From a5d8abeab454925636003f17c029a5083847f762 Mon Sep 17 00:00:00 2001 From: Kurt Himebauch <136133082+xXJSONDeruloXx@users.noreply.github.com> Date: Wed, 18 Nov 2026 14:14:26 -0500 Subject: [PATCH] feat(xbox): install legacy EAppx titles through Windows Update (on main) (#2825) * feat(xbox): install legacy EAppx titles through Windows Update A legacy Store title does not ship as MSIXVC. It ships as an encrypted EAppx bundle, which streaming rejects, so the store could list the game and never install it. Accept EAPPX and EAPPXBUNDLE beside MSIXVC, carry the format through the catalogue, the install record and the cached library, and dispatch legacy-install for those formats. Two backend patches do the work: - droiddeck-legacy-packages.patch lets the plan, license and extract verbs handle a legacy package, and expands a bundle in place (nesting bounded at three), decrypting each inner EAppx with the same keys. - droiddeck-legacy-install.patch adds legacy-install: resolve, license, download, extract, hoist AppxManifest.xml to the game root. Resolution asks the Xbox content service first and Windows Update second, because some legacy products return no files from the content service at all. The WU endpoint needs no Xbox credential but chains to Microsoft's ECC Product Root CA 2018, which Android's CA bundle does not carry, so that one public root is pinned for those requests. The manifest reader accepts AppxManifest.xml as well as MicrosoftGame.config, since a legacy package carries only the former. Running it against a real title exposed four things that only show up at full size: - A licence names a key by its 16-byte content id, while a bundle header names it as a (content id, package id) pair. Reading the 16-byte id yields a different pair, so a complete key file reported every key missing and no bundle could be opened. Keys are now aligned against the header's own ids before the header is judged. - Windows Update answers with several updates whose PackageContentId differs from the bundle's own. Licensing used to take the first and fail with content-http 400; every candidate is now tried in order, preferring the one the container is named after. - A container already in .xodus-container is reused rather than fetched again, so a failed run does not cost another download of tens of gigabytes. The space preflight no longer counts a container twice for the same reason. - A bundle that carries add-on packages unpacks each into its own folder, leaving the manifest one level down. Hoisting lifts the largest folder that holds a manifest, so the game still lands at the root beside its add-ons. Unpacking now reports bytes written, so the install bubble fills while a package is decrypted instead of holding a single unchanging word. * fix(xbox): unpack a legacy package across cores and report the bytes Unpacking one title ran as a single thread on a saturated core at about 40 MB/s. Files sit at independent offsets, so each worker now opens its own handle and takes every Nth file, with a 1 MiB read buffer in place of the 8 KiB default that turned one 64 KiB block into eight reads. Measured on the Forza Horizon 4 bundle, writes went from 40 MB/s to about 200 MB/s. Two things the first full-size run exposed: - The container was deleted only after the whole nested tree was unpacked, so a title that holds its package as one member held the archive and the decrypted copy at once. On this device that runs out of space partway through the second stage. The container is now deleted once its members are on disk, which frees the archive before they are decrypted. - A bundle member is written as a single file, so a counter that advances when a file finishes stays at zero for the whole of the longest stage. The reported figure now also follows the growth of the destination while that is the case, and never moves backwards when the container is freed. Progress is exact during the per-file stage: each file's declared length is added as it lands, instead of walking the whole destination every tick. * docs(xbox): record the completed legacy install The evidence section said a finished legacy install and its decrypted contents were unverified, because the first run was still unpacking when it was written. That install has now completed on the same device: the 85,219,173,025-byte container produced 79.3 GB at the game root with AppxManifest.xml and the package DLLs directly beneath it, and the launcher lists the title as installed. It also records what unpacking costs, since that decided the change: one thread held a core at 100% for about 40 MB/s and four reach about 200 MB/s, and releasing each container once its members are on disk is what keeps the peak footprint at one package rather than the archive plus its contents. The launch is not part of this evidence, and the entry says so. * fix(xbox): place the 64-bit framework beside a legacy title A Windows Update framework category lists one package per architecture under identifier-only names. The first package was unpacked, which on this device was the x86 build, so a 64-bit title got 32-bit DLLs and every load failed with STATUS_INVALID_IMAGE_FORMAT. Candidates are unpacked now and only the one whose manifest declares ProcessorArchitecture="x64" is kept. The framework folder was also never created, so a finished download failed at the move with "no such file or directory". unzip.rs belongs to this patch: legacy-install calls it and the recipe left it out, so a rebuild from the pinned source could not compile. * feat(xbox): report a UWP entry point at launch An unpacked package can name a COM class as its entry point instead of Windows.FullTrustApplication. That class is a UWP application, which expects the app model to activate a view; Wine has no host for it, so the launch died inside the loader with no explanation. The launch now reads the entry point and reports uwp-entry-point, and full-trust packages are unaffected. Forza Horizon 4 is such a package: its manifest names Forza.App. Its frameworks are installed and the loader is clean before that point. * fix(xbox): report why a launch was refused A failed prepare replaced the backend message with a fixed sign-in hint, so a title that was not installed or a runtime that refused all read as a bad account. The reason is kept, with paths and URLs dropped, and written to the store log. * fix(adb): dismiss the keyboard on Search and Go Search and Go were terminal IME actions but were not handled, so focus stayed in the field and the keyboard covered the results it had just fetched. * fix(xbox): keep a cache row without a legacy format plain Decoding added a format key with an empty value to every cached title, so a library restored from storage no longer matched the one that was saved. * test(ui): match the manifest message a legacy package reports The message covers both a MicrosoftGame.config and an AppxManifest, so the assertion follows the text the content dialog now prints. * fix(xbox): refuse to start a title that needs the Windows app model A legacy package that names an entry point other than Windows.FullTrustApplication is a UWP application. Wine has no app-model host, so starting its executable crashes inside the loader and the launch looks like nothing happened. Read the entry point from the manifest the installer already parses and stop the Play action with a message instead. Forza Horizon 4 names Forza.App, so it installs and reports as installed but cannot be started here yet. * fix(xbox): dispatch on the container format of the chosen package The catalogue records the format of the first installable desktop package, and the picker only replaced the content id. An offer that lists MSIXVC and a legacy container would then hand the legacy id to streaming, or the streamed id to legacy-install, and the install failed after resolution had succeeded. The choice now carries its own format and the catalogue value only fills a gap. * fix(xbox): keep an extraction reason that the sanitizer rewrote The extraction sanitizer replaces anything outside letters, digits, space, dot, underscore and dash with a question mark, so a reason such as "(os error 28)" arrived as "?os error 28?" and the allowlist dropped the line. The failure then showed only the stage code, with no cause. Add that replacement character to the class the allowlist accepts. * fix(xbox): print the reason a launch was refused The account channel sends a reason with a refusal, and the guest replaced it with a fixed sign-in message, so a title that was never installed still read as a bad account. The guest now shows the reason it was given, and a token holding a path is dropped whole instead of having its slashes removed and its directory names kept. * build(xbox): leave RUST_TEST_THREADS unset when the host has no value The recipe always passed the variable, empty when the host had none, and the pinned test harness rejects an empty count. On an ARM64 host that runs cargo test, so the default local build failed before compiling. * fix(xbox): rebuild the backend patches from the tree that ran The two patches in this branch were not the ones the verified install ran, and one of them did not compile. `unzip.rs` inflates deflated entries but the dependency that provides the inflater was never added, so `prepare.py` produced a tree that failed at `flate2::read::DeflateDecoder`. Anything else missing fell into the same gap: the install stage reported no bytes, so the bar held one value for the whole unpack, and a container a stopped run had finished was downloaded again. Rebuilt from the patched source, and fixed the defects a review of the patch found: - Unpacking reports bytes again: the workers count each file's declared length, and for the single-file bundle member the folder's growth stands in, which is what the device run showed. - A finished container is reused, so a retry after a failure past the download does not spend hours fetching it again. The space preflight counts it as reclaimable, since the retry truncates it first. - An install over a folder that already has a manifest replaces the old files instead of leaving the new package nested and reporting success. A first install still refuses to merge into an existing target. - The manifest search looks at the depth a nested bundle writes, not only one level down. - `withdraw_libraries` removes only the DLLs `copy_libraries` recorded copying, so a framework replacement cannot delete a library the title owns. - ZIP64 sizes and offsets stay `u64`; truncating them to `u32` read the wrong local header past 4 GiB. - The extraction checks the blockmap length against the footer's before writing, which the crate only does when asked. - The container being unpacked is never treated as a nested member, so `extract-eappx title.eappx .` no longer re-enters and deletes its own input. - `flate2` is declared and the lock is updated. `prepare.py` reproduces this from a pristine checkout and `cargo test` passes on it. * fix(xbox): apply the legacy install patch over the Nexus 0.2.15 backend * fix(xbox): install only the newest x64 build of a legacy title and size the space check at its real peak * fix(xbox): install a store package that is a plain archive instead of an XVD * fix(xbox): refuse a package entry that ends before its recorded size * fix(xbox): credit only complete staged containers toward the free space a legacy install needs * fix(xbox): bound each package entry to its data and check ZIP64 decoded sizes * fix(xbox): refuse duplicate package names and clear an unfinished download before measuring space * fix(xbox): bound package entries by the next header and stage a plain package until it validates * fix(xbox): hold the install folder lock through a legacy install * fix(xbox): fall back to an older build when Microsoft refuses the newest one a license * fix(xbox): keep the registered device across sign-out * feat(xbox): explain device-limit and license refusals, with a link to manage devices * fix(xbox): locate each build only once licensed, roll back a failed package swap, report the final license answer * fix(xbox): skip trial and preorder packages, read runtime behavior, keep licensing timeouts * fix(xbox): keep a failed package swap recoverable, size bundles up front, report key-write and framework errors cleanly * fix(xbox): drop a pending manifest check on any newer launch, keep committed backups out of restores, match the app-model order * fix(xbox): cancel pending launch checks on a new intent, remember entries a rollback could not remove * fix(xbox): clear the removal list before a restore moves old files back --------- Co-authored-by: TimmyCraft <46533729+TimmyTurner51@users.noreply.github.com> --- .../droiddeck/launcher/FrontEndActivity.kt | 48 +- .../launcher/stores/StoreLaunchRequests.kt | 20 +- .../launcher/stores/xbox/XboxBackend.kt | 14 +- .../launcher/stores/xbox/XboxCatalog.kt | 23 +- .../launcher/stores/xbox/XboxContent.kt | 5 +- .../launcher/stores/xbox/XboxCredentials.kt | 13 +- .../launcher/stores/xbox/XboxInstaller.kt | 68 +- .../launcher/stores/xbox/XboxLibraryCache.kt | 14 +- .../launcher/stores/xbox/XboxManifest.kt | 55 +- .../droiddeck/launcher/stores/xbox/Xodus.kt | 4 +- .../droiddeck/launcher/ui/FrontEndWidgets.kt | 33 +- .../com/droiddeck/launcher/ui/GameDetail.kt | 4 +- .../droiddeck/launcher/ui/GogMaintenanceUi.kt | 2 +- .../com/droiddeck/launcher/ui/LibraryPage.kt | 4 +- .../com/droiddeck/launcher/ui/StoreChips.kt | 15 +- .../droiddeck/launcher/ui/StoresDownloads.kt | 6 + .../launcher/ui/WirelessAdbFixPage.kt | 14 + app/src/main/res/values/strings.xml | 4 + .../launcher/stores/xbox/XboxCatalogTest.kt | 17 + .../stores/xbox/XboxDeviceKeptTest.kt | 67 + .../stores/xbox/XboxInstallerRefusalTest.kt | 22 + .../launcher/stores/xbox/XboxManifestTest.kt | 69 + .../stores/xbox/XodusDiagnosticsTest.kt | 10 + .../launcher/ui/StoreRestorationTest.kt | 2 +- docs/development/xbox-store.md | 87 +- tools/xbox/README.md | 22 +- tools/xbox/build-local.sh | 7 +- tools/xbox/droiddeck-legacy-install.patch | 2794 +++++++++++++++++ tools/xbox/droiddeck-legacy-packages.patch | 570 ++++ tools/xbox/droiddeck-legacy-run.patch | 124 + tools/xbox/prepare.py | 3 +- 31 files changed, 4072 insertions(+), 68 deletions(-) create mode 100644 app/src/test/java/com/droiddeck/launcher/stores/xbox/XboxDeviceKeptTest.kt create mode 100644 app/src/test/java/com/droiddeck/launcher/stores/xbox/XboxInstallerRefusalTest.kt create mode 100644 tools/xbox/droiddeck-legacy-install.patch create mode 100644 tools/xbox/droiddeck-legacy-packages.patch create mode 100644 tools/xbox/droiddeck-legacy-run.patch diff --git a/app/src/main/java/com/droiddeck/launcher/FrontEndActivity.kt b/app/src/main/java/com/droiddeck/launcher/FrontEndActivity.kt index 4a88837c8e..284970c9c6 100644 --- a/app/src/main/java/com/droiddeck/launcher/FrontEndActivity.kt +++ b/app/src/main/java/com/droiddeck/launcher/FrontEndActivity.kt @@ -963,13 +963,36 @@ class FrontEndActivity : ComponentActivity(), com.droiddeck.launcher.ui.AgentPet } } - private fun launchGame(game: Library.SteamGame, returnHome: Boolean = false, onHandled: () -> Unit = {}): Boolean { + private fun launchGame(game: Library.SteamGame, returnHome: Boolean = false, onHandled: () -> Unit = {}, onSettled: () -> Unit = {}) { + val request = gameLaunchRequest.incrementAndGet() if (!SessionPrefs.soundtracksIngame(this)) com.droiddeck.launcher.playback.SoundtrackController.pause(this) if (pkgStage != null) { onHandled() android.widget.Toast.makeText(this, R.string.main_wait_install_game, android.widget.Toast.LENGTH_SHORT).show() - return false + return } + val files = game.gameFiles + if (game.source != com.droiddeck.launcher.stores.Store.XBOX.id || files == null) { + if (startGame(game, returnHome, onHandled)) onSettled() + return + } + Thread({ + val refused = com.droiddeck.launcher.stores.xbox.XboxManifest.needsAppModel(files) + ui.post { + when { + request != gameLaunchRequest.get() || isFinishing || isDestroyed -> onHandled() + refused -> { + onHandled() + android.widget.Toast.makeText(this, getString(R.string.game_needs_app_model, game.name), android.widget.Toast.LENGTH_LONG).show() + onSettled() + } + startGame(game, returnHome, onHandled) -> onSettled() + } + } + }, "xbox-app-model").start() + } + + private fun startGame(game: Library.SteamGame, returnHome: Boolean, onHandled: () -> Unit): Boolean { if (com.droiddeck.launcher.session.NoSteamLaunch.enabled(this, game)) return launchWithoutSteam(game, returnHome, onHandled) onHandled() if (SessionState.running) { @@ -1184,19 +1207,19 @@ class FrontEndActivity : ComponentActivity(), com.droiddeck.launcher.ui.AgentPet if (!isDestroyed && lifecycle.currentState.isAtLeast(androidx.lifecycle.Lifecycle.State.RESUMED)) syncDualPanel() }, LINK_PANEL_MS) val game = games.firstOrNull { it.gameIdString == id } - if (game == null) { + val link = intent + fun consume() { + if (intent !== link || pendingGameLink != id) return pendingGameLink = null - intent.data = null - intent.action = Intent.ACTION_MAIN - intent.removeExtra(GameLaunchIntent.EXTRA_APP_ID) + link.data = null + link.action = Intent.ACTION_MAIN + link.removeExtra(GameLaunchIntent.EXTRA_APP_ID) + } + if (game == null) { + consume() syncDualPanel() android.widget.Toast.makeText(this, R.string.game_link_missing, android.widget.Toast.LENGTH_LONG).show() - } else if (launchGame(game, returnHome = true)) { - pendingGameLink = null - intent.data = null - intent.action = Intent.ACTION_MAIN - intent.removeExtra(GameLaunchIntent.EXTRA_APP_ID) - } + } else launchGame(game, returnHome = true, onSettled = ::consume) } private fun chooseGameShortcut(game: Library.SteamGame) { @@ -1914,6 +1937,7 @@ class FrontEndActivity : ComponentActivity(), com.droiddeck.launcher.ui.AgentPet override fun onNewIntent(intent: Intent) { super.onNewIntent(intent) setIntent(intent) + gameLaunchRequest.incrementAndGet() if (intent.action == SteamIntegration.ACTION_SETUP) { gameLinkReading = false pendingGameLink = null diff --git a/app/src/main/java/com/droiddeck/launcher/stores/StoreLaunchRequests.kt b/app/src/main/java/com/droiddeck/launcher/stores/StoreLaunchRequests.kt index af4ea74964..9ec0dad5fd 100644 --- a/app/src/main/java/com/droiddeck/launcher/stores/StoreLaunchRequests.kt +++ b/app/src/main/java/com/droiddeck/launcher/stores/StoreLaunchRequests.kt @@ -60,8 +60,7 @@ object StoreLaunchRequests { "xbox-prepare" -> runCatching { com.droiddeck.launcher.stores.xbox.XboxRuntime.prepare(app, request.getString("id")) } .getOrElse { StoresState.logLine("Xbox: launch prepare failed (${it.javaClass.simpleName}: ${it.message})") - val reason = (it as? IllegalStateException)?.message ?: "Xbox account service is unavailable. Sign in again and retry." - JSONObject().put("ok", false).put("reason", reason) + JSONObject().put("ok", false).put("reason", reasonFor(it)) } "ea-prepare" -> runCatching { lastLaunched = Store.EA to request.optString("id"); com.droiddeck.launcher.stores.ea.EaLaunch.prepare(app, request) } .getOrElse { JSONObject().put("ok", false).put("reason", "error-" + it.javaClass.simpleName) } @@ -176,4 +175,21 @@ object StoreLaunchRequests { Log.w(TAG, "could not answer ${file.name}: ${e.message}") } } + + /// What the guest prints when a launch is refused. + /// + /// The message is the only thing that separates a stale sign-in from a title + /// that is not installed, so it is kept, minus anything that could carry a + /// device path or a token: segments lookng like a path are dropped and the + /// rest is folded to plain text. + internal fun reasonFor(error: Throwable): String { + val text = error.message.orEmpty() + .split(' ') + .filterNot { it.contains('/') || it.startsWith("http") } + .joinToString(" ") + .filter { it.isLetterOrDigit() || it in " .,:;_-()'" } + .trim() + .take(160) + return if (text.isEmpty()) "Xbox account service is unavailable. Sign in again and retry." else text + } } diff --git a/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxBackend.kt b/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxBackend.kt index d678a2f410..32597faeb7 100644 --- a/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxBackend.kt +++ b/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxBackend.kt @@ -112,11 +112,21 @@ object XboxBackend : StoreBackend { fun selectPackage(context: Context, choice: org.json.JSONObject) { val pending = packageChoice ?: return packageChoice = null - val item = pending.item.copy(extra = pending.item.extra + ("contentId" to choice.getString("contentId"))) + val item = pending.item.chosen(choice) com.droiddeck.launcher.stores.download.SavedQueue.remember(context, item, pending.root) enqueue(context.applicationContext, item, pending.root) } + /** + * The picker can offer a package in a different container format than the + * catalogue listed, and the installer dispatches on that format, so the + * choice decides it and the catalogue value only fills a missing one. + */ + private fun CatalogItem.chosen(choice: org.json.JSONObject): CatalogItem { + val format = choice.optString("packageFormat").takeIf { it.isNotBlank() } ?: extra["format"].orEmpty() + return copy(extra = extra + mapOf("contentId" to choice.getString("contentId"), "format" to format)) + } + override fun install(context: Context, item: CatalogItem, root: File) { val app = context.applicationContext if (!item.extra["baseId"].isNullOrBlank()) { XboxContent.install(app, item); return } @@ -128,7 +138,7 @@ object XboxBackend : StoreBackend { val choices = XboxCatalog.objects(plan.getJSONArray("choices")).filter { it.optBoolean("supported") && it.optString("kind") == "Game" } check(choices.isNotEmpty()) { "This offer has no supported PC package." } if (plan.getString("status") == "ready" && choices.size == 1) { - val selected = item.copy(extra = item.extra + ("contentId" to choices.single().getString("contentId"))) + val selected = item.chosen(choices.single()) com.droiddeck.launcher.stores.download.SavedQueue.remember(app, selected, root) enqueue(app, selected, root) } else StoresState.post { packageChoice = PackageChoice(item, root, choices) } diff --git a/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxCatalog.kt b/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxCatalog.kt index 6f50a03039..cd3982ebdd 100644 --- a/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxCatalog.kt +++ b/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxCatalog.kt @@ -18,6 +18,14 @@ internal object XboxCatalog { "CFQ7TTC10QFD" to "Xbox Game Pass (Starter Edition)", ) + /** + * The formats the installer can put on disk, so the catalogue never lists a + * title it cannot install. `MSIXVC` streams in place; an EAppx container is a + * legacy package that takes `legacy-install` instead. + */ + internal fun installableFormat(format: String): Boolean = + format.equals("MSIXVC", true) || format.equals("EAPPX", true) || format.equals("EAPPXBUNDLE", true) + fun access(json: JSONObject, account: String, market: String, now: Instant = Instant.now(), report: (String) -> Unit = {}): Map { require(json.getInt("schema") == 1 && json.getString("account") == account && json.getString("market") == market) val age = java.time.Duration.between(timestamp(json.getString("checkedAt")), now).seconds @@ -102,11 +110,15 @@ internal object XboxCatalog { if (!productId.matches(id) || json.optString("ProductType") != "Game") return null val localized = objects(json.optJSONArray("LocalizedProperties")).firstOrNull() ?: return null val title = localized.optString("ProductTitle").takeIf { it.isNotBlank() } ?: return null - val packages = objects(json.optJSONArray("DisplaySkuAvailabilities")).flatMap { - objects(it.optJSONObject("Sku")?.optJSONObject("Properties")?.optJSONArray("Packages")) - } + val packages = objects(json.optJSONArray("DisplaySkuAvailabilities")) + .mapNotNull { it.optJSONObject("Sku")?.optJSONObject("Properties") } + .filter { it.opt("IsTrial") != true && it.opt("IsPreOrder") != true } + .flatMap { objects(it.optJSONArray("Packages")) } val pc = packages.filter { pkg -> objects(pkg.optJSONArray("PlatformDependencies")).any { it.optString("PlatformName") == "Windows.Desktop" } } - val installable = pc.any { it.optString("PackageFormat").equals("MSIXVC", true) } + val installable = pc.any { installableFormat(it.optString("PackageFormat")) } + // The installer needs the format, not just the fact that one exists: a + // legacy container takes a different subcommand from a streamed package. + val format = pc.firstOrNull { installableFormat(it.optString("PackageFormat")) }?.optString("PackageFormat").orEmpty() val streams = cloud?.streams(id, access[id].orEmpty()) == true if (!installable && !streams) return null val images = objects(localized.optJSONArray("Images")) @@ -129,7 +141,8 @@ internal object XboxCatalog { publisher = localized.optString("PublisherName"), developer = localized.optString("DeveloperName"), description = localized.optString("ProductDescription"), sizeBytes = if (installable) pc.maxOfOrNull { it.optLong("MaxDownloadSizeInBytes") } ?: 0L else 0L, - extra = mapOf("market" to Xodus.market(), "access" to kind.orEmpty(), "pc" to flag(installable), "cloud" to flag(streams)), + extra = mapOf("market" to Xodus.market(), "access" to kind.orEmpty(), "pc" to flag(installable), + "cloud" to flag(streams), "format" to format), ) } diff --git a/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxContent.kt b/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxContent.kt index 1594dee927..be9731cdb7 100644 --- a/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxContent.kt +++ b/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxContent.kt @@ -40,7 +40,8 @@ internal object XboxContent { val item = CatalogItem(Store.XBOX, id, localized?.optString("ProductTitle")?.ifBlank { id } ?: id, imageUrl = base.sidecar.cover, owned = access == "owned", subscription = access == "gamepass", sizeBytes = pkg.optLong("MaxDownloadSizeInBytes"), - extra = mapOf("baseId" to base.sidecar.id, "contentId" to cid, "market" to market, "family" to pkg.getString("PackageFamilyName"))) + extra = mapOf("baseId" to base.sidecar.id, "contentId" to cid, "market" to market, + "family" to pkg.getString("PackageFamilyName"), "format" to pkg.optString("PackageFormat"))) Offer(item, File(base.folder, ".xbox-content/$id/.droiddeck-content.json").isFile) } StoresState.post { plan = Plan(base, offers) } @@ -51,7 +52,7 @@ internal object XboxContent { private fun packages(product: JSONObject?): List = XboxCatalog.objects(product?.optJSONArray("DisplaySkuAvailabilities")) .filter { it.optJSONObject("Sku")?.optJSONObject("Properties")?.optBoolean("IsTrial", true) == false } .flatMap { XboxCatalog.objects(it.optJSONObject("Sku")?.optJSONObject("Properties")?.optJSONArray("Packages")) } - .filter { pkg -> pkg.optString("PackageFormat") == "MSIXVC" && + .filter { pkg -> XboxCatalog.installableFormat(pkg.optString("PackageFormat")) && XboxCatalog.objects(pkg.optJSONArray("PlatformDependencies")).any { it.optString("PlatformName") == "Windows.Desktop" } } private val parents = java.util.concurrent.ConcurrentHashMap() diff --git a/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxCredentials.kt b/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxCredentials.kt index 59403dac0f..0daee79ccd 100644 --- a/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxCredentials.kt +++ b/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxCredentials.kt @@ -249,10 +249,17 @@ internal class XboxCredentials(private val app: Context, expectedGeneration: Str block(current) } fun invalidate(app: Context, revoked: () -> Unit = {}) = storage(app) { - java.io.File(StoreAccounts.dir(app, Store.XBOX), "generation").writeText(UUID.randomUUID().toString()) - StoreAccounts.clear(app, Store.XBOX) - revoked() + try { + java.io.File(StoreAccounts.dir(app, Store.XBOX), "generation").writeText(UUID.randomUUID().toString()) + val device = StoreAccounts.read(app, Store.XBOX)?.optJSONObject("slots")?.opt(DEVICE_SLOT) + val kept = device != null && runCatching { StoreAccounts.write(app, Store.XBOX, deviceOnly(device)) }.isSuccess + if (!kept) StoreAccounts.clear(app, Store.XBOX) + } finally { + revoked() + } } + internal const val DEVICE_SLOT = "dev_license" + internal fun deviceOnly(device: Any): JSONObject = JSONObject().put("slots", JSONObject().put(DEVICE_SLOT, device)) fun storage(app: Context, block: () -> T): T = synchronized(lock) { java.io.RandomAccessFile(java.io.File(StoreAccounts.dir(app, Store.XBOX), "account.lock"), "rw").use { file -> file.channel.lock().use { block() } diff --git a/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxInstaller.kt b/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxInstaller.kt index 3d02e4db57..85ea9fad69 100644 --- a/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxInstaller.kt +++ b/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxInstaller.kt @@ -15,6 +15,10 @@ internal class XboxInstaller(private val app: Context, private val item: Catalog private val paused = CountDownLatch(1) @Volatile private var child: Xodus.Child? = null @Volatile private var delete = false + /** A legacy container has no pause protocol: the downloader is killed instead. */ + private val legacy = item.extra["format"]?.lowercase() in setOf("eappx", "eappxbundle") + /** Release builds hide logcat from adb, so the tail is reported in the failure itself. */ + private val errors = java.util.concurrent.ConcurrentLinkedQueue() override fun run(handle: DownloadQueue.JobHandle): String? { if (cancelled.get()) return null @@ -41,13 +45,21 @@ internal class XboxInstaller(private val app: Context, private val item: Catalog val source = if (content) selected else item.id val guestFolder = StoreInstallRoot.guestPathForXodus(app, folder) val selection = if (content) emptyList() else listOf("--selected-content-id", selected) - Xodus.start(app, listOf("streaming", source, guestFolder, "--market", market, - "--progress-json", "--control-stdin") + selection).use { process -> + // Streaming rejects a legacy container outright, so dispatch on the + // format the catalogue recorded rather than fail after a download. + val command = if (legacy) "legacy-install" else "streaming" + Xodus.start(app, if (legacy) { + listOf(command, source, guestFolder, "--market", market, "--progress-json", "--store-id", item.id) + selection + } else { + listOf(command, source, guestFolder, "--market", market, "--progress-json", "--control-stdin") + selection + }).use { process -> child = process val rejected = AtomicBoolean(false) val rejectedLine = Regex(Xodus.SIGN_IN_REJECTED) val diagnostics = Xodus.diagnostics(process) { line -> StoresState.logLine("Xbox: $line") + errors.add(line) + while (errors.size > MAX_ERROR_LINES) errors.poll() if (line.matches(rejectedLine)) rejected.set(true) } if (cancelled.get()) process.process.destroy() @@ -60,7 +72,16 @@ internal class XboxInstaller(private val app: Context, private val item: Catalog when (progress.getString("phase")) { "complete" -> complete = true "downloading", "download", "transfer" -> handle.progress(progress.getLong("completedBytes"), progress.optLong("totalBytes")) - "finalizing", "installing" -> handle.stage(DownloadStage.INSTALL) + // Unpacking reports files, not bytes: the frontend hands + // over one container and the backend counts what it has + // written, which is the only measure that moves while a + // single multi-gigabyte member is decrypted. + "installing" -> { + handle.stage(DownloadStage.INSTALL) + val total = progress.optLong("totalBytes") + if (total > 0L) handle.stageProgress(progress.getLong("completedBytes"), total) + } + "finalizing" -> handle.stage(DownloadStage.INSTALL) else -> handle.stage(DownloadStage.MANIFEST) } } @@ -69,7 +90,14 @@ internal class XboxInstaller(private val app: Context, private val item: Catalog diagnostics.join(2000) if (cancelled.get()) return null check(!rejected.get()) { app.getString(com.droiddeck.launcher.R.string.stores_xbox_sign_in_again) } - check(process.process.exitValue() == 0 && complete) { "Xbox download did not complete. Resume to retry with fresh licenses." } + val exit = process.process.exitValue() + if (exit != 0 || !complete) { + val tail = errors.toList().takeLast(MAX_ERROR_LINES) + report(command, source, market, exit, tail) + licenseRefusal(tail)?.let { error(app.getString(it)) } + error("Xbox download did not complete. Resume to retry with fresh licenses." + + tail.takeLast(3).joinToString("") { "\n$it" }) + } } if (content) { XboxContent.complete(app, item, folder) @@ -87,6 +115,21 @@ internal class XboxInstaller(private val app: Context, private val item: Catalog } } + /** Writes the backend's words beside the app, where adb can read them. */ + private fun report(command: String, source: String, market: String, exit: Int, tail: List) { + val text = buildString { + appendLine("command: $command $source --market $market") + appendLine("legacy: $legacy") + appendLine("exit: $exit") + appendLine("--- stderr (last ${tail.size} lines) ---") + tail.forEach { appendLine(it) } + } + runCatching { + app.getExternalFilesDir(null)?.let { File(it, "xbox-install.log").writeText(text) } + } + StoresState.logLine("Xbox install failed (exit $exit): ${tail.lastOrNull().orEmpty()}") + } + override fun cancel(deleteFiles: Boolean) { cancelled.set(true) delete = deleteFiles @@ -98,14 +141,25 @@ internal class XboxInstaller(private val app: Context, private val item: Catalog } Thread({ runCatching { - process.send(JSONObject().put("schema", 1).put("command", "pause")) - paused.await(3, TimeUnit.SECONDS) + if (!legacy) { + process.send(JSONObject().put("schema", 1).put("command", "pause")) + paused.await(3, TimeUnit.SECONDS) + } + process.close() } - process.close() }, "xbox-download-stop").start() } companion object { + private const val MAX_ERROR_LINES = 40 + const val DEVICES_URL = "https://account.microsoft.com/devices/content" + private const val LICENSE_DIAGNOSTIC = "NEXUS_CONTENT_LICENSE_DIAGNOSTIC_V1 " + internal fun licenseRefusal(tail: List): Int? = when (tail.lastOrNull { it.startsWith(LICENSE_DIAGNOSTIC) } + ?.removePrefix(LICENSE_DIAGNOSTIC)?.substringBefore(' ')) { + "content-device-limit" -> com.droiddeck.launcher.R.string.stores_xbox_device_group_full + "content-device-name" -> com.droiddeck.launcher.R.string.stores_xbox_license_refused + else -> null + } internal fun executable(folder: File): String = XboxManifest.executable(folder) } } diff --git a/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxLibraryCache.kt b/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxLibraryCache.kt index 5de1abcac8..074e1e9025 100644 --- a/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxLibraryCache.kt +++ b/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxLibraryCache.kt @@ -29,7 +29,10 @@ internal object XboxLibraryCache { .put("image", game.imageUrl).put("poster", game.tallImageUrl).put("description", game.description) .put("developer", game.developer).put("publisher", game.publisher).put("size", game.sizeBytes) .put("access", if (game.subscription) "gamepass" else if (game.owned) "owned" else "none") - .put("pc", game.xboxPc).put("cloud", game.xboxCloud).put("cloudId", game.extra["cloudId"].orEmpty())) + .put("pc", game.xboxPc).put("cloud", game.xboxCloud).put("cloudId", game.extra["cloudId"].orEmpty()) + // The installer picks its subcommand from this; without it a + // cached legacy game would fall back to streaming. + .put("format", game.extra["format"].orEmpty())) }).toString() internal fun decode(text: String?, account: String, market: String, now: Long = System.currentTimeMillis()): Snapshot? = runCatching { @@ -46,13 +49,18 @@ internal object XboxLibraryCache { val title = row.getString("title") val access = row.getString("access") val cloudId = row.optString("cloudId").takeIf { it.matches(Regex("[A-Z0-9]{12}")) }.orEmpty() + val format = row.optString("format").takeIf { XboxCatalog.installableFormat(it) }.orEmpty() require(id.matches(Regex("[A-Z0-9]{12}")) && title.isNotBlank() && access in setOf("owned", "gamepass")) + // A title with no legacy format keeps the plain shape a cache written + // before formats existed has, so a restart round-trips unchanged. + val extra = linkedMapOf("market" to market, "access" to access, + "pc" to XboxCatalog.flag(row.getBoolean("pc")), "cloud" to XboxCatalog.flag(row.getBoolean("cloud")), "cloudId" to cloudId) + if (format.isNotEmpty()) extra["format"] = format CatalogItem(Store.XBOX, id, title, row.optString("image").takeIf { it.isNotEmpty() }, tallImageUrl = row.optString("poster").takeIf { it.isNotEmpty() }, description = row.optString("description"), developer = row.optString("developer"), publisher = row.optString("publisher"), sizeBytes = row.optLong("size"), owned = access == "owned", subscription = access == "gamepass", - storeUrl = "https://www.xbox.com/games/store/$id", extra = mapOf("market" to market, "access" to access, - "pc" to XboxCatalog.flag(row.getBoolean("pc")), "cloud" to XboxCatalog.flag(row.getBoolean("cloud")), "cloudId" to cloudId)) + storeUrl = "https://www.xbox.com/games/store/$id", extra = extra) } require(games.map { it.id }.distinct().size == games.size) Snapshot(games, savedAt) diff --git a/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxManifest.kt b/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxManifest.kt index d738193ed7..bb8bc0408d 100644 --- a/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxManifest.kt +++ b/app/src/main/java/com/droiddeck/launcher/stores/xbox/XboxManifest.kt @@ -5,12 +5,25 @@ import org.xmlpull.v1.XmlPullParser internal object XboxManifest { data class Info(val storeId: String, val name: String, val publisher: String, val version: String, - val dependency: String, val related: List, val executables: List) + val dependency: String, val related: List, val executables: List, val entryPoint: String, + val runtimeBehavior: String = "") + + /** A GameCore title describes itself with the config; a legacy package does not. */ + private const val GAME_CONFIG = "MicrosoftGame.config" + private const val APPX_MANIFEST = "AppxManifest.xml" + private const val FULL_TRUST = "Windows.FullTrustApplication" + private const val APP_MODEL = "windowsApp" + private val directBehaviors = setOf("packagedClassicApp", "win32App") fun read(folder: File): Info { - val manifests = folder.listFiles().orEmpty().filter { it.name.equals("MicrosoftGame.config", true) && it.isFile } - require(manifests.size == 1) { "Missing or ambiguous MicrosoftGame.config." } - val manifest = manifests.single() + // Both manifests are read the same way: an identity, an executable path and, + // for a Store package, related content. + val files = folder.listFiles().orEmpty() + val config = files.filter { it.name.equals(GAME_CONFIG, true) && it.isFile } + val legacy = files.filter { it.name.equals(APPX_MANIFEST, true) && it.isFile } + require(config.size + legacy.size == 1) { "Missing or ambiguous game manifest." } + val manifest = (config + legacy).single() + val rootTag = if (config.isEmpty()) "Package" else "Game" require(manifest.length() <= 1_048_576) val parser = android.util.Xml.newPullParser().apply { setFeature(XmlPullParser.FEATURE_PROCESS_DOCDECL, false) } val identities = ArrayList>() @@ -18,7 +31,11 @@ internal object XboxManifest { val dependencies = ArrayList() val related = ArrayList() val entries = ArrayList() + val entryPoints = ArrayList() + val behaviors = ArrayList() fun attribute(name: String) = parser.getAttributeValue(null, name).orEmpty() + fun localAttribute(name: String) = (0 until parser.attributeCount) + .firstOrNull { parser.getAttributeName(it).substringAfterLast(':') == name }?.let(parser::getAttributeValue).orEmpty() var root = false var elements = 0 manifest.inputStream().use { input -> @@ -29,16 +46,22 @@ internal object XboxManifest { XmlPullParser.DOCDECL -> error("Xbox game manifests cannot contain a DTD.") XmlPullParser.START_TAG -> { require(++elements <= 10_000 && parser.depth <= 64) { "Xbox game manifest is too complex." } - if (parser.depth == 1) { require(!root && parser.name == "Game"); root = true } + if (parser.depth == 1) { require(!root && parser.name == rootTag); root = true } when (parser.name) { "Identity" -> identities.add(listOf(attribute("Name"), attribute("Publisher"), attribute("Version"))) "StoreId" -> storeIds.add(parser.nextText().trim()) "MainPackageDependency" -> dependencies.add(attribute("Name")) "RelatedProduct" -> related.add(parser.nextText().trim()) - "Executable" -> { - val path = attribute("Name").replace('\\', '/') + // Both carry the launch path: an attribute on the config's + // Executable, or on the AppxManifest's Application. + "Executable", "Application" -> { + val path = attribute("Name").ifEmpty { attribute("Executable") }.replace('\\', '/') if (path.endsWith(".exe", true) && attribute("TargetDeviceFamily") in listOf("", "PC", "Windows.Desktop") && !attribute("IsDevOnly").equals("true", true)) entries.add(path) + if (parser.name == "Application") { + attribute("EntryPoint").takeIf { it.isNotEmpty() }?.let(entryPoints::add) + localAttribute("RuntimeBehavior").takeIf { it.isNotEmpty() }?.let(behaviors::add) + } } } } @@ -48,9 +71,25 @@ internal object XboxManifest { require(root && identities.size <= 1 && storeIds.size <= 1 && dependencies.size <= 1) { "Invalid Xbox game manifest." } val ids = related.distinct() require(ids.size <= 128 && ids.all { it.matches(Regex("[A-Z0-9]{12}")) }) + val entry = entryPoints.distinct() + val behavior = behaviors.distinct() + require(entry.size <= 1 && behavior.size <= 1) { "Invalid Xbox game manifest." } val identity = identities.singleOrNull() return Info(storeIds.singleOrNull().orEmpty(), identity?.get(0).orEmpty(), identity?.get(1).orEmpty(), identity?.get(2).orEmpty(), - dependencies.singleOrNull().orEmpty(), ids, entries) + dependencies.singleOrNull().orEmpty(), ids, entries, entry.singleOrNull().orEmpty(), behavior.singleOrNull().orEmpty()) + } + + /** + * A full trust package starts its executable directly. Anything else is activated + * through the Windows app model, which this runtime has no host for, so starting + * the executable would only crash. + */ + fun needsAppModel(folder: File): Boolean { + val info = runCatching { read(folder) }.getOrNull() ?: return false + val behavior = info.runtimeBehavior + if (behavior.equals(APP_MODEL, true)) return true + if (directBehaviors.any { it.equals(behavior, true) }) return false + return info.entryPoint.isNotEmpty() && !info.entryPoint.equals(FULL_TRUST, true) } fun executable(folder: File): String { diff --git a/app/src/main/java/com/droiddeck/launcher/stores/xbox/Xodus.kt b/app/src/main/java/com/droiddeck/launcher/stores/xbox/Xodus.kt index f27794d1b5..39866fceea 100644 --- a/app/src/main/java/com/droiddeck/launcher/stores/xbox/Xodus.kt +++ b/app/src/main/java/com/droiddeck/launcher/stores/xbox/Xodus.kt @@ -155,10 +155,12 @@ internal object Xodus { const val SIGN_IN_REJECTED = "Microsoft content licensing rejected sign-in \\(HTTP 401\\)\\. Sign in again in Settings\\." + internal val reportable = Regex("NEXUS_CREDENTIAL_ERROR_V1 [a-z-]{1,48}|NEXUS_LIBRARY_DIAGNOSTIC_V1 [a-z-]{1,48} [0-9]{1,3} [0-9]{1,6}|NEXUS_SESSION_DIAGNOSTIC_V1 [a-z-]{1,64}|Account library unavailable \\([a-z-]{1,64}\\); credentials and installed games are unchanged\\.|Account library timed out; installed games are unchanged\\.|Xbox profile is temporarily unavailable \\([a-z-]{1,48}\\); stored account is unchanged\\.|Xbox profile request timed out; stored account is unchanged\\.|NEXUS_CONTENT_LICENSE_DIAGNOSTIC_V1 [a-z-]{1,48} [0-9]{1,3}|NEXUS_LEGACY_INSTALL_DIAGNOSTIC_V1 [a-z0-9-]{1,48}( [0-9]{1,6})?|NEXUS_LEGACY_INSTALL_DETAIL_V1 [A-Za-z0-9 .,;-]{1,220}|NEXUS_EXTRACT_DIAGNOSTIC_V1 [a-z-]{1,48}|NEXUS_EXTRACT_DETAIL_V1 [A-Za-z0-9 ._?-]{1,220}|$SIGN_IN_REJECTED") + fun diagnostics(child: Child, report: (String) -> Unit = { com.droiddeck.launcher.stores.StoresState.logLine("Xbox: $it") }): Thread = Thread({ runCatching { child.process.errorStream.bufferedReader().useLines { lines -> lines.forEach { line -> - if (line.matches(Regex("NEXUS_CREDENTIAL_ERROR_V1 [a-z-]{1,48}|NEXUS_LIBRARY_DIAGNOSTIC_V1 [a-z-]{1,48} [0-9]{1,3} [0-9]{1,6}|NEXUS_SESSION_DIAGNOSTIC_V1 [a-z-]{1,64}|Account library unavailable \\([a-z-]{1,64}\\); credentials and installed games are unchanged\\.|Account library timed out; installed games are unchanged\\.|Xbox profile is temporarily unavailable \\([a-z-]{1,48}\\); stored account is unchanged\\.|Xbox profile request timed out; stored account is unchanged\\.|NEXUS_CONTENT_LICENSE_DIAGNOSTIC_V1 [a-z-]{1,48} [0-9]{3}|$SIGN_IN_REJECTED"))) + if (line.matches(reportable)) report(line) } } } diff --git a/app/src/main/java/com/droiddeck/launcher/ui/FrontEndWidgets.kt b/app/src/main/java/com/droiddeck/launcher/ui/FrontEndWidgets.kt index f12ad81ddb..6d360d7357 100644 --- a/app/src/main/java/com/droiddeck/launcher/ui/FrontEndWidgets.kt +++ b/app/src/main/java/com/droiddeck/launcher/ui/FrontEndWidgets.kt @@ -54,6 +54,7 @@ import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier import androidx.compose.ui.draw.alpha import androidx.compose.ui.draw.clip +import androidx.compose.ui.draw.drawBehind import androidx.compose.ui.draw.rotate import androidx.compose.ui.graphics.Brush import androidx.compose.ui.focus.focusProperties @@ -112,7 +113,8 @@ internal fun Actions(content: @Composable () -> Unit) = FlowRow( ) { content() } @Composable -internal fun ActionChip(t: String, ok: Boolean) = Box(contentAlignment = Alignment.Center, modifier = Modifier.heightIn(min = 44.dp)) { Chip(t, ok) } +internal fun ActionChip(t: String, ok: Boolean, fraction: Float? = null, key: Any? = null) = + Box(contentAlignment = Alignment.Center, modifier = Modifier.heightIn(min = 44.dp)) { Chip(t, ok, fraction, key) } internal val Shape16 = RoundedCornerShape(16.dp) @@ -139,13 +141,32 @@ internal fun BackLink(label: String, compact: Boolean = false, onClick: () -> Un } @Composable -internal fun Chip(t: String, ok: Boolean) { +internal fun Chip(t: String, ok: Boolean, fraction: Float? = null, key: Any? = null) { val colors = MaterialTheme.colorScheme val pal = LocalPalette.current - Text( - t, fontSize = 12.sp, color = if (ok) pal.good else colors.onSurfaceVariant, maxLines = 1, overflow = TextOverflow.Ellipsis, - modifier = Modifier.clip(RoundedCornerShape(99.dp)).background(colors.surfaceVariant).border(1.dp, if (ok) pal.good.copy(alpha = 0.3f) else pal.line, RoundedCornerShape(99.dp)).padding(horizontal = 9.dp, vertical = 4.dp), - ) + val shape = RoundedCornerShape(99.dp) + val ink = if (ok) pal.good else colors.onSurfaceVariant + Box(Modifier.clip(shape).background(colors.surfaceVariant).border(1.dp, if (ok) pal.good.copy(alpha = 0.3f) else pal.line, shape)) { + if (fraction != null) ChipFill(fraction, key, pal.signal.copy(alpha = 0.3f)) + Text( + t, fontSize = 12.sp, color = ink, maxLines = 1, overflow = TextOverflow.Ellipsis, + modifier = Modifier.padding(horizontal = 9.dp, vertical = 4.dp), + ) + } +} + +@Composable +internal fun androidx.compose.foundation.layout.BoxScope.ChipFill(fraction: Float, key: Any?, color: Color) { + androidx.compose.runtime.key(key) { + val target = fraction.coerceIn(0f, 1f) + val shown = remember { Animatable(target) } + LaunchedEffect(target) { shown.animateTo(target, androidx.compose.animation.core.tween(250, easing = androidx.compose.animation.core.LinearEasing)) } + Box( + Modifier.matchParentSize().drawBehind { + drawRect(color, size = androidx.compose.ui.geometry.Size(size.width * shown.value, size.height)) + }, + ) + } } @Composable diff --git a/app/src/main/java/com/droiddeck/launcher/ui/GameDetail.kt b/app/src/main/java/com/droiddeck/launcher/ui/GameDetail.kt index e50176e09c..b99a4331bb 100644 --- a/app/src/main/java/com/droiddeck/launcher/ui/GameDetail.kt +++ b/app/src/main/java/com/droiddeck/launcher/ui/GameDetail.kt @@ -458,7 +458,7 @@ internal fun GameActions( if (installed != null && id != null) { LaunchedEffect(id) { BattleNetInstall.checkUpdate(ctx, installed) } if (download != null) { - ActionChip(downloadLabel(download), ok = false) + ActionChip(downloadLabel(download), ok = false, fraction = download.stageFraction.takeIf { it >= 0f }, key = download.stage) if (download.state != DownloadState.PAUSED) SecondaryButton(stringResource(R.string.stores_dl_pause)) { DownloadQueue.pause(download.key) } else SecondaryButton(stringResource(R.string.stores_dl_resume)) { DownloadQueue.resume(ctx, download.key) } } else if (BattleNetInstall.updates[id] == true) { @@ -473,7 +473,7 @@ internal fun GameActions( } } download != null -> { - ActionChip(downloadLabel(download), ok = false) + ActionChip(downloadLabel(download), ok = false, fraction = download.stageFraction.takeIf { it >= 0f }, key = download.stage) if (download.state != DownloadState.PAUSED) SecondaryButton(stringResource(R.string.stores_dl_pause), modifier = leadOn) { DownloadQueue.pause(download.key) } else PrimaryButton(stringResource(R.string.stores_dl_resume), main = true, modifier = leadOn) { DownloadQueue.resume(ctx, download.key) } ConfirmButton(stringResource(R.string.stores_dl_cancel), stringResource(R.string.stores_dl_cancel_confirm)) { DownloadQueue.cancel(ctx, download.key) } diff --git a/app/src/main/java/com/droiddeck/launcher/ui/GogMaintenanceUi.kt b/app/src/main/java/com/droiddeck/launcher/ui/GogMaintenanceUi.kt index 57f6923396..165905ac51 100644 --- a/app/src/main/java/com/droiddeck/launcher/ui/GogMaintenanceUi.kt +++ b/app/src/main/java/com/droiddeck/launcher/ui/GogMaintenanceUi.kt @@ -68,7 +68,7 @@ internal fun GogMaintenanceActions(game: InstalledStoreGame) { } } if (active) { - ActionChip(downloadLabel(download!!), ok = false) + ActionChip(downloadLabel(download!!), ok = false, fraction = download!!.stageFraction.takeIf { it >= 0f }, key = download!!.stage) if (download.interruptible) { SecondaryButton(stringResource(if (download.state == DownloadState.PAUSED) R.string.stores_dl_resume else R.string.stores_dl_pause)) { if (download.state == DownloadState.PAUSED) DownloadQueue.resume(ctx, key) else DownloadQueue.pause(key) diff --git a/app/src/main/java/com/droiddeck/launcher/ui/LibraryPage.kt b/app/src/main/java/com/droiddeck/launcher/ui/LibraryPage.kt index a90aa79ca8..272f42840c 100644 --- a/app/src/main/java/com/droiddeck/launcher/ui/LibraryPage.kt +++ b/app/src/main/java/com/droiddeck/launcher/ui/LibraryPage.kt @@ -834,7 +834,9 @@ internal fun ArtCard( }, ) badge?.invoke(this) - if (chip != null) Box(Modifier.align(Alignment.TopEnd).padding(6.dp)) { StateChip(chip, ChipTone.BUSY, small = true) } + if (chip != null) Box(Modifier.align(Alignment.TopEnd).padding(6.dp)) { + StateChip(chip, ChipTone.BUSY, small = true, fraction = download?.stageFraction?.takeIf { it >= 0f }, key = download?.stage) + } val showName = LibraryDisplay.title || hot val showDetail = LibraryDisplay.size || hot if (showName || showDetail) Column( diff --git a/app/src/main/java/com/droiddeck/launcher/ui/StoreChips.kt b/app/src/main/java/com/droiddeck/launcher/ui/StoreChips.kt index 3921df7f32..a822548739 100644 --- a/app/src/main/java/com/droiddeck/launcher/ui/StoreChips.kt +++ b/app/src/main/java/com/droiddeck/launcher/ui/StoreChips.kt @@ -1,6 +1,7 @@ package com.droiddeck.launcher.ui import androidx.compose.foundation.background +import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.padding import androidx.compose.foundation.shape.RoundedCornerShape import androidx.compose.material3.MaterialTheme @@ -73,7 +74,7 @@ internal fun SourceChip(source: String, small: Boolean = false, modifier: Modifi } @Composable -internal fun StateChip(text: String, tone: ChipTone, small: Boolean = false) { +internal fun StateChip(text: String, tone: ChipTone, small: Boolean = false, fraction: Float? = null, key: Any? = null) { val pal = LocalPalette.current val (fill, ink) = when (tone) { ChipTone.OK -> Color(0xFF16321F) to pal.good @@ -82,10 +83,14 @@ internal fun StateChip(text: String, tone: ChipTone, small: Boolean = false) { ChipTone.DEAL -> Color(0xFF1E3A1A) to Color(0xFF9BE27A) ChipTone.PLAIN -> MaterialTheme.colorScheme.surfaceVariant to MaterialTheme.colorScheme.onSurfaceVariant } - Text( - text, fontSize = if (small) 9.sp else 11.sp, fontWeight = FontWeight.Bold, letterSpacing = 0.3.sp, color = ink, maxLines = 1, - modifier = Modifier.clip(RoundedCornerShape(6.dp)).background(fill).padding(horizontal = if (small) 5.dp else 8.dp, vertical = if (small) 1.dp else 2.dp), - ) + val shape = RoundedCornerShape(6.dp) + Box(Modifier.clip(shape).background(fill)) { + if (fraction != null) ChipFill(fraction, key, ink.copy(alpha = 0.26f)) + Text( + text, fontSize = if (small) 9.sp else 11.sp, fontWeight = FontWeight.Bold, letterSpacing = 0.3.sp, color = ink, maxLines = 1, + modifier = Modifier.padding(horizontal = if (small) 5.dp else 8.dp, vertical = if (small) 1.dp else 2.dp), + ) + } } internal enum class ChipTone { OK, BUSY, WARN, DEAL, PLAIN } diff --git a/app/src/main/java/com/droiddeck/launcher/ui/StoresDownloads.kt b/app/src/main/java/com/droiddeck/launcher/ui/StoresDownloads.kt index ed382d8d5d..59ae4df3b8 100644 --- a/app/src/main/java/com/droiddeck/launcher/ui/StoresDownloads.kt +++ b/app/src/main/java/com/droiddeck/launcher/ui/StoresDownloads.kt @@ -61,6 +61,7 @@ import androidx.compose.foundation.layout.padding import androidx.compose.foundation.layout.width import androidx.compose.foundation.shape.RoundedCornerShape import androidx.compose.material.icons.Icons +import androidx.compose.material.icons.automirrored.filled.OpenInNew import androidx.compose.material.icons.filled.PlayArrow import androidx.compose.material3.MaterialTheme import androidx.compose.material3.Text @@ -371,6 +372,11 @@ private fun DownloadCard(d: DownloadEntry, s: FrontEndState, a: FrontEndActions) CancelAction { DownloadQueue.cancel(ctx, d.key) } } DownloadState.FAILED -> { + if (d.store == Store.XBOX && d.error == stringResource(R.string.stores_xbox_device_group_full)) { + RoundAction(Icons.AutoMirrored.Filled.OpenInNew, stringResource(R.string.stores_xbox_manage_devices)) { + runCatching { ctx.startActivity(android.content.Intent(android.content.Intent.ACTION_VIEW, android.net.Uri.parse(com.droiddeck.launcher.stores.xbox.XboxInstaller.DEVICES_URL)).addFlags(android.content.Intent.FLAG_ACTIVITY_NEW_TASK)) } + } + } RoundAction(Icons.Filled.Refresh, stringResource(R.string.stores_dl_resume), primary = true) { DownloadQueue.retry(ctx, d.key) } CancelAction { DownloadQueue.clear(ctx, d.key) } } diff --git a/app/src/main/java/com/droiddeck/launcher/ui/WirelessAdbFixPage.kt b/app/src/main/java/com/droiddeck/launcher/ui/WirelessAdbFixPage.kt index 276f82f79f..ebcfcb1ea7 100644 --- a/app/src/main/java/com/droiddeck/launcher/ui/WirelessAdbFixPage.kt +++ b/app/src/main/java/com/droiddeck/launcher/ui/WirelessAdbFixPage.kt @@ -453,6 +453,20 @@ internal fun AdbTextField( onDoneLatest.value?.invoke() true } + // Search and Go are terminal: leaving them unhandled + // keeps focus in the field, so the keyboard would stay + // up over the results it was used to find. + imeAction in setOf(ImeAction.Search, ImeAction.Go) && + (actionId == EditorInfo.IME_ACTION_SEARCH || actionId == EditorInfo.IME_ACTION_GO || isEnter) -> { + onDoneLatest.value?.invoke() + post { + clearFocus() + (context.getSystemService(android.content.Context.INPUT_METHOD_SERVICE) + as? android.view.inputmethod.InputMethodManager) + ?.hideSoftInputFromWindow(windowToken, 0) + } + true + } else -> false } } diff --git a/app/src/main/res/values/strings.xml b/app/src/main/res/values/strings.xml index dbad01b39c..6c7f319166 100644 --- a/app/src/main/res/values/strings.xml +++ b/app/src/main/res/values/strings.xml @@ -1771,6 +1771,7 @@ This game is no longer in your DroidDeck library. Stop the current session before launching a Steam game. Could not send the game to Steam. Try launching it again. + %1$s is a UWP app. This runtime has no Windows app-model host to start it. Retry removal Removing Removal incomplete @@ -2992,6 +2993,9 @@ Close Try again Finishing sign-in… + This Microsoft account is at its device limit. Remove an old device under Manage devices, then resume. + Manage devices + Microsoft does not license any available version of this game to this device. Microsoft rejected the Xbox sign-in. Sign out of Xbox in Stores, sign in again, then resume. The sign-in page could not be loaded: %1$s This provider refuses to sign in from an embedded page. Sign in with your store password instead. diff --git a/app/src/test/java/com/droiddeck/launcher/stores/xbox/XboxCatalogTest.kt b/app/src/test/java/com/droiddeck/launcher/stores/xbox/XboxCatalogTest.kt index a59450e341..c2106a9bf9 100644 --- a/app/src/test/java/com/droiddeck/launcher/stores/xbox/XboxCatalogTest.kt +++ b/app/src/test/java/com/droiddeck/launcher/stores/xbox/XboxCatalogTest.kt @@ -184,6 +184,23 @@ class XboxCatalogTest { assertFalse(XboxCatalog.product(product(game, "Windows.Desktop"), mapOf(game to "gamepass"), null)!!.xboxCloud) } + @Test fun trialAndPreorderSkusDoNotMakeAGamePcInstallable() { + fun sku(format: String, vararg flags: String) = JSONObject().put("Sku", JSONObject().put("Properties", JSONObject().apply { + flags.forEach { put(it, true) } + put("Packages", JSONArray().put(JSONObject().put("PackageFormat", format).put("MaxDownloadSizeInBytes", 7L) + .put("PlatformDependencies", JSONArray().put(JSONObject().put("PlatformName", "Windows.Desktop"))))) + })) + fun offer(vararg skus: JSONObject) = JSONObject().put("ProductId", game).put("ProductType", "Game") + .put("LocalizedProperties", JSONArray().put(JSONObject().put("ProductTitle", "A Game"))) + .put("DisplaySkuAvailabilities", JSONArray(skus.toList())) + assertNull(XboxCatalog.product(offer(sku("EAPPX", "IsTrial")), mapOf(game to "owned"))) + assertNull(XboxCatalog.product(offer(sku("MSIXVC", "IsPreOrder")), mapOf(game to "owned"))) + val streamed = XboxCatalog.product(offer(sku("EAPPX", "IsTrial")), mapOf(game to "gamepass"), XboxCloudCatalog(setOf(game), emptySet()))!! + assertFalse(streamed.xboxPc); assertTrue(streamed.xboxCloud); assertEquals("", streamed.extra["format"]); assertEquals(0L, streamed.sizeBytes) + val retail = XboxCatalog.product(offer(sku("MSIXVC", "IsTrial"), sku("EAPPX")), mapOf(game to "owned"))!! + assertTrue(retail.xboxPc); assertEquals("EAPPX", retail.extra["format"]); assertEquals(7L, retail.sizeBytes) + } + @Test fun aCloudOnlyEditionFoldsIntoThePcCardWithTheSameTitle() { val cloud = XboxCloudCatalog(gamePass = setOf(parent), all = emptySet()) val pc = XboxCatalog.product(product(game, "Windows.Desktop"), mapOf(game to "gamepass"), cloud)!! diff --git a/app/src/test/java/com/droiddeck/launcher/stores/xbox/XboxDeviceKeptTest.kt b/app/src/test/java/com/droiddeck/launcher/stores/xbox/XboxDeviceKeptTest.kt new file mode 100644 index 0000000000..b2227ae288 --- /dev/null +++ b/app/src/test/java/com/droiddeck/launcher/stores/xbox/XboxDeviceKeptTest.kt @@ -0,0 +1,67 @@ +package com.droiddeck.launcher.stores.xbox + +import com.droiddeck.launcher.stores.CredentialCipher +import com.droiddeck.launcher.stores.Store +import com.droiddeck.launcher.stores.StoreAccounts +import org.json.JSONArray +import org.json.JSONObject +import org.junit.After +import org.junit.Assert.assertEquals +import org.junit.Assert.assertNull +import org.junit.Assert.assertTrue +import org.junit.Before +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.RuntimeEnvironment +import org.robolectric.annotation.Config +import java.security.SecureRandom +import javax.crypto.spec.SecretKeySpec + +@RunWith(RobolectricTestRunner::class) +@Config(sdk = [33], application = android.app.Application::class) +class XboxDeviceKeptTest { + private val app get() = RuntimeEnvironment.getApplication() + private val key = SecretKeySpec(ByteArray(32).also { SecureRandom().nextBytes(it) }, "AES") + + @Before fun setUp() { StoreAccounts.keys = CredentialCipher.KeyProvider { key }; StoreAccounts.keystoreFailed = false } + @After fun tearDown() { + StoreAccounts.clear(app, Store.XBOX) + StoreAccounts.keys = CredentialCipher.Keystore; StoreAccounts.keystoreFailed = false + } + + @Test fun signingOutKeepsOnlyTheRegisteredDevice() { + val device = JSONArray(listOf(1, 2, 3)) + StoreAccounts.write(app, Store.XBOX, JSONObject().put("username", "player") + .put("slots", JSONObject().put(XboxCredentials.DEVICE_SLOT, device).put("user-tokens", JSONArray(listOf(9))))) + XboxCredentials.invalidate(app) + val record = StoreAccounts.read(app, Store.XBOX)!! + assertNull(StoreAccounts.signedInAs(app, Store.XBOX)) + val slots = record.getJSONObject("slots") + assertEquals(listOf(XboxCredentials.DEVICE_SLOT), slots.keys().asSequence().toList()) + assertEquals(device.toString(), slots.getJSONArray(XboxCredentials.DEVICE_SLOT).toString()) + } + + @Test fun aFailedDeviceRewriteStillSignsOutAndRevokes() { + var sealing = true + StoreAccounts.keys = object : CredentialCipher.KeyProvider { + override fun key() = if (sealing) key else throw java.security.KeyStoreException("sealing refused") + override fun existing() = key + } + StoreAccounts.write(app, Store.XBOX, JSONObject().put("username", "player") + .put("slots", JSONObject().put(XboxCredentials.DEVICE_SLOT, JSONArray(listOf(1))))) + sealing = false + var revoked = false + XboxCredentials.invalidate(app) { revoked = true } + assertTrue(revoked) + assertTrue(StoreAccounts.keystoreFailed) + assertNull(StoreAccounts.read(app, Store.XBOX)) + assertNull(StoreAccounts.signedInAs(app, Store.XBOX)) + } + + @Test fun signingOutWithoutADeviceLeavesNoRecord() { + StoreAccounts.write(app, Store.XBOX, JSONObject().put("username", "player").put("slots", JSONObject())) + XboxCredentials.invalidate(app) + assertNull(StoreAccounts.read(app, Store.XBOX)) + } +} diff --git a/app/src/test/java/com/droiddeck/launcher/stores/xbox/XboxInstallerRefusalTest.kt b/app/src/test/java/com/droiddeck/launcher/stores/xbox/XboxInstallerRefusalTest.kt new file mode 100644 index 0000000000..0e717f0c17 --- /dev/null +++ b/app/src/test/java/com/droiddeck/launcher/stores/xbox/XboxInstallerRefusalTest.kt @@ -0,0 +1,22 @@ +package com.droiddeck.launcher.stores.xbox + +import com.droiddeck.launcher.R +import org.junit.Assert.assertEquals +import org.junit.Assert.assertNull +import org.junit.Test + +class XboxInstallerRefusalTest { + private val refused = "NEXUS_CONTENT_LICENSE_DIAGNOSTIC_V1 content-device-name 400" + private val full = "NEXUS_CONTENT_LICENSE_DIAGNOSTIC_V1 content-device-limit 200" + private val granted = "NEXUS_CONTENT_LICENSE_DIAGNOSTIC_V1 content-parsed 200" + + @Test fun theLastLicensingAnswerDecidesTheMessage() { + assertEquals(R.string.stores_xbox_license_refused, XboxInstaller.licenseRefusal(listOf(refused, "NEXUS_LEGACY_INSTALL_DIAGNOSTIC_V1 license-failed"))) + assertEquals(R.string.stores_xbox_device_group_full, XboxInstaller.licenseRefusal(listOf(full))) + } + + @Test fun aRefusalAnOlderBuildRecoveredFromIsNotReported() { + assertNull(XboxInstaller.licenseRefusal(listOf(refused, "NEXUS_LEGACY_INSTALL_DETAIL_V1 newest build has no license for this device, trying an older build", granted, "NEXUS_TRANSFER_DIAGNOSTIC_V1 transfer-short 0"))) + assertNull(XboxInstaller.licenseRefusal(listOf("NEXUS_TRANSFER_DIAGNOSTIC_V1 transfer-short 0"))) + } +} diff --git a/app/src/test/java/com/droiddeck/launcher/stores/xbox/XboxManifestTest.kt b/app/src/test/java/com/droiddeck/launcher/stores/xbox/XboxManifestTest.kt index 99ab0a0eb6..d2396db935 100644 --- a/app/src/test/java/com/droiddeck/launcher/stores/xbox/XboxManifestTest.kt +++ b/app/src/test/java/com/droiddeck/launcher/stores/xbox/XboxManifestTest.kt @@ -47,6 +47,75 @@ class XboxManifestTest { assertEquals(listOf("Retail.exe"), info.executables) assertEquals("Retail.exe", XboxManifest.executable(root)) } + @Test fun legacyPackageManifestNamesItsExecutableTheSameWay() { + // A legacy EAppx package carries AppxManifest.xml and has no config; the + // launcher must still find the identity and the launch path, or an install + // that completed cannot be registered or started. + val root = temporary.newFolder() + java.io.File(root, "ForzaHorizon4.exe").writeText("encrypted") + java.io.File(root, "AppxManifest.xml").writeText(""" + + + + """) + val info = XboxManifest.read(root) + assertEquals("Microsoft.SunriseBaseGame", info.name) + assertEquals("2023.1122.1704.2", info.version) + assertEquals(listOf("ForzaHorizon4.exe"), info.executables) + assertEquals("Windows.FullTrustApplication", info.entryPoint) + assertEquals("ForzaHorizon4.exe", XboxManifest.executable(root)) + assertFalse(XboxManifest.needsAppModel(root)) + } + @Test fun uwpEntryPointNeedsAnAppModelHost() { + // Forza Horizon 4's Store build declares Forza.App, so its executable is a view + // provider rather than a program: starting it directly only crashes. + val root = temporary.newFolder() + java.io.File(root, "ForzaHorizon4.exe").writeText("encrypted") + java.io.File(root, "AppxManifest.xml").writeText(""" + + + """) + assertEquals("Forza.App", XboxManifest.read(root).entryPoint) + assertTrue(XboxManifest.needsAppModel(root)) + } + private fun legacy(application: String): java.io.File { + val root = temporary.newFolder() + java.io.File(root, "Game.exe").writeText("encrypted") + java.io.File(root, "AppxManifest.xml").writeText(""" + + + $application""") + return root + } + @Test fun windowsAppRuntimeBehaviorNeedsAnAppModelWithoutAnEntryPoint() { + val root = legacy("""""") + assertEquals("windowsApp", XboxManifest.read(root).runtimeBehavior) + assertEquals("", XboxManifest.read(root).entryPoint) + assertTrue(XboxManifest.needsAppModel(root)) + } + @Test fun classicRuntimeBehaviorsStartTheExecutableDirectly() { + for (behavior in listOf("packagedClassicApp", "win32App")) { + val root = legacy("""""") + assertEquals(behavior, XboxManifest.read(root).runtimeBehavior) + assertFalse(behavior, XboxManifest.needsAppModel(root)) + val declared = legacy("""""") + assertFalse(behavior, XboxManifest.needsAppModel(declared)) + } + } + @Test fun anEntryPointStillDecidesWithoutARuntimeBehavior() { + assertTrue(XboxManifest.needsAppModel(legacy(""""""))) + assertFalse(XboxManifest.needsAppModel(legacy(""""""))) + } + @Test fun aTitleWithoutALegacyManifestNeverNeedsAnAppModel() { + val root = temporary.newFolder() + java.io.File(root, "Game.exe").writeText("encrypted") + java.io.File(root, "MicrosoftGame.config").writeText("") + assertEquals("", XboxManifest.read(root).entryPoint) + assertFalse(XboxManifest.needsAppModel(root)) + assertFalse(XboxManifest.needsAppModel(temporary.newFolder())) + } @Test fun signInBridgeRejectsUntrustedAndNonHttpsOrigins() { assertTrue(XboxLoginActivity.trusted("https://login.live.com/ppsecure/InlineLogin.srf")) for (url in listOf("http://login.live.com/", "https://login.live.com.evil.test/", "https://evil.test@login.live.com/", "https://login.live.com:8443/")) diff --git a/app/src/test/java/com/droiddeck/launcher/stores/xbox/XodusDiagnosticsTest.kt b/app/src/test/java/com/droiddeck/launcher/stores/xbox/XodusDiagnosticsTest.kt index c2d723f7b6..3e2310be5c 100644 --- a/app/src/test/java/com/droiddeck/launcher/stores/xbox/XodusDiagnosticsTest.kt +++ b/app/src/test/java/com/droiddeck/launcher/stores/xbox/XodusDiagnosticsTest.kt @@ -1,6 +1,8 @@ package com.droiddeck.launcher.stores.xbox +import org.junit.Assert.assertEquals import org.junit.Assert.assertFalse +import org.junit.Assert.assertNull import org.junit.Assert.assertTrue import org.junit.Test @@ -16,4 +18,12 @@ class XodusDiagnosticsTest { assertFalse("Microsoft content licensing rejected sign-in (HTTP 403). Sign in again in Settings.".matches(rejected)) assertFalse("NEXUS_CONTENT_LICENSE_DIAGNOSTIC_V1 content-http 401".matches(rejected)) } + + @Test fun aLicensingTimeoutAfterARefusalReachesTheTailAndIsReported() { + val stderr = listOf("NEXUS_CONTENT_LICENSE_DIAGNOSTIC_V1 content-device-name 400", "NEXUS_CONTENT_LICENSE_DIAGNOSTIC_V1 content-timeout 0") + val tail = stderr.filter { it.matches(Xodus.reportable) } + assertEquals(stderr, tail) + assertNull(XboxInstaller.licenseRefusal(tail)) + assertFalse("NEXUS_CONTENT_LICENSE_DIAGNOSTIC_V1 content-timeout 1000".matches(Xodus.reportable)) + } } diff --git a/app/src/test/java/com/droiddeck/launcher/ui/StoreRestorationTest.kt b/app/src/test/java/com/droiddeck/launcher/ui/StoreRestorationTest.kt index 4f914f0b55..1f214aba0c 100644 --- a/app/src/test/java/com/droiddeck/launcher/ui/StoreRestorationTest.kt +++ b/app/src/test/java/com/droiddeck/launcher/ui/StoreRestorationTest.kt @@ -64,7 +64,7 @@ class StoreRestorationTest { } compose.mainClock.advanceTimeBy(100) compose.onNodeWithText(ctx.getString(R.string.stores_xbox_content_title)).assertExists() - compose.onNodeWithText("Missing or ambiguous MicrosoftGame.config.").assertExists() + compose.onNodeWithText("Missing or ambiguous game manifest.").assertExists() compose.onNodeWithText(ctx.getString(android.R.string.ok)).performClick() compose.runOnIdle { assertNull(XboxContent.plan) } } diff --git a/docs/development/xbox-store.md b/docs/development/xbox-store.md index bc4aecf661..7133e5c078 100644 --- a/docs/development/xbox-store.md +++ b/docs/development/xbox-store.md @@ -1,7 +1,8 @@ # Xbox / PC Game Pass on Thor -This integrates PC Microsoft Store MSIXVC packages, -not Xbox console emulation or UWP/APPX packages. +This integrates PC Microsoft Store packages, both streamed MSIXVC packages and the legacy +EAppx containers older desktop titles still ship as. It is not Xbox console emulation and +does not run UWP titles in a packaged-app host. ## Testing on Thor @@ -69,14 +70,96 @@ not Xbox console emulation or UWP/APPX packages. cleanup must preserve installed game files reached through view symlinks. 10. Sign out from Xbox. Installed files should remain, account library data should clear, and Play/download licensing should require sign-in again. +11. Install a legacy title that ships an EAppx bundle, such as Forza Horizon 4 + (`9PNQKHFLD2WQ`). The download shows as a percentage rather than a streamed + install, Stop kills it rather than pausing, and a finished install registers + the executable named by `AppxManifest.xml`. Expect tens of gigabytes: check + free space first, since the preflight needs the download plus two copies of + the largest container. Do not uninstall DroidDeck or clear application data to troubleshoot this build. The signed APK is installed in place to retain existing Steam and store data. +## Legacy EAppx packages + +A title that predates MSIXVC ships as an EAppx bundle: an archive of Appx packages, encrypted, +with an `AppxManifest.xml` per package instead of `MicrosoftGame.config`. `streaming` rejects the +format, so the installer reads the format the catalogue recorded and runs `xodus-cli +legacy-install` instead. `MSIXVC`, `EAPPX` and `EAPPXBUNDLE` are the only formats listed for a +`Windows.Desktop` package; the format is carried into the install record so a later update or +resume dispatches the same way. + +The verb resolves, licenses, downloads, extracts, then hoists the manifest to the game root: + +- **Resolution.** The Xbox content service is asked first. Some legacy bundles it answers with no + files at all; Windows Update (`fe3cr.delivery.mp.microsoft.com`, SOAP `GetCookie`, + `SyncUpdates`, `GetExtendedUpdateInfo2`) is asked next through the title's `WuCategoryId` and + returns the full bundle with a time-signed CDN URL. That endpoint is a public Windows Update + service and needs no Xbox credential, but it presents a certificate chained to Microsoft's ECC + Product Root CA 2018, which Android's CA bundle does not carry, so `wu-root.pem` is pinned for + those requests only. A WU result is reduced to its `.eappx`/`.eappxbundle` files: the plain + `.appx` framework packages delivered beside it (VCLibs) and the differential `.cab` metadata are + not containers this path can unpack. +- **Licensing.** The existing `license` verb runs before the first byte is fetched and must return + content keys; no keys means no install. +- **Space.** Free space is checked up front against the download plus two copies of the largest + container, because a bundle is expanded in place. Each container is removed as soon as it is + unpacked, so the peak is the archive plus its contents rather than both plus the earlier + archives. A container a stopped attempt finished is counted as reclaimable, since the retry + truncates it before writing. +- **Pause and resume.** A legacy download has no control channel: Stop kills the downloader, and + the next attempt starts an unfinished container again from byte zero, while a container that + completed is kept and reused. Nothing is checkpointed inside a container. +- **Frameworks.** An installed package gets its `PackageDependency` frameworks from the package + graph; an unpacked folder has none, so the loader fails with `0xc0000135` on + `VCRUNTIME140_APP.dll` and friends. The verb fetches each declared framework from its Windows + Update category and copies its libraries beside the title, which is where the loader looks. A + category lists one package per architecture under identifier-only names, so each is unpacked + until one declares `ProcessorArchitecture="x64"`; taking the first one puts 32-bit DLLs into a + 64-bit folder and every load fails with `0xc000007b`. Only the libraries that copy put there + are withdrawn when a framework is replaced, so a title's own DLL of the same name survives. +- **Updates.** Installing over a folder that already holds a manifest lifts the new package over + the old files; on a first install an existing target is an error rather than a merge. +- **Failure detail.** A release build hides logcat from adb, so a failed install writes the + backend's stderr tail to `files/xbox-install.log` beside the app and repeats the last lines in + the failure message. + ## Evidence and limits Verified during development: +- The catalogue lists a legacy title beside the MSIXVC ones, and its detail page + offers Install with the recorded format. +- Resolution of a real legacy PC bundle on Thor without any Xbox token: Forza + Horizon 4 (`9PNQKHFLD2WQ`) through Windows Update as + `Microsoft.SunriseBaseGame_2023.1122.1704.2_neutral_~_8wekyb3d8bbwe.eappxbundle`, + 85,219,173,025 bytes, with a signed CDN URL. The Xbox content service returns + no files for that product, which is why the WU route exists. +- `AppxManifest.xml` identity and executable extraction, and its rejection rules, + are unit-tested against a manifest of that shape. +- An install of that bundle completed on Thor. The 85,219,173,025-byte container + produced 79.3 GB at the game root with `AppxManifest.xml` and the package DLLs + directly beneath it, which the launcher reports as installed. +- Unpacking that bundle ran at about 40 MB/s on one thread and about 200 MB/s + across four, measured as growth of the destination folder. The container is + deleted once its members are on disk, so the peak footprint is one package at + a time rather than the archive plus everything under it. The install bubble + advances through this stage: each file's declared length is counted as it is + written, and a bundle member is one file, so the folder's growth covers it. +- A launch of this title through Xodus/ARM64 Proton reaches the loader with every declared + framework in place, then fails inside Wine: its manifest names `EntryPoint="Forza.App"`, a UWP + application that expects the app model to activate a view, which Wine has no host for. The + Play action reads that entry point from `AppxManifest.xml` and refuses the launch with a + message, and a launch that does reach the runtime reports `uwp-entry-point` rather than dying + silently. A package naming `Windows.FullTrustApplication` starts as an ordinary Win32 + executable instead. +- Activation itself is not the missing piece. A DLL registered under + `HKLM\Software\Microsoft\WindowsRuntime\ActivatableClassId\Windows.ApplicationModel.Core.CoreApplication` + is loaded by `combase.dll`, and its `DllGetActivationFactory` is called for that class. Such a + title still stops earlier: `GetCurrentPackageFullName` and `GetCurrentPackageFamilyName` are + Wine stubs, so a package has no identity, and an activation factory alone cannot supply the + `IFrameworkViewSource`/`CoreWindow` host the view asks for. + - Native AArch64 CLI execution on Thor's existing Linux runtime. - Abstract-socket credential transport with a synthetic account on Thor; this does not establish a real Microsoft sign-in. diff --git a/tools/xbox/README.md b/tools/xbox/README.md index 0f106585bc..fb8705b166 100644 --- a/tools/xbox/README.md +++ b/tools/xbox/README.md @@ -108,11 +108,29 @@ lets ARM64EC Wine load the AMD64 modules through its normal native DLL path. Only each Xbox game's prefix receives these Windows modules. Existing Steam games and other store prefixes retain their selected runtimes. +`droiddeck-legacy-packages.patch` lets the plan, license and extract verbs handle a legacy +package. A bundle is expanded in place, nesting is bounded at three, and each inner EAppx +container is decrypted with the same content keys; inner files without an EAppx header are left +as extracted. `streaming` still serves MSIXVC only. + +`droiddeck-legacy-install.patch` adds `xodus-cli legacy-install`: resolve, license, download, +extract, hoist. Resolution asks the Xbox content service first and Windows Update +(`fe3cr.delivery.mp.microsoft.com`, SOAP) second, because some legacy products return no files +from the content service. The WU endpoint needs no Xbox credential but chains to Microsoft's ECC +Product Root CA 2018, which Android's CA bundle does not carry, so that one public root is +pinned in `wu-root.pem` for these requests. Only `.eappx`/`.eappxbundle` files from a WU result +are kept; the `.appx` framework packages and `.cab` metadata beside them are dropped. Licensing +runs before the first byte and must return content keys. Free space is checked against the +download plus two copies of the largest container, and each container is deleted once unpacked. +The download has no control channel: cancelling kills it and the next attempt starts that +container from zero. + Cloud saves and in-game identity use the reused XGameRuntime/service interfaces; those are game-dependent. Entitlements distinguish purchases from Game Pass access, and the backend performs a fresh license check for downloads and launch. -The store lists PC MSIXVC games only. It does not emulate Xbox consoles or add -support for UWP/APPX packages. Update / verify refreshes an installed package through the normal licensed +The store lists PC packages it can install: streamed MSIXVC, and the legacy EAppx bundles older +desktop titles still ship as. It does not emulate Xbox consoles or host UWP titles in a +packaged-app host. Update / verify refreshes an installed package through the normal licensed streaming path. Manage content resolves manifest-related licensed PC DLC into separate content roots and checks its parent identity after download. These flows still require game/account validation; runtime API presence does not diff --git a/tools/xbox/build-local.sh b/tools/xbox/build-local.sh index c833a403d0..3e61e8eae9 100755 --- a/tools/xbox/build-local.sh +++ b/tools/xbox/build-local.sh @@ -32,7 +32,10 @@ else '"$build" fi mkdir -p "$cache" +threads=() +[[ -n "${RUST_TEST_THREADS:-}" ]] && threads=(-e RUST_TEST_THREADS="$RUST_TEST_THREADS") docker run --rm --platform "$platform" -v "$repo:/src" -v "$cache:/work" -w /src \ - -e CARGO_HOME=/work/cargo -e RUSTUP_HOME=/work/rustup -e XBOX_BUILD_DIR=/work/build "$image" \ + -e CARGO_HOME=/work/cargo -e RUSTUP_HOME=/work/rustup -e XBOX_BUILD_DIR=/work/build \ + ${threads[@]+"${threads[@]}"} "$image" \ bash -lc "set -euo pipefail - trap 'chown -R $(id -u):$(id -g) /work /src/app/src/main/assets/linuxfs 2>/dev/null || true' EXIT$build" + trap 'chown -R $(id -u):$(id -g) /work /src/app/src/main/assets/linuxfs /src/app/build 2>/dev/null || true' EXIT$build" diff --git a/tools/xbox/droiddeck-legacy-install.patch b/tools/xbox/droiddeck-legacy-install.patch new file mode 100644 index 0000000000..bf2013fa8e --- /dev/null +++ b/tools/xbox/droiddeck-legacy-install.patch @@ -0,0 +1,2794 @@ +diff --git a/Cargo.lock b/Cargo.lock +index 0b0f1e0..5dfc09a 100644 +--- a/Cargo.lock ++++ b/Cargo.lock +@@ -7185,12 +7185,14 @@ dependencies = [ + "console-subscriber", + "ctrlc", + "eappx", ++ "flate2", + "fs2", + "futures-util", + "indicatif", + "inquire", + "msixvc", + "nix", ++ "quick-xml 0.42.0", + "reqwest 0.13.4", + "rustix", + "serde_json", +diff --git a/crates/xodus-cli/Cargo.toml b/crates/xodus-cli/Cargo.toml +index 5894ef9..f49470b 100644 +--- a/crates/xodus-cli/Cargo.toml ++++ b/crates/xodus-cli/Cargo.toml +@@ -5,6 +5,7 @@ edition = "2024" + default-run = "xodus-cli" + + [dependencies] ++quick-xml = "0.42.0" + xodus = { path = "../xodus" } + msixvc = { path = "../msixvc" } + eappx = { git = "https://github.com/tuxuser/eappx-rs" } +@@ -25,6 +26,7 @@ futures-util = "0.3.34" + uuid = "1.25.0" + indicatif = "0.18.6" + fs2 = "0.4.3" ++flate2 = "1.0.28" + tempfile = "3.27.0" + rustix = "1.1.4" + ctrlc = "3.5.2" +diff --git a/crates/xodus-cli/src/commands.rs b/crates/xodus-cli/src/commands.rs +index 088a6d0..825c24e 100644 +--- a/crates/xodus-cli/src/commands.rs ++++ b/crates/xodus-cli/src/commands.rs +@@ -2,11 +2,14 @@ pub mod clep; + pub mod download; + pub mod extract; + pub mod extract_eappx; ++pub mod unzip; + pub mod license; ++pub mod legacy_install; + pub mod login; + pub mod logout; + pub mod offline_pc; + pub mod profile; ++pub mod store_delivery; + pub mod cloud_token; + pub mod social; + pub mod game_activity; +diff --git a/crates/xodus-cli/src/commands/extract_eappx.rs b/crates/xodus-cli/src/commands/extract_eappx.rs +index 5ea43da..0bad98e 100644 +--- a/crates/xodus-cli/src/commands/extract_eappx.rs ++++ b/crates/xodus-cli/src/commands/extract_eappx.rs +@@ -51,6 +51,8 @@ pub struct Meter { + pub banked: std::sync::atomic::AtomicU64, + /// Bytes the package being unpacked now will write in total. + pub total: std::sync::atomic::AtomicU64, ++ /// Bytes that packages not yet opened will write. ++ pub planned: std::sync::atomic::AtomicU64, + turn: std::sync::Mutex<()>, + } + +@@ -61,7 +63,14 @@ impl Meter { + let _turn = self.turn.lock().unwrap_or_else(|poisoned| poisoned.into_inner()); + let banked = self.banked.load(Relaxed); + let total = self.total.load(Relaxed); +- (banked + self.written.load(Relaxed).min(total), banked + total) ++ (banked + self.written.load(Relaxed).min(total), banked + total + self.planned.load(Relaxed)) ++ } ++ ++ /// Announces packages that will be opened later, so the total does not grow when they are. ++ fn expect(&self, bytes: u64) { ++ use std::sync::atomic::Ordering::Relaxed; ++ let _turn = self.turn.lock().unwrap_or_else(|poisoned| poisoned.into_inner()); ++ self.planned.fetch_add(bytes, Relaxed); + } + + pub fn done(&self) -> u64 { +@@ -79,6 +88,8 @@ impl Meter { + self.banked.fetch_add(self.total.load(Relaxed), Relaxed); + self.written.store(0, Relaxed); + self.total.store(total, Relaxed); ++ let planned = self.planned.load(Relaxed); ++ self.planned.store(planned.saturating_sub(total), Relaxed); + } + } + +@@ -260,6 +271,9 @@ fn derive( + return Err(Failure::at("nesting", infile, "bundle nesting exceeds the limit")); + } + ++ if let Some(meter) = meter { ++ meter.expect(nested.iter().map(|child| package_size(child)).sum()); ++ } + for child in nested { + let name = child + .file_stem() +@@ -275,6 +289,14 @@ fn derive( + Ok(()) + } + ++/// What a package's blockmap says it will write, or nothing when its header is unreadable. ++fn package_size(path: &Path) -> u64 { ++ let Ok(file) = std::fs::File::open(path) else { return 0 }; ++ EAppxFile::from_stream(&mut std::io::BufReader::new(file)) ++ .map(|package| package.blockmap.files.iter().map(|entry| entry.size).sum()) ++ .unwrap_or(0) ++} ++ + /// Bytes currently on disk under `dir`. + /// + /// A bundle member is written as one file, so a per-file counter says nothing +@@ -414,3 +436,29 @@ fn unpack( + } + Ok(()) + } ++ ++#[cfg(test)] ++mod meter_tests { ++ use super::Meter; ++ ++ #[test] ++ fn announced_packages_keep_the_total_from_growing() { ++ let meter = Meter::default(); ++ meter.roll(100); ++ meter.expect(300); ++ assert_eq!(meter.snapshot(), (0, 400)); ++ meter.written.store(100, std::sync::atomic::Ordering::Relaxed); ++ meter.roll(300); ++ assert_eq!(meter.snapshot(), (100, 400)); ++ meter.written.store(300, std::sync::atomic::Ordering::Relaxed); ++ assert_eq!(meter.snapshot(), (400, 400)); ++ } ++ ++ #[test] ++ fn written_bytes_never_exceed_the_package_being_unpacked() { ++ let meter = Meter::default(); ++ meter.roll(10); ++ meter.written.store(25, std::sync::atomic::Ordering::Relaxed); ++ assert_eq!(meter.snapshot(), (10, 10)); ++ } ++} +diff --git a/crates/xodus-cli/src/commands/legacy_install.rs b/crates/xodus-cli/src/commands/legacy_install.rs +new file mode 100755 +index 0000000..d43c5fc +--- /dev/null ++++ b/crates/xodus-cli/src/commands/legacy_install.rs +@@ -0,0 +1,1198 @@ ++//! Installs a legacy Store title: resolve, license, download, extract. ++//! ++//! `streaming` only understands MSIXVC, where the container *is* the game and ++//! the bytes arrive in the block layout the runtime reads. A legacy title ships ++//! as an EAppx bundle instead, so its bytes have to be fetched whole, licensed ++//! and decrypted before anything is launchable. ++//! ++//! Two delivery routes exist and both are needed in practice: the Xbox content ++//! service serves MSIXVC and some legacy packages, and Windows Update serves the ++//! legacy desktop bundles it refuses (see `store_delivery`). Each is tried in ++//! turn and whichever names a container is used. ++//! ++//! The three verbs under that (`license`, `download`, `extract-eappx`) are chained ++//! here rather than from the app because each process would restart the progress ++//! sequence, and the installer requires it to increase within one stream. ++ ++use std::path::{Path, PathBuf}; ++use std::process::ExitCode; ++use std::time::{Duration, Instant}; ++ ++use futures_util::StreamExt; ++use tokio::io::AsyncWriteExt; ++use xodus::tokens::TokenManager; ++ ++/// Downloaded containers live here until extraction succeeds. They are large and ++/// useless afterwards, so they are removed on success and kept on failure. ++const CONTAINER_DIR: &str = ".xodus-container"; ++/// Content keys for the containers above, removed alongside them. ++const KEYS_DIR: &str = ".xodus-keys"; ++/// The manifest the launcher resolves executables against, expected at the root. ++const MANIFEST: &str = "MicrosoftGame.config"; ++/// A legacy package carries this instead of the config above. ++const LEGACY_MANIFEST: &str = "AppxManifest.xml"; ++/// Framework packages unpack here, and their libraries are copied up beside the title. ++const FRAMEWORK_DIR: &str = ".droiddeck-frameworks"; ++/// Marks a framework whose libraries are already beside the title. ++const INSTALLED: &str = ".installed"; ++/// The libraries `copy_libraries` put beside the title, one name per line. ++const COPIED: &str = ".droiddeck-copied"; ++ ++fn is_legacy_name(name: &str) -> bool { ++ let name = name.to_ascii_lowercase(); ++ name.ends_with(".eappx") || name.ends_with(".eappxbundle") ++} ++ ++/// Free bytes on the filesystem holding `path`, or `None` when it cannot be read. ++/// ++/// The install runs inside the device's own Linux environment, where `df -P -k` is ++/// the one portable free-space query that needs no extra crate, and a failed read ++/// only skips the check rather than blocking a valid install. ++fn free_bytes(path: &Path) -> Option { ++ let output = std::process::Command::new("df").arg("-kP").arg(path).output().ok()?; ++ if !output.status.success() { ++ return None; ++ } ++ let text = String::from_utf8_lossy(&output.stdout); ++ let available = text.lines().nth(1)?.split_whitespace().nth(3)?; ++ available.parse::().ok().map(|kib| kib.saturating_mul(1024)) ++} ++ ++/// Reports why the install stopped without disclosing anything sensitive. ++/// ++/// The app only forwards stderr lines that are on a known allowlist, so a bare ++/// message is invisible there. Two lines are emitted instead: a stable stage code, ++/// and the message reduced to lowercase words with every token that could carry a ++/// path, URL or identifier dropped. ++fn fail(code: &str, message: &str) -> ExitCode { ++ eprintln!("NEXUS_LEGACY_INSTALL_DIAGNOSTIC_V1 {code}"); ++ let detail = bounded(&plain(message)); ++ if !detail.is_empty() { ++ eprintln!("NEXUS_LEGACY_INSTALL_DETAIL_V1 {detail}"); ++ } ++ eprintln!("{message}"); ++ ExitCode::FAILURE ++} ++ ++/// The message reduced to whatever can be shown over a diagnostic line. ++/// Android keeps a detail line only up to 220 characters, so the whole line is cut, not each part. ++fn bounded(line: &str) -> String { ++ let mut cut = line.len().min(220); ++ while !line.is_char_boundary(cut) { ++ cut -= 1; ++ } ++ line[..cut].trim_end().to_string() ++} ++ ++fn plain(message: &str) -> String { ++ let mut detail = String::new(); ++ for word in message.split_whitespace() { ++ if word.contains('/') || word.contains(':') || word.contains('\\') { ++ continue; ++ } ++ let cleaned: String = word ++ .chars() ++ .filter(|c| c.is_ascii_alphanumeric() || *c == '-' || *c == '.') ++ .collect::() ++ .to_ascii_lowercase(); ++ if cleaned.is_empty() { ++ continue; ++ } ++ if !detail.is_empty() { ++ detail.push(' '); ++ } ++ detail.push_str(&cleaned); ++ if detail.len() >= 200 { ++ break; ++ } ++ } ++ detail ++} ++ ++/// `plain` for callers outside this file. ++pub(crate) fn plain_for_log(message: &str) -> String { ++ bounded(&plain(message)) ++} ++ ++/// The installer parses this and only this, so it must match the streaming ++/// schema exactly; `phase` drives its stage display. ++struct Progress { ++ json: bool, ++ sequence: u64, ++ phase: &'static str, ++ completed: u64, ++ total: Option, ++ previous: u64, ++ last: Instant, ++} ++ ++impl Progress { ++ fn new(json: bool) -> Self { ++ let mut progress = Self { ++ json, ++ sequence: 0, ++ phase: "metadata", ++ completed: 0, ++ total: None, ++ previous: 0, ++ last: Instant::now(), ++ }; ++ progress.emit(); ++ progress ++ } ++ ++ fn total(&mut self, total: u64) { ++ self.total = Some(total); ++ self.emit(); ++ } ++ ++ /// Stage transitions are always reported; byte updates are rate limited so a ++ /// multi-gigabyte download cannot flood the pipe. ++ fn phase(&mut self, phase: &'static str) { ++ self.phase = phase; ++ self.completed = 0; ++ self.previous = 0; ++ self.emit(); ++ } ++ ++ fn advance(&mut self, delta: u64) { ++ self.completed = self.completed.saturating_add(delta); ++ if self.last.elapsed() >= Duration::from_millis(200) { ++ self.emit(); ++ } ++ } ++ ++ /// Reports unpacking, which counts bytes neither the download nor a file list ++ /// can see, so the bar advances while a multi-gigabyte member is decrypted. ++ fn bytes(&mut self, completed: u64, total: u64) { ++ self.phase = "installing"; ++ self.completed = completed; ++ self.total = Some(total); ++ if self.last.elapsed() >= Duration::from_millis(700) { ++ self.emit(); ++ } ++ } ++ ++ fn complete(&mut self, success: bool) { ++ self.phase = if success { "complete" } else { "failed" }; ++ self.emit(); ++ } ++ ++ fn emit(&mut self) { ++ let elapsed = self.last.elapsed(); ++ let rate = (self.completed.saturating_sub(self.previous) as f64 / elapsed.as_secs_f64().max(0.001)) as u64; ++ self.previous = self.completed; ++ self.last = Instant::now(); ++ if self.json { ++ let record = serde_json::json!({ ++ "schema": 1, ++ "event": "progress", ++ "sequence": self.sequence, ++ "phase": self.phase, ++ "paused": false, ++ "canPause": false, ++ "completedBytes": self.completed, ++ "totalBytes": self.total, ++ "bytesPerSecond": rate, ++ }); ++ println!("NEXUS_PROGRESS {record}"); ++ } else { ++ match self.total { ++ Some(total) => println!("{}: {}/{} bytes", self.phase, self.completed, total), ++ None => println!("{}: {} bytes", self.phase, self.completed), ++ } ++ } ++ self.sequence += 1; ++ } ++} ++ ++/// A container to fetch, wherever its location came from. ++struct Container { ++ name: String, ++ url: String, ++ size: u64, ++} ++ ++/// What to install and which content id its keys are licensed under. ++struct Install { ++ content_id: String, ++ containers: Vec, ++ /// Why the other route was not used, for the failure message. ++ note: String, ++ /// A Windows Update build whose download locations are asked for once it is licensed. ++ build: Option, ++} ++ ++/// Resolves the legacy containers the Xbox content service is willing to serve. ++/// ++/// Every legacy-format member of the offer is tried, because a product can carry ++/// more than one content id and only some of them are served files. Measured on ++/// FH4: the desktop EAppxBundle is acknowledged with an empty file list while the ++/// console XVC of the same product is served normally, so the empty answer is ++/// reported as such rather than as a title that is not owned. ++async fn from_xbox_service( ++ client: &reqwest::Client, ++ tokens: &TokenManager, ++ product: &str, ++ market: &str, ++ preferred: Option<&str>, ++) -> Result { ++ let region = market.to_uppercase(); ++ let mut candidates: Vec = Vec::new(); ++ if let Some(preferred) = preferred.filter(|id| uuid::Uuid::try_parse(id).is_ok()) { ++ candidates.push(preferred.to_string()); ++ } else if uuid::Uuid::try_parse(product).is_ok() { ++ candidates.push(product.to_string()); ++ } else { ++ let plan = crate::package_plan::resolve(client, &product.to_uppercase(), ®ion) ++ .await ++ .map_err(|error| format!("resolve {error}"))?; ++ for choice in &plan.choices { ++ let legacy_format = choice.format == "EAPPX" || choice.format == "EAPPXBUNDLE"; ++ if legacy_format && !candidates.contains(&choice.content_id) { ++ candidates.push(choice.content_id.clone()); ++ } ++ } ++ // The catalogue names the package a listing shows, which need not be the ++ // container the content service hands over, so the resolver's own choice ++ // is asked first and the rest of the offer is only a fallback. ++ if let Ok(pick) = plan.without_durable_selection().select(None) { ++ candidates.retain(|candidate| *candidate != pick); ++ candidates.insert(0, pick); ++ } ++ } ++ if candidates.is_empty() { ++ return Err("the Store offer names no legacy package".into()); ++ } ++ ++ let mut probe: Vec = Vec::new(); ++ for (index, candidate) in candidates.iter().take(4).enumerate() { ++ match crate::package::get_packages(client, tokens, candidate.clone()).await { ++ Ok(package) => { ++ let legacy: Vec<_> = package ++ .package_files ++ .iter() ++ .filter(|file| is_legacy_name(&file.file_name)) ++ .collect(); ++ probe.push(format!("c{index} n{} l{}", package.package_files.len(), legacy.len())); ++ if legacy.is_empty() { ++ continue; ++ } ++ let mut containers = Vec::new(); ++ for file in legacy { ++ let Some(root) = file.cdn_root_paths.first() else { ++ return Err("a package file has no download location".into()); ++ }; ++ containers.push(Container { ++ name: file.file_name.clone(), ++ url: format!("{root}{}", file.relative_url), ++ size: file.file_size.max(0) as u64, ++ }); ++ } ++ return Ok(Install { content_id: candidate.clone(), containers, note: probe.join(" "), build: None }); ++ } ++ Err(error) => probe.push(format!("c{index} {error}")), ++ } ++ } ++ Err(format!("{} candidates, {}", candidates.len(), probe.join(" "))) ++} ++ ++pub async fn run( ++ client: &reqwest::Client, ++ tokens: &TokenManager, ++ product: String, ++ destination: String, ++ market: Option, ++ selected_content_id: Option, ++ store_id: Option, ++ progress_json: bool, ++ resolve_only: bool, ++) -> ExitCode { ++ let destination = PathBuf::from(destination); ++ if let Err(error) = tokio::fs::create_dir_all(&destination).await { ++ return fail("destination-failed", &format!("Could not prepare the install folder: {error}")); ++ } ++ let _folder_lock = match std::fs::OpenOptions::new() ++ .write(true) ++ .create(true) ++ .truncate(false) ++ .open(destination.join(".nexus-streaming.lock")) ++ .and_then(|file| fs2::FileExt::try_lock_exclusive(&file).map(|()| file)) ++ { ++ Ok(file) => file, ++ Err(_) => return fail("folder-busy", "Another download is using this folder"), ++ }; ++ let mut progress = Progress::new(progress_json); ++ let region = market.clone().unwrap_or("US".into()).to_uppercase(); ++ ++ // One client covers both routes: a package the content service serves is also ++ // downloaded from a delivery host, so both download and update calls use it. ++ let delivery_client = match crate::commands::store_delivery::client() { ++ Ok(client) => client, ++ Err(error) => return fail("update-client-failed", &error), ++ }; ++ let mut update = None; ++ let mut builds: Vec = match from_xbox_service(client, tokens, &product, ®ion, selected_content_id.as_deref()).await { ++ Ok(install) => vec![install], ++ Err(xbox_error) => { ++ // The content service does not serve legacy desktop bundles, so the ++ // Windows Update route is the one that has to work. Its catalogue is ++ // keyed by the Store product, also for an add-on named by content id. ++ let catalogue = store_id.as_deref().unwrap_or(&product); ++ match crate::commands::store_delivery::resolve_builds(&delivery_client, catalogue, ®ion).await { ++ Ok(list) => { ++ let found = (0..list.len()) ++ .map(|index| Install { ++ content_id: list.content_id(index), ++ containers: Vec::new(), ++ note: "windows-update".into(), ++ build: Some(index), ++ }) ++ .collect(); ++ update = Some(list); ++ found ++ } ++ Err(update_error) => { ++ return fail( ++ "no-container", ++ &format!("{xbox_error}; windows update: {update_error}"), ++ ) ++ } ++ } ++ } ++ }; ++ if resolve_only { ++ // Printing locations is what makes a failure diagnosable without spending ++ // hours on a download, so this stops before any byte is written. ++ for install in &builds { ++ println!("content-id: {} ({})", install.content_id, install.note); ++ for container in &install.containers { ++ println!("{} {} {}", container.size, container.name, container.url); ++ } ++ if let (Some(index), Some(list)) = (install.build, update.as_ref()) { ++ match list.locate(&delivery_client, index).await { ++ Ok(files) => files.iter().for_each(|file| println!("{} {} {}", file.size, file.name, file.url)), ++ Err(error) => println!("unavailable: {error}"), ++ } ++ } ++ } ++ return ExitCode::SUCCESS; ++ } ++ ++ // The license verb already writes the [Keys] text eappx reads, so reuse it ++ // rather than re-deriving the content keys here. ++ let keys = destination.join(KEYS_DIR); ++ if let Err(error) = tokio::fs::create_dir_all(&keys).await { ++ return fail("keys-folder-failed", &format!("Could not prepare the key folder: {error}")); ++ } ++ // The standalone verb defaults to "neutral" and that is the market this path ++ // was proven against, so default the same way rather than forwarding a ++ // region the license service may not accept. ++ let license_market = market.clone().unwrap_or("neutral".into()); ++ let refused = xodus::licensing::content::LicenseContentError::DeviceUnregistrable.to_string(); ++ let install = loop { ++ let mut candidate = builds.remove(0); ++ match crate::commands::license::dump( ++ client, ++ tokens, ++ candidate.content_id.clone(), ++ license_market.clone(), ++ keys.display().to_string(), ++ ) ++ .await ++ { ++ Ok(()) => { ++ let (Some(index), Some(list)) = (candidate.build, update.as_ref()) else { break candidate }; ++ match list.locate(&delivery_client, index).await { ++ Ok(files) => { ++ candidate.containers = files ++ .into_iter() ++ .map(|file| Container { name: file.name, url: file.url, size: file.size }) ++ .collect(); ++ break candidate; ++ } ++ Err(_) if !builds.is_empty() => { ++ eprintln!("NEXUS_LEGACY_INSTALL_DETAIL_V1 a licensed build has no download location, trying an older build"); ++ } ++ Err(error) => return fail("no-container", &error), ++ } ++ } ++ Err(error) if error == refused && !builds.is_empty() => { ++ eprintln!("NEXUS_LEGACY_INSTALL_DETAIL_V1 newest build has no license for this device, trying an older build"); ++ } ++ Err(error) => { ++ eprintln!("{error}"); ++ return fail("license-failed", &format!("No license for content {} ({})", candidate.content_id, candidate.note)); ++ } ++ } ++ }; ++ if install.containers.is_empty() { ++ return fail("no-container", "No package files were listed for this title"); ++ } ++ let total: u64 = install.containers.iter().map(|container| container.size).sum(); ++ let largest = install.containers.iter().map(|container| container.size).max().unwrap_or(0); ++ progress.total(total); ++ let mut names = std::collections::HashSet::new(); ++ for container in &install.containers { ++ let Some(name) = Path::new(&container.name).file_name() else { ++ return fail("unusable-name", "Microsoft returned an unusable package name"); ++ }; ++ if !names.insert(name.to_string_lossy().to_ascii_lowercase()) { ++ return fail("duplicate-container", "Microsoft listed two packages under one name"); ++ } ++ } ++ if let Ok(entries) = std::fs::read_dir(destination.join(CONTAINER_DIR)) { ++ for entry in entries.flatten() { ++ let name = entry.file_name().to_string_lossy().to_ascii_lowercase(); ++ if !names.contains(&name) && entry.file_type().is_ok_and(|kind| kind.is_file()) { ++ let _ = std::fs::remove_file(entry.path()); ++ } ++ } ++ } ++ let mut held = 0u64; ++ for container in &install.containers { ++ let Some(name) = Path::new(&container.name).file_name() else { continue }; ++ let staged = destination.join(CONTAINER_DIR).join(name); ++ let Ok(meta) = std::fs::symlink_metadata(&staged) else { continue }; ++ if meta.is_file() && container.size > 0 && meta.len() == container.size { ++ held = held.saturating_add(meta.len()); ++ continue; ++ } ++ if std::fs::remove_file(&staged).is_err() ++ && !(meta.is_file() && std::fs::OpenOptions::new().write(true).truncate(true).open(&staged).is_ok()) ++ { ++ return fail("staging-failed", "Could not clear an unfinished download from the staging folder"); ++ } ++ } ++ if let Some(free) = free_bytes(&destination) { ++ let needed = total.saturating_add(largest).saturating_add(largest / 10); ++ if free.saturating_add(held) < needed { ++ return fail( ++ "insufficient-space", ++ &format!( ++ "An install this size needs about {} GB free to unpack, and {} GB is available", ++ needed / 1_000_000_000, ++ free.saturating_add(held) / 1_000_000_000 ++ ), ++ ); ++ } ++ } ++ let key_file = keys.join("eappx-keys.txt"); ++ if !key_file.is_file() { ++ return fail("no-keys", "The license contained no content keys for this title"); ++ } ++ ++ let store = destination.join(CONTAINER_DIR); ++ if let Err(error) = tokio::fs::create_dir_all(&store).await { ++ return fail("staging-folder-failed", &format!("Could not prepare the staging folder: {error}")); ++ } ++ progress.phase("downloading"); ++ let mut downloaded: Vec = Vec::new(); ++ for container in &install.containers { ++ // Only the container's own name is used, never a server-provided path. ++ let Some(name) = Path::new(&container.name).file_name() else { ++ return fail("unusable-name", "Microsoft returned an unusable package name"); ++ }; ++ let target = store.join(name); ++ // A container a stopped attempt finished is not fetched again: it is most ++ // of an install this size, and the failure that stopped the run was almost ++ // never in its bytes. ++ if let Ok(meta) = tokio::fs::metadata(&target).await { ++ if container.size > 0 && meta.len() == container.size { ++ progress.advance(meta.len()); ++ downloaded.push(target); ++ continue; ++ } ++ } ++ let response = match delivery_client.get(&container.url).send().await { ++ Ok(response) => response, ++ Err(_) => return fail("download-request-failed", "Could not reach the download server"), ++ }; ++ if !response.status().is_success() { ++ return fail( ++ &format!("download-refused-{}", response.status().as_u16()), ++ "The download server refused this package", ++ ); ++ } ++ let mut file = match tokio::fs::File::create(&target).await { ++ Ok(file) => file, ++ Err(error) => return fail("download-write-failed", &format!("Could not write the package: {error}")), ++ }; ++ let mut stream = response.bytes_stream(); ++ while let Some(chunk) = stream.next().await { ++ let Ok(chunk) = chunk else { ++ return fail("download-interrupted", "The transfer was interrupted; resume to retry"); ++ }; ++ if let Err(error) = file.write_all(&chunk).await { ++ return fail("download-write-failed", &format!("Could not write the package: {error}")); ++ } ++ progress.advance(chunk.len() as u64); ++ } ++ if let Err(error) = file.flush().await { ++ return fail("download-flush-failed", &format!("Could not finish writing the package: {error}")); ++ } ++ downloaded.push(target); ++ } ++ ++ progress.phase("installing"); ++ let key_file_text = key_file.display().to_string(); ++ let meter = std::sync::Arc::new(crate::commands::extract_eappx::Meter::default()); ++ // Taken before the first byte is written, so growth of the folder is the ++ // progress of the unpack and not the container that was already downloaded. ++ let baseline = crate::commands::extract_eappx::tree_size(&destination); ++ let mut high = 0u64; ++ for container in &downloaded { ++ let source = container.display().to_string(); ++ // Unpacking never yields, so it runs on a blocking thread and this task ++ // reports the counter while it works. Without that, the phase would only ++ // be visible once it had finished. ++ let unpacking = tokio::task::spawn_blocking({ ++ let destination = destination.clone(); ++ let key_file_text = key_file_text.clone(); ++ let meter = meter.clone(); ++ move || { ++ crate::commands::extract_eappx::run_blocking( ++ source, ++ destination.display().to_string(), ++ Some(key_file_text), ++ Some(&meter), ++ // A full title is tens of gigabytes. Holding the container ++ // while its members are decrypted can fill the device. ++ true, ++ ) ++ } ++ }); ++ let extracted = loop { ++ if unpacking.is_finished() { ++ break unpacking.await.unwrap_or(ExitCode::FAILURE); ++ } ++ tokio::time::sleep(Duration::from_millis(700)).await; ++ let (mut done, total) = meter.snapshot(); ++ if total == 0 { ++ continue; ++ } ++ // A bundle member is one file, so its bytes are only visible on disk. ++ if meter.written.load(std::sync::atomic::Ordering::Relaxed) == 0 { ++ let on_disk = crate::commands::extract_eappx::tree_size(&destination).saturating_sub(baseline); ++ done = done.max(on_disk); ++ } ++ // Deleting the container takes bytes off the disk, so the reported ++ // figure only ever moves forward. ++ high = high.max(done.min(total)); ++ progress.bytes(high, total); ++ }; ++ if extracted != ExitCode::SUCCESS { ++ return fail("extract-failed", "Could not unpack the package; the download is kept for a retry"); ++ } ++ // Drop each container as soon as it is unpacked: a full title is tens of ++ // gigabytes, and holding the archive and its contents at once can fill ++ // the device mid-install. ++ tokio::fs::remove_file(container).await.ok(); ++ } ++ ++ // A bundle extracts each member into its own directory, so the manifest ends ++ // up one level down. The launcher resolves executables relative to the game ++ // folder and requires the manifest at its root, so lift a lone member up. ++ if let Err(error) = hoist(&destination) { ++ return fail("hoist-failed", &error); ++ } ++ ++ tokio::fs::remove_dir_all(&store).await.ok(); ++ ++ // An installed package gets its frameworks from the package graph. Nothing ++ // puts them there for an unpacked folder, and a title that needs one refuses ++ // to load without it, so they are fetched and unpacked beside the title. ++ progress.phase("frameworks"); ++ if let Err(error) = ensure_frameworks(&delivery_client, &destination, ®ion).await { ++ progress.phase("frameworks"); ++ eprintln!("NEXUS_LEGACY_INSTALL_DETAIL_V1 {}", bounded(&format!("dependency skipped; {}", plain(&error)))); ++ } ++ ++ tokio::fs::remove_dir_all(&keys).await.ok(); ++ progress.complete(true); ++ ExitCode::SUCCESS ++} ++ ++/// Installs every framework the title declares that is not there yet. ++/// ++/// Reports the first failure rather than stopping: a title whose framework is ++/// missing still installs, and the launch tries again. ++pub(crate) async fn ensure_frameworks( ++ delivery: &reqwest::Client, ++ destination: &Path, ++ region: &str, ++) -> Result<(), String> { ++ let mut failure = None; ++ for dependency in dependencies(destination) { ++ let marker = destination.join(FRAMEWORK_DIR).join(&dependency).join(INSTALLED); ++ if marker.is_file() { ++ continue; ++ } ++ match install_framework(delivery, destination, &dependency, region).await { ++ Ok(()) => eprintln!("NEXUS_LEGACY_INSTALL_DETAIL_V1 {}", bounded(&format!("framework {} installed", plain(&dependency)))), ++ Err(error) => { ++ eprintln!("NEXUS_LEGACY_INSTALL_DETAIL_V1 {}", bounded(&format!("framework {} skipped {}", plain(&dependency), plain(&error)))); ++ failure.get_or_insert(format!("{dependency}: {error}")); ++ } ++ } ++ } ++ match failure { ++ Some(error) => Err(error), ++ None => Ok(()), ++ } ++} ++/// Framework packages the manifest asks for, in declaration order. ++/// ++/// Reading the attribute rather than parsing XML keeps this off a new dependency; ++/// the manifest is small and the element has one shape. ++pub(crate) fn dependencies(destination: &Path) -> Vec { ++ let Some(manifest) = [LEGACY_MANIFEST, MANIFEST] ++ .iter() ++ .map(|name| destination.join(name)) ++ .find(|path| path.is_file()) ++ else { ++ return Vec::new(); ++ }; ++ let Ok(text) = std::fs::read_to_string(manifest) else { ++ return Vec::new(); ++ }; ++ let mut names: Vec = Vec::new(); ++ for chunk in text.split("').next() else { continue }; ++ let Some(start) = tag.find("Name=\"") else { continue }; ++ let rest = &tag[start + "Name=\"".len()..]; ++ let Some(end) = rest.find('"') else { continue }; ++ let name = rest[..end].trim(); ++ if !name.is_empty() && !names.iter().any(|seen| seen == name) { ++ names.push(name.to_string()); ++ } ++ } ++ names ++} ++ ++/// The COM class the manifest names as the application's entry point. ++/// ++/// A packaged Win32 program names `Windows.FullTrustApplication` and starts like ++/// any other executable. A UWP application names its own class instead and expects ++/// the app model to create it, which no Win32 loader does. ++pub(crate) fn entry_point(destination: &Path) -> Option { ++ let manifest = [LEGACY_MANIFEST, MANIFEST] ++ .iter() ++ .map(|name| destination.join(name)) ++ .find(|path| path.is_file())?; ++ application_entry(&std::fs::read_to_string(manifest).ok()?) ++} ++ ++/// The activation of a manifest's first application: its `EntryPoint`, or `windowsApp` ++/// when it declares that runtime behaviour instead of an entry point. ++fn application_entry(text: &str) -> Option { ++ use quick_xml::events::Event; ++ let mut reader = quick_xml::Reader::from_str(text); ++ loop { ++ match reader.read_event().ok()? { ++ Event::Start(tag) | Event::Empty(tag) if tag.local_name().into_inner() == "Application" => { ++ let mut entry = None; ++ let mut behavior = None; ++ for attribute in tag.attributes().flatten() { ++ let value = attribute.unescape_value().ok()?.trim().to_string(); ++ match attribute.key.local_name().into_inner() { ++ "EntryPoint" => entry = Some(value), ++ "RuntimeBehavior" => behavior = Some(value), ++ _ => {} ++ } ++ } ++ return match behavior.as_deref() { ++ Some(value) if value.eq_ignore_ascii_case("windowsApp") => Some("windowsApp".to_string()), ++ Some(value) if value.eq_ignore_ascii_case("packagedClassicApp") || value.eq_ignore_ascii_case("win32App") => None, ++ _ => entry.filter(|entry| !entry.is_empty()), ++ }; ++ } ++ Event::Eof => return None, ++ _ => {} ++ } ++ } ++} ++ ++/// Downloads one framework package and lays its DLLs beside the title. ++/// ++/// Windows resolves a packaged title's frameworks through the package graph. An ++/// unpacked folder has no graph, and the loader searches the executable's own ++/// directory, so the framework's libraries go there. ++/// ++/// The catalogue names the product but not a package the Xbox Live content ++/// service serves, so the bytes come from the same Windows Update category the ++/// Store client would use, and they arrive as a plain zip rather than EAppx. ++async fn install_framework( ++ delivery: &reqwest::Client, ++ destination: &Path, ++ name: &str, ++ region: &str, ++) -> Result<(), String> { ++ // Framework products from Microsoft publish under this publisher hash; any ++ // other publisher cannot be located by family name alone. ++ let family = format!("{name}_8wekyb3d8bbwe"); ++ let lookup = format!( ++ "https://displaycatalog.mp.microsoft.com/v7.0/products/lookup?alternateId=PackageFamilyName&value={family}&market={region}&languages=en-US&fieldsTemplate=Details" ++ ); ++ let response = delivery ++ .get(lookup) ++ .timeout(Duration::from_secs(20)) ++ .send() ++ .await ++ .map_err(|_| "catalogue unreachable".to_string())?; ++ if !response.status().is_success() { ++ return Err(format!("catalogue answered {}", response.status().as_u16())); ++ } ++ let document: serde_json::Value = response.json().await.map_err(|_| "catalogue answer unreadable".to_string())?; ++ let category = framework_category(&document).ok_or_else(|| "no update category was listed".to_string())?; ++ ++ let published = crate::commands::store_delivery::resolve_frameworks(delivery, &category).await?; ++ if published.files.is_empty() { ++ return Err("the category listed no files".to_string()); ++ } ++ ++ let staging = destination.join(CONTAINER_DIR); ++ tokio::fs::create_dir_all(&staging).await.map_err(|error| format!("staging folder failed: {error}"))?; ++ let unpacked = destination.join(FRAMEWORK_DIR).join(name); ++ if let Some(parent) = unpacked.parent() { ++ tokio::fs::create_dir_all(parent).await.map_err(|error| format!("framework folder failed: {error}"))?; ++ } ++ let trial = staging.join("framework-trial"); ++ // The category lists one package per architecture under names that are only ++ // identifiers, so each is unpacked until one declares x64 in its manifest. ++ if unpacked.is_dir() { ++ withdraw_libraries(&unpacked, destination); ++ } ++ let mut reasons: Vec = Vec::new(); ++ for file in published.files.iter().take(8) { ++ let Some(file_name) = Path::new(&file.name).file_name() else { continue }; ++ let container = staging.join(file_name); ++ let response = match delivery.get(&file.url).send().await { ++ Ok(response) if response.status().is_success() => response, ++ Ok(response) => { ++ reasons.push(format!("file refused with {}", response.status().as_u16())); ++ continue; ++ } ++ Err(_) => { ++ reasons.push("file unreachable".to_string()); ++ continue; ++ } ++ }; ++ let written = tokio::fs::File::create(&container) ++ .await ++ .map_err(|error| format!("download write failed: {error}"))?; ++ let mut written = written; ++ let mut stream = response.bytes_stream(); ++ let mut failed = None; ++ while let Some(chunk) = stream.next().await { ++ match chunk { ++ Ok(chunk) => { ++ if let Err(error) = written.write_all(&chunk).await { ++ failed = Some(format!("download write failed: {error}")); ++ break; ++ } ++ } ++ Err(_) => { ++ failed = Some("download interrupted".to_string()); ++ break; ++ } ++ } ++ } ++ written.flush().await.ok(); ++ drop(written); ++ if let Some(error) = failed { ++ tokio::fs::remove_file(&container).await.ok(); ++ reasons.push(error); ++ continue; ++ } ++ ++ let attempt = trial.join("a"); ++ tokio::fs::remove_dir_all(&attempt).await.ok(); ++ let unpacked_ok = crate::commands::unzip::extract(&container, &attempt); ++ tokio::fs::remove_file(&container).await.ok(); ++ if let Err(error) = unpacked_ok { ++ reasons.push(plain(&error)); ++ continue; ++ } ++ let Some(chosen) = x64_package(&attempt) else { ++ reasons.push("not x64".to_string()); ++ continue; ++ }; ++ tokio::fs::remove_dir_all(&unpacked).await.ok(); ++ tokio::fs::rename(&chosen, &unpacked).await.map_err(|error| format!("framework move failed: {error}"))?; ++ copy_libraries(&unpacked, destination)?; ++ std::fs::write(unpacked.join(INSTALLED), name.as_bytes()).map_err(|error| format!("framework mark failed: {error}"))?; ++ tokio::fs::remove_dir_all(&trial).await.ok(); ++ return Ok(()); ++ } ++ tokio::fs::remove_dir_all(&trial).await.ok(); ++ Err(plain(&format!("no x64 package among {} listed {}", published.files.len(), reasons.join(" ")))) ++} ++ ++/// The folder holding the package that declares x64, however deep a bundle wraps it. ++fn x64_package(root: &Path) -> Option { ++ let mut level = vec![root.to_path_buf()]; ++ for _ in 0..3 { ++ let mut next = Vec::new(); ++ for folder in level { ++ let manifest = folder.join(LEGACY_MANIFEST); ++ if manifest.is_file() { ++ let text = std::fs::read_to_string(&manifest).unwrap_or_default().to_ascii_lowercase(); ++ if text.contains("processorarchitecture=\"x64\"") { ++ return Some(folder); ++ } ++ } ++ if let Ok(entries) = std::fs::read_dir(&folder) { ++ next.extend(entries.filter_map(Result::ok).map(|entry| entry.path()).filter(|path| path.is_dir())); ++ } ++ } ++ if next.is_empty() { ++ break; ++ } ++ level = next; ++ } ++ None ++} ++ ++/// The Windows Update category of the framework's x64 package, or of whichever ++/// package the product lists first when no package names an architecture. ++fn framework_category(document: &serde_json::Value) -> Option { ++ let mut fallback: Option = None; ++ for product in document["Products"].as_array()? { ++ for sku in product["DisplaySkuAvailabilities"].as_array().into_iter().flatten() { ++ for package in sku["Sku"]["Properties"]["Packages"].as_array().into_iter().flatten() { ++ let Some(category) = package["FulfillmentData"]["WuCategoryId"].as_str() else { continue }; ++ if category.is_empty() { ++ continue; ++ } ++ let x64 = package["Architectures"] ++ .as_array() ++ .into_iter() ++ .flatten() ++ .any(|arch| arch.as_str() == Some("x64")); ++ if x64 { ++ return Some(category.to_string()); ++ } ++ fallback.get_or_insert_with(|| category.to_string()); ++ } ++ } ++ } ++ fallback ++} ++ ++/// Removes the libraries an earlier attempt copied, so a replaced framework does ++/// not leave the old architecture's files beside the title. ++/// ++/// Only names this code recorded as copied are removed. A title can ship a DLL of ++/// the same name, and `copy_libraries` leaves such a file alone; deleting by ++/// basename would then remove the title's own library. ++fn withdraw_libraries(from: &Path, to: &Path) { ++ let Ok(list) = std::fs::read_to_string(from.join(COPIED)) else { return }; ++ for name in list.lines().filter(|line| !line.trim().is_empty()) { ++ if Path::new(name).components().count() != 1 { ++ continue; ++ } ++ let _ = std::fs::remove_file(to.join(name)); ++ } ++} ++ ++/// Copies the framework's libraries next to the title, never over its own files, ++/// and records what it copied so a later attempt can take exactly that back. ++fn copy_libraries(from: &Path, to: &Path) -> Result<(), String> { ++ let mut copied: Vec = Vec::new(); ++ copy_libraries_into(from, to, &mut copied)?; ++ std::fs::write(from.join(COPIED), copied.join("\n")).map_err(|error| format!("framework copy list failed: {error}"))?; ++ Ok(()) ++} ++ ++fn copy_libraries_into(from: &Path, to: &Path, copied: &mut Vec) -> Result<(), String> { ++ let entries = std::fs::read_dir(from).map_err(|error| format!("framework read failed: {error}"))?; ++ for entry in entries.filter_map(Result::ok) { ++ let path = entry.path(); ++ if path.is_dir() { ++ copy_libraries_into(&path, to, copied)?; ++ continue; ++ } ++ let named = path.extension().is_some_and(|extension| extension.eq_ignore_ascii_case("dll")); ++ if !named { ++ continue; ++ } ++ let Some(file_name) = path.file_name() else { continue }; ++ let target = to.join(file_name); ++ if target.exists() { ++ continue; ++ } ++ std::fs::copy(&path, &target).map_err(|error| format!("framework copy failed: {error}"))?; ++ copied.push(file_name.to_string_lossy().into_owned()); ++ } ++ Ok(()) ++} ++ ++fn has_manifest(folder: &Path) -> bool { std::fs::read_dir(folder).is_ok_and(|entries| { ++ entries.filter_map(Result::ok).any(|entry| { ++ let name = entry.file_name(); ++ let named = name.eq_ignore_ascii_case(MANIFEST) || name.eq_ignore_ascii_case(LEGACY_MANIFEST); ++ named && entry.file_type().is_ok_and(|kind| kind.is_file()) ++ }) ++ }) ++} ++ ++/// Lifts the contents of a single nested package directory to the game folder. ++/// Add-on content is stored without a manifest, so nothing to lift is not an ++/// error; anything other than exactly one candidate is, because a wrong guess ++/// here would move game files. ++/// ++/// An install over a folder that already holds a manifest is an update: the new ++/// package is lifted over the old one file by file, since leaving it nested would ++/// report success while the previous version stayed in place. ++fn hoist(destination: &Path) -> Result<(), String> { ++ let updating = has_manifest(destination); ++ let mut candidates = nested_packages(destination); ++ if candidates.is_empty() { ++ return Ok(()); ++ } ++ if candidates.len() != 1 { ++ return Err(format!( ++ "Extraction left {} package folders; expected exactly one", ++ candidates.len() ++ )); ++ } ++ let inner = candidates.remove(0); ++ for entry in std::fs::read_dir(&inner) ++ .map_err(|error| format!("Could not list the package folder: {error}"))? ++ .filter_map(Result::ok) ++ { ++ let target = destination.join(entry.file_name()); ++ if target.exists() { ++ if !updating { ++ return Err(format!("Extraction would overwrite {}", target.display())); ++ } ++ let replaced = if target.is_dir() { ++ std::fs::remove_dir_all(&target) ++ } else { ++ std::fs::remove_file(&target) ++ }; ++ replaced.map_err(|error| format!("Could not replace an installed file: {error}"))?; ++ } ++ std::fs::rename(entry.path(), &target) ++ .map_err(|error| format!("Could not move an extracted file: {error}"))?; ++ } ++ std::fs::remove_dir(&inner).map_err(|error| format!("Could not remove the package folder: {error}"))?; ++ Ok(()) ++} ++ ++/// Folders below [root] that carry a manifest, at the depth a nested bundle can ++/// write one. A bundle of a bundle is the deepest the extractor produces. ++fn nested_packages(root: &Path) -> Vec { ++ let mut found = Vec::new(); ++ let mut level = vec![root.to_path_buf()]; ++ for _ in 0..3 { ++ let mut next = Vec::new(); ++ for folder in level { ++ let Ok(entries) = std::fs::read_dir(&folder) else { continue }; ++ for entry in entries.filter_map(Result::ok) { ++ let path = entry.path(); ++ if !path.is_dir() || path.ends_with(CONTAINER_DIR) || path.ends_with(KEYS_DIR) || path.ends_with(FRAMEWORK_DIR) { ++ continue; ++ } ++ if has_manifest(&path) { ++ found.push(path); ++ } else { ++ next.push(path); ++ } ++ } ++ } ++ if next.is_empty() { ++ break; ++ } ++ level = next; ++ } ++ found ++} ++ ++#[cfg(test)] ++mod tests { ++ use super::*; ++ ++ fn scratch(name: &str) -> PathBuf { ++ let path = std::env::temp_dir().join(format!("xodus-legacy-{name}-{}", std::process::id())); ++ let _ = std::fs::remove_dir_all(&path); ++ std::fs::create_dir_all(&path).unwrap(); ++ path ++ } ++ ++ #[test] ++ fn only_eappx_containers_are_treated_as_legacy() { ++ for name in ["a.eappx", "A.EAPPX", "game.eappxbundle", "Game.EAppxBundle"] { ++ assert!(is_legacy_name(name), "{name} should be legacy"); ++ } ++ for name in ["game.msixvc", "game.appx", "game.zip", "eappx", "notes.txt"] { ++ assert!(!is_legacy_name(name), "{name} should not be legacy"); ++ } ++ } ++ ++ #[test] ++ fn free_bytes_reads_the_available_column() { ++ let root = scratch("free"); ++ let free = free_bytes(&root).expect("df should answer for an existing folder"); ++ assert!(free > 0, "a writable filesystem reports some free space"); ++ std::fs::remove_dir_all(&root).ok(); ++ } ++ ++ #[test] ++ fn hoist_lifts_a_lone_bundle_member_to_the_game_folder() { ++ let root = scratch("hoist"); ++ let inner = root.join("Microsoft.SunriseBaseGame"); ++ std::fs::create_dir_all(&inner).unwrap(); ++ std::fs::write(inner.join(MANIFEST), "").unwrap(); ++ std::fs::write(inner.join("ForzaHorizon4.exe"), b"MZ").unwrap(); ++ ++ hoist(&root).unwrap(); ++ ++ assert!(root.join(MANIFEST).is_file()); ++ assert!(root.join("ForzaHorizon4.exe").is_file()); ++ assert!(!inner.exists()); ++ std::fs::remove_dir_all(&root).ok(); ++ } ++ ++ #[test] ++ fn hoist_lifts_a_legacy_package_folder_too() { ++ let root = scratch("legacy-hoist"); ++ let inner = root.join("Microsoft.SunriseBaseGame_2023.1122.1704.2_x64__8wekyb3d8bbwe"); ++ std::fs::create_dir_all(&inner).unwrap(); ++ std::fs::write(inner.join(LEGACY_MANIFEST), "").unwrap(); ++ std::fs::write(inner.join("ForzaHorizon4.exe"), b"MZ").unwrap(); ++ ++ hoist(&root).unwrap(); ++ ++ assert!(root.join(LEGACY_MANIFEST).is_file()); ++ assert!(root.join("ForzaHorizon4.exe").is_file()); ++ assert!(!inner.exists()); ++ std::fs::remove_dir_all(&root).ok(); ++ } ++ ++ #[test] ++ fn hoist_leaves_a_folder_that_already_has_the_manifest_alone() { ++ let root = scratch("flat"); ++ std::fs::write(root.join(MANIFEST), "").unwrap(); ++ std::fs::create_dir_all(root.join("Content")).unwrap(); ++ ++ hoist(&root).unwrap(); ++ ++ assert!(root.join("Content").is_dir()); ++ std::fs::remove_dir_all(&root).ok(); ++ } ++ ++ #[test] ++ fn hoist_replaces_the_previous_install_when_the_manifest_is_already_there() { ++ let root = scratch("update"); ++ std::fs::write(root.join(LEGACY_MANIFEST), "").unwrap(); ++ std::fs::write(root.join("ForzaHorizon4.exe"), b"old").unwrap(); ++ let inner = root.join("Microsoft.SunriseBaseGame"); ++ std::fs::create_dir_all(root.join("Content")).unwrap(); ++ std::fs::create_dir_all(inner.join("Content")).unwrap(); ++ std::fs::write(inner.join(LEGACY_MANIFEST), "").unwrap(); ++ std::fs::write(inner.join("ForzaHorizon4.exe"), b"new").unwrap(); ++ std::fs::write(inner.join("Content").join("pack.bin"), b"data").unwrap(); ++ ++ hoist(&root).unwrap(); ++ ++ assert_eq!(std::fs::read(root.join("ForzaHorizon4.exe")).unwrap(), b"new"); ++ assert!(root.join("Content").join("pack.bin").is_file()); ++ assert!(!inner.exists()); ++ std::fs::remove_dir_all(&root).ok(); ++ } ++ ++ #[test] ++ fn hoist_finds_a_manifest_inside_a_nested_bundle() { ++ let root = scratch("deep"); ++ let inner = root.join("bundle").join("Microsoft.SunriseBaseGame"); ++ std::fs::create_dir_all(&inner).unwrap(); ++ std::fs::write(inner.join(LEGACY_MANIFEST), "").unwrap(); ++ std::fs::write(inner.join("ForzaHorizon4.exe"), b"MZ").unwrap(); ++ ++ hoist(&root).unwrap(); ++ ++ assert!(root.join(LEGACY_MANIFEST).is_file()); ++ assert!(root.join("ForzaHorizon4.exe").is_file()); ++ std::fs::remove_dir_all(&root).ok(); ++ } ++ ++ #[test] ++ fn withdrawing_a_framework_keeps_a_dll_it_never_copied() { ++ let root = scratch("withdraw"); ++ let framework = root.join("framework"); ++ std::fs::create_dir_all(&framework).unwrap(); ++ std::fs::write(framework.join("vcruntime140.dll"), b"framework").unwrap(); ++ std::fs::write(framework.join("title-only.dll"), b"framework").unwrap(); ++ std::fs::write(root.join("title-only.dll"), b"title").unwrap(); ++ ++ std::fs::write(framework.join(COPIED), "vcruntime140.dll").unwrap(); ++ withdraw_libraries(&framework, &root); ++ ++ assert!(!root.join("vcruntime140.dll").exists()); ++ assert_eq!(std::fs::read(root.join("title-only.dll")).unwrap(), b"title"); ++ std::fs::remove_dir_all(&root).ok(); ++ } ++ ++ #[test] ++ fn hoist_refuses_to_guess_between_two_package_folders() { ++ let root = scratch("ambiguous"); ++ for name in ["one", "two"] { ++ let inner = root.join(name); ++ std::fs::create_dir_all(&inner).unwrap(); ++ std::fs::write(inner.join(MANIFEST), "").unwrap(); ++ } ++ ++ assert!(hoist(&root).is_err()); ++ std::fs::remove_dir_all(&root).ok(); ++ } ++ ++ #[test] ++ fn entry_point_separates_a_uwp_application_from_a_packaged_desktop_one() { ++ let root = scratch("entry"); ++ std::fs::write( ++ root.join(LEGACY_MANIFEST), ++ r#""#, ++ ) ++ .unwrap(); ++ assert_eq!(entry_point(&root).as_deref(), Some("Windows.FullTrustApplication")); ++ ++ std::fs::write( ++ root.join(LEGACY_MANIFEST), ++ r#""#, ++ ) ++ .unwrap(); ++ assert_eq!(entry_point(&root).as_deref(), Some("Forza.App")); ++ assert_eq!(application_entry("").as_deref(), Some("Forza.App")); ++ assert_eq!(application_entry(r#""#).as_deref(), Some("windowsApp")); ++ assert_eq!(application_entry(r#""#), None); ++ assert_eq!(application_entry(r#""#), None); ++ assert_eq!(application_entry(r#""#).as_deref(), Some("windowsApp")); ++ std::fs::remove_dir_all(&root).ok(); ++ } ++ ++ #[test] ++ fn download_staging_folders_are_never_hoisted() { ++ let root = scratch("staging"); ++ std::fs::create_dir_all(root.join(CONTAINER_DIR)).unwrap(); ++ std::fs::write(root.join(CONTAINER_DIR).join(MANIFEST), "").unwrap(); ++ ++ hoist(&root).unwrap(); ++ ++ assert!(root.join(CONTAINER_DIR).join(MANIFEST).is_file()); ++ std::fs::remove_dir_all(&root).ok(); ++ } ++ ++ #[test] ++ fn add_on_content_without_a_manifest_is_left_in_place() { ++ let root = scratch("content"); ++ std::fs::create_dir_all(root.join("Content")).unwrap(); ++ std::fs::write(root.join("Content").join("pack.bin"), b"data").unwrap(); ++ ++ hoist(&root).unwrap(); ++ ++ assert!(root.join("Content").join("pack.bin").is_file()); ++ std::fs::remove_dir_all(&root).ok(); ++ } ++} +diff --git a/crates/xodus-cli/src/commands/license.rs b/crates/xodus-cli/src/commands/license.rs +index b9fd91c..fbae301 100644 +--- a/crates/xodus-cli/src/commands/license.rs ++++ b/crates/xodus-cli/src/commands/license.rs +@@ -1,8 +1,6 @@ + use std::process::ExitCode; + + use base64::{Engine, engine::general_purpose::STANDARD}; +-use tokio::fs::OpenOptions; +-use tokio::io::AsyncWriteExt; + use xodus::tokens::TokenManager; + + use crate::license::get_license; +@@ -14,45 +12,47 @@ pub async fn run( + market: String, + ciks: String, + ) -> ExitCode { +- let license = get_license(client, tokens, content_id, market, None, false).await; +- if let Err(err) = license { +- eprintln!("{}", err); +- return ExitCode::FAILURE; ++ match dump(client, tokens, content_id, market, ciks).await { ++ Ok(()) => ExitCode::SUCCESS, ++ Err(err) => { ++ eprintln!("{}", err); ++ ExitCode::FAILURE ++ } + } ++} + +- let (key, game_splicense) = license.unwrap(); +- tokio::fs::create_dir_all(&ciks).await.unwrap(); ++/// Writes the content keys of `content_id` to `ciks`, or returns why Microsoft refused them. ++pub async fn dump( ++ client: &reqwest::Client, ++ tokens: &TokenManager, ++ content_id: String, ++ market: String, ++ ciks: String, ++) -> Result<(), String> { ++ let (key, game_splicense) = get_license(client, tokens, content_id, market, None, false).await?; ++ let unwritten = |error: std::io::Error| format!("The license was granted, but its content keys could not be written: {error}"); ++ tokio::fs::create_dir_all(&ciks).await.map_err(unwritten)?; + // eappx-rs reads a text [Keys] file with base64 Key IDs, while the .cik + // format below is the binary form other tools expect. Write both so the + // same license dump serves either extractor. +- let mut eappx_keys = String::from("[Keys]\n"); ++ let mut eappx_keys = String::from("[Keys] ++"); + for (uuid, content_key) in game_splicense.content_keys { +- let unpacked = content_key.unpack(&key).expect("failed to unpack"); +- let mut file = OpenOptions::new() +- .create(true) +- .write(true) +- .open(format!("{ciks}/{uuid}.cik")) +- .await +- .unwrap(); ++ let unpacked = content_key ++ .unpack(&key) ++ .map_err(|_| "The content key could not be authenticated. Retry the installation or sign in again.".to_string())?; + let uuid_buf = uuid.to_bytes_le(); +- file.write_all(&uuid_buf).await.unwrap(); +- file.write_all(&*unpacked).await.unwrap(); +- file.flush().await.unwrap(); ++ let mut cik = uuid_buf.to_vec(); ++ cik.extend_from_slice(&*unpacked); ++ tokio::fs::write(format!("{ciks}/{uuid}.cik"), cik).await.map_err(unwritten)?; + eappx_keys.push_str(&format!( +- "\"{}\" \"{}\"\n", ++ "\"{}\" \"{}\" ++", + STANDARD.encode(uuid_buf), + STANDARD.encode(&*unpacked), + )); + } +- let mut keys_file = OpenOptions::new() +- .create(true) +- .write(true) +- .truncate(true) +- .open(format!("{ciks}/eappx-keys.txt")) +- .await +- .unwrap(); +- keys_file.write_all(eappx_keys.as_bytes()).await.unwrap(); +- keys_file.flush().await.unwrap(); ++ tokio::fs::write(format!("{ciks}/eappx-keys.txt"), eappx_keys).await.map_err(unwritten)?; + +- ExitCode::SUCCESS ++ Ok(()) + } +diff --git a/crates/xodus-cli/src/commands/store_delivery.rs b/crates/xodus-cli/src/commands/store_delivery.rs +new file mode 100644 +index 0000000..7fec6e5 +--- /dev/null ++++ b/crates/xodus-cli/src/commands/store_delivery.rs +@@ -0,0 +1,577 @@ ++//! Windows Update delivery for Store packages. ++//! ++//! The Xbox content service (`GetBasePackage`) serves MSIXVC titles but answers a ++//! legacy desktop bundle with `PackageFound: true` and an empty file list, on every ++//! request shape. Measured on Forza Horizon 4: its EAppxBundle (85.2 GB) is never ++//! served that way, while the same product's console XVC is served normally as ++//! `.xsp` chunks. ++//! ++//! Those bytes are delivered through Windows Update instead, which is what the ++//! Store client itself uses for legacy packages and which needs no Xbox Live auth. ++//! Three SOAP calls do it: an anonymous cookie, a sync scoped to the product's ++//! `WuCategoryId`, then a signed URL for each file whose digest matches. ++//! ++//! Two details cost time to learn and are relied on below: ++//! * the update fragments inside `` are escaped twice, so they have to be ++//! decoded twice before they contain any tags; ++//! * a file's content digest, not its id, is what ties it to a return URL. ++ ++use std::time::Duration; ++ ++/// The service the Store client talks to. `fe3cr` and `fe3` are both live. ++const BASE: &str = "https://fe3cr.delivery.mp.microsoft.com/ClientWebService/client.asmx"; ++const SECURED: &str = "https://fe3cr.delivery.mp.microsoft.com/ClientWebService/client.asmx/secured"; ++const NAMESPACE: &str = "http://www.microsoft.com/SoftwareDistribution/Server/ClientWebService"; ++const AGENT: &str = "Windows-Update-Agent/10.0.10011.16384 Client-Protocol/2.1"; ++ ++/// The update service is issued from Microsoft's Windows Update PKI, whose root is ++/// distributed by Windows Update itself rather than by Mozilla, so neither this ++/// device's bundle nor a normal Linux install trusts it and every handshake ends ++/// with "unable to get local issuer certificate". Only the roots those hosts chain ++/// to are added, on a client used for this path alone. ++/// ++/// `wu-root.pem` is Microsoft ECC Product Root Certificate Authority 2018, fetched ++/// from the issuer URL in the server's own authority-information extension ++/// (`www.microsoft.com/pkiops/certs/...`). Its serial 14982666DC7CCD8F4053677BB999EC85 ++/// and SHA-1 06F1AA330B927B753A40E68CDF22E34BCBEF3352 match the copy Windows ships ++/// in its own root store, and 2043-02-27 20:50:46 GMT is its expiry. ++const UPDATE_ROOT: &str = include_str!("wu-root.pem"); ++ ++/// A client for the update service and its download hosts: the system trust store ++/// plus the one root above. ++pub fn client() -> Result { ++ let root = reqwest::Certificate::from_pem(UPDATE_ROOT.as_bytes()) ++ .map_err(|error| format!("the pinned update root could not be read: {error}"))?; ++ reqwest::Client::builder() ++ .user_agent(AGENT) ++ .add_root_certificate(root) ++ .build() ++ .map_err(|error| format!("the update client could not be created: {error}")) ++} ++ ++/// Timestamps are not validated by the service, so fixed values keep this free of ++/// a date dependency, as the working reference implementations do. ++const CREATED: &str = "2017-08-05T02:03:05.038Z"; ++const EXPIRES: &str = "2044-08-02T20:09:03Z"; ++ ++/// The device reports itself as a retail Windows desktop: the catalogue serves the ++/// desktop package for this shape, and `App=WU_STORE` is what exposes app categories. ++const ATTRIBUTES: &str = concat!( ++ "BranchReadinessLevel=CB;CurrentBranch=;OEMModel=Virtual Machine;FlightRing=Retail;", ++ "AttrDataVer=21;SystemManufacturer=Microsoft Corporation;InstallLanguage=en-US;", ++ "OSUILocale=en-US;InstallationType=Client;FlightingBranchName=;", ++ "FirmwareVersion=Hyper-V UEFI Release v2.5;SystemProductName=Virtual Machine;OSSkuId=48;", ++ "FlightContent=Mainline;App=WU_STORE;OEMName_Uncleaned=Microsoft Corporation;", ++ "AppVer=0.0.0.0;OSArchitecture=AMD64;SystemSKU=None;UpdateManagementGroup=2;", ++ "IsFlightingEnabled=0;IsDeviceRetailDemo=0;TelemetryLevel=3;OSVersion=10.0.22621.1;", ++ "DeviceFamily=Windows.Desktop;" ++); ++ ++/// Baseline category ids a Windows client always claims to have installed. The ++/// service silently expects them; without them the sync returns no metadata. ++const BASELINE: &[u32] = &[ ++ 1, 2, 3, 10, 11, 17, 19, 2359974, 2359977, 5143990, 5169043, 5169044, 5169047, 8788830, 8806526, ++ 9125350, 9154769, 10809856, 23110993, 23110994, 23110995, 23110996, 23110999, 23111000, ++ 23111001, 23111002, 23111003, 23111004, 24513870, 28880263, 30077688, 30486944, 59830006, ++ 59830007, 59830008, 60484010, 62450018, 62450019, 62450020, 98959022, 98959023, 98959024, ++ 98959025, 98959026, 105939029, 105995585, 106017178, 107825194, 117765322, 129905029, ++ 130040030, 130040031, 130040032, 130040033, 133399034, 138372035, 138372036, 139536037, ++ 139536038, 139536039, 139536040, 142045136, 158941041, 158941042, 158941043, 158941044, ++ 159776047, 160733048, 160733049, 160733050, 160733051, 160733055, 160733056, 161870057, ++ 161870058, 161870059, ++]; ++ ++/// One downloadable package file. ++pub struct DeliveryFile { ++ pub name: String, ++ pub url: String, ++ pub size: u64, ++} ++ ++/// A product's installable files, plus the content id its keys are licensed under. ++pub struct Delivery { ++ pub content_id: String, ++ pub files: Vec, ++} ++ ++fn unescape(text: &str) -> String { ++ // Fragments are escaped twice, so a single pass leaves `<Files>` behind. ++ let mut current = text.to_string(); ++ for _ in 0..3 { ++ let next = current ++ .replace("<", "<") ++ .replace(">", ">") ++ .replace(""", "\"") ++ .replace("'", "'") ++ .replace("&", "&"); ++ if next == current { ++ break; ++ } ++ current = next; ++ } ++ current ++} ++ ++fn element(text: &str, tag: &str) -> Option { ++ let open = format!("<{tag}>"); ++ let close = format!(""); ++ let start = text.find(&open)? + open.len(); ++ let end = start + text[start..].find(&close)?; ++ Some(text[start..end].to_string()) ++} ++ ++fn blocks(text: &str, tag: &str) -> Vec { ++ let open = format!("<{tag}>"); ++ let close = format!(""); ++ let mut found = Vec::new(); ++ let mut rest = text; ++ while let Some(start) = rest.find(&open) { ++ let after = start + open.len(); ++ let Some(length) = rest[after..].find(&close) else { break }; ++ found.push(rest[after..after + length].to_string()); ++ rest = &rest[after + length + close.len()..]; ++ } ++ found ++} ++ ++fn attribute(text: &str, name: &str) -> Option { ++ let needle = format!("{name}=\""); ++ let start = text.find(&needle)? + needle.len(); ++ let end = start + text[start..].find('"')?; ++ Some(text[start..end].to_string()) ++} ++ ++async fn soap(client: &reqwest::Client, url: &str, body: &str) -> Result { ++ let response = client ++ .post(url) ++ .header("Content-Type", "application/soap+xml; charset=utf-8") ++ .header("User-Agent", AGENT) ++ .timeout(Duration::from_secs(60)) ++ .body(body.to_string()) ++ .send() ++ .await ++ .map_err(|error| format!("update service unreachable: {error}"))?; ++ if !response.status().is_success() { ++ return Err(format!("update service returned HTTP {}", response.status().as_u16())); ++ } ++ let text = response.text().await.map_err(|_| "update service reply unreadable".to_string())?; ++ Ok(unescape(&text)) ++} ++ ++fn envelope(action: &str, to: &str, body: &str) -> String { ++ format!( ++ r#"http://www.microsoft.com/SoftwareDistribution/Server/ClientWebService/{action}urn:uuid:{message}{to}{CREATED}{EXPIRES}{body}"#, ++ action = action, ++ to = to, ++ message = uuid::Uuid::new_v4(), ++ body = body, ++ ) ++} ++ ++/// Reads the `WuCategoryId` the catalogue carries for the product's packages. ++async fn category(client: &reqwest::Client, product: &str, market: &str) -> Result { ++ let url = format!( ++ "https://displaycatalog.mp.microsoft.com/v7.0/products/{product}?market={market}&languages=en,neutral" ++ ); ++ let text = client ++ .get(url) ++ .timeout(Duration::from_secs(30)) ++ .send() ++ .await ++ .map_err(|_| "Store metadata request failed".to_string())? ++ .text() ++ .await ++ .map_err(|_| "Store metadata reply unreadable".to_string())?; ++ let start = text ++ .find("\"WuCategoryId\":\"") ++ .ok_or("This product has no Windows Update category, so it has no legacy package")?; ++ let rest = &text[start + "\"WuCategoryId\":\"".len()..]; ++ let end = rest.find('"').ok_or("Unreadable Windows Update category")?; ++ Ok(rest[..end].to_string()) ++} ++ ++async fn cookie(client: &reqwest::Client) -> Result<(String, String), String> { ++ let body = format!( ++ r#"2016-07-27T07:18:09Z2015-10-21T17:01:07.1472913Z{CREATED}2.50"# ++ ); ++ let reply = soap(client, BASE, &envelope("GetCookie", BASE, &body)).await?; ++ let encrypted = element(&reply, "EncryptedData").ok_or("Update service issued no session cookie")?; ++ let expiration = element(&reply, "Expiration").unwrap_or_else(|| EXPIRES.to_string()); ++ Ok((encrypted, expiration)) ++} ++ ++struct Revision { ++ update_id: String, ++ revision: String, ++ digest: String, ++ name: String, ++ size: u64, ++ moniker: String, ++ content_id: String, ++} ++ ++fn package_identity(moniker: &str) -> Option<((String, String), Vec)> { ++ let mut parts = moniker.split('_'); ++ let name = parts.next()?.to_ascii_lowercase(); ++ let version = parts.next()?.split('.').map(|part| part.parse::().ok()).collect::>>()?; ++ let arch = parts.next()?.to_ascii_lowercase(); ++ if name.is_empty() { ++ return None; ++ } ++ Some(((name, arch), version)) ++} ++ ++fn newest_builds(revisions: Vec) -> Vec { ++ let desktop: std::collections::HashSet = revisions ++ .iter() ++ .filter_map(|revision| package_identity(&revision.moniker)) ++ .filter(|((_, arch), _)| arch == "x64" || arch == "neutral") ++ .map(|((name, _), _)| name) ++ .collect(); ++ let mut newest: std::collections::HashMap<(String, String), Vec> = std::collections::HashMap::new(); ++ for revision in &revisions { ++ if let Some((key, version)) = package_identity(&revision.moniker) { ++ let entry = newest.entry(key).or_default(); ++ if version > *entry { ++ *entry = version; ++ } ++ } ++ } ++ let mut seen = std::collections::HashSet::new(); ++ revisions ++ .into_iter() ++ .filter(|revision| match package_identity(&revision.moniker) { ++ Some((key, version)) => { ++ newest.get(&key) == Some(&version) ++ && (!desktop.contains(&key.0) || key.1 == "x64" || key.1 == "neutral") ++ } ++ None => true, ++ }) ++ .filter(|revision| seen.insert(revision.name.to_ascii_lowercase())) ++ .collect() ++} ++ ++/// One packager per file: the revision identity and the digest are what the ++/// download URL is issued against. ++async fn revisions( ++ client: &reqwest::Client, ++ cookie: &str, ++ expiration: &str, ++ content_id: &str, ++ wanted: fn(&str) -> bool, ++) -> Result<(Vec, String), String> { ++ let baseline: String = BASELINE.iter().map(|id| format!("{id}")).collect(); ++ let body = format!( ++ r#"{expiration}{cookie}false{baseline}falsetrue{content_id}truefalseExtendeden-USenen-USfalse{ATTRIBUTES}Interactive=1;IsSeeker=0;"# ++ ); ++ let reply = soap(client, BASE, &envelope("SyncUpdates", BASE, &body)).await?; ++ ++ let core = blocks(&reply, "UpdateInfo"); ++ let extended = blocks(&reply, "ExtendedUpdateInfo"); ++ let Some(extended) = extended.first() else { ++ return Err("Update service returned no package metadata for this category".into()); ++ }; ++ let mut core_by_id: Vec<(String, String)> = Vec::new(); ++ for entries in &core { ++ if let (Some(id), Some(xml)) = (element(entries, "ID"), element(entries, "Xml")) { ++ core_by_id.push((id, xml)); ++ } ++ } ++ ++ let mut found = Vec::new(); ++ let mut package_content_id = String::new(); ++ for entries in blocks(extended, "Update") { ++ let (Some(id), Some(xml)) = (element(&entries, "ID"), element(&entries, "Xml")) else { continue }; ++ // The file list lives in one fragment and the identity the URL is issued ++ // against in the other, so both are needed for the same revision. ++ let Some((_, core_xml)) = core_by_id.iter().find(|(key, _)| *key == id) else { continue }; ++ let combined = format!("{xml}{core_xml}"); ++ if !combined.contains("` node. ++fn file_attributes(text: &str) -> Vec { ++ let mut found = Vec::new(); ++ let mut rest = text; ++ while let Some(start) = rest.find("') else { break }; ++ found.push(after[..end].trim_end_matches('/').to_string()); ++ rest = &after[end..]; ++ } ++ found ++} ++ ++async fn download_url( ++ client: &reqwest::Client, ++ cookie: &str, ++ expiration: &str, ++ revision: &Revision, ++) -> Result { ++ let body = format!( ++ r#"{expiration}{cookie}{update}{number}FileUrlFileDecryption{ATTRIBUTES}"#, ++ update = revision.update_id, ++ number = revision.revision ++ ); ++ let reply = soap(client, SECURED, &envelope("GetExtendedUpdateInfo2", SECURED, &body)).await?; ++ for location in blocks(&reply, "FileLocation") { ++ if element(&location, "FileDigest").as_deref() != Some(revision.digest.as_str()) { ++ continue; ++ } ++ if let Some(url) = element(&location, "Url") { ++ return Ok(url); ++ } ++ } ++ Err(format!("No download location was issued for {}", revision.name)) ++} ++ ++/// Resolves the legacy packages of a Store product through Windows Update. ++pub async fn resolve( ++ client: &reqwest::Client, ++ product: &str, ++ market: &str, ++) -> Result { ++ let category = category(client, product, market).await?; ++ resolve_category(client, &category).await ++} ++ ++/// Every build of a Store product Windows Update lists, newest first. Each build ++/// has its own content id, and Microsoft can refuse keys for one but not another. ++pub struct Builds { ++ cookie: String, ++ expiration: String, ++ category: String, ++ groups: Vec>, ++} ++ ++impl Builds { ++ pub fn len(&self) -> usize { ++ self.groups.len() ++ } ++ ++ pub fn content_id(&self, index: usize) -> String { ++ self.groups[index] ++ .first() ++ .map(|revision| revision.content_id.clone()) ++ .filter(|id| !id.is_empty()) ++ .unwrap_or_else(|| self.category.clone()) ++ } ++ ++ /// Download locations for one build, asked for only when that build is used. ++ pub async fn locate(&self, client: &reqwest::Client, index: usize) -> Result, String> { ++ let mut files = Vec::new(); ++ for revision in &self.groups[index] { ++ let url = download_url(client, &self.cookie, &self.expiration, revision).await?; ++ files.push(DeliveryFile { name: revision.name.clone(), url, size: revision.size }); ++ } ++ Ok(files) ++ } ++} ++ ++pub async fn resolve_builds(client: &reqwest::Client, product: &str, market: &str) -> Result { ++ let category = category(client, product, market).await?; ++ let (cookie, expiration) = cookie(client).await?; ++ let (found, _) = revisions(client, &cookie, &expiration, &category, title_container).await?; ++ let groups = builds_newest_first(found); ++ if groups.is_empty() { ++ return Err("Windows Update lists no package files for this product".into()); ++ } ++ Ok(Builds { cookie, expiration, category, groups }) ++} ++ ++fn title_container(name: &str) -> bool { ++ let lower = name.to_ascii_lowercase(); ++ lower.ends_with(".eappx") || lower.ends_with(".eappxbundle") ++} ++ ++fn builds_newest_first(revisions: Vec) -> Vec> { ++ let mut groups: Vec<(String, Vec)> = Vec::new(); ++ for revision in revisions { ++ match groups.iter_mut().find(|(id, _)| *id == revision.content_id) { ++ Some((_, group)) => group.push(revision), ++ None => groups.push((revision.content_id.clone(), vec![revision])), ++ } ++ } ++ let mut groups: Vec> = groups.into_iter().map(|(_, group)| newest_builds(group)).collect(); ++ let newest = |group: &Vec| { ++ group.iter().filter_map(|revision| package_identity(&revision.moniker)).map(|(_, version)| version).max() ++ }; ++ groups.sort_by(|a, b| newest(b).cmp(&newest(a))); ++ groups.into_iter().filter(|group| !group.is_empty()).collect() ++} ++ ++/// The packages a Windows Update category lists, in the containers a title is ++/// packed in. ++pub async fn resolve_category(client: &reqwest::Client, category: &str) -> Result { ++ resolve_wanted(client, category, true, |name| { ++ let lower = name.to_ascii_lowercase(); ++ lower.ends_with(".eappx") || lower.ends_with(".eappxbundle") ++ }) ++ .await ++} ++ ++/// The plain `.appx` packages a category lists, which is how a framework ships. ++pub async fn resolve_frameworks(client: &reqwest::Client, category: &str) -> Result { ++ resolve_wanted(client, category, false, |name| name.to_ascii_lowercase().ends_with(".appx")).await ++} ++ ++async fn resolve_wanted( ++ client: &reqwest::Client, ++ category: &str, ++ newest_only: bool, ++ wanted: fn(&str) -> bool, ++) -> Result { ++ let (cookie, expiration) = cookie(client).await?; ++ let (revisions, package_content_id) = revisions(client, &cookie, &expiration, category, wanted).await?; ++ let revisions = if newest_only { newest_builds(revisions) } else { revisions }; ++ if revisions.is_empty() { ++ return Err("Windows Update lists no package files for this product".into()); ++ } ++ let mut files = Vec::new(); ++ for revision in &revisions { ++ let url = download_url(client, &cookie, &expiration, revision).await?; ++ files.push(DeliveryFile { name: revision.name.clone(), url, size: revision.size }); ++ } ++ Ok(Delivery { ++ // A framework's revision carries no package content id, and only a title's ++ // keys are licensed, so the category stands in when none was named. ++ content_id: if package_content_id.is_empty() { ++ category.to_string() ++ } else { ++ package_content_id ++ }, ++ files, ++ }) ++} ++ ++#[cfg(test)] ++mod tests { ++ use super::*; ++ ++ #[test] ++ fn the_pinned_root_is_the_expected_microsoft_update_authority() { ++ // The PEM travels with the patch, so a swapped or truncated file must fail ++ // here rather than silently widening what the update client trusts. ++ assert_eq!(UPDATE_ROOT.matches("BEGIN CERTIFICATE").count(), 1); ++ let der = UPDATE_ROOT ++ .lines() ++ .filter(|line| !line.starts_with("-----")) ++ .collect::(); ++ use base64::{Engine, engine::general_purpose::STANDARD}; ++ let der = STANDARD.decode(der).expect("the pinned root is not valid base64"); ++ assert!(der.len() > 500 && der[0] == 0x30, "the pinned root is not a DER SEQUENCE"); ++ // Serial number and subject live verbatim in the DER for this certificate. ++ assert!(der.windows(4).any(|w| w == [0x14, 0x98, 0x26, 0x66]), "unexpected serial"); ++ let subject = b"Microsoft ECC Product Root Certificate Authority 2018"; ++ assert!( ++ der.windows(subject.len()).any(|window| window == subject), ++ "unexpected subject" ++ ); ++ assert!(client().is_ok()); ++ } ++ ++ fn build(name: &str, moniker: &str) -> Revision { ++ Revision { update_id: String::new(), revision: "1".into(), digest: String::new(), name: name.into(), size: 1, moniker: moniker.into(), content_id: String::new() } ++ } ++ ++ fn licensed(name: &str, moniker: &str, content_id: &str) -> Revision { ++ Revision { content_id: content_id.into(), ..build(name, moniker) } ++ } ++ ++ #[test] ++ fn each_build_is_offered_under_its_own_content_id_newest_first() { ++ let builds = builds_newest_first(vec![ ++ licensed("old.eappx", "Microsoft.MSEsper_1.3.25854.2_x64__8wekyb3d8bbwe", "old-id"), ++ licensed("new.eappx", "Microsoft.MSEsper_1.3.25854.70_x64__8wekyb3d8bbwe", "new-id"), ++ licensed("new-arm.eappx", "Microsoft.MSEsper_1.3.25854.70_arm64__8wekyb3d8bbwe", "new-id"), ++ ]); ++ let shape: Vec<(String, Vec)> = builds ++ .iter() ++ .map(|group| (group[0].content_id.clone(), group.iter().map(|r| r.name.clone()).collect())) ++ .collect(); ++ assert_eq!(shape, [("new-id".to_string(), vec!["new.eappx".to_string()]), ("old-id".to_string(), vec!["old.eappx".to_string()])]); ++ } ++ ++ #[test] ++ fn only_the_newest_desktop_build_of_each_package_is_kept() { ++ let kept = newest_builds(vec![ ++ build("old.eappx", "Microsoft.MSEsper_1.3.25854.2_x64__8wekyb3d8bbwe"), ++ build("new.eappx", "Microsoft.MSEsper_1.3.25854.70_x64__8wekyb3d8bbwe"), ++ build("arm.eappx", "Microsoft.MSEsper_1.3.25854.70_arm64__8wekyb3d8bbwe"), ++ build("bare.eappxbundle", ""), ++ build("NEW.eappx", "Microsoft.MSEsper_1.3.25854.70_x64__8wekyb3d8bbwe"), ++ ]); ++ let names: Vec<_> = kept.iter().map(|r| r.name.as_str()).collect(); ++ assert_eq!(names, ["new.eappx", "bare.eappxbundle"]); ++ assert_eq!(newest_builds(vec![build("x86.eappx", "Game_1.0.0.0_x86__p")]).len(), 1); ++ } ++ ++ #[test] ++ fn fragments_escaped_twice_are_decoded_until_stable() { ++ assert_eq!(unescape("&lt;Files&gt;"), ""); ++ assert_eq!(unescape("<File FileName="a.eappxbundle"/>"), ""); ++ assert_eq!(unescape("plain text"), "plain text"); ++ } ++ ++ #[test] ++ fn self_closing_file_nodes_yield_their_attributes() { ++ let text = r#""#; ++ let attributes = file_attributes(text); ++ assert_eq!(attributes.len(), 2); ++ assert_eq!(attribute(&attributes[0], "FileName").as_deref(), Some("x.eappxbundle")); ++ assert_eq!(attribute(&attributes[0], "Size").as_deref(), Some("99")); ++ assert_eq!(attribute(&attributes[1], "FileName").as_deref(), Some("Abm_1.cab")); ++ } ++ ++ #[test] ++ fn blocks_stop_at_the_matching_close_tag() { ++ let text = "12"; ++ let updates = blocks(text, "Update"); ++ assert_eq!(updates.len(), 2); ++ assert_eq!(element(&updates[1], "ID").as_deref(), Some("2")); ++ } ++ ++ #[test] ++ fn element_returns_none_when_absent() { ++ assert!(element("1", "B").is_none()); ++ assert_eq!(element("", "A").as_deref(), Some("")); ++ } ++} +diff --git a/crates/xodus-cli/src/commands/streaming.rs b/crates/xodus-cli/src/commands/streaming.rs +index a0136f8..8637cab 100644 +--- a/crates/xodus-cli/src/commands/streaming.rs ++++ b/crates/xodus-cli/src/commands/streaming.rs +@@ -9,7 +9,7 @@ use indicatif::{MultiProgress, ProgressBar, ProgressStyle}; + use msixvc::streaming; + use msixvc::xvd::{SegmentFile, XvdFile}; + use tokio::fs::{File, OpenOptions}; +-use tokio::io::AsyncRead; ++use tokio::io::{AsyncRead, AsyncReadExt, AsyncSeekExt}; + use tokio::sync::mpsc::{Receiver, Sender}; + use uuid::Uuid; + use xodus::tokens::TokenManager; +@@ -354,6 +354,15 @@ where + Ok(file) => file, + Err(error) => { failure::report(&error); return ExitCode::FAILURE; } + }; ++ let mut magic = [0u8; 4]; ++ let plain = remote_file.read_exact(&mut magic).await.is_ok() && magic == [0x50, 0x4b, 0x03, 0x04]; ++ if let Err(error) = remote_file.seek(std::io::SeekFrom::Start(0)).await { ++ failure::report(&error); ++ return ExitCode::FAILURE; ++ } ++ if plain { ++ return plain_package(remote_file, l, out).await; ++ } + let remote_xvd = match XvdFile::parse(&mut remote_file).await { + Ok(xvd) => xvd, + Err(error) => { failure::report_metadata(error.as_ref()); return ExitCode::FAILURE; } +@@ -692,3 +701,337 @@ where + } + ExitCode::SUCCESS + } ++ ++const PLAIN_LIMIT: u64 = 1 << 30; ++ ++const PENDING: &str = ".pending"; ++ ++fn publish(staging: &Path, out: &Path) -> Result<(), String> { ++ let previous = out.join(".xodus-plain-previous"); ++ if previous.join(PENDING).exists() { ++ restore(&previous, out).map_err(|error| format!("an earlier replacement could not be undone: {error}"))?; ++ } else if previous.exists() && std::fs::remove_dir_all(&previous).is_err() { ++ let stale = out.join(format!(".xodus-plain-stale-{}", std::process::id())); ++ std::fs::rename(&previous, &stale).map_err(|error| format!("an earlier replacement left files behind: {error}"))?; ++ } ++ std::fs::create_dir_all(&previous).map_err(|error| format!("could not prepare the replacement: {error}"))?; ++ std::fs::write(previous.join(PENDING), b"").map_err(|error| format!("could not prepare the replacement: {error}"))?; ++ let mut names: Vec = std::fs::read_dir(staging) ++ .map_err(|error| format!("staged package unreadable: {error}"))? ++ .map(|entry| entry.map(|entry| entry.file_name())) ++ .collect::>() ++ .map_err(|error| format!("staged package unreadable: {error}"))?; ++ names.retain(|name| name != PENDING); ++ names.sort(); ++ let mut placed = Vec::new(); ++ let result = (|| { ++ for name in &names { ++ let target = out.join(name); ++ if std::fs::symlink_metadata(&target).is_ok() { ++ std::fs::rename(&target, previous.join(name)) ++ .map_err(|error| format!("could not replace {}: {error}", target.display()))?; ++ } ++ std::fs::rename(staging.join(name), &target) ++ .map_err(|error| format!("could not place {}: {error}", target.display()))?; ++ placed.push(name.clone()); ++ } ++ Ok::<(), String>(()) ++ })(); ++ match result { ++ Ok(()) => { ++ std::fs::remove_file(previous.join(PENDING)).ok(); ++ std::fs::remove_dir_all(&previous).ok(); ++ Ok(()) ++ } ++ Err(error) => { ++ let stuck: Vec = placed ++ .iter() ++ .filter(|name| remove(&out.join(name)).is_err()) ++ .map(|name| name.to_string_lossy().into_owned()) ++ .collect(); ++ if !stuck.is_empty() { ++ let listed = std::fs::write(previous.join(PENDING), stuck.join(" ++")); ++ let names = stuck.join(", "); ++ return Err(match listed { ++ Ok(()) => format!("{error}; {names} could not be removed, the previous files are kept in {}", previous.display()), ++ Err(write) => format!("{error}; {names} could not be removed and could not be recorded ({write})"), ++ }); ++ } ++ match restore(&previous, out) { ++ Ok(()) => Err(error), ++ Err(undo) => Err(format!("{error}; the previous files are kept in {} ({undo})", previous.display())), ++ } ++ } ++ } ++} ++ ++/// Moves every entry kept in `previous` back into `out`, then removes `previous`. ++/// On any failure `previous` stays, holding what has not been put back. ++fn restore(previous: &Path, out: &Path) -> Result<(), String> { ++ let placed = std::fs::read_to_string(previous.join(PENDING)).map_err(|error| error.to_string())?; ++ for name in placed.lines().filter(|name| !name.is_empty() && !name.contains(['/', '\\']) && *name != "." && *name != "..") { ++ let target = out.join(name); ++ if std::fs::symlink_metadata(&target).is_ok() { ++ remove(&target).map_err(|error| format!("could not clear {}: {error}", target.display()))?; ++ } ++ } ++ if !placed.is_empty() { ++ std::fs::write(previous.join(PENDING), b"").map_err(|error| error.to_string())?; ++ } ++ let entries = std::fs::read_dir(previous).map_err(|error| error.to_string())?; ++ for entry in entries { ++ let entry = entry.map_err(|error| error.to_string())?; ++ if entry.file_name() == PENDING { ++ continue; ++ } ++ let target = out.join(entry.file_name()); ++ if std::fs::symlink_metadata(&target).is_ok() { ++ remove(&target).map_err(|error| format!("could not clear {}: {error}", target.display()))?; ++ } ++ std::fs::rename(entry.path(), &target).map_err(|error| format!("could not restore {}: {error}", target.display()))?; ++ } ++ std::fs::remove_file(previous.join(PENDING)).map_err(|error| error.to_string())?; ++ std::fs::remove_dir(previous).map_err(|error| error.to_string()) ++} ++ ++fn remove(path: &Path) -> std::io::Result<()> { ++ if std::fs::symlink_metadata(path)?.is_dir() { std::fs::remove_dir_all(path) } else { std::fs::remove_file(path) } ++} ++ ++async fn plain_package(mut remote: R, length: u64, out: &Path) -> ExitCode { ++ if length > PLAIN_LIMIT { ++ eprintln!("NEXUS_TRANSFER_DIAGNOSTIC_V1 plain-package-too-large 0"); ++ eprintln!("This package is a plain archive larger than this installer unpacks."); ++ return ExitCode::FAILURE; ++ } ++ let archive = out.join(".xodus-plain-package.zip"); ++ let written = async { ++ let mut file = File::create(&archive).await?; ++ let copied = tokio::io::copy(&mut remote, &mut file).await?; ++ tokio::io::AsyncWriteExt::flush(&mut file).await?; ++ Ok::(copied) ++ } ++ .await; ++ match written { ++ Ok(copied) if copied == length => {} ++ Ok(_) => { ++ std::fs::remove_file(&archive).ok(); ++ eprintln!("NEXUS_TRANSFER_DIAGNOSTIC_V1 transfer-short 0"); ++ eprintln!("The package ended early. Retry this installation to obtain a fresh package link."); ++ return ExitCode::FAILURE; ++ } ++ Err(error) => { ++ std::fs::remove_file(&archive).ok(); ++ failure::report(&error); ++ return ExitCode::FAILURE; ++ } ++ } ++ let staging = out.join(".xodus-plain-staging"); ++ std::fs::remove_dir_all(&staging).ok(); ++ let unpacked = crate::commands::unzip::extract(&archive, &staging).and_then(|()| publish(&staging, out)); ++ std::fs::remove_file(&archive).ok(); ++ std::fs::remove_dir_all(&staging).ok(); ++ match unpacked { ++ Ok(()) => ExitCode::SUCCESS, ++ Err(error) => { ++ eprintln!("NEXUS_TRANSFER_DIAGNOSTIC_V1 plain-package-invalid 0"); ++ eprintln!("Could not unpack the package: {error}"); ++ ExitCode::FAILURE ++ } ++ } ++} ++ ++#[cfg(test)] ++mod plain_tests { ++ fn stored_zip(name: &str, data: &[u8]) -> Vec { ++ let mut out = Vec::new(); ++ let n = name.len() as u16; ++ let size = data.len() as u32; ++ out.extend_from_slice(&[0x50, 0x4b, 0x03, 0x04, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]); ++ out.extend_from_slice(&size.to_le_bytes()); ++ out.extend_from_slice(&size.to_le_bytes()); ++ out.extend_from_slice(&n.to_le_bytes()); ++ out.extend_from_slice(&0u16.to_le_bytes()); ++ out.extend_from_slice(name.as_bytes()); ++ out.extend_from_slice(data); ++ let central = out.len() as u32; ++ out.extend_from_slice(&[0x50, 0x4b, 0x01, 0x02, 20, 0, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]); ++ out.extend_from_slice(&size.to_le_bytes()); ++ out.extend_from_slice(&size.to_le_bytes()); ++ out.extend_from_slice(&n.to_le_bytes()); ++ out.extend_from_slice(&[0u8; 12]); ++ out.extend_from_slice(&0u32.to_le_bytes()); ++ out.extend_from_slice(name.as_bytes()); ++ let central_size = out.len() as u32 - central; ++ out.extend_from_slice(&[0x50, 0x4b, 0x05, 0x06, 0, 0, 0, 0, 1, 0, 1, 0]); ++ out.extend_from_slice(¢ral_size.to_le_bytes()); ++ out.extend_from_slice(¢ral.to_le_bytes()); ++ out.extend_from_slice(&0u16.to_le_bytes()); ++ out ++ } ++ ++ fn stored_zip_of(entries: &[(&str, &[u8], u32)]) -> Vec { ++ let mut out = Vec::new(); ++ let mut central = Vec::new(); ++ for (name, data, declared) in entries { ++ let offset = out.len() as u32; ++ let n = name.len() as u16; ++ out.extend_from_slice(&[0x50, 0x4b, 0x03, 0x04, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]); ++ out.extend_from_slice(&declared.to_le_bytes()); ++ out.extend_from_slice(&declared.to_le_bytes()); ++ out.extend_from_slice(&n.to_le_bytes()); ++ out.extend_from_slice(&0u16.to_le_bytes()); ++ out.extend_from_slice(name.as_bytes()); ++ out.extend_from_slice(data); ++ central.extend_from_slice(&[0x50, 0x4b, 0x01, 0x02, 20, 0, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]); ++ central.extend_from_slice(&declared.to_le_bytes()); ++ central.extend_from_slice(&declared.to_le_bytes()); ++ central.extend_from_slice(&n.to_le_bytes()); ++ central.extend_from_slice(&[0u8; 12]); ++ central.extend_from_slice(&offset.to_le_bytes()); ++ central.extend_from_slice(name.as_bytes()); ++ } ++ let at = out.len() as u32; ++ let size = central.len() as u32; ++ out.extend_from_slice(¢ral); ++ let count = entries.len() as u16; ++ out.extend_from_slice(&[0x50, 0x4b, 0x05, 0x06, 0, 0, 0, 0]); ++ out.extend_from_slice(&count.to_le_bytes()); ++ out.extend_from_slice(&count.to_le_bytes()); ++ out.extend_from_slice(&size.to_le_bytes()); ++ out.extend_from_slice(&at.to_le_bytes()); ++ out.extend_from_slice(&0u16.to_le_bytes()); ++ out ++ } ++ ++ #[tokio::test] ++ async fn an_entry_that_runs_into_the_next_header_is_refused() { ++ let out = std::env::temp_dir().join(format!("xodus-plain-next-{}", std::process::id())); ++ std::fs::create_dir_all(&out).unwrap(); ++ let fine = stored_zip_of(&[("a.txt", b"aaaa", 4), ("b.txt", b"bbbb", 4)]); ++ let code = super::plain_package(std::io::Cursor::new(fine.clone()), fine.len() as u64, &out).await; ++ assert_eq!(code, std::process::ExitCode::SUCCESS); ++ assert_eq!(std::fs::read(out.join("b.txt")).unwrap(), b"bbbb"); ++ let bad = stored_zip_of(&[("a.txt", b"zzzz", 10), ("b.txt", b"yyyy", 4)]); ++ let code = super::plain_package(std::io::Cursor::new(bad.clone()), bad.len() as u64, &out).await; ++ assert_eq!(code, std::process::ExitCode::FAILURE); ++ assert_eq!(std::fs::read(out.join("a.txt")).unwrap(), b"aaaa"); ++ std::fs::remove_dir_all(&out).ok(); ++ } ++ ++ #[test] ++ fn a_failed_swap_puts_every_replaced_entry_back() { ++ let root = tempfile::tempdir().unwrap(); ++ let (staging, out) = (root.path().join("staging"), root.path().join("out")); ++ std::fs::create_dir_all(staging.join(".xodus-plain-previous")).unwrap(); ++ std::fs::create_dir_all(&out).unwrap(); ++ std::fs::write(staging.join("+a"), b"new").unwrap(); ++ std::fs::write(out.join("+a"), b"old").unwrap(); ++ assert!(super::publish(&staging, &out).is_err()); ++ assert_eq!(std::fs::read(out.join("+a")).unwrap(), b"old"); ++ assert!(!out.join(".xodus-plain-previous").exists()); ++ } ++ ++ #[test] ++ fn entries_a_rollback_could_not_remove_are_cleared_before_the_backup_returns() { ++ let root = tempfile::tempdir().unwrap(); ++ let (staging, out) = (root.path().join("staging"), root.path().join("out")); ++ let previous = out.join(".xodus-plain-previous"); ++ std::fs::create_dir_all(&staging).unwrap(); ++ std::fs::create_dir_all(previous.join("data")).unwrap(); ++ std::fs::write(previous.join(".pending"), "stray").unwrap(); ++ std::fs::write(previous.join("data").join("old"), b"old").unwrap(); ++ std::fs::create_dir_all(out.join("stray")).unwrap(); ++ std::fs::write(out.join("stray").join("new"), b"new").unwrap(); ++ std::fs::create_dir_all(out.join("data")).unwrap(); ++ std::fs::write(staging.join("fresh"), b"new").unwrap(); ++ super::publish(&staging, &out).unwrap(); ++ assert!(!out.join("stray").exists()); ++ assert_eq!(std::fs::read(out.join("data").join("old")).unwrap(), b"old"); ++ assert!(!previous.exists()); ++ } ++ ++ #[test] ++ fn a_restore_interrupted_after_its_removals_never_deletes_what_it_put_back() { ++ let root = tempfile::tempdir().unwrap(); ++ let (staging, out) = (root.path().join("staging"), root.path().join("out")); ++ let previous = out.join(".xodus-plain-previous"); ++ std::fs::create_dir_all(&staging).unwrap(); ++ std::fs::create_dir_all(&previous).unwrap(); ++ std::fs::write(previous.join(".pending"), "a").unwrap(); ++ std::fs::write(previous.join("a"), b"old").unwrap(); ++ std::fs::write(out.join("a"), b"stray").unwrap(); ++ super::restore(&previous, &out).unwrap(); ++ assert_eq!(std::fs::read(out.join("a")).unwrap(), b"old"); ++ std::fs::create_dir_all(&previous).unwrap(); ++ std::fs::write(previous.join(".pending"), "").unwrap(); ++ std::fs::write(previous.join("b"), b"old-b").unwrap(); ++ super::publish(&staging, &out).unwrap(); ++ assert_eq!(std::fs::read(out.join("a")).unwrap(), b"old"); ++ assert_eq!(std::fs::read(out.join("b")).unwrap(), b"old-b"); ++ } ++ ++ #[test] ++ fn a_backup_left_by_a_finished_swap_is_discarded_not_restored() { ++ let root = tempfile::tempdir().unwrap(); ++ let (staging, out) = (root.path().join("staging"), root.path().join("out")); ++ std::fs::create_dir_all(&staging).unwrap(); ++ std::fs::create_dir_all(out.join(".xodus-plain-previous")).unwrap(); ++ std::fs::write(out.join(".xodus-plain-previous").join("kept"), b"old").unwrap(); ++ std::fs::write(out.join("kept"), b"installed").unwrap(); ++ std::fs::write(staging.join("fresh"), b"new").unwrap(); ++ super::publish(&staging, &out).unwrap(); ++ assert_eq!(std::fs::read(out.join("kept")).unwrap(), b"installed"); ++ assert!(!out.join(".xodus-plain-previous").exists()); ++ } ++ ++ #[test] ++ fn a_backup_left_by_an_interrupted_swap_is_restored_first() { ++ let root = tempfile::tempdir().unwrap(); ++ let (staging, out) = (root.path().join("staging"), root.path().join("out")); ++ std::fs::create_dir_all(&staging).unwrap(); ++ std::fs::create_dir_all(out.join(".xodus-plain-previous")).unwrap(); ++ std::fs::write(out.join(".xodus-plain-previous").join(".pending"), b"").unwrap(); ++ std::fs::write(out.join(".xodus-plain-previous").join("kept"), b"old").unwrap(); ++ std::fs::write(out.join("kept"), b"half").unwrap(); ++ std::fs::write(staging.join("fresh"), b"new").unwrap(); ++ super::publish(&staging, &out).unwrap(); ++ assert_eq!(std::fs::read(out.join("kept")).unwrap(), b"old"); ++ assert_eq!(std::fs::read(out.join("fresh")).unwrap(), b"new"); ++ assert!(!out.join(".xodus-plain-previous").exists()); ++ } ++ ++ #[tokio::test] ++ async fn a_plain_archive_is_downloaded_whole_and_unpacked() { ++ let out = std::env::temp_dir().join(format!("xodus-plain-{}", std::process::id())); ++ std::fs::create_dir_all(&out).unwrap(); ++ let zip = stored_zip("AppxManifest.xml", b""); ++ let code = super::plain_package(std::io::Cursor::new(zip.clone()), zip.len() as u64, &out).await; ++ assert_eq!(code, std::process::ExitCode::SUCCESS); ++ assert_eq!(std::fs::read(out.join("AppxManifest.xml")).unwrap(), b""); ++ assert!(!out.join(".xodus-plain-package.zip").exists()); ++ let short = super::plain_package(std::io::Cursor::new(zip[..10].to_vec()), zip.len() as u64, &out).await; ++ assert_eq!(short, std::process::ExitCode::FAILURE); ++ let mut lying = stored_zip("AppxManifest.xml", b""); ++ lying[18] = 0xff; ++ lying[22] = 0xff; ++ let length = lying.len() as u64; ++ let central = lying.windows(4).position(|w| w == [0x50, 0x4b, 0x01, 0x02]).unwrap(); ++ lying[central + 20] = 0xff; ++ lying[central + 24] = 0xff; ++ let truncated = super::plain_package(std::io::Cursor::new(lying), length, &out).await; ++ assert_eq!(truncated, std::process::ExitCode::FAILURE); ++ let mut overrun = stored_zip("AppxManifest.xml", b""); ++ let central = overrun.windows(4).position(|w| w == [0x50, 0x4b, 0x01, 0x02]).unwrap(); ++ for at in [18, 22, central + 20, central + 24] { ++ overrun[at] = 11; ++ } ++ let length = overrun.len() as u64; ++ let overran = super::plain_package(std::io::Cursor::new(overrun), length, &out).await; ++ assert_eq!(overran, std::process::ExitCode::FAILURE); ++ assert_eq!(std::fs::read(out.join("AppxManifest.xml")).unwrap(), b""); ++ assert!(!out.join(".xodus-plain-staging").exists()); ++ std::fs::remove_dir_all(&out).ok(); ++ } ++} +diff --git a/crates/xodus-cli/src/commands/unzip.rs b/crates/xodus-cli/src/commands/unzip.rs +new file mode 100644 +index 0000000..10e92fb +--- /dev/null ++++ b/crates/xodus-cli/src/commands/unzip.rs +@@ -0,0 +1,219 @@ ++//! Plain ZIP packages, which is what the store serves for a framework. ++//! ++//! Framework packages (`Microsoft.VCLibs.140.00` and its siblings) arrive from the ++//! delivery service as ordinary `.appx` zips, not the EAppx containers a title is ++//! packed in: there is nothing to decrypt and no header to parse. The packager ++//! writes them in the ZIP64 form, so the central directory and each entry may name ++//! its sizes and offset in an extra field instead of the fixed header. ++ ++use std::io::{Read, Seek, SeekFrom, Write}; ++use std::path::Path; ++ ++/// The end record is 22 bytes and its comment field can push it back 64 KiB. ++const TAIL: u64 = 65_557; ++const END: &[u8; 4] = b"PK\x05\x06"; ++const ZIP64_END: &[u8; 4] = b"PK\x06\x06"; ++const ZIP64_LOCATOR: &[u8; 4] = b"PK\x06\x07"; ++const CENTRAL: &[u8; 4] = b"PK\x01\x02"; ++const SATURATED: u32 = 0xFFFF_FFFF; ++ ++pub fn extract(archive: &Path, destination: &Path) -> Result<(), String> { ++ let mut file = std::fs::File::open(archive).map_err(|error| format!("package open failed: {error}"))?; ++ let length = file.metadata().map_err(|error| format!("package size failed: {error}"))?.len(); ++ if length < 22 { ++ return Err("the package is too small to be an archive".into()); ++ } ++ let tail = length.min(TAIL); ++ file.seek(SeekFrom::Start(length - tail)).map_err(|error| format!("package seek failed: {error}"))?; ++ let mut buffer = vec![0u8; tail as usize]; ++ file.read_exact(&mut buffer).map_err(|error| format!("package read failed: {error}"))?; ++ let end = buffer ++ .windows(END.len()) ++ .rposition(|window| window == END) ++ .ok_or("the package is not a zip archive")?; ++ if end + 22 > buffer.len() { ++ return Err("the package end record is truncated".into()); ++ } ++ let mut entries = u16::from_le_bytes([buffer[end + 10], buffer[end + 11]]) as u64; ++ let mut offset = u32::from_le_bytes([buffer[end + 16], buffer[end + 17], buffer[end + 18], buffer[end + 19]]) as u64; ++ if entries == 0xFFFF || offset == u64::from(SATURATED) || offset > length { ++ let (zip64_entries, zip64_offset) = zip64(&mut file, length, &buffer, end)?; ++ entries = zip64_entries; ++ offset = zip64_offset; ++ } ++ if offset >= length { ++ return Err(format!("the package directory starts past the archive at {offset} of {length}")); ++ } ++ std::fs::create_dir_all(destination).map_err(|error| format!("package folder failed: {error}"))?; ++ let directory = offset; ++ let mut listed = Vec::new(); ++ for _ in 0..entries { ++ file.seek(SeekFrom::Start(offset)).map_err(|error| format!("package seek failed: {error}"))?; ++ let mut header = [0u8; 46]; ++ file.read_exact(&mut header).map_err(|error| format!("package header failed at {offset} of the directory, {error}"))?; ++ if &header[0..4] != CENTRAL { ++ return Err(format!("the package directory is malformed at {offset}")); ++ } ++ let method = u16::from_le_bytes([header[10], header[11]]); ++ let uncompressed = u32::from_le_bytes([header[24], header[25], header[26], header[27]]); ++ let mut compressed = u64::from(u32::from_le_bytes([header[20], header[21], header[22], header[23]])); ++ let name_length = u16::from_le_bytes([header[28], header[29]]) as usize; ++ let extra_length = u16::from_le_bytes([header[30], header[31]]) as usize; ++ let comment_length = u16::from_le_bytes([header[32], header[33]]) as usize; ++ let mut local = u64::from(u32::from_le_bytes([header[42], header[43], header[44], header[45]])); ++ let mut size = u64::from(uncompressed); ++ let mut name = vec![0u8; name_length]; ++ file.read_exact(&mut name).map_err(|error| format!("package name failed: {error}"))?; ++ let mut extra = vec![0u8; extra_length]; ++ file.read_exact(&mut extra).map_err(|error| format!("package extra failed: {error}"))?; ++ offset += 46 + name_length as u64 + extra_length as u64 + comment_length as u64; ++ if compressed == u64::from(SATURATED) || local == u64::from(SATURATED) || uncompressed == SATURATED { ++ let wide = zip64_entry(&extra, uncompressed == SATURATED, compressed == u64::from(SATURATED), local == u64::from(SATURATED)); ++ if compressed == u64::from(SATURATED) { ++ compressed = wide.compressed.ok_or("a package entry has no wide size")?; ++ } ++ if local == u64::from(SATURATED) { ++ local = wide.local.ok_or("a package entry has no wide offset")?; ++ } ++ if uncompressed == SATURATED { ++ size = wide.size.ok_or("a package entry has no wide uncompressed size")?; ++ } ++ } ++ let name = String::from_utf8(name).map_err(|_| "a package entry has an unreadable name".to_string())?; ++ listed.push(Listed { name, method, compressed, size, local }); ++ } ++ let mut starts: Vec = listed.iter().map(|entry| entry.local).collect(); ++ starts.sort_unstable(); ++ starts.dedup(); ++ for Listed { name, method, compressed, size, local } in listed { ++ let limit = starts.get(starts.partition_point(|start| *start <= local)).copied().unwrap_or(directory).min(directory); ++ if name.ends_with('/') { ++ continue; ++ } ++ let Some(relative) = safe(&name) else { continue }; ++ let target = destination.join(relative); ++ if let Some(parent) = target.parent() { ++ std::fs::create_dir_all(parent).map_err(|error| format!("package folder failed: {error}"))?; ++ } ++ file.seek(SeekFrom::Start(local)).map_err(|error| format!("package seek failed: {error}"))?; ++ let mut local_header = [0u8; 30]; ++ file.read_exact(&mut local_header).map_err(|error| format!("package header failed: {error}"))?; ++ let local_name = u16::from_le_bytes([local_header[26], local_header[27]]) as usize; ++ let local_extra = u16::from_le_bytes([local_header[28], local_header[29]]) as usize; ++ let data = local + 30 + local_name as u64 + local_extra as u64; ++ if data.checked_add(compressed).is_none_or(|end| end > limit) { ++ return Err(format!("the package entry {name} runs past its data")); ++ } ++ if method == 0 && compressed != size { ++ return Err(format!("the package entry {name} has inconsistent sizes")); ++ } ++ file.seek(SeekFrom::Start(data)).map_err(|error| format!("package seek failed: {error}"))?; ++ let mut body = (&mut file).take(compressed); ++ let mut written = std::fs::File::create(&target).map_err(|error| format!("package write failed: {error}"))?; ++ match method { ++ 0 => { ++ let copied = std::io::copy(&mut body, &mut written).map_err(|error| format!("package copy failed: {error}"))?; ++ if copied != compressed { ++ return Err(format!("the package entry {name} is truncated")); ++ } ++ } ++ 8 => { ++ let mut inflate = flate2::read::DeflateDecoder::new(body); ++ let copied = std::io::copy(&mut inflate, &mut written).map_err(|error| format!("package inflate failed: {error}"))?; ++ if copied != size { ++ return Err(format!("the package entry {name} is truncated")); ++ } ++ } ++ _ => return Err(format!("a package entry uses compression {method}, which is not supported")), ++ } ++ written.flush().ok(); ++ } ++ Ok(()) ++} ++ ++/// The end record of the ZIP64 archive that precedes the ordinary one. ++fn zip64(file: &mut std::fs::File, length: u64, buffer: &[u8], end: usize) -> Result<(u64, u64), String> { ++ let locator = buffer[..end] ++ .windows(ZIP64_LOCATOR.len()) ++ .rposition(|window| window == ZIP64_LOCATOR) ++ .ok_or("the package names a wide directory with no locator")?; ++ if locator + 20 > buffer.len() { ++ return Err("the package locator is truncated".into()); ++ } ++ let at = u64::from_le_bytes(buffer[locator + 8..locator + 16].try_into().map_err(|_| "unreadable locator")?); ++ if at + 56 > length { ++ return Err(format!("the package names a wide end record past the archive at {at} of {length}")); ++ } ++ file.seek(SeekFrom::Start(at)).map_err(|error| format!("package seek failed: {error}"))?; ++ let mut record = [0u8; 56]; ++ file.read_exact(&mut record).map_err(|error| format!("package end record failed: {error}"))?; ++ if &record[0..4] != ZIP64_END { ++ return Err("the package wide end record is malformed".into()); ++ } ++ let entries = u64::from_le_bytes(record[32..40].try_into().map_err(|_| "unreadable entry count")?); ++ let offset = u64::from_le_bytes(record[48..56].try_into().map_err(|_| "unreadable directory offset")?); ++ Ok((entries, offset)) ++} ++ ++/// The compressed size and local header offset a ZIP64 entry keeps in its extra, ++/// in the order the format wrote them: only the fields that were saturated. ++fn zip64_entry(extra: &[u8], has_size: bool, has_compressed: bool, has_local: bool) -> Wide { ++ let mut wide = Wide::default(); ++ let mut at = 0; ++ while at + 4 <= extra.len() { ++ let id = u16::from_le_bytes([extra[at], extra[at + 1]]); ++ let size = u16::from_le_bytes([extra[at + 2], extra[at + 3]]) as usize; ++ let body = at + 4; ++ if body + size > extra.len() { ++ break; ++ } ++ if id == 0x0001 { ++ let fields = &extra[body..body + size]; ++ let mut cursor = 0usize; ++ let mut take = |present: bool| -> Option { ++ if !present || cursor + 8 > fields.len() { ++ return None; ++ } ++ let value = u64::from_le_bytes(fields[cursor..cursor + 8].try_into().ok()?); ++ cursor += 8; ++ Some(value) ++ }; ++ wide.size = take(has_size); ++ wide.compressed = take(has_compressed); ++ wide.local = take(has_local); ++ return wide; ++ } ++ at = body + size; ++ } ++ wide ++} ++ ++struct Listed { ++ name: String, ++ method: u16, ++ compressed: u64, ++ size: u64, ++ local: u64, ++} ++ ++#[derive(Default)] ++struct Wide { ++ size: Option, ++ compressed: Option, ++ local: Option, ++} ++ ++/// The entry's relative path, or nothing when it would land outside the folder. ++fn safe(name: &str) -> Option { ++ let mut parts = Vec::new(); ++ for part in name.split('/') { ++ if part.is_empty() || part == "." { ++ continue; ++ } ++ if part == ".." || part.contains('\\') || part.contains(':') { ++ return None; ++ } ++ parts.push(part); ++ } ++ if parts.is_empty() { None } else { Some(parts.iter().collect()) } ++} +diff --git a/crates/xodus-cli/src/commands/wu-root.pem b/crates/xodus-cli/src/commands/wu-root.pem +new file mode 100644 +index 0000000..63467d7 +--- /dev/null ++++ b/crates/xodus-cli/src/commands/wu-root.pem +@@ -0,0 +1,19 @@ ++-----BEGIN CERTIFICATE----- ++MIIDIzCCAqigAwIBAgIQFJgmZtx8zY9AU2d7uZnshTAKBggqhkjOPQQDAzCBlDEL ++MAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1v ++bmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjE+MDwGA1UEAxM1TWlj ++cm9zb2Z0IEVDQyBQcm9kdWN0IFJvb3QgQ2VydGlmaWNhdGUgQXV0aG9yaXR5IDIw ++MTgwHhcNMTgwMjI3MjA0MjA4WhcNNDMwMjI3MjA1MDQ2WjCBlDELMAkGA1UEBhMC ++VVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNV ++BAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjE+MDwGA1UEAxM1TWljcm9zb2Z0IEVD ++QyBQcm9kdWN0IFJvb3QgQ2VydGlmaWNhdGUgQXV0aG9yaXR5IDIwMTgwdjAQBgcq ++hkjOPQIBBgUrgQQAIgNiAATHERYqdh1Wjr65YmXUw8608MMw7I9t1245vMhJq6u4 ++40N41YEGXe/HfZ/O1rOQdd4MsJDeI7rI0T5n4BmpG4YxHl80Le4X/RX7fieKMqHq ++yY/JfhjLLzssSHp9pvQBB6yjgbwwgbkwDgYDVR0PAQH/BAQDAgGGMA8GA1UdEwEB ++/wQFMAMBAf8wHQYDVR0OBBYEFEPvcIe4nb/siBncxsRrdQ11NDMIMBAGCSsGAQQB ++gjcVAQQDAgEAMGUGA1UdIAReMFwwBgYEVR0gADBSBgwrBgEEAYI3TIN9AQEwQjBA ++BggrBgEFBQcCARY0aHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraW9wcy9Eb2Nz ++L1JlcG9zaXRvcnkuaHRtADAKBggqhkjOPQQDAwNpADBmAjEAocBJRF0yVSfMPpBu ++JSKdJFubUTXHkUlJKqP5b08czd2c4bVXyZ7CIkWbBhVwHEW/AjEAxdMo63LHPrCs ++Jwl/Yj1geeWS8UUquaUC5GC7/nornGCntZkU8rC+8LsFllZWj8Fo ++-----END CERTIFICATE----- +diff --git a/crates/xodus-cli/src/main.rs b/crates/xodus-cli/src/main.rs +index b2a3383..855f667 100644 +--- a/crates/xodus-cli/src/main.rs ++++ b/crates/xodus-cli/src/main.rs +@@ -166,6 +166,29 @@ enum SubCommand { + #[arg(short, long)] + market: Option, + }, ++ #[command( ++ about = "Install a legacy EAppx/EAppxBundle title: license, download, extract" ++ )] ++ LegacyInstall { ++ product: String, ++ destination: String, ++ #[arg(short, long)] ++ market: Option, ++ #[arg(long, help = "Content ID explicitly selected from resolve-package for this Store product")] ++ selected_content_id: Option, ++ #[arg(long, help = "Store product ID for the Windows Update catalogue when PRODUCT is a content ID")] ++ store_id: Option, ++ #[arg( ++ long, ++ help = "Emit Nexus progress protocol v1 as newline-delimited JSON" ++ )] ++ progress_json: bool, ++ #[arg( ++ long, ++ help = "Resolve the download locations and stop without downloading" ++ )] ++ resolve_only: bool, ++ }, + #[cfg(unix)] + #[command(about = "Run a Game with xodus wine")] + Run { +@@ -435,6 +458,28 @@ async fn main() -> ExitCode { + ) + .await + } ++ SubCommand::LegacyInstall { ++ product, ++ destination, ++ market, ++ selected_content_id, ++ store_id, ++ progress_json, ++ resolve_only, ++ } => { ++ commands::legacy_install::run( ++ &client, ++ &tokens, ++ product, ++ destination, ++ market, ++ selected_content_id, ++ store_id, ++ progress_json, ++ resolve_only, ++ ) ++ .await ++ } + #[cfg(unix)] + SubCommand::Run { + source, +diff --git a/crates/xodus/src/licensing/content_response.rs b/crates/xodus/src/licensing/content_response.rs +index e87421d..8b5b570 100644 +--- a/crates/xodus/src/licensing/content_response.rs ++++ b/crates/xodus/src/licensing/content_response.rs +@@ -53,6 +53,10 @@ pub enum LicenseContentError { + "Microsoft Store device limit reached. Open https://account.microsoft.com/devices/content, unlink an old device you no longer use, then try again. You do not need to sign out or reset the wallet." + )] + DeviceLimit, ++ #[error( ++ "Microsoft refused a license for this version of the package on this device." ++ )] ++ DeviceUnregistrable, + #[error( + "Microsoft returned a content license without a key. Share the backend diagnostic log." + )] +@@ -87,6 +91,7 @@ impl LicenseContentError { + Self::Schema => "content-schema", + Self::NotEntitled => "content-not-entitled", + Self::DeviceLimit => "content-device-limit", ++ Self::DeviceUnregistrable => "content-device-name", + Self::EmptyKeys => "content-empty-keys", + Self::Base64 => "content-base64", + Self::Utf8 => "content-utf8", +@@ -133,6 +138,12 @@ async fn receive( + let result = async { + // A valid-looking license inside a refused HTTP response is not permission. + if !response.status().is_success() { ++ if status == 400 ++ && let Some(body) = bounded_body(&mut response, REFUSAL_BODY).await ++ && unregistrable(&body) ++ { ++ return Err(LicenseContentError::DeviceUnregistrable); ++ } + return Err(LicenseContentError::http(status)); + } + if response +@@ -167,6 +178,27 @@ async fn receive( + result + } + ++const REFUSAL_BODY: usize = 64 * 1024; ++ ++async fn bounded_body(response: &mut reqwest::Response, limit: usize) -> Option> { ++ if response.content_length().is_some_and(|length| length > limit as u64) { ++ return None; ++ } ++ let mut body = Vec::new(); ++ while let Some(chunk) = response.chunk().await.ok()? { ++ if chunk.len() > limit - body.len() { ++ return None; ++ } ++ body.extend_from_slice(&chunk); ++ } ++ Some(body) ++} ++ ++fn unregistrable(body: &[u8]) -> bool { ++ serde_json::from_slice::(body) ++ .is_ok_and(|value| value["innererror"]["code"] == "DeviceNameNotHexBinary") ++} ++ + fn parse(bytes: &[u8]) -> Result<(LicenseContent, License), LicenseContentError> { + if bytes.len() > MAX_BODY { + return Err(LicenseContentError::Size); +diff --git a/crates/xodus/src/licensing/content_response_tests.rs b/crates/xodus/src/licensing/content_response_tests.rs +index 205aab3..93c138b 100644 +--- a/crates/xodus/src/licensing/content_response_tests.rs ++++ b/crates/xodus/src/licensing/content_response_tests.rs +@@ -373,3 +373,18 @@ async fn content_license_transport_timeout_and_cancellation_have_no_background_r + ); + server.await.unwrap(); + } ++ ++#[tokio::test] ++async fn a_refused_device_name_claim_has_its_own_stage() { ++ let refused = serde_json::to_vec(&json!({"code":"BadRequest","innererror":{"faultcode":3043,"code":"DeviceNameNotHexBinary","message":"private"}})).unwrap(); ++ let other = serde_json::to_vec(&json!({"code":"BadRequest","innererror":{"code":"Other"}})).unwrap(); ++ let mut oversized = refused.clone(); ++ oversized.splice(1..1, std::iter::repeat_n(b' ', 70 * 1024)); ++ for (body, expected) in [(refused, "content-device-name"), (other, "content-http"), (b"not json".to_vec(), "content-http"), (oversized, "content-http")] { ++ let (url, server) = server(format!("HTTP/1.1 400 Fixture\r\nContent-Length: {}\r\nConnection: close\r\n\r\n", body.len()), body).await; ++ let error = fetch(reqwest::Client::new().get(url)).await.err().unwrap(); ++ assert_eq!(error.stage(), expected); ++ assert!(!error.to_string().contains("private")); ++ server.await.unwrap(); ++ } ++} diff --git a/tools/xbox/droiddeck-legacy-packages.patch b/tools/xbox/droiddeck-legacy-packages.patch new file mode 100644 index 0000000000..daec856e54 --- /dev/null +++ b/tools/xbox/droiddeck-legacy-packages.patch @@ -0,0 +1,570 @@ +diff --git a/crates/xodus-cli/src/commands/extract_eappx.rs b/crates/xodus-cli/src/commands/extract_eappx.rs +index c8d5ebf..5ea43da 100644 +--- a/crates/xodus-cli/src/commands/extract_eappx.rs ++++ b/crates/xodus-cli/src/commands/extract_eappx.rs +@@ -1,80 +1,416 @@ +-use std::path::PathBuf; ++use std::path::{Path, PathBuf}; + use std::process::ExitCode; + + use eappx::EAppxFile; +-use eappx::keys::KeyCollection; ++use eappx::keys::{KeyCollection, KeyId}; ++ ++/// A bundle header names each content key as a pair of GUIDs, `(content id, ++/// package id)`, while the licence names the same key by its 16-byte content id ++/// alone. The crate reads a 16-byte id as `(static prefix, content id)`, so the ++/// two never compare equal and a complete key file looks incomplete. Pair every ++/// key the header asks for with the licence entry that carries its content id. ++const SHORT_KEY_PREFIX: uuid::Uuid = uuid::uuid!("BB1755DB-5052-4B10-B2AB-F3ABF5CA5B41"); ++ ++fn align_keys(keys: &mut KeyCollection, needed: &[KeyId]) { ++ for id in needed { ++ if keys.keys.contains_key(id) { ++ continue; ++ } ++ let KeyId::Guid((content, _)) = id else { continue }; ++ let carried = keys.keys.get(&KeyId::Guid((SHORT_KEY_PREFIX, *content))).cloned(); ++ if let Some(data) = carried { ++ keys.add(id.clone(), data); ++ } ++ } ++} ++ ++/// A bundle can hold another bundle, but nesting is bounded so a malformed ++/// archive cannot drive unbounded recursion. ++const MAX_BUNDLE_DEPTH: usize = 3; + + /// Extracts a locally-stored EAppx/EMSIX package. + /// +-/// This is intentionally local-only for now: no store lookup or download. +-/// See issue #91 - the initial goal is understanding/unpacking the format, +-/// not full download integration, which can build on this once it lands. ++/// A bundle is extracted in place, then each inner package that is itself an ++/// EAppx container is decrypted with the same content keys. Inner files that ++/// are plain ZIP-based packages are left as extracted: they carry no EAppx ++/// header, so there is nothing to decrypt. + pub async fn run(path: String, destination: String, key_file: Option) -> ExitCode { +- let infile = PathBuf::from(&path); +- let outdir = PathBuf::from(&destination); +- +- let file = match std::fs::File::open(&infile) { +- Ok(f) => f, +- Err(e) => { +- eprintln!("Failed to open {:?}: {e}", infile); +- return ExitCode::FAILURE; +- } +- }; +- let mut reader = std::io::BufReader::new(file); ++ run_metered(path, destination, key_file, None).await ++} + +- let mut eappx = match EAppxFile::from_stream(&mut reader) { +- Ok(e) => e, +- Err(e) => { +- eprintln!("Failed to parse EAppx header: {e}"); +- return ExitCode::FAILURE; +- } +- }; ++/// Where the extraction of the current package begins and how much it will write. ++/// ++/// A bundle's inner package is one enormous member, so counting finished files ++/// leaves the bar frozen for the whole of it. The counter is exact: it advances by ++/// each file's declared length as that file is written. ++#[derive(Default)] ++pub struct Meter { ++ /// Bytes written by the package being unpacked now. ++ pub written: std::sync::atomic::AtomicU64, ++ /// Bytes written by packages that already finished. ++ pub banked: std::sync::atomic::AtomicU64, ++ /// Bytes the package being unpacked now will write in total. ++ pub total: std::sync::atomic::AtomicU64, ++ turn: std::sync::Mutex<()>, ++} + +- println!("{eappx}"); ++impl Meter { ++ /// Bytes written and bytes to write, read as one consistent pair. ++ pub fn snapshot(&self) -> (u64, u64) { ++ use std::sync::atomic::Ordering::Relaxed; ++ let _turn = self.turn.lock().unwrap_or_else(|poisoned| poisoned.into_inner()); ++ let banked = self.banked.load(Relaxed); ++ let total = self.total.load(Relaxed); ++ (banked + self.written.load(Relaxed).min(total), banked + total) ++ } + ++ pub fn done(&self) -> u64 { ++ self.snapshot().0 ++ } ++ ++ pub fn all(&self) -> u64 { ++ self.snapshot().1 ++ } ++ ++ /// Closes the package being unpacked and opens the next one. ++ fn roll(&self, total: u64) { ++ use std::sync::atomic::Ordering::Relaxed; ++ let _turn = self.turn.lock().unwrap_or_else(|poisoned| poisoned.into_inner()); ++ self.banked.fetch_add(self.total.load(Relaxed), Relaxed); ++ self.written.store(0, Relaxed); ++ self.total.store(total, Relaxed); ++ } ++} ++ ++/// Extracts a package, publishing the size of the work in progress to `meter`. ++/// ++/// `release` deletes the container once its members are on disk. A bundle of a ++/// full title is tens of gigabytes, and holding it while the members are ++/// decrypted can fill the device partway through. ++pub async fn run_metered( ++ path: String, ++ destination: String, ++ key_file: Option, ++ meter: Option<&Meter>, ++) -> ExitCode { ++ run_blocking(path, destination, key_file, meter, false) ++} ++ ++/// The extraction itself, without the async wrapper. ++/// ++/// Callers that keep reporting while it works run this on a blocking thread, ++/// because it never yields: decryption and inflation are plain synchronous work. ++pub fn run_blocking( ++ path: String, ++ destination: String, ++ key_file: Option, ++ meter: Option<&Meter>, ++ release: bool, ++) -> ExitCode { + let mut key_collection = KeyCollection::default(); + if let Some(key_file) = key_file { + let mut keyfile = match std::fs::File::open(&key_file) { + Ok(f) => f, + Err(e) => { +- eprintln!("Failed to open key file {:?}: {e}", key_file); +- return ExitCode::FAILURE; ++ return report_failure( ++ &Failure::new("key-file-open", &key_file, &e.to_string()), ++ ExitCode::FAILURE, ++ ); + } + }; + match KeyCollection::from_reader(&mut keyfile) { + Ok(loaded) => key_collection.extend(loaded.keys), + Err(e) => { +- eprintln!("Failed to parse key file: {e}"); +- return ExitCode::FAILURE; ++ return report_failure( ++ &Failure::new("key-file-parse", &key_file, &e.to_string()), ++ ExitCode::FAILURE, ++ ); + } + } + } + +- if !key_collection.has_required_keys(&eappx.header.key_ids) { +- eprintln!("Missing one or more required content keys for this package"); +- return ExitCode::FAILURE; ++ match derive( ++ Path::new(&path), ++ Path::new(&destination), ++ &key_collection, ++ 0, ++ meter, ++ release, ++ ) { ++ Ok(()) => ExitCode::SUCCESS, ++ Err(failure) => report_failure(&failure, ExitCode::FAILURE), + } ++} ++ ++/// A failure the launcher can name without printing a device path. ++/// ++/// The app only forwards stderr lines that are on a known allowlist, so the two ++/// lines below are what make a stopped install say why it stopped. The reason is ++/// reduced to letters, digits and a few punctuation marks, which keeps a path or ++/// a key out of the log while still naming the cause. ++struct Failure { ++ code: &'static str, ++ name: String, ++ reason: String, ++} + +- if let Err(e) = eappx.load_keys(&key_collection) { +- eprintln!("Failed to load keys: {e}"); +- return ExitCode::FAILURE; ++impl Failure { ++ fn new(code: &'static str, name: &str, reason: &str) -> Self { ++ Self { ++ code, ++ name: sanitize(name, 60), ++ reason: sanitize(reason, 120), ++ } + } + +- if !outdir.exists() { +- if let Err(e) = std::fs::create_dir_all(&outdir) { +- eprintln!("Failed to create output directory {:?}: {e}", outdir); +- return ExitCode::FAILURE; ++ fn at(code: &'static str, path: &Path, reason: &str) -> Self { ++ let name = path.file_name().map(|name| name.to_string_lossy().into_owned()); ++ Self::new(code, &name.unwrap_or_default(), reason) ++ } ++} ++ ++fn sanitize(text: &str, limit: usize) -> String { ++ text.chars() ++ .map(|c| if c.is_ascii_alphanumeric() || matches!(c, ' ' | '.' | '_' | '-') { c } else { '?' }) ++ .take(limit) ++ .collect::() ++ .trim() ++ .to_string() ++} ++ ++fn report_failure(failure: &Failure, code: ExitCode) -> ExitCode { ++ eprintln!("NEXUS_EXTRACT_DIAGNOSTIC_V1 {}", failure.code); ++ eprintln!("NEXUS_EXTRACT_DETAIL_V1 {} {}", failure.name, failure.reason); ++ eprintln!("Extraction failed: {}: {}: {}", failure.code, failure.name, failure.reason); ++ code ++} ++ ++// EAppxMagic is EXPH (single), EXSH and EXBH (bundle). Anything else is not an ++// EAppx container, so a bundle's plain ZIP members are not recursed into. ++fn is_eappx(path: &Path) -> bool { ++ use std::io::Read; ++ let Ok(mut file) = std::fs::File::open(path) else { return false }; ++ let mut magic = [0u8; 4]; ++ if file.read_exact(&mut magic).is_err() { ++ return false; ++ } ++ matches!(&magic, b"EXPH" | b"EXSH" | b"EXBH") ++} ++ ++fn derive( ++ infile: &Path, ++ outdir: &Path, ++ keys: &KeyCollection, ++ depth: usize, ++ meter: Option<&Meter>, ++ release: bool, ++) -> Result<(), Failure> { ++ let file = std::fs::File::open(infile) ++ .map_err(|e| Failure::at("open", infile, &e.to_string()))?; ++ let mut reader = std::io::BufReader::new(file); ++ ++ let mut eappx = EAppxFile::from_stream(&mut reader) ++ .map_err(|e| Failure::at("header", infile, &e.to_string()))?; ++ ++ println!("{eappx}"); ++ ++ // The licence and the header name the same key differently, so the set is ++ // rebuilt per package and reconciled before the header is judged. ++ let mut required = KeyCollection::new(&keys.keys); ++ align_keys(&mut required, &eappx.header.key_ids); ++ ++ if !required.has_required_keys(&eappx.header.key_ids) { ++ // The count is the useful part: a license that carries one key cannot ++ // open a bundle whose members each declare their own. ++ return Err(Failure::new( ++ "missing-keys", ++ &infile.file_name().map(|name| name.to_string_lossy().into_owned()).unwrap_or_default(), ++ &format!("{} of {} content keys present", required.keys.len(), eappx.header.key_ids.len()), ++ )); ++ } ++ ++ eappx ++ .load_keys(&required) ++ .map_err(|e| Failure::at("load-keys", infile, &e.to_string()))?; ++ ++ std::fs::create_dir_all(outdir) ++ .map_err(|e| Failure::at("create-dir", outdir, &e.to_string()))?; ++ ++ unpack(infile, &eappx, &mut reader, outdir, meter, release)?; ++ // The handle must close before a released container gives its space back. ++ drop(reader); ++ ++ println!("Extracted {}", infile.file_name().unwrap_or_default().to_string_lossy()); ++ ++ let mut nested: Vec = std::fs::read_dir(outdir) ++ .map_err(|e| Failure::at("list", outdir, &e.to_string()))? ++ .filter_map(|entry| entry.ok().map(|entry| entry.path())) ++ // The container being unpacked may itself sit in the destination, which is ++ // what `extract-eappx ./title.eappx .` asks for. Re-entering it would ++ // recurse into the same bytes and then delete the caller's file. ++ .filter(|path| std::fs::canonicalize(path).ok() != std::fs::canonicalize(infile).ok()) ++ .filter(|path| path.is_file() && is_eappx(path)) ++ .collect(); ++ nested.sort(); ++ ++ if nested.is_empty() { ++ return Ok(()); ++ } ++ if depth >= MAX_BUNDLE_DEPTH { ++ return Err(Failure::at("nesting", infile, "bundle nesting exceeds the limit")); ++ } ++ ++ for child in nested { ++ let name = child ++ .file_stem() ++ .map(|name| name.to_string_lossy().into_owned()) ++ .ok_or_else(|| Failure::at("no-name", &child, "package without a usable name"))?; ++ derive(&child, &outdir.join(&name), keys, depth + 1, meter, release)?; ++ // The member's bytes are no longer needed once its contents are on disk. ++ // A bundle of a full title holds tens of gigabytes, and keeping every ++ // member beside the decrypted copy doubles what the install occupies. ++ std::fs::remove_file(&child).ok(); ++ } ++ ++ Ok(()) ++} ++ ++/// Bytes currently on disk under `dir`. ++/// ++/// A bundle member is written as one file, so a per-file counter says nothing ++/// until it is finished. Measuring the destination catches the bytes as they ++/// land, whatever is writing them. ++pub fn tree_size(dir: &Path) -> u64 { ++ let mut total = 0; ++ let Ok(entries) = std::fs::read_dir(dir) else { return 0 }; ++ for entry in entries.flatten() { ++ let Ok(kind) = entry.file_type() else { continue }; ++ if kind.is_dir() { ++ total += tree_size(&entry.path()); ++ } else if let Ok(meta) = entry.metadata() { ++ total += meta.len(); + } + } ++ total ++} ++ ++/// Buffer sizes for each worker's own handle on the container. ++/// ++/// The crate's default 8 KiB reader turns one 64 KiB block into eight system ++/// calls, and a full title is millions of blocks. ++const READ_BUFFER: usize = 1 << 20; ++ ++/// Workers used to unpack one package. ++/// ++/// Files sit at independent offsets, so each worker opens its own handle and ++/// takes every Nth file. Decryption and inflation are both CPU-bound and one ++/// core was saturated, so this is where the headroom is. The limit is the ++/// device's write speed, which is reached a few workers in. ++fn workers() -> usize { ++ std::thread::available_parallelism() ++ .map(|count| count.get()) ++ .unwrap_or(1) ++ .clamp(1, 4) ++} + +- match eappx.extract(&mut reader, &outdir) { +- Ok(_) => { +- println!("Extracted {:?} to {:?}", infile, outdir); +- ExitCode::SUCCESS ++/// Writes a package's files, publishing what is left to write. ++/// ++/// This is the crate's own `extract`, split across workers and reported as it ++/// goes. The blockmap already totals every file's uncompressed length, so the ++/// end point is known before the first byte is written. ++fn unpack( ++ infile: &Path, ++ eappx: &EAppxFile, ++ reader: &mut std::io::BufReader, ++ outdir: &Path, ++ meter: Option<&Meter>, ++ release: bool, ++) -> Result<(), Failure> { ++ let total: u64 = eappx.blockmap.files.iter().map(|package| package.size).sum(); ++ if let Some(meter) = meter { ++ meter.roll(total); ++ } ++ ++ eappx ++ .extract_footprint_files(reader, outdir) ++ .map_err(|e| Failure::at("extract", infile, &e.to_string()))?; ++ ++ if eappx.header.is_bundle() { ++ eappx ++ .extract_blockmap_files(reader, outdir) ++ .map_err(|e| Failure::at("extract", infile, &e.to_string()))?; ++ eappx ++ .extract_bundle_files(reader, outdir) ++ .map_err(|e| Failure::at("extract", infile, &e.to_string()))?; ++ if release { ++ std::fs::remove_file(infile).ok(); + } +- Err(e) => { +- eprintln!("Extraction failed: {e}"); +- ExitCode::FAILURE ++ return Ok(()); ++ } ++ ++ let count = workers().min(eappx.blockmap.files.len().max(1)); ++ let failure = std::sync::Mutex::new(None::); ++ std::thread::scope(|scope| { ++ for shard in 0..count { ++ let failure = &failure; ++ scope.spawn(move || { ++ let Ok(handle) = std::fs::File::open(infile) else { ++ if let Ok(mut slot) = failure.lock() { ++ *slot = Some("a worker could not reopen the package".to_string()); ++ } ++ return; ++ }; ++ let mut stream = std::io::BufReader::with_capacity(READ_BUFFER, handle); ++ for (index, package) in eappx.blockmap.files.iter().enumerate() { ++ if index % count != shard { ++ continue; ++ } ++ if failure.lock().map(|slot| slot.is_some()).unwrap_or(true) { ++ return; ++ } ++ let Some(footer) = eappx.find_footer_for_file(package.id()) else { ++ if let Ok(mut slot) = failure.lock() { ++ *slot = Some("the blockmap names a file the package does not hold".to_string()); ++ } ++ return; ++ }; ++ let mut info = eappx::FileInfo::from(footer); ++ // The crate checks these only when asked, but filling them ++ // keeps the per-file path identical to its own extraction. ++ info.filehash = package.filehash_bytes(); ++ info.block_hashes = Some(package.block_hashes()); ++ // The blockmap's length and the footer's must agree, or the ++ // file written would be silently short. ++ if info.uncompressed_length != package.size { ++ if let Ok(mut slot) = failure.lock() { ++ *slot = Some("a package entry disagrees with its blockmap about its size".to_string()); ++ } ++ return; ++ } ++ if let Err(error) = eappx.save_file_to_fs(&mut stream, info, outdir, &package.name) { ++ if let Ok(mut slot) = failure.lock() { ++ *slot = Some(error.to_string()); ++ } ++ return; ++ } ++ if let Some(meter) = meter { ++ meter ++ .written ++ .fetch_add(package.size, std::sync::atomic::Ordering::Relaxed); ++ } ++ } ++ }); + } ++ }); ++ ++ if let Some(reason) = failure.into_inner().unwrap_or(None) { ++ return Err(Failure::at("extract", infile, &reason)); ++ } ++ ++ if release { ++ // The files are on disk; the container is not read again unless the ++ // install is retried from scratch. ++ std::fs::remove_file(infile).ok(); + } ++ Ok(()) + } +diff --git a/crates/xodus-cli/src/commands/license.rs b/crates/xodus-cli/src/commands/license.rs +index 4540549..b9fd91c 100644 +--- a/crates/xodus-cli/src/commands/license.rs ++++ b/crates/xodus-cli/src/commands/license.rs +@@ -1,5 +1,6 @@ + use std::process::ExitCode; + ++use base64::{Engine, engine::general_purpose::STANDARD}; + use tokio::fs::OpenOptions; + use tokio::io::AsyncWriteExt; + use xodus::tokens::TokenManager; +@@ -21,6 +22,10 @@ pub async fn run( + + let (key, game_splicense) = license.unwrap(); + tokio::fs::create_dir_all(&ciks).await.unwrap(); ++ // eappx-rs reads a text [Keys] file with base64 Key IDs, while the .cik ++ // format below is the binary form other tools expect. Write both so the ++ // same license dump serves either extractor. ++ let mut eappx_keys = String::from("[Keys]\n"); + for (uuid, content_key) in game_splicense.content_keys { + let unpacked = content_key.unpack(&key).expect("failed to unpack"); + let mut file = OpenOptions::new() +@@ -33,7 +38,21 @@ pub async fn run( + file.write_all(&uuid_buf).await.unwrap(); + file.write_all(&*unpacked).await.unwrap(); + file.flush().await.unwrap(); ++ eappx_keys.push_str(&format!( ++ "\"{}\" \"{}\"\n", ++ STANDARD.encode(uuid_buf), ++ STANDARD.encode(&*unpacked), ++ )); + } ++ let mut keys_file = OpenOptions::new() ++ .create(true) ++ .write(true) ++ .truncate(true) ++ .open(format!("{ciks}/eappx-keys.txt")) ++ .await ++ .unwrap(); ++ keys_file.write_all(eappx_keys.as_bytes()).await.unwrap(); ++ keys_file.flush().await.unwrap(); + + ExitCode::SUCCESS + } +diff --git a/crates/xodus-cli/src/commands/streaming.rs b/crates/xodus-cli/src/commands/streaming.rs +index 66e4135..a0136f8 100644 +--- a/crates/xodus-cli/src/commands/streaming.rs ++++ b/crates/xodus-cli/src/commands/streaming.rs +@@ -160,7 +160,19 @@ pub async fn run( + .iter() + .find(|p| p.file_name.ends_with(".msixvc")) + else { +- eprintln!("No .msixvc file found"); ++ // Legacy EAppx titles carry no .msixvc member: their bytes come ++ // from the download command and are unpacked by extract-eappx. ++ let legacy = package.package_files.iter().any(|p| { ++ let name = p.file_name.to_ascii_lowercase(); ++ name.ends_with(".eappx") || name.ends_with(".eappxbundle") ++ }); ++ if legacy { ++ eprintln!( ++ "This title uses a legacy EAppx package; use download then extract-eappx for this title" ++ ); ++ } else { ++ eprintln!("No .msixvc file found"); ++ } + return ExitCode::FAILURE; + }; + let Some(root) = file.cdn_root_paths.first() else { +diff --git a/crates/xodus-cli/src/package_plan.rs b/crates/xodus-cli/src/package_plan.rs +index 6187d56..f784c07 100644 +--- a/crates/xodus-cli/src/package_plan.rs ++++ b/crates/xodus-cli/src/package_plan.rs +@@ -93,7 +93,11 @@ fn inspect(product: &Value, id: &str) -> Result<(Vec, Vec), Stri + .filter(|name| package_family(name)).map(str::to_string); + let durable = kind == "Durable" && parent_product_id.is_some() && main_package_family_name.is_some() + && list(package, "Architectures").iter().any(|arch| arch.as_str() == Some("x64")); +- let supported = desktop && format == "MSIXVC" ++ // Legacy Store packages are installable on PC too. FH4 and other ++ // pre-MSIXVC titles ship as EAPPXBUNDLE/EAPPX and were refused here, ++ // which made the Store offer look unavailable even when owned. ++ let supported = desktop ++ && matches!(format.as_str(), "MSIXVC" | "EAPPX" | "EAPPXBUNDLE") + && (kind == "Game" || kind == "GameAddOn" || durable) + && properties["IsTrial"].as_bool() != Some(true) + && properties["IsPreOrder"].as_bool() != Some(true); +@@ -326,6 +330,17 @@ mod tests { + assert!(plan.select(None).is_err()); assert!(plan.select(Some(CID)).is_ok()); + assert_eq!(plan.json("9CCCCCCCCCCC","US")["status"],"choice"); + } ++ #[test] fn legacy_desktop_package_formats_are_installable() { ++ for format in ["EAPPX", "EAPPXBUNDLE"] { ++ let plan = from_products(ROOT,&BTreeMap::from([(ROOT.into(),package(ROOT,"Windows.Desktop",format))])).unwrap(); ++ assert_eq!(plan.select(None).unwrap(), CID, "{format}"); ++ assert_eq!(plan.json(ROOT,"US")["status"], "ready", "{format}"); ++ } ++ // A console legacy package is still not installable on PC. ++ let plan = from_products(ROOT,&BTreeMap::from([(ROOT.into(),package(ROOT,"Xbox.One","EAPPXBUNDLE"))])).unwrap(); ++ assert!(plan.select(None).is_err()); ++ assert!(plan.select(Some(CID)).is_err()); ++ } + #[test] fn durable_entitlement_back_references_do_not_form_bundle_cycles() { + const DURABLE: &str = "9DDDDDDDDDDD"; + let durable = json!({"ProductId":DURABLE,"ProductType":"Durable", diff --git a/tools/xbox/droiddeck-legacy-run.patch b/tools/xbox/droiddeck-legacy-run.patch new file mode 100644 index 0000000000..d7fbed5853 --- /dev/null +++ b/tools/xbox/droiddeck-legacy-run.patch @@ -0,0 +1,124 @@ +diff --git a/crates/xodus-cli/src/commands/run.rs b/crates/xodus-cli/src/commands/run.rs +index 44ea870..1d8e218 100644 +--- a/crates/xodus-cli/src/commands/run.rs ++++ b/crates/xodus-cli/src/commands/run.rs +@@ -20,6 +20,7 @@ use tokio::process::Command; + use xodus::models::secrets::Token; + use xodus::tokens::TokenManager; + ++use crate::commands::legacy_install::{dependencies, ensure_frameworks}; + use crate::license::get_license; + + const PLAYFAB_RELYING_PARTY: &str = "http://playfab.xboxlive.com/"; +@@ -135,6 +136,98 @@ fn wine_exit_code(status: std::process::ExitStatus) -> u8 { + status.code().unwrap_or_else(|| 128 + status.signal().unwrap_or(1)) as u8 + } + ++/// Start a title that was unpacked into its install folder. ++/// ++/// There is no container to read a content key for or to build a launch view ++/// from, so the named executable runs as it lies, with the PlayFab header the ++/// game signs in with. ++async fn run_unpacked( ++ client: &reqwest::Client, ++ tokens: &TokenManager, ++ folder: &Path, ++ wine: String, ++ exe: Option<&str>, ++ materialize_chain: bool, ++ market: Option, ++) -> ExitCode { ++ let Some(name) = exe.filter(|name| !name.is_empty()) else { ++ eprintln!("no executable was named for the unpacked folder {}", folder.display()); ++ return ExitCode::FAILURE; ++ }; ++ let image = folder.join(name); ++ if !image.is_file() { ++ eprintln!("{} is not in the unpacked folder {}", name, folder.display()); ++ return ExitCode::FAILURE; ++ } ++ // A packaged Win32 program starts like any executable. A UWP application ++ // expects the app model to activate a view first, and this runtime has none, ++ // so say so when the launch dies instead of leaving a loader failure. ++ if let Some(entry) = crate::commands::legacy_install::entry_point(folder) { ++ if !entry.eq_ignore_ascii_case("Windows.FullTrustApplication") { ++ eprintln!("NEXUS_LEGACY_INSTALL_DIAGNOSTIC_V1 uwp-entry-point"); ++ eprintln!("NEXUS_LEGACY_INSTALL_DETAIL_V1 {name} is activated by {entry}, a UWP app model this runtime has no host for"); ++ } ++ } ++ // An unpacked title gets no package graph, so the frameworks its manifest ++ // declares are fetched here the first time it launches. ++ let declared = dependencies(folder); ++ eprintln!("NEXUS_LEGACY_INSTALL_DETAIL_V1 unpacked launch {name} with {} declared frameworks", declared.len()); ++ if !declared.is_empty() { ++ if let Ok(delivery) = crate::commands::store_delivery::client() { ++ let region = market.unwrap_or_else(|| "US".into()).to_uppercase(); ++ if let Err(error) = ensure_frameworks(&delivery, folder, ®ion).await { ++ eprintln!("NEXUS_LEGACY_INSTALL_DETAIL_V1 framework install failed {}", crate::commands::legacy_install::plain_for_log(&error)); ++ } ++ } ++ } ++ let playfab = playfab_auth_header(client, tokens).await; ++ ++ if wine == "droiddeck:prepare" { ++ use std::io::Write; ++ if !materialize_chain { ++ return ExitCode::FAILURE; ++ } ++ let ready = serde_json::json!({ ++ "exe": image, ++ "playfab": playfab, ++ }); ++ println!("DROIDDECK_LAUNCH_V1 {}", ready); ++ let _ = std::io::stdout().flush(); ++ let mut release = String::new(); ++ let _ = std::io::stdin().read_line(&mut release); ++ return ExitCode::SUCCESS; ++ } ++ ++ let mut command = Command::new(&wine); ++ command.arg(&image); ++ match playfab { ++ Some(header) => { ++ command.env("XODUS_XSTS_PLAYFAB", header); ++ } ++ None => eprintln!("could not authorise a PlayFab token; in-game sign-in may fail"), ++ } ++ ++ let mut wn = match command.spawn() { ++ Ok(child) => child, ++ Err(err) => { ++ eprintln!("failed to start Wine: {}", err); ++ return ExitCode::FAILURE; ++ } ++ }; ++ ++ let pid = wn.id().unwrap_or(0); ++ ++ if pid > 0 { ++ let _ = ctrlc::set_handler(move || { ++ let _ = kill(Pid::from_raw(pid as i32), Signal::SIGINT); ++ }); ++ } ++ ++ let status = wn.wait().await.unwrap(); ++ ++ ExitCode::from(wine_exit_code(status)) ++} ++ + // Real files in a full launch view must not be redirected back into memfds. + // Keep source mappings and encrypted DLL mappings; no title/EAC special case. + fn launch_file_maps( +@@ -542,6 +635,12 @@ pub async fn run( + let out_absolute = std::fs::canonicalize(out).unwrap(); + let final_path = out.join(".xodus-streaming.msixvc"); + ++ // A legacy package unpacks where it was installed and keeps no streaming ++ // container, so the folder itself is what gets launched. ++ if !final_path.is_file() { ++ return run_unpacked(client, tokens, &out_absolute, wine, exe.as_deref(), materialize_chain, market).await; ++ } ++ + let mut file = OpenOptions::new() + .read(true) + .open(final_path.to_owned()) diff --git a/tools/xbox/prepare.py b/tools/xbox/prepare.py index 3066d9edda..cb243221b2 100755 --- a/tools/xbox/prepare.py +++ b/tools/xbox/prepare.py @@ -10,7 +10,8 @@ cache = Path(sys.argv[1]).resolve() lock = json.loads((recipe / 'backend.lock.json').read_text()) local_patches = ['droiddeck-credentials.patch', 'droiddeck-launch-view.patch', 'droiddeck-library-subscriptions.patch', 'droiddeck-credential-fallback.patch', 'droiddeck-credential-diagnostics.patch', 'droiddeck-cloud-token.patch', - 'droiddeck-licensing-device-ticket.patch', 'droiddeck-dlc-mount.patch'] + 'droiddeck-licensing-device-ticket.patch', 'droiddeck-dlc-mount.patch', + 'droiddeck-legacy-packages.patch', 'droiddeck-legacy-install.patch', 'droiddeck-legacy-run.patch'] identity = hashlib.sha256((recipe / 'backend.lock.json').read_bytes() + b''.join((recipe / name).read_bytes() for name in local_patches)).hexdigest() source = cache / ('source-' + identity[:16]) if not (source / '.prepared').is_file():