mirror of
https://github.com/Droid-Deck/DroidDeck.git
synced 2026-11-19 21:06:09 +00:00
* feat(xbox): install legacy EAppx titles through Windows Update A legacy Store title does not ship as MSIXVC. It ships as an encrypted EAppx bundle, which streaming rejects, so the store could list the game and never install it. Accept EAPPX and EAPPXBUNDLE beside MSIXVC, carry the format through the catalogue, the install record and the cached library, and dispatch legacy-install for those formats. Two backend patches do the work: - droiddeck-legacy-packages.patch lets the plan, license and extract verbs handle a legacy package, and expands a bundle in place (nesting bounded at three), decrypting each inner EAppx with the same keys. - droiddeck-legacy-install.patch adds legacy-install: resolve, license, download, extract, hoist AppxManifest.xml to the game root. Resolution asks the Xbox content service first and Windows Update second, because some legacy products return no files from the content service at all. The WU endpoint needs no Xbox credential but chains to Microsoft's ECC Product Root CA 2018, which Android's CA bundle does not carry, so that one public root is pinned for those requests. The manifest reader accepts AppxManifest.xml as well as MicrosoftGame.config, since a legacy package carries only the former. Running it against a real title exposed four things that only show up at full size: - A licence names a key by its 16-byte content id, while a bundle header names it as a (content id, package id) pair. Reading the 16-byte id yields a different pair, so a complete key file reported every key missing and no bundle could be opened. Keys are now aligned against the header's own ids before the header is judged. - Windows Update answers with several updates whose PackageContentId differs from the bundle's own. Licensing used to take the first and fail with content-http 400; every candidate is now tried in order, preferring the one the container is named after. - A container already in .xodus-container is reused rather than fetched again, so a failed run does not cost another download of tens of gigabytes. The space preflight no longer counts a container twice for the same reason. - A bundle that carries add-on packages unpacks each into its own folder, leaving the manifest one level down. Hoisting lifts the largest folder that holds a manifest, so the game still lands at the root beside its add-ons. Unpacking now reports bytes written, so the install bubble fills while a package is decrypted instead of holding a single unchanging word. * fix(xbox): unpack a legacy package across cores and report the bytes Unpacking one title ran as a single thread on a saturated core at about 40 MB/s. Files sit at independent offsets, so each worker now opens its own handle and takes every Nth file, with a 1 MiB read buffer in place of the 8 KiB default that turned one 64 KiB block into eight reads. Measured on the Forza Horizon 4 bundle, writes went from 40 MB/s to about 200 MB/s. Two things the first full-size run exposed: - The container was deleted only after the whole nested tree was unpacked, so a title that holds its package as one member held the archive and the decrypted copy at once. On this device that runs out of space partway through the second stage. The container is now deleted once its members are on disk, which frees the archive before they are decrypted. - A bundle member is written as a single file, so a counter that advances when a file finishes stays at zero for the whole of the longest stage. The reported figure now also follows the growth of the destination while that is the case, and never moves backwards when the container is freed. Progress is exact during the per-file stage: each file's declared length is added as it lands, instead of walking the whole destination every tick. * docs(xbox): record the completed legacy install The evidence section said a finished legacy install and its decrypted contents were unverified, because the first run was still unpacking when it was written. That install has now completed on the same device: the 85,219,173,025-byte container produced 79.3 GB at the game root with AppxManifest.xml and the package DLLs directly beneath it, and the launcher lists the title as installed. It also records what unpacking costs, since that decided the change: one thread held a core at 100% for about 40 MB/s and four reach about 200 MB/s, and releasing each container once its members are on disk is what keeps the peak footprint at one package rather than the archive plus its contents. The launch is not part of this evidence, and the entry says so. * fix(xbox): place the 64-bit framework beside a legacy title A Windows Update framework category lists one package per architecture under identifier-only names. The first package was unpacked, which on this device was the x86 build, so a 64-bit title got 32-bit DLLs and every load failed with STATUS_INVALID_IMAGE_FORMAT. Candidates are unpacked now and only the one whose manifest declares ProcessorArchitecture="x64" is kept. The framework folder was also never created, so a finished download failed at the move with "no such file or directory". unzip.rs belongs to this patch: legacy-install calls it and the recipe left it out, so a rebuild from the pinned source could not compile. * feat(xbox): report a UWP entry point at launch An unpacked package can name a COM class as its entry point instead of Windows.FullTrustApplication. That class is a UWP application, which expects the app model to activate a view; Wine has no host for it, so the launch died inside the loader with no explanation. The launch now reads the entry point and reports uwp-entry-point, and full-trust packages are unaffected. Forza Horizon 4 is such a package: its manifest names Forza.App. Its frameworks are installed and the loader is clean before that point. * fix(xbox): report why a launch was refused A failed prepare replaced the backend message with a fixed sign-in hint, so a title that was not installed or a runtime that refused all read as a bad account. The reason is kept, with paths and URLs dropped, and written to the store log. * fix(adb): dismiss the keyboard on Search and Go Search and Go were terminal IME actions but were not handled, so focus stayed in the field and the keyboard covered the results it had just fetched. * fix(xbox): keep a cache row without a legacy format plain Decoding added a format key with an empty value to every cached title, so a library restored from storage no longer matched the one that was saved. * test(ui): match the manifest message a legacy package reports The message covers both a MicrosoftGame.config and an AppxManifest, so the assertion follows the text the content dialog now prints. * fix(xbox): refuse to start a title that needs the Windows app model A legacy package that names an entry point other than Windows.FullTrustApplication is a UWP application. Wine has no app-model host, so starting its executable crashes inside the loader and the launch looks like nothing happened. Read the entry point from the manifest the installer already parses and stop the Play action with a message instead. Forza Horizon 4 names Forza.App, so it installs and reports as installed but cannot be started here yet. * fix(xbox): dispatch on the container format of the chosen package The catalogue records the format of the first installable desktop package, and the picker only replaced the content id. An offer that lists MSIXVC and a legacy container would then hand the legacy id to streaming, or the streamed id to legacy-install, and the install failed after resolution had succeeded. The choice now carries its own format and the catalogue value only fills a gap. * fix(xbox): keep an extraction reason that the sanitizer rewrote The extraction sanitizer replaces anything outside letters, digits, space, dot, underscore and dash with a question mark, so a reason such as "(os error 28)" arrived as "?os error 28?" and the allowlist dropped the line. The failure then showed only the stage code, with no cause. Add that replacement character to the class the allowlist accepts. * fix(xbox): print the reason a launch was refused The account channel sends a reason with a refusal, and the guest replaced it with a fixed sign-in message, so a title that was never installed still read as a bad account. The guest now shows the reason it was given, and a token holding a path is dropped whole instead of having its slashes removed and its directory names kept. * build(xbox): leave RUST_TEST_THREADS unset when the host has no value The recipe always passed the variable, empty when the host had none, and the pinned test harness rejects an empty count. On an ARM64 host that runs cargo test, so the default local build failed before compiling. * fix(xbox): rebuild the backend patches from the tree that ran The two patches in this branch were not the ones the verified install ran, and one of them did not compile. `unzip.rs` inflates deflated entries but the dependency that provides the inflater was never added, so `prepare.py` produced a tree that failed at `flate2::read::DeflateDecoder`. Anything else missing fell into the same gap: the install stage reported no bytes, so the bar held one value for the whole unpack, and a container a stopped run had finished was downloaded again. Rebuilt from the patched source, and fixed the defects a review of the patch found: - Unpacking reports bytes again: the workers count each file's declared length, and for the single-file bundle member the folder's growth stands in, which is what the device run showed. - A finished container is reused, so a retry after a failure past the download does not spend hours fetching it again. The space preflight counts it as reclaimable, since the retry truncates it first. - An install over a folder that already has a manifest replaces the old files instead of leaving the new package nested and reporting success. A first install still refuses to merge into an existing target. - The manifest search looks at the depth a nested bundle writes, not only one level down. - `withdraw_libraries` removes only the DLLs `copy_libraries` recorded copying, so a framework replacement cannot delete a library the title owns. - ZIP64 sizes and offsets stay `u64`; truncating them to `u32` read the wrong local header past 4 GiB. - The extraction checks the blockmap length against the footer's before writing, which the crate only does when asked. - The container being unpacked is never treated as a nested member, so `extract-eappx title.eappx .` no longer re-enters and deletes its own input. - `flate2` is declared and the lock is updated. `prepare.py` reproduces this from a pristine checkout and `cargo test` passes on it. * fix(xbox): apply the legacy install patch over the Nexus 0.2.15 backend * fix(xbox): install only the newest x64 build of a legacy title and size the space check at its real peak * fix(xbox): install a store package that is a plain archive instead of an XVD * fix(xbox): refuse a package entry that ends before its recorded size * fix(xbox): credit only complete staged containers toward the free space a legacy install needs * fix(xbox): bound each package entry to its data and check ZIP64 decoded sizes * fix(xbox): refuse duplicate package names and clear an unfinished download before measuring space * fix(xbox): bound package entries by the next header and stage a plain package until it validates * fix(xbox): hold the install folder lock through a legacy install * fix(xbox): fall back to an older build when Microsoft refuses the newest one a license * fix(xbox): keep the registered device across sign-out * feat(xbox): explain device-limit and license refusals, with a link to manage devices * fix(xbox): locate each build only once licensed, roll back a failed package swap, report the final license answer * fix(xbox): skip trial and preorder packages, read runtime behavior, keep licensing timeouts * fix(xbox): keep a failed package swap recoverable, size bundles up front, report key-write and framework errors cleanly * fix(xbox): drop a pending manifest check on any newer launch, keep committed backups out of restores, match the app-model order * fix(xbox): cancel pending launch checks on a new intent, remember entries a rollback could not remove * fix(xbox): clear the removal list before a restore moves old files back --------- Co-authored-by: TimmyCraft <46533729+TimmyTurner51@users.noreply.github.com>
571 lines
23 KiB
Diff
571 lines
23 KiB
Diff
diff --git a/crates/xodus-cli/src/commands/extract_eappx.rs b/crates/xodus-cli/src/commands/extract_eappx.rs
|
|
index c8d5ebf..5ea43da 100644
|
|
--- a/crates/xodus-cli/src/commands/extract_eappx.rs
|
|
+++ b/crates/xodus-cli/src/commands/extract_eappx.rs
|
|
@@ -1,80 +1,416 @@
|
|
-use std::path::PathBuf;
|
|
+use std::path::{Path, PathBuf};
|
|
use std::process::ExitCode;
|
|
|
|
use eappx::EAppxFile;
|
|
-use eappx::keys::KeyCollection;
|
|
+use eappx::keys::{KeyCollection, KeyId};
|
|
+
|
|
+/// A bundle header names each content key as a pair of GUIDs, `(content id,
|
|
+/// package id)`, while the licence names the same key by its 16-byte content id
|
|
+/// alone. The crate reads a 16-byte id as `(static prefix, content id)`, so the
|
|
+/// two never compare equal and a complete key file looks incomplete. Pair every
|
|
+/// key the header asks for with the licence entry that carries its content id.
|
|
+const SHORT_KEY_PREFIX: uuid::Uuid = uuid::uuid!("BB1755DB-5052-4B10-B2AB-F3ABF5CA5B41");
|
|
+
|
|
+fn align_keys(keys: &mut KeyCollection, needed: &[KeyId]) {
|
|
+ for id in needed {
|
|
+ if keys.keys.contains_key(id) {
|
|
+ continue;
|
|
+ }
|
|
+ let KeyId::Guid((content, _)) = id else { continue };
|
|
+ let carried = keys.keys.get(&KeyId::Guid((SHORT_KEY_PREFIX, *content))).cloned();
|
|
+ if let Some(data) = carried {
|
|
+ keys.add(id.clone(), data);
|
|
+ }
|
|
+ }
|
|
+}
|
|
+
|
|
+/// A bundle can hold another bundle, but nesting is bounded so a malformed
|
|
+/// archive cannot drive unbounded recursion.
|
|
+const MAX_BUNDLE_DEPTH: usize = 3;
|
|
|
|
/// Extracts a locally-stored EAppx/EMSIX package.
|
|
///
|
|
-/// This is intentionally local-only for now: no store lookup or download.
|
|
-/// See issue #91 - the initial goal is understanding/unpacking the format,
|
|
-/// not full download integration, which can build on this once it lands.
|
|
+/// A bundle is extracted in place, then each inner package that is itself an
|
|
+/// EAppx container is decrypted with the same content keys. Inner files that
|
|
+/// are plain ZIP-based packages are left as extracted: they carry no EAppx
|
|
+/// header, so there is nothing to decrypt.
|
|
pub async fn run(path: String, destination: String, key_file: Option<String>) -> ExitCode {
|
|
- let infile = PathBuf::from(&path);
|
|
- let outdir = PathBuf::from(&destination);
|
|
-
|
|
- let file = match std::fs::File::open(&infile) {
|
|
- Ok(f) => f,
|
|
- Err(e) => {
|
|
- eprintln!("Failed to open {:?}: {e}", infile);
|
|
- return ExitCode::FAILURE;
|
|
- }
|
|
- };
|
|
- let mut reader = std::io::BufReader::new(file);
|
|
+ run_metered(path, destination, key_file, None).await
|
|
+}
|
|
|
|
- let mut eappx = match EAppxFile::from_stream(&mut reader) {
|
|
- Ok(e) => e,
|
|
- Err(e) => {
|
|
- eprintln!("Failed to parse EAppx header: {e}");
|
|
- return ExitCode::FAILURE;
|
|
- }
|
|
- };
|
|
+/// Where the extraction of the current package begins and how much it will write.
|
|
+///
|
|
+/// A bundle's inner package is one enormous member, so counting finished files
|
|
+/// leaves the bar frozen for the whole of it. The counter is exact: it advances by
|
|
+/// each file's declared length as that file is written.
|
|
+#[derive(Default)]
|
|
+pub struct Meter {
|
|
+ /// Bytes written by the package being unpacked now.
|
|
+ pub written: std::sync::atomic::AtomicU64,
|
|
+ /// Bytes written by packages that already finished.
|
|
+ pub banked: std::sync::atomic::AtomicU64,
|
|
+ /// Bytes the package being unpacked now will write in total.
|
|
+ pub total: std::sync::atomic::AtomicU64,
|
|
+ turn: std::sync::Mutex<()>,
|
|
+}
|
|
|
|
- println!("{eappx}");
|
|
+impl Meter {
|
|
+ /// Bytes written and bytes to write, read as one consistent pair.
|
|
+ pub fn snapshot(&self) -> (u64, u64) {
|
|
+ use std::sync::atomic::Ordering::Relaxed;
|
|
+ let _turn = self.turn.lock().unwrap_or_else(|poisoned| poisoned.into_inner());
|
|
+ let banked = self.banked.load(Relaxed);
|
|
+ let total = self.total.load(Relaxed);
|
|
+ (banked + self.written.load(Relaxed).min(total), banked + total)
|
|
+ }
|
|
|
|
+ pub fn done(&self) -> u64 {
|
|
+ self.snapshot().0
|
|
+ }
|
|
+
|
|
+ pub fn all(&self) -> u64 {
|
|
+ self.snapshot().1
|
|
+ }
|
|
+
|
|
+ /// Closes the package being unpacked and opens the next one.
|
|
+ fn roll(&self, total: u64) {
|
|
+ use std::sync::atomic::Ordering::Relaxed;
|
|
+ let _turn = self.turn.lock().unwrap_or_else(|poisoned| poisoned.into_inner());
|
|
+ self.banked.fetch_add(self.total.load(Relaxed), Relaxed);
|
|
+ self.written.store(0, Relaxed);
|
|
+ self.total.store(total, Relaxed);
|
|
+ }
|
|
+}
|
|
+
|
|
+/// Extracts a package, publishing the size of the work in progress to `meter`.
|
|
+///
|
|
+/// `release` deletes the container once its members are on disk. A bundle of a
|
|
+/// full title is tens of gigabytes, and holding it while the members are
|
|
+/// decrypted can fill the device partway through.
|
|
+pub async fn run_metered(
|
|
+ path: String,
|
|
+ destination: String,
|
|
+ key_file: Option<String>,
|
|
+ meter: Option<&Meter>,
|
|
+) -> ExitCode {
|
|
+ run_blocking(path, destination, key_file, meter, false)
|
|
+}
|
|
+
|
|
+/// The extraction itself, without the async wrapper.
|
|
+///
|
|
+/// Callers that keep reporting while it works run this on a blocking thread,
|
|
+/// because it never yields: decryption and inflation are plain synchronous work.
|
|
+pub fn run_blocking(
|
|
+ path: String,
|
|
+ destination: String,
|
|
+ key_file: Option<String>,
|
|
+ meter: Option<&Meter>,
|
|
+ release: bool,
|
|
+) -> ExitCode {
|
|
let mut key_collection = KeyCollection::default();
|
|
if let Some(key_file) = key_file {
|
|
let mut keyfile = match std::fs::File::open(&key_file) {
|
|
Ok(f) => f,
|
|
Err(e) => {
|
|
- eprintln!("Failed to open key file {:?}: {e}", key_file);
|
|
- return ExitCode::FAILURE;
|
|
+ return report_failure(
|
|
+ &Failure::new("key-file-open", &key_file, &e.to_string()),
|
|
+ ExitCode::FAILURE,
|
|
+ );
|
|
}
|
|
};
|
|
match KeyCollection::from_reader(&mut keyfile) {
|
|
Ok(loaded) => key_collection.extend(loaded.keys),
|
|
Err(e) => {
|
|
- eprintln!("Failed to parse key file: {e}");
|
|
- return ExitCode::FAILURE;
|
|
+ return report_failure(
|
|
+ &Failure::new("key-file-parse", &key_file, &e.to_string()),
|
|
+ ExitCode::FAILURE,
|
|
+ );
|
|
}
|
|
}
|
|
}
|
|
|
|
- if !key_collection.has_required_keys(&eappx.header.key_ids) {
|
|
- eprintln!("Missing one or more required content keys for this package");
|
|
- return ExitCode::FAILURE;
|
|
+ match derive(
|
|
+ Path::new(&path),
|
|
+ Path::new(&destination),
|
|
+ &key_collection,
|
|
+ 0,
|
|
+ meter,
|
|
+ release,
|
|
+ ) {
|
|
+ Ok(()) => ExitCode::SUCCESS,
|
|
+ Err(failure) => report_failure(&failure, ExitCode::FAILURE),
|
|
}
|
|
+}
|
|
+
|
|
+/// A failure the launcher can name without printing a device path.
|
|
+///
|
|
+/// The app only forwards stderr lines that are on a known allowlist, so the two
|
|
+/// lines below are what make a stopped install say why it stopped. The reason is
|
|
+/// reduced to letters, digits and a few punctuation marks, which keeps a path or
|
|
+/// a key out of the log while still naming the cause.
|
|
+struct Failure {
|
|
+ code: &'static str,
|
|
+ name: String,
|
|
+ reason: String,
|
|
+}
|
|
|
|
- if let Err(e) = eappx.load_keys(&key_collection) {
|
|
- eprintln!("Failed to load keys: {e}");
|
|
- return ExitCode::FAILURE;
|
|
+impl Failure {
|
|
+ fn new(code: &'static str, name: &str, reason: &str) -> Self {
|
|
+ Self {
|
|
+ code,
|
|
+ name: sanitize(name, 60),
|
|
+ reason: sanitize(reason, 120),
|
|
+ }
|
|
}
|
|
|
|
- if !outdir.exists() {
|
|
- if let Err(e) = std::fs::create_dir_all(&outdir) {
|
|
- eprintln!("Failed to create output directory {:?}: {e}", outdir);
|
|
- return ExitCode::FAILURE;
|
|
+ fn at(code: &'static str, path: &Path, reason: &str) -> Self {
|
|
+ let name = path.file_name().map(|name| name.to_string_lossy().into_owned());
|
|
+ Self::new(code, &name.unwrap_or_default(), reason)
|
|
+ }
|
|
+}
|
|
+
|
|
+fn sanitize(text: &str, limit: usize) -> String {
|
|
+ text.chars()
|
|
+ .map(|c| if c.is_ascii_alphanumeric() || matches!(c, ' ' | '.' | '_' | '-') { c } else { '?' })
|
|
+ .take(limit)
|
|
+ .collect::<String>()
|
|
+ .trim()
|
|
+ .to_string()
|
|
+}
|
|
+
|
|
+fn report_failure(failure: &Failure, code: ExitCode) -> ExitCode {
|
|
+ eprintln!("NEXUS_EXTRACT_DIAGNOSTIC_V1 {}", failure.code);
|
|
+ eprintln!("NEXUS_EXTRACT_DETAIL_V1 {} {}", failure.name, failure.reason);
|
|
+ eprintln!("Extraction failed: {}: {}: {}", failure.code, failure.name, failure.reason);
|
|
+ code
|
|
+}
|
|
+
|
|
+// EAppxMagic is EXPH (single), EXSH and EXBH (bundle). Anything else is not an
|
|
+// EAppx container, so a bundle's plain ZIP members are not recursed into.
|
|
+fn is_eappx(path: &Path) -> bool {
|
|
+ use std::io::Read;
|
|
+ let Ok(mut file) = std::fs::File::open(path) else { return false };
|
|
+ let mut magic = [0u8; 4];
|
|
+ if file.read_exact(&mut magic).is_err() {
|
|
+ return false;
|
|
+ }
|
|
+ matches!(&magic, b"EXPH" | b"EXSH" | b"EXBH")
|
|
+}
|
|
+
|
|
+fn derive(
|
|
+ infile: &Path,
|
|
+ outdir: &Path,
|
|
+ keys: &KeyCollection,
|
|
+ depth: usize,
|
|
+ meter: Option<&Meter>,
|
|
+ release: bool,
|
|
+) -> Result<(), Failure> {
|
|
+ let file = std::fs::File::open(infile)
|
|
+ .map_err(|e| Failure::at("open", infile, &e.to_string()))?;
|
|
+ let mut reader = std::io::BufReader::new(file);
|
|
+
|
|
+ let mut eappx = EAppxFile::from_stream(&mut reader)
|
|
+ .map_err(|e| Failure::at("header", infile, &e.to_string()))?;
|
|
+
|
|
+ println!("{eappx}");
|
|
+
|
|
+ // The licence and the header name the same key differently, so the set is
|
|
+ // rebuilt per package and reconciled before the header is judged.
|
|
+ let mut required = KeyCollection::new(&keys.keys);
|
|
+ align_keys(&mut required, &eappx.header.key_ids);
|
|
+
|
|
+ if !required.has_required_keys(&eappx.header.key_ids) {
|
|
+ // The count is the useful part: a license that carries one key cannot
|
|
+ // open a bundle whose members each declare their own.
|
|
+ return Err(Failure::new(
|
|
+ "missing-keys",
|
|
+ &infile.file_name().map(|name| name.to_string_lossy().into_owned()).unwrap_or_default(),
|
|
+ &format!("{} of {} content keys present", required.keys.len(), eappx.header.key_ids.len()),
|
|
+ ));
|
|
+ }
|
|
+
|
|
+ eappx
|
|
+ .load_keys(&required)
|
|
+ .map_err(|e| Failure::at("load-keys", infile, &e.to_string()))?;
|
|
+
|
|
+ std::fs::create_dir_all(outdir)
|
|
+ .map_err(|e| Failure::at("create-dir", outdir, &e.to_string()))?;
|
|
+
|
|
+ unpack(infile, &eappx, &mut reader, outdir, meter, release)?;
|
|
+ // The handle must close before a released container gives its space back.
|
|
+ drop(reader);
|
|
+
|
|
+ println!("Extracted {}", infile.file_name().unwrap_or_default().to_string_lossy());
|
|
+
|
|
+ let mut nested: Vec<PathBuf> = std::fs::read_dir(outdir)
|
|
+ .map_err(|e| Failure::at("list", outdir, &e.to_string()))?
|
|
+ .filter_map(|entry| entry.ok().map(|entry| entry.path()))
|
|
+ // The container being unpacked may itself sit in the destination, which is
|
|
+ // what `extract-eappx ./title.eappx .` asks for. Re-entering it would
|
|
+ // recurse into the same bytes and then delete the caller's file.
|
|
+ .filter(|path| std::fs::canonicalize(path).ok() != std::fs::canonicalize(infile).ok())
|
|
+ .filter(|path| path.is_file() && is_eappx(path))
|
|
+ .collect();
|
|
+ nested.sort();
|
|
+
|
|
+ if nested.is_empty() {
|
|
+ return Ok(());
|
|
+ }
|
|
+ if depth >= MAX_BUNDLE_DEPTH {
|
|
+ return Err(Failure::at("nesting", infile, "bundle nesting exceeds the limit"));
|
|
+ }
|
|
+
|
|
+ for child in nested {
|
|
+ let name = child
|
|
+ .file_stem()
|
|
+ .map(|name| name.to_string_lossy().into_owned())
|
|
+ .ok_or_else(|| Failure::at("no-name", &child, "package without a usable name"))?;
|
|
+ derive(&child, &outdir.join(&name), keys, depth + 1, meter, release)?;
|
|
+ // The member's bytes are no longer needed once its contents are on disk.
|
|
+ // A bundle of a full title holds tens of gigabytes, and keeping every
|
|
+ // member beside the decrypted copy doubles what the install occupies.
|
|
+ std::fs::remove_file(&child).ok();
|
|
+ }
|
|
+
|
|
+ Ok(())
|
|
+}
|
|
+
|
|
+/// Bytes currently on disk under `dir`.
|
|
+///
|
|
+/// A bundle member is written as one file, so a per-file counter says nothing
|
|
+/// until it is finished. Measuring the destination catches the bytes as they
|
|
+/// land, whatever is writing them.
|
|
+pub fn tree_size(dir: &Path) -> u64 {
|
|
+ let mut total = 0;
|
|
+ let Ok(entries) = std::fs::read_dir(dir) else { return 0 };
|
|
+ for entry in entries.flatten() {
|
|
+ let Ok(kind) = entry.file_type() else { continue };
|
|
+ if kind.is_dir() {
|
|
+ total += tree_size(&entry.path());
|
|
+ } else if let Ok(meta) = entry.metadata() {
|
|
+ total += meta.len();
|
|
}
|
|
}
|
|
+ total
|
|
+}
|
|
+
|
|
+/// Buffer sizes for each worker's own handle on the container.
|
|
+///
|
|
+/// The crate's default 8 KiB reader turns one 64 KiB block into eight system
|
|
+/// calls, and a full title is millions of blocks.
|
|
+const READ_BUFFER: usize = 1 << 20;
|
|
+
|
|
+/// Workers used to unpack one package.
|
|
+///
|
|
+/// Files sit at independent offsets, so each worker opens its own handle and
|
|
+/// takes every Nth file. Decryption and inflation are both CPU-bound and one
|
|
+/// core was saturated, so this is where the headroom is. The limit is the
|
|
+/// device's write speed, which is reached a few workers in.
|
|
+fn workers() -> usize {
|
|
+ std::thread::available_parallelism()
|
|
+ .map(|count| count.get())
|
|
+ .unwrap_or(1)
|
|
+ .clamp(1, 4)
|
|
+}
|
|
|
|
- match eappx.extract(&mut reader, &outdir) {
|
|
- Ok(_) => {
|
|
- println!("Extracted {:?} to {:?}", infile, outdir);
|
|
- ExitCode::SUCCESS
|
|
+/// Writes a package's files, publishing what is left to write.
|
|
+///
|
|
+/// This is the crate's own `extract`, split across workers and reported as it
|
|
+/// goes. The blockmap already totals every file's uncompressed length, so the
|
|
+/// end point is known before the first byte is written.
|
|
+fn unpack(
|
|
+ infile: &Path,
|
|
+ eappx: &EAppxFile,
|
|
+ reader: &mut std::io::BufReader<std::fs::File>,
|
|
+ outdir: &Path,
|
|
+ meter: Option<&Meter>,
|
|
+ release: bool,
|
|
+) -> Result<(), Failure> {
|
|
+ let total: u64 = eappx.blockmap.files.iter().map(|package| package.size).sum();
|
|
+ if let Some(meter) = meter {
|
|
+ meter.roll(total);
|
|
+ }
|
|
+
|
|
+ eappx
|
|
+ .extract_footprint_files(reader, outdir)
|
|
+ .map_err(|e| Failure::at("extract", infile, &e.to_string()))?;
|
|
+
|
|
+ if eappx.header.is_bundle() {
|
|
+ eappx
|
|
+ .extract_blockmap_files(reader, outdir)
|
|
+ .map_err(|e| Failure::at("extract", infile, &e.to_string()))?;
|
|
+ eappx
|
|
+ .extract_bundle_files(reader, outdir)
|
|
+ .map_err(|e| Failure::at("extract", infile, &e.to_string()))?;
|
|
+ if release {
|
|
+ std::fs::remove_file(infile).ok();
|
|
}
|
|
- Err(e) => {
|
|
- eprintln!("Extraction failed: {e}");
|
|
- ExitCode::FAILURE
|
|
+ return Ok(());
|
|
+ }
|
|
+
|
|
+ let count = workers().min(eappx.blockmap.files.len().max(1));
|
|
+ let failure = std::sync::Mutex::new(None::<String>);
|
|
+ std::thread::scope(|scope| {
|
|
+ for shard in 0..count {
|
|
+ let failure = &failure;
|
|
+ scope.spawn(move || {
|
|
+ let Ok(handle) = std::fs::File::open(infile) else {
|
|
+ if let Ok(mut slot) = failure.lock() {
|
|
+ *slot = Some("a worker could not reopen the package".to_string());
|
|
+ }
|
|
+ return;
|
|
+ };
|
|
+ let mut stream = std::io::BufReader::with_capacity(READ_BUFFER, handle);
|
|
+ for (index, package) in eappx.blockmap.files.iter().enumerate() {
|
|
+ if index % count != shard {
|
|
+ continue;
|
|
+ }
|
|
+ if failure.lock().map(|slot| slot.is_some()).unwrap_or(true) {
|
|
+ return;
|
|
+ }
|
|
+ let Some(footer) = eappx.find_footer_for_file(package.id()) else {
|
|
+ if let Ok(mut slot) = failure.lock() {
|
|
+ *slot = Some("the blockmap names a file the package does not hold".to_string());
|
|
+ }
|
|
+ return;
|
|
+ };
|
|
+ let mut info = eappx::FileInfo::from(footer);
|
|
+ // The crate checks these only when asked, but filling them
|
|
+ // keeps the per-file path identical to its own extraction.
|
|
+ info.filehash = package.filehash_bytes();
|
|
+ info.block_hashes = Some(package.block_hashes());
|
|
+ // The blockmap's length and the footer's must agree, or the
|
|
+ // file written would be silently short.
|
|
+ if info.uncompressed_length != package.size {
|
|
+ if let Ok(mut slot) = failure.lock() {
|
|
+ *slot = Some("a package entry disagrees with its blockmap about its size".to_string());
|
|
+ }
|
|
+ return;
|
|
+ }
|
|
+ if let Err(error) = eappx.save_file_to_fs(&mut stream, info, outdir, &package.name) {
|
|
+ if let Ok(mut slot) = failure.lock() {
|
|
+ *slot = Some(error.to_string());
|
|
+ }
|
|
+ return;
|
|
+ }
|
|
+ if let Some(meter) = meter {
|
|
+ meter
|
|
+ .written
|
|
+ .fetch_add(package.size, std::sync::atomic::Ordering::Relaxed);
|
|
+ }
|
|
+ }
|
|
+ });
|
|
}
|
|
+ });
|
|
+
|
|
+ if let Some(reason) = failure.into_inner().unwrap_or(None) {
|
|
+ return Err(Failure::at("extract", infile, &reason));
|
|
+ }
|
|
+
|
|
+ if release {
|
|
+ // The files are on disk; the container is not read again unless the
|
|
+ // install is retried from scratch.
|
|
+ std::fs::remove_file(infile).ok();
|
|
}
|
|
+ Ok(())
|
|
}
|
|
diff --git a/crates/xodus-cli/src/commands/license.rs b/crates/xodus-cli/src/commands/license.rs
|
|
index 4540549..b9fd91c 100644
|
|
--- a/crates/xodus-cli/src/commands/license.rs
|
|
+++ b/crates/xodus-cli/src/commands/license.rs
|
|
@@ -1,5 +1,6 @@
|
|
use std::process::ExitCode;
|
|
|
|
+use base64::{Engine, engine::general_purpose::STANDARD};
|
|
use tokio::fs::OpenOptions;
|
|
use tokio::io::AsyncWriteExt;
|
|
use xodus::tokens::TokenManager;
|
|
@@ -21,6 +22,10 @@ pub async fn run(
|
|
|
|
let (key, game_splicense) = license.unwrap();
|
|
tokio::fs::create_dir_all(&ciks).await.unwrap();
|
|
+ // eappx-rs reads a text [Keys] file with base64 Key IDs, while the .cik
|
|
+ // format below is the binary form other tools expect. Write both so the
|
|
+ // same license dump serves either extractor.
|
|
+ let mut eappx_keys = String::from("[Keys]\n");
|
|
for (uuid, content_key) in game_splicense.content_keys {
|
|
let unpacked = content_key.unpack(&key).expect("failed to unpack");
|
|
let mut file = OpenOptions::new()
|
|
@@ -33,7 +38,21 @@ pub async fn run(
|
|
file.write_all(&uuid_buf).await.unwrap();
|
|
file.write_all(&*unpacked).await.unwrap();
|
|
file.flush().await.unwrap();
|
|
+ eappx_keys.push_str(&format!(
|
|
+ "\"{}\" \"{}\"\n",
|
|
+ STANDARD.encode(uuid_buf),
|
|
+ STANDARD.encode(&*unpacked),
|
|
+ ));
|
|
}
|
|
+ let mut keys_file = OpenOptions::new()
|
|
+ .create(true)
|
|
+ .write(true)
|
|
+ .truncate(true)
|
|
+ .open(format!("{ciks}/eappx-keys.txt"))
|
|
+ .await
|
|
+ .unwrap();
|
|
+ keys_file.write_all(eappx_keys.as_bytes()).await.unwrap();
|
|
+ keys_file.flush().await.unwrap();
|
|
|
|
ExitCode::SUCCESS
|
|
}
|
|
diff --git a/crates/xodus-cli/src/commands/streaming.rs b/crates/xodus-cli/src/commands/streaming.rs
|
|
index 66e4135..a0136f8 100644
|
|
--- a/crates/xodus-cli/src/commands/streaming.rs
|
|
+++ b/crates/xodus-cli/src/commands/streaming.rs
|
|
@@ -160,7 +160,19 @@ pub async fn run(
|
|
.iter()
|
|
.find(|p| p.file_name.ends_with(".msixvc"))
|
|
else {
|
|
- eprintln!("No .msixvc file found");
|
|
+ // Legacy EAppx titles carry no .msixvc member: their bytes come
|
|
+ // from the download command and are unpacked by extract-eappx.
|
|
+ let legacy = package.package_files.iter().any(|p| {
|
|
+ let name = p.file_name.to_ascii_lowercase();
|
|
+ name.ends_with(".eappx") || name.ends_with(".eappxbundle")
|
|
+ });
|
|
+ if legacy {
|
|
+ eprintln!(
|
|
+ "This title uses a legacy EAppx package; use download then extract-eappx for this title"
|
|
+ );
|
|
+ } else {
|
|
+ eprintln!("No .msixvc file found");
|
|
+ }
|
|
return ExitCode::FAILURE;
|
|
};
|
|
let Some(root) = file.cdn_root_paths.first() else {
|
|
diff --git a/crates/xodus-cli/src/package_plan.rs b/crates/xodus-cli/src/package_plan.rs
|
|
index 6187d56..f784c07 100644
|
|
--- a/crates/xodus-cli/src/package_plan.rs
|
|
+++ b/crates/xodus-cli/src/package_plan.rs
|
|
@@ -93,7 +93,11 @@ fn inspect(product: &Value, id: &str) -> Result<(Vec<Choice>, Vec<String>), Stri
|
|
.filter(|name| package_family(name)).map(str::to_string);
|
|
let durable = kind == "Durable" && parent_product_id.is_some() && main_package_family_name.is_some()
|
|
&& list(package, "Architectures").iter().any(|arch| arch.as_str() == Some("x64"));
|
|
- let supported = desktop && format == "MSIXVC"
|
|
+ // Legacy Store packages are installable on PC too. FH4 and other
|
|
+ // pre-MSIXVC titles ship as EAPPXBUNDLE/EAPPX and were refused here,
|
|
+ // which made the Store offer look unavailable even when owned.
|
|
+ let supported = desktop
|
|
+ && matches!(format.as_str(), "MSIXVC" | "EAPPX" | "EAPPXBUNDLE")
|
|
&& (kind == "Game" || kind == "GameAddOn" || durable)
|
|
&& properties["IsTrial"].as_bool() != Some(true)
|
|
&& properties["IsPreOrder"].as_bool() != Some(true);
|
|
@@ -326,6 +330,17 @@ mod tests {
|
|
assert!(plan.select(None).is_err()); assert!(plan.select(Some(CID)).is_ok());
|
|
assert_eq!(plan.json("9CCCCCCCCCCC","US")["status"],"choice");
|
|
}
|
|
+ #[test] fn legacy_desktop_package_formats_are_installable() {
|
|
+ for format in ["EAPPX", "EAPPXBUNDLE"] {
|
|
+ let plan = from_products(ROOT,&BTreeMap::from([(ROOT.into(),package(ROOT,"Windows.Desktop",format))])).unwrap();
|
|
+ assert_eq!(plan.select(None).unwrap(), CID, "{format}");
|
|
+ assert_eq!(plan.json(ROOT,"US")["status"], "ready", "{format}");
|
|
+ }
|
|
+ // A console legacy package is still not installable on PC.
|
|
+ let plan = from_products(ROOT,&BTreeMap::from([(ROOT.into(),package(ROOT,"Xbox.One","EAPPXBUNDLE"))])).unwrap();
|
|
+ assert!(plan.select(None).is_err());
|
|
+ assert!(plan.select(Some(CID)).is_err());
|
|
+ }
|
|
#[test] fn durable_entitlement_back_references_do_not_form_bundle_cycles() {
|
|
const DURABLE: &str = "9DDDDDDDDDDD";
|
|
let durable = json!({"ProductId":DURABLE,"ProductType":"Durable",
|