Windows components: install .msi packages (XNA, PhysX, GFWL, MSXML...) without running them (#347)

* Windows components: install .msi packages (XNA, PhysX, GFWL, MSXML...) without running them

droiddeck-msi-install reads a Windows Installer package's tables with msiinfo and unpacks its
cabinets with cabextract, then lays out what Wine's msiexec would have installed: files at their
64-bit Windows paths (Program Files (x86), syswow64...), .NET assemblies in the GAC, side-by-side
assemblies in winsxs, and the registry values in the view the component writes through. Conditions,
features and the folder-setting custom actions are evaluated as on Windows 10 64-bit; what needs
the package's own code (DXSETUP, self-registration) is listed in the component's notes.

droiddeck-wincomponents puts a component's drive_c tree into the prefix (keeping the prefix's own
spelling of each folder) and writes its registry values into system.reg/user.reg before launch,
following Wine's link keys (Wow6432Node\Classes, CurrentControlSet...). Turning it off puts every
value back and takes an added PATH entry out again.

The app installs these from the Windows components menu like the DLL ones: it downloads the .msi
into the runtime and runs the installer there (GuestCommand). msitools 0.106 and cabextract 1.11
come from Arch Linux ARM, pinned in tools/msitools/packages.txt and hosted on the msitools-0.106-r1
release. wine-gecko stays hidden (Proton ships it) and wine-mono keeps waiting (uninstall step).

* Windows components: open with the top switch focused, by touch or by pad

The dialog's window starts in touch mode, where a switch cannot take focus, and the first switch
was asked once, two frames after the list loaded, only when a pad opened it - racing the dialog's
own Done focus. Now it always puts the window in pad mode, holds Done while the list loads and
then asks the top switch every frame until it has focus.

* Windows components: give msitools its own libraries; show a failed install on its row

msiinfo and cabextract sit with their libraries in /usr/local/lib/droiddeck-msitools, which the
loader does not search: every .msi install stopped at "libmsi-1.0.so.0: cannot open shared
object file". droiddeck-msi-install now runs them with that folder on LD_LIBRARY_PATH.

A failed install also says so under the component itself ("Could not install: ..."), not
only at the foot of the dialog, past the list.

---------

Co-authored-by: The412Banner <the412banner@users.noreply.github.com>
This commit is contained in:
The412BannerandThe412Banner authored and GitHub committed 2026-10-06 14:14:28 -04:00
1 parent 1d93626453
commit 08559a02f8
14 files changed
+3163 -28

No files matched your search

+771
View File
@@ -0,0 +1,771 @@
#!/usr/bin/python3
"""Installs a Windows Installer package (.msi) the way Wine's own msiexec would, without running it.
droiddeck-msi-install <package.msi> <component dir>
A .msi is a database: which files go to which folder, which registry values to write, which .NET
and side-by-side assemblies to register. This reads those tables (msiinfo), unpacks the package's
cabinets (cabextract, which knows every compression a cabinet can use) and lays the result out
for droiddeck-wincomponents, which puts it into a game's prefix at launch:
<component dir>/drive_c/... the files, at the paths they get on 64-bit Windows
<component dir>/registry.json the registry values, in the view (32/64-bit) Windows uses
What a package does by running its own code (custom actions such as DXSETUP, self-registering
DLLs) cannot be done this way; each is named in the notes, so nothing is left out silently.
Prints "progress <text>" lines while it works and ends with "result <json>": the counts and notes.
"""
import json
import os
import re
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path
TOOLS = Path(os.environ.get("DROIDDECK_MSITOOLS", "/usr/local/lib/droiddeck-msitools"))
MSIINFO = os.environ.get("DROIDDECK_MSIINFO", str(TOOLS / "msiinfo"))
CABEXTRACT = os.environ.get("DROIDDECK_CABEXTRACT", str(TOOLS / "cabextract"))
def tool_env():
"""The environment the tools run in: their own libraries (libmsi, libgsf, libgcab) beside them."""
env = dict(os.environ)
env["LD_LIBRARY_PATH"] = ":".join(p for p in (str(TOOLS), env.get("LD_LIBRARY_PATH", "")) if p)
return env
# The folders Windows Installer knows by name, as a 64-bit Windows 10 prefix lays them out (Proton's
# user is steamuser). None is a folder whose files a game never needs: installer temp files and
# shortcuts.
SKIP = None
KNOWN_FOLDERS = {
"TARGETDIR": "C:\\", "ROOTDRIVE": "C:\\", "WindowsVolume": "C:\\",
"WindowsFolder": "C:\\windows\\",
"SystemFolder": "C:\\windows\\syswow64\\",
"System64Folder": "C:\\windows\\system32\\",
"System16Folder": "C:\\windows\\system\\",
"FontsFolder": "C:\\windows\\Fonts\\",
"ProgramFilesFolder": "C:\\Program Files (x86)\\",
"ProgramFiles64Folder": "C:\\Program Files\\",
"CommonFilesFolder": "C:\\Program Files (x86)\\Common Files\\",
"CommonFiles64Folder": "C:\\Program Files\\Common Files\\",
"CommonAppDataFolder": "C:\\ProgramData\\",
"AppDataFolder": "C:\\users\\steamuser\\AppData\\Roaming\\",
"LocalAppDataFolder": "C:\\users\\steamuser\\AppData\\Local\\",
"PersonalFolder": "C:\\users\\steamuser\\Documents\\",
}
for _name in ("TempFolder", "DesktopFolder", "ProgramMenuFolder", "StartMenuFolder", "StartupFolder",
"SendToFolder", "FavoritesFolder", "NetHoodFolder", "PrintHoodFolder", "RecentFolder",
"TemplateFolder", "MyPicturesFolder", "AdminToolsFolder"):
KNOWN_FOLDERS[_name] = SKIP
# What Wine's msi reports on a 64-bit Windows 10 prefix, for the packages' conditions.
SYSTEM_PROPERTIES = {
"VersionNT": "603", "VersionNT64": "603", "WindowsBuild": "9600", "ServicePackLevel": "0",
"Msix64": "6", "ALLUSERS": "1", "Privileged": "1", "AdminUser": "1", "MsiNTProductType": "1",
"UILevel": "2", "INSTALLLEVEL": "1", "ProgramFilesFolder": None, "VersionMsi": "5.00",
}
# Win64 component (msidbComponentAttributes64bit); file attributes.
COMPONENT_64BIT = 0x100
FILE_NONCOMPRESSED = 0x2000
# Custom action types that only set a property or a folder: done here, the rest are named in notes.
SET_PROPERTY, SET_DIRECTORY, ERROR_ACTION = 51, 35, 19
# Microsoft's "SFP CA" merge module (MDAC, MSXML): on Windows 2000 and later it parks system DLLs in a
# cache folder and its own code copies them into the system folder past Windows File Protection.
# Skipping the folder change puts them where that copy would. Its actions, and the side-by-side
# merge module's (done here from MsiAssembly), are then left out of the notes.
SFP_CACHE_ACTION = "CDirSystemSFPCacheDir"
HANDLED_ACTIONS = ("Wdsfpca_", "SxsInstallCA", "SxsUninstallCA")
# Tables whose work needs the package's own code or is not done here yet.
UNSUPPORTED_TABLES = ("SelfReg", "Class", "ProgId", "TypeLib", "Extension", "Verb", "MIME", "AppId",
"IniFile", "ServiceInstall", "MoveFile", "IsolatedComponent", "PublishComponent",
"ODBCDriver", "ODBCDataSource")
class MsiError(Exception):
pass
def progress(text):
print("progress " + text, flush=True)
# ---------------------------------------------------------------- reading the package
IDT_ESCAPES = str.maketrans({"\x19": "\n", "\x11": "\r", "\x15": "\t", "\x1b": ";"})
def msiinfo(*args, binary=False):
try:
result = subprocess.run([MSIINFO, *args], capture_output=True, timeout=600, env=tool_env())
except OSError as error:
raise MsiError("msitools is not available (%s)" % error)
if result.returncode != 0:
raise MsiError("msiinfo %s: %s" % (args[0], result.stderr.decode("utf-8", "replace").strip()[:300]))
return result.stdout if binary else result.stdout.decode("utf-8", "replace")
class Package:
def __init__(self, path):
self.path = str(path)
self.table_names = set(msiinfo("tables", self.path).split())
self._tables = {}
self.summary = {}
for line in msiinfo("suminfo", self.path).splitlines():
name, sep, value = line.partition(":")
if sep:
self.summary[name.strip().lower()] = value.strip()
def table(self, name):
"""The table's rows as dicts by column name; an absent table has none."""
if name in self._tables:
return self._tables[name]
rows = []
if name in self.table_names:
lines = [line.rstrip("\r") for line in msiinfo("export", self.path, name).split("\n")]
columns = lines[0].split("\t") if lines else []
for line in lines[3:]:
if not line:
continue
cells = [cell.translate(IDT_ESCAPES) for cell in line.split("\t")]
cells += [""] * (len(columns) - len(cells))
rows.append(dict(zip(columns, cells)))
self._tables[name] = rows
return rows
def stream(self, name, target):
Path(target).write_bytes(msiinfo("extract", self.path, name, binary=True))
def long_name(value):
"""A Directory/File name column: "short|long", the long one when there are two."""
return value.split("|")[-1]
def to_int(value, default=0):
try:
return int(str(value).strip())
except ValueError:
return default
# ---------------------------------------------------------------- conditions
TOKEN = re.compile(r'\s*(?:(?P<str>"[^"]*")|(?P<num>-?\d+)(?![\w.])|(?P<op>~?(?:<>|<=|>=|><|<<|>>|=|<|>))'
r'|(?P<paren>[()])|(?P<word>[%$?&!]?[A-Za-z_][\w.]*))')
KEYWORDS = {"not", "and", "or", "xor", "eqv", "imp"}
class Conditions:
"""Windows Installer's condition language, enough of it for what packages ask at install time."""
def __init__(self, props, feature_state=None, component_state=None):
self.props = props
self.feature_state = feature_state or (lambda name: None)
self.component_state = component_state or (lambda name: None)
def evaluate(self, text, default=True):
text = (text or "").strip()
if not text:
return True
try:
self.tokens = self.tokenize(text)
self.pos = 0
value = self.expression()
if self.pos != len(self.tokens):
return default
return self.truth(value)
except (ValueError, IndexError):
return default
def tokenize(self, text):
tokens, pos = [], 0
while pos < len(text):
if text[pos:].strip() == "":
break
match = TOKEN.match(text, pos)
if not match or match.end() == pos:
raise ValueError(text)
pos = match.end()
kind = match.lastgroup
value = match.group(kind)
if kind == "word" and value.lower() in KEYWORDS:
tokens.append(("kw", value.lower()))
else:
tokens.append((kind, value))
return tokens
def peek(self):
return self.tokens[self.pos] if self.pos < len(self.tokens) else (None, None)
def take(self, kind, value=None):
token = self.peek()
if token[0] == kind and (value is None or token[1] == value):
self.pos += 1
return True
return False
def expression(self):
value = self.binary(0)
return value
LEVELS = ("imp", "eqv", "xor", "or", "and")
def binary(self, level):
if level == len(self.LEVELS):
return self.negation()
left = self.binary(level + 1)
op = self.LEVELS[level]
while self.take("kw", op):
right = self.truth(self.binary(level + 1))
a = self.truth(left)
left = {"and": a and right, "or": a or right, "xor": a != right,
"eqv": a == right, "imp": (not a) or right}[op]
return left
def negation(self):
if self.take("kw", "not"):
return not self.truth(self.negation())
return self.comparison()
def comparison(self):
left = self.operand()
kind, op = self.peek()
if kind != "op":
return left
self.pos += 1
right = self.operand()
return self.compare(left, op, right)
def operand(self):
kind, value = self.peek()
self.pos += 1
if kind == "paren" and value == "(":
inner = self.expression()
if not self.take("paren", ")"):
raise ValueError("unbalanced")
return inner
if kind == "str":
return value[1:-1]
if kind == "num":
return int(value)
if kind == "word":
sigil, name = (value[0], value[1:]) if value[0] in "%$?&!" else ("", value)
if sigil == "%":
return os.environ.get(name, "")
if sigil in ("&", "!"):
state = self.feature_state(name)
return state if state is not None else -1
if sigil in ("$", "?"):
state = self.component_state(name)
return state if state is not None else -1
found = self.props.get(name)
return "" if found is None else found
raise ValueError("operand expected")
@staticmethod
def number(value):
if isinstance(value, bool):
return int(value)
if isinstance(value, int):
return value
if isinstance(value, str) and re.fullmatch(r"-?\d+", value.strip()):
return int(value)
return None
def compare(self, left, op, right):
fold = op.startswith("~")
op = op.lstrip("~")
a, b = self.number(left), self.number(right)
if a is not None and b is not None:
return {"=": a == b, "<>": a != b, "<": a < b, ">": a > b, "<=": a <= b, ">=": a >= b,
"><": (a & b) != 0, "<<": (a >> 16) == b, ">>": (a & 0xFFFF) == b}[op]
if isinstance(left, (int, bool)) or isinstance(right, (int, bool)):
return op == "<>"
a, b = str(left), str(right)
if fold:
a, b = a.lower(), b.lower()
return {"=": a == b, "<>": a != b, "<": a < b, ">": a > b, "<=": a <= b, ">=": a >= b,
"><": b in a, "<<": a.startswith(b), ">>": a.endswith(b)}[op]
@staticmethod
def truth(value):
if isinstance(value, bool):
return value
if isinstance(value, int):
return value != 0
return value not in ("", None)
# ---------------------------------------------------------------- the install
def win_to_rel(path):
"""C:\\windows\\x\\ -> drive_c/windows/x; None for a path off drive C or a folder left out."""
if not path:
return None
match = re.match(r"^[Cc]:\\?(.*)$", path)
if not match:
return None
rest = [part for part in match.group(1).replace("/", "\\").split("\\") if part not in ("", ".")]
if any(part == ".." for part in rest):
return None
return "/".join(["drive_c"] + rest)
def rel_to_win(rel, folder=True):
parts = rel.split("/")[1:]
return "C:\\" + "\\".join(parts) + ("\\" if folder and parts else "")
def clean_version(version):
parts = [str(to_int(p)) for p in version.split(".") if p != ""]
return ".".join(parts) if parts else "0.0.0.0"
def clr_version(path):
"""The .NET runtime an assembly was built for ("v4.0.30319"), from its metadata header."""
try:
data = Path(path).read_bytes()
except OSError:
return ""
at = data.find(b"BSJB")
if at < 0 or at + 16 > len(data):
return ""
length = int.from_bytes(data[at + 12:at + 16], "little")
return data[at + 16:at + 16 + min(length, 32)].split(b"\0")[0].decode("ascii", "replace")
class Installer:
def __init__(self, msi, out, package=None):
self.pkg = package or Package(msi)
self.out = Path(out)
self.notes = []
self.registry = []
self.files = {} # File key -> rel path it was installed to
self.counts = {"files": 0, "assemblies": 0, "registry": 0}
template = self.pkg.summary.get("template", "")
self.platform = template.split(";")[0].strip().lower()
self.props = {k: v for k, v in SYSTEM_PROPERTIES.items() if v is not None}
if self.platform in ("", "intel"):
self.props["Intel"] = "6"
for name, path in KNOWN_FOLDERS.items():
self.props[name] = path if path is not None else "\0skip"
for row in self.pkg.table("Property"):
# The package's own defaults, but never over the folders and the system it runs on.
if row["Property"] not in self.props or row["Property"] in ("INSTALLLEVEL", "ALLUSERS", "UILevel"):
self.props[row["Property"]] = row["Value"]
self.props["ALLUSERS"] = "1"
self.directories = {row["Directory"]: row for row in self.pkg.table("Directory")}
self.components = {row["Component"]: row for row in self.pkg.table("Component")}
self.included_features = set()
self.included_components = set()
self.conditions = Conditions(self.props, self.feature_state, self.component_state)
self._resolving = set()
self._set_by_action = set()
# -- states, folders, formatted text
def feature_state(self, name):
return 3 if name in self.included_features else -1
def component_state(self, name):
return 3 if name in self.included_components else -1
def directory(self, key):
"""The folder a Directory key resolves to, as drive_c/..., or None when it is left out."""
if key in self._resolving:
return None
value = self.props.get(key)
row = self.directories.get(key)
# A folder Windows knows by name, one a custom action set, or a property naming a path.
if key in KNOWN_FOLDERS or key in self._set_by_action or row is None:
return None if value in (None, "", "\0skip") else win_to_rel(value)
parent_key = row.get("Directory_Parent", "")
default = long_name(row.get("DefaultDir", "").split(":")[0])
if not parent_key or parent_key == key:
# A root: TARGETDIR (SourceDir) is drive C.
return "drive_c"
self._resolving.add(key)
try:
parent = self.directory(parent_key)
finally:
self._resolving.discard(key)
if parent is None:
return None
if default in (".", ""):
return parent
if "\\" in default or "/" in default or default == "..":
return None
return parent + "/" + default
def format(self, text):
"""Formatted text: [Property], [Folder], [#file], [$component], [%env], [\\x], [~]."""
if not text:
return text
def one(match):
inner = match.group(1)
if inner.startswith("\\") and len(inner) == 2:
return inner[1]
if inner == "~":
return "\0"
if inner.startswith("#") or inner.startswith("!"):
rel = self.files.get(inner[1:])
return rel_to_win(rel, folder=False) if rel else ""
if inner.startswith("$"):
comp = self.components.get(inner[1:])
rel = self.directory(comp["Directory_"]) if comp else None
return rel_to_win(rel) if rel else ""
if inner.startswith("%"):
return os.environ.get(inner[1:], "")
if inner in self.directories or inner in KNOWN_FOLDERS:
rel = self.directory(inner)
return rel_to_win(rel) if rel else ""
value = self.props.get(inner, "")
return "" if value == "\0skip" else value
previous = None
for _ in range(8):
if text == previous:
break
previous = text
text = re.sub(r"\[([^\[\]]*)\]", one, text)
return text
# -- what gets installed
def run_set_actions(self):
"""The property/folder-setting custom actions, in sequence order; the rest go to the notes."""
actions = {row["Action"]: row for row in self.pkg.table("CustomAction")}
sequence = {}
for table in ("InstallUISequence", "InstallExecuteSequence"):
for row in self.pkg.table(table):
if row["Action"] in actions and row["Action"] not in sequence:
sequence[row["Action"]] = (to_int(row.get("Sequence"), 0), row.get("Condition", ""))
skipped = []
sfp = "_SfpCaInf" in self.pkg.table_names
for name, (_, condition) in sorted(sequence.items(), key=lambda item: item[1][0]):
row = actions[name]
kind = to_int(row["Type"]) & 0x3F
if name.startswith(HANDLED_ACTIONS) or (sfp and name.startswith(SFP_CACHE_ACTION)):
continue
if kind in (SET_PROPERTY, SET_DIRECTORY):
if not self.conditions.evaluate(condition, default=True):
continue
value = self.format(row.get("Target", ""))
self.props[row["Source"]] = value
self._set_by_action.add(row["Source"])
elif kind != ERROR_ACTION and self.conditions.evaluate(condition, default=True):
skipped.append(name)
if skipped:
self.notes.append("setup actions not run (they need the installer's own code): " + ", ".join(skipped[:8])
+ (" and %d more" % (len(skipped) - 8) if len(skipped) > 8 else ""))
def select(self):
install_level = to_int(self.props.get("INSTALLLEVEL"), 1) or 1
features = {row["Feature"]: dict(row) for row in self.pkg.table("Feature")}
for row in self.pkg.table("Condition"):
feature = features.get(row["Feature_"])
if feature is not None and self.conditions.evaluate(row["Condition"], default=False):
feature["Level"] = row["Level"]
def included(name, depth=0):
feature = features.get(name)
if feature is None or depth > 32:
return False
level = to_int(feature.get("Level"), 0)
if level <= 0 or level > install_level:
return False
parent = feature.get("Feature_Parent", "")
return not parent or parent == name or included(parent, depth + 1)
self.included_features = {name for name in features if included(name)}
mapped = {}
for row in self.pkg.table("FeatureComponents"):
mapped.setdefault(row["Component_"], set()).add(row["Feature_"])
for name, row in self.components.items():
owners = mapped.get(name)
if owners is not None and not owners & self.included_features:
continue
if not self.conditions.evaluate(row.get("Condition", ""), default=True):
continue
self.included_components.add(name)
def assemblies(self):
"""Component -> where its files go when it is a global assembly (the GAC or winsxs)."""
names = {}
for row in self.pkg.table("MsiAssemblyName"):
names.setdefault(row["Component_"], {})[row["Name"].lower()] = row["Value"]
placed = {}
for row in self.pkg.table("MsiAssembly"):
comp = row["Component_"]
if comp not in self.included_components or row.get("File_Application"):
continue
ident = names.get(comp, {})
placed[comp] = (to_int(row.get("Attributes"), 0), ident, row.get("File_Manifest", ""))
return placed
def gac_folder(self, ident, assembly_file):
name = ident.get("name", "")
token = ident.get("publickeytoken", "").lower()
if not name or not token or "/" in name or "\\" in name:
return None
version = clean_version(ident.get("version", "0.0.0.0"))
culture = ident.get("culture", "neutral")
culture = "" if culture.lower() == "neutral" else culture
arch = ident.get("processorarchitecture", "").lower()
gac = {"x86": "GAC_32", "amd64": "GAC_64", "x64": "GAC_64", "msil": "GAC_MSIL"}.get(arch, "GAC_MSIL" if arch else "GAC")
if clr_version(assembly_file).startswith("v4"):
return "drive_c/windows/Microsoft.NET/assembly/%s/%s/v4.0_%s_%s_%s" % (gac, name, version, culture, token)
return "drive_c/windows/assembly/%s/%s/%s_%s_%s" % (gac, name, version, culture, token)
@staticmethod
def sxs_names(ident):
"""Wine's winsxs names for a side-by-side assembly: its folder and, for a policy, the policy folder."""
arch = ident.get("processorarchitecture", "x86").lower() or "x86"
name = ident.get("name", "").lower()
token = ident.get("publickeytoken", "").lower()
version = ident.get("version", "")
if not name or not token or not version or "/" in name or "\\" in name:
return None
if ident.get("type", "").lower() == "win32-policy":
target = re.sub(r"^policy\.\d+\.\d+\.", "", name)
return "policy", "drive_c/windows/winsxs/policies/%s_%s_%s_none_deadbeef" % (arch, target, token), version
return "assembly", "%s_%s_%s_%s_none_deadbeef" % (arch, name, token, version), version
def unpack(self, work):
"""Every cabinet in the package, unpacked; returns File key -> unpacked file."""
found = {}
for index, row in enumerate(self.pkg.table("Media")):
cabinet = row.get("Cabinet", "")
if not cabinet:
continue
if not cabinet.startswith("#"):
self.notes.append("external cabinet %s is not part of the package; its files are missing" % cabinet)
continue
progress("unpacking %s" % cabinet[1:])
cab = Path(work) / ("media%d.cab" % index)
self.pkg.stream(cabinet[1:], cab)
dest = Path(work) / ("media%d" % index)
dest.mkdir()
result = subprocess.run([CABEXTRACT, "-q", "-d", str(dest), str(cab)], capture_output=True, timeout=1800,
env=tool_env())
if result.returncode != 0:
raise MsiError("cabextract: %s" % result.stderr.decode("utf-8", "replace").strip()[:300])
cab.unlink()
for entry in dest.rglob("*"):
if entry.is_file():
found.setdefault(entry.relative_to(dest).as_posix(), entry)
return found
def put(self, source, rel):
target = self.out / rel
target.parent.mkdir(parents=True, exist_ok=True)
shutil.copyfile(source, target)
def install_files(self, unpacked):
assemblies = self.assemblies()
files = self.pkg.table("File")
missing = 0
destinations = {}
# Where each file goes: its component's folder, or the assembly store for a global assembly.
for row in files:
comp = row["Component_"]
if comp not in self.included_components:
continue
name = long_name(row["FileName"])
if not name or "/" in name or "\\" in name or name in (".", ".."):
continue
if comp in assemblies:
kind, ident, manifest = assemblies[comp]
if kind == 0:
source = unpacked.get(self.components[comp].get("KeyPath", ""))
folder = self.gac_folder(ident, source) if source else None
destinations[row["File"]] = folder + "/" + name if folder else None
else:
names = self.sxs_names(ident)
if names is None:
destinations[row["File"]] = None
elif names[0] == "policy":
destinations[row["File"]] = (names[1] + "/" + names[2] + ".policy") if row["File"] == manifest else None
elif row["File"] == manifest:
destinations[row["File"]] = "drive_c/windows/winsxs/manifests/%s.manifest" % names[1]
elif name.lower().endswith(".cat"):
destinations[row["File"]] = None
else:
destinations[row["File"]] = "drive_c/windows/winsxs/%s/%s" % (names[1], name)
continue
folder = self.directory(self.components[comp]["Directory_"])
destinations[row["File"]] = folder + "/" + name if folder else None
for key, rel in destinations.items():
if rel:
self.files[key] = rel
assembly_folders = set()
for key, rel in destinations.items():
if not rel:
continue
source = unpacked.get(key)
if source is None:
missing += 1
continue
self.put(source, rel)
self.counts["files"] += 1
if "/assembly/" in rel or "/winsxs/" in rel:
assembly_folders.add(rel.rsplit("/", 1)[0])
self.counts["assemblies"] = len(assembly_folders)
for row in self.pkg.table("DuplicateFile"):
if row["Component_"] not in self.included_components or row["File_"] not in self.files:
continue
folder = self.directory(row["DestFolder"]) if row.get("DestFolder") else self.files[row["File_"]].rsplit("/", 1)[0]
name = long_name(row.get("DestName", "")) or self.files[row["File_"]].rsplit("/", 1)[1]
source = self.out / self.files[row["File_"]]
if folder and source.is_file() and "/" not in name:
self.put(source, folder + "/" + name)
self.counts["files"] += 1
if missing:
self.notes.append("%d file(s) were not in the package's cabinets" % missing)
def value(self, hive, key, name, kind, data, wow64):
# A 32-bit component writes through the 32-bit view, as Wine's msi does: HKLM\Software is
# Software\Wow6432Node, and the prefix's link keys there (Classes, TypeLib, Fonts...) lead
# to the keys both views share - droiddeck-wincomponents follows them when it writes.
lower = key.lower()
if hive == "HKLM" and wow64 and lower.startswith("software\\") and not lower.startswith("software\\wow6432node\\"):
key = "Software\\Wow6432Node\\" + key[len("software\\"):]
self.registry.append({"hive": hive, "key": key, "name": name, "type": kind, "data": data})
def registry_data(self, raw):
"""A Registry table Value: (type, data) as droiddeck-wincomponents writes them."""
if raw is None or raw == "":
return "sz", ""
if raw.startswith("#x") or raw.startswith("#X"):
hexdigits = re.sub(r"[^0-9A-Fa-f]", "", self.format(raw[2:]))
if len(hexdigits) % 2:
hexdigits = "0" + hexdigits
return "binary", hexdigits.lower()
if raw.startswith("#%"):
return "expand_sz", self.format(raw[2:])
if raw.startswith("##"):
return "sz", self.format(raw[1:])
if raw.startswith("#"):
number = self.format(raw[1:])
if re.fullmatch(r"[+-]?\d+", number.strip()):
return "dword", int(number) & 0xFFFFFFFF
return "sz", number
text = self.format(raw)
if "\0" in text:
parts = [part for part in text.split("\0") if part]
return "multi_sz", parts
return "sz", text
def install_registry(self):
for row in self.pkg.table("Registry"):
comp = row["Component_"]
if comp not in self.included_components:
continue
root = to_int(row["Root"], 2)
hive, prefix = {-1: ("HKLM", ""), 0: ("HKLM", "Software\\Classes\\"), 1: ("HKCU", ""),
2: ("HKLM", "")}.get(root, (None, None))
if hive is None:
continue
key = prefix + self.format(row["Key"]).strip("\\")
wow64 = not (to_int(self.components[comp].get("Attributes"), 0) & COMPONENT_64BIT)
name = row.get("Name", "")
raw = row.get("Value", "")
if name in ("+", "-", "*") and not raw:
self.value(hive, key, None, "key", None, wow64)
continue
if not name and not raw:
self.value(hive, key, None, "key", None, wow64)
continue
kind, data = self.registry_data(raw)
self.value(hive, key, self.format(name) or "", kind, data, wow64)
self.counts["registry"] += 1
for row in self.pkg.table("Environment"):
if row["Component_"] not in self.included_components:
continue
flags = re.match(r"^[=+\-!*]*", row["Name"]).group(0)
name = row["Name"][len(flags):]
if "!" in flags or not name:
continue
raw = row.get("Value", "") or ""
hive, key = ("HKLM", "System\\CurrentControlSet\\Control\\Session Manager\\Environment") if "*" in flags \
else ("HKCU", "Environment")
mode = "set"
if raw.startswith("[~]"):
mode, raw = "append", raw[3:]
elif raw.endswith("[~]"):
mode, raw = "prepend", raw[:-3]
separator = ";"
if mode == "append" and raw[:1] in ";:":
separator, raw = raw[0], raw[1:]
elif mode == "prepend" and raw[-1:] in ";:":
separator, raw = raw[-1], raw[:-1]
value = self.format(raw)
if value:
self.registry.append({"hive": hive, "key": key, "name": name, "type": mode, "data": value, "separator": separator})
self.counts["registry"] += 1
for row in self.pkg.table("Font"):
rel = self.files.get(row["File_"])
if not rel:
continue
title = row.get("FontTitle") or Path(rel).stem + " (TrueType)"
self.value("HKLM", "Software\\Microsoft\\Windows NT\\CurrentVersion\\Fonts", title, "sz",
rel.rsplit("/", 1)[1], False)
self.counts["registry"] += 1
def check(self):
for row in self.pkg.table("LaunchCondition"):
if not self.conditions.evaluate(row["Condition"], default=True):
self.notes.append("the package says: " + self.format(row.get("Description", "")).strip())
for name in UNSUPPORTED_TABLES:
count = len(self.pkg.table(name)) if name in self.pkg.table_names else 0
if count:
what = "self-registering DLL(s) not registered" if name == "SelfReg" else "%s entries not written" % name
self.notes.append("%d %s" % (count, what))
def install(self):
progress("reading %s" % (self.pkg.summary.get("subject") or Path(self.pkg.path).name))
self.run_set_actions()
self.select()
with tempfile.TemporaryDirectory(dir=str(self.out.parent), prefix=".msi-") as work:
unpacked = self.unpack(work)
progress("placing files")
self.install_files(unpacked)
self.install_registry()
self.check()
if not self.counts["files"] and not self.registry:
raise MsiError("the package installs nothing here")
# A component made of several packages (32- and 64-bit) gets them all in one registry.json.
values = []
try:
with open(self.out / "registry.json", encoding="utf-8") as source:
values = json.load(source).get("values", [])
except (FileNotFoundError, ValueError, AttributeError):
pass
with open(self.out / "registry.json", "w", encoding="utf-8") as out:
json.dump({"version": 1, "values": values + self.registry}, out, indent=1)
return {"product": self.props.get("ProductName", ""), "version": self.props.get("ProductVersion", ""),
"counts": self.counts, "notes": self.notes}
def main():
if len(sys.argv) != 3:
print("usage: droiddeck-msi-install <package.msi> <component dir>", file=sys.stderr)
return 2
out = Path(sys.argv[2])
out.mkdir(parents=True, exist_ok=True)
try:
result = Installer(sys.argv[1], out).install()
except MsiError as error:
print("error " + str(error), flush=True)
return 1
print("result " + json.dumps(result), flush=True)
return 0
if __name__ == "__main__":
sys.exit(main())
@@ -8,9 +8,14 @@ the components it bundles ("requires"). The ids are Bannerlator's catalog names
oalinst_dll...). The user picks components per game; the picks are in SELECTION as
{"version": 1, "games": {"<appid>": ["oalinst_dll", ...]}}.
A component installed from a Windows Installer package (droiddeck-msi-install) also has drive_c/,
its files at their Windows paths (Program Files, the .NET GAC, winsxs...), and registry.json, the
values its package writes.
droiddeck-game-env calls apply() before every game launch. It copies what is missing or changed
into compatdata/<appid>/pfx and returns the overrides for WINEDLLOVERRIDES, so nothing is written
to the prefix's registry and Wine never has to be running.
into compatdata/<appid>/pfx, writes the components' registry values into the prefix's registry
files (system.reg, user.reg) and returns the overrides for WINEDLLOVERRIDES, so Wine never has to
be running. A prefix Proton has not made yet gets the registry values on the launch after.
Proton leaves the files alone afterwards: its builtin refresh (update_builtin_libs) replaces only
files that are its own builtins, and a new prefix is built around files already there (copy_pfx
@@ -22,6 +27,7 @@ import os
import re
import shutil
import sys
import time
from pathlib import Path
STORE = Path("/opt/droiddeck/wincomponents")
@@ -34,6 +40,10 @@ BACKUP = ".droiddeck-wincomponents-backup"
BUILTIN_MARKS = (b"Wine builtin DLL", b"Wine placeholder DLL")
ARCH_DIRS = ("system32", "syswow64")
ID = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,63}")
# The registry file of each hive in a prefix, and the kinds of value registry.json holds: "key"
# makes a key, "append"/"prepend" add to a list such as PATH unless it is already there.
HIVES = {"HKLM": "system.reg", "HKCU": "user.reg"}
VALUE_TYPES = {"key", "sz", "expand_sz", "multi_sz", "dword", "binary", "append", "prepend"}
def valid_id(value):
@@ -71,8 +81,28 @@ def component(store, cid):
if entry.is_file() and not entry.is_symlink():
rel = entry.relative_to(folder).as_posix().lower()
files.append(("drive_c/windows/%s/%s" % (arch, rel), entry))
drive = root / "drive_c"
if drive.is_dir():
for entry in sorted(drive.rglob("*")):
if entry.is_file() and not entry.is_symlink():
files.append(("drive_c/" + entry.relative_to(drive).as_posix(), entry))
requires = [r for r in meta.get("requires", []) if valid_id(r)]
return [str(o) for o in meta.get("overrides", [])], files, requires
return [str(o) for o in meta.get("overrides", [])], files, requires, registry_values(root)
def registry_values(root):
"""The component's registry values (registry.json), the ones that are well formed."""
try:
with open(Path(root) / "registry.json", encoding="utf-8") as source:
values = json.load(source).get("values", [])
except (FileNotFoundError, ValueError, AttributeError):
return []
good = []
for value in values if isinstance(values, list) else []:
if isinstance(value, dict) and value.get("hive") in HIVES and isinstance(value.get("key"), str) \
and value.get("type") in VALUE_TYPES and value["key"].strip("\\") and "\0" not in value["key"]:
good.append(value)
return good
def expand(store, ids):
@@ -93,13 +123,33 @@ def expand(store, ids):
return out
def spelled(path):
"""path with each folder that exists spelled as the prefix spells it (Windows names ignore case)."""
parts = path.parts
current = Path(parts[0])
for part in parts[1:]:
candidate = current / part
if not candidate.exists():
try:
with os.scandir(current) as entries:
for entry in entries:
if entry.name.lower() == part.lower() and entry.is_dir():
candidate = current / entry.name
break
except OSError:
pass
current = candidate
return current
def existing(path):
"""The file at path as the prefix spells it: Windows names ignore case, the filesystem may not."""
folder = path.parent
folder = spelled(path.parent)
path = folder / path.name
try:
with os.scandir(folder) as entries:
for entry in entries:
if entry.name.lower() == path.name:
if entry.name.lower() == path.name.lower():
return folder / entry.name
except OSError:
pass
@@ -130,6 +180,7 @@ def builtin(path):
def place(src, target, backup=None):
"""Copies src over target unless it is already there; a file of someone else's goes to backup first."""
target = spelled(target.parent) / target.name
target.parent.mkdir(parents=True, exist_ok=True)
found = existing(target)
if found is not None and same(src, found):
@@ -145,6 +196,312 @@ def place(src, target, backup=None):
return True
# ---------------------------------------------------------------- the prefix's registry files
def reg_escape(text, extra=""):
"""A string as Wine's registry files write it."""
out = []
for ch in text:
if ch == "\\" or ch == '"' or ch in extra:
out.append("\\" + ch)
elif ch == "\n":
out.append("\\n")
elif ch == "\r":
out.append("\\r")
elif ch == "\t":
out.append("\\t")
elif ch == "\0":
out.append("\\0")
elif ch < " " or ch > "~":
out.append("\\x%04x" % ord(ch))
else:
out.append(ch)
return "".join(out)
def reg_unescape(text):
out, i = [], 0
while i < len(text):
ch = text[i]
if ch != "\\" or i + 1 >= len(text):
out.append(ch)
i += 1
continue
nxt = text[i + 1]
if nxt == "x":
match = re.match(r"[0-9a-fA-F]{1,4}", text[i + 2:])
if match:
out.append(chr(int(match.group(0), 16)))
i += 2 + len(match.group(0))
continue
out.append({"n": "\n", "r": "\r", "t": "\t", "0": "\0"}.get(nxt, nxt))
i += 2
return "".join(out)
def reg_key_line(key):
return "[%s]" % "\\\\".join(reg_escape(part, "[]") for part in key.strip("\\").split("\\"))
def reg_value_line(name, kind, data):
"""name=data as one registry file line; None for a kind that has no line of its own."""
head = "@" if name == "" else '"%s"' % reg_escape(name)
if kind == "sz":
return '%s="%s"' % (head, reg_escape(str(data)))
if kind == "expand_sz":
return '%s=str(2):"%s"' % (head, reg_escape(str(data)))
if kind == "multi_sz":
parts = [str(part) for part in data] if isinstance(data, list) else [str(data)]
return '%s=str(7):"%s"' % (head, reg_escape("".join(part + "\0" for part in parts)))
if kind == "dword":
return "%s=dword:%08x" % (head, int(data) & 0xFFFFFFFF)
if kind == "binary":
digits = re.sub(r"[^0-9a-f]", "", str(data).lower())
return "%s=hex:%s" % (head, ",".join(digits[i:i + 2] for i in range(0, len(digits) - 1, 2)))
return None
def reg_string(line):
"""The text of a string value line (plain or str(2)), or None for any other kind."""
match = re.match(r'^(?:@|"(?:[^"\\]|\\.)*")=(?:str\(2\):)?"((?:[^"\\]|\\.)*)"\s*$', line)
return reg_unescape(match.group(1)) if match else None
class RegistryFile:
"""One of a prefix's registry files, as Wine keeps them: [key] sections of name=value lines."""
def __init__(self, path):
self.path = Path(path)
with open(self.path, encoding="utf-8", errors="surrogateescape", newline="") as source:
self.lines = source.read().split("\n")
self.changed = False
self.index = {}
self.links = {}
root = ""
for line in self.lines[:4]:
match = re.match(r"^;; All keys relative to (.*)$", line)
if match:
root = reg_unescape(match.group(1)).strip("\\").lower()
current = None
for number, line in enumerate(self.lines):
if line.startswith("["):
key = self.parse_key(line)
current = key
if key is not None:
self.index.setdefault(key.lower(), number)
elif line == "#link" and current is not None:
target = self.link_target(number, root)
if target:
self.links[current.lower()] = target
def link_target(self, number, root):
"""Where the link key whose "#link" line is at number leads, relative to this file's root."""
for at in range(number + 1, min(number + 8, len(self.lines))):
line = self.lines[at]
if line.startswith("["):
return None
if line.startswith('"SymbolicLinkValue"=hex(6):'):
data = line.split(":", 1)[1]
while data.endswith("\\") and at + 1 < len(self.lines):
at += 1
data = data[:-1] + self.lines[at].strip()
try:
raw = bytes(int(part, 16) for part in data.split(",") if part.strip())
target = raw.decode("utf-16-le").strip("\\")
except ValueError:
return None
if root and target.lower().startswith(root + "\\"):
return target[len(root) + 1:]
return None
return None
def resolve(self, key):
"""The key Wine really keeps key in: a path through a link key leads to the link's target."""
key = key.strip("\\")
for _ in range(8):
lower = key.lower()
for link, target in self.links.items():
if lower == link or lower.startswith(link + "\\"):
key = target + key[len(link):]
break
else:
return key
return key
@staticmethod
def parse_key(line):
end, i = None, 1
while i < len(line):
if line[i] == "\\":
i += 2
continue
if line[i] == "]":
end = i
break
i += 1
return None if end is None else reg_unescape(line[1:end])
def section(self, key):
"""(start, end) line numbers of the key's values, or None."""
start = self.index.get(self.resolve(key).lower())
if start is None:
return None
end = start + 1
while end < len(self.lines) and not self.lines[end].startswith("["):
end += 1
return start + 1, end
@staticmethod
def value_name(line):
if line.startswith("@="):
return ""
match = re.match(r'^"((?:[^"\\]|\\.)*)"=', line)
return reg_unescape(match.group(1)).lower() if match else None
def find(self, key, name):
"""(first, last + 1) of the value's lines, or None."""
span = self.section(key)
if span is None:
return None
wanted = name.lower()
number = span[0]
while number < span[1]:
line = self.lines[number]
last = number
while self.lines[last].endswith("\\") and last + 1 < span[1]:
last += 1
if self.value_name(line) == wanted:
return number, last + 1
number = last + 1
return None
def get(self, key, name):
found = self.find(key, name)
return None if found is None else "\n".join(self.lines[found[0]:found[1]])
def ensure_key(self, key):
key = self.resolve(key)
if key.lower() in self.index:
return
now = time.time()
while self.lines and self.lines[-1] == "":
self.lines.pop()
self.lines += ["", "%s %d" % (reg_key_line(key), int(now)),
"#time=%x" % (int(now * 10000000) + 116444736000000000)]
self.index[key.lower()] = len(self.lines) - 2
self.lines.append("")
self.changed = True
def put(self, key, name, raw):
"""Sets the value's line(s) to raw, or removes the value when raw is None; True if that changed it."""
found = self.find(key, name)
if raw is None:
if found is None:
return False
del self.lines[found[0]:found[1]]
self.shift(found[0], found[0] - found[1])
self.changed = True
return True
new = raw.split("\n")
if found is not None:
if self.lines[found[0]:found[1]] == new:
return False
self.lines[found[0]:found[1]] = new
self.shift(found[0], len(new) - (found[1] - found[0]))
else:
self.ensure_key(key)
start, end = self.section(key)
while end > start and self.lines[end - 1] == "":
end -= 1
self.lines[end:end] = new
self.shift(end, len(new))
self.changed = True
return True
def shift(self, at, by):
if by:
for key, number in self.index.items():
if number >= at:
self.index[key] = number + by
def save(self):
if not self.changed:
return
temp = self.path.with_name("." + self.path.name + ".droiddeck")
with open(temp, "w", encoding="utf-8", errors="surrogateescape", newline="") as out:
out.write("\n".join(self.lines))
shutil.copymode(self.path, temp)
os.replace(temp, self.path)
self.changed = False
def list_line(name, current, item, separator, add, at_end=True):
"""current (a string value line) with item added to or taken out of its list, None if it had
nothing else; a value that is not a string is left as it is."""
text = reg_string(current) if current is not None else ""
if text is None:
return current
items = [part for part in text.split(separator) if part and part.lower() != item.lower()]
if add:
items = items + [item] if at_end else [item] + items
if not items:
return None
expand = current is None or "=str(2):" in current
return reg_value_line(name, "expand_sz" if expand else "sz", separator.join(items))
def apply_registry(pfx, values, state):
"""Writes the picked components' registry values into the prefix and takes out the ones of
components turned off: a value goes back to what it was before, an entry added to a list such
as PATH just leaves the list. Returns (values written, notes)."""
files = {}
for hive, name in HIVES.items():
if (pfx / name).is_file():
files[hive] = RegistryFile(pfx / name)
old = state.get("registry") if isinstance(state.get("registry"), dict) else {}
keep, notes, written, waiting = {}, [], 0, 0
for cid, value in values:
reg = files.get(value["hive"])
if reg is None:
waiting += 1
continue
if value["type"] == "key":
reg.ensure_key(value["key"])
continue
name = value.get("name") or ""
ident = "%s\t%s\t%s" % (value["hive"], value["key"].lower(), name.lower())
current = reg.get(value["key"], name)
record = keep.get(ident) or (old.get(ident) if isinstance(old.get(ident), dict) else None) or {"prev": current}
if value["type"] in ("append", "prepend"):
line = list_line(name, current, str(value.get("data", "")), value.get("separator") or ";", True,
value["type"] == "append")
record = dict(record, item=str(value.get("data", "")), separator=value.get("separator") or ";")
else:
line = reg_value_line(name, value["type"], value.get("data"))
if line != current and reg.put(value["key"], name, line):
written += 1
keep[ident] = dict(record, cid=cid, hive=value["hive"], key=value["key"], name=name)
for ident, record in old.items():
if ident in keep or not isinstance(record, dict) or not isinstance(record.get("key"), str):
continue
reg = files.get(record.get("hive"))
if reg is None:
keep[ident] = record
continue
name = record.get("name") or ""
if "item" in record:
line = list_line(name, reg.get(record["key"], name), record["item"], record.get("separator") or ";", False)
else:
line = record.get("prev")
if reg.put(record["key"], name, line):
written += 1
for reg in files.values():
reg.save()
state["registry"] = keep
if waiting:
notes.append("%d registry value(s) wait for the prefix: they go in on the next launch" % waiting)
return written, notes
def load_state(compat):
try:
with open(compat / STATE, encoding="utf-8") as source:
@@ -167,19 +524,21 @@ def apply(compat_path, app_id, selection=SELECTION, store=STORE):
pfx = compat / "pfx"
ids = wanted(app_id, selection)
state = load_state(compat)
if not ids and not state["files"]:
if not ids and not state["files"] and not state.get("registry"):
return [], []
notes, overrides, keep = [], [], {}
copied = 0
installed = []
values = []
for cid in expand(store, ids):
found = component(store, cid)
if found is None:
notes.append("%s is not installed; skipped" % cid)
continue
installed.append(cid)
names, files, _ = found
names, files, _, registry = found
overrides += names
values += [(cid, value) for value in registry]
for rel, src in files:
# A file this prefix already has from us is ours to replace; anything else is kept.
if place(src, pfx / rel, None if rel in state["files"] else compat / BACKUP / rel):
@@ -206,10 +565,16 @@ def apply(compat_path, app_id, selection=SELECTION, store=STORE):
except FileNotFoundError:
pass
compat.mkdir(parents=True, exist_ok=True)
save_state(compat, {"files": keep, "components": installed})
if copied or removed or restored:
new_state = {"files": keep, "components": installed, "registry": state.get("registry", {})}
written = 0
if values or new_state["registry"]:
written, registry_notes = apply_registry(pfx, values, new_state)
notes += registry_notes
save_state(compat, new_state)
if copied or removed or restored or written:
note = "components %s: %d file(s) placed, %d removed" % (",".join(ids) or "none", copied, removed)
notes.append(note + (", %d put back as they were" % restored if restored else ""))
note += ", %d put back as they were" % restored if restored else ""
notes.append(note + (", %d registry value(s) written" % written if written else ""))
return overrides, notes