Mirrors/DroidDeck
1
0 Fork 0
mirror of https://github.com/Droid-Deck/DroidDeck.git synced 2026-11-11 02:51:10 +00:00
Code Issues Packages Projects Releases Wiki Activity

Windows components: picked automatically by each game's source (#554)

Browse Source 
Each game's Windows components now come from the list its own source
trusts, and the ones that list names turn on by themselves. The user's own
on/off switches always win; each automatic row says why it is on.

- Steam games: the Steamworks Shared redistributables Steam installs with
  the game (appmanifest SharedDepots, else the client's appinfo.vdf).
  Steam keeps its install scripts marked as already run.
- GOG: the build manifest's dependencies. Epic: the manifest's
  prerequisite (Unreal's maps to VC++ and DirectX). Amazon: fuel.json
  PostInstall. Older GOG/Epic installs get their list fetched once with the
  saved sign-in; game files are never downloaded again.
- Custom games (the + button, Add all, migrated folders): one shared Steam
  match per game, also used for artwork. An appid from the game's files
  turns components on; a name-only match stays Recommended; no match falls
  back to the folder scan.
- Steam app info for apps the client never saw comes from an anonymous
  Steam PICS lookup (CM websocket, anonymous logon, product info), cached
  per appid; the user's Steam sign-in is never used.
- Missing components download in the background on one low-priority
  worker that pauses during sessions and waits on battery saver, heat or a
  metered network unless the user asked. A game Steam installs during a
  session is readied right away so it can be played without ending the
  session; the launch waits up to 10 s only for a component still on its
  way.
- Page: "On for this game" with reasons, a quiet "Checking what this game
  needs..." only for games never worked out, rows that ease in, pieces that
  need a download greyed out while offline. The intro names no other app.
- The folder scan skips Unity's *_Data/Managed (Mono, not .NET Framework).
This commit is contained in:
The412Banner authored and GitHub committed 2026-10-10 21:15:56 -04:00
1 parent d45933e559
commit a2e596a200
35 files changed
+3810 -146
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/development/windows-components-sources.md
+116
View File
@@ -0,0 +1,116 @@
# Windows components by source
A game's Windows components (VC++ runtimes, DirectX's d3dx9, XNA, PhysX...) are picked for it by
itself, from the list its source keeps. The source badge (Steam, GOG, Epic Games, Amazon Games,
Custom) decides whose list is trusted (`WinCompSources`):
| Source | List | On by itself |
| --- | --- | --- |
| Steam | The Steamworks Shared redistributables Steam installs with the game: the appmanifest's `SharedDepots`, else the client's app info (`SteamAppInfo`) | yes |
| GOG | The build manifest's `dependencies`, kept in the sidecar at install (`extra.dependencies`) | yes |
| Epic Games | The manifest's prerequisite (`PrereqName`, `PrereqPath`), kept in the sidecar at install | yes, for the mapped parts |
| Amazon Games | fuel.json's `PostInstall` installers | yes, for the mapped parts |
| Custom, appid in its files | Steam's list for that app (`SteamMatch` certainty `FILES`: steam_appid.txt, also under steam_settings/ or a parent; an emulator ini; an appmanifest) | yes |
| Custom, matched by name | Steam's list for that app (certainty `NAME`, a store search with the same name) | no, Recommended |
| Custom, no match | the folder scan (`DependencyDetector`) | no, Recommended |
The folder scan is offered as Recommended for every game.
## Steam's data
Never scraped from a website. `SteamAppInfo` reads the Steam client's own app-info cache in the
runtime (`Steam/appcache/appinfo.vdf`, versions 27-29): the app's depots that point at app 228980
(Steamworks Shared) and its name. Each appid's answer is kept in `files/steam-appinfo.json`, so it
works offline; an app the client has not seen is looked for again after a week. A Steam game with
its appmanifest beside it uses the manifest first. `droiddeck-seed-redists` still marks Steam's
install scripts for those redistributables as already run, so the client never runs the real
installers.
## One Steam match per game
`SteamMatch` keeps, per added game (by its folder), `{appId, certainty FILES|NAME|NONE, source}`
in `files/steam-matches.json`. The art fetch (`AddedGameArt`) and the components read and write
the same record; an earlier build's art lookup (`added-art/<appid>/steam-appid`) is taken over as
a name match. A miss is tried again after a week.
## Mapping
All in `WinCompSources`:
- GOG ids: `MSVC<year>[_x64]` → `vcredist<year>` (2017 → the 2015 family, as the folder scan),
`DirectX*`/`DX9`/`D3DX*` → `d3dx9`, `DOTNET<n>` → `dotnet35`...`dotnet48`, `PhysX` → `physx`,
`OpenAL` → `oalinst`, `XNA3`/`XNA31` → `xna31`, other `XNA*` → `xna40`, `XLive*`/`GFWL*` →
`XLiveRedist`, `XACT*` → `xact`.
- Epic: Unreal's `UE4PrereqSetup`/`UEPrereqSetup` (or a "UE4/UE5/Unreal Engine Prerequisites" name)
→ `vcredist2022` + `d3dx9`; any other installer by its path, as the folder scan names it.
- Amazon: each `PostInstall` command by its path, as the folder scan names it (Unreal's
prerequisite as above).
- Steam: Steamworks Shared depots 228981-228990 (`SteamRedists.DEPOTS`).
Anything that maps to nothing is listed under "Also listed" with its own name. A mapped component
that needs a Windows installer here, or is not in the catalog, stays Recommended with its status.
## On by itself, and the user's switches
All of it runs on one background thread (`AutoComponents`), low priority (the runtime helpers an
install starts run under `nice -n 19` and `ionice -c3`), one job at a time from one queue: a game's
list (worked out again only when the files it is read from change, by modification time, and kept
in `files/wincomp-findings.json`, so a page opens on it at once) and a component's download (each
component once, for every game that wants it). What the user asked for goes first: the page of a
game (on a metered network too), a store install or an add. A game's list is queued when it is
installed, added or edited, when its page opens, when it is launched from DroidDeck (the launch
itself only takes what is on already), and for every Steam game once per app start, ten seconds
after the launcher is up (Steam installs them without a word to DroidDeck). The queue waits while
a session runs, and a download also waits for a validated network (the page then greys what is not
downloaded yet, "Available when online", and follows the network live); what nobody asked for also waits on battery saver, thermal status moderate or
worse, or a metered network. A component turns on for its games when its download lands, for the
next launch, and an open page shows the progress and reads the picks again. One log line per
queued and per finished download.
`wincomponents.json` keeps, per game,
`auto` (each with its reason) and `user` (each switch the user pressed, on or off) beside `games`,
the effective list the launch reads (still version 1). A user switch always wins over an automatic
pick, either way, and a new automatic list never touches it. Picks saved before this build count
as the user's own.
The page shows "On for this game" with each reason ("From GOG's list for this game", "Steam
installs VC++ 2015 with <game> (app N)"), then Recommended, then "Also listed".
## Older store installs
A GOG or Epic game installed before its sidecar kept the store's list gets it once, on the worker
(`StoreListBackfill`): GOG's build manifest for the installed build (`dependencies`), Epic's
manifest (prerequisite name and path), fetched with the saved sign-in - the manifest alone, never a
game file - and kept in the sidecar as an install keeps it. Fresh installs now keep the keys even
when empty. Without a sign-in or when the store refuses: skipped quietly, tried again a week later;
offline is no try. Like any work nobody asked for it waits for an unmetered network, unless the
game's page is open.
## Apps the client never saw
A Steam app the client in the runtime never saw (not owned, never viewed: a Custom game's
steam_appid.txt) is missing from its `appinfo.vdf`. `SteamPics` asks Steam for it anonymously:
Steam's public CM list (`ISteamDirectory/GetCMListForConnect`), a WebSocket connection to a
connection manager (`wss://<cm>/cmsocket/`, the transport Steam's web clients use), an anonymous
logon (an AnonUser SteamID, no account, nothing of the user's Steam sign-in), a
`ClientPICSAccessTokenRequest` and a `ClientPICSProductInfoRequest` (at most 25 apps each, all in
one connection), the app KeyValues read for depots with `depotfromapp` 228980 and the name, and a
logoff. Kotlin only, no library (TLS from the platform, a minimal WebSocket and protobuf reader).
Answers go into `steam-appinfo.json` like the client's (a miss is asked again after a week).
Order for an appid: the client's `appinfo.vdf` first (free), then the anonymous lookup. It runs on
the components' worker: one batch for every game in the start/after-session pass (before their
lists), or the one app of a game whose page is open; never during a session, and as work nobody
asked for only on an unmetered, favourable device. Log: `steam pics: N apps, M with shared redists`.
## A game installed during a session
`SessionInstalls` looks at the Steam libraries' appmanifests every five seconds while a session
runs (a manifest re-read only when its time changes); a game whose StateFlags turn to 4 (fully
installed) becomes one job allowed to run in the session (`AutoComponents.queueInSession`): its
list (local appmanifest, appinfo.vdf, the kept PICS answer; one small anonymous lookup only for an
app nothing knows), its automatic picks written to `wincomponents.json` (the runtime's copy too) and
only its missing components downloaded, at low priority. It waits while a game runs in the session
(Steam's reaper `SteamLaunch AppId=`, or Proton's `waitforexitandrun`). All other work stays
paused. While its components download, the app lists them per game in the runtime's
`~/.config/droiddeck/wincomponents-pending.json`; `droiddeck-wincomponents` at launch waits up to
10 s for those (no wait when nothing is pending), then applies what is there: log
`wincomp: <appid> waited Nms for <ids>` or `... launched without <ids> (not ready)`.
docs/progress-log.md
+61
View File
@@ -7,6 +7,67 @@ the timeline, then lessons and backlog. Companion to the README (what the app *d
---
## 2026-10-10 - `feat/wincomp-auto-sources`: Windows components picked by the game's source
- **Per source** (`WinCompSources`): Steam - Steamworks Shared depots from the appmanifest or the
client's app info (`SteamAppInfo`, appinfo.vdf v27-29, cached per appid, weekly retry); GOG - the
build's `dependencies` (now kept in the sidecar); Epic - the manifest's prerequisite (now parsed
and kept); Amazon - fuel.json `PostInstall`; Custom - Steam's list when its files name the appid
(on by itself), when only the name matches (Recommended); else the folder scan. Unmapped entries
are listed with their own names; mapping tables in one place.
- **One Steam match per game** (`SteamMatch`, `files/steam-matches.json`), used by the art fetch
and the components alike; the earlier art lookup is taken over.
- **On by itself** (`AutoComponents`): at store install, add/edit, page open and before a launch
from DroidDeck; `wincomponents.json` keeps `auto` (with reasons) and `user` switches beside the
effective `games` list (still version 1). A user switch always wins. The page gets "On for this
game" with reasons and "Also listed".
- **Device fix:** on a Steam game (Need for Speed Payback) only the already-downloaded d3dx9 went
on; vcredist2013/2019 stayed Recommended, never downloaded (Steam games have no install event
here). Every refresh (page open, launch, an app-start sweep over Steam games, install/add/edit)
now queues the missing automatic picks for a background download and turns each on when it
lands; the open page shows the progress and reloads.
- **Gentle on the device:** one low-priority worker and one queue (games and components each once,
what the user asked for first); nothing while a session runs; the start-up sweep and pre-launch
looks only queue, wait ten seconds after start, and skip battery saver, a warm device or a
metered network; runtime helpers under nice 19 / ionice idle; a game's list kept and worked out
again only when its files change, on the worker. Tests: download lands, queue paused while a
session runs, unfavourable device, dedupe.
- **Fluid:** after a session the Games tab reads Steam's manifests first (no wait for the added
games' walk), a newly installed game eases into the list and its components' list is queued on
the worker. The components page shows "Checking what this game needs…" (small spinner) until
the list is worked out, never "nothing found" before; "On for this game" expands in, components
on their way sit there with their progress and a "Turning on N components…" line, and rows ease
in (~250 ms; instant with animations off).
- **Quieter:** "Checking…" only when the game's list was never worked out; a kept list shows at
once and is checked again silently, its changes easing in. The lists wait for the catalog and
fade in once; "Loading the components…" only after 300 ms.
- **Offline:** with no validated network, components not downloaded yet are greyed and cannot be
switched ("Available when online"), automatic picks on their way included (no progress bar);
downloaded ones stay usable. The page follows the network live, and downloads wait in the queue
until it is back.
- **Every source, older installs:** the start/after-session pass covers all games (Custom, GOG,
Epic, Amazon). GOG/Epic games installed before the sidecar kept the list get it once from the
store's build manifest with the saved sign-in (`StoreListBackfill`; no game files; weekly retry
on failure). Fresh installs keep the keys even when empty.
- **Apps the client never saw** (Dragon's Dogma 2, 2054970 not in appinfo.vdf): the client ask
gave nothing (no `SteamClient` call returns depots), so Steam is asked anonymously instead
(`SteamPics`: CM list, WebSocket CM, anonymous logon, PICS access tokens + product info, ≤25
apps a request, one connection a pass, on the worker; Kotlin, no new library). Prototyped first
against Steam from the build host: 2054970 → depot 228988 (VC++ 2019), 1190460 → one shared
depot. Cached in steam-appinfo.json like the client's, weekly retry on a miss.
- **Installed during a session:** the Steam libraries' appmanifests are looked at every 5 s while
a session runs; a game turning fully installed gets one in-session job (waits while a game runs)
that writes its automatic picks and downloads only its missing components; the launch waits up
to 10 s for components listed in `wincomponents-pending.json`, never when nothing is pending.
- **Checks:** a store list counts as read only when the manifest was read whole (`listRead`;
GOG: a manifest with depots, Epic: the prerequisite fields read); older empty lists are fetched
once more. Log: `backfill: <game> gog deps=[...]` / `epic prereq=<name|none>`, plus the
manifest's depot/file count. Unity's `*_Data/Managed` is skipped by the folder scan (Mono
assemblies, not .NET Framework).
- Docs: `docs/development/windows-components-sources.md`. Tests: `WinCompSourcesTest` (each
source's fixture, certainty rules, appinfo reading, auto picks, user overrides, shared match),
`AddedGameArtLookupTest` updated. Not yet on a device.
## 2026-10-10 - `fix/games-folder-migration`: Games folders become entries, not Steam libraries
- **Seen on a device:** Steam's Install dialog listed "Games (Internal shared storage)" and "Games