docs(portable): Phase 6 verification and findings; harness collection-add

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

crates/coa-core/examples/portable_smoke.rs

@@ -23,6 +23,7 @@
 //!   dump <character-id>                         the canonical JSON of the current revision (what a core import job carries)
 //!   collection-states <out.json>                (owner store)  the canonical account collections as messages
 //!   collection-apply <states.json> <account> <server-id>   write the canonical collections to the realm account (known ids only, INSERT IGNORE)
+//!   collection-add <kind> <id>...               (owner store)  add ids to a profile collection by hand (a test aid)
 //!   collection-show [<account>]                 the Owner's collections (revision, count, hash) and, with an account, the realm's
 //!
 //!   --collection-interval <seconds>   how often host-run looks at the account collections (default 300; a session start and the final
@@ -213,6 +214,13 @@ fn run() -> Result<(), String> {
                 println!("{}: {:?}", state.kind, applied);
             }
         }
+        ("collection-add", [kind, ids @ ..]) => {
+            let profile = store.default_profile().map_err(|e| e.to_string())?;
+            let ids = ids.iter().map(|i| i.parse::<u32>().map_err(|_| format!("{i:?} is not an id"))).collect::<Result<Vec<_>, _>>()?;
+            let set = coa_core::portable::collection::IdSet::from_ids(ids).map_err(|e| e.to_string())?;
+            let merge = store.merge_collection(profile, kind, &set).map_err(|e| e.to_string())?;
+            println!("{kind}: {} id(s) added{}", merge.added, merge.info.map(|i| format!(", revision {}, {} ids", i.revision, i.count)).unwrap_or_default());
+        }
         ("collection-show", rest) => {
             let profile = store.default_profile().map_err(|e| e.to_string())?;
             for kind in coa_core::portable::session::protocol::COLLECTION_KINDS {

docs/PORTABLE_APPEARANCE.md

@@ -77,3 +77,141 @@ The table registry (`realm/registry.rs`) gets a new class `Appearance` for the t
    both of which already require the character to be offline.
 5. Woodworking appearances (synthesised ids) are not in the DBC: the Manager treats them as unknown to a destination, so
    they stay canonical and are not applied. This is a documented limitation (they are crafted-item cosmetics).
+
+## 5. The selected appearance in the character (`PortableCharacter::wardrobe`)
+
+```text
+wardrobe: { active: { "<category>": <appearance id> }, can_see_item, can_see_spell, outfits: { "<name>": [<appearance id>...] } }
+```
+
+* Absent when empty (`skip_serializing_if`), so every snapshot made before Phase 6 keeps exactly its bytes and its hash; there
+  is no format-version bump (`format_version` stays 1: the field is optional and `deny_unknown_fields` still refuses anything
+  else).
+* Ids are plain numbers in the character's `content_namespace` (`coa`), like spells and quests. The namespaced forms
+  `coa:appearance:<id>` / `coa:item:<id>` are used where an id leaves the character: the collection kinds.
+* **Export** (`script.rs`, `export.rs`): three fixed `SELECT`s, only where the three tables exist. Rows the realm itself would
+  ignore (a category outside 1..68, an appearance of 0) are left out, so the model is what the realm shows.
+* **Merge** (`merge.rs`): the same keyed-map / scalar rules as `settings`. In a session `merge3(C0, B0, B1)` only the
+  `B0 -> B1` choices are progress; what the realm never showed stays in the canonical character through any number of
+  sessions. In an update `merge3(realm now, synced, canonical)` reports a conflict only where both sides changed the same
+  category differently.
+* **Import / update** (`wardrobe.rs`, `plan.rs`, `update.rs`): written only with the knowledge of the destination
+  (section 4.1). `writable()` is the part of the wanted state the realm can hold plus whatever it **already shows** (an id the
+  realm put there itself is never deleted because the Manager's table does not list it). An outfit that mentions an unknown id
+  is held back whole. The update replaces the three tables of the character as a whole when what the realm can hold differs
+  from what it shows, in the same transaction as the rest of the update, counted in `keyed_rows`. No `item_instance` or
+  `character_inventory` statement is ever produced by the appearance code (a test asserts the item counts of the script).
+* **Online import** (`online.rs`): `job_model()` carries the `wardrobe` restricted to what the destination knows (none without
+  knowledge). The core's job parser accepts the section with the same bounds as the model (section 8).
+* **Registry**: `character_appearance`, `character_appearance_outfit`, `character_appearance_settings` are the new table class
+  `Appearance` (optional-module tables: they are not required to exist, unlike the `Portable` ones).
+
+## 6. Account collections (`coa:appearance`, `coa:vanity`)
+
+They belong to the Owner's profile, not to a character: `collection(profile, kind) = revision + compact sorted id set + hash`
+(the Phase 1 design, unchanged: delta + LEB128 varint, about one byte per id for a dense wardrobe). `canonical = canonical U
+incoming`; a realm that lacks an unlock never removes it.
+
+```text
+Host                                              Owner
+ fingerprint(account, kind)  ----(cheap)---->     (nothing)
+   changed?  read ids, hash
+   hash != last acknowledged?
+ CollectionObserved{server, kind, set} -------->  union into the profile collection
+                                       <--------  CollectionAck{outcome, revision, hash, canonical?}
+ canonical present (it has ids the realm lacked)?
+   apply the known ones to the account (INSERT IGNORE)
+```
+
+* **Change detection** is a fingerprint of the realm rows (count, max, sum, checksum): one aggregate query. The set is read and
+  sent only when it differs from the last one the Owner acknowledged; an unchanged account costs one query per kind every
+  `collection_interval_secs` (default 300) and nothing else. A session start and a final checkpoint always look. The first look
+  of an account reports even an empty set: that is how the Owner learns of the realm and answers with what it holds.
+* **Never in a checkpoint.** A `PortableCheckpoint` carries the character only (a test asserts that no checkpoint message
+  mentions a collection). The 60-second cadence never touches the collections.
+* **Transport-neutral.** `CollectionObserved`, `CollectionState`, `CollectionAck` are canonical JSON with
+  `protocol_version = 1`, `deny_unknown_fields`, a size cap checked before parsing (`MAX_COLLECTION_MESSAGE_BYTES`), the set as
+  base64 of the compact encoding, its hash and its count. The Owner re-derives hash and count from the decoded set and
+  answers `Rejected` (changing nothing) on any mismatch, an unknown kind, or garbage; a malformed message is an error and
+  changes nothing.
+* **Idempotent and restart-safe.** A lost acknowledgement re-sends the same set; the union adds nothing and no second revision
+  is made. The Host keeps the pending message and the last acknowledged hash in `host_collection` (schema 6), so a restart
+  neither loses a message nor reads or sends what was acknowledged.
+* **Owner / Host separation.** `OwnerService` is the only code that touches the canonical collection; the Host knows its own
+  `host_collection` rows and the realm. The Host never opens the Owner's SQLite file.
+* **Writing to a realm** (`realm/collections.rs`): the ids the destination knows and does not hold yet, `INSERT IGNORE`, in
+  chunks of 2000 rows in one transaction, guarded by "the account exists". It is safe on a running realm for the same reason
+  the core's own writes are: the core loads an account's collection once at login and never deletes or updates a row. The bank
+  vanity items are never read nor written. `source_item` is written as 0 (the core never reads it back).
+* **A realm that learns more later** (a newer client data directory) is brought up to date by applying the canonical state
+  again; only the new known ids are written.
+
+## 7. Where the destination's knowledge comes from
+
+`ImportOptions::knowledge` is an `Arc<RealmKnowledge>` read from `<Data>/dbc/Appearances.dbc` and `VanityCollection.dbc` of the
+destination realm (`RealmKnowledge::from_data_dir`, a strict WDBC reader with a size/header/record check). Without it the Manager
+writes **no** appearance and **no** collection to the realm and reports `wardrobe` as not applied: everything stays canonical.
+The harness takes it as `--data-dir`.
+
+## 8. Core changes (`feat/portable-session-bridge`)
+
+Only the online import job needed the core: the job format gained the optional `wardrobe` object (same shape as the model),
+parsed by the strict `CoAPortableJson` reader with the model's bounds (68 categories, 100 outfits, 64-byte names, 69 ids per
+outfit), written with three new prepared statements (`CHAR_INS_PORTABLE_APPEARANCE`, `_APPEARANCE_SETTINGS`,
+`_APPEARANCE_OUTFIT`, `CONNECTION_BOTH`) inside the existing import transaction; the result file reports the number of rows as
+`wardrobe`. Nothing of the session gate, the checkpoint or the release operations was touched. The collections need no core
+change: they are written by the Manager as `INSERT IGNORE`.
+
+## 9. Verification (Phase 6 gate)
+
+| What | Result |
+|---|---|
+| `cargo test --workspace --locked` | 436 passed (413 before the phase), 39 live tests ignored without a database |
+| Live tests, two real MySQL servers (A, B) | 39 passed: the whole earlier suite (now with a wardrobe on the fixture character 1002) plus 5 new ones in `live_wardrobe.rs` |
+| Real worldserver (new core build), 3 tests | passed: automatic `B0` and checkpoints, the gate, and the **online import including the wardrobe** (read back equal to what the offline importer writes) |
+| `node tools/check-i18n.mjs`, `npm run build` | green |
+| Large collections, simulated realm | 10 000 and 50 000 scattered ids: compact message under 4 bytes per id (about 1 per id when dense), unchanged collection costs one fingerprint and **zero** reads, empty-realm arrival gets the whole set once |
+| Large collections, real MySQL | 10 000 ids: write 0.36 s, fingerprint 0.14 s, full read 0.08 s; 50 000 ids: write 0.75 s, fingerprint 0.08 s, full read 0.10 s, second apply writes 0 rows, no duplicates, compact set 50 KB |
+
+What the tests prove, by property:
+
+* a checkpoint never carries a collection; a 60-second cadence never reads one;
+* union only: a realm with fewer unlocks never removes one, a realm's own row is never removed by an apply;
+* an id the destination does not know is **not written** and stays canonical (selection, outfit, collection), through sessions and
+  updates; a destination that knows more gets the rest on the next apply;
+* reconciling an appearance changes no item: item sets are identical before and after (merge, simulated session, update script,
+  and the real database round trip);
+* a lost acknowledgement, a duplicate, a restart of the Host between observe and acknowledge, hostile or oversized or
+  mismatching messages, a kind that is not carried: nothing is merged twice or at all;
+* the bank vanity items are never read from or written to a realm.
+
+### The real-client gate (disposable pair, the owner's client)
+
+Realm B "PT Guest" and realm A "PT Home" with `CoA.UnlockLocalAppearanceCatalog = 0` and `CoA.UnlockAllVanity = 0`, so unlocks
+come only from play. The Host ran against B with a 20 s checkpoint and a 30 s collection look.
+
+1. On B: `.additem` of two weapons, a helm, a vanity mount and a vanity pet; their appearances applied (head and weapon); the
+   added items then destroyed. Result in the Owner after logout: 16 appearances and 7 vanity items in the profile collections
+   (revisions 5 and 3), the selection `{1: Lionheart helm, 14: Thunderfury}` in the character, final checkpoint acknowledged.
+2. Back on A: the character in place from canonical, the collections applied by union (16 appearance rows, 7 vanity rows, no
+   duplicates, nothing lost), the one synthetic id unknown to A kept in the Owner and **not** written to A.
+3. Logged in on A with the real client: the wardrobe unlocks, the vanity unlocks, the selected transmog and the original gameplay
+   items were all there (owner's check, 2026-10-07).
+
+## 10. Limits and findings
+
+* **Realm switches decide whether there is anything to carry.** The repack ships `UnlockLocalAppearanceCatalog = 1` and
+  `UnlockAllVanity = 1`; there the collection tables hold nothing (the core keeps the unlock-all in memory) and a wardrobe sync is
+  empty by design. It is meaningful on realms that unlock progressively.
+* **The destination's client data must be known.** `ImportOptions::knowledge` is read from `Data/dbc`; the Manager integration has
+  to supply the realm's data directory. Without it nothing is written (and `wardrobe` is reported as not applied).
+* **A core without the Phase 6 job parser refuses a job that carries a wardrobe** (`wardrobe` is an unknown field to it). There
+  is no version negotiation yet; the online import needs the Phase 6 core build when the character has a wardrobe.
+* **Woodworking appearances** (synthesised ids that are not in `Appearances.dbc`) are treated as unknown by a destination: kept,
+  never applied.
+* **An id a realm learns later** is written by the next collection apply, but a selection that was held back is not retroactively
+  re-applied by an update (it is "left alone" because canonical did not change since the last sync).
+* **A realm copy that diverged without a session cannot be updated in place** (`UpdateConflicts`, by design, nothing written). In the
+  gate the old A copy had been played unmanaged since Phase 4 and had to be re-imported. A conflict-resolution policy is a later decision.
+* The collection look is polled by the Host (`collection_interval_secs`); the Host driver is still a library plus the harness
+  (`host-run`), not yet part of the Manager application.