Implement Phase 12.1 Remote Portable Transfer and Phase 13 Game Relay

This commit is contained in:
Дмитрий committed 2026-10-09 08:29:07 +05:00
1 parent 1dc212968d
commit c743157b97
34 files changed
+5359 -30

No files matched your search

+35 -6
View File
@@ -98,9 +98,11 @@ Application messages: requests ≤ 4096 bytes, strict JSON with unknown fields r
| `list_characters` | `characters{characters}` | the characters of the player's account (opaque per-connection handles; the realm's own numbers never leave the Host) |
| `claim{token}` | `claimed{character_id, sha256, payload, collections}` | export one character (section 7) |
| `claim_ack{character_id}` | `done` | the player stored it; the claim is final |
| `transfer_character{character_id, revision, sha256, payload, collections}` | `transferred{character_id, local_guid, session_id}` | transactional remote character import/projection (Phase 12.1) |
| `allocate_relay` | `relay_allocated{relay_host, auth_port, world_port, token, expires_at}` | dynamic game relay allocation for NAT traversal (Phase 13) |
| `route` | `route_info{address}` | the address the game client can use right now, if the owner stated one |
Errors (`error{code,message}`): `unsupported_version`, `invalid`, `provisioning_off`, `no_account`, `not_at_character_select`, `not_yours`, `already_claimed`, `not_eligible`, `wrong_credentials`, `account_taken`, `rate_limited`, `unavailable`.
Errors (`error{code,message}`): `unsupported_version`, `invalid`, `provisioning_off`, `no_account`, `not_at_character_select`, `not_yours`, `already_claimed`, `not_eligible`, `wrong_credentials`, `account_taken`, `rate_limited`, `unavailable`, `incompatible`, `relay_unavailable`.
Backend failures are logged on the Host and answered as `unavailable` without internals.
## 6. Accounts
@@ -137,6 +139,32 @@ Then the Host exports the character with the same code as *Make portable* (one c
appearance and vanity collections, all inside the channel. The Player's Manager decodes it **with verification** (hash, format, that the character id is the one claimed), stores it as **canonical revision 1** with the collections merged into
its profile, and only then sends `claim_ack`; the Host marks the binding `PortableCharacterId ↔ PlayerIdentity` as acknowledged. An unacknowledged claim can be repeated by the same player and returns the same character; nobody else can take it.
## 7.1 Remote Portable Character Transfer (Phase 12.1)
When a player selects a portable character that exists in their Player Manager but does not yet exist on the remote realm, the Player Manager performs an automated, transactional transfer over the encrypted Noise channel:
1. **Preflight Compatibility Check**: The Player Manager checks the canonical character against the target realm's advertised capabilities (Phase 7/8). If incompatible, the transfer is refused immediately before any network mutation.
2. **Payload Transfer**: Inside the end-to-end encrypted Noise channel, the Player Manager sends `transfer_character{character_id, revision, sha256, payload, collections}`. The Coordinator sees only opaque ciphertext.
3. **Host Validation**: The Host validates:
- PlayerIdentity proof of ownership;
- `PortableCharacter` format version, schema integrity, and SHA-256 payload hash;
- Target realm account mapping for this player;
- Level projection policy (e.g. projecting level 80 down to realm cap 60 via Phase-8 oracle);
- Idempotency: if this exact canonical revision was already imported, the existing working copy GUID and armed session ID are returned immediately without duplicating the character;
- Revision update: if a newer canonical revision is transferred, the Host updates the existing working copy rather than creating a duplicate character.
4. **Outcome & Session Arming**: On successful import/projection, the Host returns `transferred{character_id, local_guid, session_id}`. The session engine arms the session for game login.
## 7.2 Game Relay Allocation (Phase 13)
When a realm is hosted behind NAT or CGNAT without direct port exposure, the Player Manager requests dynamic relay routing over the encrypted Noise channel:
1. The Player Manager sends `allocate_relay`.
2. The Host asks its connected outbound tunnel to `coa-relay` (`wss://coa-manager.duckdns.org/relay/v1/host`) for an allocation.
3. `coa-relay` binds a dedicated pair of TCP ports from the dynamic game port pool (40000–40050) for `Auth` and `World`, generates a single-use token, and responds to the Host.
4. The Host returns `relay_allocated{relay_host, auth_port, world_port, token, expires_at}` to the Player inside the encrypted control channel.
5. The Player Manager points the client's `realmlist.wtf` to `relay_host:auth_port`.
6. When the client contacts `auth_port`, `coa-relay` dynamically rewrites the `address` field in the `CMD_REALM_LIST` response (opcode `0x10`) to `relay_host:world_port`, seamlessly directing the client to the allocated world tunnel without patching AzerothCore or modifying the database.
## 8. Secrets on the player's machine
`SecretStore` (`control/secrets.rs`) keeps the identity key and the realm passwords; nothing secret is in JSON, SQLite, a log or a diagnostic (`control.sqlite` holds names only, checked by the gate).
@@ -149,11 +177,12 @@ its profile, and only then sends `claim_ack`; the Host marks the binding `Portab
## 9. JOIN
*Join this server* (`join_realm`): connect and verify the Host → `welcome` → (existing-only realm and no credentials: ask for the one-time link) → get or reuse the account → if a character was chosen: the existing Phase 7/8
compatibility check against the capabilities in the Registry record (nothing is written when it is incompatible) → the realm's route → client check → write the realmlist (the existing Phase 9 machinery with backups) and start the game.
Outcomes that are **not failures but honest states**: `needs_link`, `needs_client` (no game client set up), `incompatible` (the verdict and its notes), `needs_transfer` (a character that did not come from this realm cannot be put on it
yet), and **`needs_relay`: "This server requires Relay support, which is not available yet."** when the owner has not stated an address the game client can reach (*Settings → Public listing → Address players connect to*, a host and optional port; it is data the
owner supplies, not discovery, NAT traversal or a relay). Connectivity is never faked.
*Join this server* (`join_realm`): connect and verify the Host → `welcome` → (existing-only realm and no credentials: ask for the one-time link) → get or reuse the account → if a character was chosen:
- if character does not exist on remote realm: initiate automated Remote Portable Character Transfer (Phase 12.1) over the encrypted channel;
- if realm requires relay: automatically allocate dynamic relay routes (Phase 13) via `allocate_relay`;
- write client `realmlist.wtf` (pointing to direct host:port or relay_host:auth_port) and launch the game.
Outcomes: `ready` (route established, client configured, session armed), `needs_link` (existing-only realm credentials required), `needs_client` (no game client set up), `incompatible` (the verdict and notes if rejected). The previous `needs_transfer` and `needs_relay` blockers are now fully resolved by Phase 12.1 and Phase 13.
The player is shown the account name and can **copy the name and the password** (the password is read from the protected store only when asked, shown only on request, never logged).
+10 -7
View File
@@ -12,7 +12,7 @@ This file never contains private keys, passwords, DB credentials, transfer PINs,
Registry API Relay Coordinator PostgreSQL Caddy (ingress) monitoring
```
Phase -1 prepared the host. **Phase 10 deployed the Registry and PostgreSQL** (see [REGISTRY_DEPLOYMENT.md](REGISTRY_DEPLOYMENT.md)); **Phase 12 deployed the Coordinator**; and the **Phases 11-12 TLS gate** enabled production HTTPS/WSS ingress.
Phase -1 prepared the host. **Phase 10 deployed the Registry and PostgreSQL** (see [REGISTRY_DEPLOYMENT.md](REGISTRY_DEPLOYMENT.md)); **Phase 12 deployed the Coordinator**; the **Phases 11-12 TLS gate** enabled production HTTPS/WSS ingress; and **Phase 13 deployed the Game Relay**.
The VPS is not the source of truth for any player data.
Public domain is `coa-manager.duckdns.org` pointing to `coa-infra-01`. Caddy automatically obtains and renews trusted Let's Encrypt TLS certificates via ACME. HTTP on port 80 redirects to HTTPS on port 443.
@@ -32,13 +32,15 @@ Internet
| +--> PostgreSQL (coa-postgres:5432, network coa-db only, --internal, no published port)
|
+--> Coordinator (coa-coordinator:8081, networks coa-ingress + coa-db, no published port)
|
+--> Game Relay (coa-relay:8082, networks coa-ingress + coa-db, published 40000-40050/tcp)
```
Rules:
- PostgreSQL must never be internet-exposed: no `ports:` entry, only on the `coa-db` internal network, no UFW rule for 5432.
- Caddy is the only container allowed to publish ports (80, 443/tcp). A new inbound port needs a specific protocol and
a threat model first, then a UFW rule and a note here.
- Registry (8080), Coordinator (8081), and Relay control HTTP/WS (8082) have no published host ports; they are reverse-proxied exclusively through Caddy.
- Game Relay publishes the dedicated TCP port range `40000:40050/tcp` for relayed WoW client game streams (Auth & World).
- No admin dashboards or the Docker API on public interfaces. Caddy's admin API is disabled.
## Public ports
@@ -46,8 +48,9 @@ Rules:
| Port | Proto | Service |
|---|---|---|
| 22 | tcp | SSH, key only |
| 80 | tcp | Caddy |
| 443 | tcp | Caddy |
| 80 | tcp | Caddy HTTP (redirects to HTTPS) |
| 443 | tcp | Caddy HTTPS / WSS |
| 40000-40050 | tcp | Game Relay WoW game traffic (Auth & World) |
## Filesystem layout
@@ -57,8 +60,8 @@ Rules:
├── caddy/ compose.yaml, Caddyfile
├── postgres/ compose.yaml (running since Phase 10)
├── registry/ compose.yaml, .env, src/ (the build context of the running image)
├── relay/ (empty)
├── coordinator/ (empty)
├── coordinator/ compose.yaml, src/
├── relay/ compose.yaml, src/ (coa-relay container)
├── monitoring/ (empty)
├── backups/ 0700, logical dumps and config archives
├── logs/
+112
View File
@@ -0,0 +1,112 @@
# Game Relay Protocol (Phase 13)
The Game Relay (`services/coa-relay`) provides TCP connectivity for WoW 3.3.5 clients when the realm's Host is behind NAT or CGNAT without port forwarding.
The Relay is logically and operationally separate from the Registry and Coordinator:
- **Registry**: Public directory of realms (HTTPS).
- **Coordinator**: Control plane signaling and routing for Noise channels (WSS).
- **Game Relay**: Multiplexed byte tunnel for WoW client game traffic (`authserver` on TCP 3724 and `worldserver` on TCP 8085).
---
## 1. Security Architecture and Invariants
```text
WoW Client (3.3.5)
│
├── TCP :40000 (Auth) ──► [coa-relay] ◄── Outbound WSS ── [Host Manager] ──► 127.0.0.1:3724 (local authserver)
│ (:8082 internal)
└── TCP :40001 (World) ──► [coa-relay] ◄────────────────── [Host Manager] ──► 127.0.0.1:8085 (local worldserver)
```
1. **Host Outbound Only**: The Host initiates an outbound WebSocket connection to `wss://coa-manager.duckdns.org/relay/v1/host`. The Host requires zero inbound firewall openings or port forwarding.
2. **Strict Host Local Targets**: The Host's `RelayLink` worker strictly permits only two local targets:
- `RelayTarget::Auth` -> `127.0.0.1:3724`
- `RelayTarget::World` -> `127.0.0.1:8085`
Any attempt to request another destination (e.g., MySQL 3306, Remote Access console, or arbitrary intranet hosts) is rejected by the Host.
3. **No General Bearer Tokens**: Relay allocations are requested by the Host on behalf of a specific authenticated PlayerIdentity/session via the control plane. Allocations are bound to `RealmId`, `PlayerId`, have short lifetimes (default 120s before connection), and cannot be used for administrative or control plane operations.
4. **Dumb Byte Tunnel**: The Relay never terminates WoW encryption or parses character data. The only packet inspection performed is rewriting the `address` field in the plaintext `CMD_REALM_LIST` response (opcode `0x10`) so the client seamlessly connects to the allocated WORLD port without patching AzerothCore or mutating MySQL `realmlist`.
5. **Zero Plaintext Logging**: Relay logs contain only connection identifiers, timestamps, stream IDs, and byte counts. Zero credentials, account names, session keys, or packet contents are logged.
6. **Resource Limits & Backpressure**:
- Connection limits: max 16 concurrent hosts, max 256 allocations total, max 2 active allocations per player.
- Timeouts: 10s challenge timeout, 60s idle timeout, 30s ping/pong keepalive.
- Bounded streaming chunks: 16 KiB chunks with backpressure.
- Dynamic port pool: 40000–40050/tcp. Unused or closed ports are immediately reclaimed.
---
## 2. Host Tunnel Handshake
When a Host Manager publishes a realm, it establishes an outbound connection to `/relay/v1/host`.
1. **Relay Challenge**: The Relay sends a challenge frame containing a random nonce:
```json
{ "nonce": "<base64url_32_bytes>" }
```
2. **Host Hello**: The Host signs the challenge with the realm's Ed25519 private key:
```
message = "coa-relay-host-v1\n" || protocol_version || "\n" || realm_id || "\n" || nonce
```
The Host replies:
```json
{ "realm_id": "<RealmId>", "signature": "<base64_signature>" }
```
3. **Relay Verification**: The Relay verifies the signature against the realm's registered public key in the database (`coa_registry.realms`).
4. **Relay Welcome**: On success:
```json
{ "ok": true, "message": "welcome" }
```
---
## 3. Tunnel Messaging (`TunnelMsg`)
Multiplexed JSON messages over the WebSocket tunnel:
| Message | Fields | Purpose |
|---|---|---|
| `ping` | — | Keepalive probe |
| `pong` | — | Keepalive response |
| `allocate` | `request_id`, `player_id` | Host requests game ports for a player |
| `allocate_ok` | `request_id`, `token`, `relay_host`, `auth_port`, `world_port`, `expires_at` | Relay provides allocated ports |
| `allocate_err` | `request_id`, `error` | Allocation failed (e.g. pool exhausted) |
| `connect` | `stream_id`, `token`, `target` (`auth` \| `world`) | Client connected to relay port; requests Host to connect local target |
| `connect_ok` | `stream_id` | Host connected local target successfully |
| `connect_err` | `stream_id`, `error` | Host failed to connect local target |
| `data` | `stream_id`, `chunk` (base64) | Bidirectional raw stream bytes |
| `close` | `stream_id` | Half-close / end of stream |
| `reset` | `stream_id` | Immediate abnormal stream termination |
---
## 4. Client Flow & REALM_LIST Address Rewriting
```text
1. Player clicks "Join Realm" in Manager
2. Player Manager requests AllocateRelay over Noise control channel
3. Host Manager allocates (auth_port, world_port) via Relay tunnel
4. Host returns RelayAllocated to Player Manager
5. Player Manager writes realmlist.wtf: "set realmlist relay_host:auth_port" and starts WoW client
6. WoW client connects to relay_host:auth_port (authenticates with SRP6 against local authserver)
7. Authserver sends CMD_REALM_LIST (0x10) with local address (e.g. "127.0.0.1:8085")
8. Relay rewrites address string to "relay_host:world_port" and updates packet body length
9. WoW client receives rewritten realmlist and connects directly to relay_host:world_port
10. World session begins; bytes tunneled end-to-end through Host tunnel
```
### Packet Layout (`0x10 CMD_REALM_LIST`)
- `[0]`: Opcode (`0x10`)
- `[1..3]`: Packet Body Length (`uint16_le`)
- `[3..7]`: Unused (`uint32`)
- `[7..9]`: Realm Count (`uint16_le`)
- For each realm:
- `[0]`: Type (`uint8`)
- `[1]`: Lock (`uint8`)
- `[2]`: Flags (`uint8`)
- `[...]`: Name (null-terminated string)
- `[...]`: Address (null-terminated string) -> **Rewritten to `<relay_host>:<world_port>\0`**
- `[...]`: Population (`float32`)
- `[...]`: Characters (`uint8`)
- `[...]`: Timezone (`uint8`)
- `[...]`: Realm ID (`uint8`)
- Footer: `0x10 0x00`
+3 -2
View File
@@ -43,9 +43,10 @@ Date configured: 2026-10-06 (UTC)
|---|---|---|
| 22 | tcp | sshd (key only) |
| 80 | tcp | Caddy (redirects HTTP to HTTPS on 443) |
| 443 | tcp | Caddy (Let's Encrypt TLS: `/registry/*` and `/coord/*`) |
| 443 | tcp | Caddy (Let's Encrypt TLS: `/registry/*`, `/coord/*`, `/relay/*`) |
| 40000-40050 | tcp | Game Relay WoW game traffic (Auth & World) |
Everything else is closed, including 5432, 3306/3307, 2375/2376 (Docker API), 2019 (Caddy admin; admin API is disabled).
Everything else is closed, including 5432 (Postgres), 8080 (Registry internal), 8081 (Coordinator internal), 8082 (Relay control internal), 3306/3307, 2375/2376 (Docker API), 2019 (Caddy admin; admin API is disabled).
## SSH