From b5e4a38d7d9feb422b762596f0700ab28bb22ab7 Mon Sep 17 00:00:00 2001 From: "Fabien F." <8136686+lostmind84@users.noreply.github.com> Date: Sat, 3 Oct 2026 22:15:22 +0200 Subject: [PATCH] feat: install a new server as a Docker installation (Linux) docker::install is the Linux counterpart of install::install_base and shares none of its code (the Windows installer and its folder checks are untouched). Same order of events: signed manifest, folder / space / game data checks, verified download, extraction into .installing, database, one rename. Docker must be usable; a failure removes the containers, the volume and the staging folder. The databases start from the package's own dumps, Database/baseline/.sql.zst, in the format of the Manager's backups, taken in a state where every migration of the package is applied. The installer imports them, records the migrations as applied (baseline) in a few bulk statements, and creates the acore account, the realm port and the console account. Rebuilding from the repository's SQL does not work: pending migrations such as rev_20260903_01_live_class_baseline are guards generated from the maintainers' live database and refuse to run on a database built from the base scripts, whatever the order. - coa-release export-baseline --server DIR --out DIR produces the dumps from a prepared server (repack or Docker). - The game data stays where the person has it (dataDir in docker.json, mounted read-only), checked at install time. - ensure_main_configs creates worldserver.conf / authserver.conf from their .dist templates when missing, at install and at every start, never overwriting. - Fix (affects the backend merged in 0.6.0): the database container's health check is now a ping over TCP. On a new volume the image first runs a temporary server that answers on its socket but takes no network connection, so the container was healthy about four seconds (10 s against 14 s measured) before TCP worked. - docs/ARCHITECTURE.md: D13 (installation, package contract) and D14 (how Windows and Linux code are kept apart). Tested: 12 new tests (folder rules, settings, package checks, bulk ledger SQL, failure cleanup with an in-memory docker), and real runs of the whole chain through examples/docker_install: baseline dumps exported from a prepared server, package signed with a throwaway key, installed in 171 s, started by the Manager (database healthy, world ready in 8 s, auth up), console reached with the account the installer created, 1630 migrations recorded as applied. --- crates/coa-core/examples/docker_install.rs | 21 + crates/coa-core/src/docker/install.rs | 574 +++++++++++++++++++++ crates/coa-core/src/docker/lifecycle.rs | 51 +- crates/coa-core/src/docker/mod.rs | 66 ++- crates/coa-core/src/install.rs | 6 +- crates/coa-core/src/package.rs | 6 + crates/coa-core/src/registry.rs | 1 + crates/coa-core/src/release.rs | 15 + crates/coa-release/src/main.rs | 7 + docs/ARCHITECTURE.md | 39 +- 10 files changed, 771 insertions(+), 15 deletions(-) create mode 100644 crates/coa-core/examples/docker_install.rs create mode 100644 crates/coa-core/src/docker/install.rs diff --git a/crates/coa-core/examples/docker_install.rs b/crates/coa-core/examples/docker_install.rs new file mode 100644 index 0000000..8a25248 --- /dev/null +++ b/crates/coa-core/examples/docker_install.rs @@ -0,0 +1,21 @@ +//! Install a new Docker server from a package folder, for a manual end-to-end check on Linux (needs docker). +//! usage: docker_install +//! The key is the one the package was signed with (the production key is only used by the Manager itself). +use coa_core::docker::install::{install, Params}; +use coa_core::download::Cancel; +use coa_core::install::Source; +use coa_core::registry::Registry; +use std::path::PathBuf; + +fn main() { + let a: Vec = std::env::args().skip(1).collect(); + assert!(a.len() == 4, "usage: docker_install "); + let registry = Registry::at(std::env::temp_dir().join("coa-docker-install/installs.json")); + let t = std::time::Instant::now(); + let done = install( + &Params { source: Source::Dir(PathBuf::from(&a[0])), dest: PathBuf::from(&a[1]), data_dir: PathBuf::from(&a[2]), trusted_key: &a[3], registry: ®istry, cancel: Cancel::default() }, + &|s| eprintln!(" {:>3}% {} {}", s.percent, s.step, s.detail.unwrap_or_default()), + ) + .unwrap_or_else(|e| panic!("install failed: {e}")); + println!("installed {} at {} in {:?}", done.version, done.path, t.elapsed()); +} diff --git a/crates/coa-core/src/docker/install.rs b/crates/coa-core/src/docker/install.rs new file mode 100644 index 0000000..ddd1dbd --- /dev/null +++ b/crates/coa-core/src/docker/install.rs @@ -0,0 +1,574 @@ +//! Installing a new server as a Docker installation, from a Linux package. +//! +//! The order of events is the one of `install::install_base`: nothing appears at the destination until the package is +//! signed-off, downloaded, verified, extracted and its database built, all of it in a sibling `.installing` +//! folder. But this is code of its own: the Windows installer is not touched and nothing here is specific to the +//! Windows layout. What both installers use is platform-neutral: the signed manifest, the download, the extraction, +//! the migration ledger and the registry. +//! +//! The databases start from the package's own dumps (`Database/baseline/.sql.zst`, the format of the Manager's +//! backups), as a Windows package starts from its prepared database: they hold the state in which every migration of the +//! package is already applied, and the installer records them as such. They cannot be rebuilt from the repository's +//! SQL files: some migrations are guards that only apply to the maintainers' own database. The game data is not part +//! of the package: the person points at a folder they already have. + +use std::fs; +use std::path::{Path, PathBuf}; + +use super::cli::{Docker, SystemDocker}; +use super::{Config, MARKER}; +use crate::db::{self, Account, Db}; +use crate::download::Cancel; +use crate::driver::Verb; +use crate::error::{Error, Result}; +use crate::fsx; +use crate::install::{free_port, problem, random_hex, Installed, Preflight, Step}; +use crate::manifest::{self, Migration}; +use crate::migrations; +use crate::package::BASELINE_DIR; +use crate::pkgsource::{fetch_manifest, fetch_parts, Source}; +use crate::registry::{metadata_dir_for, InstallKind, InstallMeta, MetaDir, Registry, LAYOUT_DOCKER_V1}; + +/// Created in the staging folder so a leftover of an earlier attempt can be told from somebody else's folder. +const STAGING_MARKER: &str = ".coa-installing"; +/// The three game databases: kind (the name of the dump), schema. +const DATABASES: [(&str, &str); 3] = [("auth", "acore_auth"), ("characters", "acore_characters"), ("world", "acore_world")]; +/// Ports the game uses by default on this computer. +const DEFAULT_PORTS: (u16, u16, u16) = (3724, 8085, 3443); + +pub struct Params<'a> { + pub source: Source, + pub dest: PathBuf, + /// A folder with the game data (dbc, maps, vmaps, mmaps) that the person already has. Used in place, read-only. + pub data_dir: PathBuf, + /// Public key to verify the manifest with (production: `signing::EMBEDDED_PUBLIC_KEY`). + pub trusted_key: &'a str, + pub registry: &'a Registry, + pub cancel: Cancel, +} + +/// Checks whether `dest` and the game data folder are sensible for a new server on Linux. +pub fn preflight(dest: &Path, data_dir: &Path, needed_bytes: u64, registry: &Registry) -> Preflight { + let mut problems = Vec::new(); + let s = dest.to_string_lossy(); + + if !dest.is_absolute() { + problems.push(problem("relative", "Choose a full folder path, such as /home/you/CoaServer.")); + } + if s.contains(':') { + problems.push(problem("colon", "Choose a folder whose path has no colon in it.")); + } + let trimmed = s.trim_end_matches('/'); + const SYSTEM: [&str; 13] = ["", "/bin", "/boot", "/dev", "/etc", "/lib", "/lib64", "/proc", "/run", "/sbin", "/sys", "/usr", "/var"]; + if SYSTEM.iter().any(|p| trimmed == *p || (!p.is_empty() && trimmed.starts_with(&format!("{p}/")))) { + problems.push(problem("system_folder", "Please choose a normal folder for your games, not a system location.")); + } + if let Some(home) = std::env::var_os("HOME") { + let home = home.to_string_lossy().trim_end_matches('/').to_string(); + let special = ["", "/Documents", "/Desktop", "/Downloads", "/Pictures", "/Music", "/Videos", "/.config", "/.local", "/.local/share"]; + if !home.is_empty() && special.iter().any(|sfx| trimmed == format!("{home}{sfx}")) { + problems.push(problem("personal_folder", "This folder contains unrelated files. Choose another folder or create a new CoA Server folder.")); + } + } + + if dest.exists() { + match fs::read_dir(dest) { + Ok(mut rd) => { + if rd.next().is_some() { + match crate::layout::scan(dest).map(|r| r.classification) { + Ok(crate::layout::Classification::Incompatible) | Err(_) => { + if dest.join("Wow.exe").is_file() || dest.join("Ascension.exe").is_file() { + problems.push(problem("client_folder", "This is a game client folder, not a place for a server.")); + } else { + problems.push(problem("not_empty", "This folder contains unrelated files. Choose another folder or create a new CoA Server folder.")); + } + } + Ok(_) => problems.push(problem("already_server", "A server is already in this folder. Use \"I already have a server\" to add it instead.")), + } + } + } + Err(_) => problems.push(problem("unreadable", "This folder cannot be read. Choose another one.")), + } + } + + if let Ok(list) = registry.list() { + for (_, existing) in list { + if let (Ok(a), Ok(b)) = (fsx::canonicalize_lenient(dest), fsx::canonicalize_lenient(&existing)) { + if fsx::starts_with_ci(&a, &b) || fsx::starts_with_ci(&b, &a) { + problems.push(problem("registered", "A server is already registered at or around this location.")); + break; + } + } + } + } + + problems.extend(data_problems(data_dir)); + + let free_bytes = fsx::free_space(dest).unwrap_or(0); + let need = needed_bytes.saturating_add(needed_bytes / 5).saturating_add(512 * 1024 * 1024); + if free_bytes < need { + problems.push(problem("space", &format!("Not enough free space: about {} GB needed, {} GB available.", need / (1 << 30) + 1, free_bytes / (1 << 30)))); + } + Preflight { ok: problems.is_empty(), problems, free_bytes } +} + +fn data_problems(data_dir: &Path) -> Vec { + let s = data_dir.to_string_lossy(); + if !data_dir.is_absolute() || s.contains(':') { + return vec![problem("data_path", "Choose the game data folder with its full path, without a colon in it.")]; + } + let missing: Vec<&str> = ["dbc", "maps"].into_iter().filter(|d| !data_dir.join(d).is_dir()).collect(); + if missing.is_empty() { + Vec::new() + } else { + vec![problem("data_missing", &format!("This folder does not look like the game data: {} not found in it. It should hold dbc, maps, vmaps and mmaps.", missing.join(" and ")))] + } +} + +/// Port the game should use: the usual one when it is free on this computer, else any free one. +fn pick_port(preferred: u16) -> Result { + if std::net::TcpListener::bind(("127.0.0.1", preferred)).is_ok() { + Ok(preferred) + } else { + free_port() + } +} + +/// Settings of a new installation: which containers belong to it, where the game data is, the ports, the passwords. +/// Written before anything runs, so the database can be started from the staging folder. +struct Settings { + app_password: String, + world_port: u16, +} + +fn write_settings(root: &Path, data_dir: &Path) -> Result { + let project = random_hex(10); + let (auth, world, ra) = (pick_port(DEFAULT_PORTS.0)?, pick_port(DEFAULT_PORTS.1)?, pick_port(DEFAULT_PORTS.2)?); + let (root_password, app_password) = (random_hex(48), random_hex(48)); + fs::create_dir_all(root.join("Settings"))?; + let docker = Config { project, bind_address: "127.0.0.1".into(), mysql_image: super::MYSQL_IMAGE.into(), data_dir: Some(data_dir.to_string_lossy().into_owned()) }; + fsx::atomic_write_json(&root.join(MARKER), &docker)?; + // Same file and keys as a repack, so the console, the ports and the settings screens work unchanged. The database is + // not published on the host; its port is only there to fill the key. + fsx::atomic_write_json( + &root.join("Settings/repack.json"), + &serde_json::json!({ "mysqlPort": 3307, "authPort": auth, "worldPort": world, "raPort": ra, "raUsername": db::SERVICE_ACCOUNT, "raPassword": "" }), + )?; + let secrets = root.join("Settings/database.json"); + fsx::atomic_write_json(&secrets, &serde_json::json!({ "rootPassword": root_password, "appPassword": app_password }))?; + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + fs::set_permissions(&secrets, fs::Permissions::from_mode(0o600))?; + } + Ok(Settings { app_password, world_port: world }) +} + +/// The package must hold what a Docker installation is made of; say what is missing instead of failing later. +fn check_extracted(root: &Path) -> Result<()> { + let mut missing = Vec::new(); + for rel in ["Core/worldserver", "Core/authserver", "Core/configs/worldserver.conf.dist", "Core/configs/authserver.conf.dist"] { + if !root.join(rel).is_file() { + missing.push(rel.to_string()); + } + } + for (kind, _) in DATABASES { + if !root.join(BASELINE_DIR).join(format!("{kind}.sql.zst")).is_file() { + missing.push(format!("{BASELINE_DIR}/{kind}.sql.zst")); + } + } + if missing.is_empty() { + Ok(()) + } else { + Err(Error::Invalid(format!("This package is not a complete Linux server package; missing: {}.", missing.join(", ")))) + } +} + +/// Record every migration of the package as applied without running it, in a few statements (one `docker exec` for each +/// of more than a thousand would take minutes). Same ledger rows as `migrations::baseline`. +fn baseline_statements(list: &[Migration]) -> Result> { + let ok = |t: &str| !t.is_empty() && t.len() <= 190 && t.chars().all(|c| c.is_ascii_alphanumeric() || matches!(c, '_' | '.' | '-')); + let mut rows = Vec::with_capacity(list.len()); + for m in list { + if !matches!(m.db.as_str(), "auth" | "characters" | "world") || !ok(&m.id) || m.sha256.len() != 64 || !m.sha256.chars().all(|c| c.is_ascii_hexdigit()) { + return Err(Error::InvalidManifest(format!("migration {:?} is not valid", m.id))); + } + rows.push(format!("('{}','{}','{}','applied',NOW(),NULL,1)", m.db, m.id, m.sha256)); + } + Ok(rows + .chunks(400) + .map(|c| format!("REPLACE INTO `acore_world`.`coa_manager_migrations` (`db`,`id`,`sha256`,`status`,`applied_at`,`error`,`baseline`) VALUES {};", c.join(","))) + .collect()) +} + +/// Create the three schemas and the game servers' account, load the starting databases and record the package's migrations +/// as applied. The database container is started from `root` and stopped again by the caller. +fn build_database(d: &dyn Docker, root: &Path, list: &[Migration], s: &Settings, say: &dyn Fn(u8, String)) -> Result<()> { + let started = super::run_with(d, root, Verb::StartMysql)?; + if !started.ok { + return Err(Error::Invalid(started.human.map(|h| h.message.to_string()).unwrap_or_else(|| "The database could not be started.".into()))); + } + let db = Db::from_repack(root, Account::Admin)?; + let mut sql = String::new(); + for (_, schema) in DATABASES { + sql.push_str(&format!("CREATE DATABASE `{schema}` CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;\n")); + } + // The pattern also covers the schemas of other realms (acore_world_wildcard, ...). + sql.push_str(&format!("CREATE USER 'acore'@'%' IDENTIFIED BY '{}';\nGRANT ALL PRIVILEGES ON `acore\\_%`.* TO 'acore'@'%';\nFLUSH PRIVILEGES;\n", s.app_password)); + db.query(&sql)?; + + for (i, (kind, schema)) in DATABASES.iter().enumerate() { + say(83 + (i * 4) as u8, format!("Loading the {kind} database")); + db.import_from(schema, &root.join(BASELINE_DIR).join(format!("{kind}.sql.zst")))?; + } + + say(95 - 3, format!("Recording {} database updates as applied", list.len())); + // The ledger table is created by the first read of it. + migrations::Store::load(&db)?; + for statement in baseline_statements(list)? { + db.query(&statement)?; + } + + // The realm advertises the port the game server really uses on this computer. + db.query(&format!("UPDATE acore_auth.realmlist SET port={} WHERE id=1;", s.world_port))?; + // The server console gets its own account with a random password. + let console_pw = db.provision_service_account()?; + db::write_console_credentials(root, &console_pw)?; + if !Db::from_repack(root, Account::Admin)?.ping() { + return Err(Error::Invalid("The database did not answer after it was set up.".into())); + } + Ok(()) +} + +pub fn install(p: &Params, report: &dyn Fn(Step)) -> Result { + install_with(&SystemDocker, p, report) +} + +pub fn install_with(d: &dyn Docker, p: &Params, report: &dyn Fn(Step)) -> Result { + let say = |step: &'static str, percent: u8, detail: Option| report(Step { step, percent, detail }); + let dest = p.dest.clone(); + let staging_root = { + let mut n = dest.file_name().ok_or_else(|| Error::Invalid("bad destination".into()))?.to_os_string(); + n.push(".installing"); + dest.with_file_name(n) + }; + let meta_dir = metadata_dir_for(&dest)?; + + say("Checking your computer", 2, None); + super::check_docker(d)?; + // The manifest first: it is signed, and tells us how much space we need. + let (m, manifest_bytes) = fetch_manifest(&p.source, p.trusted_key)?; + if !matches!(m.kind, manifest::Kind::Base | manifest::Kind::Update) || !m.compatible_with_manager(crate::MANAGER_VERSION) { + return Err(Error::Invalid("This package needs a newer version of CoA Server Manager.".into())); + } + let archive = m.archive.clone().ok_or_else(|| Error::InvalidManifest("no archive".into()))?; + let download_size: u64 = archive.parts.iter().map(|x| x.size).sum(); + + let pre = preflight(&dest, &p.data_dir, archive.unpacked_size + download_size, p.registry); + if !pre.ok { + return Err(Error::Invalid(pre.problems.iter().map(|x| x.message.clone()).collect::>().join(" "))); + } + + let parts_dir = fetch_parts(&p.source, &m, &meta_dir.join("staging").join("download"), &p.cancel, &|frac, detail| say("Downloading server", 5 + (frac * 45.0) as u8, detail))?; + + if staging_root.exists() { + if staging_root.join(STAGING_MARKER).is_file() { + fs::remove_dir_all(&staging_root)?; // leftover of an earlier failed attempt that we created + } else { + return Err(Error::Invalid(format!("{} already exists and was not created by the Manager.", staging_root.display()))); + } + } + fs::create_dir_all(&staging_root)?; + fs::write(staging_root.join(STAGING_MARKER), b"")?; + + let mut project: Option = None; + let result = (|| -> Result { + say("Unpacking", 50, None); + crate::package::extract(&parts_dir, &m, &staging_root, &|done, total| say("Unpacking", 50 + (done * 30 / total.max(1)) as u8, None))?; + check_extracted(&staging_root)?; + + say("Preparing database", 82, None); + let settings = write_settings(&staging_root, &p.data_dir)?; + project = Some(Config::load(&staging_root)?); + let progress = |percent: u8, detail: String| say("Preparing database", percent, Some(detail)); + let built = build_database(d, &staging_root, &m.migrations, &settings, &progress); + let _ = super::run_with(d, &staging_root, Verb::StopAll); + built?; + fs::remove_file(staging_root.join(STAGING_MARKER))?; + + // Commit: the only moment the destination changes. The database lives in a docker volume, so it does not care + // that the folder moves. + say("Finishing", 95, None); + if dest.exists() { + fs::remove_dir(&dest)?; // only succeeds for an empty folder (preflight verified) + } + fs::rename(&staging_root, &dest)?; + super::ensure_main_configs(&dest)?; + crate::config::materialize_module_configs(&dest)?; + + let mut meta = InstallMeta::new(InstallKind::New, &dest); + meta.layout = LAYOUT_DOCKER_V1.into(); + meta.core.commit = m.core.commit.clone(); + meta.core.version = Some(m.version.clone()); + meta.database.schemas = DATABASES.iter().map(|(_, s)| s.to_string()).collect(); + meta.managed_files = m.files.iter().map(|f| f.path.clone()).collect(); + for f in &m.files { + meta.original_hashes.insert(f.path.clone(), f.sha256.clone()); + } + let md = MetaDir::create(&dest, &meta)?; + fsx::atomic_write(&md.root.join("manifests").join("base.json"), &manifest_bytes)?; + p.registry.register(&meta.id, &dest)?; + say("Ready", 100, None); + Ok(Installed { id: meta.id, path: dest.to_string_lossy().into_owned(), version: m.version.clone() }) + })(); + + if result.is_err() && staging_root.join(STAGING_MARKER).exists() { + // Unfinished work of ours; the destination was never touched. Give back what docker holds for it. + if let Some(cfg) = &project { + super::destroy(d, cfg); + } + let _ = fs::remove_dir_all(&staging_root); + } + result +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::docker::cli::{Call, Output}; + use crate::package::{self, BuildOptions}; + use base64::Engine; + use ed25519_dalek::{Signer, SigningKey}; + use std::cell::RefCell; + + struct Fake { + calls: RefCell>>, + unavailable: bool, + } + + impl Fake { + fn new(unavailable: bool) -> Fake { + Fake { calls: Default::default(), unavailable } + } + } + + impl Docker for Fake { + fn run(&self, call: &Call) -> Result { + self.calls.borrow_mut().push(call.args.clone()); + if self.unavailable { + return Err(Error::Invalid("docker could not be started: not found".into())); + } + Ok(Output { code: Some(0), stdout: "27.3.1".into(), stderr: String::new() }) + } + + fn pause(&self, _d: std::time::Duration) {} + } + + fn reg(d: &Path) -> Registry { + Registry::at(d.join("reg/installs.json")) + } + + fn codes(p: &Preflight) -> Vec<&'static str> { + p.problems.iter().map(|x| x.code).collect() + } + + fn data_folder(d: &Path) -> PathBuf { + let data = d.join("data"); + fs::create_dir_all(data.join("dbc")).unwrap(); + fs::create_dir_all(data.join("maps")).unwrap(); + data + } + + /// A signed package folder holding `files`, and the key that verifies it. + fn signed_package(d: &Path, files: &[(&str, &str)]) -> (PathBuf, String) { + let src = d.join("src"); + for (rel, content) in files { + let p = src.join(rel); + fs::create_dir_all(p.parent().unwrap()).unwrap(); + fs::write(p, content).unwrap(); + } + let pkg = d.join("pkg"); + let opts = BuildOptions { kind: manifest::Kind::Update, version: "0.1.0".into(), core_commit: None, built_at: "x".into(), part_size: 1 << 20, bots_commit: None, migrations: vec![] }; + package::build(&src, &pkg, &opts, &|_| {}).unwrap(); + let key = SigningKey::generate(&mut rand_core::OsRng); + let engine = base64::engine::general_purpose::STANDARD; + let sig = engine.encode(key.sign(&fs::read(pkg.join("manifest.json")).unwrap()).to_bytes()); + fs::write(pkg.join("manifest.json.sig"), sig).unwrap(); + (pkg, engine.encode(key.verifying_key().to_bytes())) + } + + const FULL: [(&str, &str); 7] = [ + ("Core/worldserver", "w"), + ("Core/authserver", "a"), + ("Core/configs/worldserver.conf.dist", "x"), + ("Core/configs/authserver.conf.dist", "x"), + ("Database/baseline/auth.sql.zst", "dump"), + ("Database/baseline/characters.sql.zst", "dump"), + ("Database/baseline/world.sql.zst", "dump"), + ]; + + #[cfg(unix)] + #[test] + fn the_destination_rules_fit_a_linux_computer() { + let d = tempfile::tempdir().unwrap(); + let data = data_folder(d.path()); + let ok = |dest: &Path| codes(&preflight(dest, &data, 1000, ®(d.path()))); + assert!(!ok(&d.path().join("CoaServer")).iter().any(|c| *c != "space"), "a new folder is fine"); + for bad in ["/usr/local/coa", "/etc", "/", "/proc/x"] { + assert!(ok(Path::new(bad)).contains(&"system_folder"), "{bad}"); + } + assert!(ok(Path::new("relative/dir")).contains(&"relative")); + assert!(ok(Path::new("/srv/with:colon")).contains(&"colon")); + let existing = d.path().join("existing"); + fs::create_dir_all(&existing).unwrap(); + fs::write(existing.join("notes.txt"), b"x").unwrap(); + assert!(ok(&existing).contains(&"not_empty")); + let r = reg(d.path()); + r.register("1", &existing).unwrap(); + assert!(codes(&preflight(&existing.join("inner"), &data, 1000, &r)).contains(&"registered")); + // The same name in another case is another folder on Linux. + assert!(!codes(&preflight(&d.path().join("EXISTING"), &data, 1000, &r)).contains(&"registered")); + } + + // The tests marked for unix use absolute folders of the temporary directory, which on Windows start with a drive letter and a + // colon; the Docker backend refuses a colon in a folder it mounts (it would be read as a separator of `--volume`) and only runs + // on Linux, so Windows has nothing to check here. + #[cfg(unix)] + #[test] + fn the_game_data_folder_must_look_like_game_data() { + let d = tempfile::tempdir().unwrap(); + let reg = reg(d.path()); + let dest = d.path().join("srv"); + let empty = d.path().join("empty"); + fs::create_dir_all(&empty).unwrap(); + assert!(codes(&preflight(&dest, &empty, 1, ®)).contains(&"data_missing")); + assert!(codes(&preflight(&dest, Path::new("relative/data"), 1, ®)).contains(&"data_path")); + assert!(!codes(&preflight(&dest, &data_folder(d.path()), 1, ®)).iter().any(|c| c.starts_with("data"))); + } + + #[cfg(unix)] + #[test] + fn new_settings_name_the_containers_and_keep_the_secrets_private() { + let d = tempfile::tempdir().unwrap(); + let root = d.path().join("srv"); + let data = data_folder(d.path()); + let s = write_settings(&root, &data).unwrap(); + + let cfg = Config::load(&root).unwrap(); + assert_eq!(cfg.bind_address, "127.0.0.1", "this computer only until the person opens it"); + assert_eq!(cfg.project.len(), 10); + assert_eq!(cfg.data_path(&root), data); + + let ports = crate::layout::read_ports(&root); + assert_eq!(ports.world, s.world_port); + assert!(ports.auth > 0 && ports.ra > 0); + let repack: serde_json::Value = fsx::read_json(&root.join("Settings/repack.json")).unwrap(); + assert_eq!(repack["raUsername"], db::SERVICE_ACCOUNT); + + let (root_pw, app_pw) = db::credentials(&root).unwrap(); + assert!(root_pw.len() == 48 && app_pw.len() == 48 && root_pw != app_pw); + assert_eq!(app_pw, s.app_password); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let mode = fs::metadata(root.join("Settings/database.json")).unwrap().permissions().mode() & 0o777; + assert_eq!(mode, 0o600, "the passwords are for their owner only"); + } + // Two installations never share a project name. + let other = write_settings(&d.path().join("srv2"), &data).unwrap(); + assert_ne!(Config::load(&d.path().join("srv2")).unwrap().project, cfg.project); + let _ = other; + } + + #[test] + fn a_busy_default_port_is_replaced_by_a_free_one() { + let held = std::net::TcpListener::bind("127.0.0.1:0").unwrap(); + let busy = held.local_addr().unwrap().port(); + let got = pick_port(busy).unwrap(); + assert_ne!(got, busy); + drop(held); + assert_eq!(pick_port(busy).unwrap(), busy, "free again: the usual port is used"); + } + + #[test] + fn a_package_is_checked_for_everything_an_installation_needs() { + let d = tempfile::tempdir().unwrap(); + let root = d.path(); + assert!(check_extracted(root).unwrap_err().to_string().contains("Core/worldserver")); + for (rel, c) in FULL { + let p = root.join(rel); + fs::create_dir_all(p.parent().unwrap()).unwrap(); + fs::write(p, c).unwrap(); + } + check_extracted(root).unwrap(); + fs::remove_file(root.join("Database/baseline/world.sql.zst")).unwrap(); + assert!(check_extracted(root).unwrap_err().to_string().contains("Database/baseline/world.sql.zst")); + } + + fn migration(db: &str, id: &str) -> Migration { + Migration { db: db.into(), id: id.into(), sha256: "ab".repeat(32), destructive: false } + } + + #[test] + fn the_package_migrations_are_recorded_as_applied_in_a_few_statements() { + let list: Vec = (0..1000).map(|i| migration(["auth", "characters", "world"][i % 3], &format!("rev_{i}"))).collect(); + let statements = baseline_statements(&list).unwrap(); + assert_eq!(statements.len(), 3, "400 rows at most in a statement"); + assert!(statements[0].starts_with("REPLACE INTO `acore_world`.`coa_manager_migrations`")); + assert_eq!(statements.iter().map(|s| s.matches("'applied'").count()).sum::(), 1000); + assert!(statements[0].contains("('auth','rev_0',") && statements[0].contains(",NOW(),NULL,1)"), "marked as a baseline"); + assert!(baseline_statements(&[]).unwrap().is_empty()); + } + + #[test] + fn a_migration_that_could_carry_sql_is_refused_before_anything_is_written() { + for bad in [migration("world", "x'); DROP TABLE y;--"), migration("mysql", "ok"), migration("world", ""), Migration { sha256: "zz".into(), ..migration("world", "ok") }] { + assert!(baseline_statements(&[migration("auth", "fine"), bad]).is_err()); + } + } + + #[test] + fn without_docker_nothing_is_downloaded_or_created() { + let d = tempfile::tempdir().unwrap(); + let (pkg, key) = signed_package(d.path(), &FULL); + let dest = d.path().join("dest"); + let r = reg(d.path()); + let fake = Fake::new(true); + let err = install_with(&fake, &Params { source: Source::Dir(pkg), dest: dest.clone(), data_dir: data_folder(d.path()), trusted_key: &key, registry: &r, cancel: Cancel::default() }, &|_| {}).unwrap_err(); + assert!(err.to_string().to_lowercase().contains("docker"), "{err}"); + assert!(!dest.exists() && !dest.with_file_name("dest.installing").exists()); + } + + #[cfg(unix)] + #[test] + fn a_package_without_the_server_is_refused_and_leaves_nothing() { + let d = tempfile::tempdir().unwrap(); + let (pkg, key) = signed_package(d.path(), &[("Core/configs/worldserver.conf.dist", "x"), ("README.txt", "hello")]); + let dest = d.path().join("dest"); + let r = reg(d.path()); + let fake = Fake::new(false); + let err = install_with(&fake, &Params { source: Source::Dir(pkg), dest: dest.clone(), data_dir: data_folder(d.path()), trusted_key: &key, registry: &r, cancel: Cancel::default() }, &|_| {}).unwrap_err(); + assert!(err.to_string().contains("not a complete Linux server package"), "{err}"); + assert!(!dest.exists() && !dest.with_file_name("dest.installing").exists()); + assert!(r.list().unwrap().is_empty()); + } + + #[cfg(unix)] + #[test] + fn a_failed_database_setup_gives_back_what_docker_holds_and_leaves_no_server() { + let d = tempfile::tempdir().unwrap(); + let (pkg, key) = signed_package(d.path(), &FULL); + let dest = d.path().join("dest"); + let r = reg(d.path()); + // This fake docker answers every command with success and no container, so the database never becomes ready. + let fake = Fake::new(false); + let err = install_with(&fake, &Params { source: Source::Dir(pkg), dest: dest.clone(), data_dir: data_folder(d.path()), trusted_key: &key, registry: &r, cancel: Cancel::default() }, &|_| {}).unwrap_err(); + assert!(err.to_string().to_lowercase().contains("database"), "{err}"); + assert!(!dest.exists() && !dest.with_file_name("dest.installing").exists(), "no half-installed server"); + assert!(r.list().unwrap().is_empty()); + let calls = fake.calls.borrow(); + let removed = |suffix: &str| calls.iter().any(|c| c[0] == "rm" && c.last().is_some_and(|n| n.starts_with("coa-") && n.ends_with(suffix))); + assert!(removed("-db") && removed("-world") && removed("-auth"), "containers removed: {calls:?}"); + assert!(calls.iter().any(|c| c[..2] == ["volume", "rm"]) && calls.iter().any(|c| c[..2] == ["network", "rm"])); + } +} diff --git a/crates/coa-core/src/docker/lifecycle.rs b/crates/coa-core/src/docker/lifecycle.rs index cdac849..bbf813a 100644 --- a/crates/coa-core/src/docker/lifecycle.rs +++ b/crates/coa-core/src/docker/lifecycle.rs @@ -80,6 +80,26 @@ fn docker(d: &dyn Docker, args: &[&str], timeout: Duration) -> Step Result { + let mut log = Log::default(); + match preflight(d, &mut log) { + Ok(()) => Ok(log.0.join(" ")), + Err(f) => Err(crate::error::Error::Invalid(format!("{} {}", f.code.human().message, f.output.trim()).trim().to_string())), + } +} + +/// Remove everything docker holds for an installation: containers, network and the database volume. Used when an +/// installation fails before it is finished, and later to uninstall. Missing pieces are not an error. +pub(crate) fn destroy(d: &dyn Docker, cfg: &Config) { + let n = cfg.names(); + for name in [&n.auth, &n.world, &n.db] { + let _ = d.run(&Call::new(&["rm", "--force", name], QUICK)); + } + let _ = d.run(&Call::new(&["network", "rm", &n.network], QUICK)); + let _ = d.run(&Call::new(&["volume", "rm", &n.volume], QUICK)); +} + fn preflight(d: &dyn Docker, log: &mut Log) -> Step<()> { let o = docker(d, &["version", "--format", "{{.Server.Version}}"], QUICK)?; if !o.ok() { @@ -204,6 +224,7 @@ fn start(d: &dyn Docker, root: &Path, cfg: &Config, with_game: bool, log: &mut L } ensure_image(d, log)?; + super::ensure_main_configs(root).map_err(|e| Failure { code: ErrorCode::ServerFilesIncomplete, output: e.to_string() })?; std::fs::create_dir_all(root.join("Core/Logs")).map_err(|e| Failure { code: ErrorCode::ServerFilesIncomplete, output: e.to_string() })?; let owner = owner_of(root); let host = std::fs::canonicalize(root).unwrap_or_else(|_| root.to_path_buf()); @@ -212,7 +233,7 @@ fn start(d: &dyn Docker, root: &Path, cfg: &Config, with_game: bool, log: &mut L if !state.get(&n.world).is_some_and(Container::running) { remove(d, &n.world); let mut call = Call::new(&[], Duration::from_secs(60)); - call.args = game_args(cfg, &n, GameKind::World, &host, ports.world, ports.ra, owner.as_deref()); + call.args = game_args(cfg, &n, GameKind::World, &host, &cfg.data_path(&host), ports.world, ports.ra, owner.as_deref()); call.env = database_env(&secrets, true); run_container(d, &call, &n.world, log)?; } @@ -221,7 +242,7 @@ fn start(d: &dyn Docker, root: &Path, cfg: &Config, with_game: bool, log: &mut L if !state.get(&n.auth).is_some_and(Container::running) { remove(d, &n.auth); let mut call = Call::new(&[], Duration::from_secs(60)); - call.args = game_args(cfg, &n, GameKind::Auth, &host, ports.auth, 0, owner.as_deref()); + call.args = game_args(cfg, &n, GameKind::Auth, &host, &cfg.data_path(&host), ports.auth, 0, owner.as_deref()); call.env = database_env(&secrets, false); run_container(d, &call, &n.auth, log)?; } @@ -320,8 +341,11 @@ fn db_args(cfg: &Config, n: &Names) -> Vec { a.extend(["--volume".into(), format!("{}:/var/lib/mysql", n.volume)]); // The password comes from the environment of the docker client, not from this command line. a.extend(["--env".into(), "MYSQL_ROOT_PASSWORD".into()]); - // "mysqladmin ping" succeeds as soon as the server answers, with or without a login. - a.extend(["--health-cmd".into(), "mysqladmin ping --silent".into()]); + // "mysqladmin ping" succeeds as soon as the server answers, with or without a login. It must go over TCP, like + // everything the Manager sends: while a new database initialises, the image first runs a temporary server that + // answers on its socket but takes no network connection, and a socket ping would call the container healthy for a few + // seconds in which every TCP connection is still refused. + a.extend(["--health-cmd".into(), "mysqladmin ping --protocol=tcp --host=127.0.0.1 --silent".into()]); a.extend(["--health-interval".into(), "5s".into(), "--health-timeout".into(), "5s".into(), "--health-retries".into(), "40".into()]); a.extend(["--stop-timeout".into(), "60".into()]); a.push(cfg.mysql_image.clone()); @@ -345,7 +369,7 @@ fn database_env(s: &Secrets, world: bool) -> Vec<(String, String)> { env } -fn game_args(cfg: &Config, n: &Names, kind: GameKind, host: &Path, port: u16, ra_port: u16, owner: Option<&str>) -> Vec { +fn game_args(cfg: &Config, n: &Names, kind: GameKind, host: &Path, data: &Path, port: u16, ra_port: u16, owner: Option<&str>) -> Vec { let (name, alias, binary, conf, inner_port) = match kind { GameKind::World => (&n.world, "world", "./worldserver", "configs/worldserver.conf", WORLD_PORT), GameKind::Auth => (&n.auth, "auth", "./authserver", "configs/authserver.conf", AUTH_PORT), @@ -370,7 +394,7 @@ fn game_args(cfg: &Config, n: &Names, kind: GameKind, host: &Path, port: u16, ra // the container and stops the server when it cannot find one. a.extend(env("AC_UPDATES_ENABLE_DATABASES", "0")); if kind == GameKind::World { - a.extend(["--volume".into(), format!("{}:{DATA}:ro", host.join("Data").display())]); + a.extend(["--volume".into(), format!("{}:{DATA}:ro", data.display())]); a.extend(env("AC_DATA_DIR", DATA)); // The remote console is only ever reachable from this computer. a.extend(["--publish".into(), format!("127.0.0.1:{ra_port}:{RA_PORT}")]); @@ -674,6 +698,17 @@ mod tests { assert!(runs[1].args.windows(2).any(|w| w == ["--env", "AC_WORLD_DATABASE_INFO"]), "name only, the value is in the environment"); } + #[test] + fn the_database_is_only_healthy_once_it_takes_tcp_connections() { + let (_d, root) = server("t1"); + let sim = Sim::new(); + assert!(start_all(&sim, &root).ok); + let db = &sim.calls_of("run")[0].args; + let i = db.iter().position(|a| a == "--health-cmd").expect("the database has a health check"); + // Found on a new volume: a socket-only ping reported healthy about four seconds before TCP worked. + assert!(db[i + 1].contains("--protocol=tcp") && db[i + 1].contains("--host=127.0.0.1"), "{}", db[i + 1]); + } + #[test] fn the_world_container_is_wired_like_the_repack_layout() { let (_d, root) = server("t1"); @@ -714,8 +749,8 @@ mod tests { let (_d, root) = server("ports"); let cfg = Config::load(&root).unwrap(); let n = cfg.names(); - let world = game_args(&cfg, &n, GameKind::World, &root, 18085, 13443, None); - let auth = game_args(&cfg, &n, GameKind::Auth, &root, 13724, 0, None); + let world = game_args(&cfg, &n, GameKind::World, &root, &root.join("Data"), 18085, 13443, None); + let auth = game_args(&cfg, &n, GameKind::Auth, &root, &root.join("Data"), 13724, 0, None); assert!(world.iter().any(|a| a == "127.0.0.1:18085:8085")); assert!(world.iter().any(|a| a == "127.0.0.1:13443:3443")); assert!(world.iter().any(|a| a == "AC_WORLD_SERVER_PORT=8085")); diff --git a/crates/coa-core/src/docker/mod.rs b/crates/coa-core/src/docker/mod.rs index ffa10ce..04cc121 100644 --- a/crates/coa-core/src/docker/mod.rs +++ b/crates/coa-core/src/docker/mod.rs @@ -15,10 +15,12 @@ //! ``` mod cli; +pub mod install; mod lifecycle; pub use cli::{Call, Docker, Output, SystemDocker}; -pub use lifecycle::{observe, observe_with, run, run_with}; +pub use lifecycle::{check_docker, observe, observe_with, run, run_with}; +pub(crate) use lifecycle::destroy; use std::net::IpAddr; use std::path::Path; @@ -31,7 +33,7 @@ use crate::layout::{self, Classification, ScanReport}; /// Present in a Docker installation, absent from a repack. pub const MARKER: &str = "Settings/docker.json"; -const MYSQL_IMAGE: &str = "mysql:8.4"; +pub(crate) const MYSQL_IMAGE: &str = "mysql:8.4"; const RUNTIME_DOCKERFILE: &str = include_str!("Dockerfile.runtime"); pub fn is_docker(root: &Path) -> bool { @@ -57,6 +59,10 @@ pub struct Config { pub bind_address: String, #[serde(default = "mysql_image")] pub mysql_image: String, + /// Where the game data (dbc, maps, vmaps, mmaps) is, when it is not the `Data` folder of the server. Used in place and + /// mounted read-only, so one copy can serve several servers. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub data_dir: Option, } pub(crate) struct Names { @@ -74,6 +80,11 @@ impl Config { Ok(cfg) } + /// The folder mounted as the game data. + pub fn data_path(&self, root: &Path) -> std::path::PathBuf { + self.data_dir.as_deref().map(std::path::PathBuf::from).unwrap_or_else(|| root.join("Data")) + } + fn validate(&self) -> Result<()> { let name_ok = !self.project.is_empty() && self.project.len() <= 32 @@ -85,6 +96,12 @@ impl Config { if self.bind_address.parse::().is_err() { return Err(Error::Invalid(format!("{} is not an IP address.", self.bind_address))); } + if let Some(d) = &self.data_dir { + // The folder goes into a `--volume host:container:ro` option, where a colon would be read as a separator. + if !Path::new(d).is_absolute() || d.contains(':') { + return Err(Error::Invalid("The game data folder must be a full path without a colon.".into())); + } + } let image_ok = !self.mysql_image.is_empty() && self.mysql_image.chars().all(|c| c.is_ascii_alphanumeric() || "._/:@-".contains(c)); if !image_ok { return Err(Error::Invalid("The database image name is not valid.".into())); @@ -103,6 +120,21 @@ impl Config { } } +/// Create the world and auth configuration files from their `.dist` templates when they do not exist. A package only +/// carries the templates, and on Windows the launcher writes the active files at every start; here nothing else would. +/// An existing file is never touched, so the person's settings survive updates. Returns the files created. +pub fn ensure_main_configs(root: &Path) -> Result> { + let mut created = Vec::new(); + for name in ["worldserver", "authserver"] { + let (conf, dist) = (root.join(format!("Core/configs/{name}.conf")), root.join(format!("Core/configs/{name}.conf.dist"))); + if !conf.exists() && dist.is_file() { + std::fs::copy(&dist, &conf)?; + created.push(format!("{name}.conf")); + } + } + Ok(created) +} + /// Name of the runtime image: it follows the content of its Dockerfile, so changing the libraries builds a new image. pub(crate) fn runtime_image() -> String { format!("coa-runtime:{}", &fsx::sha256_bytes(RUNTIME_DOCKERFILE.as_bytes())[..12]) @@ -123,7 +155,8 @@ pub(crate) fn scan(root: &Path) -> Result { let bot_active = modules_dir.join("mod_coa_playerbots.conf"); let bot_conf = if bot_active.is_file() { bot_active } else { modules_dir.join("mod_coa_playerbots.conf.dist") }; let bot_config_keys = layout::count_bot_keys(&bot_conf); - let has_data = layout::exists(root, "Data/dbc") && layout::exists(root, "Data/maps"); + let data = cfg.as_ref().map(|c| c.data_path(root)).unwrap_or_else(|_| root.join("Data")); + let has_data = data.join("dbc").is_dir() && data.join("maps").is_dir(); let has_confs = layout::exists(root, "Core/configs/worldserver.conf") && layout::exists(root, "Core/configs/authserver.conf"); let items = vec![ @@ -162,3 +195,30 @@ pub(crate) fn scan(root: &Path) -> Result { }) } + +#[cfg(test)] +mod tests { + use super::*; + use std::fs; + + #[test] + fn the_main_configs_are_created_from_their_templates_and_never_overwritten() { + let d = tempfile::tempdir().unwrap(); + let cfg = d.path().join("Core/configs"); + fs::create_dir_all(&cfg).unwrap(); + fs::write(cfg.join("worldserver.conf.dist"), "Setting = default\n").unwrap(); + fs::write(cfg.join("authserver.conf.dist"), "Auth = default\n").unwrap(); + + assert_eq!(ensure_main_configs(d.path()).unwrap(), ["worldserver.conf", "authserver.conf"]); + assert_eq!(fs::read_to_string(cfg.join("worldserver.conf")).unwrap(), "Setting = default\n"); + + // The person's own settings survive the next start. + fs::write(cfg.join("worldserver.conf"), "Setting = mine\n").unwrap(); + assert!(ensure_main_configs(d.path()).unwrap().is_empty()); + assert_eq!(fs::read_to_string(cfg.join("worldserver.conf")).unwrap(), "Setting = mine\n"); + // Without a template there is nothing to create and nothing fails. + fs::remove_file(cfg.join("authserver.conf")).unwrap(); + fs::remove_file(cfg.join("authserver.conf.dist")).unwrap(); + assert!(ensure_main_configs(d.path()).unwrap().is_empty()); + } +} diff --git a/crates/coa-core/src/install.rs b/crates/coa-core/src/install.rs index bea801d..2e88aa4 100644 --- a/crates/coa-core/src/install.rs +++ b/crates/coa-core/src/install.rs @@ -35,7 +35,7 @@ pub struct Preflight { pub free_bytes: u64, } -fn problem(code: &'static str, message: &str) -> Problem { +pub(crate) fn problem(code: &'static str, message: &str) -> Problem { Problem { code, message: message.into() } } @@ -133,7 +133,7 @@ pub struct Installed { pub version: String, } -fn random_hex(n: usize) -> String { +pub(crate) fn random_hex(n: usize) -> String { let mut s = String::new(); while s.len() < n { s.push_str(&uuid::Uuid::new_v4().simple().to_string()); @@ -148,7 +148,7 @@ pub(crate) fn bootstrap_database(root: &Path) -> Result<()> { with_scratch_ports(root, || bootstrap_database_inner(root)) } -fn free_port() -> Result { +pub(crate) fn free_port() -> Result { Ok(std::net::TcpListener::bind("127.0.0.1:0")?.local_addr()?.port()) } diff --git a/crates/coa-core/src/package.rs b/crates/coa-core/src/package.rs index e03b745..f0914f2 100644 --- a/crates/coa-core/src/package.rs +++ b/crates/coa-core/src/package.rs @@ -21,6 +21,12 @@ pub const DEFAULT_PART_SIZE: u64 = 1_900_000_000; /// Archive path of the one-time database credentials shipped with a base package. pub const BOOTSTRAP_CREDENTIALS: &str = "Settings/database.bootstrap.json"; +/// Folder of a Linux (Docker) package that holds the starting databases: `.sql.zst` for `auth`, `characters` +/// and `world`, in the format of the Manager's own backups. They hold the state in which every migration of the package is +/// already applied (a world database cannot be rebuilt from the repository's SQL files: some migrations are guarded and +/// only apply to the maintainers' own database). +pub const BASELINE_DIR: &str = "Database/baseline"; + /// Files that never belong in a shipped package: per-install state, logs, secrets, old binaries. pub fn excluded(rel: &str) -> bool { if rel.replace('\\', "/").to_lowercase().starts_with(".realms/") { return true; } diff --git a/crates/coa-core/src/registry.rs b/crates/coa-core/src/registry.rs index e3e79a2..e59dff4 100644 --- a/crates/coa-core/src/registry.rs +++ b/crates/coa-core/src/registry.rs @@ -14,6 +14,7 @@ use crate::fsx; pub const META_SCHEMA: u32 = 1; pub const LAYOUT_REPACK_V1: &str = "repack-v1"; +pub const LAYOUT_DOCKER_V1: &str = "docker-v1"; const META_SUBDIRS: [&str; 6] = ["manifests", "backups", "migrations", "logs", "cache", "staging"]; #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] diff --git a/crates/coa-core/src/release.rs b/crates/coa-core/src/release.rs index 427b03c..caa8323 100644 --- a/crates/coa-core/src/release.rs +++ b/crates/coa-core/src/release.rs @@ -87,6 +87,21 @@ pub fn collect_core_sql(core: &Path) -> Result> { Ok(out) } +/// Dump the three game databases of `server` (a repack or a Docker server, started if needed) into `out` as the starting +/// databases of a Linux package: `.sql.zst`. The databases must be in the state in which every migration of the +/// package is applied. Returns (kind, compressed bytes) per database. +pub fn export_baseline(server: &Path, out: &Path) -> Result> { + fs::create_dir_all(out)?; + crate::backup::with_database(server, |db| { + let mut done = Vec::new(); + for (kind, schema) in crate::db::SCHEMAS { + let (bytes, _) = db.dump_to(schema, &out.join(format!("{kind}.sql.zst")))?; + done.push((kind.to_string(), bytes)); + } + Ok(done) + }) +} + /// SQL a bots module checkout ships for the characters database (its `dist/sql`). pub fn collect_bots_sql(bots: &Path) -> Result> { let mut out = Vec::new(); diff --git a/crates/coa-release/src/main.rs b/crates/coa-release/src/main.rs index 2956080..7551547 100644 --- a/crates/coa-release/src/main.rs +++ b/crates/coa-release/src/main.rs @@ -6,6 +6,7 @@ //! coa-release extract-schema-base --package DIR --fixture NEW-coa-schema-fixture-DIR //! coa-release schema-contract --repack FIXTURE --tree DIR --core DIR [--bots DIR] [--repairs DIR] //! coa-release clean-base --repack DIR --core DIR --tree DIR --out DIR [--bots DIR] [--data DIR] +//! coa-release export-baseline --server DIR --out DIR (the three databases of a prepared server, for a Linux package) //! coa-release sign --dir DIR (key: env COA_SIGNING_KEY, or ~/.coa-manager/signing/manifest-signing.key) //! coa-release verify --dir DIR (against the public key built into this tool) @@ -93,6 +94,12 @@ fn run() -> Result<(), String> { coa_core::cleanbase::build(&coa_core::cleanbase::Params { repack: &repack, core: &core, bots: bots.as_deref(), tree: &tree, data: data.as_deref(), out: &out }, &|s| eprintln!("{s}")).map_err(e)?; println!("clean base tree at {}", out.display()); } + "export-baseline" => { + let done = coa_core::release::export_baseline(&PathBuf::from(need(&a, "server")?), &PathBuf::from(need(&a, "out")?)).map_err(e)?; + for (kind, bytes) in done { + println!("{kind}.sql.zst: {:.1} MB", bytes as f64 / 1e6); + } + } "sign" => { let dir = PathBuf::from(need(&a, "dir")?); require_contract_entry(&dir)?; diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 3634aca..80a4ff9 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -374,9 +374,46 @@ hand over to `coa_core::docker`; nothing else needed to change for start, stop, commands do on the host (password through `MYSQL_PWD`, never on a command line). Nothing else changes: accounts, backups, the population query, migrations and the RA service account all go through `Db`, and the host needs no MySQL client. A repack keeps using its bundled tools. -* **Not done yet**: installation and the database setup (schemas, `acore` user), updates, port conflict detection, +* **Health of the database container** is a ping over TCP, like every connection the Manager makes. On a new volume the + image first runs a temporary server that answers on its socket but takes no network connection; a socket ping reported + the container healthy about four seconds before TCP worked (measured: 10 s against 14 s). +* **Main configuration files**: a package carries only `worldserver.conf.dist` and `authserver.conf.dist` (the release + tool leaves active files out). On Windows the launcher writes them at each start; on Docker `ensure_main_configs` creates + them from the templates when they are missing, at installation and at every start, and never overwrites one. +* **Installing a new server** (`docker/install.rs`, separate from `install::install_base`, which is not touched). Same + order of events: signed manifest, space and folder checks, verified download, extraction into `.installing`, + database, then one rename. The package must hold `Core/worldserver`, `Core/authserver`, the `.dist` templates, + `_migrations/` and **`Database/baseline/{auth,characters,world}.sql.zst`**. The baseline is a dump, in the format of the + Manager's backups, of databases in which **every migration of the package is already applied**; the installer imports + it, records those migrations as applied (`baseline = 1`, a few bulk statements) and creates the `acore` account, the + realm port and the console account. It is the same arrangement as a Windows package, which ships its prepared database. + The databases are **not rebuilt from the repository's SQL**: some migrations are guards generated from the maintainers' + live database (`rev_20260903_01_live_class_baseline` refuses to run unless the table holds exactly the rows it expects), + so replaying the SQL files from the base scripts fails, in any order. `coa-release export-baseline --server DIR --out DIR` + produces the dumps from a prepared server (repack or Docker). The game data stays where the person has it + (`dataDir` in `docker.json`, mounted read-only). A failure removes the containers, the volume and the staging folder. +* **Not done yet**: wiring the installer into the screens, downloading the game data, updates, port conflict detection, the firewall and exposure checks, the Wildcard realm profiles, and the client under Wine/Proton. +### D14. Keeping Windows and Linux apart + +A fix for one platform must not be able to break the other. The rules: + +1. **Platform code lives in its own modules**: `coa_core::docker` for Linux and Docker; the repack code (`driver`, `process`, + `install`, `layout` and the bundled-tool half of `db`) for Windows. A new Linux feature is a new file there, not an edit + of a repack function. +2. **Shared modules hold platform-neutral logic only** (manifest, signing, download, extraction, migrations, backups, + configuration editing, the registry). +3. **The difference enters in as few places as possible**: the early `is_docker` return at the top of `driver::run`, + `process::observe`, `layout::scan` and `Db::from_repack`, and the platform checks of the Tauri layer. Everything past + that point belongs to one platform. +4. **No `cfg` in the middle of a shared algorithm.** Where one is unavoidable (file modes, a path comparison), it is small, + commented, and the other platform's branch is the code that was there before. +5. **Reuse by calling, not by editing**: a Linux feature that needs a helper from a Windows-side file makes it + `pub(crate)` and calls it (visibility only); it does not change what the helper does. +6. **Both platforms are tested on every pull request** (`check.yml`: Windows and Linux). Tests of one platform's behaviour + are marked for that platform, so they cannot fail on the other. + --------------------------------------------------------------------------------------------------------- ## 3. Flows