Player saves

Trace revisioned checkpoints from request to completion.

Memory is the authority. The database is a copy that catches up through one writer:

  1. The game thread marks the affected component bits and seals an immutable snapshot only when dirty work exists.
  2. The snapshot goes straight to the one persistence writer, and the character is clean again: the writer has it.
  3. The writer is a single background thread (src/player/player_save_worker.c). It applies every queued save in capture order: player saves (with their pets), corpse saves, locker saves, saved room items, shopkeeper saves, bank deltas, critical commands, flat artifact/soul captures, and the game thread's SQL (see below). A newer save of the same owner replaces its queued one only when that is the last job queued. Otherwise it is queued behind, because a job queued after the owner's save may rely on it being written first (the owner money or an item leaves is saved before the one it reaches).
  4. The game pulse consumes typed completions. A lost connection never reaches it: the writer retries that job at the head of the queue, with a backoff capped at five seconds. Any other failure is reported once, the job is dropped, and the owner is marked dirty so its next save carries the state again.

Admission is bounded to 4,096 outstanding jobs (queued, in flight, or awaiting completion consumption) and 128 MiB of declared queued/in-flight capture bytes. At capacity, submission returns unavailable with errno=ENOSPC; admitted work keeps its order and is never evicted. A tail replacement uses the old capture's slot and only its net byte increase. Dirty-state callers retry a refused capture; other callers retain their existing refusal behavior, including logged SQL drops. The byte limit bounds reported payload sizes, not total process memory. It allows one maximum artifact catalog capture; these limits are not throughput guarantees. The artifact/guild lifecycle drain consumes writer completions before retrying dirty captures, releasing admission slots while the normal game loop is stopped.

The game-thread checkpoint and completion paths perform no MySQL, Redis, or filesystem operation. Nothing is journaled. A crash loses whatever had not reached durable storage. The 30-second dirty-player-checkpoint interval is a scheduling target, not an upper bound on loss: batching, refused admission, writer retries, or an outage can leave older state unwritten. Inspect the writer's oldest age and the player's written revision when assessing durability.

Game-thread SQL

Code on the game thread that needs the database does not query it. It queues the SQL on the writer instead (src/sql/sql_async.h):

  • sql_queue() queues a write. sql_queue_statements() queues several, and sql_queue_work() queues work that reads before it writes.
  • sql_read() and sql_read_for() queue a read, which sees every write queued before it. The writer copies the rows, and the game thread gets them on a later pulse.

Each sql job runs in one transaction. A lost connection is retried, and a commit whose outcome is unknown is reported instead of retried. Whatever still queries the game thread's connection while the loop runs is counted (game_loop_queries in world persistence) and logged once per site (game loop query site ... in logs/log/status).

Boot is the exception. The writer starts only after every fatal world-data load, so until then (sql_async_boot_done() in run_the_game()) the same calls apply their SQL at once on the game thread's connection, as the rest of boot queries it. A write that fails is logged (sql job failed at boot in logs/log/file) and its call returns false. A read's callback runs on the first pulse. After the writer start, a job the writer refuses is logged (sql job not queued) and dropped.

What a save writes

Flight discoveries (learned_flight_stops) and EXP share PLAYER_COMPONENT_STATUS. A checkpoint saves both together, and unrelated component saves preserve them. Snapshot schema 9 retains field IDs and reads versions 1, 3, 5, 7 and 8. Versions before 8 receive an empty discovery mask; no transport recovery or reward journal is required. The appended last_read_news_hash status string defaults to empty for all older versions and records the SHA-256 identity acknowledged by news. It shares the status checkpoint, SQL player row, and flat-file snapshot. Other component saves preserve it. Malformed hash text becomes unread during load; structural snapshot corruption still refuses the load. An old binary cannot decode newly written schema-9 snapshots.

Opening news queues the acknowledgement through the normal save pipeline. Admission failure restores the previous hash and leaves (news) visible; accepted work follows the normal worker retry and terminal-save rules. A crash before that save lands can show the reminder again. No per-prompt save or direct SQL update is performed.

A player save writes the wallet, epic points, frags and old frags with the rest of the character; a bank change is its own bank job (see Money lives in memory and Epic points and frags live in memory). An admitted save applies its owner's captured state without a revision-conflict refusal. Submission can still be refused at the capacity bound described above. In one transaction it writes what its owner holds in memory and makes item_current_owner agree (claim_items() in src/item/item_claim_repository.c; the flat-file backend does the same in flatfile_item_repository_prepare_claim() and commits it with the player file):

  • an item with no ownership row gets one;
  • a row naming this owner is corrected if the item moved;
  • a row naming anyone else (a player, corpse, locker, room, pet, auction, shopkeeper or the collector) is taken, and an item_owner_audit row records the item, its vnum, the old owner, the new owner and the time (flat-file: a line in logs/log/item_claims). An auction listing, a sale and a collection take their items out of memory before their command, so a save captured afterwards never holds what the economy took;
  • a row that says the item was destroyed is left alone: the item and its contents are left out of the save and logged to logs/log/dupes. Only a save captured before the destruction can still hold it;
  • a coin pile is claimed like any item. A pile an older server's coin transaction spent stays spent, like any destroyed item.

The owner's revision, and the revision of each owner that lost an item, advances once per save that changes them. There is no revision fence: with one writer, every save is newer than the last one for that owner. A character with no player_data row yet gets one.

A save never releases a row: an item the owner dropped, used up or lost keeps its last holder's row until another owner's save claims it. A row whose item then stops existing (extracted by a zone reset or a shutdown, eaten, decayed, dissolved) is never claimed again, so each boot deletes them before the writer starts: every active row naming a player as the holder of an item that no payload row carries, whoever's it is: player_items, player_pet_items, locker_items, corpse_items or saved_items (reap_unheld_player_items() in src/item/item_claim_repository.c). The flat-file backend reads every player file, the locker, corpse and room stores, and what a record's player still has to be delivered from a committed transfer, which its next load materializes (flatfile_item_repository_reap_unheld_player_items()); a file or store it cannot read stops the reap. At boot no character is in memory, so such a row cannot be a held item. A row whose item an older copy still carries stays: a load skips a copy whose row names someone else (below), so after a crash that follows a hand-over the row is what keeps the giver's copy out, until that holder's next save drops the copy and the boot after it reaps the row. A row an auction's custody row, an artifact_domain_state row or another row (its contents) still references is kept for the foreign keys; the pass repeats until it deletes nothing, so a container goes after its contents. logs/log/status records the count when it is above zero. Only player-owned rows are reaped: lockers, corpses, shops, auctions and the collector keep their own. The reap does not run inside a save, where a delete that met a foreign key would fail the save (ADR 0002).

logs/log/dupes has one line per item a save left out or a load skipped, naming the item, its vnum, the owner that lost it and the owner that has it.

What a load reads

A load takes an item row only if item_current_owner has no row for that uid or names the loading owner, whatever the row's state. A row naming anyone else makes the item a stale or duplicate copy: it is skipped, logged to logs/log/dupes as load_skipped, and the owner's next save no longer writes it. What a skipped container holds moves up a level (to the top of the inventory, the corpse, the locker chest or the room item), so one stale row never takes the rest of the graph with it. A root that disagrees with the graph is corrected, not refused. The same rule applies to player and pet items (player_load_repository.c; flat-file: flatfile_player_repository.c), and to corpses, lockers and saved room items through sql_persistence_item_owner_matches_identity(). The flat-file corpse and room loaders still use their own reconciliation until those owners are saved through the writer. A player's row with no payload row of that player is only counted; the boot reap above deletes it once no payload carries the item. The count is logged on login (missing_payload_rows) only with DURIS_PERSISTENCE_TRACE, as is the save's unowned_object line for each held item the in-memory ledger has no entry for: under the claim model that is every item picked up since the last login, until the next login.

Stale ownership rows alone do not refuse the load; matching rows pass the filter. Malformed required components, storage errors, worker capacity, or a deadline can still refuse or time out the load. There is no degraded admission, stale-row refusal threshold, or read-only quarantine. Logins never block the game loop: the account menu submits the load and the descriptor waits in CON_PLAYER_LOAD until the load worker answers. If the worker refuses the request, the player is asked to try again. Copyover restore, which runs before the game loop starts, is the only caller that still waits for a load.

A character is not loaded while it still has a save queued on the writer, or while a staff target fence holds it (player_save_pipeline_load_held()): the load worker moves it to the back of its queue and loads the others meanwhile, so a quick relog always reads the latest state. A held load that reaches its deadline is answered timed_out. A locker cannot be reopened while its save slot is dirty or in flight.

Configuration And Health

The writer needs no configuration. world persistence reports the pipeline (marks, captures, replacements, unchanged checkpoints, write failures, terminal waits, drain failures) and the writer (queued and in-flight jobs, bytes, oldest age, high-water marks, connection retries, failures, capture-to-apply and apply latency). Output contains no player identity or snapshot value.

Persistence reporting severity

persistence_report(severity, level, domain, owner, item_uid, event_id, action, format, ...) separates event severity from the immortal audience selected by level:

Severity Structured outcome Routing
persistence_severity::ok outcome=ok LOG_FILE and LOG_WIZ file records
persistence_severity::info outcome=info LOG_FILE and LOG_WIZ file records
persistence_severity::alert outcome=alert Both file records and the existing red immortal broadcast

persistence_alert(...) remains an alert-only compatibility entry point. Unknown severity values also alert. Both entry points use the same category sanitization and numeric-only detail filtering; owner, item UID and event ID arguments are omitted from the output at every severity.

Use ok after a successful durable operation and info for expected progress. A death is quiet unless its terminal save fails (terminal_save_failed alerts); the character leaves at once either way.

A successful deferred-save flush uses ok and a failed one alerts. Failed I/O, rejected mutations, dropped or undrained work and unavailable workers alert even when a recovery path is available.

File delivery runs on a dedicated worker started during boot. Admission uses a fixed 128-record queue and a try-lock: the game loop never opens, writes, closes, or waits for a reporting file. Each record is bounded to 4095 bytes; the reporter bounds numeric details to 1023 bytes and the formatted event to 2047 bytes. Formatting may truncate long numeric details. The worker receives only copied text and enqueue time; it never accesses characters, descriptors, or the legacy logit formatter.

A full or contended queue rejects the new file record and increments rejected; there is no synchronous fallback. Alert broadcasts still run immediately even if file admission fails. world persistence exposes cumulative accepted/completed, rejected, and independent file/wiz failure counters. The game pulse broadcasts a reporting-delivery alert when failures increase or pending delivery makes no progress for 30 seconds, at most once per 30 seconds. These notices do not re-enter the queue. Counters remain inspectable when no immortal was online for the notice.

The worker creates missing parent directories, opens each sink with append and close-on-exec, and accepts only regular files. Open, short-write, write, and close failures are counted per sink; the other sink is still attempted. No ambiguous write is replayed, so a partial write may leave a truncated record. Rename/create rotation is supported: a record goes to the file opened for that append and subsequent opens follow the new path. Use rename/create rotation; concurrent copytruncate cannot promise lossless records. The two sinks are independent, not an atomic transaction.

Shutdown and copyover wait up to three seconds for queued and in-flight attempts. A timeout reports possible diagnostic loss to stderr and does not veto authoritative save/recovery gates. A failed exec leaves the worker available. Worker storage lives until process exit, avoiding an unbounded destructor join if filesystem I/O hangs. These diagnostic records are not a durable gameplay journal: no fsync or crash replay is promised. Failure counts are also printed on ordinary shutdown.

Validate bounded admission, blocked I/O, independent sink failures, rotation, and drain behavior with python3 tests/async/test_persistence_log.py.

Validate routing and privacy with python3 tests/async/test_persistence_severity.py.

Terminal Saves And Process Drain

Logging out never waits. Camp, rent, quit, death, idle and link-loss cleanup capture a full save with the terminal intent and room, queue it on the writer like any other, and extract the character at once (persistence_save_character_terminal() in src/cmd/actoth.c). A save that cannot be queued is reported; the character still leaves. On flat-file only, a new character's first save is written before its domains are read back, within five seconds.

Shutdown queues every player's save, gives the writer 30 seconds for what is queued and always goes; a write it could not finish is named in the log. Copyover waits the same 30 seconds and is called off, resuming the game loop, when the writer cannot drain. world persistence reports the writer's queue, retries, failures and drain failures without player identity.

Deferred and manual saves

src/cmd/actoth.c::persistence_pulse_character_saves() services deferred capture and manual acknowledgement checks from the game-loop persistence path, independent of world-event debt. It attempts at most 32 due deferred saves per call with a round-robin cursor and checks the bounded 512-slot manual-status table. Deferred slots use monotonic due times and character runtime identities so storage reuse cannot apply work to a different character. save says Save complete once the writer has written the save: the writer's completion advances the player's written revision (player_revision_record_written()). The revision the pipeline acknowledges when the writer accepts a save only marks the character clean. A save not written within 30 seconds is reported as failed.

Terminal saves are queued and the character leaves at once (see above). These guarantees do not prevent legitimate storage timeouts or operating-system starvation. The controlled retry/crash modes are documented in Testing.

Player deaths

A death happens at once: make_corpse() moves the player's items into the corpse in memory, the corpse save claims them, the player's save follows, and the character is extracted. There is no recovery hold, corpse handoff batch, or disputed-death disposition. The wallet becomes a coin pile in the corpse; the player's save, with the wallet empty, is queued before the corpse's, so a crash between them can lose the coins but never leave them in both places. See the persistence reset decision and tests/async/test_deaths_happen_at_once.py.