Documentation/Run your server
Backup & restore4 min read

Versions, migration, backup, and restore

On this page

All commands here take the exclusive data-root lock and refuse to run while scribe-server serve or another maintenance command owns the root.

Versioned things

componentversionwhere recorded
TCP protocol2 (protocol 3 = same frames with a TLS/auth hello)Hello handshake
auth protocolSAU1every auth frame
object envelope1every stored object
manifest / change set1 / 1payload headers
commit1 (ordinary), 2 (merge, two parents)payload header
object store layout1objects/SCRIBE_FORMAT
repository schema3 (serving requires it; 0, 1, 2 are migratable)repos/<id>/SCRIBE_FORMAT, meta.schema
catalog schema1catalog.redb meta
auth-state schema1auth-state.redb meta; root required feature auth-ownership-v1
workspace schema2 (1 is upgraded on the first branch, merge, or resolve operation).scribe/SCRIBE_FORMAT
chunking profile1 (FastCDC 2020, 1/4/16 MiB)every manifest
database engineredb-4descriptors
backup manifest1BACKUP_MANIFEST
metadata exportSCMX 1export file header

Startup inspects the data-root descriptor and refuses unknown kinds, newer versions, or unknown required features. It never migrates silently.

scribe-server backup --to <empty dir>

Copies every durable file: descriptors, catalog.redb, auth-state.redb when present, and for each repository SCRIBE_FORMAT, metadata.redb, objects/SCRIBE_FORMAT, objects/locator.redb, objects/loose/**, and objects/packs/**. Each copy is flushed, source and copy are hashed, and BACKUP_MANIFEST records per-file BLAKE3 and size, the source incarnation, and each repository's head change and commit id. Staging, LOCK, and runtime files are not copied. The destination must be empty. Run doctor first; a backup of a corrupt root is a corrupt backup.

The auth service is backed up separately: stop it and copy its whole data directory, keeping the administrator token file elsewhere.

scribe-server restore --from <backup> --to <empty dir>

Verifies every backup file against the manifest before copying, copies with per-file verification, assigns a new data-root incarnation and a new incarnation to every repository, writes RESTORE_MARKER, and runs structural verification. Clients whose workspaces remember the old incarnation perform a full reconcile on their next sync. Restoring an older backup changes history: any change accepted after the backup is gone, and clients that had it see base mismatches, which is the intended honest outcome. Repository ids and therefore auth grants are preserved; a restore is recovery of the same repository, not a clone.

scribe-server export-metadata --repo <name> --to <file>

Engine-neutral logical dump: "SCMX", version 1, repository id, then every table as (key, value) byte rows, followed by a BLAKE3 trailer over the whole body. Values are the Scribe record encodings (little-endian, versioned), so a future engine or redb major version can import them without reading redb pages. It is not a backup: objects are not included.

scribe-server import-metadata --repo <name> --from <file>

Requires the repository directory to have its object store but no metadata.redb. Builds a new database from the export, renames it into place, and runs structural verification. A tampered export fails the checksum and nothing is written.

scribe-server migrate

sh
scribe-server migrate --data-dir DIR --dry-run                # report only
scribe-server migrate --data-dir DIR                          # repository schema 0 -> 1
scribe-server migrate --data-dir DIR --branches [--dry-run]   # repository schema 1 or 2 -> 3
scribe-server migrate --data-dir DIR --rollback               # undo a 0 -> 1 migration if nothing was accepted since

Schema 0 to 1 (the pre-branch layout: no branch prefix in keys, no branches table, scalar head and lock generation). Per repository: read the legacy database; write metadata.redb.gen2 with re-keyed tables, a generated main branch id, the current_fold index, and the repo-meta record, then checkpoint converted; verify the new generation against the source (head, row counts, every current path, every commit), then checkpoint verified; activate by renaming metadata.redb to metadata.redb.gen1 and gen2 into place, rewriting the descriptor, writing MIGRATED, and removing the checkpoint. Then the data-root descriptor is rewritten. Every step restarts from its checkpoint; the active generation is decided by descriptors and file names, never timestamps. metadata.redb.gen1 stays until the operator deletes it. --rollback swaps it back only while the head still equals the head recorded at migration; after any accepted change it refuses. It also refuses schema 2 or 3 roots, because branch creation and rename change accepted metadata without advancing the change number.

Schema 1 or 2 to 3 (--branches). Adds the branch head, merge parent, branch operation, and pending snapshot tables and builds the historical collision index fold_history_v3 in 1,024-row durable transactions. A descriptor written before the database upgrade fences older binaries; the schema marker advances only after the whole index is durable. Rerunning after an interruption is idempotent. Existing commits and objects are unchanged. New repositories are created at schema 3, and this build serves nothing older.

Assets are never rewritten by a metadata migration; object ids and manifests are unchanged.

Placement

  • Keep at least one backup on storage independent of the server's disks.
  • The recovery window is the backup interval.
  • Rehearse restore on the real project before trusting it, and keep any pre-Scribe authority until import, history, backup, and restore have been rehearsed.
Source docs/migration-backup-restore.mdSnapshot 93d02b17