Upgrade a deployment by hand

Previous: certificates and credentials · Manual setup

No Scribe service converts data when it starts: a release that needs a newer format refuses the old one and names the command that migrates it. An upgrade is therefore always the same sequence: check, back up, stop, install the new release, migrate explicitly, start, verify.

scribe-ctl upgrade runs this sequence for a cluster it manages, natively or in Docker, including the web interface, and rolls back by itself on failure; see upgrade to a new release. This guide is for deployments you run by hand (bare metal, Docker Compose, web interface).

Scribe is unreleased: formats may change between development builds, and a migration exists only for the conversions listed in format versions. Finish pending submits and branch operations with the old build where you can, and keep the old executables or images until the new release has proved itself.

1. Stop

With a web interface, first run the new release's migrate.js --check while the old one still serves (web interface). Then stop the web interface, the gateway, every storage server, and auth last, each gracefully (Ctrl-C, SIGTERM or its shutdown-request file).

2. Back up

Take backups the old release can restore (everything a deployment keeps is listed in what to back up):

• Each storage root: scribe-server backup --data-dir ./data/storage-a --to ./backups/2026-10-01/storage-a into a new directory; it hashes every file and records each repository's head (backup).
• Auth: copy the stopped data/auth directory, and separately a copy of the revision witness taken at the same moment. The pair is for this upgrade's rollback only (the exception): delete the witness copy once the new release has proved itself, since a backup must not carry its own witness. A migration does not change the pepper; it only needs its usual separate backup.
• The gateway ledger: copy the stopped data/gateway directory.
• The web database and object store: see web interface below.

3. Install the new release

Install the new executables into a directory of their own, so the old ones stay where they are: scripts/install.sh --bin-dir DIR (scripts/install.ps1 -BinDir DIR on Windows) replaces the files it finds in its destination, so name a new one, or copy the old executables aside first. Then point your service definitions (or PATH) at the new directory. For Docker, build the new images with a new tag and change the image: lines of compose.yaml to it:

docker build -f docker/Dockerfile.storage -t scribe/storage:0.2.0 .
docker build -f docker/Dockerfile.auth -t scribe/auth:0.2.0 .
docker build -f docker/Dockerfile.gateway -t scribe/gateway:0.2.0 .

Every executable reports its release and build with --build-info; use one release for every service.

4. Migrate auth

With the new scribe-auth and auth stopped (the command takes its lock), check first, then migrate. A real run names the witness, which a migration rewrites:

scribe-auth migrate --data-dir ./data/auth --dry-run
scribe-auth migrate --data-dir ./data/auth --revision-file ./secrets/auth.revision

./data/auth is already at auth schema 4; nothing to do

A dry run validates every record and writes nothing; a directory that is already current reports that there is nothing to do. The conversion from schema 3 replaces every credential and needs more options (upgrading the auth data directory).

5. Migrate each storage root

With the new scribe-server:

scribe-server migrate --data-dir ./data/storage-a --dry-run
scribe-server migrate --data-dir ./data/storage-a
scribe-server doctor --data-dir ./data/storage-a --quick

nothing to migrate: selected target is current
nothing to migrate: selected target is current
...
doctor: healthy (1 repositories, 0 problem(s), 0 warning(s)) in 4 ms

Repeat for every storage root. An interrupted migration resumes when run again; a completed one is a no-op. What each step converts is in scribe-server migrate.

The gateway ledger has no migrate command. If the new scribe-gateway refuses the ledger's format, keep the old gateway release until a procedure for that change exists; the ledger is small, and its routes can be checked against each store's catalog.

6. Start and verify

Start auth, then the storage servers, then the gateway, then the web interface. Check that a client lists its repositories and syncs through the usual endpoint, that scribe-gateway status shows fresh inventory from every instance, and that the web interface answers /api/v1/health.

If the new release fails before it serves, stop it, put the old executables or image tags back and, for a migrated root, restore the backup: scribe-server restore for storage, and for auth the copied directory together with the witness copied with it, which is allowed only as long as nothing has changed at auth since (upgrade rollback: the one exception).

Docker Compose

The same steps run through the stopped services; the data roots are the host directories the services mount (Docker Compose layout). Stop the project and back it up with the old images still named in compose.yaml, into a new directory per upgrade:

docker compose stop
mkdir -p backups/2026-10-01/storage-a backups/2026-10-01/storage-b
docker compose run --rm --no-deps -v ./backups/2026-10-01/storage-a:/backup storage-a backup --data-dir /data --to /backup
docker compose run --rm --no-deps -v ./backups/2026-10-01/storage-b:/backup storage-b backup --data-dir /data --to /backup
cp -Rp data/auth backups/2026-10-01/auth
cp -p secrets/auth/auth.revision backups/2026-10-01/auth.revision
cp -Rp data/gateway backups/2026-10-01/gateway

One backup line per store; the witness copy is the one step 2 describes. Then build the new images with a new tag (step 3), change the image: lines, and migrate with them:

docker compose run --rm --no-deps auth migrate --data-dir /data --dry-run
docker compose run --rm --no-deps auth migrate --data-dir /data --revision-file /witness/auth.revision
docker compose run --rm --no-deps storage-a migrate --data-dir /data --dry-run
docker compose run --rm --no-deps storage-a migrate --data-dir /data
docker compose run --rm --no-deps storage-a doctor --data-dir /data --quick
docker compose up -d --wait

Repeat the storage lines for every store. To roll back (step 6), stop the project, put the old tags back in compose.yaml, keep each migrated root aside and restore into an empty directory at the same place:

docker compose stop
mv data/storage-a data/storage-a.upgraded && mkdir data/storage-a
docker compose run --rm --no-deps -v ./backups/2026-10-01/storage-a:/backup:ro storage-a restore --from /backup --to /data
mv data/auth data/auth.upgraded && cp -Rp backups/2026-10-01/auth data/auth
cp -p backups/2026-10-01/auth.revision secrets/auth/auth.revision
docker compose up -d --wait

Web interface

The web database has its own versioned migrations, run by the new worker image's migrate.js. From apps/web in the new release's checkout, with the same -f files as up:

docker compose -f compose.yaml -f compose.scribe.yaml build
docker compose -f compose.yaml -f compose.scribe.yaml run --rm worker node apps/worker/dist/migrate.js --check

--check and --dry-run connect read-only and are safe while the old release still serves: --check exits 0 when the new release can start on the stored data and 3 when a migration it requires is missing. Then stop the services, back up the database and object store volumes, and migrate:

docker compose -f compose.yaml -f compose.scribe.yaml stop
mkdir -p backups/web
docker compose -f compose.yaml -f compose.scribe.yaml run --rm --no-deps --user 0 \
  -v ./backups/web:/backup --entrypoint tar database -C /data -cf /backup/database.tar .
docker compose -f compose.yaml -f compose.scribe.yaml run --rm --no-deps --user 0 \
  -v ./backups/web:/backup --entrypoint tar objects -C /data -cf /backup/objects.tar .
docker compose -f compose.yaml -f compose.scribe.yaml run --rm worker node apps/worker/dist/migrate.js --dry-run
docker compose -f compose.yaml -f compose.scribe.yaml run --rm worker node apps/worker/dist/migrate.js
docker compose -f compose.yaml -f compose.scribe.yaml up -d --wait

A migration that raises a stored version is one-way: going back means the old images and the database restored from that archive (tar -xf into the emptied volume the same way). The search index and preview files are rebuilt by the new release; the objects volume (run bundles) is not migrated, and its archive is the only other copy of what flow runs published. With WEB_OBJECTS_DIR set, copy that host directory instead. Keep the deploy/local/secrets/ files with both archives.