# Upgrade a deployment by hand

Previous: [certificates and credentials](/docs/manual/certificates.md) · [Manual setup](/docs/manual.md)

No Scribe service converts data when it starts: a release that needs a newer
format refuses the old one and names the command that migrates it. An upgrade
is therefore always the same sequence: check, back up, stop, install the new
release, migrate explicitly, start, verify.

`scribe-ctl upgrade` runs this sequence for a cluster it manages, natively or
in Docker, including the web interface, and rolls back by itself on failure;
see [upgrade to a new release](/docs/cluster-usage.md#upgrade-to-a-new-release).
This guide is for deployments you run by hand ([bare metal](/docs/manual/bare-metal.md),
[Docker Compose](/docs/manual/docker-compose.md), [web interface](/docs/manual/web-interface.md)).

Scribe is unreleased: formats may change between development builds, and a
migration exists only for the conversions listed in [format
versions](/docs/migration-backup-restore.md#format-versions). Finish pending
submits and branch operations with the old build where you can, and keep the
old executables or images until the new release has proved itself.

## 1. Stop

With a web interface, first run the new release's `migrate.js --check` while
the old one still serves ([web interface](/docs/manual/upgrades.md#web-interface)). Then stop the web
interface, the gateway, every storage server, and auth last, each gracefully
(Ctrl-C, SIGTERM or its `shutdown-request` file).

## 2. Back up

Take backups the old release can restore (everything a deployment keeps is
listed in [what to back up](/docs/migration-backup-restore.md#what-to-back-up)):

- **Each storage root:** `scribe-server backup --data-dir ./data/storage-a
  --to ./backups/2026-10-01/storage-a` into a new directory; it hashes every
  file and records each repository's head
  ([backup](/docs/migration-backup-restore.md#scribe-server-backup---to-empty-dir)).
- **Auth:** copy the stopped `data/auth` directory, and separately a copy of
  the revision witness taken at the same moment. The pair is for this
  upgrade's rollback only ([the
  exception](/docs/migration-backup-restore.md#upgrade-rollback-the-one-exception)):
  delete the witness copy once the new release has proved itself, since a
  backup must not carry its own witness. A migration does not change the
  pepper; it only needs its usual separate backup.
- **The gateway ledger:** copy the stopped `data/gateway` directory.
- **The web database and object store:** see [web interface](/docs/manual/upgrades.md#web-interface)
  below.

## 3. Install the new release

Install the new executables into a directory of their own, so the old ones
stay where they are: `scripts/install.sh --bin-dir DIR` (`scripts/install.ps1
-BinDir DIR` on Windows) replaces the files it finds in its destination, so
name a new one, or copy the old executables aside first. Then point your
service definitions (or `PATH`) at the new directory. For Docker, build the
new images with a new tag and change the `image:` lines of `compose.yaml` to
it:

```sh
docker build -f docker/Dockerfile.storage -t scribe/storage:0.2.0 .
docker build -f docker/Dockerfile.auth -t scribe/auth:0.2.0 .
docker build -f docker/Dockerfile.gateway -t scribe/gateway:0.2.0 .
```

Every executable reports its release and build with `--build-info`; use one
release for every service.

## 4. Migrate auth

With the new `scribe-auth` and auth stopped (the command takes its lock),
check first, then migrate. A real run names the witness, which a migration
rewrites:

```sh
scribe-auth migrate --data-dir ./data/auth --dry-run
scribe-auth migrate --data-dir ./data/auth --revision-file ./secrets/auth.revision
```

```text
./data/auth is already at auth schema 4; nothing to do
```

A dry run validates every record and writes nothing; a directory that is
already current reports that there is nothing to do. The conversion from
schema 3 replaces every credential and needs more options ([upgrading the auth
data directory](/docs/auth.md#upgrading-the-auth-data-directory)).

## 5. Migrate each storage root

With the new `scribe-server`:

```sh
scribe-server migrate --data-dir ./data/storage-a --dry-run
scribe-server migrate --data-dir ./data/storage-a
scribe-server doctor --data-dir ./data/storage-a --quick
```

```text
nothing to migrate: selected target is current
nothing to migrate: selected target is current
...
doctor: healthy (1 repositories, 0 problem(s), 0 warning(s)) in 4 ms
```

Repeat for every storage root. An interrupted migration resumes when run
again; a completed one is a no-op. What each step converts is in
[`scribe-server migrate`](/docs/migration-backup-restore.md#scribe-server-migrate).

The gateway ledger has no migrate command. If the new `scribe-gateway` refuses
the ledger's format, keep the old gateway release until a procedure for that
change exists; the ledger is small, and its routes can be checked against each
store's catalog.

## 6. Start and verify

Start auth, then the storage servers, then the gateway, then the web
interface. Check that a client lists its repositories and syncs through the
usual endpoint, that `scribe-gateway status` shows fresh inventory from every
instance, and that the web interface answers `/api/v1/health`.

If the new release fails before it serves, stop it, put the old executables or
image tags back and, for a migrated root, restore the backup:
`scribe-server restore` for storage, and for auth the copied directory
together with the witness copied with it, which is allowed only as long as
nothing has changed at auth since ([upgrade rollback: the one
exception](/docs/migration-backup-restore.md#upgrade-rollback-the-one-exception)).

## Docker Compose

The same steps run through the stopped services; the data roots are the host
directories the services mount ([Docker Compose layout](/docs/manual/docker-compose.md)).
Stop the project and back it up with the old images still named in
`compose.yaml`, into a new directory per upgrade:

```sh
docker compose stop
mkdir -p backups/2026-10-01/storage-a backups/2026-10-01/storage-b
docker compose run --rm --no-deps -v ./backups/2026-10-01/storage-a:/backup storage-a backup --data-dir /data --to /backup
docker compose run --rm --no-deps -v ./backups/2026-10-01/storage-b:/backup storage-b backup --data-dir /data --to /backup
cp -Rp data/auth backups/2026-10-01/auth
cp -p secrets/auth/auth.revision backups/2026-10-01/auth.revision
cp -Rp data/gateway backups/2026-10-01/gateway
```

One `backup` line per store; the witness copy is the one [step
2](/docs/manual/upgrades.md#2-back-up) describes. Then build the new images with a new tag ([step 3](/docs/manual/upgrades.md#3-install-the-new-release)),
change the `image:` lines, and migrate with them:

```sh
docker compose run --rm --no-deps auth migrate --data-dir /data --dry-run
docker compose run --rm --no-deps auth migrate --data-dir /data --revision-file /witness/auth.revision
docker compose run --rm --no-deps storage-a migrate --data-dir /data --dry-run
docker compose run --rm --no-deps storage-a migrate --data-dir /data
docker compose run --rm --no-deps storage-a doctor --data-dir /data --quick
docker compose up -d --wait
```

Repeat the storage lines for every store. To roll back ([step
6](/docs/manual/upgrades.md#6-start-and-verify)), stop the project, put the old tags back in
`compose.yaml`, keep each migrated root aside and restore into an empty
directory at the same place:

```sh
docker compose stop
mv data/storage-a data/storage-a.upgraded && mkdir data/storage-a
docker compose run --rm --no-deps -v ./backups/2026-10-01/storage-a:/backup:ro storage-a restore --from /backup --to /data
mv data/auth data/auth.upgraded && cp -Rp backups/2026-10-01/auth data/auth
cp -p backups/2026-10-01/auth.revision secrets/auth/auth.revision
docker compose up -d --wait
```

## Web interface

The web database has its own [versioned
migrations](/docs/web-interface.md#stored-data-versions), run by the new worker
image's `migrate.js`. From `apps/web` in the new release's checkout, with the
same `-f` files as `up`:

```sh
docker compose -f compose.yaml -f compose.scribe.yaml build
docker compose -f compose.yaml -f compose.scribe.yaml run --rm worker node apps/worker/dist/migrate.js --check
```

`--check` and `--dry-run` connect read-only and are safe while the old
release still serves: `--check` exits 0 when the new release can start on the
stored data and 3 when a migration it requires is missing. Then stop the
services, back up the database and object store volumes, and migrate:

```sh
docker compose -f compose.yaml -f compose.scribe.yaml stop
mkdir -p backups/web
docker compose -f compose.yaml -f compose.scribe.yaml run --rm --no-deps --user 0 \
  -v ./backups/web:/backup --entrypoint tar database -C /data -cf /backup/database.tar .
docker compose -f compose.yaml -f compose.scribe.yaml run --rm --no-deps --user 0 \
  -v ./backups/web:/backup --entrypoint tar objects -C /data -cf /backup/objects.tar .
docker compose -f compose.yaml -f compose.scribe.yaml run --rm worker node apps/worker/dist/migrate.js --dry-run
docker compose -f compose.yaml -f compose.scribe.yaml run --rm worker node apps/worker/dist/migrate.js
docker compose -f compose.yaml -f compose.scribe.yaml up -d --wait
```

A migration that raises a stored version is one-way: going back means the old
images and the database restored from that archive (`tar -xf` into the emptied
volume the same way). The search index and preview files are rebuilt by the
new release; the `objects` volume (run bundles) is not migrated, and its
archive is the only other copy of what flow runs published. With
`WEB_OBJECTS_DIR` set, copy that host directory instead. Keep the
`deploy/local/secrets/` files with both archives.