# Local clusters with scribe-ctl For every controller command and option, see the [controller CLI reference](/docs/reference/cli/scribe-ctl.md). This guide covers the cluster definition, setup, and service lifecycle workflows. All command-line tools support global `--json`; see [JSON output for automation](/docs/json-output.md) for results, errors, and exit codes. `scribe-ctl` sets up and supervises named clusters from one definition file. Each cluster has one optional auth service, up to 256 storage services, and an optional gateway, run as supervised local processes, or as containers of one Docker Compose project with the web interface beside them ([Run a cluster in Docker](/docs/cluster-usage.md#run-a-cluster-in-docker)), on this machine or on another one over ssh ([Remote operation](/docs/cluster-usage.md#remote-operation)). Certificate generation is built in; no external certificate tool is required. `renew` keeps the service credentials valid ([Renew service credentials](/docs/cluster-usage.md#renew-service-credentials)). New to it? The [tutorials](/docs/tutorials.md) build a cluster step by step: one store, then auth, a gateway with a second store, everyday operation and an upgrade, and the web interface in Docker. ## Set up a new cluster Install the executables first: from a source checkout, `scripts/install.sh` (`scripts/install.ps1` on Windows) builds and installs them, and with `--setup VARIANT --config FILE` (`-Setup VARIANT -Config FILE`) hands over to `scribe-ctl setup` in the same line (see [build and install](https://github.com/Flying-Rat/scribe/blob/main/README.md#build)). Pick a variant, look at the plan, then run it: ```sh scribe-ctl --config ./scribe-cluster.toml setup --variant gateway --plan scribe-ctl --config ./scribe-cluster.toml setup --variant gateway scribe-ctl --config ./scribe-cluster.toml where scribe-ctl --config ./scribe-cluster.toml up ``` | Variant | Services | Clients connect to | | --- | --- | --- | | `storage` | One plaintext storage server; no accounts | `tcp://127.0.0.1:7447` | | `auth` | Auth and one TLS storage server | `tls://127.0.0.1:7447`, logging in at `tls://127.0.0.1:7443` | | `gateway` | Auth, one TLS storage server, and a TLS gateway in front | `tls://127.0.0.1:7557`, logging in at `tls://127.0.0.1:7443` | `setup` writes the definition (named cluster `local`, every path relative to the file, services on loopback), then creates everything the cluster needs: data roots, both server-certificate CAs (`pki/ca` for storage and gateway, `pki/auth-ca` for auth), server certificates, the auth pepper and revision witness, the bootstrap administrator credential, each storage server's credential, the gateway's `scribe-gateway` service account with its grants and credential, and a `scribe-monitor` credential that `df` and `repo list` read inventory with. `--namespace NAME` sets the authenticated server's namespace (default `studio`); `--port-offset N` moves every default port by N. Setup also records the executables it finds (beside `scribe-ctl`, or where the definition's `[binaries]` names them) as the cluster's first binary set: it checks they are one release for this machine, copies them into `/bin//`, and names that set in `/bin/current`. Services start from the set `current` names, so a later [upgrade](/docs/cluster-usage.md#upgrade-to-a-new-release) can switch releases and switch back. Last, setup writes `CLUSTER-INFO.txt` beside the definition. It lists every file and what it is for, which ones to back up, which to give clients, which to keep offline, the endpoints, where the web interface's TLS certificate goes, and the exact next commands, such as adding a person and logging in. Secrets appear there only as paths. `scribe-ctl where` prints the same summary, current, at any time; see the [reference](/docs/reference/cli/scribe-ctl.md#where) for an example. `--plan` runs every check for real (definition, binaries, permissions of existing secrets, the accounts file, whether the cluster runs, free listen ports, certificate and data-root consistency) and prints the steps with the reason for each. It writes nothing, takes no lock and starts nothing. Setup executes exactly that plan, planned again under its lock. To change ports, names or paths before anything exists, write the definition yourself (start from `scribe-ctl example` or edit the one `setup --plan` describes) and run `setup` without `--variant`: with an existing definition, setup uses it as written. A `--variant` it does not match is refused. ### Rerun, resume, and what setup refuses Setup is idempotent: a rerun on a complete cluster changes no file it made. It still starts auth briefly to confirm registrations, grants and credentials. It never replaces a credential, also one close to expiry: its plan points to [renew](/docs/cluster-usage.md#renew-service-credentials) for those. Every step is recorded in `state_dir/journal.json` before and after it runs. An interrupted setup (a crash, a killed process, a full disk) resumes when run again: finished steps are not repeated, and a step that had started is redone after its partial output is moved aside as `.interrupted-` beside the original, never deleted. If the interruption left the temporary auth running, setup refuses until `scribe-ctl down` drains it. The journal is removed when a run completes. State that setup did not make is refused, not replaced: a partial PEM pair, a CA directory without its key (bring the key back from offline storage), a data root that holds files but no format marker, an auth pepper or witness without its database, or a generated file with other contents. The message says what was found and what to do. Setup never regenerates a pepper or revision witness for an existing auth database. While the cluster runs, setup performs only its identity steps, through the running auth (for example, accounts added to `accounts_file`), and refuses steps that need the cluster stopped. ## Grow a cluster: add auth, then a gateway A cluster that started as the `storage` variant gains auth, and an `auth` cluster gains a gateway, without losing a repository: ```sh scribe-ctl --config ./scribe-cluster.toml down scribe-ctl --config ./scribe-cluster.toml add auth --plan scribe-ctl --config ./scribe-cluster.toml add auth scribe-ctl --config ./scribe-cluster.toml add gateway scribe-ctl --config ./scribe-cluster.toml up ``` Each `add` rewrites the definition canonically (the previous file is kept as `.bak`; `--plan` shows the diff, and for `add gateway` the new `scribe-gateway.toml`), then provisions the result exactly as setup would, with the same checks, journal and resume. `add auth` puts auth on `127.0.0.1:7443` (moved by as much as the store is from 7447; `--listen` chooses), creates both CAs if missing, issues the auth and store certificates, initializes auth with its pepper, revision witness and administrator credential, binds every store to its namespace (`--namespace`, default `studio`) in place, registers it and issues its credential, and provisions the `scribe-monitor` inventory account. `add gateway` writes the gateway's configuration (one instance per store), issues its certificate, initializes its ledger, and provisions the `scribe-gateway` account with reader and delegate grants and its credential. Data roots are never re-initialized or deleted: the store keeps its repositories, history and IDs, and a gateway imports them on its first inventory poll (as `studio/NAME`). The cluster must be stopped for either add; while it runs, the plan's `cluster` check fails and nothing changes. A bound store no longer starts without auth. After `add auth`, clients use `tls://` for the store with `--tls-ca pki/ca/ca.pem` and log in through auth with `--auth-ca pki/auth-ca/ca.pem`; plaintext connections are refused. Nobody has access until an administrator adds people and grants (the Next commands in `CLUSTER-INFO.txt`). A workspace created before auth has no signed-in owner: it still syncs, but submitting needs a new workspace created after logging in. After `add gateway`, clients use the gateway's address and log in once more for it, since a saved session belongs to the server it was made for. An add that finds its service complete changes nothing and says so. It refuses a cluster that is not its source variant (`add gateway` on storage only: run `add auth` first; a custom topology: change the definition by hand), options that shape a service that already exists, and a service that exists but is incomplete with no interrupted add to resume (run `setup` to complete it). See the [reference](/docs/reference/cli/scribe-ctl.md#add-auth) for a full plan. ## Upgrade to a new release Unpack the new release into a directory of its own, look at the plan, then run it with the controller you have: ```sh scribe-ctl --config ./scribe-cluster.toml upgrade --binaries /opt/scribe/0.2.0 --plan scribe-ctl --config ./scribe-cluster.toml upgrade --binaries /opt/scribe/0.2.0 ``` The release's own `scribe-ctl` runs its upgrade, because only it knows that release's migrations: when the directory holds a newer one, yours hands the command over to it and reports its result. The plan checks that the directory is one complete release for this machine (every executable answers `--build-info`), that it is not older than the active set, that no data root records a format newer than the new executables read, and which formats change. With the cluster stopped it also runs `scribe-auth migrate --dry-run` and `scribe-server migrate --dry-run` with the new executables. It measures the roots it will back up and fails its `free space` check unless the backups' filesystem has room for them plus a reserve of a tenth of their size, at least 1 GiB. `--plan` writes nothing, takes no lock and touches no service. The upgrade then installs the new set side by side in `/bin//` while the cluster still serves, drains it, backs up each root a migration will change (auth, stores) to `/backups//`, switches `bin/current`, migrates with the new executables (auth first, then each store), starts the cluster and waits until every service is ready. A cluster that was stopped is stopped again afterwards. Downtime runs from the drain to the health check. Auth's revision witness lives outside its data root and its backups, so that a restored old copy cannot revive revoked credentials unnoticed. The upgrade keeps the witness that matches auth's backup in its journal instead, and a rollback puts both back together; once the upgrade completes that copy is gone with the journal. If anything fails once a running cluster is drained (for a stopped one, from the switch on), the upgrade rolls back by itself: it stops what runs, restores each migrated root from its backup (keeping the migrated one beside it as `.upgraded-`; auth with its witness), switches back to the previous set and starts it if the cluster was running. A failed backup therefore leaves the old release serving, not a drained cluster. Each backup records a BLAKE3 digest of every file; the copy is checked against them right away, and a backup that no longer matches is refused before a restore moves anything. A crash at any point is resumed by running the same command again. Nothing is deleted: earlier sets and backups stay until you [prune](/docs/cluster-usage.md#prune-old-releases) them, and `where` lists them. `status` names the active set; use the new release's `scribe-ctl` (`/bin//scribe-ctl`, or a copy of it) from then on. Two migrations are refused with what to do by hand: auth's schema 3 to 4, which replaces every credential ([upgrading the auth data directory](/docs/auth.md#upgrading-the-auth-data-directory)), and any change of the gateway ledger, which has no migrate command. See the [reference](/docs/reference/cli/scribe-ctl.md#upgrade) for every check, a real plan and a rollback. The same sequence for a deployment `scribe-ctl` does not run is in [upgrade a deployment by hand](/docs/manual/upgrades.md). ### Prune old releases Every upgrade keeps the release it replaced, so each one leaves a binary set behind, and under Docker its images. `prune` removes what old releases left, and only what scribe-ctl made: ```sh scribe-ctl --config ./scribe-cluster.toml prune --plan scribe-ctl --config ./scribe-cluster.toml prune # --keep 1: the active set and the one before it stay ``` It removes binary sets other than the active one and the `--keep` newest before it, which stay for a rollback. Under Docker it also removes the images tagged with those sets' builds (`scribe/*:`) that no container uses; an image a container still uses stays, and the plan says which. It removes the scribe-ctl builds a remote command installed in the cluster's directory (`bin//scribe-ctl`) that no kept set, `bin/controller` or `bin/current` names, and upgrade backups older than the `--keep` newest. Docker's build cache, other images, volumes and containers belong to the machine and are never touched, since other work on it may use them. `doctor` warns when the build cache grows (`docker.build-cache`), and `docker builder prune` frees it when you choose. `renew` never prunes; you run `prune` when you choose, with `--remote` for a cluster on another machine as any command. ## Run a cluster in Docker With `--runtime docker`, setup writes `runtime = "docker"` and every service runs as a container of one Compose project that `scribe-ctl` generates and drives with the `docker` CLI. You need Docker Engine with Compose v2 and a Scribe checkout of the same version as your `scribe-ctl`: images are built from it and tagged with that `scribe-ctl`'s build. ```sh ./scripts/install.sh --setup gateway --config ~/scribe/scribe-cluster.toml -- --runtime docker --web --source . scribe-ctl --config ~/scribe/scribe-cluster.toml up scribe-ctl --config ~/scribe/scribe-cluster.toml where ``` (or, with `scribe-ctl` already installed from this checkout, `scribe-ctl --config ~/scribe/scribe-cluster.toml setup --variant gateway --runtime docker --web --source .`). [Tutorial 5](/docs/tutorials/05-start-web-interface.md) walks through it with real output. Everything else is as natively: the same definition (with a fixed port for every service and auth's pepper and witness in a directory of their own, `secrets/auth/`), the same data roots on the host, bind-mounted into the containers at fixed paths, the same plans, journal, backups and rollback. Containers reach each other by service name on the project network (`auth`, each store's name, `gateway`) and publish their ports on the definition's addresses only. Each Scribe service's container runs as the owner of the definition's directory, with a read-only root filesystem, no capabilities and no privilege escalation, and mounts only its own data root and files (a serving process never sees a CA key or the administrator credential). `scribe-ctl` itself runs on the host and provisions identities through auth's published port. `up` is `docker compose up --wait`, `down` is `docker compose stop`, `logs SERVICE` shows a container's log, and `where` prints the `docker compose` command for the project if you want to look yourself. The generated files in `/docker/` are rewritten before every start: change the definition instead. `--web` adds the web interface (apps/web) to the same project, with its database in `data/web`, its secrets in `secrets/web/`, and, with auth, the `scribe-web` service account and its reader credential. It is published on `http://localhost:8080`. Its TLS certificate is yours: put `fullchain.pem` and `key.pem` in `secrets/web-tls/`, set `[web] origin` to the `https://` address, and the next `up` serves HTTPS; see the [web interface guide](/docs/web-interface.md#run-it-with-scribe-ctl). To upgrade, update the checkout and let the install script build the new `scribe-ctl` and hand the checkout to it (it names the build by the commit, so two builds of one version differ; set `SCRIBE_BUILD_ID` to choose the name): ```sh ./scripts/install.sh --upgrade --config ~/scribe/scribe-cluster.toml -- --source . --plan ./scripts/install.sh --upgrade --config ~/scribe/scribe-cluster.toml -- --source . ``` It builds the new images first (with `--plan` too), then upgrades as natively, switching the project to the new images; with the web interface it also backs up the web database and runs the new worker's `migrate.js`, whose dry run against the running database refuses the upgrade before anything stops. The [reference](/docs/reference/cli/scribe-ctl.md#docker-runtime) has the details and real plans. Setup never moves a cluster between runtimes: an existing native cluster keeps running natively (its data roots and credentials name host addresses), and a Docker cluster starts from a new definition. ## Renew service credentials Setup issues every service credential for 365 days: each authenticated store's server credential, the gateway's (`scribe-gateway`), the inventory credential (`scribe-monitor`) and, under docker with the web interface, the web reader's (`scribe-web`); and each service's TLS server certificate for 365 days. `scribe-ctl renew` replaces each credential, the administrator credential and each server certificate that expires within 30 days, while the cluster runs. Only the web interface's own certificate (`fullchain.pem` in `[web] tls_dir`) stays yours to renew: ```sh scribe-ctl --config ./scribe-cluster.toml renew --plan # what is due, and the steps scribe-ctl --config ./scribe-cluster.toml renew ``` It issues the replacement beside the file (`.next`), keeps the current file as `.previous`, puts the new one in place, restarts the services that read their credential only at startup (the stores and the gateway, one at a time while the rest keep serving: natively the supervisor restarts them in place, under docker their containers are recreated), and proves each new credential for its purpose: a store serves a namespace read it authorizes with its new credential, the gateway polls every instance, and the inventory and web reader credentials read every namespace. Only then is each old credential revoked. If a proof fails, the old credentials go back in place, the holders restart with them, the new ones are revoked, and renew exits 1: the cluster keeps working with the old credentials until their own expiry, and the next run tries again. A stopped cluster is started for the proofs and stopped again. An interrupted renewal resumes when run again (it adopts a credential auth already issued rather than issuing a second). The administrator credential is renewed the same way (for 10 years, proved with an administrator request); the bootstrap one setup writes never expires, so it is never due. A server certificate is reissued for the key it already has, by the CA that signed it, and followed in its file by that CA (clients fingerprint the top of the chain a server presents). A reissue, a rekey and a rotation keep every name the certificate carries and add localhost, loopback, a container's service name and the names the definition lists (`tls.dns`/`tls.ip`, and for the gateway `dns`/`ip` in its table), so no name clients connect by is ever dropped. A certificate file that holds its certificate alone (set up by hand, or taken over) makes its server present no CA, so clients that do not trust it yet cannot discover it: `doctor` warns (`chain.`) and a plain `renew` writes the file again followed by its CA, the same certificate and key. Only the service it belongs to restarts, and a TLS handshake trusting the CA clients hold proves it before the kept copy is removed; clients notice nothing. The CA's key must be where renew runs: beside the cluster (`ca-key.pem` in `certificates.ca_dir` or `auth_ca_dir`, as setup writes it), or in a directory you name with `--ca-dir DIR` (the storage and gateway CA) or `--auth-ca-dir DIR` (the auth CA). When it is neither, the plan fails for that certificate and names the command. With `--remote`, the directory is on the machine you run scribe-ctl on: the certificates are signed there and only they travel to the cluster; the key never leaves. Details and real output: [renew](/docs/reference/cli/scribe-ctl.md#renew). `status` and `where` warn 30 days ahead (a CA a year ahead) and name the command. Better, run `renew` from a scheduler once a day: it renews what is due and retires CAs whose overlap ended (a rotation's old CA, auth's older internal CAs), and is a no-op (exit 0, nothing written, about a second) until something is due. It never prompts and exits 1 whenever it could not finish, always leaving working credentials in place. scribe-ctl installs no schedule itself: set one up with your system's scheduler, as the account that owns the cluster's files, and keep its output (progress on stderr, the result on stdout, or `--json`). ### Schedule renew on Linux cron, daily at 03:17: ```cron 17 3 * * * /opt/scribe/scribe-ctl --config /srv/scribe/scribe-cluster.toml renew >> /srv/scribe/renew.log 2>&1 ``` systemd, as a oneshot service and a timer. Put both files in place, then `systemctl daemon-reload && systemctl enable --now scribe-renew.timer`; `systemctl list-timers scribe-renew.timer` shows the next run and `journalctl -u scribe-renew` the last one's output: ```ini # /etc/systemd/system/scribe-renew.service [Unit] Description=Scribe: renew what is due, retire CAs whose overlap ended # A docker-runtime cluster: its containers must be up. After=docker.service network-online.target Wants=network-online.target [Service] Type=oneshot User=scribe Group=scribe WorkingDirectory=/srv/scribe ExecStart=/opt/scribe/scribe-ctl --config /srv/scribe/scribe-cluster.toml renew # /etc/systemd/system/scribe-renew.timer [Unit] Description=Daily Scribe renew [Timer] OnCalendar=daily RandomizedDelaySec=1h Persistent=true [Install] WantedBy=timers.target ``` `Persistent=true` runs a missed day's renew at the next boot. ### Schedule renew on Windows Task Scheduler, from an elevated prompt; `/ru` names the account that owns the cluster's files and `/rp *` asks for its password once: ```bat schtasks /create /tn "Scribe renew" /sc daily /st 03:17 /ru scribe /rp * ^ /tr "\"C:\Scribe\scribe-ctl.exe\" --config \"C:\Scribe\scribe-cluster.toml\" renew" ``` The same from PowerShell, with the output kept in a log: ```powershell $action = New-ScheduledTaskAction -Execute 'powershell.exe' -Argument '-NoProfile -Command "& ''C:\Scribe\scribe-ctl.exe'' --config ''C:\Scribe\scribe-cluster.toml'' renew *>> ''C:\Scribe\renew.log''"' $trigger = New-ScheduledTaskTrigger -Daily -At 3:17am -RandomDelay (New-TimeSpan -Hours 1) $settings = New-ScheduledTaskSettingsSet -StartWhenAvailable Register-ScheduledTask -TaskName 'Scribe renew' -Action $action -Trigger $trigger -Settings $settings -User 'scribe' -Password (Read-Host 'Password for scribe') ``` `-StartWhenAvailable` runs a missed day's renew once the machine is back. `Get-ScheduledTaskInfo 'Scribe renew'` shows the last result (0 when it finished). ### Schedule renew for a cluster on another machine Schedule it on the cluster's machine, where no ssh is involved. The scribe-ctl that runs its commands there is `bin//scribe-ctl` in the cluster's directory, and `bin/controller` names the build (each upgrade moves it), so the command reads it on every run; for a cluster in `/home/scribe/scribe-ctl`: ```sh /bin/sh -c 'exec /home/scribe/scribe-ctl/bin/"$(cat /home/scribe/scribe-ctl/bin/controller)"/scribe-ctl --config /home/scribe/scribe-ctl/scribe-cluster.toml renew' ``` Use that as the systemd `ExecStart=` (with `WorkingDirectory=` the cluster's directory) or as the cron command. Scheduling it here instead, with the definition that has the `[remote]` table, works too, but then this machine must be on and its ssh key usable unattended from the scheduler's account. A CA key kept on this machine (`--ca-dir`) is needed only in the months a server certificate is due; the schedule can name it every time. ### Auth under the storage and gateway CA A deployment taken over may have auth's certificate signed by the storage and gateway CA, with `[auth] ca` naming that CA's `ca.pem` (`doctor` says so: `ca.auth-shared`). That CA then renews and rekeys auth's certificate too, and `renew --rotate-ca storage` rotates it for auth as well: auth's certificate moves to the new CA with the stores' and the gateway's, and the credentials that carry the CA they trust auth by follow. There is no auth CA of its own to rotate, so `--rotate-ca auth` is refused with that command. ### CA fingerprints for clients `where` shows, under each `tls://` endpoint, the fingerprint of the CA clients verify it by (`CA BLAKE3:…`): give it to people with the address, for `scribe trust add --trust-fingerprint`. ### Replace server keys A plain renew keeps each server's private key. `renew --rekey` replaces it: for a key you suspect has leaked, or to move servers set up before Scribe generated ML-DSA-65 keys to a post-quantum one. It is never done by a scheduled `renew`, only by you: ```sh scribe-ctl --config ./scribe-cluster.toml renew --rekey storage-a --plan scribe-ctl --config ./scribe-cluster.toml renew --rekey storage-a,gateway scribe-ctl --config ./scribe-cluster.toml renew --rekey # every TLS server ``` Each named service (auth, a store with TLS, the gateway) gets a new private key, generated where the cluster runs as `.next` (only the cluster's account can read it), and a certificate for it from the CA that signs the server now, for the same names; during a rotation's overlap it is followed by the cross certificate, as a reissue is. The new key and certificate go in place together, the old ones kept as `.previous`; only that service restarts, and a TLS handshake trusting the CA clients hold proves the new pair before the old key is removed. If the proof fails, the old key and certificate go back together and the service restarts with them. Clients trust the CA, not the key: they notice nothing and trust nothing again, and sessions and credentials are unaffected. A name that is not a TLS server of the cluster is refused with the ones that are. With the CA key on another machine (`--remote` with `--ca-dir DIR`), the new key is still made on the cluster's machine and never leaves it: only its public key travels to be certified, and only the certificate comes back. `doctor` warns about a `.next` or `.previous` file left beside a certificate or key with no renewal pending; a staged `.next` is what the next `renew --rekey` for that service uses. ## Rotate a CA The two server-certificate CAs (the storage and gateway CA, and the auth CA) are valid for 10 years. `status`, `doctor` and `renew` say so a year before one expires; a rotation replaces it without asking clients to trust anything by hand, as long as they connect during an overlap. It is never started by a scheduled `renew`, only by you: ```sh scribe-ctl --config ./scribe-cluster.toml renew --rotate-ca storage --plan scribe-ctl --config ./scribe-cluster.toml renew --rotate-ca storage --overlap-days 90 ``` A rotation makes a new CA and a *cross certificate*: the new CA's name and key, signed by the old CA and valid for the overlap (90 days by default, never past the old CA). Every server certificate of that CA is reissued by the new CA for the key it has, in one run, and presented followed by the cross certificate, so a client that trusts only the old CA still verifies it, and learns the new CA as it does; a client given the new CA verifies it directly. Before any server changes, the web interface's services (which follow no cross certificate) are restarted trusting both CAs. The new CA then becomes `ca.pem` (the old one moves to `previous/`), the trust files the definition names follow, and for the auth CA every credential file ctl manages (servers', the gateway's, the inventory and web reader credentials, the administrator's) is rewritten to carry the new auth CA; the services that read them restart. Each step is proved with TLS handshakes trusting the old and the new CA alone, and a failure puts the old CA back. Give new clients the new `ca.pem`. Why server CAs have path length 1, and why an older CA with path length 0 still rotates, is in [the four certificate authorities](/docs/manual/certificates.md#the-four-certificate-authorities). `status` shows the rotation and the day its overlap ends. From that day the scheduled plain `renew` retires the old CA: each certificate is followed by the new CA itself, the old CA leaves every trust file, the services restart one at a time, and the old CA with its key is archived as `retired--/` in the CA directory (never deleted: move it offline or remove it yourself). `renew --retire-ca storage` does it before the end, with a warning: clients that have not connected since the rotation started then see "CA changed" with both fingerprints and must trust the new CA again ([CA rotation and handover](/docs/auth.md#ca-rotation-and-handover)). Credential files ctl does not manage (people's machines, CI) carry the auth CA too: after an auth CA rotation retires, issue them new ones. With the CA key kept away from the cluster, name its directory: `renew --rotate-ca storage --ca-dir DIR` creates the new CA in `DIR/next/`, signs there, and after the run makes it current in DIR too (the old one in `DIR/previous/`, archived when the rotation retires with `--ca-dir DIR`). Auth's two internal client CAs (credential and access, 20 years; only Scribe's services trust them, clients never see them) rotate with `renew --rotate-ca credential|access`: the cluster drains for a moment while auth gains a successor, which every service trusts from its start and which signs 15 minutes later. Nobody has to do anything meanwhile: people who sign in get a session from the new CA. A plain `renew` (the scheduled one) retires the old CA once its successor has signed for the overlap: a day for the access CA, `--overlap-days` (90, a session's default lifetime) for the credential CA. It renews the credentials the cluster holds first, so the new CA signs them, then retires the old CA and revokes the sessions and credentials people still hold from it: they sign in again. `renew --retire-ca credential|access` does the same now. Only an administrator credential is never revoked this way: if every one comes from the old CA, the retirement is refused until one is renewed. `doctor` shows both CAs, their keys and the day the old one retires. ## Remote operation `scribe-ctl` operates a cluster on another machine over the system `ssh`: it makes sure the remote has a `scribe-ctl` of this release, pushes what the command needs, and runs the same command there, passing its output and exit code through. Discovery, planning, locks, journals and execution all happen on the remote; the definition file here is the source of truth, and the secrets stay there. It replaces hand-written deploy scripts that copy a checkout to a VM, rebuild, migrate, renew credentials and open a shell. Name the machine in the definition (or pass `--remote` to any command): ```toml [clusters.prod.remote] host = "scribe-host" # a host name, an address, or an alias from ~/.ssh/config user = "deploy" # optional port = 22 # optional directory = "/srv/scribe" # holds the cluster there; absolute, or relative to the login's home ``` Only keys, the agent and `~/.ssh/config` are used: ssh runs in batch mode and never asks for a password or to accept a host key. Make `ssh scribe-host true` work without a prompt first (`ssh-copy-id`, connect once to accept the host key, or a `Host` alias with `User`, `Port` and `IdentityFile`). Private details such as host names belong in your own definition file, never in shared ones. A new cluster there: ```sh # A native cluster: executables built for the remote's OS and architecture, # or none when this machine matches it (this scribe-ctl's directory is pushed). scribe-ctl --config ./prod.toml --cluster prod --remote deploy@scribe-host:/srv/scribe setup --variant gateway --binaries ./linux-x86_64 # A docker cluster: the checkout is pushed, scribe-ctl and the images are built there. scribe-ctl --config ./prod.toml --cluster prod --remote deploy@scribe-host:/srv/scribe setup --variant gateway --runtime docker --web --source . scribe-ctl --config ./prod.toml up ``` Setup writes the definition there and brings it back here with its `[remote]` table, plus the summary as `CLUSTER-INFO-prod.txt`; after that every command (`up`, `status`, `logs`, `df`, `repo list`, `access`, `renew`, `upgrade`, `add`, `down`, `where`) goes there without `--remote`. Setup's layouts listen on the remote's loopback. For clients on other machines, write the definition here first (from `scribe-ctl example`), with the listen addresses, the names and addresses clients use in each `[tls]` table's `dns` and `ip`, and the `[remote]` table; `setup` pushes it and sets it up as written. Edits here are pushed before every command (the remote keeps its previous copy as `.bak`). Upgrades push the new release: `upgrade --binaries DIR` (built for the remote; a newer scribe-ctl in it takes the upgrade over there) or, under docker, `upgrade --source DIR` (a checkout of this scribe-ctl's release, whose images are built there). `scribe-ctl ssh` opens a shell in the cluster's directory on the remote, and `scribe-ctl ssh -- COMMAND…` runs one there. What is pushed, and where it goes in the remote directory: | What | There | When | | --- | --- | --- | | This file's cluster, without `[remote]` | `` (other clusters there kept, a differing copy kept as `.bak`) | every command, for a cluster this file defines | | A generated gateway file beside the definition | the same relative path | with the definition | | `--source DIR` | `upload/source` | setup, upgrade | | `--binaries DIR` | `upload/binaries` | setup, upgrade | | scribe-ctl of this release (and, from a release directory, its service executables), or with `--source` one built there from the pushed checkout (build `-src.`) | `bin//`, named by `bin/current` and `bin/controller` | when missing | | `bootstrap --source DIR` or `--binaries DIR` | `upload/controller/source` or `upload/controller/binaries`, then `bin//`, named by `bin/controller` only | `bootstrap` | A pushed checkout carries only what the image builds read, chosen without git (a checkout need not be a git one): the Rust workspace (`Cargo.toml`, `Cargo.lock`, `rust-toolchain.toml`, `rustfmt.toml`, `LICENSE` and every directory `[workspace] members` names), `docker/`, and `apps/web` whenever the checkout has it (whether or not the cluster runs the web interface; without its build outputs it is a few megabytes). Inside those, the build context's `.dockerignore` decides (the root one for the workspace and `docker/`, `apps/web/.dockerignore` for the web interface), read as Docker reads it for the forms Scribe's files use (`#` comments, `!` exceptions, `*`, `?` and `**` as a whole segment); another form is refused rather than guessed at. Never pushed, wherever they are: version control, build outputs and dependencies (`.git`, `target`, `target-*`, `node_modules`, `.next`, `*.tsbuildinfo`), local state (`/.scribe/_state`, `apps/web/deploy/local`, `.DS_Store`) and secrets (`.env`, `.env.local`, `*.credential`, `*.pepper`, `*.revision`, `/secrets`, `/pki`, `/cluster-state`). For this repository that is about 13 MB instead of the whole working tree. rsync 3 transfers when both machines have it; otherwise a tar stream through ssh replaces the previous upload. Only the summary (`CLUSTER-INFO-.txt`) and, after setup and add, the definition (and a generated gateway file) come back; CA keys, the pepper, credentials and data never leave the remote. Each command makes one ssh connection and runs everything else through it (a `ControlMaster` socket in a private directory under `/tmp`, ended with the command; Windows' OpenSSH has none, so there each step connects on its own). A first connection the remote times out or refuses is tried four more times, after 10, 30, 60 and 90 s (a firewall that throttles bursts of new connections lets one through within that). With `--source`, the scribe-ctl that runs the command there is built from exactly the pushed checkout and named by a digest of it, so a changed checkout never runs a stale scribe-ctl there. Without `--source`, a command runs this machine's own build there, installing it first when the remote has the same OS and architecture (or from pushed `--binaries` of this release). From a machine of another platform (a macOS laptop and a Linux server, say) that cannot be done, so the command runs the newest scribe-ctl of this version installed there (`bin/controller`), else the one `bin/current` names, and says so. That one knows only the commands of the checkout it was built from. `bootstrap` refreshes it without touching the cluster: ```sh scribe-ctl --config ./prod.toml bootstrap --source . --plan scribe-ctl --config ./prod.toml bootstrap --source . ``` It pushes the checkout to its own `upload/controller/source`, builds only scribe-ctl there in Docker (no images), installs it as `bin//` and names it in `bin/controller`. `bin/current`, the definition there, the cluster's binary set and images, its services, lock and journal stay as they are, so it needs no restart and is safe while another command (an upgrade, say) runs there. `bootstrap --binaries DIR` installs a release directory built for the remote instead. A later `--source` setup or upgrade still names its own build in both files. `doctor` says which of these fixes a remote without a usable scribe-ctl ([bootstrap](/docs/reference/cli/scribe-ctl.md#bootstrap)). A remote command does not depend on its connection: when the connection drops (or the terminal here closes), the scribe-ctl there finishes the command, or rolls it back, on its own. Run `status` to see how it went; a command it left pending (a stopped rollback, say) resumes when you run it again. Ctrl-C here asks the remote scribe-ctl to stop at its next safe point, as Ctrl-C does locally: through the remote terminal, or, without one (`--json`, redirected output), by an interrupt sent through the connection. A second Ctrl-C then leaves it running there and returns. The remote needs a POSIX shell, `tar`, and for a docker cluster Docker with Compose and BuildKit. A scribe-ctl built there by Docker links the build image's glibc (Debian bookworm) and needs that or newer on the host. The [reference](/docs/reference/cli/scribe-ctl.md#remote-operation) has the full sequence and real output. ## The definition file The complete template is [example.toml](https://github.com/Flying-Rat/scribe/blob/main/crates/scribe-ctl/example.toml); `scribe-ctl example` prints it. All relative paths resolve against the definition's directory. `[clusters.NAME.binaries]` (keys `auth`, `storage`, and `gateway`) names where setup takes the executables it records as the first binary set; without it they are found beside `scribe-ctl`. Once a set is recorded, services run from `/bin//` and `[binaries]` is not read again; `upgrade` changes the set. ctl owns the file's layout: setup writes it in a canonical form with a comment on every setting. When setup adds a cluster to a file that defines others, or `add auth` or `add gateway` changes a cluster, ctl rewrites the file in that form and keeps the previous one as `.bak`; comments of your own are only in the backup. `variant` records what setup built; a definition whose services do not match its `variant` is refused. Omit `variant` for a topology of your own (for example, plaintext and authenticated storage side by side). The definition names where secrets live but never holds one. Unknown fields are rejected to catch mistakes. To bring services you started by hand under the controller, stop them and write a definition that names their existing files: data roots, the pepper and revision witness, the administrator credential, each store's credential, certificates, and the gateway's configuration (keep `auth.ca` when auth's certificate comes from the storage CA), with `generate_missing = false` so nothing missing is replaced. For the [bare-metal guide](/docs/manual/bare-metal.md)'s deployment after stage 4, in `/srv/scribe/scribe-cluster.toml`: ```toml [clusters.local] variant = "gateway" daemon = true [clusters.local.certificates] generate_missing = false ca_dir = "pki/ca" auth_ca_dir = "pki/auth-ca" [clusters.local.auth] listen = "127.0.0.1:7443" data_dir = "data/auth" pepper_file = "secrets/auth.pepper" revision_file = "secrets/auth.revision" admin_credential = "secrets/admin.credential" [clusters.local.auth.tls] cert = "pki/auth/server.pem" key = "pki/auth/server-key.pem" [[clusters.local.storage]] name = "storage-a" listen = "127.0.0.1:7447" data_dir = "data/storage-a" auth = true namespace = "studio" credential = "secrets/storage-a.credential" [clusters.local.storage.tls] cert = "pki/storage-a/server.pem" key = "pki/storage-a/server-key.pem" [[clusters.local.storage]] name = "storage-b" listen = "127.0.0.1:7448" data_dir = "data/storage-b" auth = true namespace = "studio" credential = "secrets/storage-b.credential" [clusters.local.storage.tls] cert = "pki/storage-b/server.pem" key = "pki/storage-b/server-key.pem" [clusters.local.gateway] config = "scribe-gateway.toml" ``` scribe-ctl starts each store from its table alone and never reads `storage-a.toml` or `storage-b.toml`: a setting you changed there beyond the listen address, data root and `[auth]` table does not carry over. The gateway keeps reading `scribe-gateway.toml`. Run and log directories move to `state_dir` (`cluster-state/local`). `setup --plan` then shows that it only confirms identities with auth; `setup` records the executables and writes `CLUSTER-INFO.txt`, and `up` starts them. The [CLI reference](/docs/reference/cli/scribe-ctl.md#defaults-and-configuration) explains binary/configuration lookup and command-line selection. ## Named clusters and selection Every definition lives under `clusters.NAME`. Names use ASCII letters, digits, hyphens, and underscores. For example: ```toml [clusters.local] daemon = true [[clusters.local.storage]] name = "assets" listen = "127.0.0.1:7447" data_dir = "data/local/assets" auth = false [clusters.staging] daemon = true [[clusters.staging.storage]] name = "assets" listen = "127.0.0.1:8447" data_dir = "data/staging/assets" auth = false ``` `--cluster NAME` is global, so it can precede or follow the command. With one cluster it is optional. With multiple clusters, `setup`, `up`, `down`, `df`, `repo list`, and `logs` require it; an ambiguous command exits before changing anything. `status` and `where` show every cluster in name order, or only the selected one: ```sh scribe-ctl --cluster local setup scribe-ctl up --cluster local scribe-ctl status scribe-ctl status --cluster staging scribe-ctl logs assets --cluster local scribe-ctl down --cluster local ``` `setup --cluster staging --variant storage` adds a cluster to an existing file: its paths start with `staging/`, and its ports move up by a multiple of 100 so both can run at once (or give `--port-offset`). Clusters start and stop independently and keep separate logs. Default `state_dir` is `cluster-state/NAME`; default `certificates.ca_dir` is `pki/NAME/ca` and default `certificates.auth_ca_dir` is `pki/NAME/auth-ca`. Other relative paths remain relative to the TOML file, so provide separate data, PEM, account and credential paths as needed. Overlapping state or data roots are rejected across clusters, including paths through symlinks. Shared listen ports are allowed for alternative clusters, but those clusters cannot run on the same address simultaneously. ## Accounts and pepper The separate `accounts_file` contains password-file references, never inline passwords. Password paths resolve against the accounts file's directory: ```toml [[users]] name = "alice" password_file = "passwords/alice" administrator = false [[users]] name = "operator" password_file = "passwords/operator" administrator = true ``` Use private password files containing 12–1024 UTF-8 bytes. A trailing line ending is allowed. Keep this file and secrets out of version control. Omit `accounts_file` if no account bootstrap is wanted. The auth table names the files setup creates and every later start reads: ```toml [clusters.local.auth] listen = "127.0.0.1:7443" data_dir = "data/auth" pepper_file = "secrets/auth.pepper" # Revision witness: keep it (like the pepper) outside the data directory and its backups. revision_file = "secrets/auth.revision" # The bootstrap administrator's credential, written by setup. Keep it private. admin_credential = "secrets/admin.credential" [clusters.local.auth.tls] cert = "pki/auth/server.pem" key = "pki/auth/server-key.pem" ``` Setup creates a missing auth database by running `scribe-auth init`: it writes the required pepper to `auth.pepper_file`, the [revision witness](/docs/auth.md#rollback-protection) to `auth.revision_file`, and the bootstrap administrator's [credential file](/docs/auth.md#credential-files) to `auth.admin_credential`. The credential names auth's listen address (a wildcard bind becomes loopback) and the CA of auth's certificate. Every auth start receives the pepper and the witness. Existing auth databases require their original pepper and witness: the controller never regenerates either for an existing database. Retain and back up the pepper separately from the database, and keep the witness out of the database's backups, as described in [auth.md](/docs/auth.md). Setup also registers authenticated storage identities and provisions their server credentials (and the gateway's and inventory credentials), then bootstraps users: - A missing credential file is issued for 365 days. - An existing file must hold a live credential of that holder; a file of another subject, or a revoked one, is refused. - An existing credential is kept, also within 30 days of its expiry: `renew` replaces it ([Renew service credentials](/docs/cluster-usage.md#renew-service-credentials)). `scribe-ctl status` and `where` warn 30 days ahead. Repeat runs skip existing users without reading their password files or changing their password, administrator status, or enabled state. The initial `admin` account is consequently left unchanged too. To add accounts later, add them to `accounts_file` and run setup again; it works while the cluster runs: ```sh scribe-ctl --config ./scribe-cluster.toml setup ``` When the cluster is stopped, setup starts auth temporarily and stops it afterward. An interrupted bootstrap resumes on the next run; both of its operation ids are durable before either request. Keep `state_dir` and the original account/password files until the operation completes. Removing a user from the file does not delete their account; use auth administration commands to disable an account or change its password. The administrator credential is required for provisioning and account bootstrap; ordinary `up` needs the pepper, the revision witness, and the server credentials. Live inventory through `scribe-ctl` needs a separate `monitor_credential`: the credential of a service account with a namespace reader grant in the store's namespace, which setup provisions; central administrator status alone reads nothing (see [live storage inventory](/docs/cluster-usage.md#live-storage-inventory)). ## Certificates and topology The controller uses two server-certificate CAs: ```toml [clusters.local.certificates] generate_missing = true # Storage and gateway certificates; auth's comes from its own CA, so no storage # key can ever pass as auth. ca_dir = "pki/ca" auth_ca_dir = "pki/auth-ca" ``` `ca_dir` signs storage and gateway certificates; `auth_ca_dir` signs only auth's certificate. Credential files name the auth CA as the only trust for auth, so a stolen storage or gateway key cannot impersonate auth to their holders. The two directories must differ. Both are server-certificate CAs only; client certificates come from auth's own CAs (see [identity: device keys and certificates](/docs/auth.md#identity-device-keys-and-certificates)). `certificates.generate_missing = true` permits setup to create a missing CA (valid 3650 days) and missing leaf certificate/key pairs (365 days). Existing PEMs are preserved and leaf pairs are validated. A partial pair is refused: preserve/move the surviving file before explicitly regenerating the pair (or setup moves its own interrupted pair aside; see [rerun, resume, and what setup refuses](/docs/cluster-usage.md#rerun-resume-and-what-setup-refuses)). This also applies to incomplete CA material. Certificates are never silently rotated on `up`. `status` warns when a server certificate is within 30 days of expiry; [`renew`](/docs/cluster-usage.md#renew-service-credentials) reissues it for the key it has, while the cluster runs. While `generate_missing` is in use, both CA private keys (`ca-key.pem` in each directory) sit on the controller host, online. Once the certificates you need exist, move the CA keys to offline storage and set `generate_missing = false`; bring a key back only to issue a certificate. Setup names both keys in `CLUSTER-INFO.txt` with that advice. Default generated SANs cover `localhost`, IPv4/IPv6 loopback, and the concrete listen IP. Set `dns` and/or `ip` arrays in a service's `tls` table for other names; explicit arrays replace these defaults. `auth.ca` can select an existing trust bundle for auth's certificate, for example when adopting an auth service whose certificate another CA issued; by default the controller uses `certificates.auth_ca_dir/ca.pem`. The administrator credential that setup creates names this CA, and credentials issued later copy it from there. Auth requires TLS and a fixed, nonzero listen port so bootstrap retries retain the same endpoint. Authenticated storage requires `auth = true`, `namespace`, `credential`, and a `tls` table: ```toml [[clusters.local.storage]] name = "assets" listen = "127.0.0.1:7447" data_dir = "data/assets" auth = true namespace = "studio" # This server's own credential, issued by setup. Keep it private. credential = "secrets/assets.credential" [clusters.local.storage.tls] cert = "pki/assets/server.pem" key = "pki/assets/server-key.pem" ``` In the current storage adapter TLS is coupled to auth integration. Standalone storage uses `auth = false` and no TLS/auth fields; non-loopback plaintext requires `allow_non_loopback = true` explicitly. Storage port zero is allowed, with its assigned endpoint shown by `status`. Each data root and the state directory must be distinct and nonoverlapping. Private keys, CA directories, the pepper, the revision witness, the administrator credential, and service credentials must be outside every service data root. Unknown TOML fields are rejected to catch configuration mistakes. The certificates and credentials themselves, with their lifetimes and how to replace each one with or without `scribe-ctl`, are in [certificates and credentials](/docs/manual/certificates.md). ## Lifecycle and diagnostics `setup` is explicit provisioning; `up` never initializes data or creates secrets. `daemon = true` (which setup writes) makes `up` detach and return once all services report readiness. Without it, the default is foreground. `up --daemon` and `up --foreground` override TOML. In foreground mode, Ctrl-C requests graceful shutdown. The supervisor starts auth first, then storage, then gateway. Shutdown drains gateway before storage, and storage before auth. Startup failures and unexpected service exits stop the remaining services. There is no automatic restart policy. Service output is appended to `state_dir/logs/.log`; detached supervisor output is in `supervisor.log`. `logs` prints a bounded tail, not a live stream. These files are not automatically rotated. `status` shows an aligned service table and warns about credentials and server certificates within 30 days of expiry, read from the files themselves; `--json` selects machine-readable output for every public controller command. Status is read-only. `doctor` is the full health check, read-only too: the definition and the files it names, the binary set against this scribe-ctl, free space against the upgrade's backup reserve, each data root's formats against the active set, auth's data-root permissions and secret files, every expiry, the services, their ports and the gateway's inventory, an interrupted command, kept backups, and under docker the images, the build cache and what an adoption left behind. Each check is ok, info, warn or fail, with the fix beside it; the exit status is 0, 1 (warnings only) or 3 (a failure), so a scheduler can run it and alert on nonzero. With a remote cluster it checks ssh from here and runs the remote's own doctor there, as one report, without pushing or installing anything. See [doctor](/docs/reference/cli/scribe-ctl.md#doctor). ```sh scribe-ctl --config ./scribe-cluster.toml doctor scribe-ctl --config ./scribe-cluster.toml doctor --json # for a monitor: key on each check's id ``` If the controller exits unexpectedly, status reports unmanaged services. Run `down` to stop them before restarting the cluster. `startup_timeout_secs` defaults to 30 (allowed range 1–300), applies per service on startup, and bounds how long `down` waits. A shutdown timeout reports that services are still draining; it never force-kills a durable commit. Leave the supervisor running and retry `status`/`down`. Controller diagnostics and build-time timing support are documented in [logging](/docs/logging.md); option syntax is in the [CLI reference](/docs/reference/cli/scribe-ctl.md). ## Optional managed gateway Point the cluster at the same configuration used by the standalone gateway: ```toml [clusters.local.gateway] config = "scribe-gateway.toml" require_healthy = true ``` The gateway file follows [gateway configuration](/docs/gateway-usage.md). Its paths resolve against its own directory. Use fixed storage listen ports so its instance endpoints remain valid across restarts. Gateway-only clusters can supervise a gateway connected to storage managed elsewhere. Setup initializes the gateway data root and includes its TLS certificate/key pair in certificate provisioning, with the default SANs described above. Existing roots and PEM pairs are preserved. `setup --variant gateway` and `add gateway` also write the gateway configuration, `scribe-gateway.toml`, beside the definition. For a TLS gateway, setup also provisions the gateway's identity: - It creates the `scribe-gateway` [service account](/docs/auth.md#service-accounts) if it is missing. - For every namespace the gateway's instances serve, it sets the account's namespace reader grant (inventory) and delegate grant (acting for users; see [gateways and delegation](/docs/auth.md#gateways-and-delegation)). Setting an existing grant changes nothing. - It issues a 365-day service credential to the gateway configuration's `[tls] credential` path when the file is missing; `renew` replaces it before it expires. A file that does not hold a live credential of `scribe-gateway` is refused. A gateway instance that matches a managed store's endpoint must name that store's namespace; the controller refuses a mismatch. By default, startup waits for a listening gateway and fresh, successful inventory from every configured instance. An unhealthy instance causes startup to time out and the cluster to drain. Set `require_healthy = false` to allow startup with unavailable instances. Later backend outages are handled by the gateway and do not automatically stop the cluster. `scribe-ctl status` shows process lifecycle; `scribe-gateway --config scribe-gateway.toml status` shows backend inventory. ## Output for people and scripts Lifecycle commands report progress on stderr and a service table on stdout. Use `--json` for machine-readable results. Foreground `up` emits its final JSON only after shutdown; daemon mode returns after readiness. Status describes process lifecycle, whereas the gateway's status describes backend observations. The [JSON output guide](/docs/json-output.md#controller-results) describes the result fields and error schema. [Logging](/docs/logging.md) covers terminal colors and diagnostics. Service and supervisor logs are separate from command results and are not automatically rotated by the controller. ## Live storage inventory Query all configured stores, including additional backends in the gateway config: ```sh scribe-ctl --cluster local status --storage scribe-ctl --cluster local df scribe-ctl --cluster local repo list --json ``` `status --storage` combines lifecycle state, disk usage and repositories; without `--cluster` it covers all named clusters. `df` summarizes storage sizes and `repo list` includes each repository's server, ID, head and apparent size. Servers are queried directly while running. Errors are retained beside successful results and produce exit 1. Free disk space is per server because roots may share a filesystem. These are live samples, not an atomic cluster snapshot. Terminal inventory uses aligned tables and readable IEC sizes (KiB, MiB, GiB, and larger units). Use `--json` for automation: size fields remain exact integer byte counts, with no rounding or unit suffixes. For authenticated stores, add these optional fields inside the existing `[[clusters.local.storage]]` table, before its `[clusters.local.storage.tls]` table: ```toml monitor_credential = "secrets/monitor.credential" ca = "pki/ca/ca.pem" ``` The monitor credential must read the store's namespace: the credential of a service account with a namespace reader grant there. Central auth administrator status alone does not grant that access, and the store's own server credential is not a user credential. Setup provisions it: the `scribe-monitor` service account, a namespace reader grant in every namespace whose store names the file, and the credential itself, which `renew` replaces like the others. A file that holds another account's credential is refused. The auth and gateway variants configure `secrets/monitor.credential` for their store. Inventory does not change permissions. Keep the credential file outside service data roots. The `ca` field defaults to the CA of an exact endpoint match in the gateway configuration, then to `certificates.ca_dir/ca.pem`. Only the store's own `monitor_credential` supplies a credential: an authenticated backend listed only in the gateway configuration is reported as a per-server error. Plaintext local stores need neither field. ## Take over an existing Docker deployment A deployment started before scribe-ctl ran it (with the removed `docker/init.sh`, in basic, auth or gateway mode, optionally with the web interface's own Compose project beside it, or the same shapes started by hand with Compose, such as the [Docker Compose guide](/docs/manual/docker-compose.md)'s) comes under scribe-ctl with `adopt`, without moving or re-creating its data or secrets. For the [Docker Compose guide](/docs/manual/docker-compose.md)'s project in `/srv/scribe` (project `scribe`), with the [web interface](/docs/manual/web-interface.md)'s project (`scribe-web`) beside it, from a checkout of this release: ```sh scribe-ctl --config ~/scribe/scribe-cluster.toml adopt --runtime docker \ --admin-credential /srv/scribe/secrets/admin/admin.credential --source . --plan scribe-ctl --config ~/scribe/scribe-cluster.toml adopt --runtime docker \ --admin-credential /srv/scribe/secrets/admin/admin.credential --source . ``` Put the definition in a directory of its own, owned by the user the containers run as (`SCRIBE_UID`/`SCRIBE_GID` in the project's `.env`): scribe-ctl's containers run as that directory's owner. `adopt` finds the Compose projects (or takes them from `--project scribe --project scribe-web`), reads from their containers where every data root, secret, certificate, credential and configuration is on the host, and writes a definition that points at them where they are. The administrator credential is not in any serving container, so it is named with `--admin-credential` (`docker/init.sh` wrote it as `operator/admin.credential` in its state directory). `--source` builds this release's images, as setup does; the web interface always needs it. `--plan` runs every check and shows the definition it would write; anything it cannot carry over (another container in the project, a flag or setting scribe-ctl's containers would not pass, a data root in a named volume, a credential that names auth differently, auth before the identity v2 switch, secrets others can read) is refused by name. Adopting stops the old projects (`docker compose -p NAME stop`: nothing is removed), copies the web database once out of its named volume into `data/web` beside the definition (the volume is kept), starts the cluster's own project on the same ports and checks that it serves. If that fails, the old projects start again exactly as they ran and the cluster is taken out of the definition again. Clients keep their endpoints, sessions and trust. Afterwards the cluster is managed like any docker cluster (`status`, `upgrade --source`, `renew`, `where`). What is left of the old deployment is yours to remove once the cluster has proven itself; the completed run lists it: ```sh docker compose -p scribe down # containers and networks; never -v docker compose -p scribe-web down docker volume rm scribe-web_database # the web database's old volume, once you no longer need it ``` (`docker/init.sh`'s projects were `scribe-gateway`, `scribe-auth` or `scribe-basic`; the completed run names the ones it stopped.) Keep the old Compose files and checkout until then: the old projects' files are what a later `docker compose -p NAME up` would need, and the web interface's other volumes (`objects`, `search`) stay in use by the adopted cluster. The [reference](/docs/reference/cli/scribe-ctl.md#adopt) lists every check and shows a real plan and rollback.