Run Scribe by hand on bare metal

Manual setup · Next: Docker Compose

This guide builds a deployment without scribe-ctl, one service at a time, with the executables themselves: one plaintext storage server, then shared authentication, then a gateway, then a second storage server. It is the same grow path the tutorials take with scribe-ctl, and the files have the same names and meaning, so you can compare each stage with the tutorial's What scribe-ctl did for you section, or hand the result to scribe-ctl later (bring existing services under the controller).

Each stage keeps the repositories of the previous one. The client side (creating repositories, working copies, logging in) is the same whichever way the services run; this guide links to the tutorials for it.

Before you start

Install the six executables (see build and install) and put them on your PATH. Choose a deployment directory owned by the account the services run as. The examples use /srv/scribe on Unix and C:\Scribe on Windows, and run every command from that directory:

umask 077
mkdir -p /srv/scribe && cd /srv/scribe

New-Item -ItemType Directory C:\Scribe | Out-Null
Set-Location C:\Scribe
# Restrict the working directory to this account once; Scribe makes the
# directories it creates private itself and refuses secrets others can read.
icacls C:\Scribe /inheritance:r /grant:r "${env:USERNAME}:(OI)(CI)F" "SYSTEM:(OI)(CI)F"

On Unix, umask 077 makes everything you create private; Scribe refuses a credential file, pepper, revision witness or runtime directory that other accounts can read or write. On Windows it refuses the same files when their access list lets another account read them, and writes them private itself. The Scribe commands below are the same in PowerShell (forward slashes work in their arguments); where a step differs, the PowerShell form follows.

The services listen on loopback with these ports. For team access, use the addresses and DNS names clients connect to, both in --listen and in the certificates' --dns/--ip:

Each service runs in the foreground in a terminal of its own (or under your service manager, see run them as services). Every service stops gracefully on Ctrl-C or when a shutdown-request file appears in its runtime directory.

1. One storage server

Create the data root and a configuration file for it:

scribe-server init --data-dir ./data/storage-a --config-out ./storage-a.toml

initialized data root ./data/storage-a
wrote configuration /srv/scribe/storage-a.toml

init refuses a directory that is not empty; nothing initializes a data root again. The configuration lists every setting with its default and ends with a commented-out [auth] table for stage 2; see server configuration. Start the server:

scribe-server serve --config ./storage-a.toml --runtime-dir ./run/storage-a --log-dir ./logs/storage-a

The flags override the file's runtime_dir and log_dir, so every service keeps its own directories. Once run/storage-a/listen holds tcp://127.0.0.1:7447, the server is ready. Create a repository and two working copies exactly as in tutorial 1, steps 3 and 4.

To stop it, press Ctrl-C, or from another terminal:

touch run/storage-a/shutdown-request

New-Item -ItemType File run\storage-a\shutdown-request | Out-Null

Plaintext storage has no accounts: anyone who can reach the port acts as anyone. It refuses a non-loopback listen address unless you set allow_non_loopback = true, which is acceptable only behind an authenticated tunnel (security).

2. Add authentication

Auth owns accounts, credentials and grants; storage switches to TLS and asks auth about every caller. Authentication explains the model; the steps below are what scribe-ctl add auth runs.

Stop and back up

Stop storage A and back it up into a new directory:

scribe-server backup --data-dir ./data/storage-a --to ./backups/storage-a-before-auth

backed up 7 files (1697096 bytes) to ./backups/storage-a-before-auth
  game head=1 commit=44d835554e66d409c1520953de839bfba0a41e69e0af75c35d47677a40da5a63

Workspaces made before auth have no authenticated owner: they still sync, but submitting needs a working copy created after logging in (ownership and existing data).

Server certificates

Create the two server-certificate CAs and a certificate for each service. The storage and gateway CA (pki/ca) and the auth CA (pki/auth-ca) are separate, so no storage or gateway key can ever pass as auth; the certificates guide shows every CA and who trusts it. Name the addresses clients use. Each server.pem holds the certificate followed by its CA, which the server presents so clients can trust it on first use by the CA fingerprint:

scribe-auth certs init-ca --out ./pki/ca
scribe-auth certs init-ca --out ./pki/auth-ca
scribe-auth certs issue --ca ./pki/auth-ca --out ./pki/auth --dns localhost --ip 127.0.0.1
scribe-server certs issue --ca ./pki/ca --out ./pki/storage-a --dns localhost --ip 127.0.0.1

certificate: /srv/scribe/pki/ca/ca.pem
private key: /srv/scribe/pki/ca/ca-key.pem
client trust CA: /srv/scribe/pki/ca/ca.pem
CA fingerprint: BLAKE3:bec9ed90312d4b568ddb7752e5a26e6e80789e9b724b23b20a25a5a86e1c82f9
...

Both executables carry the same certificate helper. Every output directory must be new. ca.pem is public: clients get it, or its fingerprint. ca-key.pem signs certificates; keep it offline once the certificates exist and bring it back only to issue one. Rules for names and lifetimes are in the certificate reference.

Initialize and start auth

scribe-auth init --data-dir ./data/auth --pepper-out ./secrets/auth.pepper \
  --revision-file ./secrets/auth.revision \
  --admin-credential-out ./secrets/admin.credential \
  --auth tls://127.0.0.1:7443 --auth-ca ./pki/auth-ca/ca.pem

initialized ./data/auth; administrator credential saved to ./secrets/admin.credential; password pepper saved to ./secrets/auth.pepper; revision witness saved to ./secrets/auth.revision

Keep all three files outside data/auth:

Start auth:

scribe-auth serve --data-dir ./data/auth --pepper-file ./secrets/auth.pepper \
  --revision-file ./secrets/auth.revision --listen 127.0.0.1:7443 \
  --tls-cert ./pki/auth/server.pem --tls-key ./pki/auth/server-key.pem \
  --runtime-dir ./run/auth --log-dir ./logs/auth

Administer auth

Every administrative command takes the administrator credential and a state directory for durable pending requests. Define a short helper in each terminal you administer from:

auth() {
  scribe-auth --admin-credential /srv/scribe/secrets/admin.credential \
    --state-dir /srv/scribe/admin-state "$@"
}

function auth {
  scribe-auth --admin-credential C:\Scribe\secrets\admin.credential `
    --state-dir C:\Scribe\admin-state @args
}

Register storage A as a server of the namespace studio and issue its own credential:

auth servers add storage-a --namespace studio
auth credentials create storage-a --server --name daemon --days 365 --out ./secrets/storage-a.credential

89b08a677564b1815c55366879caba51	./secrets/storage-a.credential

The first column is the credential ID, which auth credentials revoke takes. A server credential can check callers in its namespace; it cannot administer anything.

Switch storage to TLS

Bind the data root to the namespace (in place; a bound root no longer starts without auth):

scribe-server auth-init --data-dir ./data/storage-a --namespace studio

initialized auth ownership state for namespace studio

In storage-a.toml, replace the commented-out [auth] example at the end with this table:

[auth]
namespace = "studio"
credential = "secrets/storage-a.credential"
tls_cert = "pki/storage-a/server.pem"
tls_key = "pki/storage-a/server-key.pem"

Start storage A with the same serve command as in stage 1. It now serves TLS only (run/storage-a/listen reads tls://127.0.0.1:7447) and loads auth's access CAs through its credential.

Add people and log in

Add people, their passwords and their grants with the auth helper, and log in, as in tutorial 2, steps 4 to 7: the tutorial's layout is this one, so its commands work here unchanged. In short: auth users add alice, auth users password alice --password-file FILE, auth namespace-admins set alice studio, then on a client scribe --server tls://127.0.0.1:7447 login alice --auth tls://127.0.0.1:7443. How a client decides to trust pki/ca and pki/auth-ca (a CA file, or the fingerprint the certs commands printed) is in client trust.

3. Add a gateway

A gateway gives clients one address for several storage servers. It needs a certificate from the storage CA, an identity at auth, and a routing ledger.

Issue its certificate, create its service account with the two grants it needs in every namespace it fronts, and issue its credential:

scribe-server certs issue --ca ./pki/ca --out ./pki/gateway --dns localhost --ip 127.0.0.1
auth users add scribe-gateway --service
auth namespace-readers set scribe-gateway studio
auth delegates set scribe-gateway studio
auth credentials create scribe-gateway --service --name gateway --days 365 --out ./secrets/gateway.credential

The namespace reader grant lets the gateway poll each store's inventory; the delegate grant lets it present users' own access certificates to storage. Neither lets it write as itself (gateways and delegation). Write scribe-gateway.toml (paths are relative to it; gateway configuration lists every setting):

data_dir = "data/gateway"
listen = "127.0.0.1:7557"

[tls]
cert = "pki/gateway/server.pem"
key = "pki/gateway/server-key.pem"
credential = "secrets/gateway.credential"

[[instances]]
name = "storage-a"
namespace = "studio"
endpoint = "tls://127.0.0.1:7447"
ca = "pki/ca/ca.pem"

Initialize the ledger once, then serve:

scribe-gateway --config ./scribe-gateway.toml init
scribe-gateway --config ./scribe-gateway.toml serve --runtime-dir ./run/gateway

gateway initialized: /srv/scribe/data/gateway

scribe-gateway --config ./scribe-gateway.toml status shows what it observes:

gateway: running
storage-a: online repos=1 bytes=1815535 free=463202058240

Clients switch to tls://127.0.0.1:7557 and log in once more for it, as in tutorial 3, step 4.

4. Add storage servers

A second store gets its own data root, certificate, identity and credential; it shares auth and, here, the namespace. Never copy another store's data root.

scribe-server init --data-dir ./data/storage-b --config-out ./storage-b.toml --listen tcp://127.0.0.1:7448
scribe-server certs issue --ca ./pki/ca --out ./pki/storage-b --dns localhost --ip 127.0.0.1
auth servers add storage-b --namespace studio
auth credentials create storage-b --server --name daemon --days 365 --out ./secrets/storage-b.credential
scribe-server auth-init --data-dir ./data/storage-b --namespace studio

Give storage-b.toml its [auth] table (with storage-b's credential and certificate) and start it:

scribe-server serve --config ./storage-b.toml --runtime-dir ./run/storage-b --log-dir ./logs/storage-b

Append the store to scribe-gateway.toml:

[[instances]]
name = "storage-b"
namespace = "studio"
endpoint = "tls://127.0.0.1:7448"
ca = "pki/ca/ca.pem"

The gateway reads its instances only at startup, so restart it: stop it with Ctrl-C in its terminal, or from another one:

touch run/gateway/shutdown-request

New-Item -ItemType File run\gateway\shutdown-request | Out-Null

Once it has exited, start it in its terminal with the same serve command as in stage 3, then check what it observes:

scribe-gateway --config ./scribe-gateway.toml status

gateway: running
storage-a: online repos=1 bytes=1815845 free=463200911360
storage-b: online repos=0 bytes=181591 free=463200915456

New repositories created through the gateway are placed on either store; existing ones stay where they are. For stores on other machines, run each on its own host with a certificate for the name the gateway and clients use, and see placement and reconciliation and real scaling.

Run them as services

Start auth first, then every storage server, then the gateway; stop them in the reverse order. Give each process exactly one supervisor, deliver a graceful stop (SIGTERM, Ctrl-C or the shutdown-request file) and allow started durable work to finish rather than killing it after a short deadline. The production checklist has systemd units for these services and what applies on Windows, and lists what else to settle before a team uses the deployment.

What comes next:

• Certificates and credentials: renew credentials and certificates before they expire, rotate CAs, recover administration.
• Upgrades: move to a new release with explicit migrations.
• Backups: storage roots, the stopped auth directory, the stopped gateway ledger.
• Operations: health checks, capacity and shutdown.