Run Scribe by hand with Docker Compose

Previous: bare metal · Manual setup · Next: web interface

This guide runs the Scribe services as containers of a Compose project you write yourself, and grows it the same way as the bare-metal guide: one storage server, then auth, then a gateway, then a second storage server. Every file and identity has the same meaning; only where the processes run changes, so this guide explains the Docker-specific parts and links to the bare-metal guide for the rest.

scribe-ctl can generate and run such a project for you (tutorial 5). Its generated <state_dir>/docker/compose.yaml is a valid starting point to copy: it is JSON, which Compose reads as YAML, with absolute host paths and one mount per file. scribe-ctl --config FILE where prints the docker compose command that reads it.

Before you start

You need Docker Engine or Docker Desktop with Compose v2 and a Scribe checkout to build the images from. From the checkout's root:

docker build -f docker/Dockerfile.storage -t scribe/storage:local .
docker build -f docker/Dockerfile.auth -t scribe/auth:local .
docker build -f docker/Dockerfile.gateway -t scribe/gateway:local .

The three share one cached Rust build stage, so the second and third are quick. Each image holds only its executable (the storage image also the scribe client) and no configuration or data (Docker images). Tag a release you keep with its version instead of local; an upgrade switches the tags in compose.yaml.

Create the deployment directory and tell Compose which account the containers run as:

umask 077
mkdir -p /srv/scribe && cd /srv/scribe
printf 'SCRIBE_UID=%s\nSCRIBE_GID=%s\n' "$(id -u)" "$(id -g)" > .env

The images' default user is uid 10001. Running the containers as the owner of the deployment directory instead keeps every file they write yours and lets Scribe's permission checks pass: it refuses a credential file, pepper, witness or runtime directory that other accounts can read or write, and the bind-mounted files keep their host owner and mode. On Windows, keep the deployment directory in the WSL 2 file system and run these commands from a WSL shell: files on a Windows drive appear in containers with modes Scribe refuses.

Conventions

Fixed paths inside containers. A data root is /data, a service's certificate directory /pki, its credential directory /secrets, auth's pepper and witness directory /witness, and the runtime directory /runtime/scribe on a private tmpfs. Host paths appear only as bind-mount sources.
Binds never create a missing source (create_host_path: false): a missing directory fails the start instead of being replaced by an empty one owned by root. Create each directory first.
Service names are certificate names. Containers reach each other as auth, storage-a and gateway on the project network, so each certificate names its service, plus the names people use from outside.
Ports are published on loopback. Change the host address in ports to the interface clients use.
Containers are locked down: read-only root, no capabilities, no privilege escalation, a two-minute stop grace period for durable work, and readiness (/runtime/scribe/listen) as the health check.
One-shot tools (init, auth-init, certs, backup, migrate) run with docker compose run --rm --no-deps SERVICE COMMAND, which uses the service's image, user and mounts; -v adds a mount for that run only.

1. One storage server

compose.yaml:

yaml

name: scribe

x-scribe: &scribe
  user: "${SCRIBE_UID:?set SCRIBE_UID in .env}:${SCRIBE_GID:?set SCRIBE_GID in .env}"
  read_only: true
  tmpfs: ["/tmp:mode=1777", "/runtime:mode=1777"]
  cap_drop: [ALL]
  security_opt: ["no-new-privileges:true"]
  restart: unless-stopped
  stop_grace_period: 2m
  healthcheck:
    test: ["CMD-SHELL", "test -s /runtime/scribe/listen"]
    interval: 5s
    retries: 12

services:
  storage-a:
    <<: *scribe
    image: scribe/storage:local
    command: [serve, --data-dir, /data, --runtime-dir, /runtime/scribe,
              --listen, "tcp://0.0.0.0:7447", --allow-non-loopback]
    ports: ["127.0.0.1:7447:7447"]
    volumes:
      - {type: bind, source: ./data/storage-a, target: /data, bind: {create_host_path: false}}

Inside its container the server listens on every interface of the container's own network (hence --allow-non-loopback); the host publishes it on loopback only. Initialize the data root once, then start:

mkdir -p data/storage-a
docker compose run --rm storage-a init --data-dir /data
docker compose up -d --wait

text

initialized data root /data
...
 Container scribe-storage-a-1 Healthy

Clients use tcp://127.0.0.1:7447; continue with tutorial 1, steps 3 and 4. docker compose logs storage-a shows the log, docker compose stop drains and stops the containers (never down -v: nothing here lives in a volume, but the habit protects the web interface's).

2. Add authentication

What each file is for is explained in bare metal, stage 2.

Stop storage and back it up into a new directory:

docker compose stop
mkdir -p backups/storage-a-before-auth
docker compose run --rm -v ./backups/storage-a-before-auth:/backup storage-a backup --data-dir /data --to /backup

Replace compose.yaml with this one. It adds auth, switches storage-a to TLS with its credential, and adds auth-admin, a tool you run for administration (the tools profile keeps up from starting it):

yaml

name: scribe

x-scribe: &scribe
  user: "${SCRIBE_UID:?set SCRIBE_UID in .env}:${SCRIBE_GID:?set SCRIBE_GID in .env}"
  read_only: true
  tmpfs: ["/tmp:mode=1777", "/runtime:mode=1777"]
  cap_drop: [ALL]
  security_opt: ["no-new-privileges:true"]
  restart: unless-stopped
  stop_grace_period: 2m
  healthcheck:
    test: ["CMD-SHELL", "test -s /runtime/scribe/listen"]
    interval: 5s
    retries: 12

services:
  auth:
    <<: *scribe
    image: scribe/auth:local
    command: [serve, --data-dir, /data, --runtime-dir, /runtime/scribe,
              --pepper-file, /witness/auth.pepper, --revision-file, /witness/auth.revision,
              --listen, "0.0.0.0:7443", --tls-cert, /pki/server.pem, --tls-key, /pki/server-key.pem]
    ports: ["127.0.0.1:7443:7443"]
    volumes:
      - {type: bind, source: ./data/auth, target: /data, bind: {create_host_path: false}}
      - {type: bind, source: ./secrets/auth, target: /witness, bind: {create_host_path: false}}
      - {type: bind, source: ./pki/auth, target: /pki, read_only: true, bind: {create_host_path: false}}

  storage-a:
    <<: *scribe
    image: scribe/storage:local
    command: [serve, --data-dir, /data, --runtime-dir, /runtime/scribe,
              --listen, "tcp://0.0.0.0:7447", --auth-namespace, studio,
              --auth-credential, /secrets/storage-a.credential,
              --tls-cert, /pki/server.pem, --tls-key, /pki/server-key.pem]
    ports: ["127.0.0.1:7447:7447"]
    depends_on: {auth: {condition: service_healthy}}
    volumes:
      - {type: bind, source: ./data/storage-a, target: /data, bind: {create_host_path: false}}
      - {type: bind, source: ./pki/storage-a, target: /pki, read_only: true, bind: {create_host_path: false}}
      - {type: bind, source: ./secrets/storage-a, target: /secrets, read_only: true, bind: {create_host_path: false}}

  # Auth administration: `docker compose run --rm auth-admin COMMAND`.
  auth-admin:
    profiles: [tools]
    image: scribe/auth:local
    user: "${SCRIBE_UID:?}:${SCRIBE_GID:?}"
    read_only: true
    tmpfs: ["/tmp:mode=1777"]
    cap_drop: [ALL]
    security_opt: ["no-new-privileges:true"]
    entrypoint: [scribe-auth, --admin-credential, /admin/admin.credential, --state-dir, /admin/state]
    volumes:
      - {type: bind, source: ./secrets/admin, target: /admin, bind: {create_host_path: false}}
      - {type: bind, source: ./secrets/new, target: /new, bind: {create_host_path: false}}

Each container mounts only what it needs: auth its data, its certificate and secrets/auth/ (writable, because auth replaces the revision witness beside itself); storage its data, certificate and credential; auth-admin the administrator credential and secrets/new/, where it writes the credentials it issues. No serving container sees a CA key or the administrator credential.

Create the directories, the CAs and the two certificates. Auth's certificate names auth (what containers use) and loopback (what you use from the host):

mkdir -p data/auth secrets/auth secrets/admin secrets/new secrets/storage-a pki
docker compose run --rm --no-deps -v ./pki:/pki auth certs init-ca --out /pki/ca
docker compose run --rm --no-deps -v ./pki:/pki auth certs init-ca --out /pki/auth-ca
docker compose run --rm --no-deps -v ./pki:/pki auth certs issue --ca /pki/auth-ca --out /pki/auth --dns auth --dns localhost --ip 127.0.0.1
docker compose run --rm --no-deps -v ./pki:/pki auth certs issue --ca /pki/ca --out /pki/storage-a --dns storage-a --dns localhost --ip 127.0.0.1

Initialize auth. The administrator credential names tls://auth:7443, the address auth-admin reaches auth at on the project network:

docker compose run --rm --no-deps -v ./secrets/admin:/admin -v ./pki/auth-ca/ca.pem:/auth-ca.pem:ro \
  auth init --data-dir /data --pepper-out /witness/auth.pepper --revision-file /witness/auth.revision \
  --admin-credential-out /admin/admin.credential --auth tls://auth:7443 --auth-ca /auth-ca.pem

text

initialized /data; administrator credential saved to /admin/admin.credential; password pepper saved to /witness/auth.pepper; revision witness saved to /witness/auth.revision

Start auth alone, register storage A and issue its credential into secrets/new/, then move it where storage A's container reads it:

docker compose up -d --wait auth
docker compose run --rm auth-admin servers add storage-a --namespace studio
docker compose run --rm auth-admin credentials create storage-a --server --name daemon --days 365 --out /new/storage-a.credential
mv secrets/new/storage-a.credential secrets/storage-a/

text

bc907886e8a7c2dcef5d65e07bfb781e	/new/storage-a.credential

The credential names auth as tls://auth:7443, copied from the administrator credential, which is where storage's container reaches it. (For a holder that reaches auth elsewhere, give credentials create --auth and --auth-ca.) Bind the store to the namespace and start everything:

docker compose run --rm --no-deps storage-a auth-init --data-dir /data --namespace studio
docker compose up -d --wait

Every auth X command of the tutorials is docker compose run --rm auth-admin X here. A password file goes in with a mount of its own:

docker compose run --rm auth-admin users add alice
docker compose run --rm -v ./passwords:/passwords:ro auth-admin users password alice --password-file /passwords/alice
docker compose run --rm auth-admin namespace-admins set alice studio

People log in from their own machines through the published ports, trusting pki/ca/ca.pem for storage and pki/auth-ca/ca.pem for auth, as in tutorial 2, step 6:

scribe --server tls://127.0.0.1:7447 --tls-ca ./pki/ca/ca.pem login alice --auth tls://127.0.0.1:7443 --auth-ca ./pki/auth-ca/ca.pem

text

Logged in as alice; session valid for 90 days.

3. Add a gateway

Add the gateway service to compose.yaml, before auth-admin:

yaml

  gateway:
    <<: *scribe
    image: scribe/gateway:local
    command: [--config, /config/gateway.toml, serve, --runtime-dir, /runtime/scribe]
    ports: ["127.0.0.1:7557:7557"]
    depends_on:
      storage-a: {condition: service_healthy}
    volumes:
      - {type: bind, source: ./gateway.toml, target: /config/gateway.toml, read_only: true, bind: {create_host_path: false}}
      - {type: bind, source: ./data/gateway, target: /data, bind: {create_host_path: false}}
      - {type: bind, source: ./pki/gateway, target: /pki, read_only: true, bind: {create_host_path: false}}
      - {type: bind, source: ./pki/ca/ca.pem, target: /trust/ca.pem, read_only: true, bind: {create_host_path: false}}
      - {type: bind, source: ./secrets/gateway, target: /secrets, read_only: true, bind: {create_host_path: false}}

and write gateway.toml with the container's paths and each store's service name:

toml

# Paths are the gateway container's.
data_dir = "/data"
listen = "0.0.0.0:7557"

[tls]
cert = "/pki/server.pem"
key = "/pki/server-key.pem"
credential = "/secrets/gateway.credential"

[[instances]]
name = "storage-a"
namespace = "studio"
endpoint = "tls://storage-a:7447"
ca = "/trust/ca.pem"

Issue its certificate, provision its service account (the grants are explained in bare metal, stage 3), initialize the ledger and start it:

mkdir -p data/gateway secrets/gateway
docker compose run --rm --no-deps -v ./pki:/pki auth certs issue --ca /pki/ca --out /pki/gateway --dns gateway --dns localhost --ip 127.0.0.1
docker compose run --rm auth-admin users add scribe-gateway --service
docker compose run --rm auth-admin namespace-readers set scribe-gateway studio
docker compose run --rm auth-admin delegates set scribe-gateway studio
docker compose run --rm auth-admin credentials create scribe-gateway --service --name gateway --days 365 --out /new/gateway.credential
mv secrets/new/gateway.credential secrets/gateway/
docker compose run --rm --no-deps gateway --config /config/gateway.toml init
docker compose up -d --wait
docker compose exec gateway scribe-gateway --config /config/gateway.toml status

text

gateway initialized: /data
...
gateway: running
storage-a: online repos=1 bytes=1815535 free=458752000000

Clients move to tls://127.0.0.1:7557 and log in once more for it (tutorial 3, step 4).

4. Add storage servers

Add storage-b beside storage-a, and make the gateway wait for it too:

yaml

  storage-b:
    <<: *scribe
    image: scribe/storage:local
    command: [serve, --data-dir, /data, --runtime-dir, /runtime/scribe,
              --listen, "tcp://0.0.0.0:7448", --auth-namespace, studio,
              --auth-credential, /secrets/storage-b.credential,
              --tls-cert, /pki/server.pem, --tls-key, /pki/server-key.pem]
    ports: ["127.0.0.1:7448:7448"]
    depends_on: {auth: {condition: service_healthy}}
    volumes:
      - {type: bind, source: ./data/storage-b, target: /data, bind: {create_host_path: false}}
      - {type: bind, source: ./pki/storage-b, target: /pki, read_only: true, bind: {create_host_path: false}}
      - {type: bind, source: ./secrets/storage-b, target: /secrets, read_only: true, bind: {create_host_path: false}}

yaml

    depends_on:
      storage-a: {condition: service_healthy}
      storage-b: {condition: service_healthy}

Append it to gateway.toml:

toml

[[instances]]
name = "storage-b"
namespace = "studio"
endpoint = "tls://storage-b:7448"
ca = "/trust/ca.pem"

Then provision it. The certificate comes first: docker compose run refuses a service whose mount sources do not exist yet. The gateway reads its instances only at startup, so restart it last:

mkdir -p data/storage-b secrets/storage-b
docker compose run --rm --no-deps -v ./pki:/pki auth certs issue --ca /pki/ca --out /pki/storage-b --dns storage-b --dns localhost --ip 127.0.0.1
docker compose run --rm --no-deps storage-b init --data-dir /data
docker compose run --rm auth-admin servers add storage-b --namespace studio
docker compose run --rm auth-admin credentials create storage-b --server --name daemon --days 365 --out /new/storage-b.credential
mv secrets/new/storage-b.credential secrets/storage-b/
docker compose run --rm --no-deps storage-b auth-init --data-dir /data --namespace studio
docker compose up -d --wait
docker compose restart gateway

text

gateway: running
storage-a: online repos=2 bytes=3932332 free=458752000000
storage-b: online repos=0 bytes=181591 free=458752000000

Operate the project

Logs and state: docker compose ps, docker compose logs SERVICE.
Stop and start: docker compose stop drains every service (auth last, since the others depend on it); docker compose up -d --wait starts them in dependency order. restart: unless-stopped brings them back after a reboot of the Docker engine; it is not a policy for a service that keeps failing.
Offline checks and backups run through the stopped service, for example docker compose run --rm storage-a doctor --data-dir /data --quick. The data roots are host directories: with the project stopped, copy data/auth and data/gateway as they are, and back up storage with scribe-server backup as in stage 2 (what to back up, upgrades for the commands).
Renewals and CA changes: certificates and credentials; the credential and certificate directories are mounted whole, so replacing a file inside them and restarting the service is enough.
Upgrades: upgrades.
The web interface joins this project's network: web interface.
Hand it to scribe-ctl later: scribe-ctl adopt --runtime docker takes this project over where it is, keeping its data, secrets, ports and clients' trust (take over an existing Docker deployment).