# 3. Scale across storage instances with a gateway

Previous: [authentication](/docs/tutorials/02-add-auth.md) · [All tutorials](/docs/tutorials.md) · Next: [operate the cluster](/docs/tutorials/04-use-ctl.md)

You will put one gateway in front of storage A, add storage B behind it, and
move the clients to the gateway. Work from the same `scribe-demo` directory.

The scaling unit is a **repository**. New repositories can be placed on
different storage roots, so separate projects can use different disks or
machines. Existing `game` stays on A. The gateway does not stripe one
repository across nodes, rebalance existing repositories, or provide
replication or failover.

```mermaid
flowchart LR
    Client["Clients: 127.0.0.1:7557"] --> Gateway["Gateway"]
    Gateway --> A["Storage A: 7447 — game"]
    Gateway --> B["Storage B: 7448 — new repositories"]
    A --> Auth["Shared auth: 7443 — studio namespace"]
    B --> Auth
```

## 1. Add the gateway

`add` changes a stopped cluster. Stop it, look at the plan, and apply it:

```sh
scribe-ctl --config scribe-cluster.toml down
scribe-ctl --config scribe-cluster.toml add gateway --plan
scribe-ctl --config scribe-cluster.toml add gateway
```

The plan shows the definition's new `[clusters.local.gateway]` table and the
gateway's own configuration file in full:

```text
    @ scribe-gateway.toml (a new file)
    + # Scribe gateway for cluster local, written by `scribe-ctl add gateway`.
    + # Paths are relative to this file. See docs/gateway-usage.md for every setting.
    + data_dir = "data/gateway"
    + listen = "127.0.0.1:7557"
    +
    + [tls]
    + # Issued from the cluster CA (pki/ca), like the storage servers' certificates.
    + cert = "pki/gateway/server.pem"
    + key = "pki/gateway/server-key.pem"
    + # The scribe-gateway service account's credential: a namespace reader and
    + # delegate in every namespace below. Issued by `scribe-ctl add gateway`.
    + credential = "secrets/gateway.credential"
    +
    + [[instances]]
    + name = "storage-a"
    + namespace = "studio"
    + endpoint = "tls://127.0.0.1:7447"
    + ca = "pki/ca/ca.pem"
```

```text
Scribe · local · Gateway added
  wrote scribe-gateway.toml
  rewrote scribe-cluster.toml; the previous definition is kept as scribe-cluster.toml.bak
  issued pki/gateway/server.pem
  initialized data/gateway
  server storage-a: already registered in studio
  storage-a: credential kept (valid until 2027-09-27)
  scribe-gateway: created with its grants in studio
  scribe-gateway: credential issued to secrets/gateway.credential
  scribe-monitor: present with its grants in studio
  scribe-monitor: credential kept (valid until 2027-09-27)
  wrote CLUSTER-INFO.txt
```

### What scribe-ctl did for you

- **The gateway's configuration**, `scribe-gateway.toml`: where it listens, its
  TLS certificate (from `pki/ca`, like the stores'), its credential, and one
  `[[instances]]` entry per store. Paths are relative to the file; see
  [gateway configuration](/docs/gateway-usage.md) for every setting.
- **The routing ledger** in `data/gateway`, created by `scribe-gateway init`:
  which store holds which repository. Back it up with the data roots.
- **The gateway's own identity.** The gateway acts as the `scribe-gateway`
  [service account](/docs/auth.md#service-accounts): an account with no password
  that never signs in and only holds credential files. It got two grants in
  `studio`, one per job:
  - **Namespace reader.** The gateway polls every store's inventory as itself.
    Inventory is a [namespace-wide read](/docs/auth.md#namespace-wide-operations),
    so it needs this grant. It can read every repository in `studio`; it cannot
    write or administer.
  - **Delegate.** Alice and Bob present their own access certificates to the
    gateway, which passes each user's certificate on to storage. Storage checks
    the delegate grant, then that user's own permissions: the gateway never
    acts with more than the user's rights. See
    [gateways and delegation](/docs/auth.md#gateways-and-delegation).

  Its credential, `secrets/gateway.credential`, lives 365 days; stage 4's
  `renew` replaces it in time. Keep it on the gateway's machine and never give
  it to clients.

By hand, the same steps are [bare metal, stage 3](/docs/manual/bare-metal.md#3-add-a-gateway)
and [Docker Compose, stage 3](/docs/manual/docker-compose.md#3-add-a-gateway).

## 2. Add storage B

There is no `add storage`: a second store is two short edits and a `setup`,
which provisions whatever the definition names and is missing. Append this
table to the end of `scribe-cluster.toml`:

```toml
# A second storage server, added by hand.
[[clusters.local.storage]]
name = "storage-b"
listen = "127.0.0.1:7448"
data_dir = "data/storage-b"
auth = true
namespace = "studio"
credential = "secrets/storage-b.credential"
monitor_credential = "secrets/monitor.credential"
[clusters.local.storage.tls]
cert = "pki/storage-b/server.pem"
key = "pki/storage-b/server-key.pem"
```

and this entry to the end of `scribe-gateway.toml`, so the gateway routes to it:

```toml
[[instances]]
name = "storage-b"
namespace = "studio"
endpoint = "tls://127.0.0.1:7448"
ca = "pki/ca/ca.pem"
```

Then plan and apply:

```sh
scribe-ctl --config scribe-cluster.toml setup --plan
scribe-ctl --config scribe-cluster.toml setup
```

```text
Scribe · local · Setup complete
  issued pki/storage-b/server.pem
  initialized data/storage-b
  storage-b: bound to namespace studio
  server storage-a: already registered in studio
  storage-a: credential kept (valid until 2027-09-27)
  server storage-b: registered in studio
  storage-b: credential issued to secrets/storage-b.credential
  scribe-gateway: present with its grants in studio
  scribe-gateway: credential kept (valid until 2027-09-27)
  scribe-monitor: present with its grants in studio
  scribe-monitor: credential kept (valid until 2027-09-27)
  wrote CLUSTER-INFO.txt
```

Setup did for B what `add auth` did for A: a certificate, a new data root bound
to `studio`, a server registration and B's own credential. Everything that
already existed is only checked. A and B share auth and the namespace but have
distinct data roots, identities, and credentials. Never copy A's root to create B.
By hand: [bare metal, stage 4](/docs/manual/bare-metal.md#4-add-storage-servers).

## 3. Start the cluster

```sh
scribe-ctl --config scribe-cluster.toml up
```

```text
Scribe · local · running
  SERVICE    STATE        PID  ENDPOINT
  auth       running    17999  tls://127.0.0.1:7443
  storage-a  running    18000  tls://127.0.0.1:7447
  storage-b  running    18001  tls://127.0.0.1:7448
  gateway    running    18002  tls://127.0.0.1:7557
```

`up` starts auth, then both stores, then the gateway, and returns only once the
gateway has fresh inventory from every instance (`require_healthy = true` in
the definition).

## 4. Move client connections to the gateway

Edit `client-env.sh`: change only `SCRIBE_ENDPOINT` from
`tls://127.0.0.1:7447` to `tls://127.0.0.1:7557`. Reload it, then log in for the
new endpoint:

```sh
. ./client-env.sh
alice login alice --auth tls://127.0.0.1:7443 --auth-ca ./pki/auth-ca/ca.pem
bob login bob --auth tls://127.0.0.1:7443 --auth-ca ./pki/auth-ca/ca.pem
alice repo list
(
  cd work/alice
  alice sync
  alice log
)
```

```text
studio/game              head=3 id=689a965e479c1decf32f3c9ca7e3f913
```

Use the existing passwords. A saved session belongs to the server it was made
for, so the direct-storage session is not a gateway login. Through the gateway,
repository names carry their namespace (`studio/game`); a bare `game` still
resolves. Nothing else changes for clients: they present the same kind of
access certificate to the gateway as to storage.

`game` has the same repository ID and history. The authenticated workspaces
from stage 2 remain valid because requests reach the original store. The
helper's `--server` overrides the workspace's recorded direct endpoint for each
invocation; a plain `scribe sync` in that workspace still uses the saved direct
address unless `SCRIBE_SERVER` is set. Use `alice` and `bob` consistently.

## 5. Place new projects and inspect the result

Alice can create repositories because she administers `studio`:

```sh
alice repo create prototypes --storage-compression lz4
alice repo create audio --storage-compression lz4
scribe-ctl --config scribe-cluster.toml repo list
scribe-gateway --config scribe-gateway.toml status
```

```text
Scribe · local · storage inventory

  SERVER     REPOS          USED     AVAILABLE  ENDPOINT
  storage-a      2      2.80 MiB    436.11 GiB  tls://127.0.0.1:7447
  storage-b      1      2.25 MiB    436.11 GiB  tls://127.0.0.1:7448

  Total: 3 repositories · 5.05 MiB used

  storage-a · repositories
    REPOSITORY      HEAD          SIZE  ID
    game               3    608.93 KiB  689a965e479c1decf32f3c9ca7e3f913
    audio              0      2.02 MiB  a5b3de3fcdf6e500450c59bc3fedd4d4

  storage-b · repositories
    REPOSITORY      HEAD          SIZE  ID
    prototypes         0      2.02 MiB  cda14d1dd062c896472cb400953d0a41
```

```text
gateway: running
storage-a: online repos=2 bytes=1940939 free=465212555264
storage-b: online repos=1 bytes=1355998 free=465212555264
```

`repo list` asks each store directly (as `scribe-monitor`); `scribe-gateway
... status` shows what the gateway itself observes, updated each poll (add
`--json` for details). The gateway placed the two new repositories itself. Placement depends on the
observed load, free space and pending reservations, so inspect the result
rather than assuming a store. Bob's `game` grant does not cover the new
repositories; add grants for their full IDs if he needs access.

## 6. Apply this to real scaling

The two demo roots share one machine and likely one filesystem. This proves
routing, not additional hardware capacity. For independent disk, CPU, or
network capacity, run the stores on separate suitable volumes or hosts:

1. Give the new store's `data_dir` a disk of its own, or run it on another
   machine with its own cluster definition.
2. Give its `[tls]` table the DNS names and addresses clients and the gateway
   use (`dns = [...]`, `ip = [...]`), so its certificate matches them.
3. Add the gateway `[[instances]]` entry with that endpoint. Set optional
   `capacity_bytes` consistently across instances if they should have
   different placement weights. A store in another namespace also needs the
   `scribe-gateway` reader and delegate grants there, which `setup` adds.
4. Stop the cluster, run `setup`, then `up`, which waits for healthy inventory
   before you create repositories through the gateway.

Existing repositories stay pinned to their original instance; adding C does not
move data out of a full A. A failed A makes A's repositories unavailable instead
of redirecting them to B. Keep names unique across the cluster, keep every
configured instance that owns routes, and back up each data root plus the
stopped gateway's routing ledger. The gateway itself is a single process and a
potential throughput bottleneck. See [placement and
limits](/docs/gateway-usage.md#placement-and-reconciliation) and [capacity
planning](/docs/operations.md#capacity-planning).

Continue with [4. Operate the cluster](/docs/tutorials/04-use-ctl.md).