Optional repository gateway

For every gateway command and option, see the gateway CLI reference. This guide covers topology, configuration, placement, and operating the gateway.

All command-line tools support global --json; see JSON output for automation for results, errors, and exit codes.

scribe-gateway provides one normal Scribe endpoint backed by existing storage instances. It routes repository operations, streams object uploads/downloads, and places new repositories using observed instance sizes and repository counts. It does not start storage instances, move repositories, replicate data, or make an unavailable repository available from another instance.

Gateway and auth are independently optional. Start with scribe-server alone, add auth when needed, and introduce a gateway when several storage instances are useful. A gateway can also front several local plaintext servers without auth. Public deployments require TLS and authenticated storage instances using the same auth service. The instances may serve different auth namespaces: one gateway can be the single endpoint for an organization whose departments keep separate namespaces, grants and storage pools (see Namespaces).

Set it up

With scribe-ctl, add gateway writes a complete configuration and provisions the gateway's service account (tutorial 3). By hand, follow bare metal, stage 3 or Docker Compose, stage 3. The shipped plaintext example is useful for loopback-only experiments without auth.

Existing authenticated workspaces can use a server override for the gateway; accepted repository and workspace identities remain on their original instance. Saved login sessions and remembered TLS trust are endpoint-scoped, so approve and log in for the gateway separately.

TLS configuration

Use a certificate trusted by clients at the gateway endpoint. Trust this endpoint using the existing client trust workflow before sending credentials. The gateway terminates TLS, so it is part of the trusted service infrastructure (see what the gateway is trusted with).

data_dir = "gateway-data"
listen = "0.0.0.0:7557"
poll_interval_secs = 15
stale_after_secs = 60
backend_timeout_secs = 30
repository_reserve_bytes = 1073741824
min_free_bytes = 10737418240

[tls]
cert = "pki/gateway/server.pem"
key = "pki/gateway/server-key.pem"
credential = "secrets/gateway.credential"

[[instances]]
name = "storage-a"
namespace = "studio"
endpoint = "tls://storage-a.example:7447"
ca = "pki/ca/ca.pem"
capacity_bytes = 1099511627776

[[instances]]
name = "storage-b"
namespace = "studio"
endpoint = "tls://storage-b.example:7447"
ca = "pki/ca/ca.pem"
capacity_bytes = 1099511627776

Paths are relative to the TOML file. [tls] names the gateway's server certificate and key and its own credential file: the credential of the gateway's service account, a credential bundle holding the auth endpoint, the CA of auth's certificate, and the gateway's device key and certificate. The gateway reads it at startup; restart the gateway after replacing the file. Every TLS instance names the auth namespace it was registered in (scribe-server auth-init --namespace) and the CA that signed its certificate. There are no per-instance credentials: one gateway identity serves every instance.

Plaintext mode has no [tls] table. It accepts only loopback listeners and loopback backends, with no credentials and no namespaces; mixed plaintext/TLS backends are rejected. A TLS gateway requires tls:// instances with a CA file and a namespace each.

The gateway's certificate comes from the storage and gateway CA, like the stores'. scribe-gateway init creates the routing ledger once; serve refuses a data directory without one.

Identity and delegation

The gateway acts under its own service account (scribe-gateway by convention), which needs two grants in every namespace it fronts: a namespace reader grant (namespace-readers set) and a delegate grant (delegates set). Bare metal, stage 3 provisions them and the account's credential; scribe-ctl setup does the same for a managed gateway. Repeat both grants for each additional namespace. A service-account credential lives at most 365 days: replace it before it expires.

• Inventory. The gateway polls each instance as itself, presenting an access certificate for its own credential. The inventory exposes every repository's name and size, a namespace-wide read, so the account needs the namespace reader grant. Storage refuses the poll without it, so a mislabeled instance shows up as an inventory failure on the first poll.
• User requests. Clients present their own access certificates to the gateway, which verifies them against auth's access CAs (fetched from auth with the gateway's credential and refreshed in the background; until the first fetch succeeds, presented certificates are refused). Upstream, the gateway connects with its own access certificate and forwards the user's certificate as a delegated caller. Storage verifies both, checks the delegate grant for its namespace, and decides with the user's own roles; without the grant it refuses with delegate may not act for users in this namespace. Anonymous clients are forwarded as anonymous and reach only public repositories.
• Unknown repositories. A name or ID the gateway's ledger does not know is refused, but only after the same kind of upstream round trip a real refusal takes: the gateway looks the repository up (read only) on a fresh instance of the namespace the name addresses (any fresh instance for an ID), chosen by a hash keyed per process, discards the answer and refuses with the same permission denied a storage server gives for a repository the caller may not see. So neither the message nor the time taken tells whether it exists. The request itself is never forwarded: ownership and conflicts are the ledger's, so a repository an instance holds but the gateway has not taken in yet is served only after the gateway's next poll. Storage treats a name it does not hold the same way when it is reached directly.

Clients see no difference: they log in to auth as usual and present the same access certificate they would present to a storage server. A gateway does not accept a delegated caller, so gateways cannot be chained.

What the gateway is trusted with

The gateway terminates client TLS, so repository data passing through it is visible to it. Beyond that, its reach is bounded by its grants:

• As itself, it can read every repository in the namespaces where it holds the namespace reader grant. A service credential created without --write cannot write as itself.
• As a delegate, it can act only for users whose access certificate it currently holds, only until that certificate expires (at most one hour), and only in namespaces where it holds a delegate grant. What it can do there is what each user may do.
• It never sees passwords or users' private keys: clients log in to auth directly and keep their device keys.

To cut a compromised gateway off, revoke its credential (auth credentials revoke), its delegate grants (auth delegates revoke), or disable the account; auth answers from its records, so storage stops accepting it within the 30-second authorization lease.

Admission limits

The optional [limits] table bounds the gateway's own listener, like a storage server's. The defaults:

[limits]
max_connections = 1024
max_connections_per_principal = 64
max_pending_per_source = 16
handshake_timeout_secs = 10
inactivity_timeout_secs = 300

max_pending_per_source counts connections from one address (an IPv6 /64) that have not completed their handshake; when the table is full, the oldest incomplete one is closed. A principal is the user named by the client's access certificate. Connection bounds must be at least 1, handshake_timeout_secs 1–300, and inactivity_timeout_secs 1–86400.

Namespaces

Auth isolates by namespace: grants are keyed <namespace>/<repository-id>, namespace administration is per namespace, and a storage instance serves exactly one. A user's access certificate is not bound to a namespace; it is valid wherever that user holds a grant. A gateway may therefore front instances from several namespaces and still forward every request with the caller's own access certificate; storage keeps deciding what the caller may do. The gateway's account needs its namespace reader and delegate grants in each of those namespaces.

Repositories reachable through such a gateway are addressed as <namespace>/<name>, for example studio/game and publishing/game. The prefix exists only at the gateway: it selects the namespace's instance pool and the bare name goes to storage, where / is not a valid name character. Both scribe repo create studio/game and scribe workspace create studio/game game work unchanged; repo list and repo info through a namespaced gateway report the qualified address so two game repositories stay distinguishable. Repository IDs are global, so accepted workspaces and every ID-addressed operation need nothing.

default_namespace = "studio"   # optional: where a bare name resolves

[[instances]]
name = "storage-a"
namespace = "studio"
# ...

[[instances]]
name = "storage-b"
namespace = "publishing"
# ...

With one namespace, bare names resolve in it. With several, a bare name resolves in default_namespace when set and is otherwise refused with an error naming the accepted form and the namespaces on offer; discover reports the same list. Placement never crosses a namespace: a new publishing/… repository is placed on the least-loaded fresh publishing instance. default_namespace must be served by a configured instance, and an instance that owns routes cannot be moved to another namespace; restore its namespace instead. The routing ledger keys routes by the qualified address (gateway format). Storage refuses the gateway's inventory poll when its account has no namespace reader grant in the instance's namespace, so a mislabeled instance shows up as an inventory failure on the first poll.

Use storage and gateway binaries from the same Scribe revision so both support the required inventory and authorization operations. The gateway's own status command reads a private local snapshot; it does not expose inventory to ordinary remote users.

Placement and reconciliation

Polling returns the immutable data-root identity, total repository count, per-repository IDs/names/apparent sizes, total apparent data-root bytes, and filesystem space available to the service. Repository pages contain at most 128 entries. Measurements read file metadata, not object contents, and do not follow links. Sizes are observations of a live filesystem, not an atomic storage snapshot. Whole-root size includes uncataloged files, so direct growth contributes to placement even when it did not pass through the gateway.

Only connected instances with a successful inventory newer than stale_after_secs are eligible. Failed polling immediately excludes an instance. The gateway adds a configurable baseline reservation for each repository to its observed byte load, and also accounts for newly assigned repositories that were not in the last observation. It selects the lowest load, normalized by optional capacity_bytes; ties use repository count and then instance name. Use capacity quotas consistently across instances when weighting machines of different sizes. It requires at least min_free_bytes plus a new repository reservation to remain available, and rejects placement beyond a configured quota. Unknown free space is not eligible. This is a placement heuristic, not an enforceable storage quota or a prediction of future repository growth.

Before forwarding a create, the gateway durably reserves its name on one instance. Before acknowledging success, it records the returned repository ID. An interrupted create stays pinned; a later attempt checks the same instance's catalog before attempting creation. An existing completed repository still returns the standard REPOSITORY_EXISTS error. Submit and branch operation IDs and their outcomes pass through unchanged; unknown transport outcomes remain unknown and are resolved at the original instance.

Polling imports repositories created directly on a backend. Duplicate names, duplicate IDs, missing known repositories, and replaced instance identities fail closed instead of silently changing ownership. Conflicting names are recorded in the ledger and require operator investigation; this version has no automated conflict repair or relocation command. Keep repository names unique within a namespace. Do not modify catalog ownership or introduce duplicate names behind the gateway. A backend may remain directly accessible if its deployment permits it, but normal repository creation should use the gateway once introduced.

Routes survive outages and restarts. A configured instance owning routes cannot simply be removed from TOML. Requests for unavailable repositories fail rather than redirecting to another root. Authenticated lookup failures conceal placement details. A repository list fails if an instance cannot be queried, instead of silently presenting an incomplete list.

Operations and limits

status --json includes observation timestamps, instance counts/bytes/free space, health errors, and repository assignments with sampled sizes. Observations update on polls, so a just-created repository may not appear until the next poll. status does not probe or mutate instances. After shutdown, it labels the last snapshot as stopped. The default poll interval is 15 seconds, staleness limit 60 seconds, and upstream I/O timeout 10 seconds. Tune these for large inventories and slower networks. There are at most 64 configured instances and one million repositories per collected instance inventory. Metadata scans can be costly on large roots; increase the interval as needed.

The gateway reuses the bounded TCP server workers and transfer admission, with the connection bounds from [limits]. Object bytes stream through it; they are not staged on gateway disk. TLS and transfer encoding introduce proxy work. There is one active gateway owner per data root, not a highly available gateway cluster. Protect and back up its routing ledger: losing it loses pending placement decisions. Stop the gateway before copying its data directory. The backend data roots remain authoritative for repository data.

Diagnostic flags are in the CLI reference; logging explains formats and the optional timing build feature.

Running with scribe-ctl

scribe-ctl add gateway puts a managed gateway in front of an auth cluster (tutorial 3); to hand an existing gateway to the controller, name its configuration in the cluster definition and run setup (the definition file). The managed gateway configuration describes readiness policy and lifecycle behavior; scribe-ctl setup provisions the scribe-gateway service account, its grants, and the credential file named by [tls] credential. Keep using this guide for backend placement and the gateway's separate configuration file.