Architecture

Scribe is a centralized version-control system for large game projects. The server is authoritative; clients keep working copies and lightweight local metadata under .scribe/. There is no distributed history.

Crates and boundaries

crates/
  scribe-core          ids, digests, canonical paths, change numbers, error codes            (no I/O)
  scribe-format        canonical byte encodings, version dispatch, keys, descriptor          (no I/O)
  scribe-platform      every OS-specific behaviour: flush primitives, positional and zero-copy I/O,
                       cache hints, preallocation, reflink clones, rooted directory handles, free space;
                       one module per OS; the only crate with unsafe
  scribe-api           transport-neutral contract: RemoteOps trait, AuthProvider, types, limits
  scribe-objects       chunking profile, staging, loose + pack object store, locator, verification
  scribe-metadata      redb catalog and per-repository tables, the acceptance transaction
  scribe-workspace     .scribe/ state, ignore rules, scanner, journals, branch/merge workflows
  scribe-server        Service: repository operations, publication sequence, budgets, authorization
  scribe-protocol-tcp  TCP v2 framing and codecs (LE production, BE test), protocol-3 auth hello,
                       client + server adapters
  scribe-tls           blocking client and async server rustls TLS 1.3 streams
  scribe-auth          auth wire protocol (SAU1) and client; service and admin CLI behind features
  scribe-cert          offline CA and certificate generation through the OpenSSL executable
  scribe-client        workflows: add/edit/delete/revert/status/submit/sync/export/log/locks/branches/merge
  scribe-log           logging configuration, sinks, terminal detection for executables
  scribe-server-cli    the scribe-server executable
  scribe-cli           the scribe executable
  scribe-auth-cli      the scribe-auth executable
tools/
  scribe-perf          qualification harness; not a dependency of any executable
  scenario.js          ES-module entry point for reusable user-workflow plans
  scenarios.json       services, datasets, compression variants, and scale
  scenarios/           service/client objects, dataset factories, plans, and tests

Dependency direction (arrows point at what is depended on):

scribe-cli        -> scribe-client -> scribe-protocol-tcp (client) -> scribe-api -> scribe-format -> scribe-core
                                   -> scribe-workspace -> scribe-objects -> scribe-platform
                                   -> scribe-tls, scribe-auth (client only)
scribe-server-cli -> scribe-protocol-tcp (+server) -> scribe-server -> scribe-metadata, scribe-objects
                  -> scribe-auth (+admin), scribe-cert, scribe-tls
scribe-auth-cli   -> scribe-auth (+server, +admin), scribe-cert, scribe-tls

Rules the crate graph enforces:

• scribe-core and scribe-format have no networking, no database, no filesystem beyond OS randomness.
• scribe-server never depends on a wire adapter. Adapters depend on it. Authorization is enforced inside scribe-server through the AuthProvider contract; the executable constructs the provider.
• scribe-cli links no server code and no auth database: the TCP crate's server module is behind the server feature and the auth service behind scribe-auth/server, which only the service executables enable (cargo tree -p scribe-cli -e normal,build --locked).
• Client workflows program against scribe_api::RemoteOps. The in-process adapter (scribe_server::LocalRemote), TCP with the production codec, and TCP with the big-endian test codec run the same contract tests.
• Libraries return typed results and emit tracing events. Only the three executables install a subscriber and write to stdout/stderr.

Data flow of a submit

client                                  server
------                                  ------
analyze files: chunk + hash (profile 1)
persist candidate + OperationId  --.
presence(chunk ids)  ----------------->  locator lookup
put_objects(chunks, manifests,          stage to quarantine, verify digests,
            change set) in one batch --> validate structure, publish batch:
                                         pack append / loose rename, flush files,
                                         flush directories, locator commit
submit(op id, change set id) ----------> gate: candidate number = head + 1
                                         publish commit envelope (durable)
                                         accept(): one redb transaction revalidates
                                         bases + locks + collisions, writes current/history/
                                         change_paths/fold_history/commits/operations, head++
                                    <--- acknowledge only after commit
finalize baselines, drop candidate

Publication is serialized per repository by the gate mutex; uploads, hashing, verification, and authorization checks happen outside it and outside every write transaction.

Data flow of a sync

head(branch) -> incarnation check -> plan:
   fresh / target older / incarnation changed: tree pages as-of target merged with baselines
   otherwise: changed_paths pages (synced, target]
journal (redb) holds the plan -> apply in batches of <=256 files / 512 MiB:
   opened paths deferred, existing local files displaced to .scribe/recovery,
   chunks from cache or fetched in batches, temp file per target in .scribe/tmp,
   full-content verification, exclusive rename into place through rooted directory handles,
   one best-effort filesystem flush per batch where supported (Linux)
watermark advances only when every entry is done; failures leave the journal for the next run

Features and dependencies

Third-party crate versions, enabled features, licences, and replacement constraints are documented in dependencies.md.

Shared authentication

scribe-auth-cli supplies the scribe-auth executable. Its scribe-auth library separates the binary TLS auth client from feature-gated database and service code. scribe-tls wraps blocking rustls TLS 1.3 client I/O and Tokio/rustls server I/O for both auth and storage connections. The AuthProvider contract lives in scribe-api, so scribe-server::auth::AuthorizedRemote enforces repository roles, workspace ownership, and operation ownership without depending on an auth wire adapter. The storage executable constructs and injects the provider at startup.

Storage TLS sessions use protocol 3; unauthenticated TCP v2 retains its encodings. Security ownership lives in an explicitly initialized, separately versioned auth-state.redb, with a root required-feature marker that prevents older executables from opening secured data. This state is included in normal storage backups. See auth.md.

Connection scheduling and durable storage

Each server owns a Tokio runtime. Storage has up to four I/O workers (bounded by available parallelism); auth has two. No thread is assigned to an idle connection, incomplete TLS handshake, or incomplete metadata frame. Storage connection buffers are 64 KiB each for reading and writing, instead of 1 MiB each. Frame length validation and canonical codecs remain shared with the blocking clients. One request at a time owns a connection, including any buffered read-ahead from a streamed upload.

The synchronous service API runs on blocking workers admitted through a semaphore, acquired before spawning. Storage defaults to 16 mutation/transfer jobs, at most 8 active upload/download jobs, and 4 additional reserved read- query jobs. Transfer permits are acquired first, leaving mutation capacity; read queries have their own admission permits so publication and metadata-writer waits cannot consume all query workers. Handshake authentication uses the query pool. The runtime blocking-thread cap is the sum of storage and query workers. Auth defaults to 16 storage jobs and sends responses asynchronously after each job finishes. The connection caps remain 256 for storage and 128 for auth and are configurable independently of the workers. Queued requests apply backpressure at the connection; there is no unbounded stream of jobs submitted to Tokio's blocking queue.

Active storage transfers still occupy a bounded blocking worker while the synchronous streaming API consumes/produces bytes. Their socket I/O is driven by Tokio through the synchronous bridge. Plaintext file-range downloads retain the platform zero-copy path: the socket is deregistered, used solely by the transfer worker, then re-registered as nonblocking. Objects smaller than 64 KiB share buffered writes to avoid a flush and socket registration per tiny object. TLS always passes through rustls and retains buffered writes. This is an incremental server-adapter change, not a rewrite of RemoteOps, client workflows, hashing, or database acceptance.

Shutdown closes idle connections and stops admitting queued requests. Started jobs are awaited. At the drain deadline, network I/O is cancelled (including zero-copy writes), but a started storage commit is never abandoned or acknowledged early. A stalled disk can therefore extend shutdown beyond the drain deadline while the process retains its data-root lock.

Tree-page reads validate the branch head and select current or historical rows within one redb read transaction. This removes their publication-gate wait without allowing a concurrent commit to mix a newer CURRENT page with an older requested head. Reserved query workers therefore remain useful while mutations wait for durable publication. Tests hold a publication gate during queries and retain a read snapshot across a subsequent accepted change.

redb retains Durability::Immediate and the existing object -> envelope -> metadata -> response ordering. This change addresses connection overhead and bounds blocking work; it does not claim faster fsync or eliminate contention between writers. Splitting lock/workspace coordination or group commit remains a separate storage decision requiring workload measurements and recovery tests.