The quickest start is scribe-ctl: setup --runtime docker --web runs the web
interface in Docker beside a new cluster and provisions it; follow
tutorial 5: start the web interface and
see run it with scribe-ctl. To run the web stack by
hand beside a cluster you run otherwise, follow the manual web interface
guide.
The web stack is isolated under apps/web/ in this monorepo, alongside the protocol it implements. Paths below are relative to apps/web/ unless stated otherwise:
| Location | Responsibility |
|---|---|
apps/web | Next.js, client React, Tailwind v4, Motion, Iconoir and TanStack Query |
apps/api | Fastify /api/v1 for browsers, tokens and MCP: routes, OpenAPI, image previews |
apps/content | The content process: repository file downloads and previews, bundle downloads, and runners' bundle uploads |
apps/worker | The background worker: the repository watcher (submit triggers, asset-index wakeups), proposal reconciliation, bundle retention and verification |
apps/assets-index | The asset index process: keeps every repository's file catalog in OpenSearch and runs header processors over it; no HTTP |
apps/assets-derive | The asset derive process: sandbox processors that decode whole files with native tools (libvips, OpenImageIO, ffmpeg) and load 3D models with three.js, one bubblewrap-jailed child process per file, for deep metadata, thumbnails and web-ready copies (view.glb, view.webp); no HTTP |
packages/service | What the programs share: configuration, data models and adapters, flows, jobs, the native client |
packages/headers | File header readers (PNG, JPEG, WebP, GIF, TIFF, AVIF, TGA, PSD, EXR, HDR, DDS, KTX2, glTF, GLB, FBX, WAV) and the metadata fields each writes, as pure functions without third-party dependencies, shared by the asset index's header processors and @scribe/models |
packages/models | 3D model loading, facts and web-ready GLB conversion (@scribe/models), for assets-derive and the browser's preview worker |
packages/http | What the two HTTP programs share: the Fastify server, request policy, error replies, authentication, the upload routes, the event-stream writer and the tab's event hub |
packages/scribe-sdk | Reusable Node.js TypeScript adapter for Scribe TCP/TLS and SAU2 auth, with device keys and client certificates |
packages/contracts | Shared TypeScript HTTP response types |
deploy | nginx, SurrealDB, Valkey, OpenSearch and SeaweedFS images, configuration and local secret initialization |
Browser → localhost:8080 → nginx ┬→ Next.js
├→ /api/v1 → Fastify (api) ┬→ Scribe SDK → storage or gateway
│ ├→ Scribe SDK → auth
│ ├→ SurrealDB (web sessions, flows, jobs)
│ ├→ Valkey (disposable metadata cache, live run events, job queues)
│ ├→ OpenSearch (asset catalog)
│ └→ SeaweedFS S3 (run bundles, preview files)
├→ file bytes → Fastify (content), same connections
└→ /_objects/ (internal) → SeaweedFS, for objects content or api authorized
worker, assets-index, assets-derive → Scribe SDK, SurrealDB, Valkey, OpenSearch, SeaweedFS
assets-derive → scratch and socket volumes → assets-derive-sandbox-image, assets-derive-sandbox-scene
(no network; each child jailed by bubblewrap)Storage remains authoritative for repository history, contents and permissions. SurrealDB stores web sessions with their sealed device keys and certificates, and durable auth operation identities. The API contains no Scribe executable and never spawns a CLI. The SDK speaks the existing canonical protocol; it introduces no alternative server protocol or on-disk repository format. Keep it and the services on the same unreleased source revision.
Next.js enables reactCompiler and cacheComponents in apps/web/next.config.ts.
The compiler uses the pinned build-only Babel plugin. Runtime route parameters,
login return URLs and pathname-dependent navigation sit behind Suspense boundaries
so the shared shell and loading states can prerender. Repository and account data
continue to use TanStack Query and the authorization-checked API; enabling Cache
Components does not put those responses into a shared Next.js data cache.
Repository routes share one persistent workspace layout. The header and tabs stay mounted across repository views; the browser and hierarchy stay mounted while switching folders, files, history and graph. Route validation runs in that shared workspace: the URL's namespace segment must be one the session lists as served, otherwise the page is not found. Leaf routes do not introduce another whole-page loading screen.
Navigation links retain Next.js routing and add bounded API prefetch on keyboard focus or a 100 ms hover. At most two destinations warm concurrently; directory and list prefetch loads only the first page and never replaces an existing set of cached pages. It uses the destination's identity-scoped query keys and abort signals. Asset preview payloads still load on demand. Returning to the feed does not replay entrance animations, and repository search keeps previous results visible while a new query loads, only within the same identity.
See the current React Compiler configuration and Cache Components configuration.
Visual theme
apps/web/src/app/styles/theme.css owns the shared color tokens, using the
Scribe website's brand primitives: Midnight Plum (#292333), Citron (#D6EF80),
Porcelain (#F7F7FA) and Obsidian (#101012). Light mode uses porcelain and
neutral surfaces with plum text; dark mode uses obsidian and raised charcoal
surfaces. Citron marks primary actions, with a darker olive (#52651D) for
readable focus and active indicators on light backgrounds. Links, icons, avatars
and selected navigation surfaces stay neutral; citron is reserved for primary
actions and small indicators.
Component styles consume semantic tokens rather than independent purple accents.
Code highlighting and branch lanes retain distinct semantic colors in both themes.
Glass surfaces become opaque for reduced-transparency and increased-contrast
preferences, and keyboard focus indicators remain explicit.
Asset previews
Binary asset viewers are lazy-loaded React components chosen by an internal,
build-time registry (apps/web/src/lib/previews/registry.ts), one per asset kind.
Opening a file only probes its name and the asset catalog; selecting Load
preview downloads the bytes. There are two sources:
- The file itself, through
/api/v1/repositories/:repo/content?preview=1at the viewed branch and revision. The route retains repository authorization, PAT scope checks, private responses and verified streaming. Both the server and the browser enforce the format's byte limit (packages/contracts/src/assets.ts); the configured web download limit can reduce it further. - The asset catalog's web-ready copy (
preview.viewofGET /repositories/:repo/assets/file):view.glbfor models,view.webpfor images browsers cannot open. The catalog followsmainonly, so the copy is used only when the catalog entry's content digest equals the viewed file's; other branches, proposals, files not yet processed and deployments without an asset index use the file itself. Copies are not bound by the raw limit (a GLB may be hundreds of megabytes; the browser caps a GLB copy at 1 GiB and a WebP copy at 256 MiB, and shows download progress).
Models prefer the catalog's GLB, which keeps skeletons, animations and blend
shapes. Images prefer their own bytes, which keep exact pixel values, and use the
WebP copy only for PSD, PSB and TIFF (which have no raw viewer: until the file is
on main and processed, the viewer says so and offers the download) or for files
above the limit. Closing the viewer or navigating away aborts downloads,
terminates workers, revokes object URLs, stops audio and video and disposes GPU resources.
Files without a registered viewer show No browser preview available. The API's
generic large classification can mean only that the 512 KiB automatic text
preview was skipped; it does not establish a download limit or an asset viewer
limit. Supported viewers use their byte budget and the API's previewLimit to
decide admission. Download-limit errors refer to the server's configured limit.
| Viewer | Formats | Raw limit |
|---|---|---|
| Image | PNG, JPEG, WebP, GIF (first frame), AVIF, BMP; PSD, PSB, TIFF from the catalog only | 16 MiB, 16 megapixels; catalog copies up to 8192 pixels per side |
| Image (textures) | TGA, DDS (block-compressed, mip levels, cube maps), EXR and Radiance HDR (half-float values) | 16 MiB (TGA, DDS), 32 MiB (EXR, HDR); dimensions read from headers before decoding, at most 8192 per side and 16 megapixels |
| Audio | WAV, MP3, Ogg/Opus, FLAC, M4A, AAC | 32 MiB; browser codec support applies |
| Video | MP4, M4V, WebM, MOV, in the browser's own player | 128 MiB; the codec decides: a stream the browser cannot decode (ProRes, or HEVC outside Safari) falls back to the download hint |
| Model | GLB, glTF, FBX, OBJ, STL, PLY, USD/USDA/USDC/USDZ, VOX | 16 MiB, 1 million triangles, 256 meshes, converted in a worker by @scribe/models; catalog GLB beyond that |
The viewer frame
Every viewer draws inside one frame (components/viewers/frame.tsx): a toolbar of
labelled tool groups whose buttons expose pressed states, tooltips naming their
shortcuts and aria-keyshortcuts; a collapsible side panel of tabs; an optional
bottom bar (animation timeline, exposure, comparison controls); status text; and
full screen. Shortcuts act only while focus is inside that viewer and never while
a text field, select or slider has focus. Every side-panel tab stays mounted, so
what one holds (an outliner search, expanded rows) survives switching tabs.
components/asset-preview.tsx owns
admission, downloads, progress and failure states; a viewer only draws what it is
given and reports when it cannot.
The asset page (/<namespace>/<repo>/asset/<path>) opens a small file's preview
at once: at most 8 MiB for every viewer, and up to the 16 MiB raw limit for images
the browser decodes itself. A larger file opens on the catalog's 1024-pixel
poster and loads on Load preview. The catalog does not record the size of
its web-ready copy, so that copy is tried and the download stops as soon as it
passes the budget (by Content-Length, or while reading), leaving the poster and
Load preview. Files a model refers to (glTF buffers and textures) are bounded
by the model viewer's own reference limits, not by this budget. A version the
reader chose from the version strip or a comparison loads at once; lists,
reviews and change pages always wait for Load preview.
Comparing versions
Models, images and textures compare two versions (compare in the viewer
registry). On the asset page, Compare in the version strip starts at once with
the version before the one shown on the left and the shown one on the right; both
live in the query string (compare and version), so a comparison can be
shared. While comparing, the preview takes the page's full width and the file's
panels move below it. A bar above the stage (CompareBar in
features/assets/version-compare.tsx) picks each side's version from a list,
swaps the sides, and stops comparing; choosing the version the other side shows
swaps them. The strip marks the two versions on screen with their side's badge,
and a click on a tile goes to the side chosen beside Clicking a version shows it
on.
Every viewer compares in one visual language (components/viewers/compare.tsx):
each version's name over its side of the stage in its side's colour
(--compare-left, --compare-right), and a Differences tab leading the side
panel with both versions' facts in columns, changed rows at full strength and each
changed number's difference from the left version. A Compare select in the
toolbar switches the mode, or shows the shown version alone.
Images and textures
The image viewer draws on a WebGL2 canvas with one shader, so zoom, channels,
exposure, tiling and comparison cost one draw and no copies. Wheel or pinch zooms
about the pointer from 1 % to 3200 %; pixels are drawn as squares above 100 % and
outlined by a grid from 800 %. Double-click toggles fit and 1:1; +, −, 0
(fit), 1 (1:1), arrows and Home work from the keyboard, F toggles full screen.
A minimap appears when the image is larger than the view. Channel isolation (RGB,
R, G, B, A; keys C, R, G, B, A), checkerboard or solid backgrounds, 2×2
and 3×3 tiling, and a pixel inspector (I: coordinates and RGBA, four-decimal
floats for EXR and HDR) are in the toolbar. Float images get exposure and gamma;
DDS files choose a mip level and cube face. Block-compressed DDS needs the GPU's
S3TC, ETC or BPTC support and has no CPU pixel values. KTX2 is not supported: its
Basis transcoder would have to be self-hosted beside three.js's loader.
An image compared with another version (on the asset page, or on a proposal's Files changed tab when its target and result both exist) is shown once with a comparison: swipe (drag the divider or use its slider), side by side, or onion skin with an opacity slider. The Differences tab lists width, height, file size and format.
Models
Models render with three.js's WebGPURenderer, which falls back to WebGL 2 by
itself, lit and framed like the catalog's thumbnails (studioLook,
frameBounds and frameCamera in @scribe/models/look, shared with the
sandbox): Khronos PBR
Neutral tone mapping, which keeps base colours as authored, a dimmed room
environment with one key light, and a three-quarter camera with a 30° field of
view, placed so the model fills 90 % of the view on its tighter axis, over a
soft ground shadow. Framing counts points and lines as well as meshes, and
leaves out backdrops (a sky dome, a vast ground plane) and
small strays far from the rest, so a scene never shrinks to a speck. A file's own bytes are
converted in a worker by @scribe/models (convertModel), which asks the main
thread for files the model names (.bin, .mtl); the main thread fetches them
from the same branch and revision, at most 64 MiB each, 256 MiB and 256 files in
total. The GLB, from the worker or the catalog, loads on the main thread with
GLTFLoader and the meshopt decoder. Images it still names are resolved against
the repository at the viewed revision with the shared rule resolveReference
(packages/contracts/src/references.ts); anything absolute, with a scheme other
than an embedded data:image, or leaving the repository with .. is refused.
The viewer keeps the full scene graph. It plays animations (clip picker,
play/pause, frame-accurate scrubber with frame numbers, frame steps with , and
., loop, 0.25–2× speed), shows the skeleton with its bone count (K), lists the
scene in an outliner (visibility toggles; pointing at a row outlines the node),
drives blend shapes with sliders, and draws lit, clay, wireframe, normals, UV
checker, base colour and vertex colour modes over a theme, grey, black, white or
environment background. A ground grid sized to the model in metres (G), an axis
gizmo, framing (F), front/side/top/three-quarter views (1, 3, 7, 0),
turntable rotation (R) and a PNG screenshot are in the toolbar; Shift+F
toggles full screen. The Info tab shows live statistics (triangles, vertices,
meshes, materials, textures, draw calls, bones, clips, size in metres), the
catalog's or converter's facts, and the referenced files, flagging those missing
or outside the repository.
The outliner keeps no arrow column in a list with nothing to expand, so a flat
scene sits flush, and tints each node type's icon. Chips under its search narrow
the tree to one kind (meshes, skinned meshes, lights, cameras) with the path to
each, beside the skeleton switch; a button expands or collapses every row, and
Alt-click on an arrow opens or closes a whole branch.
Compared (on the asset page, or a proposal's changed model), two versions render
side by side on two stages, the compared version on the left, stacked when the
viewer is narrow. Every toolbar setting applies to both; the cameras move
together, whichever is turned, and animations follow the shown version's
transport (clip, play, pause, paused time), until Link cameras (L) is
turned off. The Differences tab compares triangles, vertices, meshes, materials,
textures, draw calls, bones, clips, size and file size; the Scene tab switches
between the two versions' outliners, and a screenshot captures both side by side.
Adding an internal viewer
- Add the format and admission limit to
packages/contracts/src/assets.ts, shared by the API and browser, with its media type in that file'smediaTypestable, or list it incatalogImageExtensionswhen only the catalog'sviewcopy shows it. - Map its kind to a lazy component in
apps/web/src/lib/previews/registry.ts. Discovery is synchronous and name-only. - Implement the component against
ViewerProps(lib/previews/types.ts): the downloaded bytes and their source, the file's location for resolving references, catalog facts for exactly those bytes, an optional second version to compare, andonError. Draw insideViewerFrame; dispose everything on unmount. - Keep expensive decoding in a cancellable worker, enforce decoded complexity
limits, and fetch only what
resolveReferenceallows. Add format, limit, cancellation and resource cleanup coverage.
There is no dynamic third-party plugin installation or execution.
Repository browser URLs
One deployment serves every auth namespace its endpoint reports through
discover: a single web interface in front of a
gateway that serves several namespaces shows all
departments, with one login, one feed, one inbox and one workspace inventory
across them. The first URL segment is the namespace and every repository is
addressed as <namespace>/<name>: in page URLs (/studio/game), in every
name or repository field the API returns, and in the :repo route
parameter (percent-encoded, /api/v1/repositories/studio%2Fgame/tree). The
session response lists the served namespaces and the default namespace.
WEB_SCRIBE_NAMESPACE is optional. When set it is the namespace a bare :repo
name resolves in (for automation clients and older callers) and must be one the
endpoint serves, otherwise the API answers namespace_unavailable (503). When
several namespaces are served and no default is configured, a bare name is
refused with namespace_required. For plain storage there are no auth
namespaces and the value is the stable URL label (default local). Unknown
namespaces return not-found; a URL never selects an arbitrary upstream endpoint.
Reserved application prefixes (admin, api, authorize, u, login, settings,
developer, r, reviews, workspaces, maintenance, notifications,
repositories), compared case-insensitively, cannot be namespace labels. The maintenance topology labels each storage server
with the namespace it serves and lists the namespaces with their server counts.
Repository lists show the bare name when one namespace is served and the full
address when several are.
Auth grants are keyed by each repository's own namespace
(NAMESPACE/REPOSITORY_ID in scribe-auth grants), independent of
WEB_SCRIBE_NAMESPACE.
The local cluster uses personal. Browser URLs are:
/personal/scribe-web-demo— the repository's home in your interface (Files in Development, Assets in Art), so one shared link suits both/personal/scribe-web-demo/tree/main/4/docs— directory at change 4/personal/scribe-web-demo/blob/main/4/src/game/player.ts— file at change 4/personal/scribe-web-demo/commits/main/latest— current branch history/personal/scribe-web-demo/commit/main/4— change 4 of main and every file it touched/personal/scribe-web-demo/graph/main— graph focused on main/personal/scribe-web-demo/assets/Characters— the asset browser at a folder of main/personal/scribe-web-demo/asset/Characters/hero.fbx— one asset's page/personal/scribe-web-demo/settings— the repository's Settings overview;…/settings/tags,…/settings/fieldsand…/settings/indexopen its sections/personal/scribe-web-demo/locks— check-outs (locks) of every branch;…/locks/<branch>narrows them to one
Revisions are full u64 decimal strings, or latest to follow the branch head.
Branch names/IDs occupy one encoded segment; file paths encode each segment
separately. Query strings remain available for filters, not repository selection.
Page parameters are decoded exactly once before path validation, so Unicode names
such as docs/café.txt display correctly and literal percent sequences remain literal.
Encoded separators and traversal segments are rejected.
All links use lib/repository-url.ts, including README links, mount pins, graph
nodes and login return paths, and take the repository address
<namespace>/<name> (lib/repository-address.ts). The superseded
/r/:repo?... page is removed. The versioned REST API keeps its
/api/v1/repositories/:repo resources, where :repo is the same address,
percent-encoded (studio%2Fgame).
Profile photos and activity
Profile settings accept JPEG, PNG or WebP photos up to 2 MiB, with dimensions from 64 through 4096 pixels per side. Cropping stays in the browser until Save; the API validates and re-encodes uploads independently, strips metadata, rejects animated uploads and produces 64/128/256/512px WebP variants. Two concurrent image jobs per API replica and a per-source upload rate limit bound processing work.
PUT /api/v1/account/avatar and DELETE /api/v1/account/avatar require a browser
session. GET /api/v1/users/:id/avatar?size=128&v=<version> checks live profile
visibility or ownership and sends private, no-store; changing a public profile
to private must not leave a cache serving its image. Variant bytes live in a
separate SurrealDB table behind the profile repository; replacing or removing a
photo atomically updates its profile version. Profile JSON never includes images.
Data layer
Application code depends on plain domain models and repository interfaces:
Location under packages/service/src | Responsibility |
|---|---|
data/models.ts | Web sessions, durable submit records, profile defaults and preference patches, flow runs and runners |
data/repositories.ts | Session, profile, submit, star, reaction, proposal and flow contracts, grouped as DataStore; optional cache contract |
data/errors.ts | Database-independent outcomes such as the automation token limit |
data/shapes/ | One module per versioned stored record kind: its current version, strict decoder, encoder and upgrade steps; index.ts lists every table as versioned or exempt |
data/migrate.ts, data/migrations/ | Full migrations: the runner, the ledger contract, the startup check and the registered migrations |
data/surreal/ | SurrealDB connections, schema setup, queries, transactions and error translation; every stored document is read and written through its shape |
data/surreal/flows.ts | Runs, logs, runners and the watch record. A run's sort key is assigned with the run's number inside one transaction, in straight-line statements: inside an IF … ELSE block SurrealDB drops assignments that read a block-local LET, which left rows without the key their listing pages by. |
data/valkey.ts | Valkey implementations of the disposable cache, rate-limit counter and live-event (pub/sub) contracts |
flows/live.ts | The flow repository decorator that announces run changes and stored log chunks on the live-event channels, and the inbox its listeners read |
flows/topics.ts | Live topics: what a subscriber is told first, the channels to follow and how messages become events; follow() runs one on its own stream |
apps/*/src/main.ts | The five composition roots (API, content process, worker, asset index, asset derive supervisor) that select adapters, refuse data that needs a migration first, and inject repositories; the derive sandboxes' executors have no data store |
apps/worker/src/migrate.ts | The migrate command: the census, the ledger and the full migrations, over the migrate store |
REST and MCP handlers receive DataStore and use its sessions, profiles and
commits repositories. Authentication and the Scribe adapter only receive the
session repository; the file submission service only receives the commits repository.
HTTP schemas remain outside persistence adapters. Database record IDs, username
indexes and computed query columns never leave an adapter as domain fields.
To replace SurrealDB, implement the repository contracts in another adapter and
change the factory selected in the five main.ts files, together with
deployment configuration.
The UI and HTTP/MCP contracts do not depend on that choice. Existing data still
requires an explicit transfer plan; changing the adapter alone does not move it.
No generic ORM, query builder or second production implementation is introduced
here; stored data changes shape through versions and migrations, below.
The portable contracts at packages/service/test/support/data-contract.mjs must pass for
every adapter. dataContract checks atomic refresh rotation, private profile defaults and
stable UUIDs, exact submit-intent preservation under concurrent insertion,
outcome persistence across reconnection, and the per-owner automation-token
limit across connections. storedDataContract checks that a document stored
before versions existed reads, that whole and partial writes stamp the current
version, that a write leaving a document unstamped is refused, and that the
migrate command's store upgrades stale documents and records the migration
startup checks. npm run check runs both with memory repositories and
also checks that services cannot import database drivers or adapters.
Run the same contract against the running local SurrealDB deployment:
cd apps/web
docker compose build api
docker compose up -d --no-build --wait api
npm run test:dataThis uses a separate temporary database and removes it afterward; it does not
modify existing web accounts or Scribe repositories. npm run test:data -- --help
describes the command. Existing SurrealDB tables and fields are preserved.
Writes use explicit RPC transactions and wait for commit acknowledgement before
returning results. Only confirmed transaction conflicts are retried; unknown
outcomes are propagated. The cross-connection contract caught and now covers a
refresh-rotation race in the previous implementation.
Stored data versions
Stored data is either authoritative (it exists nowhere else) or derived (it is
made again from what is). Only authoritative records are versioned.
data/shapes/index.ts lists every SurrealDB table as one or the other, with
the reason, and a test holds the adapter's DEFINE TABLE statements to that
list. Exempt are rows that can be made again (the star and reaction counts, the
catalog mirror's position), the job schedule the queue works from, and single
values kept in SurrealQL (the run and write counters, a person's read
watermark, the index's moves mark, a manual start's run, the token-creation
lock); changing what one of those means is a full migration. Outside
SurrealDB, Valkey holds only disposable state (cache entries, whose keys carry
a version such as flows:v2:, admission counters and queue leases), and a
flow upload's staging record in object storage lives only until its upload
ends.
Each versioned kind has a shape module under data/shapes/: its current
version, the strict decoder of that version, the encoder that makes the stored
document, and a pure upgrade step from every older version it still reads,
keyed by the version it upgrades from. A document carries its version in the
shapeVersion field, which no domain value uses; a document without it was
stored before versions existed and is version 0, read through the kind's
explicit 0 → 1 step (unchanged: those documents already have the version 1
shape). A read upgrades what it finds step by step and the current decoder
checks the result; a write stores the current version, stamped, and is checked
as a read would check it. A stamp newer than the release reads, or malformed,
or a document older than its oldest step, is corrupt, never read as a default.
SurrealDB enforces the stamp as well: every versioned table defines
shapeVersion as a number equal to the current version, so a statement that
leaves a document unstamped or stale is refused whoever runs it. The in-memory
doubles store the same documents through the same shapes.
SurrealQL that names a kind's fields (a WHERE, ORDER BY or index, a
projection, a SET or MERGE) is partial access: it assumes every stored
document has the current shape. A partial update may only touch a document
whose stored shape is current and must not change its shape, and it stamps
what it writes; anything else reads, upgrades and writes back the whole
document. Shape.partial() holds this: while an older version the code reads
has another shape (a step that is not unchanged), it refuses, and so the
adapter refuses to connect, until the shape names the full migration that
upgraded every document of the kind (requires) and startup therefore
refuses to run without it. In practice a reshaping step on a kind that
SurrealQL filters, sorts, indexes, projects or patches needs an eager
migration; a kind only ever read whole upgrades lazily.
Full migrations live in data/migrations/, one module each with a stable ID
and ordinal (0001-stamp-shape-versions) and a description, registered in
order in index.ts. They do what a shape's step cannot: backfill a field
SurrealQL queries, split, merge or move records between tables, remove or
reshape rows, or upgrade every document of a kind at once
(upgradeToCurrent(shapes) in data/migrate.ts, on which 0001 is built).
Each runs in bounded batches of conditional writes (each over the version it
read, so a document written meanwhile is left as written), is idempotent,
records its counts and checkpoint in the ledger after every batch, and has a
dry run that reports what it would change. The ledger is the versioned
data_migration table, written only by the migrate command; an entry is
running, failed (with the error and the checkpoint the next run continues
from) or applied.
Startup never migrates, but it refuses. Every process (the API, the content
process, the worker, assets-index and assets-derive) calls
requireCurrentData once connected: while a migration this release requires
(REQUIRED_MIGRATION in data/migrations/index.ts, or a shape's requires)
has not been applied, or the ledger names a migration this release does not
know (a newer release migrated the data), it stops with a message naming the
migrate command. 0001-stamp-shape-versions is not required: every existing
document reads through its unchanged step, so a deployment keeps working
before it runs.
0002-sign-out-bearer-sessions is required. A version 2 web_session holds a
sealed device key where version 1 held a Scribe bearer token, so its 1 → 2
step signs out every older browser session and API, MCP or device token,
owing nothing upstream (auth's own schema 4 migration already removed their
tokens). When upgrading to this release, run the migrate command below before
starting it; afterwards everyone signs in again and every API, MCP and device
token has to be created again.
The migrate command is the worker image's apps/worker/dist/migrate.js.
--check writes nothing and exits 0 when this release can start on the
stored data, 3 when a migration it requires has not been applied; run it
before starting a new release. --dry-run and --check connect read-only:
they leave out the new release's version checks, so both are safe while the
old release still serves, and they read the migration ledger as empty before
anything was migrated. Only applying migrations needs the services stopped. With
--dry-run it prints, as JSON lines, every versioned kind's documents by
version (census), the ledger, and each pending migration's plan; without it,
it applies the pending migrations in order, each from its checkpoint.
--only <id> applies just the next pending one and --batch <n> sets the
documents a batch reads. On a Compose deployment it runs as docker compose run --rm worker node apps/worker/dist/migrate.js, with the same -f files as up
(first start,
upgrades). Run the dry run first: it
decodes every stale document, so a document the new decoders would refuse is
named before any process reads it.
Deploy readers before writers. A document of a newer version is corrupt to an
older release, and the database refuses a stamp that is not its current one,
so a release that raises a kind's version must be running in every process
that reads the kind before any process writes it: stop every service
(docker compose stop), run the migrate command when the release requires a
migration, then start the release (docker compose up -d --wait). A version
raise is one-way; going back means restoring the database backup taken
before it. A web interface run by scribe-ctl gets that backup from every
upgrade (a cold copy of the stopped database's files, see Run it with
scribe-ctl); for a Compose deployment of your own,
back up the database volume before
migrating.
The census says when an old step may go: once every deployment's census shows
no document of a kind below version n + 1 (its upgrade migration applied
everywhere), the step from n is deleted with its fixture, and a document of
that version is corrupt from then on. Each kind keeps a golden fixture per
version it reads (packages/service/test/fixtures/shapes/<kind>.v<n>.json);
the registry test reads each through the steps, checks it against the current
fixture and checks that writing stamps it current.
Derived data never gets upgrade steps. The asset catalog's OpenSearch indices
carry a generation (generation in assets/opensearch/mappings.ts): a
mapping or document change raises it, assets-index rebuilds every repository
into the new generation's indices from Scribe, and the lower generations are
dropped. Previews in the derived bucket are keyed by processor and version
and made again when either changes, and a cached value in Valkey either sits
under a key that names its shape's version or is checked when read and worked
out again when it does not fit.
Run it with scribe-ctl
scribe-ctl can run the web interface as part of a Docker cluster: setup --runtime docker --web --source CHECKOUT builds its images from
apps/web/compose.yaml together with Scribe's, and runs its services in the
cluster's Compose project with the Scribe services beside them. It does what
deploy/init.mjs, .env and compose.scribe.yaml do by
hand:
- the secrets:
secrets/web/jwt.keyandsecrets/web/surreal.password(32 random bytes, base64, private), mounted as the Compose secrets; - the variables (
<state_dir>/docker/.env):WEB_SCRIBE_SERVERnames the gateway (or the first store) andWEB_SCRIBE_AUTH_SERVERauth by their service names on the project network, each verified against its own CA (ca.pemfor storage and the gateway,auth-ca.pemfor auth, copied into<state_dir>/docker/trust/), and the Clone menu names their published addresses; during a CA rotation both files hold the old and the new CA, since the web services (Node's TLS) follow no handover certificate; - with auth, the
scribe-webservice account, a namespace reader in every namespace of the cluster, and its credential insecrets/web-reader/scribe-web.credential, which carries auth's service address; its directory is mounted as/etc/scribe/worker; - the database's files in
data/webon the host instead of a volume, run as the owner of the cluster's directory, so upgrades can back them up; the other volumes (search, objects) are Compose's own; - a new database prepared with the worker's
migrate.jsbefore the first start.
The services that reach Scribe (those on the upstream network) also join
the cluster's network; nothing else of the web stack changes. The interface is
published by nginx on [web] listen (127.0.0.1:8080), with origin as
APP_ORIGIN.
TLS. The web interface's own certificate is yours to provide: put
fullchain.pem and key.pem in [web] tls_dir (secrets/web-tls/), set
origin to the https:// address and run scribe-ctl up; nginx then serves
HTTPS on listen and redirects HTTP on redirect_port. Until the files are
there it serves plain HTTP, so keep it on loopback. scribe-ctl where names
the directory.
Upgrades. scribe-ctl upgrade --source CHECKOUT builds the new web images
with the rest. While the old release serves, it runs the new worker's
migrate.js --dry-run against the running database (read only); a refusal
stops the upgrade before anything stops. Then it stops the cluster, backs up
data/web (a cold copy of the stopped SurrealDB files into
<state_dir>/backups/<stamp>/web/, with a manifest), switches every service to
the new images, runs the new worker's migrate.js (its dry run first) with the
database started for it, and starts everything. If a later step fails, the
rollback restores data/web from that copy (keeping the migrated one beside
it) together with the old images. Search indices and derived files are
rebuilt by the release as usual; the objects volume (run bundles) is not
migrated and not copied.
On Linux the web services run as their images' users (node, uid 1000), so
the reader credential, the secrets and the CAs must be readable by that user:
run scribe-ctl as uid 1000, or give that uid read access with an ACL
(the commands). Never
make them group- or world-readable: the credential holds a private key. Docker Desktop maps ownership for you.
Connect it to a cluster by hand
For a cluster that scribe-ctl does not run in Docker (a native cluster, a
Compose project of your own, or services on other hosts), run the web stack
with its own Compose project and point it at the cluster: the manual web
interface guide has every step, from node deploy/init.mjs and both CAs to the scribe-web reader credential, the
database preparation and your own TLS certificate.
Networks
Only nginx publishes host ports, bound to 127.0.0.1 by default, from the
edge network. upstream also leaves the host, toward Scribe; the other
networks are internal:
| Network | Members |
|---|---|
application | nginx, web, api, content |
data | SurrealDB (database), Valkey (cache), OpenSearch (search); api, content, worker, assets-index, assets-derive |
objects | SeaweedFS (objects); nginx (to fetch objects api or content authorized), api, content, worker, assets-derive |
upstream | api, content, worker, assets-index, assets-derive: the processes that reach Scribe storage and auth |
The two sandbox containers, assets-derive-sandbox-image and
assets-derive-sandbox-scene, have network_mode: none: no network at all,
not even loopback to another service. They share only two in-memory volumes
with assets-derive: derive-scratch, holding the files being decoded, and
derive-sockets, holding the executors' sockets. No Scribe data root is
mounted into the web stack.
Connection configuration
Edit apps/web/.env and run docker compose up -d --wait to apply changes. Compose
inputs deliberately use the WEB_SCRIBE_ prefix so inherited CLI environment
variables cannot inject host-only certificate paths into containers.
| Variable | Purpose / local default |
|---|---|
WEB_BIND, WEB_PORT | nginx host bind and port; 127.0.0.1, 8080 |
APP_ORIGIN | Exact browser origin including port; http://localhost:8080 |
WEB_TLS | Empty (HTTP) or 1 / true / yes / on to terminate TLS on the published port |
WEB_TLS_DIR | Host directory mounted at /etc/nginx/tls; ./deploy/local/tls |
WEB_TLS_CERT, WEB_TLS_KEY | Certificate and key paths inside the nginx container; /etc/nginx/tls/fullchain.pem, /etc/nginx/tls/key.pem |
WEB_HTTP_PORT | Host port for HTTP→HTTPS redirect when WEB_TLS=1; 8082 locally, 80 on an intranet host |
WEB_SCRIBE_SERVER | tls://host.docker.internal:7557; use storage or gateway |
WEB_SCRIBE_TLS_CA | Container path to public storage/gateway CA: /etc/scribe/trust/ca.pem |
WEB_SCRIBE_TLS_SERVER_NAME | Explicit verified identity, 127.0.0.1; blank uses endpoint hostname |
WEB_SCRIBE_AUTH_SERVER | tls://host.docker.internal:7443; blank disables login |
WEB_SCRIBE_AUTH_CA | Container path to public auth CA: /etc/scribe/trust/auth-ca.pem (a scribe-ctl cluster signs auth with its separate pki/auth-ca) |
WEB_SCRIBE_AUTH_TLS_SERVER_NAME | Explicit auth identity, 127.0.0.1; blank uses endpoint hostname |
WEB_SCRIBE_NAMESPACE | Namespace a bare repository name resolves in; blank (see repository browser URLs) |
WEB_SCRIBE_READER_CREDENTIAL_FILE | The web services' reader credential file (worker, assets-index, assets-derive), as a container path: /etc/scribe/worker/reader.credential; the file scribe-auth credentials create --service writes. Required when WEB_SCRIBE_AUTH_SERVER is set. Compose passes it to the worker and both asset processes (assets-index, assets-derive) as SCRIBE_READER_CREDENTIAL_FILE, together with WEB_SCRIBE_AUTH_TLS_SERVER_NAME as SCRIBE_AUTH_TLS_SERVER_NAME |
WEB_OBJECTS_DIR | Where object storage keeps its data (run bundles, preview files): a host directory or a volume name; the named volume objects |
WEB_BUNDLES_RUN_LIMIT_MB | Bytes one run may publish; 16384 |
WEB_BUNDLES_UPLOADS | Upload parts one process streams into object storage at once; 8, at most 16 |
WEB_DOWNLOADS | Transfers over 64 MiB one content process streams at once; 2, at most 32 |
WEB_ASSETS_INDEX_CONCURRENCY | Asset jobs (indexing and processing) the assets-index process runs at once; 2, at most 16 |
WEB_SEARCH_HEAP | OpenSearch JVM heap for the search service; 1g |
WEB_ASSETS_DERIVE_LANES | Sandbox lanes the assets-derive process runs, comma-separated; image,scene (a second copy on a larger machine can run only scene) |
WEB_ASSETS_DERIVE_IMAGE_CONCURRENCY | Files the image lane decodes at once, each in its own child process; 2, at most 16 |
WEB_ASSETS_DERIVE_IMAGE_MAX_MB, WEB_ASSETS_DERIVE_SCENE_MAX_MB | Largest file each lane reads whole; larger files are recorded as too large; 256 and 2048 |
WEB_ASSETS_DERIVE_SCRATCH | Size of the in-memory scratch volume the derive supervisor and the sandboxes share, holding each file while a child decodes it and what the child writes; 4g. It must hold at least SCENE_CONCURRENCY × (SCENE_MAX_MB + 640) + IMAGE_CONCURRENCY × (2 × IMAGE_MAX_MB + 96) MiB of the WEB_ASSETS_DERIVE_ values above (3904 MiB with the defaults): a scene's view.glb is up to 512 MiB, an image's full-size intermediate as large as the file |
WEB_ASSETS_DERIVE_IMAGE_CHILDREN | Child processes the assets-derive-sandbox-image executor runs at once; 3, at most 32 |
WEB_ASSETS_DERIVE_SCENE_CHILDREN | Child processes (models, each with a 4 GB heap) the assets-derive-sandbox-scene executor runs at once; 1, at most 32. Raise its memory limit with it |
WEB_ASSETS_DERIVE_SCENE_CONCURRENCY | Scene files the supervisor works on at once; 1, at most 4 |
WEB_ASSETS_DERIVE_IMAGE_MEMORY, WEB_ASSETS_DERIVE_SCENE_MEMORY | Memory limit of each sandbox container: one cgroup for its executor, its children and the tools they start; 4g and 8g |
WEB_ASSETS_DERIVE_IMAGE_PIDS, WEB_ASSETS_DERIVE_SCENE_PIDS | Process limit of each sandbox container, which counts threads; 512 and 256 |
WEB_ASSETS_DERIVE_APPARMOR | AppArmor setting of the sandbox containers; unconfined, or the name of a host profile that allows user namespaces and mounts (see sandbox containers) |
WEB_PREVIEW_TRANSFERS | Smaller transfers one content process runs at once — previews and small downloads; 8, at most 64. The API keeps the default for its own small reads (README, text, comparisons) |
All public certificate files in deploy/local/trust appear read-only under
/etc/scribe/trust. Different auth and storage CAs are supported.
For plain storage, set WEB_SCRIBE_SERVER=tcp://host.docker.internal:7447
and clear both CA paths, both server-name overrides and the auth endpoint.
Every repository the plain server exposes is shown as public; login is disabled.
For authenticated storage, use tls://host.docker.internal:7447 with its CA
and the auth service. For multiple storage servers, use the gateway endpoint;
the gateway owns routing and the UI/API need no list of storage backends.
For remote deployments, use actual DNS names and clear the local 127.0.0.1
verification overrides, unless a different explicitly trusted identity is needed.
The published Compose default is HTTP so a clone of this repository does not
need a certificate. Serving HTTPS with your own certificate, or one
deploy/issue-tls.sh obtains from Let's Encrypt through Cloudflare DNS-01
(the ACME client writes a TXT record on the public zone; split-horizon A/AAAA
records stay on the local resolver), is described in serve
HTTPS. Missing
certificate files refuse startup; nginx does not fall back to HTTP.
The browser always uses relative /api/v1 URLs. Changing upstream endpoints does
not rebuild Next.js. The API's process environment uses SCRIBE_* names shown
in compose.yaml, plus JWT_KEY_FILE, SURREAL_PASSWORD_FILE, SURREAL_URL,
SURREAL_USER and VALKEY_URL for deployments outside Compose, and PORT, the
HTTP listen port (default 4000, which the images set and nginx expects for
api and content). Numeric settings take whole decimal numbers within their
range: an empty one takes its default, and a malformed one ( 5, 5.0,
1e3) stops the process at startup. Files add OBJECTS_URL (object storage's S3
endpoint, for the API, the content container, the worker and both asset
processes; unset turns bundle publishing and previews off),
OBJECTS_ACCEL_PREFIX (the internal nginx location that proxies it,
/_objects/ in the shipped nginx.conf, so downloads are
X-Accel-Redirect replies and nginx fetches the object; unset makes the
process stream it itself), BUNDLES_RUN_LIMIT_MB and BUNDLES_UPLOADS.
The content service is a program of its own (apps/content, its own image)
that takes every request moving file bytes: repository file downloads and
previews (GET /repositories/:repo/content), bundle downloads and the
runners' two upload routes, plus /health. It has the same environment and
connections as the API and the same session and token authentication, and
none of the API's pages. nginx picks it with a map over the raw request URI,
where a namespaced address ns%2Frepo is still one path segment (location
matching sees it decoded). The API registers the same route modules
(@scribe/http/content-routes.js, @scribe/http/bundle-uploads.js), so its
OpenAPI document describes one surface and a request nginx does not route
still works, only in the API's process. Downloads are rate limited per address
per route (600 a minute each) in a budget of their own, not the API's 240;
the upload routes have no per-address limit, since a build farm behind one
address legitimately sends thousands of parts. nginx routes the announce and part requests unbuffered (proxy_request_buffering off, client_max_body_size 17m),
so a part never lands on nginx's disk on its way to object storage. A large
download, a gallery of previews or a farm uploading at line rate never slows
the API that serves pages. The worker's bundle.verify job reads each
published file back and compares it with the BLAKE3 the runner sent.
Browsing and authentication
The homepage has a workspace sidebar, a feed of real accepted main-branch changes, a repository list and recently created repositories. Signing in adds repositories allowed by the user's current Scribe permissions. Public visibility is administered by Scribe auth; the web application never makes an existing repository public.
Sign-in links open /login, a responsive full-page form with password visibility
control, pending/error states and the existing Scribe credentials flow. A validated
next parameter returns users to their repository, branch, revision and path;
external URLs, API endpoints and login loops are rejected. Anonymous queries are
cancelled before publishing the authenticated session so stale responses cannot
replace the newly signed-in identity. The former login dialog is removed.
Repository browsing includes branch selection, shareable explicit revisions, folder navigation, paged file trees and history, text previews with line numbers, safe raster image previews and verified downloads. Queries retain explicit branch/revision context rather than changing a workspace. Loading, empty and error states are included, with responsive layout and reduced-motion support. Repository data renders directly when queries settle; it does not wait for an exit animation while the initial URL is pinned to a revision.
Each History row expands to the files that change touched, paged by path, with an added/modified/deleted status, the file's size and a link to it at that revision (deleted files link at the change's parent). Each file expands again into a bounded, line-numbered text diff against the parent, loaded a page of lines at a time; binary and oversized files report that instead of a comparison.
Each change also has its own page at /commit/<branch>/<change>, opened from the
#N link of its History row or from the activity feed: its description (editable in
place by writers), author, time, branch and parent, then every file it touched with
its diff, a sidebar that filters and jumps between files, and Expand all / Collapse
all. Image and model files with a viewer show it behind the viewer's Open step, so
scrolling past a large asset never downloads it. The page stays responsive for very
large changes:
- A change of at most 25 files opens with every diff shown; a larger one opens with every diff collapsed and says so.
- Expand all is withheld above 100 files, since every open diff is a compare request.
- A diff reads only once its card is open and near the viewport, and pages of at most 200 rows load on request.
- A diff changing more than 400 lines waits behind Show diff.
- Files load a page at a time; the remaining ones are listed after Show more files.
The logo and plum/citron/porcelain brand palette come from the sibling
scribe-website project. The appearance menu in the header offers Light, Dark
and System themes and the language selector. Preferences are device-local;
theme changes synchronize between tabs and system mode follows OS changes.
All interface text, plural variants and mapped API errors live under
apps/web/apps/web/src/i18n, one catalog per language. To add a language,
provide a complete typed catalog, register it in src/i18n/index.ts, and add its
selector option. Dates use the selected locale through Intl. Repository names,
file contents and user-authored change messages are never translated.
The /oauth routes are first-party session endpoints, not an OAuth 2.0/OIDC
authorization server. Scribe auth verifies username/password. Each web session
holds its own Scribe credential: at sign-in the API generates a P-256 device key
and a credential id for the session and, before login is transmitted, SurrealDB
durably stores the key sealed to the session (JWE, under a key derived from the
JWT key for that purpose alone) with the login/logout operation IDs. The API then
logs in to auth with the password and the key's public half, and stores the
credential certificate auth answers. For storage it fetches hour-long access
certificates with that certificate, keeps them in memory and renews them at half
their lifetime; the private key never leaves the API. Passwords are never
persisted. Browser JWTs contain a session reference, not the key or a
certificate. See identity.
Access JWTs last five minutes; refresh JWTs rotate atomically and expire after at most seven days or the upstream expiry, whichever is earlier. Before refreshing, the browser persists a random operation ID and serializes rotation across tabs. SurrealDB atomically records the new digest and an encrypted refresh outcome. Retrying the same operation with its old or new cookie returns the accepted refresh cookie and a fresh short-lived access JWT, including after API restart. Reusing an old cookie with a different operation ID still revokes the session family. Both cookies are HttpOnly and SameSite=Strict; HTTPS origins enable Secure. Every mutating request requires the exact configured Origin, and every authenticated read checks the durable session. Browser code never stores tokens in localStorage. Logout immediately revokes web access; a background worker retries upstream logout with the persisted operation ID. Failed or uncertain attempts cannot block later revocations indefinitely.
Missing access cookies trigger renewal before the client publishes anonymous session data. Network errors, timeouts, rate limits and server failures never mean logout: cached identity remains visible while refresh retries with bounded backoff. Focus, visibility and online events resume recovery, and successful renewal revalidates session and failed queries. Only a definitive refresh 401 ends the browser session. No mutating application request is blindly retried on a network failure.
Connecting a desktop or device client
Native clients do not hold browser sessions. /authorize is a consent page a
client sends someone to in their own browser; approving it issues that client a
personal access token, which it then uses as Authorization: Bearer. No password
is ever typed into the client, and nothing has to be copied by hand.
The client generates a random verifier, opens
/authorize?name=&scopes=&days=&challenge=&state=, and waits. challenge is
base64url(SHA-256(verifier)). The page requires a signed-in browser session,
names the client and the account it would act as, lists the scopes in full, and
on approval calls POST /api/v1/oauth/authorize — a browser-only route, so a
credential can never mint a wider or longer-lived successor for itself. The code
comes back through scribe://web-auth?code=&state=, and the client exchanges it
at POST /api/v1/oauth/authorize/claim with the verifier it kept. A code that
travels through a URL is therefore worth nothing to anyone who reads it out of
shell history, a redirect log, or another handler registered for the scheme.
Every /authorize field is validated in apps/web/src/lib/device-authorization.ts;
a request that is not completely well formed is refused rather than repaired,
and name is only ever rendered as text. The destination survives the sign-in
round trip. The redirect scheme is fixed in code, not taken from the request, so
an approval cannot be aimed somewhere unintended. Because the claim request
proves itself from its own body and reads no cookie, that route is marked
cookieless and is exempt from the Origin requirement — the only such route,
since a browser tricked into calling it achieves nothing.
Claiming is single use: the credential is created pending, is denied by
middleware and swept by the revocation worker until claimed, and activating it is
what a replay then collides with. Only the read scopes in deviceScopes are
delegable this way; write access is refused, because a native client performs its
writes over the Scribe protocol under the user's own credentials. Clients sign
themselves out with DELETE /api/v1/account/tokens/self, and the credential also
appears in Settings alongside every other token, revocable from a browser.
GET /api/v1/session accepts a Bearer credential as well as a cookie, so a
client can ask who it is the same way a browser does. Credential management is
never exposed as an MCP tool.
Opening in the desktop app (scribe://open)
Open in desktop app (the asset page and the file page) hands a repository
or one of its files to the macOS or Windows app, which already owns the
scribe:// scheme for sign-in:
scribe://open?repository=<namespace/name>&server=<endpoint>&web=<origin>[&path=<path>][&branch=<branch>]| Parameter | Meaning |
|---|---|
repository | Required. namespace/name, as the server names repositories |
server | The Scribe endpoint a workspace of it connects to (tcp:// or tls://, as /repositories/:repo/clone gives it) |
web | The origin of the web interface that made the link (https://…, no path) |
path | A repository path; absent means the repository itself |
branch | The branch the page showed; absent means no preference |
At least one of server and web is required; the web interface always sends
both. Values are percent-encoded; each parameter appears at most once; the host
is open, with no user, port, path or fragment; an unknown parameter or a
value that breaks its rule (the core's repository-name, path and branch rules,
the apps' server-address rule) makes the whole link invalid. One Rust parser
in scribe-client, exported through scribe-ffi, is the only implementation:
both apps call it and neither re-validates or repairs a link.
The app matches the link against the workspaces it knows (its recent list with
their cached server and repository): a workspace matches when its repository
name is the link's and either its server is the link's server (compared as
the apps compare server addresses) or it is paired with the link's web
origin (both origins in the form scribe_web_origin writes, the one origin rule
the apps store and compare pairings by). One match: its window opens or comes
forward and the file is selected and revealed, or a notice says the path lies
outside that workspace's view (decided by the core's view rule through
scribe_view_includes; a folder that holds part of the view is shown), or that
the workspace is on another branch than the link's. Several: the app
asks which. None: it offers to create a workspace for that repository, with
the server filled in and shown for confirmation, or to locate an existing
folder.
A link changes nothing by itself: it never syncs, switches branches, takes a
lock, contacts a server the person has not confirmed, or opens the file in
another application. Any web page can start a scribe:// link, so everything
past selecting and showing waits for a click in the app.
HTTP v1 surface
apps/web/apps/api/src/routes/v1.ts is registered once with Fastify's /api/v1 prefix;
there are no unversioned aliases. A future version gets a separate route module.
Its shared withAuth pre-handler resolves and verifies the durable session once
and exposes request.webSession. Routes default to required authentication;
public discovery/browsing explicitly use auth: "optional", while health and
the auth lifecycle use auth: "none" and their dedicated credential checks.
Repository roles are enforced by authoritative Scribe operations using that
identity, including cache-hit authorization. No role claims or parallel role
database are invented in the web layer. Add role-specific middleware only when
a route needs a corresponding authoritative capability check.
Method / path under /api/v1 | Behavior |
|---|---|
GET /health | API/database readiness; does not claim upstream availability |
GET /session | Current user and whether login is configured |
POST /oauth/login | JSON username, password; sets cookies |
POST /oauth/token | JSON grant_type: "refresh_token", UUID operationId; durably rotates cookies with safe retries |
POST /oauth/logout | Revokes session and clears cookies |
POST /oauth/authorize | Browser-only. Approves a waiting device client; returns a code bound to its challenge |
POST /oauth/authorize/claim | JSON code, verifier; single use, no ambient credentials; returns the credential once |
DELETE /account/tokens/self | Revokes the credential making the request, so a client can sign itself out |
GET /repositories?q=&cursor= | Visible repositories across every served namespace, 48 per HTTP page; name is the address <namespace>/<name>. Each row carries presentation: { kind, kinds, cover }: the kind of its accepted .scribe/config.toml on main, its files by kind and a cover thumbnail (see Previews), cached per repository and main head for a minute; null when not ready within about 300 ms (the work finishes into the cache for the next listing), and for tokens without contents:read |
GET /feed?cursor=…&hide=… | Up to 12 entries per page — main-branch changes, proposal events and flow runs across at most 8 visible repositories; hide leaves kinds out |
GET /repositories/:repo/branches | Branch IDs/names/heads and repository incarnation |
GET /repositories/:repo/assets | The asset catalog of main: folder (a repository path, checked by the shared path rule; empty for the root, otherwise 400 folder) + depth (children: files and subfolder summaries; all: everything below), or q to search names and paths; kind, author, changedAfter/changedBefore (milliseconds, safe integers), tag (comma-separated; files with every tag, from any source), excludeTag (comma-separated; files with none of them), untagged=true (files with no tag from any source), status (comma-separated status IDs, none for files without one), field (comma-separated id:value of choice, person, date or number fields; 400 unknown_field, 400 unfilterable_field for a text field, 400 invalid_field for a value the field cannot hold; a number in any spelling) and range (comma-separated id:from..to of date or number fields, bounds inclusive, either may be empty; 400 invalid_range) filters; sort (name, changed, size, kind), order, cursor, limit ≤ 200. index.status is pending, building, behind or ready; locks says how much of the live lock state the items' lock carries (see Reads). Items carry tags (each with its sources), status and fields as the catalog mirrors them, an edit included once it answers |
GET /repositories/:repo/assets/file | One file's catalog entry by path, with processed metadata, missingReferences, its live lock and locks, and its references resolved from its folder |
GET /repositories/:repo/assets/facets | File counts for the same filters, refused alike: by kind, author, status and each choice or person field, each ignoring its own filter, and the 30 most frequent tags with every filter applied (a rule's tag only where its folder can give it); 300 a minute |
GET /repositories/:repo/assets/properties | One path's stored tags, status and fields as the schema has them now, every tag it has with its sources, who last edited and canEdit (see Tags, statuses and fields) |
PATCH /repositories/:repo/assets/properties | One edit for up to 500 files of branch (main when omitted): addTags, removeTags, status, fields; listings show it when it answers; Writer, token scope contents:write, 120 a minute |
GET /repositories/:repo/assets/properties/log | One path's edit log, newest first, 50 a page; reason is edit, move, delete, merge or schema |
GET /repositories/:repo/assets/schema | The repository's statuses, fields and tag aliases, its version, updatedAt and updatedBy of the last schema edit or merge, and canEdit (Maintainer) |
PUT /repositories/:repo/assets/schema | Replace the statuses and fields over the version read, with replacements for every removed status and choice (400 replacement_needed, 400 invalid_replacement); Maintainer, token scope repositories:write |
POST /repositories/:repo/assets/tags/merge | Merge (or rename) tag from into to; Maintainer, token scope repositories:write |
GET /repositories/:repo/assets/tags | Every tag of the files of main with its file count, sources and aliases; q keeps those starting with it (400 invalid_tag when it reads as no tag); 120 a minute |
GET /repositories/:repo/assets/used-by | The files of main whose references name path (a texture's models, a library's scenes), in path order, with locks; cursor, limit ≤ 200 |
GET /repositories/:repo/activity-assets | Up to four thumbnails of files a submit to main wrote and more, how many further ones it has; branch is a branch ID, as feed entries carry it, and revision the submit's change number (a safe integer) |
GET /repositories/:repo/assets/status | The report behind Settings → Asset index: whether the deployment runs an asset index (configured), index status, last progress, files, bytes, kinds; each processor's tier, sandbox lane, extensions, whether it makes previews, and its done/failed/eligible counts; the due time, attempts and generic last error of the index job, the processing job and each derive lane's job (jobs.derive, null when the deployment makes no derived data), and each job's failure (the last failed attempt's time, class, stable code, a message only when Scribe's services wrote it, and frames), sent only to a caller with canRebuild and null for everyone else; and canRefresh (Writer) and canRebuild (Maintainer) for the caller, which a token has only with repositories:write as well |
GET /repositories/:repo/assets/events | The recent history of the repository's asset jobs, newest first, at most 50 across the index job, the processing job and each derive lane's job: failed (with the failure), retrying, recovered and completed, each with its job (index, process, derive), lane, attempts and, for a processor job's runs, the files processed, failed and crashed; runs with nothing to do are not recorded. Maintainer, as for a rebuild; token scope repositories:write |
GET /repositories/:repo/assets/processors/:processor/failures | The files the report counts as failed for one processor, in path order: each file's path, the change that last wrote it and changedAt, the processor version that failed, the recorded reason and, for a sandbox processor, the attempts counted on its bytes (null when none were); total, cursor, limit ≤ 200 (50 by default); 404 unknown_processor for an ID the deployment does not run. Maintainer, as for a rebuild; token scope repositories:write |
POST /repositories/:repo/assets/refresh | Wake the index and processing jobs now (Writer); discards nothing |
POST /repositories/:repo/assets/rebuild | Discard the catalog and rebuild it from main (Maintainer); cached processor results are reused |
GET /repositories/:repo/configuration | { kind, cover, tags } from the accepted .scribe/config.toml on main: kind (generic or art; generic without one), [presentation] cover (null when unset) and the [tags] rules, each tag in its canonical spelling (empty when unset); scope contents:read, cached per repository and main head; 502 repository_config when the accepted file cannot be read as a configuration |
GET /repositories/:repo/tree | branch, revision, optional path, cursor |
GET /repositories/:repo/history | branch, revision, optional path, cursor |
GET /repositories/:repo/changes/:change | branch; the change's history entry, including the parent its files compare against. 404 change_missing when the branch has no change with that number |
GET /repositories/:repo/graph | Optional focus branch (default main) and cursor; commit ancestry across branches |
GET /repositories/:repo/file | branch, revision, path; metadata/preview |
GET /repositories/:repo/content | Same selection, optional download=1; verified bytes. ETag is the content digest (If-None-Match → 304); one Range: bytes=… → 206 |
64-bit revisions, byte counts and timestamps in repository responses are decimal
strings, avoiding JavaScript precision loss. IDs and digests are full hex strings.
Errors have { error: { code, message, requestId } } with no credentials or
upstream diagnostic contents. A path storage refuses as not a repository path
(INVALID_PATH) is 400 invalid_path.
Every request is checked against its route's schema exactly. A JSON body is
never coerced: null stays null, "5" is not a number and "true" is not a
boolean. Query values are text and are converted only where the schema says so
(a page limit, unread=true). A u64 decimal (a revision, change or lock
generation) past 18446744073709551615 is refused like any other malformed
value. A request the schema refuses, including an unknown query parameter or
body field, is 400 validation with a message naming the first problem and
where it is (Invalid request: body.events.0.exitCode: …); the old
per-field codes for these cases (revision, path, password_length,
commit_invalid, cherry_pick_invalid, resolution_text, resolution_limit,
code_range, resolution_path) are gone, and a missing or malformed X-Scribe-Lease or X-Scribe-Owner
header is refused the same way. The framework's own refusals keep their status
and say what they are: 413 too_large, 415 unsupported_media_type,
406 not_acceptable, 429 rate_limit, and refused for any other 4xx.
A branch named by name or ID that does not exist is 404 branch_missing on
every route. When Auth administration cannot be reached the maintenance routes
answer 503 admin_unavailable; a fault of the API itself is a logged 500.
Performance and present limits
The repository's Graph tab draws stored commit parents, merge parents, cherry-pick sources and branch origins with colored lanes, explicit event labels, and links to each revision. It shows current branch history independently of the file browser's pinned revision. Solid edges represent parents and merge ancestry, which does not imply that every source file's bytes were selected during conflict resolution. Dashed edges represent individual cherry-picks. Faded stubs mark sources outside the loaded page. Branch heads also label branches that have not accepted a new change since being created.
Graph reads include up to eight branches: the focused branch, main, and the
most recently active remaining branches. The response reports totalBranches;
the UI explains when branches are omitted and allows choosing another focus.
Each request reads two branch logs at a time, deduplicates shared ancestors by
the repository-wide change sequence, and returns at most 50 changes. Pagination
uses the newest continuing upstream cursor as a safe boundary, so a busy branch
cannot lose changes behind a sparse branch's older entries. Each page checks
current repository access and uses the caller's history:read scope. History and
branch responses retain the native branchId, parent, mergeParent, cherryPick and
origin fields without changing the wire protocol. The graph includes keyboard
scrolling, textual ancestry links, light/dark colors and loading placeholders;
colors are never the only way to identify a branch, merge or cherry-pick. Merge
connections use solid lines and double-ring nodes; cherry-pick provenance uses
dashed lines and diamond nodes, with separate badges and exact source links. A
cherry-pick never becomes a second parent in the layout. Sources outside the loaded
history retain an unresolved endpoint and source link until their commit is loaded.
Tree pages contain at most 100 recursive entries; history pages contain at most 30 changes. Valkey caches metadata for 30 seconds, keyed by user/session, repository incarnation, branch mapping and revision. Each cache hit still performs a live authorized Scribe read. Metadata cache read/write failures fall back to uncached reads; shared admission counters must remain available, and the database is required for sessions. Authorization remains subject to Scribe's own bounded authorization leases.
The API admits at most 12 simultaneous native workflows per process. File
transfers are admitted in two lanes of their own, per process — in content
for downloads and previews, in the API for its README, text and comparison
reads: WEB_PREVIEW_TRANSFERS (default 8)
for transfers up to 64 MiB and WEB_DOWNLOADS (default 2) for larger ones. A
transfer is sized by the bytes it sends (a range's length, otherwise the file's
size), and a full lane answers 503 transfer_busy without affecting the other
lane or metadata requests. Transfers lease connections from a separate pool, so
a slow download never holds a metadata request's slot. Connections are leased exclusively to one workflow and keyed by both
session and access certificate, so a renewed certificate opens new connections. Up to 12 idle connections may be retained for ten
seconds; failures discard the connection. Each lease checks the durable web
session, and native requests still enforce Scribe's authorization leases. Cache
hits pinned to branch IDs can use an exact repository lookup instead of listing
all branches. Alias-dependent responses still check current branch mappings.
HTTP downloads stream up to 8 GiB with at most 16 MiB of chunk payload per native batch. Worker threads verify the manifest and each chunk before emission. The last chunk is withheld until the full content digest matches; Content-Length and an aborted response prevent truncated transfers from appearing successful. HTTP backpressure holds both the native lease and download admission until delivery finishes.
Content responses carry the content digest as a strong ETag and
Cache-Control: private, no-cache: a browser may keep the bytes but revalidates
each use, so every reuse passes a live, authorized metadata read, and a matching
If-None-Match is answered 304 with no transfer. One Range: bytes=a-b,
a- or -n is answered 206 with Content-Range; only the chunks overlapping
the range are fetched, and each is verified against the accepted manifest before
its bytes are sent. A partial response cannot check the whole-file digest, which
needs every chunk; a range covering the whole file is checked as before.
If-Range with another tag, several ranges, or another unit gets the whole
file; a range starting past the end is 416. Disconnects close the native connection. No automatic retry is applied
to a partially sent response. Text previews remain bounded to 512 KiB and 5,000
displayed lines; raster previews to 16 MiB. HTML and SVG never execute inline.
The SDK's buffered convenience method retains its 64 MiB maximum.
The home feed loads its initial page and up to two more batches automatically on scroll. Older activity then requires the “Load older activity” button for each batch. Returning to an already loaded feed preserves this limit; explicitly refreshing the feed starts it over.
The home feed samples up to eight repositories with history from the first bounded catalog page, fetching at most three histories concurrently and at most twelve history entries per repository per request. It returns up to twelve merged changes and an encrypted, caller-bound cursor valid for one hour. Cursors pin main branch heads and advance only entries actually emitted, so concurrent commits and unequal native page sizes do not cause gaps or duplicates. Every page rechecks repository access and public/private visibility. Refresh starts a fresh snapshot. The browser loads another page near the bottom of the timeline, retains existing cards on a load failure, and includes a keyboard-accessible load/retry button. Refresh resets the feed to its first page; window focus does not reload accumulated pages. It reports the sample size and whether further repositories exist. It is a sample, not a complete global activity index; each repository's History view remains complete.
Each sampled repository contributes its flow runs to the same merge, from
the web's own store rather than from Scribe: every run of every branch, newest
first, alongside that repository's submits and the proposal events. Runs are
shown to whoever may read the repository, so a public repository's runs are on
the anonymous feed too, and a private one's are marked private like its
submits. Adjacent runs of one repository on one day read as a single block,
each line carrying the flow, the run number, its status, branch, bundle count
and duration, and linking to the run. Reactions work on a run as they do on a
submit ({ kind: "run", id }). An automation token without flows:read
never sees runs, as one without proposals:read never sees proposal events.
Show beside the refresh button opens a popover of the feed's item kinds —
submits, proposals, flow runs — each a checkbox; the last one shown cannot be
unchecked, and the trigger counts what is shown once something is left out.
The choice is part of the feed's query key and is fixed for a cursor when its
first page is minted (GET /feed?hide=changes,runs), so paging never mixes
shapes. A feed without submits reads
no history and resolves no branch; it pages the runs and proposals alone. A
signed-in reader's choice is saved on their profile (feedHidden, below) and
follows them to every browser; the browser keeps a copy so the first feed read
already honours it. A reader who is not signed in keeps it in the browser.
Proposal events split by what they say. What happens to a proposal —
created, published, closed, reopened, merged, archived — reaches everyone who
may read its repository, the anonymous feed of a public one included. The
discussion — comments, reviews, reviewer, assignee and label changes, automatic
updates, a rejected merge — reaches only the proposal's audience: its author,
reviewers, assignees, anyone who commented or acted on it and anyone mentioned.
Each event is stored once per audience it reaches (one row for everyone, or one
per participant) and access is rechecked on every read, so a row never grants
access to a repository. Activity written before lifecycle events became public
is folded into the shared row once, when the API connects to the database. A
reaction on a feed event is the same record as on that event in the proposal's
discussion ({ kind: "event", id: "<proposal>:<event>" }).
Repository discovery collects each identity's native directory on one leased connection, then produces stable repository-ID pages. Gateway continuation tokens are opaque and connection-bound; they are never exposed as HTTP cursors or compared lexicographically. Native discovery rejects repeated cursors and caps collection at 100,000 repositories/pages. Directory pages merge in stable repository-ID order at a shared cursor frontier so differing visibility cannot skip repositories or mislabel private results. HTTP pages return at most 48 matches. Search filters this bounded window and may return an empty page with a non-null nextCursor; clients must continue using that cursor to search later windows. REST, MCP, repository selection and the feed use the same discovery policy. Directory collection is a current cost of adapting connection-bound discovery to stateless HTTP; file, workspace and activity contents remain separately paginated. An indexed global search remains a separate feature. Folder navigation groups a recursive tree page; very large directories can need additional page loads.
Rate-limit counters are atomic fixed-window Valkey counters shared by all API
replicas, isolated by origin, route and source. Admission fails closed if the
counter store is unavailable. Valkey uses noeviction so cache pressure cannot
silently reset active limits; metadata writes remain optional and expire after
30 seconds. Monitor Valkey memory and capacity errors when sizing deployments.
Valkey also carries the live events behind a run page: every process that
changes a run or stores a log chunk publishes it (flow.run.<id>,
flow.repo.<scope>.<repo>, flow.log.<run>.<job>.<step>), the API replica
holding a tab's event stream forwards what the tab follows, and a tab's
subscription requests reach that replica over flow.hub.<stream>. Delivery is best effort — a viewer that connects reads
the store first, and a lost connection resumes from the last event it saw —
so a Valkey restart costs a reconnect, never a missed line. See
live streams.
Revocation workers atomically claim each due session for two minutes, using a
unique worker identity. Only the current claim owner can complete the job;
expired claims can be reclaimed after a worker crash. Native logout remains
idempotent if a slow worker overlaps a reclaimed lease.
Asset catalog
Every file of every repository's main branch is catalogued in OpenSearch:
path, folder, name, extension, kind (the same table that picks file icons,
fileKind in the shared contracts), size, content digest, the change that
last wrote it, that change's author and time. It powers folder listings with
counts and byte totals, search by name and path (per repository), and
filters. The catalog holds only data derived from Scribe, so it can always be
rebuilt; nothing a person types is kept only there.
Who builds it. The worker's repository watcher (the flow.watch job, which
also triggers flows) wakes an asset.index job when a repository's main head
moves, and registers each repository it sees once per worker process, so a
missing catalog is built without waiting for a submit. The separate assets-index
process claims these jobs from a queue of its own (<realm>-assets-index), so a
first index of a large repository never delays flows or proposals.
asset.index walks the tree at a pinned head on the first pass (resumable,
a bounded number of 1,000-file pages per run) and afterwards applies the paths
changed since the last indexed change. Authors come from main's history, one
call per thousand changes, cached; a change the history of main does not
name (written elsewhere and merged) is recorded with an empty author rather
than a guessed one. A restored repository (new incarnation) is removed and
rebuilt.
Processors are the layer above, in two tiers, listed together in
packages/service/src/assets/processors/registry.ts. Every processor declares
the metadata fields it writes; for any one extension a field has one owner,
the registry refuses a build where two processors would write the same field
of the same file, and a result carrying an undeclared field is recorded as a
processor error instead of stored.
Header processors run in assets-index: one module per file format, each a
pure function of at most its declared head bytes and, when it names one, one
more range of at most that size elsewhere in the file (a TIFF whose directory
follows its image data), read with verified range reads, so a 3 GB PSD costs a
small read. They return shared, filterable fields
(dimensions, channels, bit depth, colour space, layers, mip levels, duration,
sample rate, meshes, triangles, materials, animations, bones, references, and
names — clip, material and blend-shape names, found by search like a path)
and a details object stored but not indexed, each within the bounds of the
contract's metadata table; a result outside them is recorded as a processor
error, never stored. Every header processor's parsing and its field list live
in @scribe/headers (packages/headers), which the browser's model converter
shares; a processor names its reader, head and fields. asset.process runs processors once
per distinct content and processor version, recording each batch's results in
one write; a result another repository
already has for identical bytes is reused without reading them. Bytes that are
not the format are recorded as failed for that version and not retried until
the version is raised, which reprocesses every file of the format in the
background: assets-index wakes the processing job of every catalogued
repository when it starts, and assets-derive the jobs of its lanes, so a
raised version reaches repositories nobody submits to. Recording a result
replaces the fields its processor declares, so a field an earlier version
wrote does not linger. Processor IDs are at most 64 lowercase letters, digits
and dashes, not starting with a dash. Adding a format is a reader in
@scribe/headers, a descriptor module and one registry line.
Sandbox processors run in the separate assets-derive process and decode
whole files: raster (libvips: PNG, JPEG, WebP, AVIF, GIF, TIFF; whether the
alpha channel is used; sharp's default pixel limit, about 268 megapixels,
refuses larger images), texture (OpenImageIO: the PSD composite, TGA, DDS
including block-compressed formats, EXR and HDR through the sRGB transfer
curve; an image declaring more than a gigapixel is refused before any pixel is
read, and one above 8192 pixels a side is scaled down by oiiotool before
the previews are made), audio and audio-wav (ffmpeg: facts WAV's header lacks, EBU R128
loudness and true peak, a waveform), video (ffmpeg: facts and a poster
frame), model-gltf, model-fbx and model (glTF/GLB, FBX, OBJ, STL, PLY,
USD/USDA/USDC/USDZ and VOX through three.js's readers and glTF Transform, in
the shared @scribe/models package: skeletons (details.skins counts them,
however many meshes each binds), blend shapes, animation
length, UV sets, size in metres, textures, cameras and lights, the names
inside, a view.glb and a rendered thumbnail) and blend (Blender files read
without Blender: the version that saved them, what they hold, the libraries
and images they link, and the thumbnail Blender embeds). An FBX header reader
adds version, creator, units (named from metres: cm, m, in, or the scale
itself, such as 2.5cm or 2.5m), up axis and frame rate at once.
Decoders run under budgets of their own besides the lane's file size limit.
A compressed .blend may decompress to at most 16 GiB in total (eight times
the scene lane's limit), with Zstandard frames decompressed whole only up to a
bounded size and streamed beyond it. A model is refused as unreadable past its
limits (modelLimits in @scribe/models): among them 2 GiB of decoded arrays
in the sandbox — a glTF's accessors, stored, sparse or compressed, and a USDC
crate's arrays, counted before they are allocated — and, in the browser's
preview worker, the raw-preview bounds of the shared contracts. A tool or child
that fails because of where it ran (killed by the kernel for memory, or unable
to start) is a crash and retried; what the file makes a tool do (a non-zero
exit, a timeout, runaway output) is unreadable and not retried. OpenImageIO
runs on two threads (OPENIMAGEIO_THREADS=2) and ffmpeg's decoders, filters
and encoders on two (-threads 2), so several children stay within the
container's process limit. The registry
holds only descriptions; the code that decodes lives in apps/assets-derive,
so the API and the index never load native decoders. Several processors may
share one implementation, each keeping only its declared fields. Each belongs
to a lane with its own queue (<realm>-assets-derive-<lane>): image for
work measured in seconds and scene for models, which may take minutes and
gigabytes, so a lane can run on a larger machine. assets-index wakes a
repository's lane jobs whenever its catalog changed, when OBJECTS_URL is
set.
There is no DCC in the image and no GPU is needed: a model is loaded with the
same three.js readers the browser uses, written into one glTF document
(metres, +Y up) and its facts read from that document, whatever the source
format. The thumbnail is rendered by three.js's WebGPU renderer through Dawn
(webgpu) on Mesa's software Vulkan driver (lavapipe), from a fixed
three-quarter view under a studio environment, framed without backdrops or
far strays — the viewer's own look and framing (@scribe/models/look:
studioLook sets the tone mapping and a 30° camera the model fills to 90 %
on its tighter axis, and frameCamera places it; points and lines count
towards the framing like meshes). Embedded textures libvips cannot read, or
that exceed its default pixel limit, are left out of the view.glb rather
than carried at full size, and counted in details.droppedTextures.
Licensed formats (Maya, 3ds Max, ZBrush, Substance) are left to flow runners.
The work is split across three containers, all from one image that only
assets-derive builds (one image: tag, so up --build never leaves them on
different builds). assets-derive, the supervisor,
claims the jobs, saves each file whole (verified, up to the lane's size
limit; larger files are recorded as too large) to the in-memory scratch volume,
and stores the results; it never decodes. Each lane has a sandbox container of
its own, assets-derive-sandbox-image and assets-derive-sandbox-scene, with
its own memory and process limits, running one executor for that lane
(ASSETS_DERIVE_LANE): the only process that starts children, in a container
with no network at all (network_mode: none), no secrets and no credentials.
The supervisor reaches each executor over a Unix socket,
executor-<lane>.sock, in the small derive-sockets volume
(ASSETS_DERIVE_SOCKETS, /sockets), which only the supervisor and the
sandboxes mount, never the scratch volume. The two speak one JSON line at a
time: the
supervisor names its lane, the executor says it is ready (with a fingerprint
of its build's sandbox processors, which the supervisor must match) when a
child of that lane could start at once, the supervisor counts the attempt and
sends one run of a processor of that lane, the executor answers with the
outcome. Each executor runs ASSETS_DERIVE_CHILDREN children at once (Compose
sets it from WEB_ASSETS_DERIVE_IMAGE_CHILDREN, 3, and
WEB_ASSETS_DERIVE_SCENE_CHILDREN, 1). The executor starts a child Node
process for that one file, in its own process group, with the lane's
JavaScript heap (1 GB for images, 4 GB for scenes) and timeout (2 minutes for
images, 15 for scenes) and only PATH, HOME, TMPDIR and NODE_ENV in its
environment (image children also get OPENIMAGEIO_THREADS, scene children
the software renderer's thread count and no shader cache), and refuses any
path outside the scratch volume or a link in place of the input.
Every child is jailed by bubblewrap (bwrap): its own user, PID, mount, IPC,
UTS, cgroup and network namespaces, no further user namespaces, and it dies
with its parent. It sees only its own job — its input read-only at
/job/input.<ext> and its output writable at /job/output — the system's
/usr and /etc and the program read-only, and a private 64 MiB /tmp: not
the scratch volume, another job or the executor's socket. So a decoder that
crashes, exhausts memory or is exploited reaches only the file it was given.
A child raises its own oom_score_adj to 1000, so when the container's
memory runs out the kernel's OOM killer ends a child or a tool it started,
never the executor serving the others. The executor refuses to start when
bubblewrap cannot create a child's sandbox, and says why; its startup probe
jails a child and, for the scene lane, creates a WebGPU device inside the
jail. ffmpeg is given the demuxer the extension
names and may open local files only, so a playlist disguised as a video is
refused rather than followed. The child writes its answer to
result.json, which the executor reads through one handle opened with
O_NOFOLLOW | O_NONBLOCK, only as a regular file no larger than the largest
answer the protocol's schema allows, and parses strictly; the supervisor
parses the executor's answer again and opens each preview file it names
non-blocking (a FIFO is refused, not waited on), only as a regular file
directly in that job's output under its fixed name, within the name's size
limit, never through a link;
anything else is a processor error. A signal, a timeout, a
non-zero exit, a closed connection or an answer outside the protocol is a
crash.
The attempt is counted after the executor is ready and before the child starts, so an unreachable executor costs a file nothing and a crash that takes the sandbox down still counts: after three the content is recorded as crashing and is not tried again at that processor version. Stopping the supervisor cancels its running files (their children are killed) and gives their attempts back, so a deploy during a long render costs nothing either. Files the tool refuses are recorded as unreadable, as in the header tier.
Previews. A sandbox processor declares the preview files it may make,
from a fixed list, and stores them per content in the derived bucket of
object storage under previews/<content>/<processor>-<version>/<name>:
256.webp and 1024.webp (always together, never enlarged, with a 16-pixel
placeholder inline in the catalog); view.webp, a full-size WebP (at most
8192 pixels) of an image browsers cannot open (PSD, PSB, TIFF); and
view.glb, a model as one GLB for the viewer: meshopt-compressed, textures
bounded to 2048 pixels as WebP, skeletons, animations and blend shapes kept,
simplified when it would be too heavy to view (details.viewTriangles then
says to what). Each is a function of the one file's bytes: textures and other
files a model refers to are never baked in, but stay references the viewer
resolves at the revision it shows, so a changed texture can never leave a
stale copy behind. Asset items carry
preview: { thumbnail, large, placeholder, view } (view is
{ url, format: "glb" | "webp" } or null), and feed entries of submits to
main show up to four thumbnails of files that submit wrote and that are
still current (GET /repositories/:repo/activity-assets; other branches have
none), with more counting the further thumbnails among the first 64
previewable files of the submit (a lower bound for a larger submit); the feed
reads them three at a time as entries scroll in, and gives up on one after 15
seconds. A repository's card takes its cover from the presentation of its
row in GET /repositories, so a directory page costs one request: the
thumbnail of the file [presentation] cover names in
.scribe/config.toml while it is on
main and has one, else of the newest file that has one among the 64 newest
previewable files, next to its file counts by kind. Cards show it in
interfaces that lead with pictures (the Art interface); interfaces that keep
identifiers out of primary view move a card's change number into its
tooltip. An earlier result is reused for identical bytes only while all its
files are still stored; assets-derive checks that again after recording a
reused result, and makes the files again if a sweep removed them meanwhile
(asset.derive.reuse_lost).
References. Processors keep metadata.references exactly as a file writes
them (a glTF image URI, an OBJ material library, a Blender link without its
// marker), each once, at most a thousand: a result belongs to the bytes and
is shared by every file with them, in any folder of any repository, so a path
resolved against one file's folder would be wrong for its copy elsewhere. Each
catalog row resolves them for itself, by the one rule the viewers share
(resolveReference in the contracts: no scheme, no absolute or drive path, no
.. past the root), into two fields:
referencePaths: the repository paths the references name from the row's folder, each once (a keyword field, soGET /repositories/:repo/assets/used-by?path=finds a texture's models).missingReferences: how many references (each entry as written) name no path at all — a URI, an absolute or drive path, a path out of the repository — or a path the repository has no file at.
Both are worked out whenever a row's metadata changes (recording a result
reads the rows with that content a page at a time, resolves each from its own
folder and writes it back only over the version it read), and the indexer
recounts missingReferences of every row whose referencePaths hold a path a
page of changes added or removed, so a texture added later clears the count and
removing it raises it again. tag=missing-references lists the files with any.
GET /repositories/:repo/assets/file?path= returns the same resolution per
entry, references with each path and whether main has a file there, so a
model's missing textures show without loading it; the entries with exists: false are the ones missingReferences counts.
Sweep. A recurring asset.sweep job (every six hours, resumable a few
listing pages at a time) walks the derived bucket and keeps a preview file
while a file of any repository has its content at its processor version. A
stored result no file refers to is noted, and its files go only when a later
pass, at least a day on, still finds it unreferenced; the result goes first,
so it is made again rather than reused without its files, and the references
are counted once more after it goes, so a file that came to refer to the
content just before keeps its preview files. Files without a
stored result go once a day old; keys the sweep does not recognise are
reported and never touched. A view.glb is swept like a thumbnail: it never
outlives the last file with its content.
GET /repositories/:repo/previews/:content/:processor/:version/:name is
served by content: it reads the repository under the caller's identity,
checks in the catalog that a file of that repository has that content with
that preview file at that version (a digest alone never opens one), and
answers with X-Accel-Redirect to nginx's internal /_objects/derived/…
location, which streams it from object storage (a view.glb gzipped on the
way: nginx gzips model/gltf-binary only in that location, so a repository's
own .glb from /content keeps its Content-Length, ranges and If-Range).
The URL names the content and version, so the file never changes, but a
browser keeps its copy only to revalidate it: a preview found is
Cache-Control: private, no-cache with its object key as ETag, and
If-None-Match with that tag is answered 304 by content after the
repository and catalog checks, without touching object storage. A copy left in
a shared browser after sign-out or lost access is therefore never shown again
without a fresh authorization. A miss carries no tag. 404 preview_unavailable means none yet; without object storage the route answers
503 previews_unavailable.
Reads. Every asset route first reads the repository under the caller's own
identity and only then the catalog for exactly that repository and
incarnation; locks shown with files are read live, as in the file browser,
for browser sessions and for tokens with locks:read. locks in /assets,
/assets/file and /assets/used-by says how much of that the items carry:
complete; partial, when the bounded lock read stopped early, so an item
without a lock may still be locked (the asset browser then says some badges
may be missing); or unread, for anonymous readers and tokens without
locks:read, when lock says nothing either way.
While index.status is pending or building, the listing is incomplete and
the tree endpoint remains the authority. Without OPENSEARCH_URL the asset
routes answer 503 assets_unavailable and the watcher wakes no asset jobs.
The same routes are MCP tools: search_assets, asset_file, asset_facets,
asset_used_by, change_previews (/activity-assets), asset_index_status,
refresh_asset_index, rebuild_asset_index, asset_job_events and
asset_processor_failures, with
repository_configuration for /configuration, and for tags, statuses and
fields asset_properties, set_asset_properties, asset_properties_log,
asset_schema, update_asset_schema, asset_tags and merge_asset_tags.
set_asset_properties is marked idempotent: an edit states its end state, so
repeating it changes nothing more. A body a tool sends that its schema refuses
answers 400 validation naming the first problem and where it is
(Invalid request: body.paths: …).
Tags, statuses and fields
Files of any repository can carry tags, one status and custom fields. They describe a path, not a branch or a version: every branch shows the same values for the same path, a file changed on main keeps them, and when a file moves on main they move with it.
Tags come from three sources, and a tag is one name whatever gives it:
- manual: added by people, on a file's page or to a selection in the asset browser;
- rule: the
[tags]table of the accepted.scribe/config.tomlonmaingives every file below a folder its tags (see Repository configuration; changing it needs a Maintainer); - derived: the file's own facts, as the contracts'
derivedTagsdefines them:rigged(a skeleton),animated(animation) andmissing-references(references main has no file for).
Every tag, typed or configured, goes through one rule, normalizeTag in the
contracts: lowercase, NFC, surrounding white space removed and inner runs of
it made one space; at most 64 characters as written, no commas (lists are
comma-separated) and no other control characters. A file carries at most 64
manual tags. Tags are free: anyone who may edit types what they need, and the
field suggests the repository's tags as they type. A Maintainer keeps the
vocabulary tidy by merging a tag into another (cars into car), which
also renames one when the target is new: the old name becomes an alias, files
show and are found by the new one at once, typing the old name adds the new
one, and the stored rows are rewritten in the background and logged as
merges. Derived tags cannot be merged away.
Statuses and fields are defined per repository by its Maintainers. Until
one edits them there are four statuses, wip, review, approved and
deprecated, whose labels are null so every client shows its own words, and
no fields. A status has an ID, a label and one of seven colours; a field has
an ID, a label and a type: choice (one of its choices), text (up to 1,000
characters), number, date (YYYY-MM-DD) or person (a username). IDs are
1 to 32 lowercase letters, digits and dashes starting with a letter; none
is no status ID, as filters use it for files without one. A schema edit names
the version it replaces. Removing a status or a choice names what files with
it get instead, another status or choice of the edited schema or nothing,
for every one removed, used or not (400 replacement_needed names the first
one missing, 400 invalid_replacement one the new schema lacks); removing a
field, or changing its type, clears its values. The schema applies at once:
files are read as it has them, and the stored rows are rewritten in the
background and logged as schema.
Filters. Listings and counts filter by tags (tag, all of them;
excludeTag, none of them; untagged=true, no tag from any source), by
status, and by field: field=id:value for choice, person, date and number
fields (a number in any spelling, 1.0 is 1; a text field is found by q
and refused as a filter), and range=id:from..to for date and number
fields, bounds inclusive, either empty (due:..2026-02-01, lod:2..), both
empty for files with any value. The browser keeps its filters in the page's
link and reads them by the same rules against the current schema, so a link
kept from before a field was removed or retyped, or naming a date the
calendar lacks, lists files without that filter instead of failing.
Storage. Properties live in the web tier's database, keyed by realm,
repository ID and path (not by incarnation, so a restored repository keeps
them), with an append-only log of every change: who, when, and what was
added, removed or changed. Every write of a repository, schema edits
included, takes the repository's next sequence number in its own
transaction, and each row is written only over the number it was read at, so
two people editing one file both land and a schema edit never removes a
status a file is being given at the same moment. An edit is checked whole
against the schema before anything is written. The asset catalog mirrors
the rows for search: an asset.properties job in assets-index, woken by
every edit and merge, applies them in sequence order, each only over an
older copy, and the index copies a path's properties whenever it writes the
path's row. An edit also applies the rows it wrote to the catalog before it
answers, so listings show it at once; should that fail, the edit still
stands and the job catches the catalog up. Moves on main are
found through Scribe's recorded lineage for every changed path: a file whose
origin is another path, gone at the new head, and that no other path descends
from, takes that path's properties and replaces what its own path had (the log
keeps it), so swaps and chains keep every file's own values; copies and
anything storage cannot trace to one file are left alone, never guessed. A
path deleted on main and not moved loses its properties (logged delete), so
a new file there starts clean.
Who may. Reading needs what reading the repository's files needs
(contents:read for tokens). Editing a file's tags, status and fields needs
write access to the repository (Writer; tokens also contents:write), and the
paths must be files of the branch named. Editing statuses and fields and
merging tags needs Maintainer (tokens also repositories:write). The web
adds no role of its own.
In the web. A file's page has a Tags and status panel: the status, every
tag with an icon for rule and derived ones (each links to the files with it),
a field that adds tags with suggestions, the fields, and the history. The
asset browser filters by tags, status and each choice or person field, with
counts; in its Select mode clicking cards chooses them (Shift chooses a
range) and a bar adds or removes tags or sets a status on up to all shown
files at once, 500 per request. Cards show their status as a coloured dot.
A card's details line (kind, author, when it changed) fits the card's width:
as cards narrow, the author's name goes first (the avatar stays), then the
date shortens to a compact age ("5d", "2w"), then the avatar goes. Hovering the
file name always shows its path, author and full date.
The repository's Settings tab (/<namespace>/<repo>/settings, no
branch) holds sections that each need a role, read from the permissions
endpoint: Tags and Statuses and fields need Maintainer (maintain), the
Asset index Writer (write on main). The tab shows to whoever may open one
section; opening a section without its role says which role it needs, and the
API refuses edits regardless. /settings is an overview with a card per
section the reader may open, saying where it stands (how many tags and
merges, the statuses, how many fields, who changed them last; the index's
state). Tags
(/settings/tags) lists every tag of main with its file count, sources and
aliases, and merges or renames one; Statuses and fields
(/settings/fields) edits both. Sections are one table in the page, each with
its panel, its overview card and the role it needs, so a new setting is one
more entry.
Asset index. A repository's Settings have an Asset index section
(/<namespace>/<repo>/settings/index, no branch: the index follows main),
open to Writers and Maintainers, and its card on the Settings overview names
the index's state, file count and last progress. The section shows whether the index is up to date,
building or catching up, the change it has reached against main's head, when
it last made progress, file and byte totals, files by kind, and each
processor's progress. Its health line reads the durable job records: idle,
working, retrying after a failure (with the attempt count and the job's
generic error), or stalled, when work has been due for over two minutes
without a service claiming it — the line names assets-index or
assets-derive, whichever is down. Each processor shows its tier and lane,
whether it makes previews, and how many files are done, failed and waiting.
It refreshes every five seconds while anything is in progress. Writers can
Check now (wake the jobs); Maintainers can Rebuild index after a
confirmation. A pass still running during a rebuild notices before it writes
its next page that its state is gone and stops, so it never writes discarded
progress back.
Maintainers — whoever may rebuild the index: the repository's Maintainer
permission, and for a token repositories:write as well — also see why things
failed. Under the health line, the retrying job's Last failure: the error's
class and code, its message when Scribe wrote it (an error from outside Scribe
keeps only its class and code, since its text could quote a URL or a
credential), when it happened, and where in Scribe's code it was thrown. When
the web services' own account was refused, the failure also says How to
fix it, with a command to copy: for a repository the account cannot read,
scribe-auth namespace-readers set scribe-web <namespace>, which covers
repositories created later too (a reader grant on the one repository is the
alternative); for a credential Scribe refused, a new credential file from
scribe-auth credentials create scribe-web --service (see
background jobs). The health line then names the
cause instead of the generic error. A
processor's N failed opens its failed files, 25 at a time: each path links
to its asset page, with the change that last wrote it, the processor version,
the reason the processor recorded and, for a sandbox processor, the attempts
counted. Recent activity lists the asset jobs' last 50 events: failures,
retries, recoveries and runs that got through files, with the job and lane,
the counts, the failure's code and when. Everyone else sees the counts and
the generic line only; the API sends them nothing more.
Storage. search is a single OpenSearch node on the internal data
network with the security plugin off: it is reachable only from the compose
project and holds nothing that is not derived. Indices are versioned
(scribe-assets-files-v5 and its state, change, processed-result and attempt
companions); a mapping change is a new version rebuilt from Scribe, and
starting a process creates its version's indices and drops those of lower
versions, never of higher ones, so an older build starting during a rolling
deploy leaves the newer one's catalog alone. (An older process still writing
after that makes OpenSearch create its indices again; the next start of the
newer version drops them, and nothing reads them meanwhile.) The API creates
the indices at start too, so a listing before assets-index first runs is
pending rather than an error; with OPENSEARCH_URL set, it does not start
while OpenSearch is unreachable.
Object storage. objects is a single-node SeaweedFS with its S3 gateway
(OBJECTS_URL, the endpoint's origin such as http://objects:8333), the web
tier's only store for files: the other volumes are SurrealDB's, the OpenSearch
catalog and the derive scratch space. It runs
on an internal network of its own that only api, content, the worker,
assets-derive and nginx join, without credentials. It holds one bucket per
purpose, named in packages/service/src/objects.ts and created by the
processes that use it: bundles holds what flow runs publish (see
Flows), which is kept data that exists nowhere else;
derived holds preview files (thumbnails, view.glb, view.webp), which can
be made again. An X-Accel-Redirect to nginx's internal /_objects/<bucket>/<key>
location (OBJECTS_ACCEL_PREFIX) makes nginx fetch an object the content
process authorized; each bucket has its own location, so previews are cached
for good and bundles pass range requests through and are never cached.
Sandbox containers
assets-derive-sandbox-image and assets-derive-sandbox-scene run as uid
1000 with every capability dropped, no-new-privileges, a read-only root and
network_mode: none. Debian's bubblewrap is not setuid: it jails each child in
user namespaces of its own, which needs no capability but three settings
Docker's defaults forbid:
seccomp=./deploy/assets-derive/seccomp.json: Docker's default profile plus the calls namespaces take (clone,unshare,mount,umount2,pivot_root).apparmor=${WEB_ASSETS_DERIVE_APPARMOR:-unconfined}: Docker's AppArmor profile forbids mounts. Name a host profile that allows user namespaces and mounts to keep one.systempaths=unconfined: lets bubblewrap mount a fresh/procfor each child's PID namespace. The container has no capabilities, so the paths this unmasks stay read-only to it.
This trades Docker's default AppArmor confinement and /proc masking on these
two containers for a jail around every child, which sees far less than the
container does. The executor refuses to start when bubblewrap cannot create a
child's sandbox, naming these settings. On Ubuntu 23.10 and later, AppArmor
also restricts unprivileged user namespaces for unconfined processes: set the
host's kernel.apparmor_restrict_unprivileged_userns=0, or load a host profile
that allows them and name it in WEB_ASSETS_DERIVE_APPARMOR. SELinux hosts may
also need label=disable on these containers; that is untested.
Each container's memory limit is one cgroup for its executor, its children and
their tools; when it runs out, the OOM killer ends the highest-scoring
process, a child or a tool it started (children raise their own score), never
the executor; the file is reported as crashed and retried, and after three
crashes it is recorded as failed. The process limit counts threads: three image children
with their tools and the executor fit in 512. Raise
WEB_ASSETS_DERIVE_SCENE_MEMORY with WEB_ASSETS_DERIVE_SCENE_CHILDREN. To move
scene work to a larger machine, set WEB_ASSETS_DERIVE_LANES=image here and
run the scene supervisor and sandbox there.
Repository stars
Signed-in users can star or unstar a repository from its header. The header shows the total count to anyone who can read that repository. Home includes a paginated Starred repositories list, with removal controls and a compact layout on smaller screens. Anonymous visitors are sent to login with their return location preserved.
GET /api/v1/repositories/:repo/stars returns { total, starred }.
PUT and DELETE on the same path explicitly set or remove the current browser
user's star; repeated requests are idempotent. GET /api/v1/account/stars lists
the user's favorites with an optional cursor. Mutations and the account list use
the shared browser-session and origin middleware; read access is checked against
Scribe before returning counts or changing membership. Listing favorites checks
current permissions again and omits repositories the user can no longer read.
The DataStore.stars repository owns persistence through the SurrealDB adapter.
Membership is keyed by deployment scope, immutable repository ID and verified
Scribe user ID, so repository renames retain stars. Membership and aggregate counts
are updated in one transaction, including concurrent duplicate requests. Lists
page twelve IDs at a time and resolve current names and visibility from Scribe.
No repository history scan is needed. The portable data contract covers concurrency,
owner/scope isolation, pagination and persistence across connections and reconnects.
Quick reactions
Signed-in users can react to activity with one of eight fixed emoji: 👍 👎 😄 🎉 ❤️ 🚀 👀 🤔. Reactions appear under every submit and proposal story in the home feed, under each submit in a proposal's Submits tab and under each event in the discussion timeline (comments, reviews, replies, metadata changes and automatic updates; deleted comments have none). Anyone who can read the repository sees the counts; only signed-in users get the picker and can toggle their own reactions. Reacting never needs write permission and does not create a proposal event, so it neither notifies participants nor invalidates approvals. Locked or archived discussions still accept reactions.
The emoji are bundled SVG artwork in apps/web/public/emoji/ (Noto Emoji, SIL OFL
1.1, notice and licence alongside), rendered as fixed-size images through the shared
Emoji component rather than the platform emoji font. Every operating system and
browser therefore shows the same glyph; profile status emoji use the same artwork. The set is declared once as reactionEmojis in the shared
contracts; the API refuses anything else and the picker, the summaries and the
tests derive from the same list.
PUT /api/v1/repositories/:repo/reactions takes
{ target, emoji, reacted }, where target is { kind, id } for anything
that carries reactions — one shape for every kind, so nothing but a row in the
API's registry stands between a new kind and its reactions. Today's kinds are
change (a number in the repository-wide sequence), event (a proposal
event, <proposalId>:<eventId>) and run (a flow run's id); each has an
existence check of its own, and a stored reaction is the kind, the id, the
owner and the emoji, under the repository's scope. The route answers
{ reactions } for that target as the caller now sees it. The request states the
desired end state, so retries are idempotent. Read access is checked against Scribe
on every call; a change above the repository head or an unknown or deleted event
is 404, as is a run that belongs to another repository. Automation tokens need
the opt-in reactions:write scope (a reader ceiling); MCP exposes the same
route as set_reaction.
Reads never need a separate request: the feed, the proposal detail and events
routes and the proposal submits route attach reactions (emoji, count,
reacted) to each entry, one persistence read per repository on the page. The
browser toggles optimistically and patches every cached query that shows the same
target — the feed and the proposal page agree without a refetch — and restores the
previous state if the request fails. Nothing new is written into Scribe.
The DataStore.reactions repository owns persistence through the SurrealDB adapter,
keyed by realm, repository incarnation, immutable repository ID, target and verified
Scribe user ID: renames retain reactions, a recreated repository starts over.
Membership and per-emoji aggregate counts are updated in one transaction, including
concurrent duplicate requests. The portable data contract covers concurrency,
picker ordering, absent targets and scope/repository isolation.
Operations and checks
The footer shows the web package release, the connected API's route version and
package release, and its configured storage/gateway wire version and transport.
Web and API releases come from their respective package.json versions.
GET /api/v1/version publishes only version metadata, without internal endpoints
or credentials and without contacting storage. Its wire value uses the SDK's
handshake selector (TCP v2, TLS v4); it describes the protocol required by the
connection, not an upstream health check. The footer refreshes this metadata after
five minutes when revisited or refocused and shows an unavailable value on failure.
When the API uses Compose service names such as tls://gateway:7557 and
tls://auth:7443, include the network override on every deployment or recreation:
docker compose -f compose.yaml -f compose.scribe.yaml up --build -d --waitSet WEB_SCRIBE_NETWORK to the existing Scribe network (for the local cluster,
scribe-local_default). Omitting the override detaches the API, the content
process, the worker and both asset processes (assets-index, assets-derive)
from that network; the derive sandboxes never join it.
The HTTP health endpoint checks API process readiness; also check repository
discovery and sign-in to verify upstream connectivity. Server failure logs include
safe diagnostic types and codes, such as ENOTFOUND for DNS resolution failures,
without logging upstream messages, credentials, or returning diagnostics to browsers.
docker compose ps
docker compose logs --tail 50 api
curl http://localhost:8080/api/v1/health
curl http://localhost:8080/api/v1/repositories
npm ci
npm run check
npm run build
cargo build --manifest-path ../Cargo.toml --locked -p scribe-cli -p scribe-server-cli -p scribe-auth-cli -p scribe-gateway -p scribe-runner
SCRIBE_INTEGRATION=1 npm run test --workspace @scribe/sdk
npx tsx tools/web-smoke.tsThe last command requires Docker Desktop host connectivity and the built SDK
(npm run build). It creates an isolated Compose project and temporary real
Scribe services, creates a service account that reads the studio namespace
and issues its read-only credential file for the worker and both asset processes with
scribe-auth, builds every image, then checks public/private
discovery, auth, verified content, refresh, API restart persistence and logout.
It deletes only its test project/volumes and fixture directory. Ordinary Scribe
Rust checks remain node tools/check.js fast from the repository root; the manual web CI workflow handles
the separate npm and container checks.
npm run check builds the shared packages, typechecks every workspace, lints,
runs every workspace's tests and checks formatting. Run npm run lint,
npm run format or npm run format:check independently from apps/web; they
cover every workspace. node tools/workspaces.mjs <script> runs a workspace script
in dependency order (--help lists its options); npm run build and the dev:*
scripts use it. Shared Oxfmt configuration includes web/API overrides;
Oxlint has a root policy and separate app configurations for React/Next/browser
and Node API checks. Prettier is not installed or configured.
Every service image ships /usr/local/bin/healthcheck. Node scripts live
beside each app's Dockerfile (apps/web, apps/api, apps/content,
apps/worker, apps/assets-index, apps/assets-derive, whose image also runs
assets-derive-sandbox-image and assets-derive-sandbox-scene); nginx,
database, cache, search and objects scripts live next to their Dockerfiles
under deploy. Compose invokes these scripts without embedding probe logic.
The derive supervisor and each executor write a heartbeat only once they are
ready: an executor after its startup probe has jailed a child (and, for
scenes, created a WebGPU device), hence their 60-second start period. To run a
probe manually, for example:
docker compose exec api /usr/local/bin/healthcheckSurrealDB and object storage have persistent volumes (database, and
objects or the WEB_OBJECTS_DIR host directory). When backing up the web
deployment, preserve the database, the objects data and the local key/password
files together: the bundles bucket holds what flow runs published, which
exists nowhere else. Changing the key invalidates browser tokens and prevents
unsealing the sessions' and tokens' device keys, so every session and token stops
working. Valkey is disposable; the search
volume (the asset catalog) is rebuilt from Scribe by assets-index when lost,
the derived bucket's preview files can be made again, and derive-scratch
and derive-sockets are in memory. docker compose down
preserves volumes; adding --volumes deletes them. Terminate HTTPS at nginx by setting WEB_TLS=1 after installing
certificates with deploy/issue-tls.sh, or at an outer edge. APP_ORIGIN must
be that exact HTTPS origin. The default Compose listener is local HTTP.
Scaling can separate web/API hosts without changing browser URLs: keep /api/
routing at the edge and point API replicas at shared SurrealDB, Valkey and key
material. Rate limits and revocation claims are shared across replicas. The
provided Compose deployment still has a single database and cache; availability,
capacity planning and load testing of those services remain deployment concerns.
README and source presentation
GET /api/v1/repositories/:repo/readme resolves the main branch head by default,
or accepts branch and revision to pin historical content. It checks common
root README names in one bounded batch of exact file-state lookups, never scans the full repository, and
renders UTF-8 text up to 64 KiB. Like other browse caches, every hit performs a
live authorization read. Repository list excerpts load lazily through a serial
request lane. Missing READMEs use the repository's ordinary description.
READMEs use react-markdown with GFM;
raw HTML is disabled. Relative links and raster images stay on the same
repository, branch and revision. External images and inline SVG are not loaded.
Code uses Lowlight tokens rendered as React
text in light/dark palettes. Only visible blocks are highlighted, through one
worker at a time with a two-second deadline and a 64 KiB input limit; unsupported
or larger files retain the plain-text preview. No generated HTML is injected.
Clicking a line number marks that line and writes #L12 into the URL; dragging
or Shift-clicking marks a range as #L12-L40. Shared links scroll to the first
marked line. Ranges are capped at 200 lines, matching review comments.
The Clone menu has macOS / Linux (POSIX), PowerShell, and Windows CMD tabs using the native CLI's
workspace create, branch selection and sync. Public browsing uses
--anonymous. The snippet sets SCRIBE_SERVER once; signed-in users also
get SCRIBE_AUTH and SCRIBE_USERNAME defaults followed by scribe login.
PowerShell uses $env:NAME assignments and a guarded script block compatible with
Windows PowerShell 5.1 and newer. CMD uses set "NAME=value", cd /d, and caret
line continuations. Every shell stops the workflow after a failed step. CMD selections
containing expansion characters that cannot be safely quoted direct users to PowerShell.
The remaining commands omit repeated server and login arguments. Values are
quoted for the selected shell, and browser credentials are never included. At the selected
branch's head, sync omits --revision to fetch the latest version when the
command runs. Historical
revisions stay pinned; an unknown branch head also keeps the explicit revision.
WEB_SCRIBE_CLONE_SERVER and WEB_SCRIBE_CLONE_AUTH_SERVER optionally configure
client-visible addresses separately from the API's Docker connection addresses.
Without overrides, host.docker.internal is mapped to 127.0.0.1 for local use.
Installations reached remotely must set those outward-facing addresses.
The command preview uses horizontal scrolling and Bash highlighting for POSIX,
preserving one logical command per line. Copying always uses the selected tab's
original command text. Tabs support arrow keys, Home, and End.
Accounts and integrations
Avatar/name references share the UserIdentity component across the feed,
repository history and graph, workspace navigation, and account headings.
Display names take precedence; keyboard and pointer tooltips retain the original
username. Known public accounts link to /u/<UUID>; your own private identity
links to settings. Unknown authors and other private accounts remain plain text.
POST /api/v1/users/resolve accepts up to 50 distinct usernames and returns only
public profile IDs and display names. Mounted rows batch these lookups and share
cached results by username. Profile edits invalidate those results.
Mappings are registered from authenticated Scribe identities when reading or editing an account, never from submit messages. SurrealDB indexes a username digest on the profile row; lookup responses omit private profile fields and do not publish a username directory. Existing profiles become discoverable after their next authenticated account visit. An author-name match is a navigation aid, not proof that a historical submit was authenticated as that account.
/settings shows the authenticated identity and profile; /settings/appearance,
/settings/api and /settings/mcp hold the remaining panels (the former
?panel= links redirect permanently). Token panels manage API/MCP tokens. The browser
receives a secret only in the creation response; it stays in component memory until
dismissed and is never saved to local storage or the query cache. Each token has a
name, explicit scopes, fixed expiry, last-use timestamp (at most one write per minute),
and an inline-confirmed revocation action. API, MCP and device tokens are bearer
secrets of the web API alone: they authenticate HTTP requests to /api/v1 and
never reach auth or a storage server. Behind each, the API holds a key-backed
Scribe automation credential, issued through the browser session that created
the token and outliving that session's logout: a device key the API generated
and sealed for the token, and the certificate auth issued for it with a role
ceiling derived from the token's scopes (see web automation credentials). API management routes are browser-only
and protected by Origin checks. The REST surface remains under /api/v1.
Response schemas are serializers, not documentation: a reply that no branch of
a route's schema describes is refused with a 500, a field the schema omits is
dropped from the reply, and a field the schema marks required but the data
does not carry is refused as well. Every route therefore declares its request
parts and its responses as zod next to its handler, and each response schema is
checked at compile time against its contract type (Describes in
@scribe/http/schemas): a contract that gains a field or an entry kind fails to
compile until the schema gains it too. A run's frozen definition is accepted
history and is read as it was stored: a run started before jobs published
bundles or flows declared retention has no bundles or retention, and its
contract type (FrozenFlowDefinition) says so. A route also declares its OpenAPI tags and,
when a token or anyone may call it, what it is as an MCP tool (config.tool: a
name, or false); a route missing either is refused when the server starts.
/developer/restrenders the OpenAPI document generated from actual Fastify routes, request schemas, response serializers, and auth configuration. Endpoints are grouped by tag behind a sticky, searchable index (/focuses the search), and each entry opens to a parameter table, field-by-field request and response schemas with a raw-JSON toggle, and a copyable curl example. Download/api/v1/openapi.jsonto generate an integration client. Requests containing a bad token never fall back to anonymous or cookie credentials./developer/mcprenders the machine catalog generated from the REST route declarations, grouped and searchable the same way, with readable input and output schemas and a copyabletools/callexample per tool. AGETtool is marked read-only unless reading can change state: reading a proposal's current detail, rounds or submits may open its next review round, soread_proposal,proposal_roundsandproposal_submitsare not./api/v1/mcpuses the official MCP SDK v2 Streamable HTTP handler. Tool arguments use{ params, query, body }with only the sections declared by that route. Responses include both JSON text andstructuredContent.data; errors preserve RESTerror.code,error.message, anderror.requestId, withisError: true. The shared response schema describes both outcomes. Discovery is scope-filtered. In-process dispatch marks the request with a private symbol; no HTTP header can bypass token audiences. Every call still validates its bearer, live identity, scope, native repository grants, request schema and rate limit. The catalog includes reads, search, inbox/feed, stars, reactions, account preferences, notifications, workspace/lock administration, mounts, cherry-picks, native merges, branch operations, batch commits, proposal discussion and comparisons. Credential issuance/revocation and separately authenticated auth administration remain outside the MCP catalog. Raw HTTP streams use the content endpoint;read_file_bytesprovides verified base64 for files up to 512 KiB.
MCP clients must support configured Bearer headers. Manually created MCP tokens
are directly usable; OAuth discovery/dynamic client registration is not advertised.
Both token types are audience-separated, and MCP never accepts browser cookies.
Origin is checked when present, Host is checked for MCP, and machine POST requests
can omit Origin only on explicitly enabled bearer routes. nginx forwards the
existing /api/ namespace with buffering disabled. Use HTTPS for remote clients.
See auth.md for lifecycle and storage details.
Workspaces and repository locks
The global /workspaces page lists the signed-in user's native workspace
registrations, grouped by repository. It reads the authoritative storage registry
with bounded cursor pagination (at most four workspace pages per API request,
reusing one directory page within that request), without mirroring workspace state
into SurrealDB. Search can continue
through sparse pages. Details show the branch, creation time, immutable ID and
whether locks are held; the registry does not claim to know local unsubmitted work
or whether a client is currently online.
Repository details expose a repository-wide Locks tab, with owner, branch,
workspace, acquisition time and path filters. It lists every branch by default;
the branch bar under the tabs narrows it to one (/locks/<branch>, shareable),
and All branches returns. Readers see other users' locks.
Force unlock is shown only when native authorization permits Maintainer access.
The request retains the exact holder and generation, so a retry cannot release a
replacement lock. Repository ID and incarnation are checked before mutations.
For signed-in readers the Files tab also marks locks where the paths are shown:
a file row and the file toolbar name the holder (hover or focus shows since
when), and a folder row counts the locked files under it. The browser reads
GET …/locks?prefix=<folder>/ (or the file's path) for up to eight pages of 128
locks; beyond that the table says only the first locks are shown and the Locks
tab lists them all. Locks are repository-wide, so the badges do not change with
the selected branch or revision. They are a hint refreshed every 30 seconds and
on focus; storage still enforces locks at submit.
Removing a workspace retires its registration, not its local files. Native storage enforces ownership, live repository access and held-lock rejection. A persisted operation ID and full intent are reused after uncertain failures; storage resolves the outcome and rejects operation-ID reuse with different input. The browser persists pending administration requests before transmission and offers retry.
REST exposes GET /api/v1/workspaces, GET /api/v1/repositories/{repo}/locks,
POST /api/v1/repositories/{repo}/workspaces/remove and
POST /api/v1/repositories/{repo}/locks/release. MCP exposes list_workspaces,
list_locks, remove_workspace and release_lock through the same service and
validation schemas. Scopes are workspaces:read, workspaces:write, locks:read
and locks:write; write scopes are opt-in. Workspace removal needs no role above
Reader when the registration belongs to the caller. Lock management tokens have a
Maintainer ceiling, still bounded by the owner's live grants; they cannot perform
auth administration or delegate further tokens. Flow scopes follow the same
pattern: flows:read, flows:write (writer ceiling) and runners:write
(reader ceiling; registers and operates runners); MCP exposes list_flows, read_flow, list_runs,
start_run, read_run, cancel_run, delete_run, read_run_log,
list_runners, read_runner, register_runner, update_runner,
set_runner_enabled and remove_runner, while the runner's own endpoints
are not agent tools.
Change descriptions
A change's message is part of the accepted commit and is never rewritten. The history view shows each change's current description instead: the submitted message, or a correction recorded later, with who edited it and when and the original message kept beside it. Writers see an Edit description control on each history row; saving records a new revision on the server and updates the row at once. Descriptions are append-only — every earlier revision stays on the server — and the native service enforces who may record one: the change's author, or a maintainer. Two people editing the same change cannot overwrite each other; the second save is refused as stale and the row must be reloaded.
REST exposes PUT /api/v1/repositories/{repo}/changes/{change}/description
({text, expectedRevision}, scope contents:write, browser or PAT session
required) returning the recorded note, and
GET /api/v1/repositories/{repo}/changes/{change}/descriptions (scope
history:read) listing every revision oldest first. History pages carry the
current description as note (null when never corrected) beside message. A
stale expectedRevision is 409 description_stale; describing someone else's
change without maintainer rights is 403 forbidden. Cached history pages may
show a description up to 30 seconds late.
Profiles and preferences
Settings use a sidebar with Profile, Appearance, API tokens and MCP tokens panels.
PATCH /api/v1/account/profile edits display name (80 characters), plain-text bio
(500), a status emoji and status text (100), and public/private visibility. The
status emoji is one of the twelve in statusEmojis (shared contracts) or empty;
the schema refuses anything else. Like reactions, the picker, the settings preview
and the public profile draw it from the bundled SVG artwork in
apps/web/public/emoji/, so a status looks the same on every operating system.
A stored value outside the set is not displayed.
PATCH /api/v1/account/preferences independently merges theme (system, light,
dark), language (en, cs), the home feed's left-out item kinds
(feedHidden, a list of changes, proposals, runs, replaced whole; absent
until first chosen) and the Interface preference (interfaceMode: auto,
development, art; absent until first chosen, which means auto). It
chooses how repositories are presented: development with history, branches,
the graph and flows, art leading with browsing and reviewing art, and auto
by the repository (one whose accepted kind on main is art in
.scribe/config.toml opens in the Art
interface; the repository header then shows an Art badge). It is chosen in
quick settings and Settings → Appearance; signed-out visitors keep it in the
browser like the theme. One repository can be shown in another interface
from the quiet line under its page ("Art interface · View as Development"),
remembered in that browser for that repository only. Switching keeps you on
the same page when the new tab bar has it; a file or folder of main moves to
its counterpart (the file view and the asset page, the file tree and the asset
browser), and anything else opens the new interface's first tab. Art shows
Assets, Activity (main's history), Reviews, Check-outs and Settings, with Files,
Graph, Flows and Clone behind Developer tools at the end of the bar, and
branch names only as a chip off main; Development shows Files, History, Graph,
Proposals, Flows, Locks and Settings. The preference is called Interface, not
view, because "view" already means a workspace's download selection. Both require a live browser identity and same
Origin; automation credentials cannot edit profiles or preferences.
SurrealDB's user_profile rows belong to immutable upstream principal IDs.
Each receives a separate stable UUID v4 public address, with a unique index.
Profiles default to private. GET /api/v1/users/:id and /u/<uuid> expose only
public display name, bio and status. The API returns 404 for both private and
unknown profiles, uses no-store responses, and never publishes login usernames,
emails, theme, locale, token metadata, or internal identity IDs. Display names
are never inferred from a username for public views.
Signed-in preference changes, including the top-bar controls, persist to the account and apply across devices after sign-in. Anonymous preferences stay local. Profile writes and preference writes update separate fields. The frontend cancels stale account reads before saving, and renders profile text without HTML parsing.
File submissions and accessibility
Repository loading keeps the page header, navigation and columns in place. File previews and their loading placeholders use the same bounded viewport, while directory lists retain natural height. README content is a separate card and loads independently of homepage excerpt requests.
The file sidebar is an expandable hierarchy. Opening a folder loads only its immediate children; large directories expose an explicit Load more entries button. The sidebar and main pane share cached directory pages, scoped to the signed-in identity, repository, branch and revision. Folder expansion survives navigation, and a direct file link opens its ancestors without fetching every sibling page. Collapsing folders releases their active queries. Compact desktop rows and shallow indentation leave room for deeper paths; long filenames remain readable through horizontal scrolling. The panel reveals the selected file without scrolling the surrounding page. Mobile rows retain larger touch targets.
Folder chevrons expand in place; names navigate to the folder or file. Arrow keys move through the hierarchy, with Left/Right collapsing or expanding folders and Home/End moving to the first or last visible entry. On mobile, Browse files opens a compact drawer; closed drawers do not load additional directories. Virtual mounts open their target repository at the pinned revision.
The file preview takes the height the page leaves it (down to a readable minimum, and with a ceiling on very tall displays) rather than a fixed slice of the screen, and it starts at the top of every file opened: the viewport is one element across files, so its scroll position is reset when the file changes. Switching a Markdown file between rendered and source keeps the place.
The lazily loaded CodeMirror editor provides syntax highlighting, line numbers, search, undo/redo, folding and line wrapping. Language grammars load on demand; changing theme or wrapping preserves the document and undo history. Tab keeps normal keyboard navigation. File creation sits in the browsing toolbar; edit and delete controls sit beside the file preview. Submit messages have their own panel.
Authenticated writers can create and edit UTF-8 files up to 512 KiB and delete files from the repository browser. Every operation requires a message and creates one new change on the selected branch. Historical content remains available. Executable mode is preserved during edits. The browser editor accepts text; the batch API also supports binary bytes.
GET /api/v1/repositories/:repo/permissionssays what the caller may do:write,mountPinandmountTopologyfor the branch, and the repository-widemaintain(see virtual repository mounts). Tokens needcontents:writeto call it.POST /api/v1/repositories/:repo/commitsacceptsoperationId(UUID),branch, pinnedrevision,action(create,edit,delete),path,message, andtextfor create/edit only. Responses containstatus(accepted,rejected,pending); accepted responses includechangeandcommit.GET /api/v1/operations/:idresolves the authenticated owner's operation.- MCP
commit_filetakesparams.repoand the request inbody;commit_statustakesparams.id. Both requirecontents:write; write scope is unchecked by default during token creation. Read-only tokens cannot invoke these tools.
Before mutation the API stores the exact native operation ID and immutable object
bytes in SurrealDB. A client retries the identical request with the same UUID;
reuse with different content is refused. Unknown outcomes are resolved with native
operation_outcome, never inferred from a timeout. Concurrent API replicas reuse
one persisted intent and storage replays its durable result. Operation records are
owned by immutable auth principal IDs. Recording a terminal accepted or rejected
outcome atomically removes the prepared payload. Compact receipts retain the
fingerprint, native operation ID, repository ID and original outcome indefinitely
for safe replay; concurrent inserts cannot resurrect compacted payloads. Pending
operations retain their exact bytes without expiry. Include receipts and pending
operations in web database backups.
Before the browser sends a commit, its shared file/mount submission helper stores
the exact request and UUID in a strict-durability IndexedDB transaction, scoped
to namespace and immutable account identity. Storage failure prevents submission.
One unresolved request per account/repository blocks replacement intents across
tabs. The browser sends X-Scribe-Owner with its immutable account ID; the API
rejects a changed account before preparing or publishing any intent. The shell restores saved submissions after reload, checks their outcomes,
and offers an exact retry. Accepted submissions remove their local journal entry;
rejected submissions retain the request until explicitly dismissed. Browser
tokens and passwords never enter this journal. Clearing site data explicitly
removes browser recovery records.
The native submit path enforces per-file base changes, current grants, namespace rules, path validity, content hashes and repository lock policy. The web editor does not acquire or steal locks. A file requiring a lock must be submitted through a CLI workspace with that lock. Conflicts retain the editor draft for review; there is no force-overwrite action. Pending requests keep their operation ID and freeze their contents so retry cannot accidentally create a second change.
The accessibility pass targets WCAG 2.2 AA practices: keyboard/focus tooltips that can be dismissed with Escape and hovered, visible focus outlines, programmatic form labels, current navigation states, arrow/Home/End navigation for homepage tabs, focusable main/tabpanel targets, announced status/errors, stronger input borders, touch targets, forced-color support and reduced motion. This is not a formal WCAG conformance audit. Visual, screen-reader and mobile verification are left to the user's UI review; code and service tests cover behavior separately.
Visual tooltips are limited to labeled icon buttons and explicit data-tooltip
help. They appear after 300 ms of hover or immediately on keyboard focus, and
dismiss within 80 ms of leaving both the control and hint. Clicking, Escape,
activation, scrolling, window blur, and removal of the control dismiss them;
touch and click-triggered focus do not open hints. Placement uses the rendered
hint dimensions to stay within the viewport. Compact styling follows the theme
and reduced-motion preference; accessible control labels remain available when
visual hints are hidden.
Virtual repository mounts
Repository browsing includes virtual mount rows in the file list from accepted .scribe/config.toml
at the selected parent revision. Each available row shows a mount icon and @ revision pin and links to the child's exact
branch ID and revision. Mount paths containing directories appear inside those
parent folders; recursive repositories mounted inside child repositories remain
unsupported. Inaccessible targets are shown without fetched names or
click-through links. The independent mount endpoint bypasses the parent tree cache
and performs target authorization with the caller's credential. See
repository mounts for the HTTP/SDK contracts and permissions.
Writers can update mount revision pins. Adding, removing, renaming, or retargeting
mounts on main requires Maintainer access; Writers can make these changes on
other branches. The toolbar and file-list rows disable restricted topology actions
with an explanation, while the editor keeps the revision pin editable. A compact modal keeps the repository and sidebar in place. It
selects a visible repository, branch and explicit revision pin, with optional
repository search and advanced naming. The same form collects a commit message;
removal uses a short confirmation instead of a separate review page. Removing a mount changes only the parent configuration, never the child
repository. Unavailable targets can still be removed with parent topology permission.
POST /api/v1/repositories/:repo/mounts/preview requires contents:write and
returns the updated .scribe/config.toml text without publishing it. The API checks
branch-specific mount permissions and independently verifies the target pin.
GET /api/v1/repositories/:repo/permissions?branch=… returns write, mountPin,
and mountTopology capabilities for the branch, and the repository-wide
maintain (Maintainer; tokens also need repositories:write), which with
write decides the Settings sections shown; omitting the branch selects main. Branch names
and IDs resolve to the same permissions. Permission refusals keep entered values
and refresh permissions without treating a confirmed rejection as retryable. Comments and
unrelated settings are retained. The UI submits this exact text to the existing
file commit endpoint using the same parent branch/revision and one operation ID;
retries, permission checks, config validation and stale-file conflicts follow the
same durable workflow as file editing. The preview does not reserve a revision or
grant access, and storage remains authoritative at commit time.
A preview the configuration refuses answers 400 with one code per reason:
mount_required (create or edit without a mount), mount_path_invalid,
mount_path_reserved (a .scribe directory or inside the root .scribe/),
mount_overlap, mount_limit (64 mounts), mount_config_limit (the file
would exceed 64 KiB), mount_config_version, mount_config_encoding (not
UTF-8) and mount_config_syntax (not TOML the editor can change). A mount that
already exists or is missing at the pinned revision is 409 mount_changed, and a
topology change on main without Maintainer access is 403 mount_topology_forbidden.
Selective integration
Mount editing and cherry-picking use the shared RepositoryDialog frame: centered
placement, viewport-bounded scrolling, a persistent action bar, background scroll
locking, Escape handling and focus restoration. Cherry-pick starts compact and
expands for file review. History distinguishes the muted source-provenance pill
from the separate outlined cherry-pick action.
Repository history provides a Cherry-pick action for writers on ordinary changes. The editor previews a pinned target, compares file contents and records explicit conflict choices through the shared saved-submission journal. See cherry-pick for limits, permissions, API and recovery details.
Branch selection uses the shared BranchPicker in repository navigation, mount
editing and cherry-picking. It supports keyboard navigation and search, orders
branches beneath their parents, and shows each origin revision and current head.
Cherry-picking keeps the same dialog dimensions during selection and review;
long reviews scroll inside the dialog. The graph marks the selected branch with
a flat chip and checkmark while retaining its lane color on the branch icon.
Proposals
Repositories have a Proposals tab for optional browser-based branch review
and merging. Proposals follow branch changes automatically and retain immutable
review rounds. Unchanged reviewed content keeps its approvals. Drafts, independent approvals, requested
changes, reviewers, assignees, labels, referenced discussion, asset comparisons,
activity feeds and durable merge recovery use the existing auth model and native
merge protocol. REST and MCP support the same workflow through proposal scopes.
CLI merges remain independent. The web review follows a clean source-side rename
(the source moved A -> B, the target left A alone and has no B) as one
Renamed row and merges it as the CLI would, with moved_from provenance;
every other rename shape still hands off to the CLI.
See Proposals for the workflow, permissions and current bounds.
Flows
Repositories have a Flows tab. It reads .scribe/flows/**/*.yml and *.yaml at the
selected branch head and lists the flows as their files are laid out:
folders become collapsible groups (nested as deep as the tree goes, so
ps5/debug/full.yml sits in debug under ps5), each group's header
tallies what is failing or running beneath it, and the flows inside are
tiles carrying the trigger, job count, description and last run. Collapsed
groups are remembered per repository in the browser; a filter box appears
past six flows and opens every fold while searching. The tab composes every
uses: on the server, draws each flow as a graph of jobs, and keeps run
history: which job and step failed, how long
each took, and the log of every step. Runs are started from the flow page
(repository Writer) and cancelled or deleted (Writer, Maintainer) from the run
page. The same tab lists the runners bound to the repository with their live
status and registers new ones: a Maintainer names the runner and picks its
labels, and lands on the runner's page (/flows/<branch>/runners/<id>). Until
the process has checked in, that page leads with setup: a one-click token (an
API token with runners:write and flows:read, reader ceiling, shown once)
or any existing token with those scopes, and the scribe-runner run and
service install commands with the address, runner id and token filled in.
Once the runner has checked in the page is its status (last seen, current job,
platform, version, labels, owner), with the setup steps folded away under "Set
up on another machine" for a reinstall. A Maintainer edits the labels in
place, disables or removes the runner there; labels also change through
PATCH /repositories/:repo/runners/:runner. The
runner checks in with that token and its own device public key, and receives a
reader-ceiling credential certificate for that key, derived from the token's
automation credential, which it checks repositories out with; a deployment
without TLS auth configured answers 409 runner_auth_unconfigured. A run page lists the bundles its jobs
published (bundles: in the flow) as a compact grid of download links with
sizes and a verification mark (the job too, when several published), or the
same list marked removed once retention took the files. A step's log follows
its tail while the step runs, until the reader scrolls up; the search box in
its header finds lines case-insensitively, marks every match, and Enter /
Shift+Enter (or the arrows) walk through them. Terminal colours and weights
in a step's output (the SGR escapes a compiler emits — the 16-colour
palette, 256 colours, true colour, bold, dim, italic, underline) are drawn
as a terminal would draw them, on a palette tuned to the log's background;
every other escape is dropped.
The Runs tab lists runs one line each, in sections per local day, and is
narrowed by four pickers above it — flow, branch, status (any, active,
succeeded, failed, cancelled) and period (any time, today, last 7 or 30
days, or a custom day range) — which live in the page's URL, so a filtered
list is a link. The filters are applied by the API (GET /repositories/:repo/runs?flow=&branch=&status=&since=&until=), so they hold
across every page of a long history. The registration guide assumes
scribe-runner is installed and links to no download: take the
scribe-runner-<platform> archive from the manual build workflow, or build it
with scripts/install.sh / install.ps1. The
scribe-runner reference covers the
executable. The runner protocol (check-in, claim, events, logs, heartbeat) is
REST under /api/v1/runners and documented, with the definition format, in
Flows.
The branch is chosen once, in a bar directly under the repository tabs, and
the Files, History, Graph, Flows and Locks tabs all read it; Proposals are
repository-wide and show no bar. Files and History pin a change beside the
branch. Leaving Proposals or an all-branches Locks view for a branch tab opens
the branch last used in that repository during the session, never silently
main.
URLs: /<namespace>/<repo>/flows/<branch> lists flows,
/flows/<branch>/<flow-id> shows one, /flows/<branch>/runs lists runs
(/flows/<branch>/runs/<flow-id> filters to that flow),
/flows/<branch>/runners lists runners, and /runs/<run-id> shows a run.
Runs of a specific execution are not per branch, so they carry no branch
segment. runs and runners are reserved as the first path segment after
the branch so they are not flow ids.
Background work
The worker Compose service uses the generic durable job queue described in
background jobs. It updates proposals without open browser
pages. Authenticated deployments require the read-only credential file of a
service account in apps/web/deploy/local/worker/reader.credential and
WEB_SCRIBE_READER_CREDENTIAL_FILE=/etc/scribe/worker/reader.credential (the
Compose name; Compose passes it to the worker process as
SCRIBE_READER_CREDENTIAL_FILE).
The worker does not mount the API JWT signing key. The account must read each
tracked repository: a namespace-wide reader grant (scribe-auth namespace-readers set) covers every repository in the namespace, including
later ones, and a reader grant covers one. PROPOSAL_CHECK_INTERVAL_SECONDS defaults to 30 and
WORKER_CONCURRENCY to 2. Plain/public-only storage can omit the credential.
The same worker runs the flows submit watcher (flow.watch, one job per
realm): each interval it compares every repository's branch heads with what it
saw last and starts runs for the flows whose submit trigger matches a branch
that advanced. It uses the same reader credential, so repositories the worker
account cannot read are not followed. See Flows.
Workspace navigation and inboxes
The persistent navbar links to Home (/), Repositories (/repositories) and
Reviews (/reviews). Search opens with its button or Cmd/Ctrl+K. On mobile, the
navigation links occupy a second row and search becomes an icon. The account
menu groups profile, settings, appearance, developer documentation and sign-out;
developer documentation also remains in the footer.
Home is an activity dashboard with quick repository links and starred repositories.
Repositories is a separate full-width catalogue with search, visibility filters,
README excerpts loaded on demand, and paginated discovery. Search and visibility
are URL filters (q and visibility); the catalogue does not load the activity
feed. Visibility filters apply to loaded discovery pages, with further pages
available through Load more.
Reviews has three filters: awaiting your review, your proposals with outstanding
change requests, and open/draft proposals assigned to you. Approval validity uses
the same content-invalidation boundary as proposal merging. Inbox discovery lives
behind DataStore.navigation; each result rechecks current native repository
access and repository incarnation. No prepared merge objects or proposal bodies
are returned by discovery. Pages return up to 20 entries and examine at most
three candidate pages, so sparse permission-filtered pages can have a continuation
cursor. Navbar counts describe the first page, with + when more may exist.
GET /api/v1/reviews?filter=review|changes|assigned accepts browser sessions or a
PAT with proposals:read. GET /api/v1/search?q=...&kind=... searches repository
names, proposal titles, public display names or branches in a selected repo.
Search is metadata-only, not file-content indexing. People results include only
public UUIDs and display names; private profiles and usernames are not searchable.
Repository results offer a branch-search shortcut. Inside a repository, numeric
queries also offer a change-number jump. Search uses native repository pagination
and bounded database pages, with signed, expiring cursors bound to the caller and
filters. Generated REST documentation includes request and response schemas.
/notifications is each person's own durable history, one SurrealDB row per
recipient. A proposal writes its notifications in the same transaction as the
change that caused them:
- a reviewer or assignee being added reaches only the person added (review requested, assigned);
- comments, edits, resolved and reopened threads, approvals and requested changes, publication, merges, rejected merges, closes, reopens and archiving reach the proposal's audience (author, reviewers, assignees, participants and anyone mentioned), each marked with why it is theirs — mentioned first;
- content that needs a fresh review (a refresh, or automatic tracking that changed the reviewed files) reaches the reviewers.
A flow run that fails reaches whoever started it by hand or whose change started
it; a success reaches only someone who started it by hand. The run notification
is added when the run ends, keyed by the run, so later changes to an ended run
add nothing. Own actions and routine automatic tracking are excluded. Browser-only GET /api/v1/notifications supports
unread=true; POST /api/v1/notifications/read records individual event IDs or a
server-issued checkpoint. Read receipts and monotonic checkpoints persist in
SurrealDB under the realm and immutable authenticated identity. Mark-all-read
covers the supplied snapshot, so later activity remains unread. Unread filtering
happens before pagination; already-read history does not hide older unread items.
Access is rechecked even for previously delivered events. No native history is
changed.
Notifications are live: the navbar follows the notifications topic on the tab's
event stream. Every process that writes a proposal or
ends a run announces it on the recipient's channel, and marking notifications
read announces that too, so the bell and an open /notifications page refresh
at once in every tab. The topic carries no content — the page rereads over REST,
where access is rechecked. The bell polls once a minute only while the stream
is down; the review signal still refreshes once per minute while the tab is
active, and sooner when a notification arrives.
Maintenance and administration
/maintenance (linked from the footer) shows a public logical topology and disk
statistics for repositories visible to the viewer, and links to the
administration section. Topology exposes generic service labels and connections,
never endpoints, instance names, paths, capacity, errors, or private repository
inventory. The API obtains topology through native discover (opcode 37), not
from a snapshot file, server version, or web authentication settings. The diagram
shows configured storage servers, including offline ones; failed discovery is an
error, never a guessed single-storage deployment.
Repository disk usage is read over the native protocol: for each repository on
the inventory page the API calls repo_usage (opcode 38) under the viewer's own
session, so a size is shown exactly when the viewer can read that repository,
and a gateway forwards the read to the owning storage server. The number is the
apparent size of the repository's storage directory — history, objects and
metadata — measured when the page loads, at most six repositories at a time.
A repository whose storage server does not answer, or that the viewer can no
longer read, shows an unknown size, never zero; any other failure fails the
page. No snapshot file, exporter container, or mount of the gateway's data
directory is involved, and the API never opens a native database.
/admin (also linked from the footer) holds every task that needs an Auth
administrator: Permissions (/admin/permissions) and Orphaned data
(/admin/leftovers) so far. The section unlocks once, in one of two ways:
- Sign in as administrator with an Auth administrator's username and
password.
POST /api/v1/maintenance/auth/sign-in(body{username, password}, validated like the browser login and rate limited like it, five attempts a minute) generates a P-256 key in memory, sends Auth'sadmin-loginwith it, and answers{credential, username, expiresAt}: a credential file for a short-lived administrator credential (8 hours unless Auth is configured otherwise, 1 to 24) labelledadmin-sign-in (web)in Auth's credential list. A lost answer is asked once more with the same operation, which Auth resolves from its recorded outcome rather than registering a second credential. The API keeps nothing: not the password, the key nor the credential; the reply isCache-Control: no-store. A wrong username or password is401 admin_sign_in_failed, a person who is no Auth administrator403 admin_required, too many attempts at Auth503 auth_unavailable. Signing in does not touch the browser's web session. - Load a credential file: an Auth administrator's credential file, the one
scribe login --admin --out FILE(hours) orscribe-auth credentials create(orinit,admin-credential issue,migrateorrestore) wrote, chosen with a file picker; at most 32 KiB, which leaves room for ML-DSA certificates. Use it for automation and work longer than a sign-in lasts.
Either way, unlocking lists Auth's users, so Auth must accept the credential as
an administrator's before any tool opens. The browser holds the credential,
private key included, only in the admin layout's memory, shared by every tool
until the administrator locks the section or leaves it (a signed-in one shows
when it expires), and sends it in the body of each administration request; the
API reads it strictly, uses its key and credential certificate for that one
request, and neither stores nor logs it. It is neither a web session nor an
API/MCP token. Requests use same-origin POST
bodies and mutual TLS to the configured Auth service, verified with the
deployment's own auth CA; the endpoint and CA the file names are ignored. A
file that is not a credential bundle is refused with 400 admin_credential_invalid. No user-supplied endpoint, arbitrary Auth command, or
secret logging is supported. Both request-rate limits and an eight-call process
admission bound apply.
Users can read paginated Auth users and direct grants, grant reader/writer/maintainer
roles, revoke direct grants, and change repository visibility. Policies use native
namespace/repository-UUID keys; names in the UI are only labels. Administrators
may enter a UUID for a repository outside their browser account's inventory.
Namespace-admin and public read permissions still apply when a direct grant is
removed. Storage authorization caches refresh within 30 seconds of an Auth change.
Before transmission, the browser persists the nonsecret complete change intent,
namespace, immutable repository ID and operation ID. Retries query native
operation-outcome and replay the same complete intent so Auth verifies its actor
and fingerprint. Visibility additionally checks the expected policy revision.
Uncertain outcomes remain pending across reloads and require unlocking again;
authentication failure does not erase a pending operation. Native Auth retains its
existing durable operation outcomes and audit trail. Public and administration
endpoints appear in generated REST documentation; administrator credentials are
deliberately not accepted through the general MCP automation interface.
On /admin/permissions, select a repository in the shared searchable
picker, select a user in the left panel, and review their direct access on the right.
Visibility belongs to the repository toolbar. The picker uses paginated authorized
repository discovery, supports keyboard navigation and is also used by the mount
editor. The permissions page additionally accepts a repository UUID; this does not
add repositories to the browser's visible inventory. Partial grant pages are treated
as unknown until loaded, not as proof that a user has no direct access.
Orphaned data
The web tier keeps its own data about repositories: the asset search index,
previews, tags, status and fields, proposals and comments, notifications,
reactions, stars, flow runs and runner bindings, web commit records and
background jobs. Scribe never tells it when a repository is deleted, and a
restored repository is a new incarnation whose earlier web data is no longer
shown. /admin/leftovers finds that data and removes it. It uses the
administration section's credential; every request presents it and Auth must
accept it before anything is read or started. A browser session, API token or MCP
token is never enough.
Scan wakes the maintenance.leftovers job in the worker, because only the
worker's service account can list every repository. The job counts every store
by repository and incarnation scope, then walks the complete repository
directory. A gateway fails that listing when a storage server is out of reach,
so a scan never completes with a node missing. The job keeps the result until
the next scan. Each repository is reported with one reason:
- Deleted: the repository is not in the directory. Every stored row for it is left over.
- Restored: the repository is listed, but some rows belong to an earlier incarnation. Tags, status and fields are kept by repository ID and survive a restore, so they are never left over for this reason.
Rows keyed only by a scope that no row naming this deployment's realm uses (reactions, stars, catalog files, commit records) may belong to another deployment sharing the database. They are counted and never removed.
A repository missing from the directory is proof of deletion only when the
service account reads every namespace the deployment serves. The page asks
Auth, with the administrator's credential, whether the account the scan ran as
holds a namespace reader grant on each namespace. Until it does, the page says
which namespaces are missing and refuses to clean up deleted repositories
(409 leftovers_incomplete). Cleaning up an earlier incarnation does not
depend on this.
Clean up asks for the first eight characters of the repository ID and
wakes that repository's maintenance.purge job, recording which administrator
asked. Before it removes anything, the job walks the directory again and counts
the repository again. It refuses when the repository is listed again, or when a
restored repository has nothing left from before. It then empties the stores
in a fixed order: jobs first, so nothing reads or writes for the repository
while the rest goes, and tags before the search index, which the tags mirror
writes. Each store goes in bounded batches of short transactions. The job
counts again at the end and repeats if anything came back.
Tags, proposals, comments, flow history, notifications, reactions, stars and commit records cannot be recreated. The confirmation names them. Previews and flow bundle files carry no repository ID. The preview sweep and the bundle retention sweep remove them within a day, once no index row or run refers to them.
Inline confirmations
Use InlineConfirm from @/components/inline-confirm for actions that need a
simple second click, such as signing out, revoking a token, or removing a
workspace. The label and leading icon stay mounted; only the confirmation
controls appear on the right. Use the existing RepositoryDialog for forms,
conflict resolution, or changes that need a full review before submission.
<InlineConfirm
label="Remove workspace"
icon={<Trash width={16} aria-hidden="true" />}
confirmLabel="Permanently remove workspace"
description="Your local files are kept."
danger
disabled={workspace.hasLocks}
onConfirm={() => removeWorkspace(workspace.id)}
/>The component owns opening, cancellation, focus restoration, reduced-motion
handling, and pending state. variant="row" makes the trigger fill a menu row;
button styling is the default. busy accepts an external operation's state.
Return false from onConfirm to keep confirmation open when the caller already
handles an unsuccessful result; thrown errors remain visible inside the component.
Returning true or nothing closes it. Keep authorization, operation IDs, durable
retry handling, and API calls in the caller. Key the component by the target's
identity when an existing detail panel can switch targets.
Machine workflow coverage
The public OpenAPI document and MCP catalog are generated from the same declarations. Additional routes are:
REST route (under /api/v1) | MCP tool | Required scope |
|---|---|---|
PUT /repositories/:repo | ensure_repository | repositories:create |
POST /repositories/:repo/branches | mutate_branch | repositories:write |
GET /repositories/:repo/branch-operations/:operation | branch_operation_status | repositories:write |
POST /repositories/:repo/commits/batch | commit_files | contents:write |
POST /repositories/:repo/merges/preview | preview_merge | contents:read |
POST /repositories/:repo/merges | merge_branches | contents:write |
POST /repositories/:repo/compare | compare_files | contents:read |
GET /repositories/:repo/changes/:change/files | change_files | history:read |
GET /repositories/:repo/file-bytes | read_file_bytes | contents:read |
GET /repositories/:repo/proposals/:id/diff | proposal_diff | proposals:read |
GET /repositories/:repo/proposals/:id/events | proposal_events | proposals:read |
Provisioning ensures the named repository exists; retrying never changes an existing repository's compression or visibility. This separately selected scope delegates a namespace-administration ceiling, and creation still requires a current native namespace-admin grant. It does not grant auth-service administration or token delegation. Branch create/rename requests carry repository ID, incarnation, caller-persisted operation ID and branch ID. Native outcomes make retries safe, including nested branches.
Batch commits accept 1–128 create/edit/delete entries, at most 512 KiB decoded per
file and 8 MiB total. Each create/edit supplies exactly one UTF-8 text or canonical
base64 field. One accepted change contains all entries; a conflict rejects the
whole batch. The caller persists its operation ID and exact input before sending.
Larger asset uploads still use the native client. Direct native merges do not
require proposals or web review; their pinned heads and durable outcomes use the
same machinery as proposal merges.
Account/profile/preferences and avatars use account:read / account:write;
notifications use notifications:read / notifications:write; personal favorites
use stars:read / stars:write; reactions use reactions:write. Existing tokens
do not gain these scopes.
Token lifecycle endpoints remain browser-only. No CI check reporting, webhooks,
subscriptions or external event delivery are introduced here.
Proposal events are indexed separately from compact review state. Detail returns
the newest 50 events, eventCount and eventsNextCursor; the events route pages
backwards with cursor or limits to a root thread UUID. Each page is in chronological
order. Actions comment (optional replyTo), edit_comment, delete_comment,
resolve_thread and reopen_thread use the existing durable action endpoint.
Edits are author-only; deletion and resolution also allow the proposal owner with
write permission. Locked/archived conversations reject discussion changes.
Deleted comments retain a tombstone and audit event. Replies keep their original
review round. Approvals and retry receipts survive timeline pagination.