On-disk format specification

This specification describes canonical object, repository, and workspace formats. Scribe is unreleased: these layouts may change directly between development revisions, with specifications and byte fixtures updated together. Released formats will require a new version number for changes. All multi-byte integers are little-endian unless a field is explicitly marked ordered (big-endian, used only inside lexicographically compared index keys). [n] means n raw bytes. str16 is a u16 byte length followed by UTF-8; str32/bytes32 use a u32 length. Digests are BLAKE3-256, 32 bytes, never truncated.

Reference implementation: crates/scribe-format (encodings, keys, descriptor), crates/scribe-objects (pack and loose layout), crates/scribe-metadata (tables), crates/scribe-workspace (workspace records).

1. Identifiers and hashing

Hash algorithm identifier 1 = BLAKE3-256 appears in every envelope. It is the only identifier in format v1; a reader rejects any other value. A future algorithm must keep 32-byte digests and applies to a whole repository, so no fixed-width layout here changes. Hex renderings are lowercase.

2. Format descriptor file SCRIBE_FORMAT

Present at the root of a data root, a repository directory, an object store, and a workspace .scribe/_state/. UTF-8, LF or CRLF, one key = value per line, keys [a-z0-9-]+, values without newlines. The first line is exactly scribe-descriptor = 1; a kind line is mandatory. Unknown keys are kept. A required-features value is a comma-separated list; an opener that does not recognize every listed feature refuses to open for mutation.

Openers compare each version they know against the descriptor before opening any database; a mismatch is a refusal that names scribe-server migrate.

3. Object envelope v1

Every stored object, loose or packed, is an envelope:

Identity is the digest of the original canonical payload, after decompression. Codec 0 stores those bytes directly. Codec 1 stores u64 original length followed by independent blocks: u8 codec | u32 original length | u32 stored length | stored bytes, all integers little endian. Each block contains 1 to 262144 original bytes. Block codec 0 is raw (equal lengths); codec 1 is a standard LZ4 block with strictly fewer stored bytes. Unknown codecs, invalid lengths, truncation, and trailing bytes are rejected. There is no terminator; the original object length determines the end.

Repository creation fixes the storage policy for its object store. It cannot be changed through the API or CLI. Under lz4, an object retains codec 0 if the complete encoded representation, including the original-length prefix and block headers, is not smaller. Locations and compression are never part of object identity. Manifests continue to describe original chunk bytes.

4. Payloads

4.1 Manifest v1

u16 version = 1
u32 flags = 0                (unknown bits reject)
u8  content codec = 0        (logical chunk contents are original bytes)
u16 chunking profile         (1 = FastCDC-2020, level 1, seed 0, 1/4/16 MiB)
u64 total size
[32] file content id
u32 chunk count
repeat chunk count: [32] chunk id, u32 chunk length

Rules: sum(length) == total size; chunk count == 0 ⇔ total size == 0; 1 ≤ length ≤ 16 MiB; empty file content id is blake3("") (af1349b9…f3262); repeated chunk ids allowed; total size ≤ 1 PiB; chunk count ≤ 2^30. Executable bits are not in the manifest. Files no larger than the profile minimum (1 MiB) are exactly one chunk.

4.2 Change set v1

u16 version = 1
u32 entry count            (≤ 4,000,000)
repeat, strictly ascending by path bytes, no duplicates:
  str16 canonical path
  u8  action: 1 add, 2 edit, 3 delete, 4 keep
  u64 expected base change   (0 for add and for a keep of a path absent on the target;
                              the change that produced the base state otherwise)
  if action is 1 or 2: [32] manifest id, [32] file content id, u64 size, u8 mode (0 regular, 1 executable)
  u8 move present (0 or 1)
  if move present: str16 source canonical path, u64 source change

A keep records a merge or cherry-pick decision to retain the target's current state, or its current absence, for a path the source changed. It carries no file state and changes no tree; it exists so the server can verify that the submission covers every source change (section 4.2.1). A keep is legal only in a submission that also carries a merge parent or a cherry-pick source; otherwise the acceptance transaction rejects it with malformed. A keep is never a move endpoint: moved_from requires an add, and the paired predecessor must be a delete.

An explicit move is an add paired with exactly one deletion of its source in the same change set. The source revision is nonzero and equals the deletion's expected base; the server validates that base at publication. Each source may feed only one destination. Contents may change during a move. Content hashes never infer a move relationship.

4.2.1 Acceptance of keep entries

The acceptance transaction validates a keep and then writes nothing for it: no changed snapshot path, no history, change_paths, file_lineage, accepted_manifests, or moves entry, and it is excluded from the commit record's entry count. Validation is only the stated base: if the path exists at head its change must equal expected base (otherwise base_mismatch), and if it is absent the expected base must be zero (otherwise not_present). A keep takes no lock, satisfies no required-lock rule, and takes part in no path-collision check, because it can displace nothing. A merge whose entries are all keeps is still a valid submission; only an entry list that is empty and carries no merge parent or cherry-pick source is rejected as empty_submission.

4.3 Commit envelope v5

Every accepted change is stored as one v5 envelope. It wraps the v1 commit payload (4.3.1) and binds the change to its ancestry by identity.

u16 version = 5
u32 payload length, then a complete v1 commit payload
u8  authenticated actor present (0 unauthenticated, 1 authenticated)
  if 1: [16] immutable principal id, asserted by the server
u8  parent commit present    (1 exactly when the payload's parent change is nonzero)
  if 1: [32] parent commit id
u8  origin                   (0 none, 1 merge parent, 2 cherry-pick source)
  if 1: [16] source branch id, u64 source change, [32] source commit id
  if 2: [16] source branch id, u64 source change
[32] portable content root
[32] portable revision root

CommitId is blake3("scribe:commit:v5\0" ‖ envelope bytes). The parent commit id is the CommitId accepted at the payload's parent change; a merge parent's commit id is the CommitId accepted at the merged head. Because each identity covers the envelope that carries the previous link, a branch head commits to every envelope below it: replacing any accepted envelope changes its identity and breaks the link its child recorded. A cherry-pick source is provenance only, never traversed as a parent, so it is recorded by branch and change like the request that selected it.

Acceptance validates both links against the commits table inside the publication transaction: a link that does not name the identity accepted at that change is a publication-gate violation and writes nothing, not even a rejection. doctor walks 1..=head in order; every envelope's bytes verify against the identity in its index row, and every link is compared with the identity recorded for the change it names, so a rewritten envelope whose index row was repointed at it is reported by its child's link. The commits index row carries no link: it is derived from the envelope, and the parent's identity is the parent row's own commit id.

An origin is legal only on a different branch at an earlier nonzero change, and a merge parent and a cherry-pick source never appear together. A merge parent's change is also written to merge_parents_v2 (section 12); a cherry-pick source is also written to the index row (Cherry-pick provenance).

Versions 1 (bare payload), 2 (merge) and 3 (cherry-pick) were accepted before envelopes were chained. They remain readable and keep the identities they were accepted with, hashed under their own domain prefixes; they are never written again, because re-encoding them would change identities that other records name. doctor counts them and reports a warning naming how many commits carry no links.

Version 4 has its own identity prefix, scribe:commit:v4\0, so no two versions share an identity, but no reader decodes it: a stored version-4 commit is refused as an unsupported version, like any other unknown word.

4.3.1 Commit payload v1

u16 version = 1
[16] repository id
[16] branch id
u64 change number            (≥ 1)
u64 parent change            (0 = none; must be < change)
u8  branched-from present    (0 or 1)
  if 1: [16] source branch id, u64 source change
[32] change set id
[16] operation id
[16] workspace id
u64 timestamp, Unix milliseconds UTC
str16 author                 (≤ 256 bytes)
str32 message                (≤ 65,536 bytes)

5. Canonical paths

UTF-8, / separator, relative, no empty/./.. components, ≤ 8192 bytes, components ≤ 255 bytes, no control characters, none of < > : " | ? *, no reserved device name stem (CON PRN AUX NUL COM1-9 LPT1-9 COM¹ COM² COM³ LPT¹ LPT² LPT³, any case, with or without extension), no component ending in . or space, and no component equal to .scribe (under the collision fold) immediately followed by a component equal to _state (same fold). .scribe/ is Scribe's namespace inside the repository; x/.scribe/flows/build.yml is valid, while .scribe/_state/x is rejected. A change set entry carrying file state whose last component folds to .scribe is refused at validation: the name is always a directory in a working copy. Collision profile 1 = NFD, Unicode 17.0.0 default full case folding (CaseFolding.txt statuses C + F), then NFC. Dotless i (U+0131) additionally maps to ASCII i for NTFS upcase equivalence. This maps ß, ẞ, and ss to one key, and also merges ſ/s, ς/σ, and ı/i. The case table is committed in scribe-core/src/path/case_folding.rs; normalization uses the pinned Unicode 17.0.0 data in unicode-normalization 0.1.25. Rust's built-in case mappings do not define index bytes. Two live paths with equal keys cannot coexist on one branch.

Repository meta["path-fold-profile"] is the little-endian u32 value 1 (01 00 00 00), written at creation and included in logical export/import. Opening for serving or inspection requires that exact profile. Explicit migration also accepts an absent profile and rebuilds the collision indexes. Missing, malformed, or unknown profiles are refused before using collision indexes. Opening never rebuilds or converts indexes implicitly. The schema-0 conversion declares profile 1 when it constructs new indexes with this fold. The profile governs only the repository profile row and folded index keys; object identities and wire layouts do not depend on it.

6. Object store layout

<store>/SCRIBE_FORMAT
<store>/locator.redb
<store>/loose/<hh>/<kk><digest hex>      hh = first digest byte in hex, kk = kind in 2-digit hex
<store>/packs/<pack id, 8 hex digits>.pack
<store>/staging/                        quarantine temporaries (same filesystem)

Objects whose envelope length ≤ small-object-threshold (default 16,777,236) are packed; larger ones are loose files containing exactly one envelope.

6.1 Pack file

header (16 bytes): "SCPK" | u16 version = 1 | u32 pack id | u16 reserved = 0 | u32 flags = 0
entry (repeated): "SCPE" | [33] object key | u64 envelope length | envelope bytes

Entry length = 45 + envelope length. Packs are append-only. Every pack except the highest-numbered is sealed. The open pack rolls over when an append would exceed pack-seal-bytes (default 256 MiB). After a crash the open pack may have bytes past the last committed length; open truncates them (they were never referenced). A pack whose id is higher than the checkpoint pack was created by an interrupted publication before its locator commit; open removes it, since no locator row can reference it (doctor and verify report it as uncommitted without removing it). A pack shorter than its committed length is corruption and refuses to open.

6.2 Locator database locator.redb

Tables: locator key [u8; 33] object key → value:

u8 version = 1
u8 location: 1 loose, 2 packed
  loose:  u64 envelope length
  packed: u32 pack id, u64 entry offset, u64 entry length

meta (&str → bytes): open-pack (u32 LE pack id), open-pack-committed-len (u64 LE). The locator is an index: scribe-server rebuild-locator recreates it offline from packs and loose files, re-hashing every entry and indexing only bytes that match the identity they claim. The rebuilt database is written beside the old one and renamed into place once durable.

6.3 Publication order of a batch

1. staged file data flushed at staging time (loose candidates);
2. loose renames into shard directories / pack appends;
3. pack file flushed once per touched pack;
4. each touched shard directory flushed once; loose/ flushed if a shard was created;
5. locator transaction commits (Durability::Immediate).

Object bytes and identities never change. Explicit offline collection may reclaim unreferenced objects and relocate retained identities into a new store; see the atomic publication and upload lifecycle profile below.

7. Data root layout

<data>/SCRIBE_FORMAT
<data>/LOCK                    exclusive OS lock (flock / LockFileEx) held by any owning process
<data>/catalog.redb
<data>/auth-state.redb          workspace and operation ownership, newest auth revision seen, schema 1 (only after auth-init)
<data>/repos/<repository id hex>/SCRIBE_FORMAT
<data>/repos/<repository id hex>/metadata.redb
<data>/repos/<repository id hex>/objects/            object store (section 6)
<data>/repos/<repository id hex>/metadata.redb.gen1  previous generation after a migration (optional)
<data>/repos/<repository id hex>/MIGRATION, MIGRATED migration checkpoint / record (optional)

Logs and runtime files (scribe-server.pid, listen, shutdown-request) live in separately configured directories and are disposable.

7.1 Catalog catalog.redb (schema 1)

Creation writes creating, builds the directory, then ready; startup removes creating leftovers with their directories.

7.2 Auth state auth-state.redb (schema 1)

Created only by auth-init, which first adds auth-ownership-v1 to the data root's required-features. One table, auth_state_v1 (&[u8] key -> &[u8] value):

Claims are written in their own Immediate transaction before the history mutation they authorize and are never rewritten; a second claim with a different value is refused (operation_intent_mismatch when only the fingerprint differs, otherwise forbidden).

auth-revision is written in its own Immediate transaction before the server acts on an auth answer that carries a newer revision or another incarnation. An answer from a lower revision of the recorded incarnation is refused with auth_unavailable; an answer from another incarnation (after scribe-auth restore) replaces the record. A value that is not exactly 24 bytes is Corrupt. See rollback protection.

7.3 Auth data directory (schema 4)

scribe-auth owns a separate data directory, never inside a storage data root:

<auth>/SCRIBE_AUTH              format marker
<auth>/LOCK                     exclusive OS lock held by serve, migrate, and offline commands
<auth>/auth.redb                every auth record

SCRIBE_AUTH is exactly scribe-auth\nschema=4\nprotocol=2\npassword=argon2id-pepper-v1\n. init and restore write it last. A directory at schema 3 is refused at startup; scribe-auth migrate converts it and replaces the marker through a flushed SCRIBE_AUTH.next.

Two files belong to the directory but live outside it and outside its backups: the password pepper and the revision witness. The witness is the UTF-8 text <incarnation as 32 lowercase hex> <revision in decimal>\n. init creates it (refusing an existing file); migrate, restore, and every revision change rewrite it through a flushed <witness>.next renamed over it. Startup refuses a witness of another incarnation and a database whose revision is behind the witness; a database ahead of it (an interrupted witness write) rewrites the witness.

auth.redb holds one table, auth_v1 (&str key → &[u8] value). Every value is one row of strings in the SAU2 result encoding (u16 0 ‖ u16 rows=1 ‖ u16 n ‖ n × str16; wire format), except operations/:

Ids (user, server, credential, operation) are 32 lowercase hex digits of a random 128-bit value. A key id is the 64-hex BLAKE3 digest of the public key's SubjectPublicKeyInfo DER. In credentials/, the kind is one of user, session, automation, automation-write, automation-maintain, automation-admin, service-reader, service-writer, and server; the subject is the username, or the server name for kind server; the algorithm is always p256; parent is set only on a derived automation credential and generation always on session and automation*, on user exactly when administrator sign-in issued it, and never on the others. All ten fields are always present, empty when unset, and the decoder refuses any other shape. The bootstrap administrator's expiry is 18446744073709551615 (it never expires). Audit arguments are the request arguments after the verb, joined by spaces and truncated to 1,024 bytes; a public key is shown as key:<key id>, and a password never appears.

Every row is written with Durability::Immediate. The expiry sweep deletes a credential with its key, index, and certificate rows one day after it expires; audit rows and operation outcomes are never pruned. auth.redb holds the CA private keys, so its backups need the same protection as the pepper.

8. Repository metadata metadata.redb

Serving requires repository schema 3 and snapshot profile 3. It uses the tables below, the branch tables, and the shared snapshots in section 12. Stores from pre-snapshot development builds are incompatible; missing profile markers or tables are refused without mutation, including by explicit migration. The meta.schema marker is authoritative for the database: opening metadata.redb for serving or inspection refuses any other value before a single write, independently of what the repository descriptor says, and both must read 3 for a repository to be served. Only the explicit migrate command opens a database at schema 1 or 2, and acceptance on such a handle fails with UNSUPPORTED_VERSION. Versioned records start with a u8 record version (1); index-only values use the exact bytes shown below. Keys:

Ordering guarantees: paths never contain 0x00, so path ‖ 0x00 ‖ change keeps one path's revisions contiguous and before path ‖ '/' ‖ …. Branch prefixes make per-branch scans key-prefix scans. commits, history, change_paths are re-derivable from the immutable commit and change set objects (slow path). file_lineage is also re-derivable from accepted predecessor/merge metadata and is included in physical and logical backups. Everything else is authoritative and must be backed up.

file_lineage identifies a revision by repository-wide change number and exact historical path; the accepting branch comes from commits. Edit/delete entries link to their accepted base, move destinations link to their recorded source, and per-file merge entries link to the pinned source revision when present (including a source tombstone for a target deletion). Ordinary adds have no base link. Fork snapshots retain original revision numbers and add no edges. Edges have empty values and both directions publish in the same durable acceptance transaction as history. Rejections publish no edges. A prefix ending after the first path terminator selects exactly one revision's neighbors.

doctor checks edge completeness and rejects extra or nonempty edges against accepted history without mutation. Database opening requires the table; missing metadata is never treated as an empty graph or rebuilt during startup. There is no legacy fallback.

Operation fingerprint = blake3("scribe:operation-fingerprint:v3\0" ‖ repo ‖ incarnation ‖ branch ‖ workspace ‖ change set id ‖ u64 len ‖ message ‖ u8 keep locks). The author is attribution, not intent, and is not part of it: an authenticated server replaces the submitted author with the principal's display name, so a fingerprint covering it would refuse a legitimate retry of a submit whose reply was lost once that name changed.

operations rows are never pruned, by design. A retry that arrives long after its submission must resolve to the recorded outcome instead of applying a second time, so the table grows by one row per accepted or rejected operation for the life of the repository: a 16-byte key and a 53-byte value plus the rejection detail (empty for accepted operations), under about 100 bytes per operation with redb overhead. A repository accepting a thousand changes a day therefore adds roughly 35 MB of operation rows per year; no expiry or compaction removes them.

9. Workspace .scribe/_state/ (schema 1; schema 2 adds workflows_v2, section 12)

.scribe/ at the workspace root is Scribe's namespace inside the repository. Only .scribe/_state/ is local, machine-specific workspace state, reserved by Scribe itself: it is never versioned, add refuses it, and sync or checkout never deletes or overwrites it. Everything else under .scribe/ is ordinary repository content and may be changed by sync like any other path. A workspace root is recognised by .scribe/_state/SCRIBE_FORMAT; a nested child workspace is recognised by a .scribe/_state directory.

.scribe/_state/SCRIBE_FORMAT       kind = workspace
.scribe/_state/state.redb
.scribe/_state/ignore              optional workspace-local ignore rules (not versioned)
.scribe/_state/cache/              object store (section 6) holding objects ≤ 256 KiB and manifests
.scribe/_state/tmp/                reconstruction temporaries (`<hex>-<index>-<pid>.part`) and resolution staging directories, cleaned at open
.scribe/_state/recovery/           displaced local contents: `<unix ms>-<32 hex random>` with an index `<name>.origin` beside it (section 9a); intent archives
.scribe/_state/staging/            work in transit between the two renames of a move: `move-<operation id>`
.scribe/_state/LOCK                local operation lock

9a. Recovery index entries

Every entry sync, revert, delete or resolution displaces from the working copy gets a text file beside it, written before the rename so that a crash leaves an index without content rather than content without an index:

scribe-recovery 1
path=<canonical workspace path>
reason=sync-update | sync-remove | delete | revert | resolve | project
operation=<32 hex, the random half of the entry name>
retained_ms=<unix milliseconds, the timestamp half of the entry name>

Every key appears exactly once and nothing else is accepted; a malformed index is reported on its entry by scribe recovery list and that entry is never pruned. Entries without an index (submit-<operation>.pending, workflow archives) are intent archives, listed and never pruned. Every displaced file is retained, even when its bytes equal the recorded baseline, because an editor may still save through an open handle. Removal requires explicit recovery pruning.

state.redb tables:

The workspace meta table additionally records optional view-profile as UTF-8 (missing or empty means an explicit selection) and optional view-change with this canonical little-endian payload:

u8 1
u64 pinned target change
[16] repository incarnation
str16 profile name (empty means none; at most 64 bytes)
u16 inclusion count
str16 inclusion prefix × count

Prefixes are validated, sorted, deduplicated, and stripped of redundant child inclusions; an empty list selects the entire repository. No empty prefix is encoded in a normalized selection. Trailing payload bytes are rejected. The golden/truncation fixture is in scribe-workspace/src/state/view_state.rs.

view-change is committed with immediate durability before planning or file application. Existing view, view-profile, and synced remain active while the journal is applied. Once no paths are failed or deferred and the server acknowledges the replacement workspace registration, one immediate transaction publishes the new view/profile/watermark/incarnation and clears view-change and the journal. Registration reuses the existing workspace ID and protocol operation; no repository object or wire encoding changes. Missing optional keys represent no selected profile or transition, not another format version. The versioned shared TOML format is specified in repository configuration.

The merge-presence flag is exactly 0 or 1. The client durably records the pending candidate after successful immutable uploads and before the first request carrying its operation ID. A candidate retains its exact branch, merge ancestry, author, message, keep-locks choice, and operation ID until a persisted acceptance or rejection is resolved. The repository incarnation is part of that intent. Missing outcomes in the same incarnation cause replay with the same ID. After a restore changes the incarnation, an absent outcome instead causes the exact pending payload to be saved as .scribe/_state/recovery/submit-<operation>.pending. The archive file and its directory are flushed before the pending row is removed; local opened work and baselines remain intact. Existing archives must match the exact bytes. The pending record uses this exact encoding; older development layouts are not decoded.

Each baseline retains the incarnation of the server state from which it was created. A full sync after restore leaves opened baselines in their original incarnation while recording deferred targets in the new incarnation. Submit refuses those old bases until explicit resolution or revert replaces them. Nested baseline records in journals, deferred entries, and resolution records use the same encoding above.

A fingerprint is one observation of a working file: its size, its modification time, and recorded ms, the Unix millisecond at which that observation was taken (0 = unknown, never trusted). Equality of size and mtime proves unchanged bytes only when the observation is settled, that is when recorded ms is at least the filesystem's timestamp granularity ahead of the recorded mtime; otherwise the client hashes the file. The granularity is read from the mtime itself: a sub-millisecond value needs no separation, because a later write cannot reproduce it; a value with sub-second digits only needs 1 s, and a whole-second value 2 s (FAT rounds to two), each plus 1 ms for truncation to whole milliseconds.

10. Backup and export containers

BACKUP_MANIFEST: text, first line scribe-backup-manifest = 1, then key = value lines (created-ms, source-incarnation, repository-<id> = <name> head=<n> commit=<hex>) and one file = <blake3 hex> <size> <relative path> per copied file.

Metadata export (.scmx): "SCMX" | u16 version = 1 | [16] repository id | u32 table count | per table: str16 name, u64 row count, (bytes32 key, bytes32 value)* | [32] blake3 of everything before it. Encoding is streamed unchanged; import accepts at most 64 tables, 64-byte names, 1 MiB keys, and 16 MiB values.

11. Limits that are part of the format

Path 8192 B; component 255 B; file 1 PiB; chunk 16 MiB; manifest chunks 2^30; change set entries 4,000,000; author 256 B; message 64 KiB; repository name 128 B; view prefixes 256.

12. Branch extension: repository schema 2 and 3, workspace schema 2, merge commits

The v1 encodings above are unchanged. Repository schema 2 adds:

Integers use little endian. Branch rows retain their v1 byte layout under schema 2; state 2 means an unpublished snapshot. A snapshot-root row pins inherited file state without creating native path history rows. User-facing path history follows the recorded source origin.

A merge is a v5 envelope with origin 1 (section 4.3). Before envelopes were chained, a merge was commit v2: u16(2), a u32-length-prefixed complete v1 commit payload, then u8(1), BranchId and u64 ChangeNumber of the merged head, hashed under scribe:commit:v2\0. Such envelopes remain readable with their original identities. The stored commit index row keeps its layout in both cases. The repository's global change sequence orders all commits; branch heads and first/second parents describe ancestry.

Workspace schema 2 adds workflows_v2 (UTF-8 string keys to explicit versioned bytes). It holds pending branch/switch intent, one active resolution workflow, and per-path immutable base/ours/theirs/target references plus selection and completion flags. Planning completes before application starts. File application, base/opened updates, and the resolution completion marker are coordinated with recovery protection. Accepted-submit finalization clears the active workflow in the same local transaction as its pending submit and baseline changes.

Resolution rows live under resolution/<workflow-id>/<path>, and resolution-lock/<path> (value u8 1) records one repository lock the active integration acquired for its own result. The claim is written before the acquisition, so an interrupted merge still releases it on abort; a submitted merge releases it by default, or retains it with keep-locks, and the row retires with the workflow. Retirement — a completed resolution, an abort, an archived intent, or a replan discarding an earlier plan — removes rows in 1,024-row pages before the marker that makes them reachable, because one workflow may hold a row per planned path (100,000) and a single transaction over them would be neither short nor bounded in memory. An interruption therefore leaves a smaller but still resumable workflow, never an active marker whose rows are gone. Accepted-submit finalization pages the same rows out before its acceptance transaction, which the still-pending submit repeats if it is interrupted.

Pending branch-operation bytes are u8 2 ‖ [16] repository ‖ [16] incarnation ‖ [16] operation ‖ [16] branch ‖ str16 name ‖ u8 action, then [16] source branch ‖ u64 source change for create (1), or str16 expected name for rename (2). The branch fingerprint is BLAKE3 of "scribe:branch-operation:v2\0" ‖ repository ‖ incarnation ‖ branch ‖ u64 name length ‖ name ‖ u8 action, followed by the source and change for create, or u64 expected-name length ‖ expected name for rename. Lengths are UTF-8 byte counts; all integers are little endian. Operation ID selects the outcome and is not itself part of its fingerprint.

Retired branch and resolution workflows are saved in .scribe/_state/recovery as branch-<operation>.pending and resolution-<workflow-id>.pending. Their archive encoding is u8 1 ‖ u32 row count ‖ (str32 key ‖ bytes32 value)*. The active marker is first; resolution rows follow in key order. Archives are bounded to 100,001 rows and 256 MiB of keys and values. Exclusive publication, file flush and directory flush precede removal of the active marker in a bounded Durability::Immediate transaction. An existing archive must match exactly; otherwise retirement fails with the marker intact. The archived rows are then paged out with their marker, since the archive is their durable copy. Local files, baselines and opened edits are retained. There is no fallback for earlier unreleased branch request layouts.

See the reference implementations in workflows.rs and conflicts/mod.rs.

Repository schema 3: shared metadata snapshots

Snapshot profile 3 replaces the unreleased current, current_fold, and fold_history_v3 serving indexes. Immutable compressed radix trees hold exact canonical path bytes and Unicode collision-key bytes. Node IDs are local u64 metadata references, not object identities or pack offsets. Asset and manifest identities are unchanged. Commit envelope v5 binds both portable state roots.

A link is exactly 88 bytes: u64 node id ‖ u64 live values ‖ u64 newest change ‖ content digest[32] ‖ revision digest[32]. The empty link has zero numeric fields and the two domain-separated empty-tree digests described below. A nonempty link has a nonzero newest change, even when its entire subtree consists of tombstones. Live count is the sum of live values; newest change is the maximum revision in the subtree, including deletes.

snapshot_nodes values use physical page version 3: u8 3 followed by one independent compression block specified in section 3 (codec:u8 ‖ raw_len:u32 ‖ stored_len:u32 ‖ bytes). The encoder selects LZ4 only when its body is smaller; otherwise it stores the raw body. No dictionary, parent fetch, or extra metadata lookup is needed to decode a page. Declared raw length is bounded by the maximum node body (55,650 bytes) before allocation; invalid codecs, lengths, or trailing bytes are refused. Compressor output is not an object identity.

The decompressed body has this layout:

Every integer is little endian. Values have nonzero changes; a live state's change agrees with its value. The node prefix equals its value's exact path or collision key. A valueless node has at least two children. Each child extends the parent prefix by its edge byte, has a smaller node ID, and agrees with its link's summaries. Allocation and decoding reject overflow, malformed flags, out-of-order edges, oversized prefixes, and trailing bytes.

Runtime query readers keep child records packed and share identical prefix/path bytes in one immutable page buffer. Canonical hashes stream from the same field projection used for portable encoding. A read view may reuse verified pages in a FIFO cache bounded by 128 entries and 2 MiB of page data (bookkeeping is separately bounded by the entry count). Every hit still checks the full expected link; the cache does not outlive its table view. Range traversal shares parent pages while siblings remain pending. Builders, proofs, and maintenance use owned nodes without populating this cache. These are memory-layout choices: physical profile 3, canonical node version 1, and root-link version 2 bytes remain unchanged.

snapshot_roots keys are BranchId(16) ‖ ordered_u64(change); only the ordered key segment uses big endian. Values are exactly 177 bytes: u8 2 ‖ exact-path link ‖ collision-key link. Their live counts agree. Main starts with two empty roots at change 0. Branch reservation copies the source's pinned root value at the fork change, including on nested forks; activation changes no pages and creates no inherited history/index rows. Every accepted commit stores roots at its native branch/change, even when all entries are keeps. Case-only replacement applies deletions before additions so the live spelling wins in the collision tree.

Preparation groups each batch's edits by exact and folded key before traversing the affected subtrees. Each affected existing page is read once per tree per batch; each final replacement page is hashed and encoded once, after its changed children. The traversal uses an explicit stack. There are no intermediate page versions within a batch. Work remains bounded to 64 edits per preparation batch; ancestors shared across different batches are revisited. Buffered plans retain pages in memory until their bounded acceptance transaction. Larger plans stage pages in bounded Immediate transactions before the publication gate. Acceptance publishes the roots with the commit ledger and outcome. Unchanged subtrees are shared. There is no separate page-file publication or recovery ordering.

A reverse root seek selects a pinned tree directly. Live-count summaries prune deleted subtrees during historical enumeration. Prefix ranges and newest-change summaries prune unrelated and unchanged subtrees during delta reads, which return each changed path once with its last change and final state, including tombstones. Native history, change_paths, moves, and file_lineage remain the accepted event ledger for logs and integration ancestry.

history retains full accepted revision state. change_paths retains only its ordered key, since commit enumeration reads paths, not states. moves retains only the source revision needed to traverse former names; its destination and accepting change are in the key. Verification checks move sources against history, and commit-path membership against history in both directions. file_lineage retains both edge directions for bounded neighborhood reads. Manifest acceptance witnesses are written only for newly encountered manifests; repeated use preserves the first durable witness. Staged non-witness insertion uses the previous value returned by insert to detect a collision and aborts the transaction, avoiding a separate existence lookup.

Profile 3 covers pages and these compact indexes. Stores with an earlier snapshot profile are refused; no implicit migration or compatibility reader exists. Canonical objects, commit envelopes, portable proofs, and TCP v2 do not depend on the snapshot profile.

doctor and verify check pages, links, summaries, root provenance, accepted history coverage, collision state, and total live count without mutation. Physical backups and logical export/import include both snapshot tables and profile/count metadata. There is no automatic conversion from pre-snapshot development stores and no fallback reader for their former indexes.

Local offline intent

The opened-record needs reconcile flag is strictly 0 or 1. Offline edit, local add/delete/move, and retained conflict-resolution work set it. Ordinary online edit starts with 0 but preserves an existing 1. Reconciliation sets selected opened records to 1 durably before network checks and clears only ready paths after checking pinned branch states and current main lock policy. It records updates in bounded local transactions; interruption can leave conservative pending markers or a prefix of completed results. Acquired lock generations use the existing local lock table. File bytes and baselines are untouched.

The flag is advisory, never evidence of server lock ownership. Revert and accepted-submit finalization remove it with the opened record. Move journals embed the complete prior opened record, including the flag, for exact abort. The flag is local workspace state only: repository objects and wire encodings do not carry it, and there is no compatibility reader for opened records without it.

Explicit local move intent

Workspace workflows_v2["move/<operation-id>"] contains one record per pending move. The operation ID is lowercase hexadecimal. The canonical value is:

• u8 version=1, id16 operation, u64 sequence, str16 source, str16 destination;
• u8 staged, u8 complete, u8 aborting (flags are strictly 0/1);
• u32 dependency count, followed by that many id16 earlier move IDs;
• u32 tracked file count, followed by records sorted by destination: str16 physical source, str16 destination, str16 original accepted source, u64 original source change, u8 has prior opened record, and, when present, bytes16 prior opened record using the opened-record encoding above.

Sequence numbers order active moves independently of their opaque IDs. They are unique among pending records; each new move takes the greatest active sequence plus one (or one for an empty journal). Dependencies reference existing older moves whose source or destination roots overlap the new operation. A prior opened record is an Add carrying the same original accepted source and revision; this allows both chain composition and exact restoration on abort. Without a prior opened record, the original source equals the physical source. Each destination must be the exact root substitution of its physical source.

The client bounds a move to 50,000 tracked files and 100,000 discovered entries. The whole journal is bounded to 1,024 operations and 32 MiB. Vacated source roots are reserved until submission or abort. IDs must match their keys; dependencies must be unique and refer backward; at most one intent may be interrupted.

Original local work is staged at .scribe/_state/staging/move-<operation-id> using exclusive rooted renames. It never enters disposable .scribe/_state/tmp, nor the recovery area that users prune. Intent is durable before opened-state changes or the first rename, and new destination parents are flushed before publication. Destination and staging directories are flushed before completion is recorded. Interrupted application blocks competing workflows. Abort is allowed only when no pending move depends on the selected intent. It performs the inverse rename, retains current contents, restores prior opened records, and removes only its own journal record. Accepted submit clears all pending move records in the same transaction as opened records, baselines, and pending submit; journal keys are collected before entering that transaction.

Shared repository configuration and main-branch lock policy

.scribe/config.toml is an ordinary versioned file: chunks, manifest, changeset, snapshot/history records, and commit acceptance use their existing encodings. Its lock section is read from main's current file record, while profiles are read from the selected branch revision. One shared typed parser validates the entire TOML document before accepting it on any branch; its schema is specified in repository configuration. No separate persisted policy copy or migration is needed. The compiled in-memory cache is keyed by the complete accepted file record; an absent record represents no mandatory rules.

Configuration bytes are read, verified, parsed, and lock rules matched before taking the repository publication gate. After taking the gate the service rechecks the policy record; if it changed, preparation is repeated outside the gate. The accepted old policy governs its own updating changeset. The final metadata transaction checks each required lock against its live owner, alongside existing base/path checks. The service supplies a prepared set of required paths; no glob matching or object I/O occurs in that transaction. Rejections 209 (lock_required) and 210 (invalid_repository_config) use the existing operation-record encoding. Rejection 211 (merge_incomplete) is decided before the gate and is not recorded as an operation outcome; no envelope exists for it. Accepted policy changes activate atomically through the same history transaction and are reconstructed from accepted main state after restart or restore.

Maintainer authorization compares the decoded ordered rule values against the exact file revision named by the change entry's expected base. The acceptance transaction already validates that base; profile-only changes cannot bypass concurrent policy updates. File removal is compared with an empty rule set. TOML formatting and source-line changes do not alter this permission check. Missing configuration has no rules and no profiles; an existing empty or invalid file is rejected. No old filenames or text parsers participate.

Rename-aware resolution records (unreleased)

The resolution/<workflow-id>/<path> record has version byte 2 and encodes, in order: optional group path; rename-present byte (when present, str16 base path and optional ours/theirs paths); path-conflict byte (0 none, 1 competing renames, 2 rename/delete, 3 destination collision); the five existing optional base/ours/theirs/target/selected BaselineRecords; choice byte (0 unresolved, 1 ours, 2 theirs, 3 merged, 4 working); applied byte. Optional paths are a strict 0/1 presence byte followed by str16. Optional states are a presence byte followed by u16 byte length and the encoded BaselineRecord. The record key supplies the physical path. Integer fields are little endian.

Group choices publish through one workflows_v2 transaction (at most 1,024 records / 32 MiB). File application remains restartable per path and protects replaced contents in recovery. Target references retain the original physical layout for abort. New destination opened records carry an explicit MoveSource when their target-side predecessor is selected for deletion. Planning limits remain 100,000 paths / 32 MiB path bytes / 128 MiB encoded resolution data. There is no decoder for earlier layouts.

Accepted manifest membership and anonymous workspaces

Repository schema 3 requires accepted_manifests, keyed by the full 32-byte ManifestId. Its value is AcceptedManifestRecord: u8 version=1 | [16] accepting BranchId | u64 accepting change, little-endian, with a nonzero change. This is a repository-wide union of accepted manifests, across all branches and retained history. Successful acceptance writes membership atomically with history; rejected submissions write none. Reuse can update the witness to a later accepted change. The table is included in logical export/import and checked against history and commit metadata by doctor/verify. A database lacking it is refused; startup does not backfill or initialize it.

Anonymous working copies store anonymous-workspace = UTF-8 true in the local workflow table. Explicit authenticated registration first persists authenticated-registration = a fresh 16-byte WorkspaceId, then transmits the registration. After its acknowledgement, the descriptor is durably replaced and one local transaction activates the ID and removes both markers. An interruption reuses the same persisted ID. No credential enters workspace state.

Local mount coordination

Workspace meta["mount-paths"] contains a u16 count followed by str16 canonical paths (at most 256 including retained directories). It reserves accepted mount boundaries before child creation and retains removed paths while their directories exist. Child workspaces use the ordinary workspace descriptor, databases, recovery, and journal inside their own .scribe/_state/ directories. Child workflow keys mount-pin and mount-applied-pin each contain exactly one little-endian u64: requested pin and last successfully applied parent pin respectively. Keeping them separate makes a failed child download resumable and distinguishes subsequent independent child submissions. These local records contain no credentials.

Parent workflow key mount-sync stores the little-endian u64 target before child sync starts. While it exists and every parent entry is applied, the parent journal finishes even when a selected mount failed: the parent watermark describes the parent tree alone, and a child that is unavailable or still downloading is reported through this marker and the child's own journal. Any unapplied parent entry still holds the parent journal open, and a view change may then replace it to exclude unavailable mounts. Successful overall sync clears the marker.

Workflow key sync-frontier stores exactly two little-endian u64 revisions, minimum then maximum. Before replacing an interrupted sync journal the client persists the endpoints covering its watermark, local accepted submission, and all interrupted targets. It reads this row only while a journal is pending; journal and view completion remove it atomically with the watermark. Restores use full reconciliation and replace the frontier. Malformed frontiers are refused.

Local project publication

meta["last-submit"] is an optional 40-byte receipt: incarnation[16] ‖ branch-id[16] ‖ change u64. Accepted-submit finalization writes it in the same transaction as baselines and pending/opened removal, retaining the maximum receipt in the same branch/incarnation. It does not advance synced. Completed ordinary/view sync removes the receipt atomically with its watermark; explicit backward sync reconciles against it even if the requested target equals the old watermark. A receipt from another branch/incarnation is not a local pin.

Workflow key project-owner contains exactly the 16-byte publication operation ID. It prevents ordinary mutations from independently resolving or replacing project-owned candidates. Workflow key project-publish exists only at the root and has this little-endian v1 encoding:

u8 version = 1
publication-id[16]
str32 message (ordinary submit message bound)
str32 original configuration (<= 65536 bytes)
u8 config-replacement-started (0 or 1)
u8 complete (0 or 1)
u8 aborted (0 or 1)
str32 last problem (<= 65536 bytes; empty means none)
u16 participant-count (1..65)
for each participant:
  str16 mount-name (<= 64 bytes)
  str16 path (canonical mount path; final participant is ".")
  repository-id[16]
  branch-id[16]
  workspace-id[16]
  incarnation[16]
  u64 local/accepted revision
  operation-id[16]
  u8 phase

Phases are 0 planned, 1 prepared, 2 outcome_pending, 3 accepted, 4 unchanged, 5 rejected, and 6 retired_after_restore. Children are ordered by mount name; the parent is last. Identity/path declarations must match the stored original config. Counts, strings, booleans, phases, and trailing/truncated bytes are checked. The complete golden/truncation fixture lives in project/journal.rs.

The root journal reserves all participant operation IDs before any submission. Each participant stores its frozen ordinary PendingSubmit before transmission. The root marks outcome_pending durably before invoking that operation. Parent pins are rewritten only after confirmed child acceptance; only the parent's config entry is amended while its candidate is still unsent. Its analyzed content must match the planned config bytes. The replacement-started marker makes interruption between retaining the old config inode and publishing the new one resumable.

Local accepted-submit finalization may precede the root receipt: if the candidate is gone after a crash, recovery queries that original operation outcome instead of creating a new ID. Missing outcomes in an unchanged incarnation remain unknown. As with ordinary submit recovery, an absent outcome in a changed incarnation retires and archives the old candidate, preserving opened work for explicit sync. Completion/abort is durable before participant cleanup; root journal removal is last. Coordination uses no credentials and no cross-repository acceptance transaction.

Cherry-pick provenance

A selected integration is a v5 envelope with origin 2 (section 4.3): the source branch and change follow the parent link. The source is mandatory, nonzero, earlier than the accepting change and on another branch. It records intent and never adds a merge parent. Before envelopes were chained, a cherry-pick was commit v3: u16(3), a u32-length-prefixed complete v1 commit payload, then u8(1), [16] source branch, u64 source change, hashed under scribe:commit:v3\0; such envelopes remain readable with their original identities. doctor compares provenance in the envelope and metadata.

The commit-index record uses version byte 2 for cherry-picks, followed by the old v1 fields and the mandatory source branch (16 bytes) and change (LE u64). Ordinary and merge index records remain version 1. No additional table or repository schema is required. The record and accepted operation outcome are written in the existing single Immediate-durability acceptance transaction after the envelope is durable.

Workspace pending-submit integration discriminator is 0 for ordinary, 1 for merge and 2 for cherry-pick. Kinds 1 and 2 carry the same source branch, source change and target head fields. Resolution-workflow source discriminator likewise uses 0 absent, 1 merge, 2 cherry-pick; its pinned base is the selected source commit's immediate first parent. The operation kind survives restart, retry and resolution updates. Unknown kinds are rejected.

Workspace lifecycle index (unreleased repository schema 3)

workspace_state is required at open; normal startup never creates it. It is included in metadata export, backup/restore, and read-only doctor/verify checks. Keys are byte strings with a one-byte kind followed by fixed-width IDs; values use little-endian numbers:

Active owner/index/count updates are one immediate transaction. Retirement removes the active row and owner-index entry, decrements the active count, and writes its fence and operation outcome together. Held locks are detected by one index seek, without scanning the lock table. Acquire and release maintain the lock index in the same transaction as the lock row. Registration, acquisition, and acceptance refuse retired identities. Retirements do not consume a repository change number. Auth ownership claims remain permanent; the active quota is enforced by repository metadata, not lifetime registration counts in auth state.

Client pending removal files contain u8 1 ‖ [16] repository ‖ [16] incarnation ‖ [16] workspace ‖ [16] operation and are scoped to an endpoint under per-user configuration. They are flushed and published before transmission. The lifecycle index is repository metadata only; object identities and canonical object bytes do not depend on it.

Repository migration journal

Repository schema 3 has an independent additive migration revision. In the meta table, migration-version is a four-byte little-endian u32: current is 3 (bytes 03 00 00 00). Its absence denotes pre-framework revision 0, accepted only by explicit migration inspection/application. A future revision is refused. Normal opens require the current revision and no active step.

During a step, migration-active is the next revision as a four-byte little-endian u32 and migration-cursor is its opaque, nonempty progress token (at most 8,218 bytes). Both must be present together. A batch's data and cursor commit in one Immediate transaction. Only after verification does another Immediate transaction advance the revision and remove both progress keys. Missing tables belonging to completed revisions are corruption, not a request to initialize them again.

Revision 1 (workspace-state-v1) creates or adopts workspace_state. Its cursor is byte 00 followed optionally by the last 16-byte workspace id, byte 01 followed optionally by the last canonical lock path, or byte 02 for final verification. Ranges exclude the recorded last key. At most 1,024 source rows are prepared per batch outside the write transaction. An absent lifecycle table selects the pre-owner workspace layout: u8 1, then branch, view, created-ms and label, with no owner-present byte; any other first byte is corruption. Each record is rewritten as version 1 of the current layout by inserting the owner-present byte (and owner) after the version byte. The storage root's permanent auth claim supplies its owner; an absent claim remains unowned. Converted records, owner counts, and indexes are written together with the cursor. Existing valid lifecycle state is adopted with cursor 02, preserving retirement fences and operation outcomes. The completed revision marker is recorded only after workspace index verification passes. See migration framework.

Revision 2 (path-fold-v1) verifies the current Unicode path profile and shared snapshots. Its only cursor is byte 04, meaning verify. Missing or incompatible path/snapshot profiles are refused; this step does not rebuild pre-snapshot development indexes. The completed revision commits only after read-only verification succeeds.

Revision 3 (change-notes-v1) creates the empty change_notes journal. Its only cursor is byte 01, meaning "created; verify and complete". A table left by an interrupted run is adopted when every row decodes, names an accepted change, and continues its change's revisions without a gap; otherwise the step refuses rather than rebuilding, because descriptions are user-authored history and cannot be derived from anything else. A completed revision 3 whose table is missing is corruption. doctor runs the same check as metadata.change-notes.

Metadata recovery journal profile 2

Catalog, repository, and storage ownership databases contain scribe_recovery_identity_v1 (u8 key 0 -> 16-byte random database identity) and scribe_recovery_records_v1 (u64 sequence -> canonical mutation bytes), plus scribe_recovery_position (u8 key 0 = end, key 1 = retained floor; values u64) and scribe_recovery_types (table name -> key/value type bytes). Sequence allocation uses persisted end, never the last retained row. Floor and end survive complete pruning; floor <= end and both name committed boundaries. Openers require this profile. A store without it is refused; there is no converter or implicit initialization. Root and repository descriptors declare recovery-journal-v2 in required-features, so older binaries also refuse the new stores. Commit envelopes carry the authenticated-actor field (section 4.3). An absent actor is an explicit unauthenticated attribution, never an authenticated user with a zero identifier. The author display label remains recorded separately.

Every authoritative transaction writes its mutations and a final commit record in the same Immediate redb transaction. Aborted transactions expose neither. Opening tables is recorded with their key and value types, so replay does not infer types from bytes. Catalog reservations, branch operations, locks, workspace state, change notes, and rejected/idempotent operation outcomes are included. The separately operated central auth service remains a separate backup responsibility.

Each record starts with u8 version = 1, then an opcode:

Types are 1 byte string, 2 UTF-8 string, 3 fixed 16-byte identifier, 4 little-endian u64. Table names are at most 256 bytes; keys at most 1 MiB; values at most 16 MiB. Unknown tags/types, invalid widths, malformed strings, and trailing bytes refuse. Recovery table names are reserved and cannot be targets of replayed mutations.

A .scrj segment is SCRJ, u16 version = 1, database identity [16], u64 after, u64 end, then records (u64 sequence, u32 length, bytes) for after + 1 ..= end, followed by a BLAKE3-256 digest of all preceding bytes. Both cursors are committed boundaries. Replay refuses gaps, wrong identities, repeated nonempty segments, incomplete transactions, corruption, and trailing data. Replay writes quarantined databases; only complete, verified restores are published. Every writable metadata handle also archives committed mutations independently before acknowledging writes or exposing reads. The database outbox is never treated as the independent copy; see the archive layout below.

Atomic publication and upload lifecycle profile 1

Root and repository descriptors additionally require atomic-publication-v2. There is no automatic upgrade or old-format fallback: binaries require its tables and strict-width counters before serving. The explicit scribe-server migrate converts atomic-publication-v1 stores (table submit_lock_transfers holding plain LockRecord acquisitions, and meta/transfer-count, meta/transfer-owner); see the migration procedure. scribe-format::lifecycle owns the canonical records and golden fixtures.

Streamed submission preparation reserves one next repository revision and writes at most 512 data rows or 4 MiB of key/value payload per Immediate transaction. Each data row has an undo inventory entry written in the same transaction. Snapshot tree construction processes at most 64 path changes per batch. These transactions run outside the publication gate, while the repository's submit turn excludes another preparation. Lock acquisition and release also join that turn, preserving the prepared lock epoch and reserved rename generations. Ordinary reads remain available; lock mutations wait for the current preparation to finish. Staging transactions append to the local durable recovery outbox without individually flushing archive HEAD. Preparation publishes that outbox once before entering the publication gate; final acceptance is independently archived before its reply.

Undo tags are 1 history, 2 moves, 3 change_paths, 4 file_lineage, 5 accepted_manifests, 6 snapshot_nodes (key u64 LE), and 7 lock updates. Unknown tags, missing rows, invalid encodings, and an undo row naming accepted metadata are corruption. Only newly inserted manifest witnesses are inventoried.

History/index readers exclude revisions above the accepted repository head. Snapshot pages become reachable only when the final transaction publishes their roots. That transaction rechecks branch/repository heads and the lock epoch, then atomically writes the commit ledger, branch head, snapshot roots/live count, lock-update visibility, and original operation outcome. Its row count is independent of submitted path count for streamed preparation. A stale preparation records no outcome and is retried outside the gate. The existing request byte/entry limits still bound decoded change-set memory and preparation time.

Buffered preparation writes nothing: at most 512 data rows / 4 MiB of encoded metadata remain in memory, with snapshot pages shared by reference. Such plans can prepare concurrently and are checked against repository identity, incarnation, heads, lock epoch, and page allocation before acceptance. No pending marker or undo inventory is needed for a buffered plan. Its final Immediate transaction inserts the bounded rows together with the ledger and roots.

Up to eight compatible buffered submissions can share that transaction, with the same aggregate data-row and byte bounds plus bounded per-member ledger rows. Plans that publish lock updates use individual acceptance. Each member retains its own change number, parent links, envelope, state roots, and persisted operation outcome. All envelopes are durable first. The existing recovery journal's transaction boundary contains every member's mutations; its independent archive is durable before responses. A stale member aborts the entire group. Canonical records, snapshot bytes, journal framing, and wire bytes are unchanged. Large submissions retain the streamed preparation described above.

Startup and recover-submissions remove an unaccepted preparation in bounded transactions. For accepted preparations they discard only the undo inventory; accepted rows remain. Visible lock updates are consolidated into the ordinary lock/owner indexes in bounded transactions. Reads and unlocks work before, during, and after consolidation. Recovery journals and logical exports include staging rows, markers, lock-update state, and upload pins.

Upload pins are persisted before clients transmit or reuse object identities. They may name absent objects. Pinning is idempotent; it neither accepts history nor changes an operation outcome. A workspace retirement fence refuses further pins and makes that workspace's pins inactive. Active pins do not expire on a clock. Final submission outcomes deactivate that intent only and enqueue bounded pin cleanup, resumed on startup. Each cleanup transaction deletes at most 512 pins; the final transaction also removes the intent's owner and cleanup marker. A bounded read of at most 513 keys identifies the final batch outside the write transaction. Finalized intents cannot acquire new pins, so this remains safe while other intents upload. Other intents stay protected. Explicit offline operator abandonment releases a workspace’s unfinished pins and ownership bookkeeping without altering accepted history. Unresolved upload/resolution intents are retained until retirement or explicit abandonment; they never expire. The client persists meta/upload-intent:<operation> = byte 1 before pinning. Finalization removes that local marker with the pending submit.

Offline object collection and compaction

compact-objects takes the exclusive data-root lock. It traces all accepted commit envelopes/change sets and active upload pins, retaining their complete object graph, including content of deleted historical paths. Missing pin roots are allowed (an upload may not have started); missing or corrupt referenced objects stop collection. Temporary redb live/queue indexes and batches of 512 identities bound graph bookkeeping memory. Individual structured-object decode limits still apply.

Retained objects are streamed through digest verification into a new store at objects.compacting. Its pack offsets and locator index are independent of the source. The live store remains intact until every retained object is durable. The temporary marker objects.compaction-ready.tmp is flushed and renamed to objects.compaction-ready, then the repository directory is flushed. The marker is exactly SCRGC1\0\0, repository [16], and metadata journal-end u64 LE. A changed metadata journal or malformed marker refuses recovery.

The durable marker authorizes the following recoverable sequence: rename objects to objects.previous; flush; rename objects.compacting to objects; flush; remove the previous store and temporary index; flush; remove the marker; flush. Startup and locator rebuild refuse a pending marker. Rerunning the operator command completes the recorded swap. A crash before marker publication leaves the original store authoritative; a later run recreates scratch state. No live readers coexist with relocation. This implementation requires temporary space for the retained store and graph indexes; it does not perform online GC.

Portable tree commitments (portable version 1)

Physical snapshot profile 3 does not change these canonical bytes or domains. Root-link records remain version 2. Metadata compression, allocation order, and secondary-index layouts do not participate in portable identities.

Local page IDs, allocation order, database identity, compression, and pack offsets never enter either commitment. Every digest is full BLAKE3-256 with an explicit domain. Each accepted v5 envelope binds the exact-path tree’s two roots. The collision tree remains a derived validation index, not another file identity.

The portable revision node is u8 1, bytes16 full prefix, then the same value-present/value/state fields as the stored node, u16 child count, and for each ascending edge: u8 edge, u64 live count, u64 newest change, content[32], revision[32]. It contains no page IDs. Its identity is blake3("scribe:revision-tree:node:v1\0" ‖ portable bytes). This binds manifests, last revisions, tombstones, and authenticated subtree summaries.

The portable content node is u8 1, bytes16 full prefix, u8 live-value present, then (when present) FileContentId[32], u64 size, u8 mode; u16 live-child count; and ascending u8 edge, u64 live count, content[32] for each live child. Deleted values/children do not participate. A node with no live value and one live child collapses to that child’s content digest; one with no live value or children collapses to the empty digest. Other nodes hash "scribe:content-tree:node:v1\0" ‖ portable bytes. This canonical live projection makes edit/revert and temporary add/delete restore ContentRoot even when the revision tree’s tombstone topology differs. Path keys enter map identity; asset FileContentId continues to hash only exact file bytes.

Empty ContentRoot hashes "scribe:content-tree:empty:v1\0": 79a13efedff230d33cd90d2b40d9a8ee86a3846303fb8c15ef88afd21139cb7e. Empty RevisionRoot hashes "scribe:revision-tree:empty:v1\0": 630bdd85b02bd400407c262590f09e82f16b0b7fafbb5ea97c926ca19c81f9f2. An all-deleted revision tree has empty ContentRoot but a nonempty RevisionRoot.

Preparation hashes the changed radix paths before taking the acceptance writer. Acceptance refuses an envelope whose roots differ from its prepared snapshot. Doctor compares retained roots with accepted envelopes. Portable proofs use the revision projection to authenticate live state and exclusions together. Bounded wire fragments retain these canonical node bytes and identities; continuation state is in memory and adds no persisted tree or workspace format. See wire proofs.

Independent archive and consolidated recovery bases

Export refuses missing sequence numbers and malformed mutation records before publishing an archive cursor. A committed end marker alone does not establish that the intervening journal is complete.

Each metadata file defaults to a sibling .recovery directory, for example metadata.redb → metadata.recovery. An optional .recovery-location file selects an absolute UTF-8 directory (1–32,768 bytes). Relocation copies, flushes, and verifies a new directory before atomically publishing that selection; the previous archive is retained. Normal startup requires an existing matching archive and stream profile. It never initializes or converts them.

HEAD is a canonical 112-byte archive cursor: SCRARC1\0, database identity[16], u64 segment, u64 byte offset, u64 end, chain digest[32], checksum[32] over the preceding 80 bytes. Integer fields are little endian. The zero position has zero offset/segment/chain digest.

Segment names are 20-digit decimal numbers plus .journal. A complete frame is u64 payload length, previous chain digest[32], SCRJ payload, chain digest[32]. Its chain digest hashes "scribe:recovery-frame:v1\0", previous digest, the little-endian length, and the SCRJ payload checksum. Appends roll to a new segment when the previous published segment is at least 32 MiB. A frame may exceed this target but streams without buffering the whole transaction history in memory.

The metadata transaction commits first. Archive data is flushed, then HEAD is written/flushed/renamed and the directory flushed, before acknowledgement. Mutation retries also cross this barrier. Writable reads instead clone the latest archived read transaction; concurrent archive publication cannot move this pointer backward. Readers see an older published snapshot while archival work is pending. An archive failure fences writes until restart but preserves published reads; startup drains the surviving database outbox. Bytes beyond HEAD are unpublished tails and can be replaced. Inspection and archive verification only read. A crash between metadata acceptance and archival publication has no successful response and can lose that unacknowledged outcome if both the database and its unarchived outbox are destroyed.

A logical base is SCRB, u16 version 1, database identity[16], u64 end, then u32-length-prefixed table declarations and puts, a zero u32 terminator, and a BLAKE3 checksum of all preceding bytes. Tables use their recorded types and ordered rows. Recovery bookkeeping is reconstructed separately. Replay reads, decodes and hashes outside write transactions, then applies batches of at most 512 records or roughly 4 MiB (one record may exceed that byte target) to a quarantined database. The serving identity is absent until the input is complete and verified. Never publish a failed or interrupted replay file.

base-<20-digit-end>.scrb is selected by BASE: an archive cursor[112], full base-file digest[32], and checksum[32] over the preceding 144 bytes. Offline consolidation publishes and verifies this base before advancing the persisted floor and deleting covered outbox rows in batches. It then prunes covered older segments/bases; the active segment’s prefix stays until rollover. Sequence end never decreases or derives from retained rows. Pruning itself is not journaled.

Consolidation holds the exclusive data-root lock and excludes live writers and archive consumers. Independent checkpoint copies are unaffected. Online checkpoints use manifest version 2 with a full flag: a missing cursor or one below a retained floor starts a full logical base; otherwise exports continue the prior cursor. Restore follows predecessors only back to a full manifest. The previous manifest link remains a publication check even for a new full base.

Offline snapshot-page reclamation uses a temporary disk mark set, retaining both roots of every historical branch snapshot. It refuses missing accepted-commit or active-branch origin roots before sweeping pages. Child IDs decrease, allowing a bounded reverse sweep. Only unreachable preparation pages are removed. Freed redb pages become reusable; neither operation promises to shrink its physical file. Accepted history, all operation outcomes, and unresolved upload intents remain retained. Metadata archive placement on a separate surviving volume protects metadata; asset-volume loss still requires independent verified object backups.

Incremental object compaction

compact-objects --incremental uses the same roots and durable swap marker as full compaction. Its temporary redb index adds incremental_state (string → bytes), pack_live_bytes (u32 → u64) and rewrite_packs (u32 → u8). Integers are little endian. State holds the canonical compaction boundary, a strict one-byte seed marker, selected source bytes (u64), and the last copied ObjectId key (33 bytes). Initialization publishes a complete index by rename and directory flush. Mark/queue work and copied locators survive interruption; a changed, well-formed metadata boundary discards the old scratch pass. Malformed state is refused.

Packs below 80% live are selected only while their whole source sizes fit the requested budget. Whole-dead packs need no payload rewrite. Unselected packs and live loose files are hard-linked into the replacement store; new writes start above the source's last pack id, so shared packs are never appended. A cache bound to that source holds at most 1,048,576 durable pack links across copy batches; reopening re-establishes link durability before populating it again. Relocated objects pass ordinary digest verification. Reused bytes are unchanged and are not scrubbed by this mode; use doctor or full compaction for that verification. Copy batches commit target locators before advancing the durable cursor. The replacement is installed only after its locator count matches the live inventory.

Workspace history trust

Keys trusted:<branch> in the workspace meta table contain version u8 (1), repository[16], branch[16], incarnation[16], change u64 LE, commit flag u8, optional CommitId[32], and BLAKE3 of all preceding record bytes. Only change zero has no commit. Decoding rejects invalid flags, widths, trailing bytes, checksums and foreign identities.

sync-proof-target holds target change u64 LE plus that anchor after every plan page has been verified. Journal completion moves it to applied-proof:<branch> in the same transaction that advances the watermark, including view changes. An unfinished journal cannot advance the applied proof; deferred opened paths retain their separate prior baselines and deferred entries. Other workflows that finish without a proof invalidate that binding. Explicit re-anchoring invalidates it too, forcing verified full sync. trust-progress/<branch> and trust-state/<branch>/<target> workflow records hold three length-prefixed anchors plus a checksum for bounded ancestry progress. Resolved historical cursors survive plan retries and are removed after completion.

Lock lookups consult active updates before the ordinary lock table. Releases hide their original acquisitions immediately; bounded cleanup removes those physical rows and their workspace-owner indexes. Acquisitions insert both indexes. A lock mutation settles its own active update in the same transaction before acquiring or releasing, so deferred cleanup cannot remove a newer token. Effective lock count is physical count + update count - 2 × release count. Workspace removal settles updates in bounded batches under the submit turn, before taking the publication gate and entering its short write transaction. Pending submit records save local lock generations; accepted-client finalization clears only matching tokens when keep-locks is false.