Normative description of the bytes exchanged between scribe and
scribe-server. Reference implementation: crates/scribe-protocol-tcp
(frame.rs, wire/, client.rs, server.rs). Logical semantics live in
crates/scribe-api; another protocol may encode the same operations
differently without touching storage or operation identity.
1. Transport
TCP, either plaintext or inside TLS 1.3. One control connection per client;
up to four data connections bound to the same session. Exactly one request
is in flight per connection; the response carries the request id back. No
multiplexing. Default listen tcp://127.0.0.1:7447; non-loopback binding
in plaintext mode requires explicit configuration.
An authenticated server (see auth.md) accepts only TLS and only protocol 3: the same frames, codecs, and opcodes as protocol 2, with an extended hello (section 4). It never negotiates plaintext or protocol 2.
Timeouts: the server waits 10 s for the first frame (handshake) and then applies an inactivity timeout (default 300 s) between frames; there is no total limit on a long transfer. The client applies a 300 s read/write timeout and a 10 s connect timeout.
2. Frame
offset size field
0 4 magic "SCR1"
4 2 opcode u16 LE
6 2 flags u16 LE bit0 STREAM: object data follows the payload
bit1 ERROR: response payload is an error
8 8 request id u64 LE client-assigned; echoed unchanged
16 4 payload length u32 LE 0 ..= 67,108,864
20 4 reserved must be 0
24 n payloadThe header is always little-endian, whatever the negotiated codec. A receiver that sees bad magic, unknown flag bits, nonzero reserved bytes, or a length over the maximum closes the connection: its position in the stream is unknown.
3. Codecs and primitives
Payload fields are encoded by the negotiated codec. Codec 1 is little-endian
and is the only production codec. Codec 2 encodes the same layouts big-endian;
it exists so contract tests prove semantics and operation identity are
independent of byte order, and servers accept it only when built for tests
(allow_test_codec), never through configuration.
| primitive | encoding |
|---|---|
| u8, bool | one byte; bool is 0 or 1, anything else is malformed |
| u16, u32, u64 | codec byte order |
| str16 / bytes16 | u16 length, then bytes; strings must be UTF-8 |
| str32 / bytes32 | u32 length, then bytes |
| opt_str16 | u8 present, then str16 if 1 |
| digest | 32 raw bytes |
| id16 | 16 raw bytes |
| object id | u8 kind (1 chunk, 2 manifest, 3 change set, 4 commit), digest |
| path | str16 whose bytes must form a canonical path (else invalid_path) |
| prefix | str16, empty or a canonical path |
| file state | digest manifest, digest content, u64 size, u8 mode, u64 change |
| opt file state | u8 present, then file state |
| semantic error | u16 code, str16 detail (≤ 4096 bytes) |
| lock info | path, u64 generation, id16 workspace, id16 branch, str16 author, u64 acquired ms |
| submit outcome | id16 operation, u8 ok; if ok: u64 change, digest commit id; else semantic error; then u64 file flushes, u64 flush nanos |
Every count is validated against its limit and against the remaining payload before allocation. Trailing bytes after a payload are malformed.
4. Handshake
Request opcode 1 hello, payload always little-endian:
u16 protocol version (2, or 3 over TLS) | u8 codec (1 or 2) | u32 capabilities (0 = raw object transfers, 1 = LZ4 object transfers) | u64 bind session (0 = new) | str16 client string (≤ 256)Protocol 3 appends str16 token (64 bytes) and [32] binding to the same
hello. The token is the user's personal access token; the binding is zero
for a control connection and, for a data connection, the 256-bit secret the
server returned when the control session was established, proving the same
credential and a live control session.
Response opcode 1: u16 protocol version | u8 codec | u64 session id | u32 inactivity timeout seconds | u8 compression (0 none, 1 LZ4).
A protocol-3 control session additionally receives its binding secret.
Errors during the handshake are little-endian error frames:
unsupported_version for a protocol or codec the server does not accept
(there is no fallback to an older protocol; an older client is simply
refused), unauthenticated for a bad or expired token, malformed
otherwise, then the connection closes. A data connection passes the control
session's id in bind session and receives the same id back.
5. Errors
A response with the ERROR flag carries a semantic error. Codes are the stable
values from scribe_core::ErrorCode, also persisted in operation records:
| range | codes |
|---|---|
| 1..99 request | 1 malformed, 2 unsupported_version, 3 limit_exceeded, 4 unauthorized_network, 5 busy, 6 shutting_down, 7 cancelled, 8 unauthenticated, 9 forbidden, 10 auth_unavailable |
| 100..199 repository | 100 repository_not_found, 101 repository_exists, 102 branch_not_found, 103 branch_not_main, 104 workspace_not_found, 105 change_not_found, 106 path_not_found, 107 invalid_path, 108 incarnation_mismatch |
| 200..299 submission | 200 base_mismatch, 201 already_exists, 202 not_present, 203 locked_by_other, 204 path_collision, 205 object_missing, 206 object_invalid, 207 operation_intent_mismatch, 208 empty_submission |
| 300..399 locks | 300 lock_held, 301 lock_not_held, 302 lock_generation_stale |
| 500..599 storage | 500 storage_failure, 501 integrity_failure, 599 internal |
Codes 1–4, 8–9, and 100–499 are client outcomes and are not server errors.
forbidden is returned for any operation the token's role does not allow,
and repositories the user cannot read are absent from repo_list.
6. Messages
Request payload → response payload. repo, branch, workspace,
operation are id16; change fields are u64.
| op | name | request | response |
|---|---|---|---|
| 2 | server_info | (empty) | u16 protocol, str16 server version, u64 session id, u32 max batch objects, u64 max batch bytes, u32 max page entries |
| 3 | repo_create | str16 name, u8 storage compression (0 none, 1 LZ4; permanent) | repo info |
| 4 | repo_list | (empty) | u32 count, repo info × count |
| 5 | repo_info | u8 1 + id16, or u8 2 + str16 name | repo info |
| 6 | workspace_register | repo, workspace, branch, u16 view count, prefix × count, str16 label | (empty) |
| 7 | presence | repo, u32 count (≤ 8192), object id × count | u32 count, bool × count |
| 8 | put_objects | repo, u32 count (≤ 1024), (u8 kind, digest, u64 length) × count; STREAM flag set; then the concatenated payloads in order (total ≤ 256 MiB) | u32 count, per object: u8 1 stored / 2 already present / 3 rejected + semantic error; then u64 file flushes, u64 dir flushes, u64 flush nanos |
| 9 | submit | repo, branch, workspace, operation, digest change set, str16 author (≤ 256), str32 message (≤ 65536) | submit outcome |
| 10 | operation_outcome | repo, operation | u8 present, then submit outcome |
| 11 | head | repo, branch | u64 head, id16 incarnation |
| 12 | tree_page | repo, branch, u64 change, prefix, opt_str16 after, u32 limit | u32 count, (path, file state) × count, bool more |
| 13 | changed_paths | repo, branch, u64 from (exclusive), u64 to (inclusive), prefix, u8 token present (+ bytes32 token), u32 limit | u32 count, (u64 change, path, opt file state as of to, bytes32 token) × count, bool more |
| 14 | get_objects | repo, u32 count (≤ 1024), object id × count | STREAM flag set; payload u32 count; then per requested id in order: u8 present, u64 original length (zero if absent), followed by object data when present |
| 15 | log | repo, branch, opt_str16 path, u8 before present (+ u64), u32 limit | u32 count, per entry: u64 change, digest commit id, u64 parent, str16 author, str32 message, u64 timestamp ms, u32 entry count, id16 workspace, id16 operation, u8 action present (+ u8 action), opt file state |
| 16 | lock | repo, path, workspace, branch, str16 author | lock info |
| 17 | unlock | repo, path, workspace, u64 generation, bool force | (empty) |
| 18 | locks | repo, prefix, opt_str16 after, u32 limit | u32 count, lock info × count, bool more |
| 19 | ping | (empty) | (empty) |
Repo info = id16 id, str16 name, id16 main branch, u64 head, id16 incarnation,
u64 created ms, u8 storage compression. Page limits are clamped to 1..=4096 entries. after and
token are opaque continuation values taken from the previous page's last
entry.
7. Streaming rules
The client selects transfer compression in Hello; the server echoes the accepted choice in HelloOk. Unknown capability bits are rejected and the client rejects a different echoed choice. Control frames, tokens, and other metadata are not compressed. Bound data connections negotiate their own transfer policy (the client carries its existing preference forward).
With compression 0, object data is the original byte stream. With compression
1, each object is a sequence of bounded blocks with the same layout specified
in the object-envelope section of on-disk-format.md, without the stored
object's u64 original length prefix. Announced object lengths and admission
budgets always count original bytes. Block headers are little endian even
with the big-endian test metadata codec. Decoder allocation is bounded to
256 KiB per block; malformed blocks close the connection. Empty objects have
no blocks. Receivers drain unfinished objects before reading the next item.
Transfer compression is independent of repository storage. Stored compressed objects are decoded before being passed to the adapter; the initial implementation recompresses them when transfer compression is enabled. Raw storage and raw transfers retain the direct file-to-socket path.
put_objects: the sender announces every length up front and then writes object data that decodes to exactly those lengths. The server hashes the original bytes while receiving, validates manifests (all chunks present, in the store or earlier in the same batch) and change sets (all manifests present), publishes the batch durably, and only then answers. If a request-level failure occurs before all announced bytes were consumed, the server sends the error and closes the connection; the client reconnects and retries (uploads are idempotent).get_objects: the response header commits to a count; each item is self-describing so absent objects are reported inline. A receiver that stops consuming an item must drain the remaining bytes to stay in sync.- Resume is at chunk granularity: a client asks
presenceagain and re-sends only absent chunks. - Byte accounting: the harness reports "protocol bytes" as header + payload +
stream bytes as seen by the client (
TcpRemote::bytes_sent/received).
8. Semantics every adapter must preserve
- Every request naming a branch is checked against the repository's branch
table before any mutation;
branch_not_foundnames an unknown id. - A mutating response is sent only after the durable commit. A lost response
leaves the outcome unknown;
operation_outcomeresolves it. - The operation fingerprint is computed from the semantic request, so a
submit replayed through another adapter or codec resolves to the same
accepted change; a replay with different intent is
operation_intent_mismatch. - Locks, budgets, and publication gates are shared server state, identical across concurrently connected adapters.
9. Branch and merge opcodes
Framing and the opcode payloads above are those of protocol 1; protocol 2
adds the opcodes below and is the minimum this build serves. A protocol-1
hello is answered with unsupported_version and the client does not fall
back.
| Opcode | Operation |
|---|---|
| 20 | branch list |
| 21 | branch create/rename with persisted operation id |
| 22 | branch operation outcome |
| 23 | pinned merge parents and merge-base discovery |
| 24 | submit with expected target head and pinned source parent |
The canonical codecs are in scribe-protocol-tcp/src/branches.rs.
Names, counts, input lengths, and trailing bytes are checked. A merge submission
uses the existing accepted/rejected outcome encoding and operation_outcome
for retries. Data connections negotiate the same version as their bound session.
Opcode 25 (MergeRead) is read-only and requires repository schema 3.
Its request begins with RepositoryId and a u8 query tag:
- 1: u64 left revision, u64 right revision, optional str16 path cursor. A page traverses the larger first-parent revision until the chains meet, at most 128 commits or 1,024 changed paths. Reply: tag, updated left/right revisions, optional cursor, u32 path count, str16 paths. Equal revisions signal completion.
- 2: BranchId, u64 revision, u32 count, str16 canonical paths (maximum 1,024). Reply: tag, u32 count, optional FileStateAt for each input in the same order.
- 3: BranchId, u64 revision, str16 canonical path, optional str16 folded cursor. Reply: tag, optional next cursor, u32 count, (str16 path, FileStateAt) entries. The first page includes exact/ancestor matches. Each page examines at most 1,024 historical descendant paths, including deleted paths. An empty page can still carry a continuation cursor. Replies are bounded to 5,120 entries, allowing the ancestors of a maximum-length canonical path.
Optional strings are a boolean presence byte followed by str16 when present. Numeric fields follow the negotiated codec; the deployed codec is little endian. Malformed, oversized and trailing payloads are rejected. All reads are repository-scoped and enforce branch/revision bounds.
10. Upload admission under protocol 3
On an authenticated connection put_objects is answered with an admission
response before the client sends the streamed payload, so a reader cannot
force the server to consume an upload it will reject. Object payloads pass
through TLS; the zero-copy sendfile path is used only on plaintext
connections.
Gateway inventory and authorization preflights (opcodes 26–28)
These additive operations use the existing TCP v2/plaintext and v3/TLS handshake, framing and negotiated codec. Existing operation payloads and canonical object bytes are unchanged. A storage server remains usable without a gateway.
InstanceStatus (26) request: u8 has_after (0 or 1), followed by an exact
16-byte repository ID when present. Response, in order:
- Exact 16-byte data-root incarnation.
u64 total_repositories,u64 used_bytes(apparent whole-root bytes, populated on the first page only).u8 has_available_bytes(0 or 1), optionalu64 available_bytes.u32 repository_count, at most 128; each entry is the existingRepoInfoencoding followed byu64 apparent_repository_bytes.u8 has_next(0 or 1), optional exact 16-byte exclusive repository-ID cursor.
Entries are ordered by repository ID. The gateway retries inventory if the identity or total count changes between pages. On authenticated storage the operation requires namespace-administrator authorization. It is read-only and performs metadata work in the existing bounded blocking storage pool. Gateways use it for placement and discovery, not to acknowledge repository mutations.
AuthorizeUpload (27) request: exact 16-byte repository ID. Successful response:
empty payload. Storage checks the same writer authorization as upload admission.
A gateway uses this before granting its own v3 upload admission. The storage
server repeats admission for the actual upload; this preflight grants no lease.
AuthorizeRepoCreate (28) request and successful response: empty payload.
Authenticated storage requires namespace-administrator authorization; plaintext
storage permits it. A gateway checks this before persisting a new placement.
Creation itself repeats authorization at storage.
The new inventory golden fixture is in
scribe-protocol-tcp/src/wire/tests.rs. Gateways forward all existing repository
operations and stream object bodies; failed upstream transports close the client
connection so unknown mutation outcomes are not presented as definitive failures.