Tests and qualification

Run everything:

cargo fmt --all -- --check
cargo clippy --workspace --all-targets --all-features --locked -- -D warnings
cargo test --workspace --locked
cargo test --workspace --all-features --locked        # adds the process-kill fault tests
cargo bench --workspace --no-run --locked
cargo build --release --locked -p scribe-server-cli -p scribe-cli -p scribe-perf
cargo run --release --locked -p scribe-perf -- run --profile ci --root ./target/perf-sandbox --output ./target/perf/current.json
cargo run --release --locked -p scribe-perf -- compare ./tools/scribe-perf/baselines/macos-ci.json ./target/perf/current.json --policy ./tools/scribe-perf/policy/regression.toml

The all-features configuration includes the process-kill tests plus an ignored large-scale qualification. Certificate tests need an OpenSSL 3 executable (SCRIBE_TEST_OPENSSL names it when it is not openssl on PATH).

Performance harness lifecycle

scribe-perf uses prebuilt executables; it never invokes Cargo or rustup while running benchmarks. Use --server-bin and --client-bin for explicit executable paths. Compiler metadata is captured when the harness is built.

--root must be a dedicated new or empty directory with an existing parent. The harness removes it after success, failure, or a handled interrupt, after stopping/reaping its children and joining workload threads. --output must be outside that root and is the retained JSON report. A setup failure leaves an existing report untouched. --keep-data explicitly retains the root, logs, and workspaces for diagnosis; subsequent runs refuse a nonempty root, even one previously marked by the harness. Choose another root or explicitly remove the retained data yourself.

Each child receives an allowlisted environment with HOME, USERPROFILE, APPDATA, LOCALAPPDATA, XDG directories, and temporary paths under the sandbox. Client config and global-ignore paths, server data, runtime, and log paths are explicit. Inherited Scribe credentials/settings cannot redirect a run. Git metadata probes also use the isolated environment with optional locking and fsmonitor disabled. No process-wide environment mutation is needed.

Lifecycle and scenario progress goes to stderr, with elapsed time and terminal colors (NO_COLOR, TERM=dumb, and redirection disable colors). The existing stdout summary, JSON schema, metric names, and timed workload operations remain. Phase logging is outside the per-operation latency loops. SIGINT/SIGTERM/SIGHUP request a cooperative stop: an active command/storage operation finishes, then cleanup runs. Normal server cleanup first requests graceful shutdown, falling back to kill and wait after 60 seconds. SIGKILL and power loss cannot run cleanup.

Run lifecycle tests with cargo test --locked -p scribe-perf. Full CI-profile qualification additionally exercises subprocess shutdown, restore verification, and report retention. Platform coverage must be reported separately.

Lint policy

Every crate inherits [workspace.lints] from the root Cargo.toml. CI treats warnings as errors and checks every target with every feature. clippy.toml sets the minimum supported Rust version and permits test assertions to use unwrap and expect.

The default Clippy set (including perf) is augmented with:

• host_endian_bytes: serialization must choose endianness explicitly.
• cast_possible_truncation, cast_sign_loss, cast_possible_wrap: audit narrowing conversions, lengths, offsets, and signed values.
• unwrap_used, expect_used: audit production panic paths.
• print_stdout, print_stderr: libraries return results or emit tracing events.
• dbg_macro, todo, unimplemented: reject debugging and unfinished code.
• large_stack_arrays: review oversized stack buffers.

Unsafe code is denied workspace-wide; only scribe-platform explicitly allows it for documented OS interfaces. unsafe_op_in_unsafe_fn remains denied. Executable and benchmark entry points allow terminal output. Integration test helpers and qualification workloads explicitly allow fail-fast assertions; production invariant exceptions use local #[expect(..., reason = "...")] so removing the triggering code also exposes a stale exception. Keep exceptions at the smallest practical scope. Cargo's workspace lint inheritance is all-or-nothing, so target-specific exceptions live in source attributes.

Infallible encoders check that counts fit their frozen fields rather than silently truncating them. Their callers remain responsible for validating application limits. These checks do not replace golden fixtures, recovery and fault-injection tests, or performance qualification.

What is covered

Fault injection is compiled only with the fault-injection feature on the server crates and armed by SCRIBE_FAULT; release builds have no such switch.

Where it runs

.github/workflows/ci.yml is manual (workflow_dispatch) only; it does not run on push or pull request. When started, it runs format, clippy, both test configurations, benchmark compilation, and the ci perf profile on ubuntu-latest, windows-latest, and macos-latest, checks the dependency graph of each executable, and cross-tests scribe-core and scribe-format on emulated s390x for big-endian correctness. The perf step builds scribe-server-cli and scribe-cli first; scribe-perf never compiles those executables itself. Jobs and long steps have timeouts so a hung test cannot run for the GitHub-hosted six-hour limit. Windows-only and Linux-only tests are #[cfg]-gated and therefore run only on those runners; a green macOS run says nothing about them. GitHub re-enables a disabled workflow if this file is pushed, which is why the automatic triggers are omitted.

Tokio qualification on macOS (2026-09-07)

The server migration passed workspace formatting, Clippy with all targets and features, default and all-feature tests, and benchmark compilation on macOS. The CI performance profile passed its correctness gates. An additional release probe kept 1,024 connections alive per service: storage used 9 OS threads and about 36 MiB RSS; auth used 5 threads and about 26 MiB RSS with TLS. Ping and administration remained responsive, and both services drained in about 39 ms. These are measurements from this machine, not production capacity guarantees.

Performance qualification remains incomplete: the measured mixed-workload sync p99 was 9,733 ms versus the stored baseline's 427 ms, exceeding the regression budget. Read p99 was 0.264 ms versus 0.277 ms. Concurrent qualification work was observed sharing the same machine and disk during follow-up diagnosis; repeat on an otherwise idle machine before attributing or accepting the sync result. The baseline and durability requirements were not relaxed. Local reports are target/perf/current.json, target/tokio-perf-compare.log, and target/tokio-connection-probe.json.

Linux and Windows execution remains a gap for this migration. Local cross-build attempts lacked the native compiler/assembler needed by ring; they do not substitute for platform CI.

Not covered by any automated test

• Power-loss ordering. Process kills do not prove it. A VM or device-level power interruption with doctor afterwards is required before claiming the OS-crash row of the failure model on real hardware.
• The million-paths and scale perf profiles: implemented, opt-in, run only on dedicated hardware.
• Kernel-level syscall counts: the harness reports its own instrumented opens, renames, and flushes.
• Auth replication or failover: not implemented, so not tested.

Hidden password prompt (Unix terminal regression)

cargo build -p scribe-cli -p scribe-auth-cli --locked
python3 crates/scribe-platform/tests/password_prompt.py

The stdlib-only PTY check runs the auth service and client, verifies hidden input with spaces and Unicode, confirms a saved session, and checks cancellation and terminal echo restoration. On macOS it also checks Secure Input during entry and restoration after success, wrong passwords, Ctrl+C, and EOF. A native guard test covers panic cleanup and preserving another Secure Input assertion. It runs on Unix; it does not claim Windows console coverage. The normal client tests also cover redirected input and password files.

Password pepper and authenticated demo

Auth tests cover salt uniqueness, required matching pepper on restart, rejection of pepper files under auth data or .scribe, verification failure without the pepper, password-reset replay, session invalidation, and secret exclusion from database bytes and service logs. Wire fixtures pin the transient password-change payload separately from persisted intent.

The scenario runner provisions passwords and logs users in over TLS before exercising saved sessions through collaboration and all four compression variants:

cargo build --release --locked -p scribe-auth-cli -p scribe-server-cli -p scribe-cli
node tools/scenario.js --report /tmp/scribe-results.json
node --test tools/scenario.test.js tools/scenarios/scenarios.test.js

Choose a new report filename for each run. See runner documentation.