Verified object repair

scribe-server repair-objects repairs damaged or missing object bytes from a verified copy, without restoring older metadata. It preserves accepted history, repository and root incarnations, branches, workspaces, locks, operation outcomes, and object identities. Clients do not need to accept a different history after this repair. Canonical object, metadata, and wire formats are unchanged.

The command is explicit and offline. Stop the destination server and any server using the donor root: repair takes exclusive ownership of both roots. The online integrity checker never invokes repair automatically.

Preview and apply

Use a separate physical backup made by scribe-server backup, or an offline data root containing the same repository ID. For an online checkpoint, first run restore-checkpoint into a separate root and use that restored root as the donor. The donor may be older than the destination and use a different compression policy. Every replacement is verified against the destination's original full object identity; donor history is never imported.

# Read-only preview of repository data; neither store is repaired or cleaned.
scribe-server repair-objects --data-dir /srv/scribe/data --repo game --from /backups/game

# Recompute and apply the plan, then verify the result.
scribe-server repair-objects --data-dir /srv/scribe/data --repo game --from /backups/game --apply

Add global --json for one structured result. The preview lists object identities, the destination failure, whether the donor copy verifies (source_error is null), and packs that must be rewritten. A damaged pack header or unindexed entry is reported under pack_problems. A plan is recomputed on apply, so preview output does not authorize trusting bytes that changed since the preview.

The command checks every indexed object and walks accepted changes to discover missing commit, changeset, manifest, and chunk locator entries, including deleted historical files. --object kind:full-digest can additionally name an absent object outside that accepted graph, such as an unfinished upload. Repeat the flag for additional identities. No unrelated objects are imported from the donor. If an accepted structured parent is unavailable, its descendants cannot all be discovered; the unavailable parent is reported and installation is refused.

Exit codes:

0: the preview found verified replacements for all discovered failures, or the requested application/resume/abort succeeded. A successful preview is not a declaration that the destination is healthy.
1: a required donor object failed verification, or replacement verification found remaining problems. No unverified replacement is installed.
2: the command could not run, for example because a root is in use, an index is unreadable, a link was encountered, or a pending repair needs another action.

A damaged locator database must be made readable with the existing rebuild-locator command first. Repair can recover missing rows afterward from accepted metadata and the donor. Damaged metadata, missing store descriptors, or an unreadable donor database require their respective recovery tools; object repair does not invent metadata or reconstruct bytes without a good copy. The destination's packs, loose, and staging directories must exist.

What changes on disk

Repair prepares a replacement object store beside the current one. It rebuilds the locator inventory, copies verified replacement objects, and rewrites the other indexed objects sharing each affected pack. Healthy packs and loose files use durable hard links, and the new writer reserves pack IDs beyond the old writer so it can never append to a shared evidence pack. Repair retains all indexed objects, including unaccepted uploads; it does not perform collection.

The complete candidate is checked against current metadata, with full object verification and a pack audit, before a durable two-phase marker permits the directory switch. The result is checked again before normal serving is enabled. No metadata transaction accepts or rewrites history during repair.

This avoids copying all healthy payloads, but it is not a constant-time patch: planning and final verification scan the repository, and a new locator index is built. Budget downtime for that work, and free space for the new index and all rewritten packs/loose objects. Up to 4096 object findings and 4096 affected packs are supported in one plan; more widespread damage requires whole-root recovery. Hard-link support on the destination filesystem is required, as for incremental compaction. The donor only supplies content and need not be on that filesystem.

Evidence and interrupted repair

Each operation owns repos/<id>/objects.repair-<operation-id>/. After success:

previous/ preserves the entire original object store, including damaged bytes.
plan.json records the verified preparation plan and donor path. It is a plan, not proof that a later interrupted switch finished.
The authoritative store is still repos/<id>/objects/.

Evidence is never automatically removed or included in locator rebuilds. Healthy files may share inodes with the live store: never edit evidence files in place. If you need to alter evidence for investigation, make an independent copy first. Standard backups/checkpoints cover the active store, not these evidence folders; archive evidence separately. After diagnosis and a verified independent backup, an operator may explicitly remove a completed evidence directory while the server is stopped. Keep the pending marker and all referenced directories during recovery.

objects.repair-pending is checksummed and bound to the repository's metadata recovery boundary and incarnation. Serving, verification through normal opens, and locator rebuild refuse an incomplete repair. They never guess which store should win. A damaged marker or changed metadata is refused for diagnosis.

If preparation was interrupted, rerun the same --from ... --apply command with the same additional --object identities. Unaccepted preparation is rebuilt; the original store remains untouched.
Once the candidate is durably ready, finish without the donor:
sh
```
scribe-server repair-objects --data-dir /srv/scribe/data --repo game --resume
```
To discard preparation while the original store is still in place:
sh
```
scribe-server repair-objects --data-dir /srv/scribe/data --repo game --abort
```
Abort refuses once the old store has moved to evidence; use --resume then. Aborting does not fix the original corruption.

The marker's file and directory flushes, both parents of each store rename, and final marker removal are separate crash windows exercised by process tests. Lost progress never turns a partially prepared object into an accepted repair.

After repair, the online checker's old findings remain historical reports. To start fresh coverage immediately, while the server is stopped preserve and remove only the repaired repository's checker .json and .writing progress files as described in online integrity checking. Restart the server normally; no data migration or history-trust reset is needed.