Files
ngit-grasp/docs/explanation/git-family-object-storage.md
T
DanConwayDev 8c2ee32ee6 fix(git): isolate staged expiry recovery failures
A malformed or unreadable staging record should not keep the entire relay offline. Skipping that record creates no new expiry authority and preserves live refs and their history.

Recover each record independently, log failures with the record path, and continue recovering healthy views. Leave suspect metadata and data intact for inspection. Registry enumeration failure remains fatal; this change does not add automatic metadata repair or batch retained-root writes.

Validation: reproduced the corrupt JSON failure before the fix. All 18 staging unit tests and 62 integration/helper tests passed, including invalid-path isolation, healthy-record recovery, data preservation and relay restart with corrupt metadata. Workspace all-target clippy with warnings denied, cargo fmt --check and git diff --check passed.

Assisted-by: GPT-6
2026-09-29 11:43:55 +00:00

684 lines
34 KiB
Markdown

# Identifier-family Git object storage
**Status:** Accepted for implementation
**Date:** 2026-08-17
## Decision
ngit-grasp will store Git objects once per repository identifier and object
format, while continuing to expose a separate bare repository view for every
owner and every GRASP-06 contributor route.
The shared object inventory is always enabled. Its default durable backend is
the local filesystem. An S3-compatible backend is an explicit operator opt-in
that keeps repository views and metadata local, stores immutable packs in
object storage, and hydrates a bounded local cache on demand.
We deliberately do not garbage-collect Git objects in this change. Deleting or
rolling back a Nostr state changes which refs a view exposes; it does not remove
objects from the identifier family. This preserves the current ability to
recover from deletion-state mistakes and late rollback decisions.
## Context
The current layout creates a complete bare repository at both
`<npub>/<identifier>.git` and, when GRASP-06 is enabled,
`prs/<submitter>/<identifier>.git`. State and PR synchronization copy missing
objects between those repositories. Repositories with the same NIP-34 `d` tag
therefore store the same large blobs and history repeatedly.
That duplication has two costs:
- local installations consume space for every owner and contributor copy;
- the first push to an apparently empty `/prs/` or related owner route uploads
history the service already possesses.
Buzz's Git-on-object-storage implementation demonstrates useful S3 mechanics:
create-only content-addressed packs, verified hydration, a local pack cache,
and publication only after durable writes. Its manifest is repository-scoped,
however, so byte-identical pack files deduplicate globally but independently
packed copies of the same Git objects do not. ngit-grasp instead needs a shared
semantic inventory at the identifier boundary.
## Goals
- Deduplicate objects across all local owner and `/prs/` views that share an
identifier.
- Let receive-pack advertise already stored family history anonymously so a
client does not resend it.
- Preserve the existing URL, authorization, ref, HEAD, purgatory, deletion,
archive, and rollback semantics.
- Keep local storage as the zero-configuration default.
- Make S3-compatible storage opt-in and make a cache miss affect latency, not
correctness.
- Upgrade existing installations automatically, idempotently, and before the
server accepts traffic.
## Non-goals
- Reclaiming unreachable objects or old packs.
- Changing NIP-34 identifiers, repository URLs, or public ref names.
- Making one Git object family span independent ngit-grasp installations.
- Introducing a distributed writer lock or claiming multi-instance S3 writes
in the first implementation.
- Converting archives and holding snapshots to S3 in the first implementation.
Restores import their objects back into the family through the same write
path.
## Model and terminology
A **coordinate** is an owner plus identifier, such as
`30617:<owner>:<identifier>`. A **view** is the small bare repository served at
that coordinate. A view owns only refs, `HEAD`, configuration, hooks if any,
and an alternate link; it does not own the shared object inventory.
A **family** is:
```text
(service storage root, Git object format, NIP-34 identifier)
```
The service storage root is an implicit tenant boundary. Object-format
separation prevents a future SHA-256 repository from being mixed with SHA-1
objects. Owner is intentionally absent: related owner announcements and
contributor submissions for the same identifier share objects.
An identifier collision means the service may reveal that it already has a
reachable object ID for another repository with that identifier. This is an
accepted GRASP storage and delivery trade-off. Authorization still determines
which named refs a client can see or update.
## Local layout
Existing public paths remain stable. Internal state lives below `.grasp`, which
normal repository scans must ignore:
```text
<git-data>/
.grasp/
storage-version
migration/
families/
sha1/
<identifier>.git/ # bare family inventory and internal refs
s3-cache/ # present only for the S3 backend
<owner-npub>/
<identifier>.git/ # thin view: refs + HEAD + alternate
prs/
<submitter-hex>/
<identifier>.git/ # thin view: refs + HEAD + alternate
```
Identifiers are already constrained to one safe filesystem component. Family
path construction must reuse the same validation and must never accept an
unvalidated event tag as a path.
Each view's `objects/info/alternates` names its family's object directory. The
view config sets:
```ini
[core]
alternateRefsPrefixes = refs/grasp/bases/
```
This limits anonymous receive-pack negotiation to current family base tips.
The family can retain additional history without advertising every retained
tip on every push.
## Ref ownership
Views keep all client-visible refs:
- `refs/heads/*` and `refs/tags/*` follow the authorized repository state;
- `refs/nostr/*` follows accepted or purgatory PR state;
- `HEAD` follows the authorized state for that owner.
The family repository uses internal refs only:
- `refs/grasp/bases/<digest>` records tips useful for receive negotiation;
- `refs/grasp/retained/<digest>` is append-only and records every accepted tip
that must remain recoverable.
The suffix is derived from the ref name and object ID rather than user input.
Updating or deleting a view ref may update the base set, but must never delete
a retained ref in this phase.
## Why Git alternates solve the upload problem
Git receive-pack includes tips from alternate repositories as anonymous
`.have` entries. With a view's alternate pointing at the family repository, an
empty `/prs/` view can say “this object graph is already here” without
advertising another owner's `refs/heads/main`.
The service already opts into `uploadpack.allowReachableSHA1InWant` and the
related tip capability. Fetching a known reachable object from the family is
therefore also an accepted behavior. Named-ref visibility and push
authorization remain view-specific.
For local writes, receive-pack writes its quarantine and final objects into the
family object directory while it updates refs in the selected view. Other Git
commands that can create objects, including proactive fetch, copies between
views and archive restore, must use the same family object directory. Read-only
commands can use the view normally because its alternate resolves the
inventory. The one exception is an upload that no signed event names yet,
described next.
## Staging of unsigned uploads
**Added:** 2026-09-29
A push to `refs/nostr/<event-id>` is accepted before its PR event is known.
Until this change its objects went straight into the family, which is never
garbage-collected, so anyone could fill permanent storage without signing
anything. Such an upload is now **staged**: receive-pack writes its objects to
the view's own object directory, and they reach the family only when a signed
event accepts them.
### What earns family storage
A signed event earns family storage at push time. Push authorization already
decides, for each pushed ref, whether a signed event names it:
| Pushed ref | Objects go to |
| ------------------------------------------------------------ | ------------- |
| Branch or tag named by a State, accepted or in purgatory | family |
| `refs/nostr/<id>` whose PR event is accepted or in purgatory | family |
| `refs/nostr/<id>` with no event, or only a placeholder | view staging |
A push that carries any unsigned ref is staged as a whole, because one push is
one pack. While a view holds staged objects, every push to it is staged too:
the view advertises its pending refs, so a client may omit objects that only
staging holds, and receive-pack could not check such a push against the family
alone.
Trade-off: a pack from a signed push is stored whole, including any object in
it that the signed tip does not reach. This is accepted because a signer can
make any object reachable from their own tip. Staging defends against uploads
nobody signed for, not against a signer's choice of content. Purgatory sync
fetches and integrity repair fetch history for signed events and keep writing
to the family.
### Promotion
Promotion copies a tip's history from the view into the family with
`git fetch`, checks that the family alone holds it, and installs the usual
retained and base roots. The check walks from the tip down to existing retained
roots. Those were complete when installed and the family is append-only, so the
cost follows the new history instead of the whole repository. Detecting damage
behind a root remains the integrity pass's job; a damaged root fails the check
and therefore the promotion.
Promotion happens at these boundaries:
- **PR acceptance.** A PR or PR Update event is accepted only after its tip is
promoted. On failure the event is rejected and its placeholder is kept, so
the upload expires normally and the client can send the event again.
- **Signed push into a staged view.** Its signed tips are promoted when
receive-pack finishes, while the push still holds the family lease.
- **State acceptance and purgatory release.** All concrete branch and tag tips
are promoted before ref alignment and acceptance. A State can adopt an
unsigned upload already in the view without another push. Failure rejects
a directly submitted State or leaves a waiting State in purgatory; its
history must become durable before later ref changes can make it disposable.
### Owed history
Rollback after a State deletion depends on history that no current ref names.
The absence of a ref therefore never proves that staged history is disposable.
Before receive-pack runs, the signed tips of a staged push are recorded as
**owed** in the view's registry record. A tip stays owed until its history is
complete in the family alone. If promotion at the end of the push fails, or
the server stops before it runs, the tip is still owed after restart and
maintenance promotes it by object ID, whether or not a ref still names it. An
owed tip whose object is in neither store belongs to a push that never
completed and is dropped. An empty `/prs/` view is not removed while it owes
history.
A view that already holds objects when it is first staged has them moved into
the family first. They predate staging and cannot be told apart from accepted
history.
### Compaction
Staging is reclaimed by Git itself. For one view, maintenance:
1. promotes owed tips, and stops if any remain owed;
2. runs `git repack -a -d -l` and `git prune --expire=now`, which keep the
history live refs need and the family lacks, and drop everything else;
3. removes loose objects the family also holds;
4. removes the registry record if nothing is left, and the view itself if it
is an empty `/prs/` view.
Every live ref is a root, so a pending upload keeps exactly its own history
and an abandoned one disappears once its ref expires.
Compaction must not overlap a push to the same view. Git can report a
successful push and install a ref whose parent a concurrent repack has just
removed; `tests/git_cruft_concurrency.rs` reproduces this. A grace period
cannot prevent it, because the push may start after the repack has chosen
what to keep. Compaction therefore takes the family write lease, which every
push already holds from before receive-pack starts until its refs and roots
are installed. The lease is fair: maintenance queues behind active writers for
at most 250 ms, then gives way and retries with backoff from one second to five
minutes, so it cannot hold up the pushes behind it.
Fetches take no lease. A fetch of a pending ref that expires while it is being
served may fail. That ref named an upload nobody signed for and is being
removed deliberately.
### Registry and maintenance worker
`.grasp/staging/<digest>.json` records each staged view, its owed tips, and
unsigned ref intentions with absolute expiry deadlines. It is written and
fsynced before receive-pack can write an object. One worker
processes maintenance requests, which come from staged pushes, promotions and
ref deletions, including placeholder expiry. At startup it reads the registry,
so interrupted work resumes. Before cleanup, sync and request handling start,
recovery reconstructs missing purgatory placeholders from surviving unsigned
refs and their durable deadlines. Accepted events and signed events already in
purgatory are preserved. Failed pushes with no matching ref create no
placeholder. Legacy records without deadlines receive one grace period that
is persisted before recovery proceeds, so repeated restarts do not extend it.
Unreadable or invalid individual records are logged and skipped without
removing their metadata, refs or objects; healthy records still recover. A
skipped upload with no checkpoint placeholder remains retained for operator
inspection. Failure to enumerate the registry itself still aborts startup.
This closes the window between receive-pack and the periodic purgatory
checkpoint. A view whose remaining staged objects belong to pending refs is
parked until the next request rather than polled.
### Limits
- Staging is not a storage quota. It bounds how long an unsigned upload is
kept, not how much one may upload.
- Archiving a repository stores its view directory as it is, staged objects
included. Restoring the archive imports them into the family.
- Staged history exists in one view only. Other views cannot read it until it
is promoted.
## Durability and the success fence
The invariant is:
> When a client observes a successful push, every Git object needed by the
> accepted ref updates is durable in the configured family backend, or in the
> view's staging for an upload that no signed event names yet.
The local backend satisfies the fence when Git has atomically installed the
objects in the family object directory, or in view staging, and receive-pack
has completed. Accepting a PR event additionally requires its history to be
complete in the family.
The S3 backend cannot release receive-pack's terminal success immediately.
The handler must retain the final protocol status until it has:
1. indexed and verified the received objects;
2. written new immutable packs to S3 using create-only, content-addressed keys;
3. durably published the updated family manifest;
4. installed append-only retained roots and the intended view refs; and
5. fsynced the small local metadata needed to reconstruct the views.
Only then may the terminal success reach the client. A failure before the
fence returns a push error and leaves the previously published family manifest
and view refs authoritative. Staged packs may become harmless orphans; no old
pack is deleted.
## S3 backend
S3 stores immutable data; local disk remains the execution surface for Git:
```text
packs/<object-format>/<sha256(pack-bytes)>
indexes/<object-format>/<sha256(pack-bytes)>
manifests/<object-format>/<sha256(canonical-manifest)>
families/<object-format>/<encoded-identifier>/pointer
```
A canonical manifest contains its schema version, object format, identifier,
complete pack-key set, and parent manifest digest. The mutable family pointer
names the current immutable manifest. Initial implementation permits one
ngit-grasp writer per storage root; conditional pointer writes are still used
to detect an accidental second writer rather than silently losing an update.
Hydration resolves pointer to manifest, verifies every downloaded pack against
its key digest, installs or regenerates its index, and links the pack/index
pair into the local family object directory. Cache entries are immutable,
byte-bounded, and pinned for the lifetime of the Git request. Eviction may make
the next request slower but cannot remove durable data.
S3 lifecycle rules must not expire `packs/`, `indexes/`, `manifests/`, or
family pointers. Garbage collection requires a separate design that accounts
for rollback roots, old manifests, in-flight hydrations, and archives.
## No-GC recovery invariant
Until explicit garbage collection is designed and approved:
> A family's durable inventory contains every Git object ever accepted for
> that family by this service.
Consequences:
- disable automatic Git maintenance and pruning for family repositories;
- create retained roots for every accepted branch, tag, and PR tip;
- unsigned uploads stay outside this inventory until a signed event accepts
them;
- do not use S3 lifecycle deletion on family objects;
- never compact the family by packing only the current visible ref closure
(view staging is compacted this way because it is not part of the
inventory, and only once nothing in it is owed to the family);
- if pack-count compaction becomes necessary, repack the union of every object
in all selected packs, publish the replacement in addition to the old
immutable packs, and leave physical deletion to future GC work.
This intentionally spends storage to preserve rollback choices. Deduplication
still removes the much larger multiplier caused by owner and `/prs/` copies.
## `/prs/` behavior
A missing `/prs/<submitter>/<identifier>.git` route is no longer synthesized
from a truly empty temporary repository. It is synthesized as an empty thin
view whose alternate is the identifier family. Therefore:
- upload-pack still advertises no named refs for a missing route;
- receive-pack may advertise family base tips as anonymous `.have` lines;
- the first contributor push sends only objects the family does not have;
- pushes remain restricted to `refs/nostr/<event-id>`;
- a push whose PR event is not yet known is staged in the view;
- placeholder and expiry cleanup removes view refs or an empty view, never
family objects or retained roots. An empty view that still owes history to
the family is kept.
Mirroring a PR into an owner view becomes a ref update after an object
availability check. It no longer copies the object graph.
## V2 to v3 migration boundary
V3 changes the on-disk meaning of every served Git repository. Owner and
`/prs/` paths become thin views whose objects and write serialization belong to
the identifier family under `.grasp`. Migration happens automatically on the
first v3 launch, but the result is not safe for v2: v2 has no family lock,
retained-root, or shared-object write model. A software rollback therefore
requires restoring the Git and relay-data snapshot taken before v3; changing
only the binary is not supported.
V3 can encounter an object graph which was already incomplete in legacy
storage. Migration copies every Git-readable object and preserves the backup;
it does not manufacture a missing ancestor or infer its provenance. Earlier
manual copies or data migrations, an incomplete source repository, and other
pre-existing storage damage can all produce the same unmarked symptom. The
integrity report therefore describes the evidence (`missing_oids`, broken
links, and `shallow_views`) rather than assigning a cause from a missing object
alone.
One known and narrower source has an unambiguous marker. State Git-data fallback
was introduced by commit `623cae5` on 2026-01-05 with
`git fetch --depth=1`, carried into the rewritten purgatory sync path, and
changed to a full fetch by commit `f25eea8` on 2026-01-12. The first tagged
release was v1.0.0 on 2026-02-26 and already contained the fix, so no tagged v1
or v2 release shipped the shallow-fetch behavior. Only operators who deployed
an untagged source revision from that seven-day interval can have repositories
created by this bug. The affected fallback ran only when a pending State or PR
event named OIDs which had not arrived through an ordinary push.
V3 treats that legacy `shallow` marker as compatibility state rather than a
reason to delete or disable the repository. Migration preserves the marker on
the thin view and retains the legacy backup. After the listener starts, the
ordinary integrity worker requests the complete closure from other accepted
clone servers. On success it removes the marker; the next v3 launch retires the
now-redundant backup. On failure the existing shallow repository and backup
remain, and an `ERROR` identifies the family with a non-zero `shallow_views`
count.
An incomplete graph with `shallow_views=0` is a different case. V3 still tries
every accepted clone source, but all listed servers may share the same missing
history or the complete source may no longer be announced. Such a family stays
unresolved until an operator obtains the missing closure from a known-complete
maintainer clone or bundle. Matching a signed State event is not enough to
prove this closure: State authorizes ref tips, while storage integrity also
checks every object reachable behind those tips.
## Startup migration
Migration runs after configuration validation and before purgatory restoration,
deletion reconciliation, background sync, or accepting HTTP connections. It is
versioned, exclusive, crash-safe, and idempotent.
For each legacy bare repository:
1. Classify the path as an owner view, `/prs/` view, archive/holding data, or
internal data. Only owner and `/prs/` views migrate in this version.
2. Discover the object format and identifier; create the family inventory if
absent with automatic maintenance disabled.
3. Copy every legacy object—including objects not currently reachable from a
visible ref—into a staging family inventory. Copying the entire object
database, not only `rev-list --all`, preserves rollback material.
4. Record every legacy ref tip as an append-only retained root and useful tips
as base roots.
5. Verify with `git fsck`, verify every legacy object ID exists in the family,
and verify the proposed view's refs and `HEAD` exactly match the legacy
repository.
6. Write and fsync a per-repository migration journal entry.
7. Atomically rename the legacy repository to a migration backup, atomically
install the thin view, then mark that journal entry complete.
On restart, the journal determines whether to resume copying, finish a rename,
or restore the legacy directory. Every state transition is safe to repeat.
Backups are retired family by family rather than accumulating until the whole
migration finishes. After the last view of a family is installed, each of its
backups must pass a retirement gate: the family contains every Git-readable
object of the backup, every affected view is a correctly wired thin view whose
refs and `HEAD` match its journal snapshot, and every family pack is indexed
and passes `git verify-pack`. The ordinary family integrity inspection must
also report the family and all of its views healthy. This final check catches
corrupt loose objects, missing history, broken ref targets, invalid alternates,
and shallow views. An unhealthy family keeps its backups while the non-blocking
integrity worker attempts repair after startup. Only a healthy family writes a
durable `retired` journal state, deletes the backup, and removes the journal
before the next family is migrated. Peak migration overhead is therefore
bounded by the family currently in flight except for the small set of families
still awaiting repair.
Backup deletion is itself durable: every directory entry removed while
deleting and pruning a backup is fsynced before its journal is removed. A
power loss therefore leaves either a discoverable `retired` journal that
finishes deletion or no backup entry to rediscover.
Packs without an index are Git-invisible, so the superset proof cannot vouch
for them; they are moved into content-addressed directories beneath
`.grasp/migration/unindexed-packs/` before their backup is deleted. Repeated
migrations preserve different payloads even when their original pack filenames
match, while an identical payload converges on the same quarantine path.
A server-side `shallow` marker is a compatibility boundary, not valid final
family storage. Migration copies it onto the installed thin view so the view
keeps serving exactly what the legacy repository served, and excludes its
backup from retirement until the family holds the complete reachable closure
of the backup's refs. Closure recovery is the ordinary integrity repair: the
family reports truncated parents as missing objects and marked views as
`shallow_views`, repair fetches the closure from accepted clone sources, and
once complete removes the marker. The backup retires on the next launch. An
unrecoverable closure retains both marker and backup and keeps the family
reported as unresolved; it never blocks startup because recovery depends on
remote servers. Client-requested shallow clones and fetches remain supported
throughout.
Retirement failures during an active migration are fail-closed and keep the
backup. Journals from a migration that already committed are handled
leniently on later launches: their ref snapshots are stale once the server
has served traffic, so the gate skips snapshot equality, and an unverifiable
backup (for example one whose migrated view was later deleted by repository
lifecycle) is retained with a warning as operator-managed rollback material.
Installations whose backups were verified and deleted manually simply have
their completed journals removed.
The global `storage-version` advances only after every eligible repository is
complete. A server must fail startup on an unrepairable mismatch rather than
serve a partially converted storage root.
Fresh installations create the current version marker and family layout on
their first launch. Switching from local to S3 is a separate backend migration:
upload and verify all local family inventories first, then change the backend
marker. It must never reinterpret an absent S3 pointer as an empty family when
local objects exist.
## Concurrency
Operations that mutate one family are serialized by a per-family lock. This
covers pushes to different owner or `/prs/` views with the same identifier,
proactive fetches, archive restores, retained/base ref updates, promotion and
compaction of view staging, and S3 manifest publication. Read requests take a
hydrated family lease; they do not hold the writer lock after their pack set
has been pinned.
View lifecycle locks remain responsible for “may this path be removed?” The
family lock is responsible for “is this object inventory and manifest update
atomic?” Neither lock grants authorization.
## Storage and authorization integrity
Storage integrity is defined for an object-format/identifier family, not for
one owner path. The storage pass checks the family pack set and object graph,
then checks every owner and `/prs/` view with the same identifier for the
correct alternate and for refs whose targets are available in the family. A
ref of a staged view whose history is complete in that view's staging is a
pending upload, not a fault. Multiple independent histories in one identifier are valid, and unreachable
objects are not an error: retained delete-state and rollback data intentionally
remain present.
The relay starts one non-blocking pass after migration, database
initialization, and construction of the hardened outbound Git client. Broken
alternate wiring is repaired locally. For missing OIDs, the pass tries clone
URLs from accepted repository announcements, excluding this service and
applying the same SSRF policy, DNS pinning, credentials, process containment,
and missing-OID behavior used by proactive sync. It then runs the same
integrity check again. An unresolved family produces an `ERROR` log containing
bounded counts and OID/diagnostic samples; it does not make availability depend
on remote servers.
Authorization integrity is the second, view-scoped layer. It derives each
owner view's branch, tag, and `HEAD` set from the NIP-01-preferred accepted
State event published by the confirmed maintainer component resolved from
that selected owner coordinate. It derives
`refs/nostr/<event-id>` from accepted PR and PR Update events when the same
confirmed-maintainer overlap as normal event processing selects the view, or
when an exact standard clone URL names that owner and identifier on this
service. Foreign hosts, different coordinates, URL suffixes, and `/prs/` URLs
cannot authorize a standard owner view. GRASP-06 views separately require the
signer and this relay's clone URL to name that exact submitter/identifier
coordinate. Exactly scoped active purgatory entries are recognized so an
in-flight push is not mistaken for corruption.
The pass creates or updates missing/wrong authorized refs once their objects
are available, deletes stale State-governed branch and tag refs, and repairs
`HEAD`. Before giving up on a missing expected object it uses the same accepted
clone URLs and hardened fetch path as storage healing. It does not delete an
unexplained PR or unknown-namespace ref: that ref may be evidence of a past
authorization bypass. Instead it emits a bounded, structured `ERROR` with
`manual_inspection=true`, the view and ref, and actual/expected targets. An
authorized State still in purgatory also defers State mutation and is reported
for inspection rather than racing event promotion.
Both startup passes run in the background after database initialization. By
default they cover every installed identifier family. A temporary
`NGIT_STARTUP_INTEGRITY_IDENTIFIERS` scope can limit both passes to named
families during staged release-candidate validation; an empty scope is required
for the final full sweep. The passes do not extend the offline migration window
or make relay availability depend on a remote Git server. Their stable terminal
log messages are `Git
storage-integrity startup pass completed` and `Git authorization-integrity
startup pass completed`. Because the authorization pass is online, it refreshes
the accepted events immediately before mutation and refuses to overwrite any
ref that changed after its initial snapshot.
This is also the migration repair path. Migration remains a deterministic,
offline conversion that preserves every Git-readable object and the exact
legacy refs. Once those paths are thin family views, the ordinary family pass
can heal pre-existing missing objects. Unindexed legacy packs are preserved
under `.grasp/migration/unindexed-packs/` when their backup is retired; there
is no separate legacy repair subsystem.
Operators can queue an identifier-scoped storage and event-authorization check
in the live process:
```console
ngit-grasp integrity-check --identifier example
ngit-grasp integrity-check --identifier example --repair
```
Without `--repair`, the command reports structural faults, ref differences,
and manual-inspection findings without changing objects, refs, or `HEAD`. With
`--repair`, it also heals storage from accepted clone sources and applies the
same unambiguous State/PR/PR-Update ref fixes as the startup pass. Unexplained
PR refs remain preserved in both modes. The command writes a durable request
beneath `.grasp/integrity-requests/`. The server consumes it while holding its
normal in-process family locks, so a manual repair cannot race an
object-producing request in another view. Completion is reported separately as
`Manual Git storage-integrity request completed` and `Manual Git
authorization-integrity request completed` with the identifier and repair mode.
## Security and privacy trade-offs
- Sharing is restricted to one validated identifier, object format, and
service storage root. It is not a process-global object pool.
- Anonymous `.have` entries disclose object IDs already useful to the family.
The user has explicitly accepted this in exchange for avoiding duplicate
uploads.
- `allowReachableSHA1InWant` can deliver a known object reachable through a
related repository's family inventory. This is also explicitly accepted.
- Clients still cannot enumerate another view's named refs through their own
URL. Upload-pack materializes only the selected view's refs and HEAD.
- S3 credentials and bucket details are operator secrets/configuration and
must not be written into manifests, logs, or Nostr events.
## Rejected alternatives
### Keep full per-owner repositories and run periodic repack
Alternates would still be absent during the first push, so bandwidth remains
duplicated. Cross-repository repack also has no natural safe deletion rule while
rollback recovery is open.
### One global object pool
This maximizes deduplication but turns any known object ID on the service into
a cross-repository reachability surface. Identifier families give the desired
related-repository behavior with a smaller disclosure and failure domain.
### Repository-scoped S3 manifests only
This follows Buzz closely but deduplicates only byte-identical packs. Two packs
containing the same large blob may have different bytes and keys, so the main
ngit-grasp duplication remains.
### Partial clone or LFS
Both require client/repository participation and change repository semantics.
The service must deduplicate ordinary Git repositories transparently.
### Garbage-collect unreachable objects during migration
This would reduce the migration footprint but discard exactly the history used
for rollback after delete-state events. It is deferred until rollback retention
and physical deletion have an explicit policy.
## Delivery sequence
The implementation is intentionally reviewable as a local-first stack:
1. this decision and its invariants;
2. family paths, local inventory, thin-view construction, and Git-level tests;
3. standard and `/prs/` handler integration plus ref-only synchronization;
4. launch-time legacy migration, restart recovery, and operator documentation;
5. one optional S3 layer containing manifests, verified hydration, cache,
local-family adoption, configuration, and the durability fence.
Steps 3 and 4 ship together as the first deployable local-storage milestone.
It can be operated and observed before the S3 layer is considered. Every layer
retains the public repository layout, and S3 never changes the default from
local storage.