mirror of
https://relay.ngit.dev/npub15qydau2hjma6ngxkl2cyar74wzyjshvl65za5k5rl69264ar2exs5cyejr/ngit-grasp.git
synced 2026-10-05 15:08:24 +00:00
DanConwayDev
fd7c6bb851
The v3.0.1 authorization fix is intentionally small. Follow it with a separate structural pass so the implementation and documentation express the present-tense maintainer model directly instead of leaving the security behavior hidden behind owner-oriented names and repeated raw-tag interpretation.
Parse indexed roles once into a current-only snapshot of active maintainers, active lead targets, and announcement-author activity. Preserve detailed lead-resolution failures internally while policy callers continue to fail closed, distinguish selected authorization coordinates from physical owner views, and name broad announcement admission as discovery rather than authority.
Keep history relevant only while deriving current activity and retain active leads only for selected-coordinate resolution. Preserve the v3.0 public API through compatibility projections and deprecated aliases; this commit is not intended to change the authorization outcome established by 650cfb57.
Refresh architecture, inline authorization, storage, sync, and audit documentation. Correct the audit fixture description that claimed a listed maintainer authorized with no reciprocal announcement even though its setup already published one.
Validated with cargo test --lib (903 tests), cargo test --test state_authorization (53 tests), cargo test -p grasp-audit --lib (54 passed, 5 ignored), cargo test --test push_authorization (56 tests), and cargo clippy --tests -- -D warnings.
173 lines
8.6 KiB
Markdown
173 lines
8.6 KiB
Markdown
# Upgrade from v2 to v3 Git family storage
|
|
|
|
Version 3 replaces complete per-owner and `/prs/` bare repositories with thin
|
|
views backed by one local object family per repository identifier. The upgrade
|
|
requires no new configuration and runs automatically before the server accepts
|
|
traffic.
|
|
|
|
**Plan a maintenance window.** The relay does not serve HTTP or WebSocket
|
|
traffic until the storage migration finishes. The largest GRASP service
|
|
migrated so far spent 50 minutes 59 seconds in the storage migration, from
|
|
21:44:16 UTC until completion was logged at 22:35:14 UTC. The HTTP server began
|
|
listening about two seconds later, so total relay downtime was almost exactly
|
|
51 minutes. That run migrated 2,294 repository views into 2,014 identifier
|
|
families, retired 2,291 backups, and retained three for automatic integrity
|
|
repair. Treat this as a production scale reference rather than a fixed
|
|
estimate: migration time depends on the dataset and storage performance.
|
|
|
|
This is a one-way Git-data migration. A v2 binary does not understand the
|
|
family locking and write model and must not be run against a migrated
|
|
`NGIT_GIT_DATA_PATH`. Rolling back means restoring a complete pre-upgrade
|
|
snapshot, not only installing the older binary.
|
|
|
|
## Before upgrading
|
|
|
|
1. Schedule and announce a maintenance window during which the relay will not
|
|
accept HTTP or WebSocket traffic.
|
|
2. Stop writes to the relay and take a filesystem snapshot of both the Git and
|
|
relay data directories. Keep it for the rollback window.
|
|
3. Check free space on `NGIT_GIT_DATA_PATH`. Plan for the current footprint
|
|
plus roughly one extra copy of the largest identifier family: every owner
|
|
and `/prs/` repository sharing one identifier. The migration retires each
|
|
healthy family's legacy repositories before processing the next family.
|
|
4. Keep Git and relay data on durable storage. Do not enable Git garbage
|
|
collection for this rollout; retained objects preserve recovery from
|
|
deletion-state mistakes.
|
|
|
|
## Upgrade
|
|
|
|
Deploy v3 with the existing configuration and start the relay. It remains
|
|
unavailable while the launch migration runs. The migration is journaled,
|
|
crash-safe, and safe to resume by restarting the same v3 release. It:
|
|
|
|
1. Builds and verifies one identifier family at a time, including unreachable
|
|
objects retained for rollback.
|
|
2. Replaces each owner and `/prs/` repository with a thin view that preserves
|
|
its refs, `HEAD`, configuration, and observable Git behavior.
|
|
3. Deletes a legacy backup only after its object superset, view wiring, packs,
|
|
and ordinary family integrity report all verify.
|
|
4. Keeps any unhealthy family's backup while the running server automatically
|
|
attempts repair from accepted clone URLs.
|
|
|
|
Progress is stored below `.grasp/migration/`; the completed layout is marked by
|
|
`.grasp/storage-version`. Do not edit these files while the service is running.
|
|
A later v3 restart sees that completed marker and does not repeat the offline
|
|
v2 conversion; the production-scale 51-minute cost applies to the first
|
|
v2-to-v3 migration, not every v3 deployment.
|
|
|
|
Confirm that the service reaches its normal listening state, then follow the
|
|
logs until both terminal integrity summaries appear:
|
|
|
|
```text
|
|
Git storage-integrity startup pass completed
|
|
Git authorization-integrity startup pass completed
|
|
```
|
|
|
|
For a release-candidate deployment, operators may temporarily limit both
|
|
startup passes to selected identifiers while validating runtime and log output:
|
|
|
|
```bash
|
|
NGIT_STARTUP_INTEGRITY_IDENTIFIERS=repo-one,repo-two
|
|
```
|
|
|
|
The equivalent NixOS option is
|
|
`startupIntegrityIdentifiers = [ "repo-one" "repo-two" ];`. A scoped summary
|
|
establishes integrity only for those names; it is not the v3 security sweep.
|
|
Remove the scope and observe a successful all-family pair of terminal summaries
|
|
before declaring the upgrade complete. The relay stays online during both
|
|
scoped and full passes.
|
|
|
|
Any service that enabled GRASP-06 on a tagged release through v2.1.2 should
|
|
treat its hosted Git repositories as potentially containing unauthorized
|
|
objects or refs until the full v3 integrity pass has completed and every
|
|
reported exception has been resolved. Conforming Nostr Git clients that resolve
|
|
refs from valid signed State, PR, and PR Update events are not expected to
|
|
accept this unauthorized data; direct Git consumers do not have that
|
|
protection. Passing a selected scope is useful validation evidence, but it does
|
|
not clear repositories that were not named.
|
|
|
|
These passes are non-blocking: the relay is online while they inspect and heal
|
|
the migrated views. In the storage summary, an `unresolved` or `failed` count
|
|
above zero has a corresponding `ERROR` naming the identifier. In the
|
|
authorization summary, a `manual_inspection` or `failed` count above zero has
|
|
a corresponding `ERROR` naming the exact view, ref, actual target, expected
|
|
target, and reason. The server has already attempted safe automatic repair.
|
|
It deliberately preserves unexplained `refs/nostr/*` and unknown-namespace
|
|
refs as evidence instead of guessing that deletion is safe.
|
|
|
|
The authorization pass treats the accepted event database as authoritative:
|
|
|
|
- the latest State event from the confirmed maintainer component resolved for
|
|
the selected owner coordinate defines
|
|
all and only `refs/heads/*`, `refs/tags/*`, and `HEAD`;
|
|
- accepted PR and PR Update events define `refs/nostr/<event-id>` in an owner
|
|
view when either the confirmed-maintainer overlap selects that view or an
|
|
exact standard clone URL names that owner and identifier on this service;
|
|
foreign hosts, different coordinates, suffixes, and `/prs/` URLs do not
|
|
authorize the owner view;
|
|
- a GRASP-06 contributor view additionally requires the event signer, clone
|
|
URL, and repository identifier to match that exact `/prs/` coordinate;
|
|
- precisely scoped active purgatory entries are tolerated as in-flight state.
|
|
Legacy unscoped placeholders are preserved but reported because they cannot
|
|
prove which owner and identifier originally received the push.
|
|
|
|
To repeat both checks for one family without restarting the relay, queue a
|
|
check-only request first and inspect the two manual completion summaries:
|
|
|
|
```console
|
|
ngit-grasp integrity-check --identifier repo-one
|
|
ngit-grasp integrity-check --identifier repo-one --repair
|
|
```
|
|
|
|
Only the second command applies safe fixes. Re-run the check-only command after
|
|
repair; `repair_needed`, `manual_inspection`, and `failed` must all be zero for
|
|
that identifier to be considered clean.
|
|
|
|
A retained backup protects an unhealthy family's legacy data. A non-zero
|
|
`shallow_views` count identifies the marked depth-one fallback used only by a
|
|
seven-day untagged development interval; no tagged v1 or v2 release shipped
|
|
that behavior. Do not attribute an unmarked `missing_oids` result to that bug.
|
|
It means the incomplete graph predated the v3 conversion, but its exact origin
|
|
cannot be inferred from the missing object alone. Earlier manual copies or data
|
|
migrations, an incomplete source, and other legacy storage damage can have the
|
|
same result.
|
|
|
|
### Recover an unresolved object graph
|
|
|
|
Automatic repair can only use objects available from the accepted clone URLs.
|
|
If all announced servers inherited the same gap, ask a maintainer with a
|
|
known-complete local clone to run the following, substituting the OIDs from the
|
|
relay's `ERROR` log:
|
|
|
|
```bash
|
|
git fsck --full
|
|
git cat-file -e '<missing-oid>^{commit}'
|
|
git cat-file -e '<child-oid>^{commit}'
|
|
git cat-file -p '<child-oid>'
|
|
git update-ref refs/integrity-repair/<child-short-id> '<child-oid>'
|
|
git bundle create repository-integrity-repair.bundle \
|
|
refs/integrity-repair/<child-short-id>
|
|
git update-ref -d refs/integrity-repair/<child-short-id>
|
|
git bundle verify repository-integrity-repair.bundle
|
|
sha256sum repository-integrity-repair.bundle
|
|
```
|
|
|
|
The `fsck` and both `cat-file` checks must succeed. Transfer the bundle and its
|
|
SHA-256 digest through agreed channels; an ordinary push or `ngit sync` may be
|
|
a no-op when the relay already advertises the authorized tip. Import the bundle
|
|
under a temporary ref, remove that ref after the objects are present, queue the
|
|
identifier with `--repair`, and finish with a check-only request. Do not delete
|
|
the retained migration backup until storage reports `unresolved=0` and
|
|
`failed=0`.
|
|
|
|
## Roll back
|
|
|
|
Stop v3 and restore the pre-upgrade Git and relay-data snapshot together before
|
|
starting v2. Do not point v2 at thin family views, even if ordinary clones
|
|
appear to work: v2 does not coordinate writes through the shared family or
|
|
preserve its durability invariants.
|
|
|
|
The storage model, legacy incomplete-history repair, and detailed retirement
|
|
guarantees are described in
|
|
[Identifier-family Git object storage](../explanation/git-family-object-storage.md#v2-to-v3-migration-boundary).
|