diff --git a/CHANGELOG.md b/CHANGELOG.md index f92d4d6..bb65dca 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -170,14 +170,17 @@ Expect a bit of downtime as a git data migraiton is performed on startup. The la a warning, backup-directory removals are fsynced before their journals are removed, and completed installations whose backups were already removed manually start unchanged. -- Reconcile server-side shallow repositories left by untagged development - builds between 2026-01-05 (`623cae5`) and 2026-01-12 (`f25eea8`). No tagged - v1 or v2 release shipped the depth-one fallback fetch. The affected - non-happy path fetched missing state or PR data from another listed Git - server when the objects had not arrived by push. V3 preserves existing - shallow clone behavior and the legacy backup, automatically requests the - complete closure from accepted clone sources, and logs any family that - remains shallow without making it less available. +- Detect and reconcile incomplete object graphs inherited from legacy storage. + The v3 migration preserves the legacy repository and its backup; the online + integrity worker then requests missing objects from every accepted clone + source and logs any family that remains incomplete without making it less + available. A server-side + `shallow` marker specifically identifies repositories created by the + depth-one fallback in untagged development builds between 2026-01-05 + (`623cae5`) and 2026-01-12 (`f25eea8`); no tagged v1 or v2 release shipped + that behavior. Missing ancestors without a `shallow` marker are not + attributed to that bug and can require a complete maintainer clone or bundle + when every announced server inherited the same incomplete history. - Scope bare log levels to ngit-grasp while keeping dependencies at warnings; explicit tracing filter expressions remain unchanged. - Keep per-event discovery and validation details at debug, retain aggregate diff --git a/docs/explanation/git-family-object-storage.md b/docs/explanation/git-family-object-storage.md index 94166c5..fda4eb6 100644 --- a/docs/explanation/git-family-object-storage.md +++ b/docs/explanation/git-family-object-storage.md @@ -259,31 +259,41 @@ retained-root, or shared-object write model. A software rollback therefore requires restoring the Git and relay-data snapshot taken before v3; changing only the binary is not supported. -One pre-release development interval can leave server-side shallow -repositories for v3 to reconcile. State Git-data fallback was introduced by -commit `623cae5` on 2026-01-05 with `git fetch --depth=1`, carried into the -rewritten purgatory sync path, and changed to a full fetch by commit `f25eea8` -on 2026-01-12. The first tagged release was v1.0.0 on 2026-02-26 and already -contained the fix, so no tagged v1 or v2 release shipped the shallow-fetch -behavior. Only operators who deployed an untagged source revision from that -seven-day interval can have repositories created by this bug. +V3 can encounter an object graph which was already incomplete in legacy +storage. Migration copies every Git-readable object and preserves the backup; +it does not manufacture a missing ancestor or infer its provenance. Earlier +manual copies or data migrations, an incomplete source repository, and other +pre-existing storage damage can all produce the same unmarked symptom. The +integrity report therefore describes the evidence (`missing_oids`, broken +links, and `shallow_views`) rather than assigning a cause from a missing object +alone. -The affected path was the non-happy-path Git-data fallback. It ran only when a -pending state or PR event named OIDs which had not arrived through an ordinary -push, then fetched those OIDs into an existing local repository from another -announcement or PR clone URL. A repository whose required data arrived through -the normal push path did not invoke this fallback. +One known and narrower source has an unambiguous marker. State Git-data fallback +was introduced by commit `623cae5` on 2026-01-05 with +`git fetch --depth=1`, carried into the rewritten purgatory sync path, and +changed to a full fetch by commit `f25eea8` on 2026-01-12. The first tagged +release was v1.0.0 on 2026-02-26 and already contained the fix, so no tagged v1 +or v2 release shipped the shallow-fetch behavior. Only operators who deployed +an untagged source revision from that seven-day interval can have repositories +created by this bug. The affected fallback ran only when a pending State or PR +event named OIDs which had not arrived through an ordinary push. -V3 treats a legacy `shallow` marker as compatibility state rather than a reason -to delete or disable the repository. Migration preserves the marker on the thin -view so its shallow clones and current tree continue to work, and retains the -legacy backup. After the listener starts, the ordinary integrity worker -requests the complete closure from other accepted clone servers. On success it -removes the marker; the next v3 launch retires the now-redundant backup. On -failure the existing shallow clone and current tree remain available, the -backup remains, and an `ERROR` log identifies the family with a non-zero -`shallow_views` count. The operator does not need to take a separate action for -the v3 upgrade. +V3 treats that legacy `shallow` marker as compatibility state rather than a +reason to delete or disable the repository. Migration preserves the marker on +the thin view and retains the legacy backup. After the listener starts, the +ordinary integrity worker requests the complete closure from other accepted +clone servers. On success it removes the marker; the next v3 launch retires the +now-redundant backup. On failure the existing shallow repository and backup +remain, and an `ERROR` identifies the family with a non-zero `shallow_views` +count. + +An incomplete graph with `shallow_views=0` is a different case. V3 still tries +every accepted clone source, but all listed servers may share the same missing +history or the complete source may no longer be announced. Such a family stays +unresolved until an operator obtains the missing closure from a known-complete +maintainer clone or bundle. Matching a signed State event is not enough to +prove this closure: State authorizes ref tips, while storage integrity also +checks every object reachable behind those tips. ## Startup migration diff --git a/docs/how-to/upgrade-git-family-storage.md b/docs/how-to/upgrade-git-family-storage.md index 15ac253..445a5b3 100644 --- a/docs/how-to/upgrade-git-family-storage.md +++ b/docs/how-to/upgrade-git-family-storage.md @@ -122,9 +122,42 @@ Only the second command applies safe fixes. Re-run the check-only command after repair; `repair_needed`, `manual_inspection`, and `failed` must all be zero for that identifier to be considered clean. -A retained backup protects an unhealthy family's legacy data, and a legacy -shallow view continues serving at its pre-upgrade level rather than being made -less usable. +A retained backup protects an unhealthy family's legacy data. A non-zero +`shallow_views` count identifies the marked depth-one fallback used only by a +seven-day untagged development interval; no tagged v1 or v2 release shipped +that behavior. Do not attribute an unmarked `missing_oids` result to that bug. +It means the incomplete graph predated the v3 conversion, but its exact origin +cannot be inferred from the missing object alone. Earlier manual copies or data +migrations, an incomplete source, and other legacy storage damage can have the +same result. + +### Recover an unresolved object graph + +Automatic repair can only use objects available from the accepted clone URLs. +If all announced servers inherited the same gap, ask a maintainer with a +known-complete local clone to run the following, substituting the OIDs from the +relay's `ERROR` log: + +```bash +git fsck --full +git cat-file -e '^{commit}' +git cat-file -e '^{commit}' +git cat-file -p '' +git update-ref refs/integrity-repair/ '' +git bundle create repository-integrity-repair.bundle \ + refs/integrity-repair/ +git update-ref -d refs/integrity-repair/ +git bundle verify repository-integrity-repair.bundle +sha256sum repository-integrity-repair.bundle +``` + +The `fsck` and both `cat-file` checks must succeed. Transfer the bundle and its +SHA-256 digest through agreed channels; an ordinary push or `ngit sync` may be +a no-op when the relay already advertises the authorized tip. Import the bundle +under a temporary ref, remove that ref after the objects are present, queue the +identifier with `--repair`, and finish with a check-only request. Do not delete +the retained migration backup until storage reports `unresolved=0` and +`failed=0`. ## Roll back @@ -133,6 +166,6 @@ starting v2. Do not point v2 at thin family views, even if ordinary clones appear to work: v2 does not coordinate writes through the shared family or preserve its durability invariants. -The storage model, automatic shallow-history compatibility repair, and detailed -retirement guarantees are described in +The storage model, legacy incomplete-history repair, and detailed retirement +guarantees are described in [Identifier-family Git object storage](../explanation/git-family-object-storage.md#v2-to-v3-migration-boundary). diff --git a/tests/common/git_server.rs b/tests/common/git_server.rs index f4a5e50..edc986e 100644 --- a/tests/common/git_server.rs +++ b/tests/common/git_server.rs @@ -43,8 +43,8 @@ //! - **SimpleGitServer**: Fast, lightweight, good for basic `git fetch` without depth limits //! - **SmartGitServer**: Full protocol support, required for `--depth=1` shallow fetches //! -//! The purgatory sync system uses `git fetch --depth=1`, so tests involving purgatory -//! sync should use `SmartGitServer`. +//! Tests which explicitly exercise shallow client fetches should use +//! `SmartGitServer`; ordinary purgatory sync performs full fetches. use grasp_audit::git_command; use std::net::SocketAddr; @@ -525,7 +525,7 @@ mod tests { /// - Shallow fetches (`git fetch --depth=1`) /// - Full protocol negotiation /// -/// This is required for testing purgatory sync, which uses `git fetch --depth=1`. +/// This is required for tests which explicitly exercise shallow client fetches. pub struct SmartGitServer { /// Shutdown signal sender shutdown_tx: Option>, diff --git a/tests/purgatory_sync.rs b/tests/purgatory_sync.rs index eb13b9a..28b8200 100644 --- a/tests/purgatory_sync.rs +++ b/tests/purgatory_sync.rs @@ -940,7 +940,7 @@ async fn test_pr_event_clone_tag_sync_with_partial_oid_aggregation_from_multiple let mock_relay = MockRelay::start().await; // 3. git_server - SmartGitServer with PR commit only - // Using SmartGitServer because purgatory sync uses `git fetch --depth=1` + // Using SmartGitServer because this fixture exercises smart HTTP fetches // which requires the Git Smart HTTP protocol (not dumb HTTP) let git_server = SmartGitServer::start(repo_b.path()).await;