mirror of
https://relay.ngit.dev/npub15qydau2hjma6ngxkl2cyar74wzyjshvl65za5k5rl69264ar2exs5cyejr/ngit-grasp.git
synced 2026-10-05 15:08:24 +00:00
docs(storage): distinguish legacy incomplete histories
Motivation: Production v3 rehearsal found unmarked missing ancestors that the previous shallow-fetch wording could misattribute to a narrow pre-release bug. Approach: Describe incomplete legacy object graphs without inferring provenance, retain the shallow marker as a specific historical case, and document a minimal maintainer-bundle recovery procedure. Remove stale test comments claiming current purgatory fetches are shallow. Correctness: Current fetches have no depth limit, migration logged the affected graphs as pre-existing, and an unmarked missing parent is not evidence of the marked January fallback behavior. Excluded scope: No integrity policy, object import automation, migration behavior, or production data is changed. Validation: Ran git diff --check and reviewed the production sweep evidence; tests were intentionally skipped because this commit changes documentation and comments only.
This commit is contained in:
+11
-8
@@ -170,14 +170,17 @@ Expect a bit of downtime as a git data migraiton is performed on startup. The la
|
|||||||
a warning, backup-directory removals are fsynced before their journals are
|
a warning, backup-directory removals are fsynced before their journals are
|
||||||
removed, and completed installations whose backups were already removed
|
removed, and completed installations whose backups were already removed
|
||||||
manually start unchanged.
|
manually start unchanged.
|
||||||
- Reconcile server-side shallow repositories left by untagged development
|
- Detect and reconcile incomplete object graphs inherited from legacy storage.
|
||||||
builds between 2026-01-05 (`623cae5`) and 2026-01-12 (`f25eea8`). No tagged
|
The v3 migration preserves the legacy repository and its backup; the online
|
||||||
v1 or v2 release shipped the depth-one fallback fetch. The affected
|
integrity worker then requests missing objects from every accepted clone
|
||||||
non-happy path fetched missing state or PR data from another listed Git
|
source and logs any family that remains incomplete without making it less
|
||||||
server when the objects had not arrived by push. V3 preserves existing
|
available. A server-side
|
||||||
shallow clone behavior and the legacy backup, automatically requests the
|
`shallow` marker specifically identifies repositories created by the
|
||||||
complete closure from accepted clone sources, and logs any family that
|
depth-one fallback in untagged development builds between 2026-01-05
|
||||||
remains shallow without making it less available.
|
(`623cae5`) and 2026-01-12 (`f25eea8`); no tagged v1 or v2 release shipped
|
||||||
|
that behavior. Missing ancestors without a `shallow` marker are not
|
||||||
|
attributed to that bug and can require a complete maintainer clone or bundle
|
||||||
|
when every announced server inherited the same incomplete history.
|
||||||
- Scope bare log levels to ngit-grasp while keeping dependencies at warnings;
|
- Scope bare log levels to ngit-grasp while keeping dependencies at warnings;
|
||||||
explicit tracing filter expressions remain unchanged.
|
explicit tracing filter expressions remain unchanged.
|
||||||
- Keep per-event discovery and validation details at debug, retain aggregate
|
- Keep per-event discovery and validation details at debug, retain aggregate
|
||||||
|
|||||||
@@ -259,31 +259,41 @@ retained-root, or shared-object write model. A software rollback therefore
|
|||||||
requires restoring the Git and relay-data snapshot taken before v3; changing
|
requires restoring the Git and relay-data snapshot taken before v3; changing
|
||||||
only the binary is not supported.
|
only the binary is not supported.
|
||||||
|
|
||||||
One pre-release development interval can leave server-side shallow
|
V3 can encounter an object graph which was already incomplete in legacy
|
||||||
repositories for v3 to reconcile. State Git-data fallback was introduced by
|
storage. Migration copies every Git-readable object and preserves the backup;
|
||||||
commit `623cae5` on 2026-01-05 with `git fetch --depth=1`, carried into the
|
it does not manufacture a missing ancestor or infer its provenance. Earlier
|
||||||
rewritten purgatory sync path, and changed to a full fetch by commit `f25eea8`
|
manual copies or data migrations, an incomplete source repository, and other
|
||||||
on 2026-01-12. The first tagged release was v1.0.0 on 2026-02-26 and already
|
pre-existing storage damage can all produce the same unmarked symptom. The
|
||||||
contained the fix, so no tagged v1 or v2 release shipped the shallow-fetch
|
integrity report therefore describes the evidence (`missing_oids`, broken
|
||||||
behavior. Only operators who deployed an untagged source revision from that
|
links, and `shallow_views`) rather than assigning a cause from a missing object
|
||||||
seven-day interval can have repositories created by this bug.
|
alone.
|
||||||
|
|
||||||
The affected path was the non-happy-path Git-data fallback. It ran only when a
|
One known and narrower source has an unambiguous marker. State Git-data fallback
|
||||||
pending state or PR event named OIDs which had not arrived through an ordinary
|
was introduced by commit `623cae5` on 2026-01-05 with
|
||||||
push, then fetched those OIDs into an existing local repository from another
|
`git fetch --depth=1`, carried into the rewritten purgatory sync path, and
|
||||||
announcement or PR clone URL. A repository whose required data arrived through
|
changed to a full fetch by commit `f25eea8` on 2026-01-12. The first tagged
|
||||||
the normal push path did not invoke this fallback.
|
release was v1.0.0 on 2026-02-26 and already contained the fix, so no tagged v1
|
||||||
|
or v2 release shipped the shallow-fetch behavior. Only operators who deployed
|
||||||
|
an untagged source revision from that seven-day interval can have repositories
|
||||||
|
created by this bug. The affected fallback ran only when a pending State or PR
|
||||||
|
event named OIDs which had not arrived through an ordinary push.
|
||||||
|
|
||||||
V3 treats a legacy `shallow` marker as compatibility state rather than a reason
|
V3 treats that legacy `shallow` marker as compatibility state rather than a
|
||||||
to delete or disable the repository. Migration preserves the marker on the thin
|
reason to delete or disable the repository. Migration preserves the marker on
|
||||||
view so its shallow clones and current tree continue to work, and retains the
|
the thin view and retains the legacy backup. After the listener starts, the
|
||||||
legacy backup. After the listener starts, the ordinary integrity worker
|
ordinary integrity worker requests the complete closure from other accepted
|
||||||
requests the complete closure from other accepted clone servers. On success it
|
clone servers. On success it removes the marker; the next v3 launch retires the
|
||||||
removes the marker; the next v3 launch retires the now-redundant backup. On
|
now-redundant backup. On failure the existing shallow repository and backup
|
||||||
failure the existing shallow clone and current tree remain available, the
|
remain, and an `ERROR` identifies the family with a non-zero `shallow_views`
|
||||||
backup remains, and an `ERROR` log identifies the family with a non-zero
|
count.
|
||||||
`shallow_views` count. The operator does not need to take a separate action for
|
|
||||||
the v3 upgrade.
|
An incomplete graph with `shallow_views=0` is a different case. V3 still tries
|
||||||
|
every accepted clone source, but all listed servers may share the same missing
|
||||||
|
history or the complete source may no longer be announced. Such a family stays
|
||||||
|
unresolved until an operator obtains the missing closure from a known-complete
|
||||||
|
maintainer clone or bundle. Matching a signed State event is not enough to
|
||||||
|
prove this closure: State authorizes ref tips, while storage integrity also
|
||||||
|
checks every object reachable behind those tips.
|
||||||
|
|
||||||
## Startup migration
|
## Startup migration
|
||||||
|
|
||||||
|
|||||||
@@ -122,9 +122,42 @@ Only the second command applies safe fixes. Re-run the check-only command after
|
|||||||
repair; `repair_needed`, `manual_inspection`, and `failed` must all be zero for
|
repair; `repair_needed`, `manual_inspection`, and `failed` must all be zero for
|
||||||
that identifier to be considered clean.
|
that identifier to be considered clean.
|
||||||
|
|
||||||
A retained backup protects an unhealthy family's legacy data, and a legacy
|
A retained backup protects an unhealthy family's legacy data. A non-zero
|
||||||
shallow view continues serving at its pre-upgrade level rather than being made
|
`shallow_views` count identifies the marked depth-one fallback used only by a
|
||||||
less usable.
|
seven-day untagged development interval; no tagged v1 or v2 release shipped
|
||||||
|
that behavior. Do not attribute an unmarked `missing_oids` result to that bug.
|
||||||
|
It means the incomplete graph predated the v3 conversion, but its exact origin
|
||||||
|
cannot be inferred from the missing object alone. Earlier manual copies or data
|
||||||
|
migrations, an incomplete source, and other legacy storage damage can have the
|
||||||
|
same result.
|
||||||
|
|
||||||
|
### Recover an unresolved object graph
|
||||||
|
|
||||||
|
Automatic repair can only use objects available from the accepted clone URLs.
|
||||||
|
If all announced servers inherited the same gap, ask a maintainer with a
|
||||||
|
known-complete local clone to run the following, substituting the OIDs from the
|
||||||
|
relay's `ERROR` log:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git fsck --full
|
||||||
|
git cat-file -e '<missing-oid>^{commit}'
|
||||||
|
git cat-file -e '<child-oid>^{commit}'
|
||||||
|
git cat-file -p '<child-oid>'
|
||||||
|
git update-ref refs/integrity-repair/<child-short-id> '<child-oid>'
|
||||||
|
git bundle create repository-integrity-repair.bundle \
|
||||||
|
refs/integrity-repair/<child-short-id>
|
||||||
|
git update-ref -d refs/integrity-repair/<child-short-id>
|
||||||
|
git bundle verify repository-integrity-repair.bundle
|
||||||
|
sha256sum repository-integrity-repair.bundle
|
||||||
|
```
|
||||||
|
|
||||||
|
The `fsck` and both `cat-file` checks must succeed. Transfer the bundle and its
|
||||||
|
SHA-256 digest through agreed channels; an ordinary push or `ngit sync` may be
|
||||||
|
a no-op when the relay already advertises the authorized tip. Import the bundle
|
||||||
|
under a temporary ref, remove that ref after the objects are present, queue the
|
||||||
|
identifier with `--repair`, and finish with a check-only request. Do not delete
|
||||||
|
the retained migration backup until storage reports `unresolved=0` and
|
||||||
|
`failed=0`.
|
||||||
|
|
||||||
## Roll back
|
## Roll back
|
||||||
|
|
||||||
@@ -133,6 +166,6 @@ starting v2. Do not point v2 at thin family views, even if ordinary clones
|
|||||||
appear to work: v2 does not coordinate writes through the shared family or
|
appear to work: v2 does not coordinate writes through the shared family or
|
||||||
preserve its durability invariants.
|
preserve its durability invariants.
|
||||||
|
|
||||||
The storage model, automatic shallow-history compatibility repair, and detailed
|
The storage model, legacy incomplete-history repair, and detailed retirement
|
||||||
retirement guarantees are described in
|
guarantees are described in
|
||||||
[Identifier-family Git object storage](../explanation/git-family-object-storage.md#v2-to-v3-migration-boundary).
|
[Identifier-family Git object storage](../explanation/git-family-object-storage.md#v2-to-v3-migration-boundary).
|
||||||
|
|||||||
@@ -43,8 +43,8 @@
|
|||||||
//! - **SimpleGitServer**: Fast, lightweight, good for basic `git fetch` without depth limits
|
//! - **SimpleGitServer**: Fast, lightweight, good for basic `git fetch` without depth limits
|
||||||
//! - **SmartGitServer**: Full protocol support, required for `--depth=1` shallow fetches
|
//! - **SmartGitServer**: Full protocol support, required for `--depth=1` shallow fetches
|
||||||
//!
|
//!
|
||||||
//! The purgatory sync system uses `git fetch --depth=1`, so tests involving purgatory
|
//! Tests which explicitly exercise shallow client fetches should use
|
||||||
//! sync should use `SmartGitServer`.
|
//! `SmartGitServer`; ordinary purgatory sync performs full fetches.
|
||||||
|
|
||||||
use grasp_audit::git_command;
|
use grasp_audit::git_command;
|
||||||
use std::net::SocketAddr;
|
use std::net::SocketAddr;
|
||||||
@@ -525,7 +525,7 @@ mod tests {
|
|||||||
/// - Shallow fetches (`git fetch --depth=1`)
|
/// - Shallow fetches (`git fetch --depth=1`)
|
||||||
/// - Full protocol negotiation
|
/// - Full protocol negotiation
|
||||||
///
|
///
|
||||||
/// This is required for testing purgatory sync, which uses `git fetch --depth=1`.
|
/// This is required for tests which explicitly exercise shallow client fetches.
|
||||||
pub struct SmartGitServer {
|
pub struct SmartGitServer {
|
||||||
/// Shutdown signal sender
|
/// Shutdown signal sender
|
||||||
shutdown_tx: Option<oneshot::Sender<()>>,
|
shutdown_tx: Option<oneshot::Sender<()>>,
|
||||||
|
|||||||
@@ -940,7 +940,7 @@ async fn test_pr_event_clone_tag_sync_with_partial_oid_aggregation_from_multiple
|
|||||||
let mock_relay = MockRelay::start().await;
|
let mock_relay = MockRelay::start().await;
|
||||||
|
|
||||||
// 3. git_server - SmartGitServer with PR commit only
|
// 3. git_server - SmartGitServer with PR commit only
|
||||||
// Using SmartGitServer because purgatory sync uses `git fetch --depth=1`
|
// Using SmartGitServer because this fixture exercises smart HTTP fetches
|
||||||
// which requires the Git Smart HTTP protocol (not dumb HTTP)
|
// which requires the Git Smart HTTP protocol (not dumb HTTP)
|
||||||
let git_server = SmartGitServer::start(repo_b.path()).await;
|
let git_server = SmartGitServer::start(repo_b.path()).await;
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user