mirror of
https://relay.ngit.dev/npub15qydau2hjma6ngxkl2cyar74wzyjshvl65za5k5rl69264ar2exs5cyejr/ngit-grasp.git
synced 2026-10-05 15:08:24 +00:00
DanConwayDev
b3ca015968
Production logs after deploying a8964bb to gitnostr.com showed ~41
incomplete negentropy retries and 20 batches completing with partial
results within six minutes, some batches missing hundreds of events.
Negentropy reconciliation identifies event IDs missing locally, but a
relay's exact-ID response can return only a subset (or nothing on the
retry). Batches without repository/root-event metadata - the generic
Layer 1 announcements batch - cannot build a semantic REQ+EOSE
fallback, so handle_eose finalized them "with partial results" and
dropped the missing IDs entirely. Nothing retried them until the next
daily sync up to 25 hours later, leaving repository announcements and
their dependencies absent indefinitely.
Missing IDs from a batch that finalizes incomplete are now registered
in a per-relay recovery index (sync::missing_events), and the existing
sync maintenance timer refetches them over the relay's live connection
with bounded exponential backoff (30s doubling to 15min, one in-flight
attempt per relay, 300 IDs per fetch). Network I/O runs outside the
sync actor lock. Startup remains non-blocking: the batch still
finalizes as failed, the relay transitions to
ConnectedHistoricSyncFailures, and traffic is served while recovery
runs in the background.
Semantics:
- progress clears only the IDs actually recovered and resets backoff;
- duplicate incomplete responses merge into the pending set without
duplicating work;
- IDs satisfied by live sync or user submission are cleared on the
next tick without consuming attempt budget;
- attempts against a disconnected relay are deferred, not counted, so
an unavailable relay neither expires its work nor loops tightly;
- 12 consecutive zero-progress attempts expire the pending IDs with an
explicit warning; the relay stays observably degraded until the
daily sync re-discovers the gap;
- full recovery promotes the relay back to Connected unless an
unrelated batch failure was observed for it;
- nothing persists across restarts: historic sync re-runs from scratch
and re-detects any still-missing events, so incomplete work is never
falsely reported as complete.
Also fixes the retry-subscription-failure path, which confirmed an
incomplete batch without marking it failed (falsely reporting
Connected), and bounds the previously unbounded missing_ids log arrays
to a five-ID sample.
Regression coverage: a new censoring WebSocket proxy fixture sits
between a syncing relay and a real ngit-grasp bootstrap relay,
forwarding NIP-77 frames unchanged while withholding chosen EVENT
frames. The integration test reproduces the full production sequence
(subset response, zero-progress retry, no semantic fallback,
ConnectedHistoricSyncFailures) and proves the withheld event is
recovered and the relay promoted to Connected once the event becomes
available - without a restart and while live sync continues unstarved.
Unit tests cover registration dedupe, partial clears, backoff growth
and cap, explicit expiry, deferral, and health-restoration poisoning.
Full cargo test suite passes.
tokio-tungstenite was added as a dev-dependency for the proxy fixture;
it was already present transitively in Cargo.lock, so no Nix hash
updates are required (crates.io dependency under cargoLock).
Deliberately out of scope: durable persistence of pending recovery
work, retrying missing IDs against other relays, outbound-target
policy changes, and broader logging cleanup.
Confirms the closed issue
nostr:nevent1qqs94up6nnkzjlz4fcy5tesh8yxvr63xqjhg79etmc573fuunjt0qeqpz3mhxue69uhhyetvv9ujumn8d96zuer9wc5tdht6
ngit-grasp Documentation
Welcome to the ngit-grasp documentation! We use the Diátaxis framework to organize our documentation into four types, each serving a different purpose.
PRACTICAL THEORETICAL
───────── ───────────
LEARNING │ Tutorials │ Explanation │
│ │ │
│ Getting │ Architecture │
│ Started │ Decisions │
│ │ │
├────────────────┼──────────────────┤
│ │ │
WORKING │ How-To │ Reference │
│ Guides │ │
│ │ API Docs │
│ Deployment │ Protocols │
│ Testing │ │
│ │ │
📚 Documentation Types
🎓 Tutorials - Learning by Doing
Purpose: Learn the basics through practical steps
For: Newcomers getting started
Style: Step-by-step lessons with guaranteed outcomes
- Getting Started - Your first ngit-grasp setup
- Running Your First Audit - Using grasp-audit tool
🔧 How-To Guides - Solving Problems
Purpose: Accomplish specific tasks
For: Users with basic knowledge solving real problems
Style: Practical recipes and solutions
- Deploy ngit-grasp - Production deployment guide
- Configure Nix Flakes - Nix development environment
- Run Compliance Tests - GRASP compliance testing
- Upgrade nostr-sdk - Handling SDK upgrades
📖 Reference - Technical Information
Purpose: Look up technical details
For: Users who know what they're looking for
Style: Dry, factual, comprehensive
- Git Protocol - Git Smart HTTP protocol details
- GRASP Protocol - GRASP specification details
- Configuration - All config options
- API Reference - Internal API documentation
💡 Explanation - Understanding Concepts
Purpose: Understand the "why" and design decisions
For: Users wanting deeper understanding
Style: Discussion, context, alternatives
- Architecture Overview - System design and components
- Inline Authorization - Why we chose this approach
- Comparison with ngit-relay - How we differ from reference
- Design Decisions - Key architectural choices
🚀 Quick Start Paths
I'm brand new to ngit-grasp
- Read README.md for project overview
- Follow Getting Started Tutorial
- Understand Architecture Overview
I want to deploy ngit-grasp
- Review Configuration Reference
- Follow Deployment How-To
- Set up monitoring and backups
I want to develop on ngit-grasp
- Follow Getting Started Tutorial
- Read Architecture Overview
- Check Nix Flakes How-To
- Review Test Strategy
I want to understand the design
- Read Inline Authorization Explanation
- Review Design Decisions
- Compare with ngit-relay Comparison
I'm looking for specific information
- Protocol details? → Reference
- Configuration options? → Configuration Reference
- Git protocol? → Git Protocol Reference
📂 Additional Resources
Archive
Historical session notes and completed work. Useful for understanding project evolution but not required reading.
Learnings
DEPRECATED - Being migrated to Diátaxis structure:
- Gotchas → How-To Guides
- Patterns → Reference or Explanation
- Notes → Appropriate category
🤝 Contributing to Documentation
When adding documentation, ask yourself:
Is it a tutorial?
- Does it teach a beginner?
- Is it a complete lesson with guaranteed outcome?
- → Add to
tutorials/
Is it a how-to guide?
- Does it solve a specific problem?
- Is it a recipe for accomplishing a task?
- → Add to
how-to/
Is it reference material?
- Is it technical information?
- Will people look it up when needed?
- → Add to
reference/
Is it explanation?
- Does it explain "why"?
- Does it discuss alternatives or design?
- → Add to
explanation/
See Diátaxis documentation for more guidance.
📊 Project Status
ALPHA - Under active development. Core functionality working, API may change.
Completed
- ✅ grasp-audit compliance testing tool
- ✅ Nix flake development environment
- ✅ nostr-sdk 0.43 upgrade
- ✅ Documentation restructure (Diátaxis)
In Progress
- 🔄 Core ngit-grasp server implementation
- 🔄 GRASP-01 compliance
Planned
- 🔜 GRASP-02 (Proactive Sync)
- 🔜 GRASP-05 (Archive)
🔗 External Links
Documentation structure based on Diátaxis
Last updated: November 4, 2025