mirror of
https://relay.ngit.dev/npub15qydau2hjma6ngxkl2cyar74wzyjshvl65za5k5rl69264ar2exs5cyejr/ngit-grasp.git
synced 2026-10-05 23:18:24 +00:00
DanConwayDev
18eddeedb5
nostr:nevent1qgsx2lyl2e4zvfadwcvkd9fkrcwczj7mf858hy85mwqclwgut8wpg2spz3mhxue69uhhyetvv9ujumn8d96zuer9wcq3yamnwvaz7tm8d96xummnw3ezucm0d5q3kamnwvaz7tmwva5hgtnyv9hxxmmwwashjer9wchxxmmdqqsyuueftzc68d5a3x5d0pecs66y0nacv4pc87kkcfw499adyl5t3ycxqgtta PR-Author: DanConwayDev's Agent nostr:npub1v47f74n2ycn66asev62nv8sas99akj0g0wg0fkup37u3ckwuzs4q7cwtp0 CoverNote: ## What this adds Repository conversations can continue on relays used only by a non-root participant. For example, a reaction to an issue may be stored on one relay, while a reply to that reaction exists only on the reaction author's NIP-65 write relay. Root-author relay discovery and recursive reference queries cannot find that reply unless the participant's mailbox is also searched. This PR adds bounded, historic participant-mailbox coverage: 1. Preserve exact repository-root provenance while deriving the accepted recursive descendant frontier. 2. Treat authors of accepted replies, comments, reactions, zaps and other descendants as thread participants. 3. Accept kind `10002` relay lists only for repository owners, maintainers, root authors and those accepted participants. 4. Derive each participant's NIP-65 read, write and unmarked mailbox relays. 5. Query those mailboxes only for the repository roots and accepted descendant IDs associated with that participant. 6. Pass every returned event through the ordinary write policy, deletion/replacement rules and persistence pipeline. This discovers replies to reactions and other indirect descendants without fetching a participant's unrelated notes or trusting their relay list to bypass repository policy. ## Bounds and scheduling - The existing `NGIT_SYNC_RECURSIVE_DESCENDANT_LIMIT` bounds which indirect descendant IDs remain query roots. Each direct event that tags a repository root has its own bounded branch; the default is 500. - Once a branch reaches that bound, later indirect descendants cannot add more participant authors or query roots through that branch. - Exact root provenance prevents roots from one repository leaking into another participant query. - Participant mailbox coverage is historic only. It does not add permanent non-root participant subscriptions or expand the existing live-sync tier. - Filters use the existing byte-bounded grouping and paginated `RelayConnection::fetch_events` path, including request pacing, background priority, subscription-ledger capacity and EOSE/CLOSED handling. - At most one mailbox worker runs per relay, and at most one new due relay is started per maintenance pass. - Relays progress independently: there is no global mailbox lane, cross-relay success condition or shared completion counter. ## Restart and failure behavior - A probe starts only after both the WebSocket and the sync actor's connection lifecycle are ready. - Ready due relays are preferred, so old unreachable mailbox sources cannot starve connected work. - Completed mailbox workers are handled before new connection results, so a large startup connection queue cannot delay cursor progress or resource release. - Successful filters advance an in-memory, stable-sorted cursor immediately. Completing the last filter schedules the next historic rotation after 24 hours. - Failed filters release their relay worker and retry after five minutes without blocking other relays. - On restart, accepted roots, participants and kind `10002` ownership are reconstructed from LMDB. Filter cursors deliberately restart from the first current group; they are best-effort coverage, not durable exactly-once state. The implementation reuses the existing relay transport. It does not add a mailbox-specific protocol state machine, pending-batch purpose, CLOSE API, NIP-42 retry system, watchdog or connection-queue coordinator. ## Coverage and production evidence The audit motivating this change found 72 events visible through gitworkshop but absent from the pre-change production relay across the `gitworkshop` and `ngit` repositories. The persistent canary served 29 exact misses: six kind-1 notes, three reposts, ten reactions, six comments, two issues, one zap receipt and one repository-follow event. The exact candidate `de5fa6625aa5693afcde7335a572456680aef9df` activated on the isolated archive at 2026-08-14 19:23:49 UTC. Restart reconstruction found 4,291 accepted roots, 403 participant authors and 351 mailbox relays. During the recorded soak: - 63 mailbox filters started and all 63 reached terminal handling: 59 successes and four bounded failures. - Failures received the intended five-minute retry and did not block successful progress on other relays. - `relay.ngit.dev`, `nostr.land`, `nostr.mom`, `haven.danconwaydev.com/inbox` and other relays advanced independently. - `nostr.azzamo.net` completed all six filter groups; the final terminal reported `completed_cycle=true` and scheduled the next probe in 86,400 seconds. - The archive remained active with zero restarts and no process panic or fatal error. The temporary archive test override of `NGIT_SYNC_RECURSIVE_DESCENDANT_LIMIT=2` has been removed. Both the archive and public gitnostr.com have no override and therefore use the default of 500. Public gitnostr.com was not restarted or changed by the archive deployment. ## Validation and review state - Exact tested head: `de5fa6625aa5693afcde7335a572456680aef9df`. - One commit; 1,157 additions and 140 deletions across 11 files. - 767 library tests passed in the exact Nix release build locally and on the production host. - All three proactive Sync+ integration scenarios pass together, including a child found only through a participant reaction author's write mailbox. - Strict all-target Clippy, formatting, diff checks, Nix flake evaluation and a conflict-free merge-tree check against current `master` passed. - One remote build attempt hit the pre-existing `test_entries_expired_during_downtime` timing flake; its source is unchanged by this PR, and the unchanged candidate passed all 767 tests on retry before activation. Recommended for merge. The archive demonstrates successful and failed terminal paths, per-relay independence, restart reconstruction, cursor advancement and a complete historic-to-24-hour-refresh rotation under real startup load.
ngit-grasp Documentation
Welcome to the ngit-grasp documentation! We use the Diátaxis framework to organize our documentation into four types, each serving a different purpose.
PRACTICAL THEORETICAL
───────── ───────────
LEARNING │ Tutorials │ Explanation │
│ │ │
│ Getting │ Architecture │
│ Started │ Decisions │
│ │ │
├────────────────┼──────────────────┤
│ │ │
WORKING │ How-To │ Reference │
│ Guides │ │
│ │ API Docs │
│ Deployment │ Protocols │
│ Testing │ │
│ │ │
📚 Documentation Types
🎓 Tutorials - Learning by Doing
Purpose: Learn the basics through practical steps
For: Newcomers getting started
Style: Step-by-step lessons with guaranteed outcomes
- Getting Started - Your first ngit-grasp setup
- Running Your First Audit - Using grasp-audit tool
🔧 How-To Guides - Solving Problems
Purpose: Accomplish specific tasks
For: Users with basic knowledge solving real problems
Style: Practical recipes and solutions
- Deploy ngit-grasp - Production deployment guide
- Configure Nix Flakes - Nix development environment
- Run Compliance Tests - GRASP compliance testing
- Upgrade nostr-sdk - Handling SDK upgrades
📖 Reference - Technical Information
Purpose: Look up technical details
For: Users who know what they're looking for
Style: Dry, factual, comprehensive
- Git Protocol - Git Smart HTTP protocol details
- GRASP Protocol - GRASP specification details
- Configuration - All config options
- API Reference - Internal API documentation
💡 Explanation - Understanding Concepts
Purpose: Understand the "why" and design decisions
For: Users wanting deeper understanding
Style: Discussion, context, alternatives
- Architecture Overview - System design and components
- Inline Authorization - Why we chose this approach
- Comparison with ngit-relay - How we differ from reference
- Design Decisions - Key architectural choices
🚀 Quick Start Paths
I'm brand new to ngit-grasp
- Read README.md for project overview
- Follow Getting Started Tutorial
- Understand Architecture Overview
I want to deploy ngit-grasp
- Review Configuration Reference
- Follow Deployment How-To
- Set up monitoring and backups
I want to develop on ngit-grasp
- Follow Getting Started Tutorial
- Read Architecture Overview
- Check Nix Flakes How-To
- Review Test Strategy
I want to understand the design
- Read Inline Authorization Explanation
- Review Design Decisions
- Compare with ngit-relay Comparison
I'm looking for specific information
- Protocol details? → Reference
- Configuration options? → Configuration Reference
- Git protocol? → Git Protocol Reference
📂 Additional Resources
Archive
Historical session notes and completed work. Useful for understanding project evolution but not required reading.
Learnings
DEPRECATED - Being migrated to Diátaxis structure:
- Gotchas → How-To Guides
- Patterns → Reference or Explanation
- Notes → Appropriate category
🤝 Contributing to Documentation
When adding documentation, ask yourself:
Is it a tutorial?
- Does it teach a beginner?
- Is it a complete lesson with guaranteed outcome?
- → Add to
tutorials/
Is it a how-to guide?
- Does it solve a specific problem?
- Is it a recipe for accomplishing a task?
- → Add to
how-to/
Is it reference material?
- Is it technical information?
- Will people look it up when needed?
- → Add to
reference/
Is it explanation?
- Does it explain "why"?
- Does it discuss alternatives or design?
- → Add to
explanation/
See Diátaxis documentation for more guidance.
📊 Project Status
ALPHA - Under active development. Core functionality working, API may change.
Completed
- ✅ grasp-audit compliance testing tool
- ✅ Nix flake development environment
- ✅ nostr-sdk 0.43 upgrade
- ✅ Documentation restructure (Diátaxis)
In Progress
- 🔄 Core ngit-grasp server implementation
- 🔄 GRASP-01 compliance
Planned
- 🔜 GRASP-02 (Proactive Sync)
- 🔜 GRASP-05 (Archive)
🔗 External Links
Documentation structure based on Diátaxis
Last updated: November 4, 2025