mirror of
https://relay.ngit.dev/npub15qydau2hjma6ngxkl2cyar74wzyjshvl65za5k5rl69264ar2exs5cyejr/ngit-grasp.git
synced 2026-10-06 07:28:23 +00:00
DanConwayDev
e942199a55
Historic REQ+EOSE sync subscribed every byte-budgeted filter group of a batch in one loop and left them all awaiting EOSE concurrently, as did the REQ+EOSE fallback, negentropy ID fetches and missing-event retries. On strfry-family relays these REQs share maxSubsPerConnection with negentropy views and live subscriptions; nos.lol (budget 20) answered each gitnostr.com startup with a burst of 'ERROR: too many concurrent REQs' NOTICEs (6 in one second at the 2026-08-05 06:41 startup; 7,739 over the prior three days). A per-connection semaphore (5 permits, shared across clones) now gates every auto-close subscription inside subscribe_filters: the permit is registered against the subscription id on success and released when the connection's own event loop sees that subscription's EOSE or CLOSED frame - before forwarding the notification, so release never depends on downstream channel consumers. This keeps a plain blocking acquire deadlock-free even though the SyncManager actor both creates subscriptions and processes EOSE: bursts pipeline at five in flight, matching the precedent of the actor already stalling inline for negentropy batches. Permits are additionally freed on unsubscribe, disconnect, event-loop termination, and by a 30-second watchdog so a relay that never answers cannot starve later subscriptions. The negentropy semaphore stays separate (different lifetimes); live subscriptions are not gated. Budget: 4 NEG + 5 REQ + 2 margin leaves at least nine slots of the tightest observed budget (20) for live subscriptions. Scheduling is deliberately per-REQ rather than per-core-filter: a paginating filter chain holds no slot between pages, so queued groups interleave breadth-first. Relay-visible concurrency is identical either way and pagination chains have no durable identity across disconnects. Reproduction: a new proxy fixture mimics the strfry limit (rejecting REQs beyond 5 with the production NOTICE, exempting limit:0 live subscriptions, delaying EOSE so REQs provably overlap), and a scenario test syncs 2500 root events into persistent storage, restarts the relay - startup recomputes filters from the full index, the shape that bursts in production - and asserts zero rejections with overlap retained. Unfixed: 3 REQs rejected (opened 91, peak pinned at the limit). Fixed: zero rejections, peak <= 5, sync completes. The TestRelay fixture gains same-port restart support with persistent LMDB storage for this. Deliberately excluded: the unified budget ledger (NIP-11-aware B/M, per-query result caps), raising the 300-ID exact-ID chunks, and any configuration surface for the bound. Validated with the full test suite (nix develop -c cargo test).
ngit-grasp Documentation
Welcome to the ngit-grasp documentation! We use the Diátaxis framework to organize our documentation into four types, each serving a different purpose.
PRACTICAL THEORETICAL
───────── ───────────
LEARNING │ Tutorials │ Explanation │
│ │ │
│ Getting │ Architecture │
│ Started │ Decisions │
│ │ │
├────────────────┼──────────────────┤
│ │ │
WORKING │ How-To │ Reference │
│ Guides │ │
│ │ API Docs │
│ Deployment │ Protocols │
│ Testing │ │
│ │ │
📚 Documentation Types
🎓 Tutorials - Learning by Doing
Purpose: Learn the basics through practical steps
For: Newcomers getting started
Style: Step-by-step lessons with guaranteed outcomes
- Getting Started - Your first ngit-grasp setup
- Running Your First Audit - Using grasp-audit tool
🔧 How-To Guides - Solving Problems
Purpose: Accomplish specific tasks
For: Users with basic knowledge solving real problems
Style: Practical recipes and solutions
- Deploy ngit-grasp - Production deployment guide
- Configure Nix Flakes - Nix development environment
- Run Compliance Tests - GRASP compliance testing
- Upgrade nostr-sdk - Handling SDK upgrades
📖 Reference - Technical Information
Purpose: Look up technical details
For: Users who know what they're looking for
Style: Dry, factual, comprehensive
- Git Protocol - Git Smart HTTP protocol details
- GRASP Protocol - GRASP specification details
- Configuration - All config options
- API Reference - Internal API documentation
💡 Explanation - Understanding Concepts
Purpose: Understand the "why" and design decisions
For: Users wanting deeper understanding
Style: Discussion, context, alternatives
- Architecture Overview - System design and components
- Inline Authorization - Why we chose this approach
- Comparison with ngit-relay - How we differ from reference
- Design Decisions - Key architectural choices
🚀 Quick Start Paths
I'm brand new to ngit-grasp
- Read README.md for project overview
- Follow Getting Started Tutorial
- Understand Architecture Overview
I want to deploy ngit-grasp
- Review Configuration Reference
- Follow Deployment How-To
- Set up monitoring and backups
I want to develop on ngit-grasp
- Follow Getting Started Tutorial
- Read Architecture Overview
- Check Nix Flakes How-To
- Review Test Strategy
I want to understand the design
- Read Inline Authorization Explanation
- Review Design Decisions
- Compare with ngit-relay Comparison
I'm looking for specific information
- Protocol details? → Reference
- Configuration options? → Configuration Reference
- Git protocol? → Git Protocol Reference
📂 Additional Resources
Archive
Historical session notes and completed work. Useful for understanding project evolution but not required reading.
Learnings
DEPRECATED - Being migrated to Diátaxis structure:
- Gotchas → How-To Guides
- Patterns → Reference or Explanation
- Notes → Appropriate category
🤝 Contributing to Documentation
When adding documentation, ask yourself:
Is it a tutorial?
- Does it teach a beginner?
- Is it a complete lesson with guaranteed outcome?
- → Add to
tutorials/
Is it a how-to guide?
- Does it solve a specific problem?
- Is it a recipe for accomplishing a task?
- → Add to
how-to/
Is it reference material?
- Is it technical information?
- Will people look it up when needed?
- → Add to
reference/
Is it explanation?
- Does it explain "why"?
- Does it discuss alternatives or design?
- → Add to
explanation/
See Diátaxis documentation for more guidance.
📊 Project Status
ALPHA - Under active development. Core functionality working, API may change.
Completed
- ✅ grasp-audit compliance testing tool
- ✅ Nix flake development environment
- ✅ nostr-sdk 0.43 upgrade
- ✅ Documentation restructure (Diátaxis)
In Progress
- 🔄 Core ngit-grasp server implementation
- 🔄 GRASP-01 compliance
Planned
- 🔜 GRASP-02 (Proactive Sync)
- 🔜 GRASP-05 (Archive)
🔗 External Links
Documentation structure based on Diátaxis
Last updated: November 4, 2025