mirror of
https://relay.ngit.dev/npub15qydau2hjma6ngxkl2cyar74wzyjshvl65za5k5rl69264ar2exs5cyejr/ngit-grasp.git
synced 2026-10-05 23:18:24 +00:00
DanConwayDev
e942199a55
Historic REQ+EOSE sync subscribed every byte-budgeted filter group of a batch in one loop and left them all awaiting EOSE concurrently, as did the REQ+EOSE fallback, negentropy ID fetches and missing-event retries. On strfry-family relays these REQs share maxSubsPerConnection with negentropy views and live subscriptions; nos.lol (budget 20) answered each gitnostr.com startup with a burst of 'ERROR: too many concurrent REQs' NOTICEs (6 in one second at the 2026-08-05 06:41 startup; 7,739 over the prior three days). A per-connection semaphore (5 permits, shared across clones) now gates every auto-close subscription inside subscribe_filters: the permit is registered against the subscription id on success and released when the connection's own event loop sees that subscription's EOSE or CLOSED frame - before forwarding the notification, so release never depends on downstream channel consumers. This keeps a plain blocking acquire deadlock-free even though the SyncManager actor both creates subscriptions and processes EOSE: bursts pipeline at five in flight, matching the precedent of the actor already stalling inline for negentropy batches. Permits are additionally freed on unsubscribe, disconnect, event-loop termination, and by a 30-second watchdog so a relay that never answers cannot starve later subscriptions. The negentropy semaphore stays separate (different lifetimes); live subscriptions are not gated. Budget: 4 NEG + 5 REQ + 2 margin leaves at least nine slots of the tightest observed budget (20) for live subscriptions. Scheduling is deliberately per-REQ rather than per-core-filter: a paginating filter chain holds no slot between pages, so queued groups interleave breadth-first. Relay-visible concurrency is identical either way and pagination chains have no durable identity across disconnects. Reproduction: a new proxy fixture mimics the strfry limit (rejecting REQs beyond 5 with the production NOTICE, exempting limit:0 live subscriptions, delaying EOSE so REQs provably overlap), and a scenario test syncs 2500 root events into persistent storage, restarts the relay - startup recomputes filters from the full index, the shape that bursts in production - and asserts zero rejections with overlap retained. Unfixed: 3 REQs rejected (opened 91, peak pinned at the limit). Fixed: zero rejections, peak <= 5, sync completes. The TestRelay fixture gains same-port restart support with persistent LMDB storage for this. Deliberately excluded: the unified budget ledger (NIP-11-aware B/M, per-query result caps), raising the 300-ID exact-ID chunks, and any configuration surface for the bound. Validated with the full test suite (nix develop -c cargo test).
45 lines
1.3 KiB
Rust
45 lines
1.3 KiB
Rust
//! Proactive Sync Integration Tests
|
|
//!
|
|
//! This test file organizes tests for ngit-grasp's proactive sync functionality.
|
|
//! Tests are grouped into submodules by sync scenario:
|
|
//!
|
|
//! - `historic_sync` - Tests for sync from pre-configured bootstrap relay (historic events)
|
|
//! - `discovery` - Tests for relay discovery from announcement events
|
|
//! - `live_sync` - Tests for real-time sync after connection established
|
|
//! - `tag_variations` - Tests for different Layer 2/3 tag types
|
|
//! - `catchup` - Tests for catchup sync after disconnect (not yet implemented)
|
|
//! - `metrics` - Tests for Prometheus metrics integration
|
|
//!
|
|
//! # Running Tests
|
|
//!
|
|
//! ```bash
|
|
//! # Run all sync tests
|
|
//! cargo test --test sync
|
|
//!
|
|
//! # Run with output
|
|
//! cargo test --test sync -- --nocapture
|
|
//!
|
|
//! # Run specific test
|
|
//! cargo test --test sync test_bootstrap_syncs -- --nocapture
|
|
//!
|
|
//! # Run ignored tests (like catchup)
|
|
//! cargo test --test sync -- --ignored
|
|
//! ```
|
|
|
|
// Include the common test utilities
|
|
mod common;
|
|
|
|
// Include sync test submodules (located in tests/sync/)
|
|
mod sync {
|
|
pub mod catchup;
|
|
pub mod discovery;
|
|
pub mod historic_recovery;
|
|
pub mod historic_sync;
|
|
pub mod live_sync;
|
|
pub mod maintainer_reprocessing;
|
|
pub mod metrics;
|
|
pub mod neg_concurrency;
|
|
pub mod req_concurrency;
|
|
pub mod tag_variations;
|
|
}
|