mirror of
https://relay.ngit.dev/npub15qydau2hjma6ngxkl2cyar74wzyjshvl65za5k5rl69264ar2exs5cyejr/ngit-grasp.git
synced 2026-10-06 07:28:23 +00:00
DanConwayDev
a0520c3d6b
Historic sync opened one NIP-77 negentropy diff per filter with no bound:
handle_add_filters batches launched every diff simultaneously through an
unbounded join_all, so a large watched set (146 filters on the bootstrap
relay at startup) burst far past relay per-connection subscription
budgets. On strfry-family relays negentropy views share
maxSubsPerConnection with ordinary subscriptions; nos.lol (budget 20)
answered a gitnostr.com startup with 34 'too many concurrent NEG
requests' rejections, 61 per-filter timeouts, and 21 failed fallback
subscription creations in two minutes (2026-08-04, PR commit ecb6c8b6
soak). The cycle-2 transient cooldown contains the damage; this removes
the cause.
Approach: a per-connection tokio semaphore (4 permits, shared across
clones) gates negentropy_sync_diff. The permit is held for the whole
round including the timeout, so concurrent batches targeting the same
relay share one bound; queued rounds re-check supports_negentropy()
after acquiring, bailing to the per-batch REQ+EOSE fallback without
recording a failure when a cooldown started while they waited. Four
permits keeps the tightest commonly observed budget (20, shared with
live subscriptions) mostly free; constraint research and the budget
model are documented in docs/explanation/sync-scaling-constraints.md.
Deliberately excluded: bounding REQ+EOSE fallback subscriptions (needs
permit lifetimes spanning EOSE handling in the manager loop; deferred to
the budget-ledger work), byte-budgeted filter chunking (next cycle), and
any configuration surface for the bound.
Validation: new scenario test drives two real relays end to end through
a proxy enforcing the strfry limit of 4 with delayed NEG responses so
rounds provably overlap; a 150-root-event batch needs six rounds. On the
unfixed code the proxy rejected 3 rounds (opened 6, peak 4, production
signature reproduced); with the fix, zero rejections, peak <= 4 with
overlap retained, and sync completes. Full test suite passes; one
unrelated grasp06_pr_hosting test was flaky in the full run and passes
standalone.
44 lines
1.3 KiB
Rust
44 lines
1.3 KiB
Rust
//! Proactive Sync Integration Tests
|
|
//!
|
|
//! This test file organizes tests for ngit-grasp's proactive sync functionality.
|
|
//! Tests are grouped into submodules by sync scenario:
|
|
//!
|
|
//! - `historic_sync` - Tests for sync from pre-configured bootstrap relay (historic events)
|
|
//! - `discovery` - Tests for relay discovery from announcement events
|
|
//! - `live_sync` - Tests for real-time sync after connection established
|
|
//! - `tag_variations` - Tests for different Layer 2/3 tag types
|
|
//! - `catchup` - Tests for catchup sync after disconnect (not yet implemented)
|
|
//! - `metrics` - Tests for Prometheus metrics integration
|
|
//!
|
|
//! # Running Tests
|
|
//!
|
|
//! ```bash
|
|
//! # Run all sync tests
|
|
//! cargo test --test sync
|
|
//!
|
|
//! # Run with output
|
|
//! cargo test --test sync -- --nocapture
|
|
//!
|
|
//! # Run specific test
|
|
//! cargo test --test sync test_bootstrap_syncs -- --nocapture
|
|
//!
|
|
//! # Run ignored tests (like catchup)
|
|
//! cargo test --test sync -- --ignored
|
|
//! ```
|
|
|
|
// Include the common test utilities
|
|
mod common;
|
|
|
|
// Include sync test submodules (located in tests/sync/)
|
|
mod sync {
|
|
pub mod catchup;
|
|
pub mod discovery;
|
|
pub mod historic_recovery;
|
|
pub mod historic_sync;
|
|
pub mod live_sync;
|
|
pub mod maintainer_reprocessing;
|
|
pub mod metrics;
|
|
pub mod neg_concurrency;
|
|
pub mod tag_variations;
|
|
}
|