diff --git a/CHANGELOG.md b/CHANGELOG.md index 2457d15..1955d01 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,14 +14,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 a finite DoS backstop while avoiding an unusually restrictive default that also charges every SDK-managed NIP-77 `NEG-MSG` continuation. Re-evaluate the 10× allowance after upstream revises that NIP-77 accounting. +- Proactively space non-urgent historic sync, pagination, hydration, retry, + and dependency query starts at one per second per relay connection. Live + subscriptions bypass the background gate, while the existing adaptive + refusal handling remains the backstop for tighter or internally generated + traffic. - Pace query starts after a relay reports its per-connection query-rate budget is exhausted, preventing fixed cooldown recovery from replaying the same fast historic-sync burst indefinitely. If an SDK-managed NIP-77 exchange exhausts that budget, use paced REQs for the rest of the connection session because the application cannot pace individual `NEG-MSG` frames. This is a - reactive compatibility backstop for an unadvertised limit; proactive pacing - of lower-priority historic work remains separate so live coverage is not - delayed. + reactive compatibility backstop for an unadvertised limit, layered over the + proactive background pacing while leaving live coverage prioritised. ## [2.1.0] - 2026-08-07 diff --git a/docs/explanation/grasp-02-proactive-sync.md b/docs/explanation/grasp-02-proactive-sync.md index da7bc6f..7abbced 100644 --- a/docs/explanation/grasp-02-proactive-sync.md +++ b/docs/explanation/grasp-02-proactive-sync.md @@ -1062,13 +1062,15 @@ Probing -> previous state: Recovery REQs succeed Probing -> RateLimited: Recovery REQ is rate limited again ``` -The subscription ledger limits simultaneous work, while some relays also -limit completed query operations per minute. A `too many queries` refusal -therefore makes already-queued starts unwind during the 65-second cooldown so -their work can be re-derived in priority order, then activates per-connection -query-start pacing: 600 ms between starts initially, doubling for a distinct -later episode up to 10 seconds. The pacing gate is shared by live and transient -REQs, exact-ID fetches and NIP-77 round starts, and resets with the relay +The subscription ledger limits simultaneous work, while a proactive +per-connection gate spaces non-urgent historic, dependency, pagination, +hydration, retry, and NIP-77 round starts at one per second. Persistent live +subscriptions bypass that background gate. Some relays also limit completed +query operations per minute. A `too many queries` refusal therefore makes +already-queued starts unwind during the 65-second cooldown so their work can be +re-derived in priority order, then activates a shared reactive gate for live +and background operations: 600 ms between starts initially, doubling for a +distinct later episode up to 10 seconds. Both gates reset with the relay connection session. Any query-rate refusal disables NIP-77 for the rest of the session and falls back to paced REQs, because rust-nostr owns the internal `NEG-MSG` frames and the application cannot guarantee their pacing. diff --git a/docs/explanation/monitoring.md b/docs/explanation/monitoring.md index 0ad5df1..1a284b5 100644 --- a/docs/explanation/monitoring.md +++ b/docs/explanation/monitoring.md @@ -150,7 +150,9 @@ After a `too many queries` response, `Activated adaptive query-start pacing` reports the learned per-connection interval. It begins at 600 ms and doubles only when a distinct later episode proves that pace too fast. Queued starts unwind during the existing 65-second cooldown before paced recovery; -reconnecting starts a fresh session without pacing. +reconnecting clears that reactive lesson. Independent proactive pacing still +spaces background historic and dependency starts at one per second; persistent +live subscriptions bypass the background gate. `Falling back to paced REQs for the query-limited connection session` means NIP-77 remains skipped until reconnect because SDK-managed `NEG-MSG` traffic cannot be passed individually through the learned gate. diff --git a/docs/explanation/sync-scaling-constraints.md b/docs/explanation/sync-scaling-constraints.md index 793fe09..9f9da5f 100644 --- a/docs/explanation/sync-scaling-constraints.md +++ b/docs/explanation/sync-scaling-constraints.md @@ -203,13 +203,13 @@ whether it applies independently to each filter or to the merged REQ, which is why the source audit above remains necessary. Relay-specific extensions can add fields, but clients cannot assume common names or semantics. -The implemented query-start pacer is therefore a reactive compatibility -backstop: it remains inactive until an explicit `too many queries` response, -then spaces all application-visible starts and selects REQ fallback because -the application cannot pace individual `NEG-MSG` frames. Proactively being -gentle with non-urgent historic work is desirable but separate. It requires -request-class priority—live coverage first, dependency recovery next, bulk -history last—rather than enabling this shared gate from connection startup. +Two gates cover the distinct concerns. A proactive background gate spaces +historic, dependency, pagination, hydration, retry, and NIP-77 round starts at +one per second from session startup; persistent live subscriptions bypass it. +A reactive compatibility gate remains inactive until an explicit +`too many queries` response, then spaces every application-visible start and +selects REQ fallback because the application cannot pace individual +`NEG-MSG` frames. The client encodes this model in per-connection `RelayPaginationSession` state (`src/sync/mod.rs`). After EOSE it learns the largest raw page seen @@ -400,15 +400,23 @@ So concurrency is not a free scaling axis; it is the residual of the ledger: - Rounds queue behind a per-connection semaphore; each completion releases the next. No timed batches or sleeps — throughput degrades smoothly instead of bursting into rejections. -- The ledger bounds simultaneous resource use, not query starts over time. If - a relay returns `rate-limited: too many queries`, the current connection +- The ledger bounds simultaneous resource use, not query starts over time. + Non-urgent historic REQs, pagination and hydration/retry pages, exact-ID + dependency fetches, and NIP-77 round starts proactively share a one-second + start interval from the beginning of each connection session. Persistent + live subscriptions bypass that background gate and retain priority. This + bounds application-visible starts, but cannot pace SDK-managed NIP-77 + `NEG-MSG` continuations. If a relay returns + `rate-limited: too many queries`, the current connection first rejects locally queued starts for the existing 65-second cooldown so their incomplete work can be re-derived in priority order, then learns a - shared query-start interval: 600 ms initially (100 starts/minute, below our - own 120/minute serving limit), doubling on a later rate-limit episode up to + shared query-start interval: 600 ms initially (100 starts/minute, below the + rust-nostr 120/minute default), doubling on a later rate-limit episode up to 10 seconds. Live, transient REQ, exact-ID fetch and NIP-77 round starts all - pass through that pacer. Relays that never report a query - rate limit remain unpaced, and reconnecting resets the per-session lesson. + pass through that reactive pacer in addition to background work retaining + its proactive spacing. Relays that never report a query rate limit still + receive proactively paced background work, while live starts remain + immediate; reconnecting resets both per-session gates. After any query-budget refusal, NIP-77 is skipped for the rest of the session: rust-nostr owns the internal `NEG-MSG` exchange, so the application cannot guarantee that each charged frame passes through its pacing gate. Historic diff --git a/src/sync/relay_connection.rs b/src/sync/relay_connection.rs index 4446b4b..72f40ac 100644 --- a/src/sync/relay_connection.rs +++ b/src/sync/relay_connection.rs @@ -58,6 +58,15 @@ const QUERY_PACING_MAX_INTERVAL: Duration = Duration::from_secs(10); /// refusal after a full rate-limit window means the learned pace was too fast. const QUERY_RATE_LIMIT_EPISODE_GAP: Duration = Duration::from_secs(60); +/// Minimum spacing between non-urgent historic and dependency query starts. +/// +/// Historic completeness matters, but does not require burst throughput. One +/// background start per second is deliberately gentle to public relays and +/// leaves query capacity for live coverage. This only controls +/// application-visible starts; rust-nostr owns NIP-77 `NEG-MSG` continuations, +/// so the reactive refusal handling remains necessary. +const BACKGROUND_QUERY_INTERVAL: Duration = Duration::from_secs(1); + fn effective_subscription_budget(advertised: Option) -> usize { advertised.unwrap_or(FALLBACK_SUBSCRIPTION_BUDGET) } @@ -87,6 +96,39 @@ struct QueryStartPacer { state: std::sync::Mutex, } +/// Serialises background query starts from the beginning of each session. +/// Live subscriptions bypass this gate and therefore retain priority. +#[derive(Debug, Default)] +struct BackgroundQueryPacer { + gate: tokio::sync::Mutex<()>, + last_start: std::sync::Mutex>, +} + +impl BackgroundQueryPacer { + async fn wait_for_start(&self) { + let _gate = self.gate.lock().await; + let deadline = self + .last_start + .lock() + .expect("background query pacing state poisoned") + .map(|last| last + BACKGROUND_QUERY_INTERVAL); + if let Some(deadline) = deadline { + tokio::time::sleep_until(deadline).await; + } + *self + .last_start + .lock() + .expect("background query pacing state poisoned") = Some(tokio::time::Instant::now()); + } + + fn reset(&self) { + *self + .last_start + .lock() + .expect("background query pacing state poisoned") = None; + } +} + impl QueryStartPacer { fn record_rate_limit(&self) -> (Duration, bool) { let now = tokio::time::Instant::now(); @@ -438,6 +480,8 @@ pub struct RelayConnection { live_req_permits_held: LiveReqPermitMap, /// Learned per-session spacing for query starts after a query-rate refusal. query_start_pacer: std::sync::Arc, + /// Proactive spacing for non-urgent historic and dependency query starts. + background_query_pacer: std::sync::Arc, } impl RelayConnection { @@ -454,6 +498,10 @@ impl RelayConnection { }) } + async fn await_background_query_start(&self) { + self.background_query_pacer.wait_for_start().await; + } + fn record_query_rate_limit(&self) { let (interval, new_episode) = self.query_start_pacer.record_rate_limit(); if new_episode { @@ -548,6 +596,7 @@ impl RelayConnection { std::collections::HashMap::new(), )), query_start_pacer: std::sync::Arc::new(QueryStartPacer::default()), + background_query_pacer: std::sync::Arc::new(BackgroundQueryPacer::default()), } } @@ -607,6 +656,7 @@ impl RelayConnection { std::collections::HashMap::new(), )), query_start_pacer: std::sync::Arc::new(QueryStartPacer::default()), + background_query_pacer: std::sync::Arc::new(BackgroundQueryPacer::default()), } } @@ -785,6 +835,7 @@ impl RelayConnection { pub fn reset_subscription_budget(&self, advertised: Option) { self.clear_subscription_permits(); self.query_start_pacer.reset(); + self.background_query_pacer.reset(); self.nip77_query_rate_limited .store(false, std::sync::atomic::Ordering::Relaxed); let budget = effective_subscription_budget(advertised); @@ -1385,6 +1436,9 @@ impl RelayConnection { // Acquire pacing before subscription/class permits: a learned remote // query-rate window must not turn local capacity into queued sleepers. self.await_query_start().await?; + if transient_class.is_some() { + self.await_background_query_start().await; + } // Transient (auto-close) subscriptions share a bounded number of // per-connection slots so historic bursts queue instead of @@ -1519,6 +1573,7 @@ impl RelayConnection { // nostr-sdk. It must share the same permit bound as historic pages, // fallbacks and retries rather than escaping the connection budget. self.await_query_start().await?; + self.await_background_query_start().await; if self.historic_capacity_consumed_by_live() { tracing::warn!( relay = %self.url, @@ -1928,6 +1983,7 @@ impl RelayConnection { // shared with live subscriptions (see MAX_CONCURRENT_NEG_DIFFS). // The permit is held for the whole round, including the timeout. self.await_query_start().await?; + self.await_background_query_start().await; if self.historic_capacity_consumed_by_live() { tracing::warn!( relay = %self.url, @@ -3203,6 +3259,27 @@ mod tests { .expect("paced query start should be admitted"); } + #[tokio::test(start_paused = true)] + async fn background_query_pacer_spaces_starts_from_session_beginning() { + let pacer = std::sync::Arc::new(BackgroundQueryPacer::default()); + pacer.wait_for_start().await; + + let waiting = { + let pacer = std::sync::Arc::clone(&pacer); + tokio::spawn(async move { pacer.wait_for_start().await }) + }; + tokio::task::yield_now().await; + assert!(!waiting.is_finished()); + tokio::time::advance(BACKGROUND_QUERY_INTERVAL - Duration::from_millis(1)).await; + tokio::task::yield_now().await; + assert!(!waiting.is_finished()); + tokio::time::advance(Duration::from_millis(1)).await; + waiting.await.expect("background query should be admitted"); + + pacer.reset(); + pacer.wait_for_start().await; + } + #[tokio::test(start_paused = true)] async fn query_pacer_deduplicates_bursts_and_slows_new_episodes() { let pacer = QueryStartPacer::default();