diff --git a/CHANGELOG.md b/CHANGELOG.md index 8c52ac1..a276ce5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -60,6 +60,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- `rx_loop` tick-arm stall under convergence-phase mesh pressure + is eliminated. Previously, the tick body's per-peer `check_*` + loops (heartbeats, bloom announces, MMP reports, tree announces) + called `transport.send` directly for every active peer. For + TCP/Tor peers whose pool entry was not yet established, + `send_async` fell through to a synchronous connect-on-send + branch that wrapped `TcpStream::connect` in + `tokio::time::timeout(connect_timeout_ms, …)` — 5 seconds by + default — and blocked the entire tick body for the duration per + unreachable peer. Under post-restart convergence on a high-peer + mesh, this cascaded into multi-second tick stalls; the same + mechanism also starved the master-only per-tick control-snapshot + republish and pushed `fipsctl show *` queries onto an mpsc + fallback that was itself queued behind the wedged `rx_loop`, + producing the five-second `fipsctl` head-of-line pattern + operators observed on loaded nodes. The send path now gates on + `transport.connection_state(addr)` before sending: proceed only + when `Connected`; on `None`, kick off a non-blocking background + `connect` (idempotent — deduplicates against the connecting + pool, spawns the timeout-bounded `TcpStream::connect` inside its + own tokio task) and fail this send fast with a clear + `transport connection not ready` error. A subsequent tick + retries once the pool has an entry. The existing reconnect + lifecycle (heartbeat-dead detection in `check_link_heartbeats`, + scheduled retries via `process_pending_retries`, background- + connect polling via `poll_pending_connects`) is unchanged. + The connect-on-send branch in `transport.send_async` itself + remains in place for code paths that legitimately need + synchronous connect (e.g., explicit operator-driven + `fipsctl connect`); the tick path just no longer trips it. + - NAT-traversal cross-init adoption is now deterministic under simultaneous dual-initiation. Previously, when two peers' Nostr-mediated UDP punches completed within the same scheduling diff --git a/src/node/mod.rs b/src/node/mod.rs index cb64298..eb76312 100644 --- a/src/node/mod.rs +++ b/src/node/mod.rs @@ -45,7 +45,8 @@ use crate::transport::tcp::TcpTransport; use crate::transport::tor::TorTransport; use crate::transport::udp::UdpTransport; use crate::transport::{ - Link, LinkId, PacketRx, PacketTx, TransportAddr, TransportError, TransportHandle, TransportId, + ConnectionState, Link, LinkId, PacketRx, PacketTx, TransportAddr, TransportError, + TransportHandle, TransportId, }; use crate::tree::TreeState; use crate::upper::hosts::HostMap; @@ -1967,6 +1968,27 @@ impl Node { .get(&transport_id) .ok_or(NodeError::TransportNotFound(transport_id))?; + // Gate: don't drive connect-on-send from the tick path. If the + // transport's connection isn't ready, kick off a non-blocking + // background connect (no-op if already in flight or pooled) and + // fail this send fast. A subsequent tick will retry once the + // pool entry exists. The historical connect-on-send wedged the + // rx_loop tick body for up to `connect_timeout_ms` (5 s default) + // per unreachable peer, which under convergence-phase mesh + // pressure cascaded into multi-tick stalls and control-RPC HOL. + match transport.connection_state(&remote_addr) { + ConnectionState::Connected => {} + other => { + if matches!(other, ConnectionState::None) { + let _ = transport.connect(&remote_addr).await; + } + return Err(NodeError::SendFailed { + node_addr: *node_addr, + reason: format!("transport connection not ready: {:?}", other), + }); + } + } + let bytes_sent = transport .send(&remote_addr, &wire_packet) .await