Merge maint into master (rx_loop tick-arm connect-on-send fix)

CHANGELOG.md

@@ -133,6 +133,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+- `rx_loop` tick-arm stall under convergence-phase mesh pressure
+  is eliminated. Previously, the tick body's per-peer `check_*`
+  loops (heartbeats, bloom announces, MMP reports, tree announces)
+  called `transport.send` directly for every active peer. For
+  TCP/Tor peers whose pool entry was not yet established,
+  `send_async` fell through to a synchronous connect-on-send
+  branch that wrapped `TcpStream::connect` in
+  `tokio::time::timeout(connect_timeout_ms, …)` — 5 seconds by
+  default — and blocked the entire tick body for the duration per
+  unreachable peer. Under post-restart convergence on a high-peer
+  mesh, this cascaded into multi-second tick stalls; the same
+  mechanism also starved the master-only per-tick control-snapshot
+  republish and pushed `fipsctl show *` queries onto an mpsc
+  fallback that was itself queued behind the wedged `rx_loop`,
+  producing the five-second `fipsctl` head-of-line pattern
+  operators observed on loaded nodes. The send path now gates on
+  `transport.connection_state(addr)` before sending: proceed only
+  when `Connected`; on `None`, kick off a non-blocking background
+  `connect` (idempotent — deduplicates against the connecting
+  pool, spawns the timeout-bounded `TcpStream::connect` inside its
+  own tokio task) and fail this send fast with a clear
+  `transport connection not ready` error. A subsequent tick
+  retries once the pool has an entry. The existing reconnect
+  lifecycle (heartbeat-dead detection in `check_link_heartbeats`,
+  scheduled retries via `process_pending_retries`, background-
+  connect polling via `poll_pending_connects`) is unchanged.
+  The connect-on-send branch in `transport.send_async` itself
+  remains in place for code paths that legitimately need
+  synchronous connect (e.g., explicit operator-driven
+  `fipsctl connect`); the tick path just no longer trips it.
+
 - NAT-traversal cross-init adoption is now deterministic under
   simultaneous dual-initiation. Previously, when two peers'
   Nostr-mediated UDP punches completed within the same scheduling

src/node/mod.rs

@@ -49,7 +49,8 @@ use crate::transport::tcp::TcpTransport;
 use crate::transport::tor::TorTransport;
 use crate::transport::udp::UdpTransport;
 use crate::transport::{
-    Link, LinkId, PacketRx, PacketTx, TransportAddr, TransportError, TransportHandle, TransportId,
+    ConnectionState, Link, LinkId, PacketRx, PacketTx, TransportAddr, TransportError,
+    TransportHandle, TransportId,
 };
 use crate::tree::TreeState;
 use crate::upper::hosts::HostMap;
@@ -2123,6 +2124,27 @@ impl Node {
             .get(&transport_id)
             .ok_or(NodeError::TransportNotFound(transport_id))?;
 
+        // Gate: don't drive connect-on-send from the tick path. If the
+        // transport's connection isn't ready, kick off a non-blocking
+        // background connect (no-op if already in flight or pooled) and
+        // fail this send fast. A subsequent tick will retry once the
+        // pool entry exists. The historical connect-on-send wedged the
+        // rx_loop tick body for up to `connect_timeout_ms` (5 s default)
+        // per unreachable peer, which under convergence-phase mesh
+        // pressure cascaded into multi-tick stalls and control-RPC HOL.
+        match transport.connection_state(&remote_addr) {
+            ConnectionState::Connected => {}
+            other => {
+                if matches!(other, ConnectionState::None) {
+                    let _ = transport.connect(&remote_addr).await;
+                }
+                return Err(NodeError::SendFailed {
+                    node_addr: *node_addr,
+                    reason: format!("transport connection not ready: {:?}", other),
+                });
+            }
+        }
+
         let bytes_sent = transport
             .send(&remote_addr, &wire_packet)
             .await