The transport layer used Mutex::lock().unwrap() at ten sites across the
UDP, BLE, and Ethernet code. A std mutex poisons if a thread panics
while holding it, after which every lock().unwrap() on that same mutex
also panics, turning one fault into a cascade. These critical sections
only perform short HashMap/Vec operations on locally constructed values
and are not reachable from peer input, but the idiom is fragile against
any future in-section panic. Replace each with
lock().unwrap_or_else(|e| e.into_inner()), which recovers the guarded
data and removes the cascade with no new dependency and no call-graph
change.
Also replace four self.local_addr.unwrap() calls in the UDP start and
adopt paths with a sentinel fallback. The value is provably set just
above each log line today, but the unwrap is brittle against a future
reordering; logging an unbound sentinel is harmless and cannot panic.
When two peers each auto_connect to the other, each runs both an
initiator and a responder NAT-traversal session and binds a separate
UDP socket per session. Each side adopts only the first Established
event and drops the loser session's socket; when the two sides adopt
mismatched sessions, each sends its Noise msg1 to a peer port the peer
has already stopped draining, and both handshakes stall.
Deterministically keep the session initiated by the smaller NodeAddr,
decided on the responder path: decline an incoming offer only when we
also have an in-flight outbound initiator for the same peer and our
NodeAddr is smaller. The peer's redundant initiator then times out,
leaving a single matching socket pair on both ends. Asymmetric
(one-sided) auto_connect has no co-active initiator and is never
suppressed, so connectivity is preserved; an undecidable NodeAddr falls
through to answering.
Reuses the NodeAddr tie-breaker convention already used by the
cross-connection and rekey dual-init paths. Adds a unit test for the
election helper.
Add a Loopback variant to TransportHandle backed by an unbounded
in-process channel and a shared address-to-receiver registry, so
node-level multi-node tests deliver packets directly between nodes
instead of over real localhost UDP sockets. This removes the kernel
UDP receive-buffer overflow that dropped handshake packets when many
tests ran in parallel under CPU contention, and lets the large-network
convergence tests run reliably in the default suite again (their
parallel-load ignore markers are removed).
The new transport and its enum variant are cfg(test)-gated, so the
daemon build is unaffected.
The FMP rekey msg1 resend driver retransmitted indefinitely with no cap
and no abandon, so a rekey that never completed kept resending msg1
forever. Give it a retransmission budget: cap resends at
handshake_max_resends with exponential backoff and abandon the rekey
cycle cleanly once the budget is exhausted, mirroring the FSP session
rekey msg3 driver.
With the cap in place the link-dead heartbeat can safely become
rekey-aware: check_link_heartbeats now suppresses teardown while a rekey
is in progress with msg1 budget remaining, instead of reaping a link
that is still actively carrying rekey-handshake traffic. The suppression
terminates deterministically (the budget abandons on exhaustion, cutover
clears the in-progress flag), so a genuinely dead link is still reaped on
the next cycle.
Adds a rekey_msg1_resend_count counter on ActivePeer reset at every
rekey-clear and cutover site, msg1 resend-budget unit tests, and two-node
heartbeat suppression/resume/regression integration tests.
Remove the duplicated immutable fields (config, identity, startup_epoch,
started_at, is_leaf_only, max_connections/peers/links) from the Node
struct so the Arc<NodeContext> bundle is the single source of truth.
Previously Node owned these fields and a parallel context copy, kept in
lockstep by rebuild_context() at every mutation site — pure overhead that
existed only because of the duplication.
- Replace rebuild_context() with replace_context(): a clone-edit-swap of
the whole Arc. The per-instance context stays immutable; mutation swaps
the Arc. This is the sole runtime mutation path (constructors, leaf_only,
update_peers).
- Add Copy-returning accessors startup_epoch() and max_connections()/
max_peers()/max_links(); migrate the remaining direct field readers onto
the accessors. node_addr()/npub()/Debug now read identity/is_leaf_only
from the context.
- update_peers reads the pre-update peer set from the live context Arc
before building a fresh Config + context and swapping — preserving the
read-before-write ordering its mutation-window test depends on.
- Remove the test-only set_max_* setters; tests set the limits on Config at
construction instead (new make_node_with_max_peers/links helpers).
- Add a ci-local guard that fails if the Node struct re-declares a bundled
field, so the single-store invariant can't silently regress.
cargo test --lib 1291/0; clippy -D warnings and release build clean.
Store node counters in an atomic metric registry read through &self, and
introduce a shared NodeContext bundle holding the effectively-immutable
fields (config, identity, startup epoch, capability limits). Source the
immutable config and identity reads across the receive hot path, the
handshake/session/mmp/encrypted state machines, and the discovery, tree,
bloom, retry, and lifecycle modules through the context accessors rather
than direct field reads. The Node fields and the context are rebuilt in
lockstep at every mutation site.
PeerRecvDrain::drop previously called std::thread::join on the worker
thread synchronously. The worker uses packet_tx.blocking_send on a
tokio mpsc Sender, which internally parks the worker via
tokio::block_on on the same current_thread runtime that drives
rx_loop. Calling join from inside remove_active_peer (which runs on
the runtime thread, the runtime's sole driver) created a circular
wait:
- rx_loop blocks in libc futex via Thread::join
- the worker being joined cannot observe the stop flag because the
runtime that polls it is the very thread now blocked joining it
- all other PeerRecvDrain workers park on the same runtime via
block_on, so a single peer's removal wedges every worker on the
daemon
The /proc snapshot from a production wedge showed exactly this
shape: 107 of 108 threads in futex_do_wait, 101 of them named
fips-peer-drain. fipsctl became unresponsive (EAGAIN on control
socket), SIGTERM was ignored, and Docker SIGKILLed the container
after the 10 s grace period. Two confirmed wedges on the public
test deployment (52 min and 23 min uptime), plus a third on the
admission-gate-Msg2-silent-drop build at 2 min 21 sec — all ending
with the identical "Peer removed and state cleaned up
tree_changed=false" final log line preceding total silence.
Fix: detach the std::thread instead of joining. The stop flag plus
self-pipe write already signal the worker to exit; the worker's
kernel-level libc::poll inside the drain loop sees the wake, checks
the flag, exits, and the OS reclaims the thread state independently
of the JoinHandle being dropped.
The trigger was statistically amplified by aggressive multi-npub-
from-one-NAT peer reconnect patterns at the moment of the 30 s
link-dead-timeout peer-removal, but not bounded to them. Any
peer whose disconnect happens with the per-peer drain worker
parked in block_on can fire the bug. The admission-gate work
that landed earlier in this branch line compressed more handshake
work per rx_loop tick, increasing the rate at which workers are
parked in block_on and so reducing time-to-wedge — but the
underlying bug pre-dated the admission gate and pre-dated this
fix branch.
The deployed wedged daemon is mitigated operationally by blocking
the trigger IP at the host firewall; this commit removes the bug
class entirely.
The discovery request dedup cache (recent_requests) silently dropped
LookupRequests once it reached MAX_RECENT_DISCOVERY_REQUESTS, with no
counter to surface the condition. Add a DiscoveryReject::ReqDedupCacheFull
reject reason backed by a req_dedup_cache_full counter on DiscoveryStats,
mirroring the existing duplicate-request counter, and record it at the
drop site so the rejection is visible in show_routing.
Bring the runtime peer-list refresh and opt-in mDNS LAN discovery work
on master into the receive-path RejectReason / reloadable-config
integration branch. Code files auto-merge clean; the only conflict is
the CHANGELOG Unreleased section, resolved as the union of both sets of
entries.
Add scoped mDNS / DNS-SD discovery for peers on the same local link,
giving sub-second pairing without a relay or NAT-traversal roundtrip.
A node advertises its npub, protocol version, and an optional network
scope over link-local multicast, and browses for matching adverts to
initiate Noise handshakes against same-LAN peers.
LAN discovery is disabled by default; operators enable it with
node.discovery.lan.enabled: true. Default-off avoids reintroducing a
per-LAN identity broadcast on nodes that have deliberately disabled
other discovery channels, and avoids any multicast surprise on upgrade.
The startup advertised-port picker now excludes bootstrap transports
and selects a non-bootstrap operational UDP transport with a stable
lowest-id selector, so the advertised port is deterministic across
restarts rather than dependent on HashMap iteration order. This
matches the per-dial transport selection used for discovered peers.
Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
Add Node::update_peers for runtime peer-list refresh. It re-derives the
active peer connections from a new peer configuration, adding newly
configured peers and removing those no longer present, while keeping
links to peers that remain in the set rather than tearing every
connection down. The call returns an UpdatePeersOutcome summarizing the
added, removed, and retained peers.
PeerAddress gains a seen_at_ms recency field (with_seen_at_ms). Active
path selection now sorts address candidates by recency so the most
recently observed address wins when concurrent path probes race.
complete_rekey_msg2 now returns the remote peer's startup epoch
alongside the new Noise session, letting the rekey path detect a peer
restart and clear stale session state. A stale FSP session is cleared
when a peer restart is detected during FMP rekey or cross-connection
promotion, so the session-layer map no longer lingers out of sync with
the freshly promoted peer.
Per-tick work budgets bound the connection churn in a single node tick
(MAX_DISCOVERY_CONNECTS_PER_TICK, MAX_RETRY_CONNECTIONS_PER_TICK,
MAX_PARALLEL_PATH_CANDIDATES_PER_PEER); work beyond a tick's budget is
deferred to the next tick rather than discarded.
Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
Move PeerAclReloader onto the Reloadable trait: its ACL snapshot is now
published through an arc_swap::ArcSwap so the authorization hot path reads
it without locking, and the former check_reload becomes the trait's
reload(). The node tick calls self.peer_acl.reload().await.
Wire the host map into the tick as well. The host map snapshot was
previously taken once at construction and never polled; it now hot-reloads
on /etc/fips/hosts mtime changes once per tick, alongside the ACL, so
hostname display reflects edits without a restart.
The path_mtu_lookup cache (event-driven, populated from observed traffic)
and the nostr_discovery subsystem (an async spawned task) are deliberately
left off the trait: neither reloads from a backing file, so a no-op reload()
would be misleading. The rationale is documented on the trait module.
The host map and the ACL's embedded alias reloader still stat /etc/fips/hosts
independently each tick. A single small-file stat per tick is cheap, so the
duplicate is left in place; sharing one mtime observation between the two is
a possible future cleanup.
Tests: a node-level test exercises the host-map tick reload end to end
through peer_display_name; the ACL reloader tests are updated to drive the
async reload().
Add a `Reloadable` trait that normalizes the node's reloadable
configuration/resource pattern onto a single contract built around an
`arc_swap::ArcSwap` snapshot: a lock-free `load()` for the hot read path
and an async `reload()` that re-reads the backing source and atomically
swaps in a fresh snapshot. The trait carries the canonical Arc-wrapper
template documentation (single-writer node tick, many-reader hot path,
whole-snapshot swap so readers never observe a partial update).
Migrate the host map to this trait via a new `HostMapReloadable` that
reuses the existing load/merge/mtime helpers in upper::hosts. The Node
`host_map` field changes from `Arc<HostMap>` to `HostMapReloadable`, and
`peer_display_name` reads through a lock-free guard. The initial snapshot
is byte-identical to the previous construction, so behavior is unchanged.
The host map is still snapshotted once at construction and not polled;
`reload()` is exercised only by unit tests for now. Wiring the periodic
poll into the node tick, and deduplicating the hosts-file stat against
the ACL reloader's embedded copy, is left as a follow-up.
Add `arc-swap` as a dependency. Unit tests cover initial load (base +
file, base only), change/no-change/deletion/creation detection,
base-preserved-on-reload, and equivalence of the initial snapshot to the
pre-migration construction.
Introduce a typed RejectReason enum and a NodeStats::record_reject
dispatch so every receive-path rejection-and-return site bumps a
machine-readable per-subsystem counter while keeping its operator-facing
log line. The top-level variants mirror the existing NodeStats subsystem
split (Tree, Bloom, Discovery, Forwarding) and add Handshake, Session,
Mmp, and Transport categories; HandshakeStats, SessionStats, and MmpStats
are new sub-stats.
Wired clusters: tree and MMP outbound sign-failure; the FSP session
unknown-session and state-machine cluster; the Noise IK handshake
state-machine cluster (msg1/msg2); and the decode / crypto / cap /
semantic tail across bloom, discovery, forwarding, mmp, and tree. The
TreeStats::ancestry_invalid counter, present since the scaffold but never
incremented, is now bumped from the validate_semantics ancestry rejection.
Several handshake, MMP, tree, and discovery paths that previously had no
counter at all are now counted, including the send_lookup_response
no-route drop (DiscoveryStats::resp_no_route).
Existing direct counters at the bloom / discovery / forwarding sites are
retained alongside the new dispatch while the rollout is in progress (the
bloom_poison tests expect the transitional +2 delta); a later change
collapses the duplicate increment.
Raise the in-process backpressure headroom in make_test_node_with_mtu
(request an 8 MiB recv_buf_size on UdpConfig and grow packet_channel from
256 to 8192) to reduce localhost-UDP receive overflow under parallel-CPU
scheduler contention, and mark the large-network convergence tests
#[ignore] so cargo test --lib stays green by default. The ignored tests
remain runnable on demand with --ignored or --test-threads=1.
Add per-direction pool_inbound/pool_outbound counters to TcpStats and
TorStats, updated at every pool-insert, receive-loop-exit, transport-stop,
and send-failure removal site. Compare the max_inbound_connections cap
against pool_inbound rather than the combined pool length, so outbound
connect-on-send connections no longer consume the operator-facing inbound
budget. The configuration field name and operator semantics are preserved;
only the cap-check comparison and accounting change.
Move the max_peers cap check in handle_msg1 forward, from the late
check inside promote_connection (which fires after Msg2 has already
been built and put on the wire) to an early position after identity
verification but before index allocation and the Msg2 send. When the
gate fires for a net-new identity, the Msg1 is silent-dropped — no
response goes back to the peer, no AEAD compute or wire bytes are
spent.
Bypass preserved for known peers (reconnect / cross-connection): if
the sender's NodeAddr is already in self.peers, or if a pending
outbound connection is in flight to the same identity, the gate is
skipped so legitimate maintenance traffic continues to work. The
late check inside promote_connection is intentionally retained as
defense-in-depth against future call sites or a disconnect racing
between the early-gate decision and promotion.
Wire-cost rationale: a 45 s tcpdump at saturation observed ~3.6
cap-denials/s steady-state, each previously paying the full Noise IK
responder crypto + Msg2 (~104 B) on the wire before being rejected.
The bigger value is cleaner peer-side semantics — the peer no longer
sees a fake-completed handshake whose data frames subsequently fail
decryption locally.
Two new unit tests cover the cases:
- handle_msg1_silent_drops_at_cap_for_new_peer drives a wire-pumped
Msg1 from a fresh identity into a saturated node and asserts no
Msg2 reaches the sender socket. Stash-verifies as FAIL on the
pre-fix tree (Msg2 hits the wire) and PASS post-fix.
- handle_msg1_admits_existing_peer_at_cap drives a Msg1 from an
identity already in self.peers and asserts the gate does not evict
it. This is a regression check (the no-gate tree behaves the same
way here, but the test guards against an accidental future gate
that breaks known-peer admit).
Brings two structural fixes landed on maint:
- compute_mesh_size: explicit parent skip in the children loop, so the
disjoint-subtree invariant no longer depends on peer_declaration cache
freshness.
- max_peers: outbound connection-initiation gated on the cap (auto-reconnect
retries, Nostr-mediated discovery established adoption, and both sides
of the NAT-traversal punch sequence). Inbound msg1 admission gate
unchanged.
node.limits.max_peers was honored only on inbound msg1 admission
(handshake.rs handle_msg1 returns PeerLimitExceeded when peers.len
is at the cap). Four outbound initiation paths proceeded unconditionally
at capacity: auto-reconnect retries (process_pending_retries),
Nostr-mediated discovery's BootstrapEvent::Established adoption
(poll_nostr_discovery), NAT-traversal punch initiation (the outgoing
side of the offer/answer/punch sequence in the Nostr discovery
runtime), and NAT-traversal punch response (the incoming side of the
same sequence). A saturated node burned CPU, UDP probes, STUN
observations, and Nostr relay traffic on connections that the inbound
gate would reject the moment they reached msg1.
Introduce Node::outbound_admission_check (peers.len < max_peers, or
true when max_peers == 0 as the no-cap sentinel) and gate the four
paths. The discovery runtime lives in a separate task and does not
hold a Node reference; bridge via an Arc<AtomicBool> the runtime
reads and Node refreshes once per tick from outbound_admission_check.
The atomic granularity is intentionally loose: one-tick lag is
acceptable because the inbound msg1 gate continues to be the
authoritative cap, and in-flight handshakes started below the cap
are allowed to complete.
Inbound gate at handshake.rs is unchanged.
The mesh-size estimator's children loop relied on the cached
peer_declaration(parent_id).parent_id() != my_addr check to exclude
the parent. That cached view briefly disagrees with our own latest
my_declaration().parent_id() during the window between a local
parent-switch and the new parent's next inbound TreeAnnounce: the
peer-declaration cache still names us as the parent's parent, so the
parent is iterated as if it were a child and its (typically dominant)
bloom cardinality is added a second time. Symptom: estimated mesh size
displayed in fipsctl show status and fipstop nearly-but-not-exactly
doubles during tree rebalancing.
Make the invariant structural with an explicit peer_addr == parent_id
skip at the head of the children loop. Per-peer 500 ms rate-limiter
and overall recompute cadence are unchanged.
Adds a regression test that constructs the stale-peer-declaration
scenario directly and asserts the parent is not double-counted.
Closes the eventually-consistent gap in spanning-tree state
distribution. Every existing send_tree_announce_to_all call site
gates on a local state-change event (parent switch, self-root
promotion, ancestry change, peer promotion, parent loss). Once a
partition latches — for example a parent-switch announce stranded
in the brief cross-init handshake swap window, where the announce
arrives on a session-index whose decrypt-worker entry has been
unregistered — neither side's state changes again, so neither
side ever re-broadcasts. The existing 60 s check_periodic_parent_reeval
was a re-evaluation, not a re-broadcast: it short-circuited
silently on no-change. Production-side healing depended on
incidental link churn; lab harnesses with stable docker-bridge
links had no equivalent path.
Add a final else branch that fires send_tree_announce_to_all
unconditionally on the no-change path, alongside the existing
switch and self-promote arms. Receivers coalesce by sequence
comparison (ParentDeclaration::is_fresher_than) and short-circuit
at the `if !updated` gate in handle_tree_announce; same-sequence
repeats drop silently with no cascade. The per-peer 500 ms
rate-limiter is well below this 60 s cadence and does not suppress
the heartbeat broadcast.
The fix is a general protocol-robustness improvement: it addresses
any in-flight TreeAnnounce loss class, not only the specific
cross-init swap-window drop site.
testing/static/scripts/rekey-test.sh BASELINE_CONVERGENCE_TIMEOUT
60 -> 65 so a partition healed by the periodic broadcast at T+60
lands inside the convergence window. wait_for_full_baseline
early-exits on PASS, so successful reps see no extra wall-clock.
The cross-connection-won path in handle_msg1 removes the old peer and frees
its allocated index, but does not unregister the old (transport_id, our_index)
cache_key from the decrypt worker pool. The orphan entry sits in the
per-shard HashMap until the index allocator recycles old_idx to a different
peer and that peer's register_decrypt_worker_session call overwrites it.
In the interim, any decrypt job that lands at the recycled cache_key
resolves to the wrong session and AEAD silently fails — observed as
multi-hop routing failure in 5-node static-mesh on next-branch where
bidirectional auto_connect drives cross-connections at every peer pair
on startup.
The tick body's per-peer check_* loops (heartbeats, bloom
announces, MMP reports, tree announces) called transport.send
for every active peer, which on TCP/Tor fell through to a 5 s
connect-on-send wait for any peer whose pool entry was not yet
established. That wedged the entire tick body for the full
connect_timeout_ms per unreachable peer; under post-restart
convergence on a high-peer mesh, this cascaded into multi-
second tick stalls. On master, the same mechanism also starved
the per-tick control-snapshot republish and pushed fipsctl
queries onto an mpsc fallback that was itself queued behind
the wedged rx_loop, producing the 5-second fipsctl head-of-line
pattern operators observed on loaded nodes.
Gate send_encrypted_link_message_with_ce on
transport.connection_state before the send: proceed only when
Connected; on None, kick off a non-blocking background connect
(idempotent — TransportHandle::connect dedupes against the
connecting pool and spawns the timeout-bounded TcpStream::connect
inside its own tokio task) and fail this send fast with a
clear "transport connection not ready" error. A subsequent
tick retries once the pool has an entry. The reconnect
lifecycle (check_link_heartbeats, process_pending_retries,
poll_pending_connects) is unchanged. The connect-on-send
branch in transport.send_async itself remains in place for
code paths that legitimately need synchronous connect (e.g.,
explicit operator-driven fipsctl connect).
When both peers' Nostr-mediated UDP punches complete within the
same scheduling window, each side's `BootstrapEvent::Established`
event arrives with `is_connecting_to_peer` already true: each side
received an inbound msg1 from the peer's pre-punch outbound
attempt, which created a connecting-state record. The deduplication
skip then fires on both sides, neither installs the fresh
traversal socket as canonical, and the peer-adoption budget
(45 s) expires. Cross-node wall-clock alignment of the skip log
line in observed failures was within ~1 ms — simultaneous dual-
fire under contention, the dual-initiation pattern.
Apply the deterministic NodeAddr tie-breaker already used at
`handlers/handshake.rs:269` for rekey dual-initiation and in
`peer::cross_connection_winner` for cross-connection resolution.
Smaller NodeAddr wins as adopter: enumerate the in-flight
connections whose `expected_identity` points at this peer, tear
them down via the canonical `cleanup_stale_connection` helper, and
fall through to `adopt_established_traversal`. Larger NodeAddr
loses and keeps the existing `continue` semantics; the loser's
in-flight outbound is reconciled by `handle_msg1`'s cross-
connection logic when the winner's fresh msg1 arrives over the
adopted socket.
`cleanup_stale_connection` visibility bumped from module-private
to `pub(in crate::node)` so it is callable from `lifecycle.rs`.
The defensive re-check inside `adopt_established_traversal`
itself is left as-is — after the outer cleanup the winner reaches
it with `is_connecting_to_peer == false`, so the inner skip
won't trip. The `BootstrapEvent::Failed` arm is unchanged: there
is no winning outcome on dual failure, and the existing skip +
retry-schedule semantics are correct.
The cross-connection-won branch of `promote_connection` builds a
fresh ActivePeer with a new Noise session and our_index, inserts
it into peers, and registers identity, but did not hand the new
session to the decrypt shard worker pool. The normal-promotion
tail in the same function does make that call. A session
established via the cross-connection race path therefore missed
the worker fast-path for its lifetime, falling back to inline
decryption on the rx loop. Correctness was unaffected, but the
throughput/latency benefit of the worker pool was lost for peerspromoted through that path.
Mirror the normal-promotion tail and call
`register_decrypt_worker_session` after the fresh ActivePeer is
inserted into `self.peers` in the `this_wins` arm.
Logs source, destination, and payload size at the existing no-route
drop site so investigations can attribute transit drops without
enabling trace-level instrumentation. Diagnostic-only; no behavior
change on the success path.
An FSP session rekey could leave the two endpoints holding different
key sets for a brief window: if a handshake message was lost in
transit, one side rotated to the new keys while the other did not.
Traffic sealed in one key epoch then reached a peer still on the
other epoch and failed to decrypt, producing bursts of AEAD
decryption failures and dropped connectivity until a later rekey
cycle reconverged the pair. Choreographing the cutover order cannot
close this window: any fixed ordering still leaves a skew that
packet reordering widens.
Make rekey correctness independent of cutover timing by overlapping
the key epochs on the receive path. During a rekey transition the
receiver trial-decrypts each frame against every live session it
holds: current, the not-yet-promoted pending session, and the
draining previous session. The K-bit becomes a hint that orders the
trial-decrypt cascade rather than a hard gate, and a frame that
authenticates against the pending session is itself the cutover
signal. No rotation ordering and no packet reordering can then cause
a decryption failure.
The pre-rekey Noise session is held in the `previous` slot until the
peer has demonstrably moved off it. Its drain deadline is anchored
on the most recent frame the peer authenticated against that slot,
refreshed each time the trial-decrypt cascade lands there, rather
than on a fixed wall-clock timer started unilaterally at the local
cutover. A peer that never received the new keys keeps authenticating
against `previous` and the slot stays live; without this, a fixed
timer would erase the only key set that could decrypt the peer's
frames, producing a permanent silent decrypt failure on a live data
path. A peer that never catches up is handled by the existing FSP
session liveness path rather than by silent decrypt failure.
The lost-handshake liveness gap is closed separately by retransmitting
the third rekey handshake message until the peer is confirmed on the
new keys, with a bounded retry budget after which the rekey cycle is
cleanly abandoned and retried on the next timer.
Adds unit tests covering the trial-decrypt cascade (epoch selection,
promotion on pending decrypt, reordered old-epoch stragglers after
cutover, per-slot replay-window integrity), the msg3 retransmission
lifecycle, and the peer-progress-aware drain retirement.
Moves both AEAD layers (ChaCha20-Poly1305, one round per layer per
packet) plus the sendmsg syscall off the rx_loop task onto a per-shard
worker pool, adds per-peer connect(2)-ed UDP with SO_REUSEPORT, and
uses Linux UDP GSO (sendmsg+UDP_SEGMENT — kernel splits one super-skb
into N on-the-wire datagrams in a single TX-stack walk) when packets
in a batch are uniform-size. Same kernel primitive WireGuard's
in-kernel module and BoringTun use to hit 2.5–3.2 Gbps single-stream.
Single TCP stream on a 5-node docker-bridge mesh, 5 x 15 s x P=1:
A→D: 1379 → 2708 Mbps (1.96x, RTT +0.12 ms)
A→E: 1394 → 2663 Mbps (1.91x, RTT +0.11 ms)
E→A: 1406 → 2624 Mbps (1.87x, RTT +0.19 ms)
Static-peer pairs only — every CoV under 3%, 0 outliers, 0% ICMP
loss. The ~+100 µs RTT is the worker queue handoff cost; AEAD +
sendmmsg now run on a separate core in exchange.
What lands:
- src/node/encrypt_worker.rs: std::thread + crossbeam_channel
workers; hash-by-destination dispatch pins a TCP flow to one
worker so wire ordering is preserved; per-worker sendmmsg(2)
batching up to 32; Linux uses sendmsg(2)+UDP_SEGMENT when
packets in a group are uniform-size.
- src/node/decrypt_worker.rs: receive-side mirror. Each shard owns
its session's recv cipher + replay window in a thread-local
HashMap (no shared RwLock/Mutex). Sessions are handed off at
promote_connection and re-registered on K-bit flip / rekey
cutover.
- src/node/handlers/session.rs try_send_session_data_pipelined:
FSP+FMP both seal in-place in the worker on one wire-buffer
alloc; no intermediate inner_plaintext / fsp_payload Vecs.
- src/transport/udp/connected_peer.rs + peer_drain.rs: per-peer
connect(2)-ed UDP socket with SO_REUSEPORT (set on the listen
socket too — without that, EADDRINUSE on activation and every
packet falls back to the wildcard path); the worker sends with
msg_name=NULL and the kernel uses its cached 5-tuple. Tick-
driven activation in handlers/connected_udp.rs, idempotent.
- src/transport/udp/mod.rs: mem::replace the recvmmsg backing buffer
instead of buf.to_vec() per packet — single pointer swap, no
MTU-sized memcpy.
- src/protocol/link.rs SessionDatagramRef: zero-copy borrowed view
used by handle_session_datagram for the bulk local-delivery
path; handle_session_payload takes the borrowed payload
directly (no payload[35..].to_vec()).
- src/transport/mod.rs TransportAddr::from_socket_addr: collapses
the two-alloc from_string(addr.to_string()) pattern to one.
- src/node/handlers/rx_loop.rs: decrypt-fallback drain promoted
ahead of packet_rx in the select! (TCP ACK starvation fix);
interleaved fallback drain every 32 packets inside the rx burst
loop.
- noise::Session: send_cipher_clone / recv_cipher_clone /
recv_replay_snapshot_owned / take_send_counter / accept_replay
so off-task workers can hold a cloned cipher + reserved counter
while the dispatcher keeps replay/counter sequencing serial.
CipherState::cipher_clone returns a refcount-bumped LessSafeKey.
AsyncUdpSocket: AsRawFd so workers issue raw sendmmsg / sendmsg
without going through the tokio reactor.
- Worker pool sizing: both default to num_cpus, overridable via
FIPS_ENCRYPT_WORKERS=N / FIPS_DECRYPT_WORKERS=N. Per-peer
connected UDP can be disabled via FIPS_CONNECTED_UDP=0.
- src/perf_profile.rs: optional per-stage timing reporter under
FIPS_PERF=1 (or FIPS_PIPELINE_TRACE=1). Off by default; zero
overhead when disabled.
- All cfg(unix)-gated. Windows continues on the existing tokio-
based send/recv.
Decrypt worker session lifecycle:
- Node::unregister_decrypt_worker_session mirrors the existing
register helper. Wired at the two natural sites that already
iterate peers_by_index: the rekey drain-completion block in
handlers/rekey.rs (drops the worker entry for the old our_index
once the drain window has expired and the cache_key is
unreachable to any in-flight OLD-K packet), and remove_active_peer
in handlers/dispatch.rs (drops the worker entry for each of the
four index slots: current, rekey, pending, previous). Only
our_index is normally registered; unregister_session is fire-
and-forget for missing entries, so calling unconditionally on
all four slots is correct and bounds the cleanup without per-
slot accounting. Without these callers the per-worker sessions
HashMap and the Node's decrypt_registered_sessions set would
grow monotonically per rekey on long-lived peers.
Testing:
- testing/static/scripts/bench-multirun.sh: multi-run iperf3 +
ping bench. N reruns (default 5), median / min / max / CoV % /
per-run outlier flag, avg ping RTT, ICMP loss %, TCP retransmit
total. Plain client→dest labels + topology header. Pre-bench
peer-convergence check (FIPS_BENCH_CONVERGE_SECS, default 15);
per-path route verification via stats.bytes_sent deltas — fails
fast if traffic exits via a non-static-peer link.
- testing/static/docker-compose.yml: passes FIPS_ENCRYPT_WORKERS /
FIPS_DECRYPT_WORKERS / FIPS_PERF through to containers for A/B
benchmarking without rebuilds.
- testing/static/scripts/iperf-test.sh: same plain client→dest
labels + topology header (was multihop/direct/N hop, which
conflated topology distance with on-wire path).
- .config/nextest.toml: synthetic UDP node tests serialized
through a max-threads=1 test group. Localhost handshakes drop
on shared CI runners under parallel load; one-at-a-time keeps
assertions reliable.
- src/node/tests/spanning_tree.rs: repair_missing_edge_handshakes
— retries up to 5 times for synthetic edges whose msg1 was
dropped, with a drain after each edge retry instead of after
each attempt's full burst.
- src/node/decrypt_worker.rs::tests: two unit tests asserting
WorkerMsg::UnregisterSession removes the worker-thread session
HashMap entry (handle_msg_unregister_session_removes_entry) and
is a no-op for never-seen cache_keys
(handle_msg_unregister_session_idempotent_on_unknown_key), which
is the safety invariant the unconditional unregister calls at
the four index slots in remove_active_peer rely on.
- src/node/encrypt_worker.rs::unix_tests
pipelined_send_wire_layout_roundtrips_canonical_decoders: mirrors
the encoder geometry of try_send_session_data_pipelined (no
coords, the common established-session path), runs the worker's
real seal + send via flush_direct_batch_sync, and decodes the
resulting wire packet using only canonical receive-side decoders
(EncryptedHeader::parse, SessionDatagramRef::decode, FSP header
parse, noise::open). Any divergence between the hand-rolled
encoder offsets (fsp_aad_offset, fsp_plaintext_offset) and the
decoders fails at one of the parse / open / decode steps before
the inner-plaintext assertion fires. Complements the existing
fsp_preseal_runs_before_outer_fmp_seal test which covers the
seal-ordering invariant with synthetic headers but does not
exercise the wire-layout invariant.
CHANGELOG.md [Unreleased] # Changed entry added describing the
worker-pool threading model, hash-by-destination dispatch,
sendmmsg/UDP_GSO, per-peer connected UDP, the operator-facing env
vars, and the bench numbers above.
Cherry-picks from mmalmi/master (paths translated from
crates/fips-core/src/ to src/): 9b7c723, 0deb5cb, 13f7339, e036c0e,
3740a68, 3792f83, 8510193, 4910b07, e53f545, e4e2896, 5fe4af5,
1d01ada, 8c37008, e12469e, 6eb2860.
Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
The non-Linux test build was emitting warnings from code that is
intentionally platform-specific: the nftables firewall parser is
Linux-only, the utun address-family helper is only used in macOS
TUN paths, and one macOS Ethernet test module trips a clippy
layout lint. These warnings made focused test runs noisy and
encouraged bundling unrelated warning fixes into behavioral PRs.
- Gate the firewall parser dead-code allowance to non-Linux
targets, where the parser is compiled but not used.
- Mark the macOS utun helper and long TUN reader entry point with
narrow allowances.
- Rewrite the small MAC-copy loop to satisfy clippy and mark the
macOS Ethernet test module layout explicitly.
No runtime behavior change.
Replaces the unconditional `CoordCache::clear()` calls at parent-switch,
become-root, and loop-detection sites with two targeted invalidation
methods scoped to what actually makes an entry stale:
- `invalidate_via_node(node_addr)`: drop entries whose cached
destination ancestry contains `node_addr`. Used at parent-position-
change sites — our prefix changed, so destinations downstream of
us have stale-prefix coords.
- `invalidate_other_roots(current_root)`: drop entries rooted under
a different root than the current one. Used at root-change sites.
Under the previous global flush, parent switches blanked the cache
across the board, leaving `find_next_hop` returning `None` for every
non-direct-peer destination until the cache passively re-warmed via
incoming TreeAnnounces / SessionSetup. Surgical invalidation
preserves entries that remain correct after the topology change.
The cached coord describes a destination's tree position; that
position only goes stale relative to our own routing decisions when
our own prefix changes (entries we are downstream of) or the root
changes (entries in a different tree). Peer removal does not
invalidate cached coords: `Node::find_next_hop` recomputes the
next-hop decision on every call against the current peer set, bloom
filters, and tree state, and Discovery already triggers on
`no route to destination` errors when a destination becomes
unroutable through us. The peer-removal site retains the original
"no cache invalidation" behavior.
Each method returns the count of entries removed for observability.
Unit tests cover each method against the cases enumerated in the
acceptance criterion.
- Borrowed SessionDatagramRef decoder is used in the forwarding
handler so local delivery and coordinate-cache warming no longer
allocate or copy the session payload. The owned SessionDatagram is
materialized only when re-encoding for the next hop.
- Owned SessionDatagram::decode is reimplemented as Ref::decode +
into_owned, so the two decoders cannot drift.
- recvmmsg / recvmsg_x (Linux + macOS) receive loop moves each filled
slot buffer into ReceivedPacket via mem::replace instead of cloning
it; a fresh empty buffer is installed for the next syscall.
- TransportAddr is formatted directly from the SocketAddr without
going through an intermediate String.
Focused decode bench: ref 1.6 ns/op vs owned 34.7 ns/op (21.4x).
End-to-end iperf is neutral as expected for a ~30 ns saving per
packet.
Unit tests added:
- test_session_datagram_ref_decode_borrows_payload (verifies the
payload slice pointer equals the input slice's offset 35, a real
zero-copy invariant guard against accidental future to_vec)
- bench_session_datagram_decode_owned_vs_ref (ignored, run with
--ignored --nocapture)
- test_transport_addr_from_socket_addr
The Linux recv path drains up to 32 datagrams per kernel wakeup via
recvmmsg(2), amortising the per-syscall + per-task-wakeup cost across
the burst. macOS still fell through to single-packet recv_from, so
the same overhead capped inbound rate on Apple builds.
Add an equivalent batch path for Darwin using recvmsg_x(2). It is a
xnu-private syscall (not in the public SDK) but is the canonical
amortisation primitive on macOS — same shape used by quinn-udp for
the same reason. ABI is the public msghdr layout plus a trailing
msg_datalen (per-datagram bytes-received output), declared via
`unsafe extern "C"` against a local repr(C) `msghdr_x`.
Same `(count, kernel_drops)` contract as the Linux `recv_batch`. macOS
has no SO_RXQ_OVFL equivalent, so `kernel_drops` is always 0 — the
1Hz `sample_transport_congestion()` detector simply sees no kernel
drop signal on Apple hosts (it already tolerates that, since the
field has been 0 there pre-batching too).
cmsg buffer is intentionally null: we never consume ancillary data on
this path, and quinn-udp documents that `recvmsg_x` does not overwrite
`msg_controllen` on macOS 10.15+ (zeroed init is the only safe state).
The udp_receive_loop dispatch widens from cfg(linux) to
cfg(any(linux, macos)); the per-packet recv_from path is now used
only on the remaining unix targets (BSDs etc.) and Windows.
Add test_burst_recv_batch exercising 10 in-flight datagrams to
verify per-datagram boundaries and arrival order across the batch.
Add an ignored bench_udp_recv_amortization measuring recv-side
syscall amortization across 1/2/4/8 sender threads on dedicated
blocking std threads (kernel rx queue stays saturated regardless of
tokio scheduling). Sample numbers on aarch64-apple-darwin (100B
payloads, 3s windows):
senders=1: recv_from 398k pps recv_batch 432k pps 1.09x
senders=2: recv_from 353k pps recv_batch 608k pps 1.72x
senders=4: recv_from 322k pps recv_batch 503k pps 1.56x
senders=8: recv_from 353k pps recv_batch 515k pps 1.46x
Gate the Linux-only IpAddr import in control::listening behind a
cfg(target_os = "linux") so the macOS test build is warning-clean
now that test code paths there compile.
Skip BootstrapEvent::Established and BootstrapEvent::Failed dispatch
in poll_nostr_discovery for peers that are already connected or
actively handshaking. Without these guards, stale traversal events
arriving after a peer connected through a different path would
either attempt to adopt a redundant socket against the live
connection (Established) or poison the per-peer failure-state
cooldown and trigger redundant retraversal via schedule_retry /
try_peer_addresses (Failed).
The four guard sites use a new is_connecting_to_peer helper extracted
from the existing closure inside initiate_peer_connection; the helper
checks for an in-flight outbound handshake state. adopt_established_traversal
gains a defense-in-depth check returning PeerAlreadyExists when called
against an already-promoted peer, so the invariant holds if a future
caller bypasses the outer dispatch guard.
Side benefit: narrows a cooldown-poisoning vector previously available
to an attacker injecting stale failure events for an active peer.
Test coverage for the new behavior:
- test_try_peer_addresses_skips_connected_peer
- test_try_peer_addresses_skips_connecting_peer
- test_nostr_traversal_failure_skips_connected_peer (Failed-arm event
injection)
- test_nostr_traversal_established_skips_connected_peer (Established-arm
event injection, mirror of the Failed test)
- test_adopted_traversal_skips_already_connected_peer
(adopt_established_traversal defense-in-depth)
CHANGELOG entry under [Unreleased] / Fixed.
Closes#87
Johnathan Corgan
Martti MalmiandJohnathan Corgan
