mirror of
https://github.com/jmcorgan/fips.git
synced 2026-08-09 08:14:42 +00:00
989ae65fc5114425327c82c97d44879bea10afc8
123
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Johnathan Corgan
|
08a226fb63 |
Test cost-based parent selection and kernel-drop detection as unit tests
The cost-selection chaos scenarios (cost-reeval, cost-avoidance, cost-stability, depth-vs-cost, mixed-technology, bottleneck-parent) tested TreeState::evaluate_parent's decision logic through a Docker mesh that could not exercise it reliably: the tree roots at whichever node holds the smallest NodeAddr, MMP link costs take several measurement windows to settle, and the parent hold-down plus hysteresis timing all confound the outcome. A deterministic link-cost flap still produced zero periodic parent switches in a full run. Replace those six scenarios with deterministic unit tests in src/tree/tests.rs that drive evaluate_parent directly: cheaper-link selection at equal depth, switch-on-cost-change, hysteresis suppressing a marginal change while allowing a significant one, and the depth-versus-cost effective-depth tradeoff. Each is constructed so that breaking the cost or hysteresis logic makes it fail. The congestion kernel-drop signal (SO_RXQ_OVFL) cannot be provoked deterministically in Docker: a fresh daemon reader keeps up with container-speed traffic, so the socket receive queue never overflows (an unshaped run with a 4 KB buffer and heavy traffic recorded zero drops on every node). Extract the drop-detection edge -- read the cumulative counter, fire an event only on the transition into a new drop burst -- into TransportDropState::observe_drops and unit-test it directly. congestion-stress keeps its ECN and MMP congestion-signal assertions, which do need the real shaped bottleneck queue. Remove the retired scenarios from both CI runners and update the chaos README. |
||
|
Johnathan Corgan
|
8ca2362e6c |
Follow IP semantics for the session datagram hop limit
The forwarding path tested the hop limit before testing whether the datagram was addressed to this node, and decremented only on the forwarding branch. Two consequences followed. A datagram addressed here that arrived with a zero hop limit was dropped rather than delivered. And a forwarder receiving a transit datagram with one hop left transmitted it with zero for the next hop to discard, wasting a transmission on every expiring datagram. Delivery to the addressed node is no longer gated on the hop limit, and the decrement now happens before the drop decision, so a datagram that would leave with zero is not sent. Saturating subtraction folds the two arrivals that cannot be transmitted, already exhausted and last hop, into a single test. The reachable radius does not change, which is worth stating because it is easy to assume otherwise. The old behavior refused to deliver at zero but was willing to send at zero, and the new behavior is the mirror of that, so the two cancel: a path of h links still delivers for any source value of h or more. What this actually buys is the wasted transmission, the exhaustion counter charging at the node that makes the decision rather than the one after it, conformance with the specified semantics, and one real gain during a rolling upgrade, where an unupgraded forwarder feeding an upgraded destination delivers one hop further than either version does on its own. Coordinate cache warming moved ahead of both decisions, so a transit datagram later dropped for hop limit still contributes its plaintext coordinates. The only arrivals this newly warms from are those with a zero hop limit, and every insert they can make is already achievable with a value of one, so it grants nothing that was not already available and adds no memory growth the cap does not bound. Four tests added: delivery at zero to this node, the transit drop at one, the transit pass at two, and the one per hop decrement across a three node chain, which brackets the emitted value from both sides since it is only observable through what the next hop does. One existing test was renamed because its destination was this node, so it never exercised the transit path its name claimed, and it asserted nothing at all. |
||
|
Johnathan Corgan
|
6011d233c1 |
discovery: keep tighter path_mtu when applying a LookupResponse
An originator handling a LookupResponse unconditionally overwrote the cached path_mtu_lookup entry, so a looser (larger) estimate in a later response could clobber a tighter value already learned from a reactive MtuExceeded or PathMtuNotification. Read-and-compare before writing and keep the minimum, so a looser discovery estimate no longer loosens the clamp. Add a regression test. |
||
|
Johnathan Corgan
|
cb5a32693e |
tree: remove redundant parent_switched metric counter
parent_switched was incremented on the line immediately before parent_switches at every site and never independently, so the two counters were always identical. Drop parent_switched from TreeMetrics, its snapshot, TreeStatsSnapshot, the show_tree fixture, and the fipstop render, keeping parent_switches as the sole counter. |
||
|
Johnathan Corgan
|
274b09d4ff |
node: classify transit forwards by route class; regroup fipstop routing tab
Add six forwarding counters that partition transit-forwarded packets by their tree relationship to the chosen next hop: tree-up (peer is our ancestor), tree-down (peer is our descendant and the destination is within its subtree), tree-down-cross (peer is our descendant but the destination is outside its subtree), cross-link descend (lateral peer, destination within its subtree), cross-link ascend (lateral peer, destination outside its subtree), and direct-peer. The six classes sum to forwarded_packets, asserted by a unit test. Classification is computed from tree coordinates at the transit chokepoint, so the error-signal routing callers are excluded. The two "outside the chosen peer's subtree" classes are both up-and-over forwards but differ in what they depend on. Tree-down-cross is the dive-to-tree-child cut-through: we forward down to our own child for a destination not beneath it, which is only possible because the child advertised cross-link reach upward to us, beyond its own subtree. Its count measures how much forwarding depends on that upward advertisement, i.e. what would change if cross-link advertisements were narrowed to subtree-entry only. Cross-link ascend, by contrast, uses the node's own lateral cross-link learned from a peer's split-horizon advertisement, so it does not depend on any upward advertisement. Surface the counters through the forwarding stats snapshot (control socket, show_routing and show_status) and reorganize the fipstop routing tab so its two columns separate own/endpoint traffic (received, delivered, originated) from forwarded/transit traffic (the route-class breakdown and drop reasons), with the tree-down-cross line visually flagged. |
||
|
Johnathan Corgan
|
a4802ccf9e | Merge maint into master: demote rekey-abort logs to debug | ||
|
Johnathan Corgan
|
7a74fa8ca2 |
rekey: demote exhausted-budget rekey-abort logs from warn to debug
When an FMP msg1 or FSP msg3 rekey retransmission budget is exhausted, the cycle is abandoned and retried on the next timer. On lossy or high-latency links this is an expected, self-limiting outcome: the existing session stays valid and keeps carrying traffic, so the abort is not a failure that warrants operator-level visibility. Demote both abandon-cycle messages from warn to debug to cut steady-state log noise on nodes with many flapping peers. |
||
|
Johnathan Corgan
|
81cd10d5db |
control: serve the full show_* read surface off the rx_loop via a read-snapshot plane
Complete the control-plane read-isolation work: every pure-read show_* query now renders in the control accept task from published read snapshots, so none round-trips the data-plane receive loop. Only the mutating connect/disconnect commands still reach that loop. Three subsystem snapshots are published via ArcSwap and served through the read handle's snapshot_dispatch: - A routing read view (spanning tree, bloom filters, coordinate cache, identity cache, and the discovery F-queue summary scalars), published from the tick, serving show_tree/show_bloom/show_cache/show_routing/ show_identity_cache. - A per-entity read view (peers, sessions, links, connections, transports, and the MMP link/session views) as Vec<Arc<Row>> tables reconciled against the prior snapshot so a republish reuses unchanged rows by pointer and re-allocates only changed or new rows, keeping the per-tick publish cost bounded as the peer/session count grows. Serves show_peers/show_sessions/show_links/show_connections/show_transports/ show_mmp. - The stats snapshot is extended with the peer-ACL status and a per-peer metadata map (is_active, npub, display name), resolved at publish time, serving show_acl and the two per-peer stats queries. Display names and other cross-subsystem fields are resolved at publish time; time-relative fields are derived at render time from captured absolute timestamps, so rendered output is byte-identical to the prior on-loop handlers, which are retained as the equality oracle. With every read query served off-loop, the show_* branch is removed from the rx_loop control handler and the now-dead on-loop dispatcher deleted. The snapshot projections are forward-compatible with the later structural extraction of the derived-state and session tables: they become thin views over the extracted types without changing the read-handle interface. |
||
|
Johnathan Corgan
|
c77e564462 |
control: serve high-traffic show_* queries off the rx_loop hot path
Introduce a read-snapshot plane so pure-snapshot control queries render in the control-socket task instead of round-tripping the rx_loop, removing the head-of-line coupling that let a busy or slow rx_loop time out fipsctl and fipstop observability. - ControlReadHandle: a cloneable bundle the control accept loop holds, over the node's already-shared NodeContext and MetricsRegistry plus an ArcSwap-published StatsSnapshot. A snapshot_dispatch seam serves cut-over commands off-loop and falls through to the rx_loop for the rest, keeping the rx_loop's ownership of Node intact. - StatsSnapshot is published from the tick (the natural and sole mutator of stats_history), carrying the history rings plus the scalar gauges and counts show_status reports. Readers serve the latest snapshot unconditionally, with staleness bounded by the tick interval and no IO_TIMEOUT-coupled fallback. - Off-loop now: show_status, show_stats_history, show_stats_all_history, show_listening_sockets, show_stats_list, and a new counter-only show_metrics (exposed as fipsctl "stats metrics", the enabler for a Prometheus scraper at no hot-path cost). Queries that need live per-entity state (peers, links, sessions, routing, and the per-peer stats variants) stay on the rx_loop path pending later phases. Quartet green; forward-merge to next verified clean. |
||
|
Johnathan Corgan
|
79b945b93d |
Merge maint into master (mesh-size OR-union estimate; log-level hygiene)
Brings the maint-line OR-union mesh-size estimator fix and the per-peer / capacity-cap log-level demotions forward to master. Conflict resolution: compute_mesh_size resolved to master's config() accessor form carrying maint's OR-union rewrite (drop the stale summing `total`); the log-level demotions auto-merged into master's handler versions. The equivalent master-only log "connected UDP socket installed" (connected_udp.rs, a file that does not exist on maint) was demoted info -> debug as part of this merge so the connected-UDP path matches the rest of the per-peer lifecycle logging. |
||
|
Johnathan Corgan
|
974e146bb9 |
node: demote routine per-peer and capacity-cap events from info/warn to debug
On a saturated public-mesh node the connection-lifecycle and capacity-cap events fire continuously and drown out the genuinely notable INFO/WARN lines. Demote them to debug and drop a redundant duplicate: - FMP K-bit cutover promotion (encrypted): info -> debug - "Connection promoted to active peer" (handshake): info -> debug, and remove the duplicate "Inbound peer promoted to active" line that shadowed it on the inbound path - "Peer restart detected" (handshake): info -> debug - "Peer removed and state cleaned up" (dispatch): info -> debug - "Rejecting inbound TCP connection (max_inbound_connections reached)" (tcp): warn -> debug - "Congestion detected, CE flag set on forwarded packet" (forwarding): warn -> debug - "Removing peer: link dead timeout" (mmp): warn -> debug These are expected, high-frequency conditions on a busy public node (new and reconnecting peers, ECN CE marking, the inbound connection cap, and link-dead churn), not operator-actionable signals. |
||
|
Johnathan Corgan
|
4af3730be6 | fmp: authenticate inbound frame against pending session before K-bit cutover promotion | ||
|
Johnathan Corgan
|
36c830edfd | fmp: authenticate inbound frame against pending session before K-bit cutover promotion | ||
|
Johnathan Corgan
|
22a41cb1a0 | Merge maint into master (FMP rekey msg1 resend cap + rekey-aware link-dead heartbeat) | ||
|
Johnathan Corgan
|
25fe87ff60 |
fmp: bound rekey msg1 retransmission and make link-dead heartbeat rekey-aware
The FMP rekey msg1 resend driver retransmitted indefinitely with no cap and no abandon, so a rekey that never completed kept resending msg1 forever. Give it a retransmission budget: cap resends at handshake_max_resends with exponential backoff and abandon the rekey cycle cleanly once the budget is exhausted, mirroring the FSP session rekey msg3 driver. With the cap in place the link-dead heartbeat can safely become rekey-aware: check_link_heartbeats now suppresses teardown while a rekey is in progress with msg1 budget remaining, instead of reaping a link that is still actively carrying rekey-handshake traffic. The suppression terminates deterministically (the budget abandons on exhaustion, cutover clears the in-progress flag), so a genuinely dead link is still reaped on the next cycle. Adds a rekey_msg1_resend_count counter on ActivePeer reset at every rekey-clear and cutover site, msg1 resend-budget unit tests, and two-node heartbeat suppression/resume/regression integration tests. |
||
|
Johnathan Corgan
|
d9a4a7807c |
node: make the shared context the sole store of immutable state
Remove the duplicated immutable fields (config, identity, startup_epoch, started_at, is_leaf_only, max_connections/peers/links) from the Node struct so the Arc<NodeContext> bundle is the single source of truth. Previously Node owned these fields and a parallel context copy, kept in lockstep by rebuild_context() at every mutation site — pure overhead that existed only because of the duplication. - Replace rebuild_context() with replace_context(): a clone-edit-swap of the whole Arc. The per-instance context stays immutable; mutation swaps the Arc. This is the sole runtime mutation path (constructors, leaf_only, update_peers). - Add Copy-returning accessors startup_epoch() and max_connections()/ max_peers()/max_links(); migrate the remaining direct field readers onto the accessors. node_addr()/npub()/Debug now read identity/is_leaf_only from the context. - update_peers reads the pre-update peer set from the live context Arc before building a fresh Config + context and swapping — preserving the read-before-write ordering its mutation-window test depends on. - Remove the test-only set_max_* setters; tests set the limits on Config at construction instead (new make_node_with_max_peers/links helpers). - Add a ci-local guard that fails if the Node struct re-declares a bundled field, so the single-store invariant can't silently regress. cargo test --lib 1291/0; clippy -D warnings and release build clean. |
||
|
Johnathan Corgan
|
08b8b3908e |
node: extract immutable state into a shared context and atomic metric registry
Store node counters in an atomic metric registry read through &self, and introduce a shared NodeContext bundle holding the effectively-immutable fields (config, identity, startup epoch, capability limits). Source the immutable config and identity reads across the receive hot path, the handshake/session/mmp/encrypted state machines, and the discovery, tree, bloom, retry, and lifecycle modules through the context accessors rather than direct field reads. The Node fields and the context are rebuilt in lockstep at every mutation site. |
||
|
Johnathan Corgan
|
53c6c78721 |
discovery: count dropped requests when the dedup cache is full
The discovery request dedup cache (recent_requests) silently dropped LookupRequests once it reached MAX_RECENT_DISCOVERY_REQUESTS, with no counter to surface the condition. Add a DiscoveryReject::ReqDedupCacheFull reject reason backed by a req_dedup_cache_full counter on DiscoveryStats, mirroring the existing duplicate-request counter, and record it at the drop site so the rejection is visible in show_routing. |
||
|
Johnathan Corgan
|
3c5d9fd4f2 |
Merge master into refactor-hotpath
Bring the runtime peer-list refresh and opt-in mDNS LAN discovery work on master into the receive-path RejectReason / reloadable-config integration branch. Code files auto-merge clean; the only conflict is the CHANGELOG Unreleased section, resolved as the union of both sets of entries. |
||
 Martti MalmiandJohnathan Corgan
|
7d7b551ca1 |
discovery: add opt-in mDNS LAN discovery
Add scoped mDNS / DNS-SD discovery for peers on the same local link, giving sub-second pairing without a relay or NAT-traversal roundtrip. A node advertises its npub, protocol version, and an optional network scope over link-local multicast, and browses for matching adverts to initiate Noise handshakes against same-LAN peers. LAN discovery is disabled by default; operators enable it with node.discovery.lan.enabled: true. Default-off avoids reintroducing a per-LAN identity broadcast on nodes that have deliberately disabled other discovery channels, and avoids any multicast surprise on upgrade. The startup advertised-port picker now excludes bootstrap transports and selects a non-bootstrap operational UDP transport with a stable lowest-id selector, so the advertised port is deterministic across restarts rather than dependent on HashMap iteration order. This matches the per-dial transport selection used for discovered peers. Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com> |
||
|
Martti MalmiandJohnathan Corgan
|
da0d9d39a0 |
node: refresh active peer paths without dropping links
Add Node::update_peers for runtime peer-list refresh. It re-derives the active peer connections from a new peer configuration, adding newly configured peers and removing those no longer present, while keeping links to peers that remain in the set rather than tearing every connection down. The call returns an UpdatePeersOutcome summarizing the added, removed, and retained peers. PeerAddress gains a seen_at_ms recency field (with_seen_at_ms). Active path selection now sorts address candidates by recency so the most recently observed address wins when concurrent path probes race. complete_rekey_msg2 now returns the remote peer's startup epoch alongside the new Noise session, letting the rekey path detect a peer restart and clear stale session state. A stale FSP session is cleared when a peer restart is detected during FMP rekey or cross-connection promotion, so the session-layer map no longer lingers out of sync with the freshly promoted peer. Per-tick work budgets bound the connection churn in a single node tick (MAX_DISCOVERY_CONNECTS_PER_TICK, MAX_RETRY_CONNECTIONS_PER_TICK, MAX_PARALLEL_PATH_CANDIDATES_PER_PEER); work beyond a tick's budget is deferred to the next tick rather than discarded. Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com> |
||
|
Johnathan Corgan
|
d672ed865f |
node: migrate peer ACL to the Reloadable trait and hot-reload the host map
Move PeerAclReloader onto the Reloadable trait: its ACL snapshot is now published through an arc_swap::ArcSwap so the authorization hot path reads it without locking, and the former check_reload becomes the trait's reload(). The node tick calls self.peer_acl.reload().await. Wire the host map into the tick as well. The host map snapshot was previously taken once at construction and never polled; it now hot-reloads on /etc/fips/hosts mtime changes once per tick, alongside the ACL, so hostname display reflects edits without a restart. The path_mtu_lookup cache (event-driven, populated from observed traffic) and the nostr_discovery subsystem (an async spawned task) are deliberately left off the trait: neither reloads from a backing file, so a no-op reload() would be misleading. The rationale is documented on the trait module. The host map and the ACL's embedded alias reloader still stat /etc/fips/hosts independently each tick. A single small-file stat per tick is cheap, so the duplicate is left in place; sharing one mtime observation between the two is a possible future cleanup. Tests: a node-level test exercises the host-map tick reload end to end through peer_display_name; the ACL reloader tests are updated to drive the async reload(). |
||
|
Johnathan Corgan
|
66732e89c1 |
node: route receive-path silent-rejection sites through typed RejectReason counters
Introduce a typed RejectReason enum and a NodeStats::record_reject dispatch so every receive-path rejection-and-return site bumps a machine-readable per-subsystem counter while keeping its operator-facing log line. The top-level variants mirror the existing NodeStats subsystem split (Tree, Bloom, Discovery, Forwarding) and add Handshake, Session, Mmp, and Transport categories; HandshakeStats, SessionStats, and MmpStats are new sub-stats. Wired clusters: tree and MMP outbound sign-failure; the FSP session unknown-session and state-machine cluster; the Noise IK handshake state-machine cluster (msg1/msg2); and the decode / crypto / cap / semantic tail across bloom, discovery, forwarding, mmp, and tree. The TreeStats::ancestry_invalid counter, present since the scaffold but never incremented, is now bumped from the validate_semantics ancestry rejection. Several handshake, MMP, tree, and discovery paths that previously had no counter at all are now counted, including the send_lookup_response no-route drop (DiscoveryStats::resp_no_route). Existing direct counters at the bloom / discovery / forwarding sites are retained alongside the new dispatch while the rollout is in progress (the bloom_poison tests expect the transitional +2 delta); a later change collapses the duplicate increment. |
||
|
Johnathan Corgan
|
f6429c19d2 | Merge maint into master (admission-gate Msg2 silent-drop + integration suite) | ||
|
Johnathan Corgan
|
5b229c03bf |
node: skip Msg1 → Msg2 reply when at max_peers cap
Move the max_peers cap check in handle_msg1 forward, from the late check inside promote_connection (which fires after Msg2 has already been built and put on the wire) to an early position after identity verification but before index allocation and the Msg2 send. When the gate fires for a net-new identity, the Msg1 is silent-dropped — no response goes back to the peer, no AEAD compute or wire bytes are spent. Bypass preserved for known peers (reconnect / cross-connection): if the sender's NodeAddr is already in self.peers, or if a pending outbound connection is in flight to the same identity, the gate is skipped so legitimate maintenance traffic continues to work. The late check inside promote_connection is intentionally retained as defense-in-depth against future call sites or a disconnect racing between the early-gate decision and promotion. Wire-cost rationale: a 45 s tcpdump at saturation observed ~3.6 cap-denials/s steady-state, each previously paying the full Noise IK responder crypto + Msg2 (~104 B) on the wire before being rejected. The bigger value is cleaner peer-side semantics — the peer no longer sees a fake-completed handshake whose data frames subsequently fail decryption locally. Two new unit tests cover the cases: - handle_msg1_silent_drops_at_cap_for_new_peer drives a wire-pumped Msg1 from a fresh identity into a saturated node and asserts no Msg2 reaches the sender socket. Stash-verifies as FAIL on the pre-fix tree (Msg2 hits the wire) and PASS post-fix. - handle_msg1_admits_existing_peer_at_cap drives a Msg1 from an identity already in self.peers and asserts the gate does not evict it. This is a regression check (the no-gate tree behaves the same way here, but the test guards against an accidental future gate that breaks known-peer admit). |
||
|
Johnathan Corgan
|
00bd849ee1 |
node: unregister old decrypt-worker entry on cross-connection-won promotion
The cross-connection-won path in handle_msg1 removes the old peer and frees its allocated index, but does not unregister the old (transport_id, our_index) cache_key from the decrypt worker pool. The orphan entry sits in the per-shard HashMap until the index allocator recycles old_idx to a different peer and that peer's register_decrypt_worker_session call overwrites it. In the interim, any decrypt job that lands at the recycled cache_key resolves to the wrong session and AEAD silently fails — observed as multi-hop routing failure in 5-node static-mesh on next-branch where bidirectional auto_connect drives cross-connections at every peer pair on startup. |
||
|
Johnathan Corgan
|
5dfbd05fe8 | Merge maint into master (cross-init NAT-traversal tie-breaker) | ||
|
Johnathan Corgan
|
f396d71826 |
node: deterministic tie-breaker for cross-init NAT traversal adoption
When both peers' Nostr-mediated UDP punches complete within the same scheduling window, each side's `BootstrapEvent::Established` event arrives with `is_connecting_to_peer` already true: each side received an inbound msg1 from the peer's pre-punch outbound attempt, which created a connecting-state record. The deduplication skip then fires on both sides, neither installs the fresh traversal socket as canonical, and the peer-adoption budget (45 s) expires. Cross-node wall-clock alignment of the skip log line in observed failures was within ~1 ms — simultaneous dual- fire under contention, the dual-initiation pattern. Apply the deterministic NodeAddr tie-breaker already used at `handlers/handshake.rs:269` for rekey dual-initiation and in `peer::cross_connection_winner` for cross-connection resolution. Smaller NodeAddr wins as adopter: enumerate the in-flight connections whose `expected_identity` points at this peer, tear them down via the canonical `cleanup_stale_connection` helper, and fall through to `adopt_established_traversal`. Larger NodeAddr loses and keeps the existing `continue` semantics; the loser's in-flight outbound is reconciled by `handle_msg1`'s cross- connection logic when the winner's fresh msg1 arrives over the adopted socket. `cleanup_stale_connection` visibility bumped from module-private to `pub(in crate::node)` so it is callable from `lifecycle.rs`. The defensive re-check inside `adopt_established_traversal` itself is left as-is — after the outer cleanup the winner reaches it with `is_connecting_to_peer == false`, so the inner skip won't trip. The `BootstrapEvent::Failed` arm is unchanged: there is no winning outcome on dual failure, and the existing skip + retry-schedule semantics are correct. |
||
|
Johnathan Corgan
|
de78c94d58 |
node: register decrypt worker on cross-connection-won promotion
The cross-connection-won branch of `promote_connection` builds a fresh ActivePeer with a new Noise session and our_index, inserts it into peers, and registers identity, but did not hand the new session to the decrypt shard worker pool. The normal-promotion tail in the same function does make that call. A session established via the cross-connection race path therefore missed the worker fast-path for its lifetime, falling back to inline decryption on the rx loop. Correctness was unaffected, but the throughput/latency benefit of the worker pool was lost for peerspromoted through that path. Mirror the normal-promotion tail and call `register_decrypt_worker_session` after the fresh ActivePeer is inserted into `self.peers` in the `this_wins` arm. |
||
|
Johnathan Corgan
|
050483f3bf |
forwarding: log no-route SessionDatagram drops at debug level
Logs source, destination, and payload size at the existing no-route drop site so investigations can attribute transit drops without enabling trace-level instrumentation. Diagnostic-only; no behavior change on the success path. |
||
|
Johnathan Corgan
|
3fc0178192 | Merge maint into master (FSP rekey overlapping-epoch, drain-erase fix) | ||
|
Johnathan Corgan
|
6e5cb8965f |
Make FSP session rekey hitless under packet loss and reordering
An FSP session rekey could leave the two endpoints holding different key sets for a brief window: if a handshake message was lost in transit, one side rotated to the new keys while the other did not. Traffic sealed in one key epoch then reached a peer still on the other epoch and failed to decrypt, producing bursts of AEAD decryption failures and dropped connectivity until a later rekey cycle reconverged the pair. Choreographing the cutover order cannot close this window: any fixed ordering still leaves a skew that packet reordering widens. Make rekey correctness independent of cutover timing by overlapping the key epochs on the receive path. During a rekey transition the receiver trial-decrypts each frame against every live session it holds: current, the not-yet-promoted pending session, and the draining previous session. The K-bit becomes a hint that orders the trial-decrypt cascade rather than a hard gate, and a frame that authenticates against the pending session is itself the cutover signal. No rotation ordering and no packet reordering can then cause a decryption failure. The pre-rekey Noise session is held in the `previous` slot until the peer has demonstrably moved off it. Its drain deadline is anchored on the most recent frame the peer authenticated against that slot, refreshed each time the trial-decrypt cascade lands there, rather than on a fixed wall-clock timer started unilaterally at the local cutover. A peer that never received the new keys keeps authenticating against `previous` and the slot stays live; without this, a fixed timer would erase the only key set that could decrypt the peer's frames, producing a permanent silent decrypt failure on a live data path. A peer that never catches up is handled by the existing FSP session liveness path rather than by silent decrypt failure. The lost-handshake liveness gap is closed separately by retransmitting the third rekey handshake message until the peer is confirmed on the new keys, with a bounded retry budget after which the rekey cycle is cleanly abandoned and retried on the next timer. Adds unit tests covering the trial-decrypt cascade (epoch selection, promotion on pending decrypt, reordered old-epoch stragglers after cutover, per-slot replay-window integrity), the msg3 retransmission lifecycle, and the peer-progress-aware drain retirement. |
||
|
Martti MalmiandJohnathan Corgan
|
0a5c367edc |
data-plane perf overhaul: off-task encrypt + decrypt, GSO, connected UDP
Moves both AEAD layers (ChaCha20-Poly1305, one round per layer per packet) plus the sendmsg syscall off the rx_loop task onto a per-shard worker pool, adds per-peer connect(2)-ed UDP with SO_REUSEPORT, and uses Linux UDP GSO (sendmsg+UDP_SEGMENT — kernel splits one super-skb into N on-the-wire datagrams in a single TX-stack walk) when packets in a batch are uniform-size. Same kernel primitive WireGuard's in-kernel module and BoringTun use to hit 2.5–3.2 Gbps single-stream. Single TCP stream on a 5-node docker-bridge mesh, 5 x 15 s x P=1: A→D: 1379 → 2708 Mbps (1.96x, RTT +0.12 ms) A→E: 1394 → 2663 Mbps (1.91x, RTT +0.11 ms) E→A: 1406 → 2624 Mbps (1.87x, RTT +0.19 ms) Static-peer pairs only — every CoV under 3%, 0 outliers, 0% ICMP loss. The ~+100 µs RTT is the worker queue handoff cost; AEAD + sendmmsg now run on a separate core in exchange. What lands: - src/node/encrypt_worker.rs: std::thread + crossbeam_channel workers; hash-by-destination dispatch pins a TCP flow to one worker so wire ordering is preserved; per-worker sendmmsg(2) batching up to 32; Linux uses sendmsg(2)+UDP_SEGMENT when packets in a group are uniform-size. - src/node/decrypt_worker.rs: receive-side mirror. Each shard owns its session's recv cipher + replay window in a thread-local HashMap (no shared RwLock/Mutex). Sessions are handed off at promote_connection and re-registered on K-bit flip / rekey cutover. - src/node/handlers/session.rs try_send_session_data_pipelined: FSP+FMP both seal in-place in the worker on one wire-buffer alloc; no intermediate inner_plaintext / fsp_payload Vecs. - src/transport/udp/connected_peer.rs + peer_drain.rs: per-peer connect(2)-ed UDP socket with SO_REUSEPORT (set on the listen socket too — without that, EADDRINUSE on activation and every packet falls back to the wildcard path); the worker sends with msg_name=NULL and the kernel uses its cached 5-tuple. Tick- driven activation in handlers/connected_udp.rs, idempotent. - src/transport/udp/mod.rs: mem::replace the recvmmsg backing buffer instead of buf.to_vec() per packet — single pointer swap, no MTU-sized memcpy. - src/protocol/link.rs SessionDatagramRef: zero-copy borrowed view used by handle_session_datagram for the bulk local-delivery path; handle_session_payload takes the borrowed payload directly (no payload[35..].to_vec()). - src/transport/mod.rs TransportAddr::from_socket_addr: collapses the two-alloc from_string(addr.to_string()) pattern to one. - src/node/handlers/rx_loop.rs: decrypt-fallback drain promoted ahead of packet_rx in the select! (TCP ACK starvation fix); interleaved fallback drain every 32 packets inside the rx burst loop. - noise::Session: send_cipher_clone / recv_cipher_clone / recv_replay_snapshot_owned / take_send_counter / accept_replay so off-task workers can hold a cloned cipher + reserved counter while the dispatcher keeps replay/counter sequencing serial. CipherState::cipher_clone returns a refcount-bumped LessSafeKey. AsyncUdpSocket: AsRawFd so workers issue raw sendmmsg / sendmsg without going through the tokio reactor. - Worker pool sizing: both default to num_cpus, overridable via FIPS_ENCRYPT_WORKERS=N / FIPS_DECRYPT_WORKERS=N. Per-peer connected UDP can be disabled via FIPS_CONNECTED_UDP=0. - src/perf_profile.rs: optional per-stage timing reporter under FIPS_PERF=1 (or FIPS_PIPELINE_TRACE=1). Off by default; zero overhead when disabled. - All cfg(unix)-gated. Windows continues on the existing tokio- based send/recv. Decrypt worker session lifecycle: - Node::unregister_decrypt_worker_session mirrors the existing register helper. Wired at the two natural sites that already iterate peers_by_index: the rekey drain-completion block in handlers/rekey.rs (drops the worker entry for the old our_index once the drain window has expired and the cache_key is unreachable to any in-flight OLD-K packet), and remove_active_peer in handlers/dispatch.rs (drops the worker entry for each of the four index slots: current, rekey, pending, previous). Only our_index is normally registered; unregister_session is fire- and-forget for missing entries, so calling unconditionally on all four slots is correct and bounds the cleanup without per- slot accounting. Without these callers the per-worker sessions HashMap and the Node's decrypt_registered_sessions set would grow monotonically per rekey on long-lived peers. Testing: - testing/static/scripts/bench-multirun.sh: multi-run iperf3 + ping bench. N reruns (default 5), median / min / max / CoV % / per-run outlier flag, avg ping RTT, ICMP loss %, TCP retransmit total. Plain client→dest labels + topology header. Pre-bench peer-convergence check (FIPS_BENCH_CONVERGE_SECS, default 15); per-path route verification via stats.bytes_sent deltas — fails fast if traffic exits via a non-static-peer link. - testing/static/docker-compose.yml: passes FIPS_ENCRYPT_WORKERS / FIPS_DECRYPT_WORKERS / FIPS_PERF through to containers for A/B benchmarking without rebuilds. - testing/static/scripts/iperf-test.sh: same plain client→dest labels + topology header (was multihop/direct/N hop, which conflated topology distance with on-wire path). - .config/nextest.toml: synthetic UDP node tests serialized through a max-threads=1 test group. Localhost handshakes drop on shared CI runners under parallel load; one-at-a-time keeps assertions reliable. - src/node/tests/spanning_tree.rs: repair_missing_edge_handshakes — retries up to 5 times for synthetic edges whose msg1 was dropped, with a drain after each edge retry instead of after each attempt's full burst. - src/node/decrypt_worker.rs::tests: two unit tests asserting WorkerMsg::UnregisterSession removes the worker-thread session HashMap entry (handle_msg_unregister_session_removes_entry) and is a no-op for never-seen cache_keys (handle_msg_unregister_session_idempotent_on_unknown_key), which is the safety invariant the unconditional unregister calls at the four index slots in remove_active_peer rely on. - src/node/encrypt_worker.rs::unix_tests pipelined_send_wire_layout_roundtrips_canonical_decoders: mirrors the encoder geometry of try_send_session_data_pipelined (no coords, the common established-session path), runs the worker's real seal + send via flush_direct_batch_sync, and decodes the resulting wire packet using only canonical receive-side decoders (EncryptedHeader::parse, SessionDatagramRef::decode, FSP header parse, noise::open). Any divergence between the hand-rolled encoder offsets (fsp_aad_offset, fsp_plaintext_offset) and the decoders fails at one of the parse / open / decode steps before the inner-plaintext assertion fires. Complements the existing fsp_preseal_runs_before_outer_fmp_seal test which covers the seal-ordering invariant with synthetic headers but does not exercise the wire-layout invariant. CHANGELOG.md [Unreleased] # Changed entry added describing the worker-pool threading model, hash-by-destination dispatch, sendmmsg/UDP_GSO, per-peer connected UDP, the operator-facing env vars, and the bench numbers above. Cherry-picks from mmalmi/master (paths translated from crates/fips-core/src/ to src/): 9b7c723, 0deb5cb, 13f7339, e036c0e, 3740a68, 3792f83, 8510193, 4910b07, e53f545, e4e2896, 5fe4af5, 1d01ada, 8c37008, e12469e, 6eb2860. Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com> |
||
|
Johnathan Corgan
|
f51dde647f | Merge maint into master (coord cache surgical invalidation) | ||
|
Johnathan Corgan
|
49bd210480 |
cache: scope coord cache invalidation to entries actually affected by topology change
Replaces the unconditional `CoordCache::clear()` calls at parent-switch, become-root, and loop-detection sites with two targeted invalidation methods scoped to what actually makes an entry stale: - `invalidate_via_node(node_addr)`: drop entries whose cached destination ancestry contains `node_addr`. Used at parent-position- change sites — our prefix changed, so destinations downstream of us have stale-prefix coords. - `invalidate_other_roots(current_root)`: drop entries rooted under a different root than the current one. Used at root-change sites. Under the previous global flush, parent switches blanked the cache across the board, leaving `find_next_hop` returning `None` for every non-direct-peer destination until the cache passively re-warmed via incoming TreeAnnounces / SessionSetup. Surgical invalidation preserves entries that remain correct after the topology change. The cached coord describes a destination's tree position; that position only goes stale relative to our own routing decisions when our own prefix changes (entries we are downstream of) or the root changes (entries in a different tree). Peer removal does not invalidate cached coords: `Node::find_next_hop` recomputes the next-hop decision on every call against the current peer set, bloom filters, and tree state, and Discovery already triggers on `no route to destination` errors when a destination becomes unroutable through us. The peer-removal site retains the original "no cache invalidation" behavior. Each method returns the count of entries removed for observability. Unit tests cover each method against the cases enumerated in the acceptance criterion. |
||
|
Martti MalmiandJohnathan Corgan
|
b1af151aef |
rx: avoid copies in receive hot paths
- Borrowed SessionDatagramRef decoder is used in the forwarding
handler so local delivery and coordinate-cache warming no longer
allocate or copy the session payload. The owned SessionDatagram is
materialized only when re-encoding for the next hop.
- Owned SessionDatagram::decode is reimplemented as Ref::decode +
into_owned, so the two decoders cannot drift.
- recvmmsg / recvmsg_x (Linux + macOS) receive loop moves each filled
slot buffer into ReceivedPacket via mem::replace instead of cloning
it; a fresh empty buffer is installed for the next syscall.
- TransportAddr is formatted directly from the SocketAddr without
going through an intermediate String.
Focused decode bench: ref 1.6 ns/op vs owned 34.7 ns/op (21.4x).
End-to-end iperf is neutral as expected for a ~30 ns saving per
packet.
Unit tests added:
- test_session_datagram_ref_decode_borrows_payload (verifies the
payload slice pointer equals the input slice's offset 35, a real
zero-copy invariant guard against accidental future to_vec)
- bench_session_datagram_decode_owned_vs_ref (ignored, run with
--ignored --nocapture)
- test_transport_addr_from_socket_addr
|
||
|
Johnathan Corgan
|
4f3d2f8471 |
rekey: apply symmetric jitter to desynchronize dual-initiation
Add a per-session signed jitter offset (uniform [-15, +15] seconds) to the rekey timer triggers in check_rekey (FMP) and check_session_rekey (FSP). The configured `node.rekey.after_secs` becomes the nominal interval rather than a floor; mean is preserved. Desynchronizes both endpoints in symmetric-start meshes so the dual-initiation race stops occurring rather than being resolved after the fact by the smaller-NodeAddr tie-breaker. Per-session storage means each rekey cutover reconstructs the session and redraws the jitter naturally — successive cycles get independent offsets, preventing drift back into sync. |
||
|
Martti MalmiandJohnathan Corgan
|
e81fd4b477 |
tree: fix TreeAnnounce ancestry when self is smallest visible NodeAddr
When a node was the smallest-NodeAddr peer it could see (no smaller neighbor available as a parent), the spanning-tree state was promoting it to root. But the ancestry it advertised on the next TreeAnnounce still referenced its previous parent's path, so receiving peers rejected the announce with `invalid ancestry: advertised root X is not the minimum path entry Y`, blocking mesh transit on any path that needed to traverse this node. Detect the self-root transition explicitly in `TreeState::become_root` and rebuild the advertised ancestry to start from self. Also surface the same path through the MMP receive handler so a stale ancestry inherited across reconnect is corrected eagerly rather than waiting for the next observation tick. Adds 80 unit tests in `tree::tests` covering self-root transitions, mid-chain ancestor disappearance, and ancestry validation against the new root, plus a regression in `node::tests::spanning_tree` for a 3-node chain where the middle node's only parent (the smallest-addr peer) goes away — previously it would advertise an ancestry rejected by both endpoints; now it self-roots cleanly. |
||
 Martti MalmiandDev
|
fac4450694 |
node: drain packet_rx / tun_outbound_rx in batches in run_rx_loop
The run_rx_loop's `tokio::select!` was costing one full scheduler hop + futex per inbound packet and per outbound TUN packet. Under sustained load that capped throughput at one event per scheduler quantum — independent of CPU (which sat near-idle) because every iteration parked the worker, woke it via futex, processed one event, then parked again. After the await on `packet_rx.recv()` / `tun_outbound_rx.recv()` fires, drain up to 256 additional ready items via `try_recv()` in a tight inner loop before yielding back to `select!`. `biased` ordering gives the data-plane branches priority over tick / control / DNS under sustained load. The 256 cap is empirically tuned to keep the worker on a busy stream between yield points (a contiguous burst of ~256 MTU-sized packets ≈ 400 KB of contiguous traffic) while still bounding the inner loop so a flood on one branch can't starve the periodic tick or control socket. Lower caps (64) left perf on the table; higher caps (1024+) delayed tick handling visibly under stress. Pairs with the recvmmsg(2) change in the previous commit: the kernel UDP queue now hands packets to `packet_rx` in 32-batches, and the rx_loop drains them without a per-packet scheduler hop. |
||
|
Johnathan Corgan
|
a62a0a6cf4 |
nostr: suppress retraversal of cross-FMP-version peers
Open-discovery NAT traversal succeeds at the UDP layer regardless of what FMP-protocol version the peer speaks. When the daemon discovers a peer running a different FMP version (e.g. a v0/v1 mix during a mid-rollout window, or a misconfigured peer in the same advert namespace), the punch sequence completes, the socket is adopted via `Node::adopt_established_traversal`, and we initiate an FMP handshake. The peer drops our msg1 at its own version-gate and we drop their msg1/msg2 at `Unknown FMP version, dropping`. Neither side advances the handshake. Today the bootstrap transport sits idle until the 31s stale- handshake timeout, drops, and the open-discovery sweep ~30s later fires the full STUN+offer+answer+punch sequence again — every minute, indefinitely, against peers the handshake literally cannot complete with. Add a `Node::bootstrap_transport_npubs` map populated alongside `bootstrap_transports` at adopt time. The rx loop reverse-maps the transport_id → npub on version-mismatch and bumps the discovery layer's `failure_state` to a long structural cooldown via the new `NostrDiscovery::record_protocol_mismatch` API. The next sweep skips the npub for `protocol_mismatch_cooldown_secs` (default 86400 = 24h, separate from the 30-min transient-failure `extended_cooldown_secs`). One-shot WARN per fresh observation. Repeat mismatches inside the cooldown window are silent (the failure_state method returns false when an existing comparable cooldown is already in place). The handshake/transport teardown chain is unchanged — the fix is specifically about preventing the *next* sweep cycle from re-traversing. Cleared on `cleanup_bootstrap_transport_if_unused` and on the adopt-failure rollback path so completed handshakes don't leave stale entries behind. Four new unit tests in `failure_state.rs` cover fresh-entry signaling, repeat-suppression inside the window, streak-pin behavior for `show_peers` rendering, and post-cooldown re-arming. |
||
|
Johnathan Corgan
|
7fc890b7a2 |
session: mirror proactive PathMtuNotification into path_mtu_lookup
The TUN-side TCP MSS clamp consults `path_mtu_lookup` (FipsAddress- keyed) when sizing outbound TCP flows. Until now, only the reactive `MtuExceeded` handler mirrored the bottleneck MTU into that store; the proactive end-to-end `PathMtuNotification` echoed by the destination updated only `MmpSessionState.path_mtu`, leaving the TUN mirror stale. On stable long-lived paths, the proactive echo can tighten the session-canonical MTU well before any transit router fires a `MtuExceeded` for those flows (since all current traffic is already sized by the tighter session value). New TCP flows opened during that window get clamped by the discovery-time value rather than the session-canonical one, leading to PMTU-D loss until the reactive path eventually fires. Mirror the post-apply MTU into `path_mtu_lookup` whenever `apply_notification` returns true, with the same tighter-only semantics as the reactive mirror — never loosen the clamp. Gated on the bool return so spurious writes don't happen on rejected increases or no-op same-value notifications. Four new unit tests exercise the empty-lookup write, tighten- existing, keep-tighter-existing, and no-session-no-op paths, parallel to the existing reactive-mirror test trio. |
||
|
Johnathan Corgan
|
81c0547bdf |
Pin consecutive decrypt-failure counter and threshold-20 force removal
Cover the security-relevant defensive signal: sustained decrypt failures indicate key drift or active probing; threshold-trip force- removes the peer. src/peer/active.rs (+43): two unit tests on the counter struct itself - test_increment_decrypt_failures_monotonic asserts each increment_decrypt_failures() call returns count+1 for at least 25 iterations - test_reset_decrypt_failures_zeroes_counter asserts the reset helper zeroes a non-zero counter and is idempotent src/node/tests/decrypt_failure.rs (new, 93 lines): end-to-end test - Builds a Node + connected peer via existing make_completed_connection / add_connection / promote_connection harness so peers_by_index is exercised, not just peers - Drives the peer to threshold-20 by calling handle_decrypt_failure 20 times; asserts iterations 1..20 leave the peer registered with monotonically increasing counter, then iteration 20 evicts from both peers and peers_by_index src/node/handlers/encrypted.rs: visibility-only widen on handle_decrypt_failure from private to pub(in crate::node) so the in-tree test can drive the threshold logic without re-implementing it. Same pattern as the already-pub(in crate::node) handle_encrypted_frame in the same file. Threshold pinned: DECRYPT_FAILURE_THRESHOLD = 20 at src/node/handlers/encrypted.rs:11. |
||
|
Johnathan Corgan
|
037a965a93 |
Mirror reactive MtuExceeded into path_mtu_lookup
When a transit forwarder drops an oversized data packet and reports the bottleneck back via MtuExceeded, the receive-side handler already updates per-session MmpSessionState::path_mtu (used by PTB synthesis to feed kernel TCP). It did not, however, update path_mtu_lookup — the per-destination map the TUN reader/writer consult at TCP MSS clamp time. So forward-path-asymmetry flows kept clamping at the discovery reverse-path value (too generous for the actual forward-path budget) on every subsequent SYN. Add the missing write at the same point apply_notification runs. Keep the tighter of existing-or-new — the clamp must never loosen. Same write-shape as seed_path_mtu_for_link_peer. Tests: - Three focused unit tests on handle_mtu_exceeded for the empty, tighten, and keep-tighter cases. - Extended test_multihop_pmtud_heterogeneous_mtu to assert the lookup tightens after the wire-level MtuExceeded propagation, alongside its existing PathMtuState assertion. Adds two #[cfg(test)] accessors on Node (path_mtu_lookup_get / path_mtu_lookup_insert) and bumps handle_mtu_exceeded to pub(in crate::node) for direct test invocation. |
||
|
Johnathan Corgan
|
ae607431eb |
Per-destination TCP MSS clamping at the TUN boundary
Adds source-side TCP MSS clamping informed by per-destination path MTU learned via discovery, with a conservative IPv6-minimum-derived ceiling for cold flows where discovery has not yet completed. Closes the multi-hop default-config TCP wedges observed in production where a sender's local-floor MSS exceeds what some intermediate forwarder hop is willing to carry: silent drops, no PTB feedback through the userspace TUN to the kernel TCP stack, retransmits at the same too- large MSS, application connection times out. ## Architecture A new `Arc<RwLock<HashMap<FipsAddress, u16>>>` field `path_mtu_lookup` on Node mirrors the per-destination path MTU in a form accessible from sync TUN reader/writer threads. A new `per_flow_max_mss` helper in `src/upper/tun.rs` reads the lookup at SYN-clamp time and returns the appropriate ceiling for the flow. Three write sites populate `path_mtu_lookup`: 1. **Discovery originator branch** of `handle_lookup_response`: the path MTU bottleneck accumulated through the reverse path lands here when a LookupResponse arrives at the originator. Same value also lands in `coord_cache` per the existing `insert_with_path_mtu` API. 2. **FMP peer-promotion seed** (`seed_path_mtu_for_link_peer`): when an FMP link-layer peer is promoted to active, the local outgoing-link MTU on the peer's transport seeds the lookup. Tighter existing values (learned via discovery) are preserved; the seed only writes when no entry exists or the existing value is looser than the link MTU. Without this seed, directly-configured peers (auto_connect / static peer config) would leave `path_mtu_lookup` empty for their FipsAddress because the FSP session establishes without ever issuing a LookupRequest. 3. **Target-edge fold at `send_lookup_response`**: when a node is the discovery target, it folds its own outgoing-link MTU to the response's next-hop into `path_mtu` before sending. Without this fold, the response leaves the target with `path_mtu = u16::MAX` and only intermediate transits min-fold; the target's first reverse-path hop is never represented in the bottleneck calculation. Refactored the existing transit- side min-fold into a shared `apply_outgoing_link_mtu_to_response` helper called from both sites. ## Read-side: per_flow_max_mss Two TUN call sites consume the lookup: - Outbound `handle_tun_packet` clamps SYN MSS using packet[24..40] (IPv6 destination) as the lookup key. - Inbound `TunWriter::run` clamps SYN-ACK MSS using packet[8..24] (IPv6 source). When the lookup contains a learned value, the helper computes `min(global_max_mss, effective_ipv6_mtu(path_mtu) - 60)` where 60 is IPv6 (40) + TCP (20) headers and `effective_ipv6_mtu` accounts for the FIPS encapsulation overhead. When the lookup is empty for a destination — the cold-flow case — the helper returns `min(global_max_mss, IPv6-minimum-derived ceiling)`. RFC 8200 mandates every IPv6 path accept ≥1280-byte packets, so the IPv6-minimum-derived MSS (1280 - 77 - 60 = 1143) fits any compliant path. Without this conservative ceiling, the first SYN to a destination with no learned path MTU exits the TUN at the kernel-natural MSS (TUN MTU - 60), and the application connection wedges silently before discovery completes for a corrected second SYN to fire. The fix is provably safe: the ceiling is taken with `min` against the local global so operators with even tighter local floors are never loosened upward. Subsequent flows pick up the actual learned per-destination value once discovery (or the FMP-promotion seed for direct peers) populates the lookup. ## Diagnostic logging All write and read sites emit instrumentation suitable for operators bisecting a wedged path: - `debug!` log on every `path_mtu_lookup` write (discovery originator path and FMP-promotion seed path), showing the FipsAddress, written value, prior value, and post-write map size. `warn!` on poisoned-lock failure path. - `trace!` log per `per_flow_max_mss` call covering every fall-through branch (wrong addr_bytes length, non-fd::/8 prefix, lookup poisoned, no entry for destination, empty-lookup conservative ceiling) and the success path. trace level filters out under normal log settings; capture with `RUST_LOG=info,fips::node::handlers::discovery=debug,fips::upper::tun=trace`. ## Tests 15 new unit tests across 3 files: - `per_flow_max_mss` (8 tests in `src/upper/tun.rs::tests`): empty-lookup conservative ceiling, empty-lookup global-smaller floor, learned-value-overrides-conservative, per-destination smaller, per-destination larger capped by global, non-fips addr, short addr slice, per-destination independence. - `seed_path_mtu_for_link_peer` (4 tests in `src/node/tests/unit.rs`): seed when empty, keep tighter existing, tighten looser existing, no-op for unknown transport. - Discovery integration (3 tests in `src/node/tests/discovery.rs`): apply_outgoing_link_mtu_to_response on unknown peer no-op, two-node target-edge fold (path_mtu reflects target-edge link), three-node chain transit min-fold (existing test, updated for target-edge inclusion). Two pre-existing discovery tests had assertions updated to account for the target-edge fold: - `test_response_path_mtu_two_node`: previously asserted `u16::MAX` (no transit to min-fold); now asserts 1280 (the test transport MTU, folded in by send_lookup_response). - `test_response_path_mtu_four_node_chain`: previously asserted 1350 (transit MTUs only); now asserts 1280 (target-edge MTU is the bottleneck). - `test_transit_forwards_when_mtu_sufficient`: previously asserted 1400 (transit MTU only); now asserts 1280 (target- edge MTU is the bottleneck). ## Verification Local CI on this commit: 29/29 suites pass, 1105 lib tests pass, clippy --all-targets --all-features -D warnings clean, cargo fmt clean. Production deploy verified via trace capture across the managed fleet: cold-flow conservative ceiling branch fires on first SYN, learned-lookup branch takes over once discovery completes, both behaviors observable end-to-end at the SYN MSS on the wire. No wire-format change. No config-format change. |
||
|
Johnathan Corgan
|
96c6b7dea8 |
Admit rekey msg1 from established peers when addr forms differ
Companion to the ethernet `accept_connections: false` rekey-deadlock fix from earlier this release: the same dual-init failure mode shows up over UDP when peers register by hostname, and the existing addr_to_link-only carve-out in `should_admit_msg1` doesn't cover it. The carve-out's first predicate keys `addr_to_link` by the literal `TransportAddr` that `initiate_connection` inserted, which is the hostname-form when a peer config carries a hostname (e.g., `core-vm.tail65015.ts.net:2121`). Inbound packets always arrive with numeric source addrs because `udp_receive_loop` builds the `TransportAddr` from the `SocketAddr` the kernel reports via `recvfrom`. `TransportAddr` equality is byte-exact, so the two forms don't match and the lookup misses. With `udp.accept_connections: false` (or `udp.outbound_only: true`, which forces it false) the gate then rejects the rekey msg1 from an established peer. The dual-init tie-breaker stalls because the loser side never produces msg2; both sides retry indefinitely and the winner side keeps logging "Dual rekey initiation: we win, dropping their msg1" at 1Hz. The earlier ethernet fix didn't generalize to this variant because ethernet TransportAddrs are always numeric MAC bytes — both the config-time form and the inbound-arrival form match identically. Add a second predicate to `should_admit_msg1`: an active peer's `current_addr()` matching `(transport_id, remote_addr)`. `current_addr` is updated and refreshed from inbound encrypted-frame source addrs (`handlers/encrypted.rs`), which are always numeric `SocketAddr`-form, so this catches the established peer regardless of how its `addr_to_link` key was originally inserted. The fast `addr_to_link` check stays first; the iteration over peers is bounded by peer count and only runs when the first predicate misses. Regression coverage in this commit: - Unit test `test_should_admit_msg1_admits_rekey_when_addr_form_differs` in `src/node/tests/handshake.rs`. Constructs the failing scenario in-process: `addr_to_link` populated with hostname-form key, peer's `current_addr` at the resolved numeric form, query with numeric form. Without the new predicate this fails immediately. - New integration topology `rekey-outbound-only` plus matching docker-compose profile. Same 5-node mesh shape as `rekey-accept-off` but `inject-config` sets `udp.outbound_only: true` on node-b and rewrites node-b's peer-c address from the numeric docker IP to the docker hostname (`node-c:2121`), reproducing the production hostname-vs-numeric mismatch. The test asserts no sustained "Dual rekey initiation: we win" log lines on any node (>10 = bug) and the existing rekey health checks catch the connectivity loss the loop produces. - `testing/ci-local.sh` and `.github/workflows/ci.yml` extended to run the new variant in the local sweep and the GitHub CI integration matrix alongside `rekey` and `rekey-accept-off`. Verified locally: full `bash testing/ci-local.sh` sweep passes 29/29 suites (23m 12s) with the new variant green; 1084 unit tests pass. |
||
|
Johnathan Corgan
|
e641eb5b5f |
Drop the nostr-discovery cargo feature flag
Make Nostr-mediated overlay discovery unconditional, mirroring the philosophy of PR #79's collapse of the tui/ble/gateway features in favor of platform cfg gates. nostr / nostr-sdk are pure Rust over WebSockets/TCP, so they build cleanly on every FIPS-supported platform — there is no need for the parallel feature gate. The flag was already in `default = [...]`, so no behavior change for anyone using `cargo build` without `--no-default-features`. Operators who explicitly disabled the feature will now find Nostr code present in the binary; the runtime check `node.discovery.nostr.enabled` still controls whether the runtime starts. Cargo.toml: - Remove the `[features]` table entirely. - Drop `optional = true` from `nostr` and `nostr-sdk`. Source: 27 cfg sites collapsed across 5 files — `src/discovery.rs`, `src/discovery/nostr/mod.rs`, `src/node/handlers/rx_loop.rs`, `src/node/lifecycle.rs`, `src/node/mod.rs`. Two `#[cfg(not(feature = "nostr-discovery"))]` fallback blocks (the udp:nat-without-runtime debug-log path and the "feature not compiled in" warning at startup) were removed as dead code; the always-on path already handles the missing-runtime case via `nostr_discovery: Option<NostrDiscovery>`. Packaging and tooling: - `packaging/openwrt-ipk/Makefile`: drop a stale `--features gateway` flag (the `gateway` feature was already removed in PR #79; this was a leftover that the build path tolerated only because cargo ignored unknown feature names). - `testing/scripts/build.sh`: drop `DEFAULT_CARGO_BUILD_ARGS=(--features nostr-discovery)`; defaults are empty. - `packaging/common/fips.yaml`: drop the "requires the nostr-discovery feature" comment from the discovery section. Bundled cleanup: - Apply `cargo clippy --fix` against three pre-existing warnings in `src/discovery/nostr/runtime.rs` and `src/discovery/nostr/stun.rs` (collapsed `if let Some` chain; two redundant `as i32` casts). These were always present but masked when the feature gate was off; they surface now that the code is unconditionally compiled. - `cargo fmt` settled two minor formatting drift sites in `src/bin/fips-gateway.rs` and `src/config/mod.rs`. Tests: 1083 passed, 0 failed, 4 ignored. clippy clean. fmt clean. |
||
|
Johnathan Corgan
|
6def31bcf6 |
Admit rekey msg1 from established peers regardless of accept_connections
The accept_connections gate at the top of handle_msg1 was applied unconditionally, so rekey msg1 from a peer with whom an established link already existed was dropped on the same path as fresh handshakes from strangers. Combined with the dual-init tie-breaker, this deadlocked at ~25 minutes when both sides' rekey timers fired near-simultaneously: the smaller-NodeAddr side wins as initiator and expects the larger side to consume its rekey msg1, but if the larger side has accept_connections=false the gate dropped it. Both sides retried at 1 Hz indefinitely; the affected peer fell out of MMP-active rotation. Extract the gate decision into Node::should_admit_msg1, which admits unconditionally when addr_to_link already has an entry for the (transport_id, remote_addr) pair (rekey/restart on an established session) and otherwise consults the transport's accept_connections(). Fresh msg1 from strangers is still rejected before any Noise crypto. Three unit tests pin the truth table: no transport (admit), accept_off no-link (reject, behavior unchanged), accept_off with-link (admit, the carve-out). The fix generalizes for free to BLE, which has the same Node-level gate. TCP and Tor were never subject to this deadlock because their accept condition is runtime state (bind_addr.is_some() / onion_address.is_some()), not a config flag. |
||
 
|
34e00b9f6e |
Add Nostr-mediated overlay discovery and UDP NAT traversal (#53)
Optional peer discovery and NAT hole-punching path gated behind a new
`nostr-discovery` cargo feature. Nodes publish signed overlay endpoint
adverts to public Nostr relays, consume peer adverts to populate
fallback dial addresses, and use STUN-assisted UDP hole punching with
NIP-59 gift-wrap offer/answer signaling to establish direct UDP paths
between NATed peers. Once a punched socket is up, it is handed into
the existing FIPS UDP transport and the standard Noise/FMP session
stack takes over unchanged.
The cargo feature is in the default feature set
(`default = ["nostr-discovery"]`) so stock builds include it; a
build that explicitly disables default features (or selects a
feature set without `nostr-discovery`) does not link the nostr /
nostr-sdk crates and does not emit a no-op poll in the tick loop.
Runtime behavior is independently gated by
`node.discovery.nostr.enabled`, which defaults to false; if the
config enables Nostr on a non-feature build, startup logs a
warning and continues without it.
== Cargo feature and dependencies
- New cargo feature `nostr-discovery = ["dep:nostr", "dep:nostr-sdk"]`.
Not in the default feature set.
- New optional Linux-only dependencies: `nostr 0.44` (features: std,
nip59) and `nostr-sdk 0.44`. Gift-wrap unwrap is hand-rolled in
`src/discovery/nostr/signal.rs` rather than relying on the SDK's
rumor-author check, which FIPS sidesteps by trusting `seal.pubkey`
exclusively.
== Wire format
Overlay advert event: `kind 37195`, parameterized replaceable
(NIP-01 application-defined replaceable range 30000-39999), with
`d = "fips-overlay-v1"`. The digits visually spell FIPS (7=F, 1=I,
9=P, 5=S); a relay survey confirmed the kind is unused.
Advert content carries the version tag, endpoint list
(`udp|tcp|tor` + addr), optional signal-relay and stun-server
metadata, and `issuedAt` / `expiresAt` timestamps. Endpoint
`addr: "nat"` is the sentinel that triggers traversal on the peer
side. NIP-40 `expiration` tag bounds staleness on permanent
shutdown. Lifecycle relies on parameterized-replaceable
supersession; the daemon does not emit NIP-09 kind-5 deletes —
strict relays (Damus, Primal) race delete-against-replace and can
silently drop the replacement.
Gift-wrapped signal event: `kind 21059`. Punch packets carry magic
values `PUNCH_MAGIC` / `PUNCH_ACK_MAGIC`, a sequence number, and a
16-byte session hash.
== Discovery surface
- `src/discovery.rs` (always compiled)
- `EstablishedTraversal`: bound UDP socket + selected remote +
peer npub + optional transport name/config tuning overrides.
- `BootstrapHandoffResult`: returned on successful handoff —
allocated transport id, local/remote addrs, peer NodeAddr,
session id.
- `src/discovery/nostr/` (`#![cfg(feature = "nostr-discovery")]`)
- `types.rs`: wire and control types described above. `ADVERT_KIND`
constant. `BootstrapError` enumerates failure modes (disabled,
missing advert, missing NAT endpoint, no usable relays, invalid
advert, invalid npub, signal timeout, punch timeout, replay,
STUN failure, protocol, nostr, io, serde, event-parse).
- `runtime.rs`: `NostrDiscovery` coordinator. Owns the shared
nostr-sdk `Client`, subscribes to advert + signal event kinds,
maintains a bounded advert cache and a bounded seen-sessions
replay set, drains `BootstrapEvent::{Established, Failed}` for
the node to consume, exposes `update_local_advert`,
`request_connect`, `advert_endpoints_for_peer`,
`cached_open_discovery_candidates`, and `shutdown`.
- `signal.rs`: NIP-59 gift-wrap encode/decode. Outbound wraps are
built against per-attempt ephemeral keys; inbound events are
unwrapped against the node identity.
- `stun.rs`: RFC 5389/8489 Binding Request client with
XOR-MAPPED-ADDRESS parsing for both IPv4 and IPv6; used only to
observe the initiator's own reflexive address against its
locally configured STUN list (peer-advertised STUN is
informational, never an egress target).
- `traversal.rs`: per-attempt candidate-pair punch planner.
Allocates a fresh `0.0.0.0:0` UDP socket per attempt, enumerates
LAN-private and ULA interface addresses alongside the STUN
reflexive address, schedules probe/ack exchanges at the
configured interval for the configured duration, and picks the
first candidate pair that authenticates end-to-end.
Strategy ordering is Reflexive↔Reflexive first, then LAN, then
Mixed. The STUN-observed pair is the only candidate that's reliable
across arbitrary network topologies; trying it first prevents the
planner from latching onto a misleading host-candidate path before
the reflexive path gets a chance. There is no catch-all
Local↔Local strategy: a previous design that paired every local
host candidate from one side with every local host candidate from
the other could declare success on a one-way reachable asymmetric
L3 path (corporate VPN, Tailscale subnet route, overlapping private
address space), only for the FMP handshake to stall because the
return path didn't match. The legitimate `Lan` strategy still pairs
candidates that share a subnet.
== Configuration surface
`node.discovery.nostr.*` (`NostrDiscoveryConfig`), all `serde(default)`
with `deny_unknown_fields`:
- `enabled` (default false), `advertise` (default true)
- `advert_relays`, `dm_relays`, `stun_servers`: defaults are
`wss://relay.damus.io`, `wss://nos.lol`, `wss://offchain.pub`
for both relay lists, and Google / Cloudflare / Twilio for STUN.
Operators are expected to override for production. Other
verified-working public relays for reference:
`nostr.bitcoiner.social`, `nostr-pub.wellorder.net`,
`nostr.oxtr.dev`, `nostr.mom`.
- `app` (default `"fips-overlay-v1"`), `signal_ttl_secs` (120)
- `policy`: `NostrDiscoveryPolicy::{Disabled, ConfiguredOnly (default),
Open}` — controls whether advert-derived endpoints are consumed
only for peers carrying `via_nostr = true`, or also for
non-configured peers within a budget cap.
- `share_local_candidates` (default false) — when false, the offer's
`local_addresses` list is empty and peers see only the reflexive
address. Enable per-node only for genuinely same-LAN deployments;
off-by-default eliminates the misleading-path failure mode for
the common case where peers are not on the same broadcast domain.
- `open_discovery_max_pending` (64) — caps queued open-discovery
retries; bounded by available outbound slots.
- `max_concurrent_incoming_offers` (16) — semaphore against offer
spam; excess offers are debug-logged and dropped.
- `advert_cache_max_entries` (2048) and `seen_sessions_max_entries`
(2048) — bound memory under ambient relay volume; overflow
evictions are debug-logged.
- `attempt_timeout_secs` (10), `replay_window_secs` (300)
- `punch_start_delay_ms` (2000), `punch_interval_ms` (200),
`punch_duration_ms` (10000)
- `advert_ttl_secs` (3600), `advert_refresh_secs` (1800)
Per-peer and per-transport flags:
- `PeerConfig.via_nostr: bool` — when true (and Nostr is enabled),
advert-derived addresses are appended as fallback dial candidates
after static addresses for that peer.
- `PeerConfig.addresses` is now `serde(default)` and may be empty
when `via_nostr: true`; validation requires at least one of the
two to be present per peer, and the error message names the
peer's npub.
- `UdpConfig.advertise_on_nostr: Option<bool>` and
`UdpConfig.public: Option<bool>` — UDP transports can be
advertised either as direct `host:port` (public = true) or as the
`addr: "nat"` sentinel that triggers rendezvous on the peer side.
- `TcpConfig.advertise_on_nostr` and `TorConfig.advertise_on_nostr`
— TCP and Tor onion endpoints can be advertised as directly
reachable.
- A reserved peer address `transport: udp, addr: "nat"` parses without
special-casing in YAML and routes through the bootstrap runtime.
Cross-field validation (`Config::validate`, called from `Node::new`
and `Node::with_identity`):
- Any transport with `advertise_on_nostr = true` requires
`node.discovery.nostr.enabled = true`.
- Any peer with `via_nostr = true` requires
`node.discovery.nostr.enabled = true`.
- A non-public UDP advert (`advertise_on_nostr = true`,
`public = false` — i.e. `udp:nat`) additionally requires at least
one `dm_relay` and at least one `stun_server`.
Surfaced as `ConfigError::Validation`.
== Node integration
`src/node/lifecycle.rs` is the main integration point.
- At node start (after transports are up, before TUN), if Nostr is
enabled and the feature is compiled in, `NostrDiscovery::start` is
invoked, the initial local overlay advert is built from the live
transport set and published, and the runtime handle is stored.
- The rx tick loop calls `poll_nostr_discovery` (feature-gated both
at method definition and call site), which refreshes the local
advert, drains bootstrap events, adopts established traversals,
schedules retries for failed traversals, and — under `policy:
open` — enqueues outbound retries for non-configured peers
visible in the advert cache, bounded by
`open_discovery_max_pending` and the remaining outbound slots.
- Outbound peer dialing is refactored to `try_peer_addresses`, which
first exhausts the static address list in priority order and only
then appends advert-derived fallback addresses; both lists run
through the same `attempt_peer_address_list` code path. The
`udp:nat` sentinel address triggers `NostrDiscovery::request_connect`
for the peer instead of a direct dial and returns `Ok(())`.
- `build_overlay_advert` walks operational transports, consults
per-instance `UdpConfig` / `TcpConfig` / `TorConfig` (matching by
optional transport instance name), and emits an `OverlayAdvert`
including `signalRelays` and `stunServers` when any UDP endpoint
is advertised as NAT.
- `adopt_established_traversal` is the bootstrap handoff API:
allocates a new `TransportId`, constructs a `UdpTransport` with
the user-supplied (or default) `UdpConfig`, calls the new
`adopt_socket_async` to reuse the punched socket verbatim,
registers the transport in the normal transport map, records it
in `bootstrap_transports`, and calls `initiate_connection` so the
normal handshake path runs. On failure, the transport is stopped
and removed cleanly and the set membership is rolled back.
- On clean shutdown, `NostrDiscovery::shutdown` is awaited so
background tasks stop before transports are torn down. (The
advert is not explicitly retracted; NIP-40 expiration plus the
next refresh from any live publisher supersedes it.)
New `Node` fields:
- `nostr_discovery: Option<Arc<NostrDiscovery>>` (feature-gated).
- `bootstrap_transports: HashSet<TransportId>` — per-peer UDP
transports adopted from NAT traversal, cleaned up via
`cleanup_bootstrap_transport_if_unused` whenever the link,
connection, peer, or pending-connect referencing them is removed.
Retry and error surface:
- `RetryState.expires_at_ms: Option<u64>` — optional absolute expiry
for a retry entry. `pump_retries` drops expired entries with an
info log. Used for open-discovery retries, which expire at two
times the advert TTL.
- New `NodeError::BootstrapHandoff(String)` returned from
`adopt_established_traversal` when the underlying transport
adoption fails or local address discovery fails.
- New `ConfigError::Validation(String)`.
- A small refactor extracts `Node::now_ms()` and reuses it across
lifecycle, rx-loop tick, and timeout bookkeeping.
== UDP transport
`src/transport/udp/`:
- `UdpRawSocket::adopt(std::net::UdpSocket, recv_buf, send_buf)`:
adopts an externally bound socket, makes it non-blocking, applies
the configured buffer sizes (warning if the kernel clamps), and
reports the resulting local address. Preserves the NAT mapping —
no rebind.
- `UdpTransport::adopt_socket_async(std::net::UdpSocket)`: the
`start_async` analogue for an already-bound socket, wiring the
async socket and recv task exactly as the fresh-bind path would.
- `Drop` impl for `UdpTransport`: if a transport is dropped while
still holding a recv task or socket (for example on error
teardown), aborts the task, clears the socket, and emits a debug
log so the cleanup is visible in tracing rather than silent.
== Logging and observability
Default `EnvFilter` demotes third-party relay-pool DEBUG output to
TRACE-only: `nostr_relay_pool`, `nostr_sdk`, and `nostr` are pinned
at INFO when our level is anything below TRACE, and at TRACE when
our level is TRACE — so the raw frames are still reachable when
explicitly asked for. RUST_LOG continues to override completely.
Concise one-line DEBUG events are emitted at the meaningful points
in the discovery / hole-punch sequence:
- `advert: published` (event id, relay count, endpoints, ttl)
- `advert: peer cached` (notify-loop ingress for non-self)
- `advert: resolved` (cache hit / relay fetch outcome)
- `traversal: initiator starting`
- `traversal: initiator STUN observed` (reflexive, local count)
- `traversal: offer sent` (session id, relay count, event id)
- `traversal: answer received` (accepted, reflexive, local)
- `traversal: initiator punch succeeded` (remote addr)
- `traversal: offer received` (responder side)
- `traversal: responder STUN observed`
- `traversal: answer sent`
- `traversal: responder punch succeeded`
Npubs are shortened to `npub1<4>..<4>` and event/session ids to
their first 8 hex characters.
Other operator-facing logs:
- `UdpTransport` adoption and drop paths log at info / debug.
- `adopt_established_traversal` logs at debug on entry and info on
successful return, tagged with peer npub, session id, transport
id, and both socket endpoints, so the bootstrap handoff is
traceable end-to-end alongside the `UdpTransport::drop` log.
- `cleanup_bootstrap_transport_if_unused` logs at debug when the
reference-count check drops an adopted transport.
- `connect_peer` tags its entry `debug!` with `peer_npub` so
downstream STUN, punch, and handshake logs for the same peer
correlate for operators.
- Advert-cache and seen-sessions overflow evictions log at debug so
mis-sized caps are visible under ambient relay volume.
- Gift-wrap unwrap failures on `SIGNAL_KIND` events log at trace
(hot path: fires for every unrelated signal event on the same
relay).
- Traversal-offer handler failures log at debug. Expected conditions
such as punch timeout on symmetric NAT are covered there; real
problems are reported upstream via `BootstrapEvent::Failed`.
- Inbound-offer rate-limit messages name the governing config field
(`max_concurrent_incoming_offers`) and state that the offer was
rate-limited rather than failing.
== Tests
- 18 new unit tests in `src/discovery/nostr/tests.rs` covering advert
encoding, signal envelope round-trip, STUN parsing, punch-packet
codec, and replay-window enforcement. Run under the
`nostr-discovery` feature.
- Config-validation tests in `src/config/mod.rs` covering the three
cross-field invariants and YAML parsing of the full
`node.discovery.nostr` block plus `peers[].via_nostr`, empty
`addresses` with `via_nostr: true`, and a `udp: nat` address.
- `src/node/tests/bootstrap.rs` integration tests that drive a
synthetic traversal (bound UDP socket pair + synthetic peer
identity) through `adopt_established_traversal` and assert the
Noise handshake completes over the adopted socket.
- Punch-planner tests assert reflexive-before-LAN ordering and that
same-LAN scenarios still include the LAN target in the plan.
- `testing/nat/` Docker NAT lab harness:
- Local `strfry` relay, local STUN responder, and one or two
router containers performing `iptables` NAT.
- Node LAN interfaces are provisioned with explicit `veth` pairs
injected into the node and router namespaces so every packet
traverses the router namespace (plain Docker bridges are not
used for the LAN).
- `cone` scenario: both peers behind full-cone-emulation NAT
(SNAT with source-port preservation, inbound DNAT back to the
single LAN host regardless of remote source); asserts UDP
traversal succeeds and link remote addresses are on the router
WAN subnet.
- `symmetric` scenario: `MASQUERADE --random-fully`; asserts UDP
traversal fails and TCP fallback converges over router-
published WAN addresses.
- `lan` scenario: both peers share a LAN subnet; asserts LAN
addresses are preferred over reflexive ones.
- Cleanup tears down all profile-gated services
(`--profile cone --profile symmetric --profile lan`) so no
orphan containers survive a run.
- `testing/scripts/build.sh` builds the Docker test image with
`--features "tui nostr-discovery"` by default so NAT-harness
binaries include bootstrap support.
== CI
- Linux release build and nextest unit-test job both use
`--features "gateway nostr-discovery"` so the feature-gated code
and its unit tests compile and run in CI.
- Three new integration matrix entries (`nat-cone`, `nat-symmetric`,
`nat-lan`) invoke `testing/nat/scripts/nat-test.sh`, collect
`docker compose logs` on failure, and always stop containers.
== Packaging and operations
- `packaging/common/fips.yaml` ships a fully commented
`node.discovery.nostr.*` block, plus documented
`advertise_on_nostr` / `public` examples under the UDP transport,
an `advertise_on_nostr` example under TCP, and a `via_nostr: true`
example under the static peer section with both a direct
`host:port` UDP address and a `udp: nat` fallback.
- `.github/workflows/package-openwrt.yml`: NIP-94 release event
publishes target the new default relay set.
== Documentation
- `README.md`: overlay discovery + NAT traversal moved from
"Near-term priorities" into "What works today".
- `docs/design/fips-intro.md`: rewrites the paragraphs that
previously described Nostr discovery and NAT traversal as future
work; describes the shipped mechanism and the feature gate.
- `docs/design/fips-transport-layer.md`: drops the "(future
direction)" qualifier from the Nostr Relay Discovery section,
expands with the `udp:nat` advertisement and bootstrap handoff
description, and updates the Current State callout.
- `docs/design/fips-mesh-layer.md`: notes that mid-session NAT
rebinding (roaming) and initial NAT traversal (Nostr path) are
distinct mechanisms.
- `docs/design/fips-configuration.md`: documents the full
`node.discovery.nostr.*` surface, including the three resource
caps and `share_local_candidates`.
- `docs/design/fips-nostr-discovery.md`: design and configuration
reference for the shipped mechanism, including the empty-
`addresses`-with-`via_nostr` shorthand.
- `docs/proposals/nostr-udp-hole-punch-protocol.md`: adds an
Implemented status callout, clarifies that the punch socket is
per-peer and per-attempt rather than shared with the application
listener, aligns field names with the shipped JSON
(`sessionId`, `issuedAt` / `expiresAt`, `reflexiveAddress`,
`localAddresses`, `stunServer`), sets the `d`-tag to
`fips-overlay-v1`, names the kind as 37195, and notes that
advertised STUN entries are informational.
- `docs/proposals/README.md`: adds a Status column and marks the
hole-punching proposal Implemented.
- `CHANGELOG.md`: Unreleased > Added entry covering the discovery
path, STUN/punch path, configuration surface, and Docker NAT lab.
Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
|
||
|
Johnathan Corgan
|
f16b837a12 |
Tune overly aggressive discovery rate limiting
The default 30s post-failure backoff (300s cap, doubling per consecutive failure) was set to bound traffic from chatty apps looking up unreachable targets, but in practice it dominates cold-start mesh convergence: a single timed-out lookup during initial bloom-filter propagation suppresses any retry for 30s, and the existing reset triggers (parent change, new peer, first RTT, reconnection) don't fire on a stable post-handshake topology. The suppression window winds up dictating the protocol's effective time-to-converge instead of bounding repeat traffic. Replaces the single-lookup-with-internal-retry model (`timeout_secs`/`retry_interval_secs`/`max_attempts`) with a per-attempt timeout sequence in `node.discovery.attempt_timeouts_secs`, defaulting to `[1, 2, 4, 8]`. Each attempt sends a fresh LookupRequest with a new random request_id so successive attempts can take different forwarding paths as the bloom and tree state evolve. The destination is declared unreachable only after the sequence is exhausted (15s total at the default). Disables post-failure suppression by default (`backoff_base_secs`/ `backoff_max_secs` now `0`/`0`). The `DiscoveryBackoff` machinery stays in tree (inert at zero base/cap); operators with chatty apps generating repeat lookups against unreachable destinations can opt back in. `PendingLookup` field shape unchanged so the control-socket `show_routing` JSON (`pending_lookups[].attempt`/`initiated_ms`/ `last_sent_ms`) keeps the same schema for fipstop and external consumers; `last_sent_ms` now means "current-attempt start" under the new state machine. |
||
 Alex XieandJohnathan Corgan
|
745b523ac6 |
Add peer ACL enforcement with reloadable allow/deny files (#50)
Implement TCP Wrappers-style peer access control using /etc/fips/peers.allow and /etc/fips/peers.deny files. Evaluation order: allow overrides deny, default permit when no files exist. Three enforcement points: outbound connect (before dialing), inbound handshake (msg1 receipt, after restart/rekey classification), and outbound handshake completion (msg2, before peer promotion). Files support npub, hex pubkey, host alias, and ALL wildcard entries with automatic mtime-based reload. Adds fipsctl acl show query, 954-line acl module with unit tests, and a 6-node Docker integration harness (testing/acl-allowlist/) exercising insider, outsider, and allowed-remote scenarios. CI matrix entry included. Closes #50 Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com> |