mirror of
https://github.com/jmcorgan/fips.git
synced 2026-08-09 16:24:45 +00:00
8e72326ec2aefe7cf271bd2ae236e8aecb39e3d9
121
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Johnathan Corgan
|
8e72326ec2 |
node/tests: mark leaf-nonrouting handshake test as parallel-load flake-class
The XX leaf-nonrouting rejection test shares the localhost-UDP rcvbuf overflow failure mode of the other large-network tests under parallel cargo test --lib load. Mark it #[ignore] for the default run; it stays runnable with --ignored or --test-threads=1. |
||
|
Johnathan Corgan
|
a6e4481c1f | Merge branch 'master' into next | ||
|
Johnathan Corgan
|
8d94c0f29c | Merge branch 'maint' | ||
|
Johnathan Corgan
|
e6e2a06879 |
node/tests: stabilize parallel-load flake-class large-network tests
Raise the in-process backpressure headroom in make_test_node_with_mtu (request an 8 MiB recv_buf_size on UdpConfig and grow packet_channel from 256 to 8192) to reduce localhost-UDP receive overflow under parallel-CPU scheduler contention, and mark the large-network convergence tests #[ignore] so cargo test --lib stays green by default. The ignored tests remain runnable on demand with --ignored or --test-threads=1. |
||
|
Johnathan Corgan
|
0a84a25db1 |
Merge master into next (inbound admission cap silent-drop; XX-adapted gate at handle_msg3)
Carries the inbound max_peers admission-cap silent-drop work from
master across the IK→XX structural boundary.
On master (Noise IK), the gate fires at handle_msg1 after the peer's
identity is extracted from the IK msg1 payload, before Msg2 is built
and sent. Wire savings of Msg2 (~104 B) plus the responder crypto
per cap-denied attempt motivate the gate placement.
On next (Noise XX), peer identity is not learned until msg3, by
which point Msg1, Msg2, and Msg3 have all crossed the wire. The IK
gate's wire-savings rationale does not apply on XX. The equivalent
gate is placed in handle_msg3 after the peer's static-key + signature
verification (identity now known) and before promote_connection
(ActivePeer construction, peers_by_index insert, link transition).
The value on XX is local CPU / allocation savings and cleaner peer-
side semantics: no fake-promotion whose subsequent data frames fail
decryption on this side, replaced by a silent drop the peer's
existing auto-reconnect backoff handles cleanly.
Bypass logic is the same as the IK version: known-active peers
(reconnect / cross-connection / restart) and pending-outbound peers
skip the cap so admitting them doesn't grow peers.len(). The late
cap check inside promote_connection is intentionally retained as
defense-in-depth.
Cleanup ordering for the gate's silent-drop path corrects a latent
issue in the adjacent msg3-processing-failure path: our_index is
captured before the connection is removed so the allocator slot is
actually freed (the pre-existing path's get-after-remove returns
None, leaking the index).
The two IK-shaped unit tests at handle_msg1 are marked #[ignore] on
this branch with reason. Their wire-bytes discriminator ("Msg2 must
NOT be sent at cap") is structurally false on XX, and the XX-shaped
equivalents need full three-message-handshake test scaffolding that
isn't authored here. XX-adapted unit + integration test coverage for
the handle_msg3 gate is a follow-up.
CHANGELOG [Unreleased] entry adjusted to describe the handle_msg3
placement on this branch (preserving the master-side framing on the
mainline-merge half of the entry would have been misleading on this
branch).
Three conflicts auto-resolved: the structural handshake.rs span
(~253 lines, IK identity-tied logic discarded on this branch's XX
msg1 path), and three textual ci-local.sh conflicts (admission-cap
suite list / dispatch added alongside the existing mixed-profile
suite list / dispatch).
|
||
|
Johnathan Corgan
|
f6429c19d2 | Merge maint into master (admission-gate Msg2 silent-drop + integration suite) | ||
|
Johnathan Corgan
|
5b229c03bf |
node: skip Msg1 → Msg2 reply when at max_peers cap
Move the max_peers cap check in handle_msg1 forward, from the late check inside promote_connection (which fires after Msg2 has already been built and put on the wire) to an early position after identity verification but before index allocation and the Msg2 send. When the gate fires for a net-new identity, the Msg1 is silent-dropped — no response goes back to the peer, no AEAD compute or wire bytes are spent. Bypass preserved for known peers (reconnect / cross-connection): if the sender's NodeAddr is already in self.peers, or if a pending outbound connection is in flight to the same identity, the gate is skipped so legitimate maintenance traffic continues to work. The late check inside promote_connection is intentionally retained as defense-in-depth against future call sites or a disconnect racing between the early-gate decision and promotion. Wire-cost rationale: a 45 s tcpdump at saturation observed ~3.6 cap-denials/s steady-state, each previously paying the full Noise IK responder crypto + Msg2 (~104 B) on the wire before being rejected. The bigger value is cleaner peer-side semantics — the peer no longer sees a fake-completed handshake whose data frames subsequently fail decryption locally. Two new unit tests cover the cases: - handle_msg1_silent_drops_at_cap_for_new_peer drives a wire-pumped Msg1 from a fresh identity into a saturated node and asserts no Msg2 reaches the sender socket. Stash-verifies as FAIL on the pre-fix tree (Msg2 hits the wire) and PASS post-fix. - handle_msg1_admits_existing_peer_at_cap drives a Msg1 from an identity already in self.peers and asserts the gate does not evict it. This is a regression check (the no-gate tree behaves the same way here, but the test guards against an accidental future gate that breaks known-peer admit). |
||
|
Johnathan Corgan
|
05124367db | Merge master into next (outbound admission gate + mesh-size parent skip) | ||
|
Johnathan Corgan
|
6991a152e6 |
Merge maint into master (outbound admission gate + mesh-size parent skip)
Brings two structural fixes landed on maint: - compute_mesh_size: explicit parent skip in the children loop, so the disjoint-subtree invariant no longer depends on peer_declaration cache freshness. - max_peers: outbound connection-initiation gated on the cap (auto-reconnect retries, Nostr-mediated discovery established adoption, and both sides of the NAT-traversal punch sequence). Inbound msg1 admission gate unchanged. |
||
|
Johnathan Corgan
|
d4687e5d30 |
node: gate outbound connection initiation on max_peers
node.limits.max_peers was honored only on inbound msg1 admission (handshake.rs handle_msg1 returns PeerLimitExceeded when peers.len is at the cap). Four outbound initiation paths proceeded unconditionally at capacity: auto-reconnect retries (process_pending_retries), Nostr-mediated discovery's BootstrapEvent::Established adoption (poll_nostr_discovery), NAT-traversal punch initiation (the outgoing side of the offer/answer/punch sequence in the Nostr discovery runtime), and NAT-traversal punch response (the incoming side of the same sequence). A saturated node burned CPU, UDP probes, STUN observations, and Nostr relay traffic on connections that the inbound gate would reject the moment they reached msg1. Introduce Node::outbound_admission_check (peers.len < max_peers, or true when max_peers == 0 as the no-cap sentinel) and gate the four paths. The discovery runtime lives in a separate task and does not hold a Node reference; bridge via an Arc<AtomicBool> the runtime reads and Node refreshes once per tick from outbound_admission_check. The atomic granularity is intentionally loose: one-tick lag is acceptable because the inbound msg1 gate continues to be the authoritative cap, and in-flight handshakes started below the cap are allowed to complete. Inbound gate at handshake.rs is unchanged. |
||
|
Johnathan Corgan
|
df43ac79b9 |
node: skip parent explicitly in compute_mesh_size children loop
The mesh-size estimator's children loop relied on the cached peer_declaration(parent_id).parent_id() != my_addr check to exclude the parent. That cached view briefly disagrees with our own latest my_declaration().parent_id() during the window between a local parent-switch and the new parent's next inbound TreeAnnounce: the peer-declaration cache still names us as the parent's parent, so the parent is iterated as if it were a child and its (typically dominant) bloom cardinality is added a second time. Symptom: estimated mesh size displayed in fipsctl show status and fipstop nearly-but-not-exactly doubles during tree rebalancing. Make the invariant structural with an explicit peer_addr == parent_id skip at the head of the children loop. Per-peer 500 ms rate-limiter and overall recompute cadence are unchanged. Adds a regression test that constructs the stale-peer-declaration scenario directly and asserts the parent is not double-counted. |
||
|
Johnathan Corgan
|
d6729086de |
Merge master into next (data-plane perf overhaul, packaging fixes)
Forward-merges PR #91 (off-task encrypt/decrypt worker pools, GSO,
connected UDP), the macOS #102 package-integrity fix, and the AUR
#98 fips-dns fix from master.
Six textual conflicts resolved: wire/noise imports, the K-bit-flip
and rekey-cutover handlers, ActivePeer fields, and an advert test
fixture. One semantic conflict: PR #91's worker-pool code on master
referenced FLAG_SP and MmpPeerState.spin_bit, which next removed in
|
||
 Martti MalmiandJohnathan Corgan
|
0a5c367edc |
data-plane perf overhaul: off-task encrypt + decrypt, GSO, connected UDP
Moves both AEAD layers (ChaCha20-Poly1305, one round per layer per packet) plus the sendmsg syscall off the rx_loop task onto a per-shard worker pool, adds per-peer connect(2)-ed UDP with SO_REUSEPORT, and uses Linux UDP GSO (sendmsg+UDP_SEGMENT — kernel splits one super-skb into N on-the-wire datagrams in a single TX-stack walk) when packets in a batch are uniform-size. Same kernel primitive WireGuard's in-kernel module and BoringTun use to hit 2.5–3.2 Gbps single-stream. Single TCP stream on a 5-node docker-bridge mesh, 5 x 15 s x P=1: A→D: 1379 → 2708 Mbps (1.96x, RTT +0.12 ms) A→E: 1394 → 2663 Mbps (1.91x, RTT +0.11 ms) E→A: 1406 → 2624 Mbps (1.87x, RTT +0.19 ms) Static-peer pairs only — every CoV under 3%, 0 outliers, 0% ICMP loss. The ~+100 µs RTT is the worker queue handoff cost; AEAD + sendmmsg now run on a separate core in exchange. What lands: - src/node/encrypt_worker.rs: std::thread + crossbeam_channel workers; hash-by-destination dispatch pins a TCP flow to one worker so wire ordering is preserved; per-worker sendmmsg(2) batching up to 32; Linux uses sendmsg(2)+UDP_SEGMENT when packets in a group are uniform-size. - src/node/decrypt_worker.rs: receive-side mirror. Each shard owns its session's recv cipher + replay window in a thread-local HashMap (no shared RwLock/Mutex). Sessions are handed off at promote_connection and re-registered on K-bit flip / rekey cutover. - src/node/handlers/session.rs try_send_session_data_pipelined: FSP+FMP both seal in-place in the worker on one wire-buffer alloc; no intermediate inner_plaintext / fsp_payload Vecs. - src/transport/udp/connected_peer.rs + peer_drain.rs: per-peer connect(2)-ed UDP socket with SO_REUSEPORT (set on the listen socket too — without that, EADDRINUSE on activation and every packet falls back to the wildcard path); the worker sends with msg_name=NULL and the kernel uses its cached 5-tuple. Tick- driven activation in handlers/connected_udp.rs, idempotent. - src/transport/udp/mod.rs: mem::replace the recvmmsg backing buffer instead of buf.to_vec() per packet — single pointer swap, no MTU-sized memcpy. - src/protocol/link.rs SessionDatagramRef: zero-copy borrowed view used by handle_session_datagram for the bulk local-delivery path; handle_session_payload takes the borrowed payload directly (no payload[35..].to_vec()). - src/transport/mod.rs TransportAddr::from_socket_addr: collapses the two-alloc from_string(addr.to_string()) pattern to one. - src/node/handlers/rx_loop.rs: decrypt-fallback drain promoted ahead of packet_rx in the select! (TCP ACK starvation fix); interleaved fallback drain every 32 packets inside the rx burst loop. - noise::Session: send_cipher_clone / recv_cipher_clone / recv_replay_snapshot_owned / take_send_counter / accept_replay so off-task workers can hold a cloned cipher + reserved counter while the dispatcher keeps replay/counter sequencing serial. CipherState::cipher_clone returns a refcount-bumped LessSafeKey. AsyncUdpSocket: AsRawFd so workers issue raw sendmmsg / sendmsg without going through the tokio reactor. - Worker pool sizing: both default to num_cpus, overridable via FIPS_ENCRYPT_WORKERS=N / FIPS_DECRYPT_WORKERS=N. Per-peer connected UDP can be disabled via FIPS_CONNECTED_UDP=0. - src/perf_profile.rs: optional per-stage timing reporter under FIPS_PERF=1 (or FIPS_PIPELINE_TRACE=1). Off by default; zero overhead when disabled. - All cfg(unix)-gated. Windows continues on the existing tokio- based send/recv. Decrypt worker session lifecycle: - Node::unregister_decrypt_worker_session mirrors the existing register helper. Wired at the two natural sites that already iterate peers_by_index: the rekey drain-completion block in handlers/rekey.rs (drops the worker entry for the old our_index once the drain window has expired and the cache_key is unreachable to any in-flight OLD-K packet), and remove_active_peer in handlers/dispatch.rs (drops the worker entry for each of the four index slots: current, rekey, pending, previous). Only our_index is normally registered; unregister_session is fire- and-forget for missing entries, so calling unconditionally on all four slots is correct and bounds the cleanup without per- slot accounting. Without these callers the per-worker sessions HashMap and the Node's decrypt_registered_sessions set would grow monotonically per rekey on long-lived peers. Testing: - testing/static/scripts/bench-multirun.sh: multi-run iperf3 + ping bench. N reruns (default 5), median / min / max / CoV % / per-run outlier flag, avg ping RTT, ICMP loss %, TCP retransmit total. Plain client→dest labels + topology header. Pre-bench peer-convergence check (FIPS_BENCH_CONVERGE_SECS, default 15); per-path route verification via stats.bytes_sent deltas — fails fast if traffic exits via a non-static-peer link. - testing/static/docker-compose.yml: passes FIPS_ENCRYPT_WORKERS / FIPS_DECRYPT_WORKERS / FIPS_PERF through to containers for A/B benchmarking without rebuilds. - testing/static/scripts/iperf-test.sh: same plain client→dest labels + topology header (was multihop/direct/N hop, which conflated topology distance with on-wire path). - .config/nextest.toml: synthetic UDP node tests serialized through a max-threads=1 test group. Localhost handshakes drop on shared CI runners under parallel load; one-at-a-time keeps assertions reliable. - src/node/tests/spanning_tree.rs: repair_missing_edge_handshakes — retries up to 5 times for synthetic edges whose msg1 was dropped, with a drain after each edge retry instead of after each attempt's full burst. - src/node/decrypt_worker.rs::tests: two unit tests asserting WorkerMsg::UnregisterSession removes the worker-thread session HashMap entry (handle_msg_unregister_session_removes_entry) and is a no-op for never-seen cache_keys (handle_msg_unregister_session_idempotent_on_unknown_key), which is the safety invariant the unconditional unregister calls at the four index slots in remove_active_peer rely on. - src/node/encrypt_worker.rs::unix_tests pipelined_send_wire_layout_roundtrips_canonical_decoders: mirrors the encoder geometry of try_send_session_data_pipelined (no coords, the common established-session path), runs the worker's real seal + send via flush_direct_batch_sync, and decodes the resulting wire packet using only canonical receive-side decoders (EncryptedHeader::parse, SessionDatagramRef::decode, FSP header parse, noise::open). Any divergence between the hand-rolled encoder offsets (fsp_aad_offset, fsp_plaintext_offset) and the decoders fails at one of the parse / open / decode steps before the inner-plaintext assertion fires. Complements the existing fsp_preseal_runs_before_outer_fmp_seal test which covers the seal-ordering invariant with synthetic headers but does not exercise the wire-layout invariant. CHANGELOG.md [Unreleased] # Changed entry added describing the worker-pool threading model, hash-by-destination dispatch, sendmmsg/UDP_GSO, per-peer connected UDP, the operator-facing env vars, and the bench numbers above. Cherry-picks from mmalmi/master (paths translated from crates/fips-core/src/ to src/): 9b7c723, 0deb5cb, 13f7339, e036c0e, 3740a68, 3792f83, 8510193, 4910b07, e53f545, e4e2896, 5fe4af5, 1d01ada, 8c37008, e12469e, 6eb2860. Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com> |
||
|
Martti MalmiandJohnathan Corgan
|
c0ccedb491 | nostr: start discovery without blocking node startup | ||
|
Johnathan Corgan
|
c80b3eb0e3 |
Merge master into next (coord cache surgical invalidation)
# Conflicts: # CHANGELOG.md |
||
|
Martti MalmiandJohnathan Corgan
|
87d1af0269 |
nostr: ignore stale traversal for active peers
Skip BootstrapEvent::Established and BootstrapEvent::Failed dispatch in poll_nostr_discovery for peers that are already connected or actively handshaking. Without these guards, stale traversal events arriving after a peer connected through a different path would either attempt to adopt a redundant socket against the live connection (Established) or poison the per-peer failure-state cooldown and trigger redundant retraversal via schedule_retry / try_peer_addresses (Failed). The four guard sites use a new is_connecting_to_peer helper extracted from the existing closure inside initiate_peer_connection; the helper checks for an in-flight outbound handshake state. adopt_established_traversal gains a defense-in-depth check returning PeerAlreadyExists when called against an already-promoted peer, so the invariant holds if a future caller bypasses the outer dispatch guard. Side benefit: narrows a cooldown-poisoning vector previously available to an attacker injecting stale failure events for an active peer. Test coverage for the new behavior: - test_try_peer_addresses_skips_connected_peer - test_try_peer_addresses_skips_connecting_peer - test_nostr_traversal_failure_skips_connected_peer (Failed-arm event injection) - test_nostr_traversal_established_skips_connected_peer (Established-arm event injection, mirror of the Failed test) - test_adopted_traversal_skips_already_connected_peer (adopt_established_traversal defense-in-depth) CHANGELOG entry under [Unreleased] / Fixed. Closes #87 |
||
|
Johnathan Corgan
|
a014296639 |
node/tests: adapt rekey-jitter tests to XX HandshakeState signature
The two rekey-jitter unit tests added on the IK-rekey branches
construct a HandshakeState via:
HandshakeState::new_initiator(keypair, remote_pubkey)
On next, the XX rewrite changed the initiator constructor to take
only the local keypair:
HandshakeState::new_initiator(keypair)
XX does not learn the responder's static pubkey until msg2, so
the IK-shaped signature does not apply. The merge from master
brought the test bodies verbatim; this drops the second argument
in both call sites so the tests compile against the XX API. The
test semantics (rekey-jitter range and mean checks) are unchanged
and orthogonal to the handshake-pattern choice.
Same shape as
|
||
|
Johnathan Corgan
|
74b08b9b6f |
Merge master into next (rekey jitter, Phase 5 settle, acl-allowlist, CI concurrency)
# Conflicts: # CHANGELOG.md |
||
|
Johnathan Corgan
|
4f3d2f8471 |
rekey: apply symmetric jitter to desynchronize dual-initiation
Add a per-session signed jitter offset (uniform [-15, +15] seconds) to the rekey timer triggers in check_rekey (FMP) and check_session_rekey (FSP). The configured `node.rekey.after_secs` becomes the nominal interval rather than a floor; mean is preserved. Desynchronizes both endpoints in symmetric-start meshes so the dual-initiation race stops occurring rather than being resolved after the fact by the smaller-NodeAddr tie-breaker. Per-session storage means each rekey cutover reconstructs the session and redraws the jitter naturally — successive cycles get independent offsets, preventing drift back into sync. |
||
|
Johnathan Corgan
|
dd65951244 |
Merge branch 'master' into next
# Conflicts: # src/node/tests/bootstrap.rs |
||
|
64cc30df12 |
Schedule retry on startup peer-init failure
When initiate_peer_connections() runs at boot, address resolution can fail for an entire peer (no operational transport for the configured transport types, all addresses unreachable, NAT rebind invalidated cached endpoints, etc.). Before this change the failure was logged and silently forgotten — the peer entry stayed in a dead state forever, accepting incoming pings but unable to answer them, until the daemon was manually restarted. The retry plumbing (schedule_retry / process_pending_retries with exponential backoff) already exists and is wired into the post-handshake failure paths (BootstrapEvent::Failed, MMP dead-link timeout, handshake timeout). The startup loop just wasn't calling it. Mirror the BootstrapEvent::Failed path: on a startup peer-init error, parse the peer's npub and call schedule_retry so the peer recovers without operator intervention. Includes a regression test that asserts retry_pending is populated when initiate_peer_connections() fails for a peer with no operational transport. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Martti MalmiandJohnathan Corgan
|
f0bb29ff6e |
node: inherit primary UDP config when adopting NAT-traversal sockets
`Node::adopt_established_traversal` was constructing the adopted UDP transport with `UdpConfig::default()` — MTU 1280, default recv/send buffer sizes, default accept/advertise flags. If the operator had configured a higher MTU on the primary `[transports.udp]` listener (e.g. 1500 on a path where larger frames are known viable), full-sized tunnel datagrams sent over the NAT-traversed link would exceed the adopted socket's MTU and get dropped at the socket layer with no visibility into why throughput collapsed. Inherit the primary UDP config (MTU + recv/send buffer sizes + accept / advertise flags) and clear the bind / external-address fields since the adopted socket is already bound. Lookup tries `transport_name` first so operators with multiple named `[transports.udp.<name>]` listeners pick up inheritance from the matching listener, and falls back to the unnamed `Single` listener so single-instance configs work unchanged. The previous default of MTU 1280 was deliberately the IPv6 minimum, the only value guaranteed to survive arbitrary middlebox paths. With this change, operators who set their primary listener higher (based on known-clean LAN topology) will have NAT-traversed flows initially attempting that higher MTU and possibly black-holing on tighter paths until reactive `MtuExceeded` recovery kicks in. Documented in the adoption call-site comment so future readers understand why the conservative default went away. Discovered in a downstream consumer where a `MESH_TUNNEL_MTU=1320` / encrypted wire ~1426B produced silent packet drop on every session that had been promoted onto a NAT-traversed link. Adds two sibling tests in `src/node/tests/bootstrap.rs` pinning the new behaviour for the `Single` and `Named` config variants. |
||
|
Johnathan Corgan
|
acb750c0d2 | Merge branch 'master' into next | ||
|
Johnathan Corgan
|
7fc890b7a2 |
session: mirror proactive PathMtuNotification into path_mtu_lookup
The TUN-side TCP MSS clamp consults `path_mtu_lookup` (FipsAddress- keyed) when sizing outbound TCP flows. Until now, only the reactive `MtuExceeded` handler mirrored the bottleneck MTU into that store; the proactive end-to-end `PathMtuNotification` echoed by the destination updated only `MmpSessionState.path_mtu`, leaving the TUN mirror stale. On stable long-lived paths, the proactive echo can tighten the session-canonical MTU well before any transit router fires a `MtuExceeded` for those flows (since all current traffic is already sized by the tighter session value). New TCP flows opened during that window get clamped by the discovery-time value rather than the session-canonical one, leading to PMTU-D loss until the reactive path eventually fires. Mirror the post-apply MTU into `path_mtu_lookup` whenever `apply_notification` returns true, with the same tighter-only semantics as the reactive mirror — never loosen the clamp. Gated on the bool return so spurious writes don't happen on rejected increases or no-op same-value notifications. Four new unit tests exercise the empty-lookup write, tighten- existing, keep-tighter-existing, and no-session-no-op paths, parallel to the existing reactive-mirror test trio. |
||
|
Johnathan Corgan
|
4457587e5f |
Merge branch 'master' into next
# Conflicts: # src/node/tests/discovery.rs # testing/ci-local.sh |
||
|
Johnathan Corgan
|
e08f42e3cc |
Pin discovery state machine: open-discovery sweep + per-attempt lookup timeout
Cover two adjacent runtime behaviors in the discovery state machine
that were previously unpinned at the test level.
1. Open-discovery startup sweep iterate-filter-queue contract.
Cover the runtime sweep behavior: iterate advert cache, apply
skip-filters (own-pubkey, already-connected peers), queue eligible
entries to retry_pending. The config layer was tested but the sweep's
own filtering logic was unpinned.
src/discovery/nostr/runtime.rs: add #[cfg(test)] impl block with
three pub(crate) helpers — new_for_test() builds a minimal
NostrDiscovery with empty cache and no relays/background tasks (uses
fresh nostr::Keys signer + Client::builder().autoconnect(false));
cached_advert_for_test() wraps an OverlayEndpointAdvert into a
CachedOverlayAdvert valid for 1h; insert_advert_for_test() writes
direct to the advert_cache RwLock. All three vanish from release
builds via cfg-gating.
src/node/lifecycle.rs: visibility-only widen on
run_open_discovery_sweep from private async fn to
pub(in crate::node) async fn so the in-tree test can drive it
directly. Same pattern as already-pub(in crate::node) handlers in
src/node/handlers/.
src/node/tests/discovery.rs: add #[tokio::test]
test_open_discovery_sweep_queues_eligible_skips_filtered. Builds
Node + Arc<NostrDiscovery>, injects 3 adverts (eligible, already-
connected peer, own-pubkey), invokes the sweep, asserts retry_pending
contains exactly the eligible entry with matching peer_config npub
and the two filtered entries do NOT appear.
2. Per-attempt timeout state machine in check_pending_lookups.
Cover the central new behavior of
|
||
|
Johnathan Corgan
|
81c0547bdf |
Pin consecutive decrypt-failure counter and threshold-20 force removal
Cover the security-relevant defensive signal: sustained decrypt failures indicate key drift or active probing; threshold-trip force- removes the peer. src/peer/active.rs (+43): two unit tests on the counter struct itself - test_increment_decrypt_failures_monotonic asserts each increment_decrypt_failures() call returns count+1 for at least 25 iterations - test_reset_decrypt_failures_zeroes_counter asserts the reset helper zeroes a non-zero counter and is idempotent src/node/tests/decrypt_failure.rs (new, 93 lines): end-to-end test - Builds a Node + connected peer via existing make_completed_connection / add_connection / promote_connection harness so peers_by_index is exercised, not just peers - Drives the peer to threshold-20 by calling handle_decrypt_failure 20 times; asserts iterations 1..20 leave the peer registered with monotonically increasing counter, then iteration 20 evicts from both peers and peers_by_index src/node/handlers/encrypted.rs: visibility-only widen on handle_decrypt_failure from private to pub(in crate::node) so the in-tree test can drive the threshold logic without re-implementing it. Same pattern as the already-pub(in crate::node) handle_encrypted_frame in the same file. Threshold pinned: DECRYPT_FAILURE_THRESHOLD = 20 at src/node/handlers/encrypted.rs:11. |
||
|
Johnathan Corgan
|
00f4a4c7af |
Pin bloom-not-closer-than-tree-parent fall-through to greedy tree
Add test_routing_bloom_hit_not_closer_falls_through_to_tree to
src/node/tests/routing.rs covering the regression class fixed in
|
||
|
Johnathan Corgan
|
1c8b8d55e8 | Merge branch 'master' into next | ||
|
Johnathan Corgan
|
037a965a93 |
Mirror reactive MtuExceeded into path_mtu_lookup
When a transit forwarder drops an oversized data packet and reports the bottleneck back via MtuExceeded, the receive-side handler already updates per-session MmpSessionState::path_mtu (used by PTB synthesis to feed kernel TCP). It did not, however, update path_mtu_lookup — the per-destination map the TUN reader/writer consult at TCP MSS clamp time. So forward-path-asymmetry flows kept clamping at the discovery reverse-path value (too generous for the actual forward-path budget) on every subsequent SYN. Add the missing write at the same point apply_notification runs. Keep the tighter of existing-or-new — the clamp must never loosen. Same write-shape as seed_path_mtu_for_link_peer. Tests: - Three focused unit tests on handle_mtu_exceeded for the empty, tighten, and keep-tighter cases. - Extended test_multihop_pmtud_heterogeneous_mtu to assert the lookup tightens after the wire-level MtuExceeded propagation, alongside its existing PathMtuState assertion. Adds two #[cfg(test)] accessors on Node (path_mtu_lookup_get / path_mtu_lookup_insert) and bumps handle_mtu_exceeded to pub(in crate::node) for direct test invocation. |
||
|
Johnathan Corgan
|
b02eaf2c5b |
Merge branch 'master' into next
Forward-merges the squashed per-destination TCP MSS clamping work (master `ae60743`) into next. Auto-merge of source files clean. Two pre-existing test assertions on the next-side discovery tests were updated as part of merge resolution to reflect the target-edge MTU fold introduced by the merged-in commit: - test_transit_forwards_when_mtu_sufficient: 1400 → 1280 = min(target-edge 1280, transit 1400) - test_response_path_mtu_four_node_chain: 1350 → 1280 = min(target-edge 1280, transits 1350+1500) Resulting next tree is bit-for-bit identical to the pre-squash next state previously verified by full local CI sweep (29 + 1 next-only suites pass on `6a8b519` in 23m 41s). |
||
|
Johnathan Corgan
|
ae607431eb |
Per-destination TCP MSS clamping at the TUN boundary
Adds source-side TCP MSS clamping informed by per-destination path MTU learned via discovery, with a conservative IPv6-minimum-derived ceiling for cold flows where discovery has not yet completed. Closes the multi-hop default-config TCP wedges observed in production where a sender's local-floor MSS exceeds what some intermediate forwarder hop is willing to carry: silent drops, no PTB feedback through the userspace TUN to the kernel TCP stack, retransmits at the same too- large MSS, application connection times out. ## Architecture A new `Arc<RwLock<HashMap<FipsAddress, u16>>>` field `path_mtu_lookup` on Node mirrors the per-destination path MTU in a form accessible from sync TUN reader/writer threads. A new `per_flow_max_mss` helper in `src/upper/tun.rs` reads the lookup at SYN-clamp time and returns the appropriate ceiling for the flow. Three write sites populate `path_mtu_lookup`: 1. **Discovery originator branch** of `handle_lookup_response`: the path MTU bottleneck accumulated through the reverse path lands here when a LookupResponse arrives at the originator. Same value also lands in `coord_cache` per the existing `insert_with_path_mtu` API. 2. **FMP peer-promotion seed** (`seed_path_mtu_for_link_peer`): when an FMP link-layer peer is promoted to active, the local outgoing-link MTU on the peer's transport seeds the lookup. Tighter existing values (learned via discovery) are preserved; the seed only writes when no entry exists or the existing value is looser than the link MTU. Without this seed, directly-configured peers (auto_connect / static peer config) would leave `path_mtu_lookup` empty for their FipsAddress because the FSP session establishes without ever issuing a LookupRequest. 3. **Target-edge fold at `send_lookup_response`**: when a node is the discovery target, it folds its own outgoing-link MTU to the response's next-hop into `path_mtu` before sending. Without this fold, the response leaves the target with `path_mtu = u16::MAX` and only intermediate transits min-fold; the target's first reverse-path hop is never represented in the bottleneck calculation. Refactored the existing transit- side min-fold into a shared `apply_outgoing_link_mtu_to_response` helper called from both sites. ## Read-side: per_flow_max_mss Two TUN call sites consume the lookup: - Outbound `handle_tun_packet` clamps SYN MSS using packet[24..40] (IPv6 destination) as the lookup key. - Inbound `TunWriter::run` clamps SYN-ACK MSS using packet[8..24] (IPv6 source). When the lookup contains a learned value, the helper computes `min(global_max_mss, effective_ipv6_mtu(path_mtu) - 60)` where 60 is IPv6 (40) + TCP (20) headers and `effective_ipv6_mtu` accounts for the FIPS encapsulation overhead. When the lookup is empty for a destination — the cold-flow case — the helper returns `min(global_max_mss, IPv6-minimum-derived ceiling)`. RFC 8200 mandates every IPv6 path accept ≥1280-byte packets, so the IPv6-minimum-derived MSS (1280 - 77 - 60 = 1143) fits any compliant path. Without this conservative ceiling, the first SYN to a destination with no learned path MTU exits the TUN at the kernel-natural MSS (TUN MTU - 60), and the application connection wedges silently before discovery completes for a corrected second SYN to fire. The fix is provably safe: the ceiling is taken with `min` against the local global so operators with even tighter local floors are never loosened upward. Subsequent flows pick up the actual learned per-destination value once discovery (or the FMP-promotion seed for direct peers) populates the lookup. ## Diagnostic logging All write and read sites emit instrumentation suitable for operators bisecting a wedged path: - `debug!` log on every `path_mtu_lookup` write (discovery originator path and FMP-promotion seed path), showing the FipsAddress, written value, prior value, and post-write map size. `warn!` on poisoned-lock failure path. - `trace!` log per `per_flow_max_mss` call covering every fall-through branch (wrong addr_bytes length, non-fd::/8 prefix, lookup poisoned, no entry for destination, empty-lookup conservative ceiling) and the success path. trace level filters out under normal log settings; capture with `RUST_LOG=info,fips::node::handlers::discovery=debug,fips::upper::tun=trace`. ## Tests 15 new unit tests across 3 files: - `per_flow_max_mss` (8 tests in `src/upper/tun.rs::tests`): empty-lookup conservative ceiling, empty-lookup global-smaller floor, learned-value-overrides-conservative, per-destination smaller, per-destination larger capped by global, non-fips addr, short addr slice, per-destination independence. - `seed_path_mtu_for_link_peer` (4 tests in `src/node/tests/unit.rs`): seed when empty, keep tighter existing, tighten looser existing, no-op for unknown transport. - Discovery integration (3 tests in `src/node/tests/discovery.rs`): apply_outgoing_link_mtu_to_response on unknown peer no-op, two-node target-edge fold (path_mtu reflects target-edge link), three-node chain transit min-fold (existing test, updated for target-edge inclusion). Two pre-existing discovery tests had assertions updated to account for the target-edge fold: - `test_response_path_mtu_two_node`: previously asserted `u16::MAX` (no transit to min-fold); now asserts 1280 (the test transport MTU, folded in by send_lookup_response). - `test_response_path_mtu_four_node_chain`: previously asserted 1350 (transit MTUs only); now asserts 1280 (target-edge MTU is the bottleneck). - `test_transit_forwards_when_mtu_sufficient`: previously asserted 1400 (transit MTU only); now asserts 1280 (target- edge MTU is the bottleneck). ## Verification Local CI on this commit: 29/29 suites pass, 1105 lib tests pass, clippy --all-targets --all-features -D warnings clean, cargo fmt clean. Production deploy verified via trace capture across the managed fleet: cold-flow conservative ceiling branch fires on first SYN, learned-lookup branch takes over once discovery completes, both behaviors observable end-to-end at the SYN MSS on the wire. No wire-format change. No config-format change. |
||
|
Johnathan Corgan
|
e7fc4f7d10 | Merge branch 'master' into next | ||
|
Johnathan Corgan
|
8448e38510 |
Make Node::transport_mtu() deterministic across restarts (TCP black hole fix)
Default-config TCP flows between fips peers were stalling completely (cwnd-pinned, 0 bps for 10s+) on a non-trivial fraction of restarts. Reproducible with iperf3 between any two peers. Root cause: `Node::transport_mtu()` iterated `self.transports.values()` (HashMap with default RandomState hasher) and returned `handle.mtu()` of the first one whose `is_operational()` returned true. Two stacked sources of non-determinism stacked on each other: HashMap iteration order is randomized per-process via RandomState, and async transport `.start()` completion order races each daemon restart. The returned value drives the TCP MSS clamp ceiling computed once at TUN init (src/upper/tun.rs:501-524) and stored as immutable max_mss in the reader/writer thread state. When the picker landed on a transport with MTU > 1357 (any non-UDP-1280 in the standard fleet defaults), `max_mss > 1220` (kernel's natural fips0-MTU-derived MSS), the daemon's clamp was silently a no-op, and the kernel emitted 1220-byte segments. Those wrap into 1280-byte IPv6 → 1357-byte fips datagrams that exceed UDP-1280 transports at any forwarding hop, causing silent drops with no PTB feedback to the kernel TCP stack. Fix: return min across operational transports instead of first-iterated. With UDP-1280 in the configured set (the common case), `transport_mtu = 1280`, `max_mss = 1143 < 1220`, daemon's clamp engages, MSS=1143 reaches the wire, packets fit, throughput recovers. Empirical green light from a single-UDP-config end-to-end test: iperf3-without-`-M` recovered to ~21 Mbps with no operator-side nft TCPMSS rules. Adds three unit tests: - transport_mtu_returns_min_across_operational: pin selection to smallest MTU when multiple operational transports differ. - transport_mtu_fallback_when_no_operational_transports: 1280 fallback. - transport_mtu_min_with_single_operational: trivial single-transport case. The `effective_ipv6_mtu` field reported by `fipsctl show status` was also racy (consequence of the same bug); fixed by this change as a side effect. |
||
|
Johnathan Corgan
|
5317735a91 |
Merge branch 'master' into next
# Conflicts: # src/node/tests/handshake.rs # testing/ci-local.sh |
||
|
Johnathan Corgan
|
96c6b7dea8 |
Admit rekey msg1 from established peers when addr forms differ
Companion to the ethernet `accept_connections: false` rekey-deadlock fix from earlier this release: the same dual-init failure mode shows up over UDP when peers register by hostname, and the existing addr_to_link-only carve-out in `should_admit_msg1` doesn't cover it. The carve-out's first predicate keys `addr_to_link` by the literal `TransportAddr` that `initiate_connection` inserted, which is the hostname-form when a peer config carries a hostname (e.g., `core-vm.tail65015.ts.net:2121`). Inbound packets always arrive with numeric source addrs because `udp_receive_loop` builds the `TransportAddr` from the `SocketAddr` the kernel reports via `recvfrom`. `TransportAddr` equality is byte-exact, so the two forms don't match and the lookup misses. With `udp.accept_connections: false` (or `udp.outbound_only: true`, which forces it false) the gate then rejects the rekey msg1 from an established peer. The dual-init tie-breaker stalls because the loser side never produces msg2; both sides retry indefinitely and the winner side keeps logging "Dual rekey initiation: we win, dropping their msg1" at 1Hz. The earlier ethernet fix didn't generalize to this variant because ethernet TransportAddrs are always numeric MAC bytes — both the config-time form and the inbound-arrival form match identically. Add a second predicate to `should_admit_msg1`: an active peer's `current_addr()` matching `(transport_id, remote_addr)`. `current_addr` is updated and refreshed from inbound encrypted-frame source addrs (`handlers/encrypted.rs`), which are always numeric `SocketAddr`-form, so this catches the established peer regardless of how its `addr_to_link` key was originally inserted. The fast `addr_to_link` check stays first; the iteration over peers is bounded by peer count and only runs when the first predicate misses. Regression coverage in this commit: - Unit test `test_should_admit_msg1_admits_rekey_when_addr_form_differs` in `src/node/tests/handshake.rs`. Constructs the failing scenario in-process: `addr_to_link` populated with hostname-form key, peer's `current_addr` at the resolved numeric form, query with numeric form. Without the new predicate this fails immediately. - New integration topology `rekey-outbound-only` plus matching docker-compose profile. Same 5-node mesh shape as `rekey-accept-off` but `inject-config` sets `udp.outbound_only: true` on node-b and rewrites node-b's peer-c address from the numeric docker IP to the docker hostname (`node-c:2121`), reproducing the production hostname-vs-numeric mismatch. The test asserts no sustained "Dual rekey initiation: we win" log lines on any node (>10 = bug) and the existing rekey health checks catch the connectivity loss the loop produces. - `testing/ci-local.sh` and `.github/workflows/ci.yml` extended to run the new variant in the local sweep and the GitHub CI integration matrix alongside `rekey` and `rekey-accept-off`. Verified locally: full `bash testing/ci-local.sh` sweep passes 29/29 suites (23m 12s) with the new variant green; 1084 unit tests pass. |
||
|
Johnathan Corgan
|
37c2973e2f |
Test infrastructure overhaul: gateway robustness + full CI coverage
Single combined commit covering five interlocking pieces of test and CI work that landed during the v0.3.0-prep cycle. ## fips-gateway robustness - src/bin/fips-gateway.rs DNS upstream probe converted from a 3-second hard-fail to a bounded retry loop (5 attempts × 1s timeout, 1s sleep between attempts; ~10s worst case). Covers the cold-boot race where the daemon's TUN is up but the DNS responder at [::1]:5354 is still binding. Each failed attempt logs at INFO. In production the binary's retry is the live recovery mechanism; with retry it recovers silently instead of relying on Restart=on-failure (~5s blip + spurious ERROR per cycle). - packaging/debian/fips-gateway.service `ExecStartPre` now waits up to 30 seconds for the daemon's `fips0` TUN to appear before exec'ing the gateway binary. Eliminates the cold-boot race where the gateway exits with `fips0 interface not found` and recovers via `Restart=on-failure`, producing a 5-second blip and a spurious error log per restart cycle. - testing/docker/entrypoint.sh gateway-mode waits up to 30s for the daemon's DNS responder to bind [::1]:5354 (probes once per second with `dig @::1 -p 5354 ... test.fips`) before exec'ing fips-gateway. Belt-and-suspenders with the binary's own retry: in CI we want deterministic startup ordering. On timeout, fall through so the binary's probe reports the definitive error. ## Test infrastructure DNS bind migration to ::1 After session 359's daemon DNS-bind default flipped from `127.0.0.1` to `::1` (the production fix for ISSUE-2026-0002), the static-test infrastructure was carrying a stale workaround that overrode the default back to IPv4 loopback. The fips-gateway integration test exposed the divergence: the gateway probes its DNS upstream at `[::1]:5354` (production default) while the daemon was binding `127.0.0.1:5354` from the template override — IPv6-explicit sockets do not accept v4-mapped traffic, so the upstream probe exhausted retries and the gateway exited. - Drop the explicit `bind_addr: "127.0.0.1"` line from every test config that emits it: testing/static/configs/node.template.yaml, testing/chaos/configs/node.template.yaml, the sidecar heredoc in testing/docker/entrypoint.sh, testing/acl-allowlist/generate-configs.sh (six per-node blocks), testing/nat/scripts/generate-configs.sh, and the four tor templates under testing/tor/. Daemon picks up its production `::1` default. - Flip the dnsmasq forwarder for `.fips` in testing/docker/Dockerfile from `127.0.0.1#5354` to `::1#5354` so dnsmasq on the shared test image continues to reach the daemon. Template and Dockerfile must move together since most static suites resolve `<npub>.fips` via the test-image dnsmasq. ## rekey-accept-off integration variant + UDP unit test - New `rekey-accept-off` topology and docker-compose profile under testing/static/. 2-node variant where node-b runs with `udp.accept_connections: false`. Pins the regression class that ISSUE-2026-0004 fixed (cross-connection winner's rekey msg1 was being filtered by the accept_connections gate, breaking rekey). - testing/static/scripts/rekey-test.sh accepts REKEY_TOPOLOGY and REKEY_ACCEPT_OFF_NODES env vars; its inject-config subcommand applies the per-node `udp.accept_connections: false` edit, and the test asserts no sustained "Dual rekey initiation" log lines. - New UDP variant of `should_admit_msg1` admit-rekey unit test in src/node/tests/handshake.rs. ## ci-local.sh full integration coverage - New runner functions and dispatcher entries for `acl-allowlist`, `nat-cone` / `nat-symmetric` / `nat-lan`, `rekey-accept-off`, `dns-resolver`, `deb-install`. Each integrates with the existing summary tracking via `record`. - New `--with-tor` flag (off by default) gates `tor-socks5-outbound` and `tor-directory-mode` runners. Tor stays opt-in because both harnesses depend on the live Tor network and would introduce a flake source unrelated to the FIPS code. - New suite arrays (`ACL_SUITES`, `NAT_SUITES`, `DNS_RESOLVER_SUITES`, `DEB_INSTALL_SUITES`, `TOR_SUITES`) drive both the default sweep and `--list` output. - `run_suite` extended to accept the new suite names for `--only` invocations. ## GitHub CI matrix expansions - `gateway` matrix entry runs testing/static/scripts/gateway-test.sh against the existing docker-compose `gateway` profile. - `rekey-accept-off` matrix entry exercises the new topology with REKEY_ACCEPT_OFF_NODES=b. - `deb-install` matrix (debian12 + ubuntu24 + ubuntu26) runs testing/deb-install/test.sh with privileged systemd containers. ~5-7 min cold cache, ~2 min warm per distro. Self-contained: builds its own .deb in a Debian 12 cargo-deb builder image; does not depend on the build job's pre-built artifact. - `dns-resolver` matrix entry runs the full 13-scenario harness (per-distro systemd resolver-backend tests + real-fips end-to-end scenarios) in a single job. Pins the production DNS bind path that ISSUE-2026-0002 lived in. ~7-12 min warm, ~12-15 min cold. Verified locally: full `bash testing/ci-local.sh` sweep passes, including 5/5 deb-install distros and all 13 dns-resolver scenarios. Tor-inclusive sweep (`--with-tor`) verified in a follow-up run. |
||
|
Johnathan Corgan
|
6def31bcf6 |
Admit rekey msg1 from established peers regardless of accept_connections
The accept_connections gate at the top of handle_msg1 was applied unconditionally, so rekey msg1 from a peer with whom an established link already existed was dropped on the same path as fresh handshakes from strangers. Combined with the dual-init tie-breaker, this deadlocked at ~25 minutes when both sides' rekey timers fired near-simultaneously: the smaller-NodeAddr side wins as initiator and expects the larger side to consume its rekey msg1, but if the larger side has accept_connections=false the gate dropped it. Both sides retried at 1 Hz indefinitely; the affected peer fell out of MMP-active rotation. Extract the gate decision into Node::should_admit_msg1, which admits unconditionally when addr_to_link already has an entry for the (transport_id, remote_addr) pair (rekey/restart on an established session) and otherwise consults the transport's accept_connections(). Fresh msg1 from strangers is still rejected before any Noise crypto. Three unit tests pin the truth table: no transport (admit), accept_off no-link (reject, behavior unchanged), accept_off with-link (admit, the carve-out). The fix generalizes for free to BLE, which has the same Node-level gate. TCP and Tor were never subject to this deadlock because their accept condition is runtime state (bind_addr.is_some() / onion_address.is_some()), not a config flag. |
||
|
Johnathan Corgan
|
8a6477b7ff |
Adapt bootstrap-handoff tests to XX three-way handshake
The two bootstrap-handoff tests were written against the IK two-message
handshake where peer promotion happens after msg2. Under the noise-XX
three-message handshake on this branch, identity is only revealed in
msg3, so peer promotion happens one message later: the responder learns
the initiator's identity in msg3 and only then promotes the peer.
`test_adopted_udp_traversal_completes_handshake`: Previous version ran
two `run_rx_loop` iterations (one per node) and asserted both peers
had been promoted. That works for IK but not XX: after node_a sends
msg1 from `adopt_established_traversal`, node_b's first rx_loop pass
generates msg2 (revealing node_b's identity to node_a), node_a's next
rx_loop pass generates msg3 (revealing node_a's identity to node_b),
and only then does node_b promote node_a. A third rx_loop pass on
node_b is needed.
Restructured the test to use direct `handle_msg{1,2,3}` dispatch via
`packet_rx.recv()` rather than `run_rx_loop` + select cancellation,
matching the pattern already used by
`test_third_peer_can_handshake_via_adopted_transport_socket`. This
sidesteps the issue that `run_rx_loop` does `packet_rx.take()` and a
cancelled future never returns it, so a second rx_loop call fails
with `NotStarted`.
`test_third_peer_can_handshake_via_adopted_transport_socket`: Added
explicit msg3 dispatch on both Alice/Bob and Bob/Colin sub-handshakes
so the responder side actually receives the initiator's identity
before the test asserts peer promotion.
|
||
|
Johnathan Corgan
|
5a5dbe36da |
Merge branch 'master' into next
# Conflicts: # src/node/lifecycle.rs # testing/static/scripts/rekey-test.sh |
||
 
|
34e00b9f6e |
Add Nostr-mediated overlay discovery and UDP NAT traversal (#53)
Optional peer discovery and NAT hole-punching path gated behind a new
`nostr-discovery` cargo feature. Nodes publish signed overlay endpoint
adverts to public Nostr relays, consume peer adverts to populate
fallback dial addresses, and use STUN-assisted UDP hole punching with
NIP-59 gift-wrap offer/answer signaling to establish direct UDP paths
between NATed peers. Once a punched socket is up, it is handed into
the existing FIPS UDP transport and the standard Noise/FMP session
stack takes over unchanged.
The cargo feature is in the default feature set
(`default = ["nostr-discovery"]`) so stock builds include it; a
build that explicitly disables default features (or selects a
feature set without `nostr-discovery`) does not link the nostr /
nostr-sdk crates and does not emit a no-op poll in the tick loop.
Runtime behavior is independently gated by
`node.discovery.nostr.enabled`, which defaults to false; if the
config enables Nostr on a non-feature build, startup logs a
warning and continues without it.
== Cargo feature and dependencies
- New cargo feature `nostr-discovery = ["dep:nostr", "dep:nostr-sdk"]`.
Not in the default feature set.
- New optional Linux-only dependencies: `nostr 0.44` (features: std,
nip59) and `nostr-sdk 0.44`. Gift-wrap unwrap is hand-rolled in
`src/discovery/nostr/signal.rs` rather than relying on the SDK's
rumor-author check, which FIPS sidesteps by trusting `seal.pubkey`
exclusively.
== Wire format
Overlay advert event: `kind 37195`, parameterized replaceable
(NIP-01 application-defined replaceable range 30000-39999), with
`d = "fips-overlay-v1"`. The digits visually spell FIPS (7=F, 1=I,
9=P, 5=S); a relay survey confirmed the kind is unused.
Advert content carries the version tag, endpoint list
(`udp|tcp|tor` + addr), optional signal-relay and stun-server
metadata, and `issuedAt` / `expiresAt` timestamps. Endpoint
`addr: "nat"` is the sentinel that triggers traversal on the peer
side. NIP-40 `expiration` tag bounds staleness on permanent
shutdown. Lifecycle relies on parameterized-replaceable
supersession; the daemon does not emit NIP-09 kind-5 deletes —
strict relays (Damus, Primal) race delete-against-replace and can
silently drop the replacement.
Gift-wrapped signal event: `kind 21059`. Punch packets carry magic
values `PUNCH_MAGIC` / `PUNCH_ACK_MAGIC`, a sequence number, and a
16-byte session hash.
== Discovery surface
- `src/discovery.rs` (always compiled)
- `EstablishedTraversal`: bound UDP socket + selected remote +
peer npub + optional transport name/config tuning overrides.
- `BootstrapHandoffResult`: returned on successful handoff —
allocated transport id, local/remote addrs, peer NodeAddr,
session id.
- `src/discovery/nostr/` (`#![cfg(feature = "nostr-discovery")]`)
- `types.rs`: wire and control types described above. `ADVERT_KIND`
constant. `BootstrapError` enumerates failure modes (disabled,
missing advert, missing NAT endpoint, no usable relays, invalid
advert, invalid npub, signal timeout, punch timeout, replay,
STUN failure, protocol, nostr, io, serde, event-parse).
- `runtime.rs`: `NostrDiscovery` coordinator. Owns the shared
nostr-sdk `Client`, subscribes to advert + signal event kinds,
maintains a bounded advert cache and a bounded seen-sessions
replay set, drains `BootstrapEvent::{Established, Failed}` for
the node to consume, exposes `update_local_advert`,
`request_connect`, `advert_endpoints_for_peer`,
`cached_open_discovery_candidates`, and `shutdown`.
- `signal.rs`: NIP-59 gift-wrap encode/decode. Outbound wraps are
built against per-attempt ephemeral keys; inbound events are
unwrapped against the node identity.
- `stun.rs`: RFC 5389/8489 Binding Request client with
XOR-MAPPED-ADDRESS parsing for both IPv4 and IPv6; used only to
observe the initiator's own reflexive address against its
locally configured STUN list (peer-advertised STUN is
informational, never an egress target).
- `traversal.rs`: per-attempt candidate-pair punch planner.
Allocates a fresh `0.0.0.0:0` UDP socket per attempt, enumerates
LAN-private and ULA interface addresses alongside the STUN
reflexive address, schedules probe/ack exchanges at the
configured interval for the configured duration, and picks the
first candidate pair that authenticates end-to-end.
Strategy ordering is Reflexive↔Reflexive first, then LAN, then
Mixed. The STUN-observed pair is the only candidate that's reliable
across arbitrary network topologies; trying it first prevents the
planner from latching onto a misleading host-candidate path before
the reflexive path gets a chance. There is no catch-all
Local↔Local strategy: a previous design that paired every local
host candidate from one side with every local host candidate from
the other could declare success on a one-way reachable asymmetric
L3 path (corporate VPN, Tailscale subnet route, overlapping private
address space), only for the FMP handshake to stall because the
return path didn't match. The legitimate `Lan` strategy still pairs
candidates that share a subnet.
== Configuration surface
`node.discovery.nostr.*` (`NostrDiscoveryConfig`), all `serde(default)`
with `deny_unknown_fields`:
- `enabled` (default false), `advertise` (default true)
- `advert_relays`, `dm_relays`, `stun_servers`: defaults are
`wss://relay.damus.io`, `wss://nos.lol`, `wss://offchain.pub`
for both relay lists, and Google / Cloudflare / Twilio for STUN.
Operators are expected to override for production. Other
verified-working public relays for reference:
`nostr.bitcoiner.social`, `nostr-pub.wellorder.net`,
`nostr.oxtr.dev`, `nostr.mom`.
- `app` (default `"fips-overlay-v1"`), `signal_ttl_secs` (120)
- `policy`: `NostrDiscoveryPolicy::{Disabled, ConfiguredOnly (default),
Open}` — controls whether advert-derived endpoints are consumed
only for peers carrying `via_nostr = true`, or also for
non-configured peers within a budget cap.
- `share_local_candidates` (default false) — when false, the offer's
`local_addresses` list is empty and peers see only the reflexive
address. Enable per-node only for genuinely same-LAN deployments;
off-by-default eliminates the misleading-path failure mode for
the common case where peers are not on the same broadcast domain.
- `open_discovery_max_pending` (64) — caps queued open-discovery
retries; bounded by available outbound slots.
- `max_concurrent_incoming_offers` (16) — semaphore against offer
spam; excess offers are debug-logged and dropped.
- `advert_cache_max_entries` (2048) and `seen_sessions_max_entries`
(2048) — bound memory under ambient relay volume; overflow
evictions are debug-logged.
- `attempt_timeout_secs` (10), `replay_window_secs` (300)
- `punch_start_delay_ms` (2000), `punch_interval_ms` (200),
`punch_duration_ms` (10000)
- `advert_ttl_secs` (3600), `advert_refresh_secs` (1800)
Per-peer and per-transport flags:
- `PeerConfig.via_nostr: bool` — when true (and Nostr is enabled),
advert-derived addresses are appended as fallback dial candidates
after static addresses for that peer.
- `PeerConfig.addresses` is now `serde(default)` and may be empty
when `via_nostr: true`; validation requires at least one of the
two to be present per peer, and the error message names the
peer's npub.
- `UdpConfig.advertise_on_nostr: Option<bool>` and
`UdpConfig.public: Option<bool>` — UDP transports can be
advertised either as direct `host:port` (public = true) or as the
`addr: "nat"` sentinel that triggers rendezvous on the peer side.
- `TcpConfig.advertise_on_nostr` and `TorConfig.advertise_on_nostr`
— TCP and Tor onion endpoints can be advertised as directly
reachable.
- A reserved peer address `transport: udp, addr: "nat"` parses without
special-casing in YAML and routes through the bootstrap runtime.
Cross-field validation (`Config::validate`, called from `Node::new`
and `Node::with_identity`):
- Any transport with `advertise_on_nostr = true` requires
`node.discovery.nostr.enabled = true`.
- Any peer with `via_nostr = true` requires
`node.discovery.nostr.enabled = true`.
- A non-public UDP advert (`advertise_on_nostr = true`,
`public = false` — i.e. `udp:nat`) additionally requires at least
one `dm_relay` and at least one `stun_server`.
Surfaced as `ConfigError::Validation`.
== Node integration
`src/node/lifecycle.rs` is the main integration point.
- At node start (after transports are up, before TUN), if Nostr is
enabled and the feature is compiled in, `NostrDiscovery::start` is
invoked, the initial local overlay advert is built from the live
transport set and published, and the runtime handle is stored.
- The rx tick loop calls `poll_nostr_discovery` (feature-gated both
at method definition and call site), which refreshes the local
advert, drains bootstrap events, adopts established traversals,
schedules retries for failed traversals, and — under `policy:
open` — enqueues outbound retries for non-configured peers
visible in the advert cache, bounded by
`open_discovery_max_pending` and the remaining outbound slots.
- Outbound peer dialing is refactored to `try_peer_addresses`, which
first exhausts the static address list in priority order and only
then appends advert-derived fallback addresses; both lists run
through the same `attempt_peer_address_list` code path. The
`udp:nat` sentinel address triggers `NostrDiscovery::request_connect`
for the peer instead of a direct dial and returns `Ok(())`.
- `build_overlay_advert` walks operational transports, consults
per-instance `UdpConfig` / `TcpConfig` / `TorConfig` (matching by
optional transport instance name), and emits an `OverlayAdvert`
including `signalRelays` and `stunServers` when any UDP endpoint
is advertised as NAT.
- `adopt_established_traversal` is the bootstrap handoff API:
allocates a new `TransportId`, constructs a `UdpTransport` with
the user-supplied (or default) `UdpConfig`, calls the new
`adopt_socket_async` to reuse the punched socket verbatim,
registers the transport in the normal transport map, records it
in `bootstrap_transports`, and calls `initiate_connection` so the
normal handshake path runs. On failure, the transport is stopped
and removed cleanly and the set membership is rolled back.
- On clean shutdown, `NostrDiscovery::shutdown` is awaited so
background tasks stop before transports are torn down. (The
advert is not explicitly retracted; NIP-40 expiration plus the
next refresh from any live publisher supersedes it.)
New `Node` fields:
- `nostr_discovery: Option<Arc<NostrDiscovery>>` (feature-gated).
- `bootstrap_transports: HashSet<TransportId>` — per-peer UDP
transports adopted from NAT traversal, cleaned up via
`cleanup_bootstrap_transport_if_unused` whenever the link,
connection, peer, or pending-connect referencing them is removed.
Retry and error surface:
- `RetryState.expires_at_ms: Option<u64>` — optional absolute expiry
for a retry entry. `pump_retries` drops expired entries with an
info log. Used for open-discovery retries, which expire at two
times the advert TTL.
- New `NodeError::BootstrapHandoff(String)` returned from
`adopt_established_traversal` when the underlying transport
adoption fails or local address discovery fails.
- New `ConfigError::Validation(String)`.
- A small refactor extracts `Node::now_ms()` and reuses it across
lifecycle, rx-loop tick, and timeout bookkeeping.
== UDP transport
`src/transport/udp/`:
- `UdpRawSocket::adopt(std::net::UdpSocket, recv_buf, send_buf)`:
adopts an externally bound socket, makes it non-blocking, applies
the configured buffer sizes (warning if the kernel clamps), and
reports the resulting local address. Preserves the NAT mapping —
no rebind.
- `UdpTransport::adopt_socket_async(std::net::UdpSocket)`: the
`start_async` analogue for an already-bound socket, wiring the
async socket and recv task exactly as the fresh-bind path would.
- `Drop` impl for `UdpTransport`: if a transport is dropped while
still holding a recv task or socket (for example on error
teardown), aborts the task, clears the socket, and emits a debug
log so the cleanup is visible in tracing rather than silent.
== Logging and observability
Default `EnvFilter` demotes third-party relay-pool DEBUG output to
TRACE-only: `nostr_relay_pool`, `nostr_sdk`, and `nostr` are pinned
at INFO when our level is anything below TRACE, and at TRACE when
our level is TRACE — so the raw frames are still reachable when
explicitly asked for. RUST_LOG continues to override completely.
Concise one-line DEBUG events are emitted at the meaningful points
in the discovery / hole-punch sequence:
- `advert: published` (event id, relay count, endpoints, ttl)
- `advert: peer cached` (notify-loop ingress for non-self)
- `advert: resolved` (cache hit / relay fetch outcome)
- `traversal: initiator starting`
- `traversal: initiator STUN observed` (reflexive, local count)
- `traversal: offer sent` (session id, relay count, event id)
- `traversal: answer received` (accepted, reflexive, local)
- `traversal: initiator punch succeeded` (remote addr)
- `traversal: offer received` (responder side)
- `traversal: responder STUN observed`
- `traversal: answer sent`
- `traversal: responder punch succeeded`
Npubs are shortened to `npub1<4>..<4>` and event/session ids to
their first 8 hex characters.
Other operator-facing logs:
- `UdpTransport` adoption and drop paths log at info / debug.
- `adopt_established_traversal` logs at debug on entry and info on
successful return, tagged with peer npub, session id, transport
id, and both socket endpoints, so the bootstrap handoff is
traceable end-to-end alongside the `UdpTransport::drop` log.
- `cleanup_bootstrap_transport_if_unused` logs at debug when the
reference-count check drops an adopted transport.
- `connect_peer` tags its entry `debug!` with `peer_npub` so
downstream STUN, punch, and handshake logs for the same peer
correlate for operators.
- Advert-cache and seen-sessions overflow evictions log at debug so
mis-sized caps are visible under ambient relay volume.
- Gift-wrap unwrap failures on `SIGNAL_KIND` events log at trace
(hot path: fires for every unrelated signal event on the same
relay).
- Traversal-offer handler failures log at debug. Expected conditions
such as punch timeout on symmetric NAT are covered there; real
problems are reported upstream via `BootstrapEvent::Failed`.
- Inbound-offer rate-limit messages name the governing config field
(`max_concurrent_incoming_offers`) and state that the offer was
rate-limited rather than failing.
== Tests
- 18 new unit tests in `src/discovery/nostr/tests.rs` covering advert
encoding, signal envelope round-trip, STUN parsing, punch-packet
codec, and replay-window enforcement. Run under the
`nostr-discovery` feature.
- Config-validation tests in `src/config/mod.rs` covering the three
cross-field invariants and YAML parsing of the full
`node.discovery.nostr` block plus `peers[].via_nostr`, empty
`addresses` with `via_nostr: true`, and a `udp: nat` address.
- `src/node/tests/bootstrap.rs` integration tests that drive a
synthetic traversal (bound UDP socket pair + synthetic peer
identity) through `adopt_established_traversal` and assert the
Noise handshake completes over the adopted socket.
- Punch-planner tests assert reflexive-before-LAN ordering and that
same-LAN scenarios still include the LAN target in the plan.
- `testing/nat/` Docker NAT lab harness:
- Local `strfry` relay, local STUN responder, and one or two
router containers performing `iptables` NAT.
- Node LAN interfaces are provisioned with explicit `veth` pairs
injected into the node and router namespaces so every packet
traverses the router namespace (plain Docker bridges are not
used for the LAN).
- `cone` scenario: both peers behind full-cone-emulation NAT
(SNAT with source-port preservation, inbound DNAT back to the
single LAN host regardless of remote source); asserts UDP
traversal succeeds and link remote addresses are on the router
WAN subnet.
- `symmetric` scenario: `MASQUERADE --random-fully`; asserts UDP
traversal fails and TCP fallback converges over router-
published WAN addresses.
- `lan` scenario: both peers share a LAN subnet; asserts LAN
addresses are preferred over reflexive ones.
- Cleanup tears down all profile-gated services
(`--profile cone --profile symmetric --profile lan`) so no
orphan containers survive a run.
- `testing/scripts/build.sh` builds the Docker test image with
`--features "tui nostr-discovery"` by default so NAT-harness
binaries include bootstrap support.
== CI
- Linux release build and nextest unit-test job both use
`--features "gateway nostr-discovery"` so the feature-gated code
and its unit tests compile and run in CI.
- Three new integration matrix entries (`nat-cone`, `nat-symmetric`,
`nat-lan`) invoke `testing/nat/scripts/nat-test.sh`, collect
`docker compose logs` on failure, and always stop containers.
== Packaging and operations
- `packaging/common/fips.yaml` ships a fully commented
`node.discovery.nostr.*` block, plus documented
`advertise_on_nostr` / `public` examples under the UDP transport,
an `advertise_on_nostr` example under TCP, and a `via_nostr: true`
example under the static peer section with both a direct
`host:port` UDP address and a `udp: nat` fallback.
- `.github/workflows/package-openwrt.yml`: NIP-94 release event
publishes target the new default relay set.
== Documentation
- `README.md`: overlay discovery + NAT traversal moved from
"Near-term priorities" into "What works today".
- `docs/design/fips-intro.md`: rewrites the paragraphs that
previously described Nostr discovery and NAT traversal as future
work; describes the shipped mechanism and the feature gate.
- `docs/design/fips-transport-layer.md`: drops the "(future
direction)" qualifier from the Nostr Relay Discovery section,
expands with the `udp:nat` advertisement and bootstrap handoff
description, and updates the Current State callout.
- `docs/design/fips-mesh-layer.md`: notes that mid-session NAT
rebinding (roaming) and initial NAT traversal (Nostr path) are
distinct mechanisms.
- `docs/design/fips-configuration.md`: documents the full
`node.discovery.nostr.*` surface, including the three resource
caps and `share_local_candidates`.
- `docs/design/fips-nostr-discovery.md`: design and configuration
reference for the shipped mechanism, including the empty-
`addresses`-with-`via_nostr` shorthand.
- `docs/proposals/nostr-udp-hole-punch-protocol.md`: adds an
Implemented status callout, clarifies that the punch socket is
per-peer and per-attempt rather than shared with the application
listener, aligns field names with the shipped JSON
(`sessionId`, `issuedAt` / `expiresAt`, `reflexiveAddress`,
`localAddresses`, `stunServer`), sets the `d`-tag to
`fips-overlay-v1`, names the kind as 37195, and notes that
advertised STUN entries are informational.
- `docs/proposals/README.md`: adds a Status column and marks the
hole-punching proposal Implemented.
- `CHANGELOG.md`: Unreleased > Added entry covering the discovery
path, STUN/punch path, configuration surface, and Docker NAT lab.
Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
|
||
|
Johnathan Corgan
|
be0708ac9b |
Bring acl test module into the test tree
src/node/tests/acl.rs was added by PR #50 but never declared in src/node/tests/mod.rs, so none of its 4 unit tests ran. The tests themselves still compile and pass against current master code — the fix is a one-line mod declaration. Test count goes from 1031 to 1035. No code under test changes. This only adds previously-dormant coverage of the ACL enforcement call sites (outbound connect, inbound msg1, outbound msg2). |
||
|
Johnathan Corgan
|
3edca4a84f |
Notify initiator when inbound ACL rejects a Noise XX handshake
Under Noise XX the responder only sees the initiator's static key in msg3, so the inbound-handshake ACL check cannot fire until msg3 has already been processed. By then the initiator has received msg2, passed its own outbound ACL check, and promoted its side of the peering. Silently tearing down the responder's state left the initiator as a "connected" peer with no traffic until the link-dead timeout fired tens of seconds later; the acl-allowlist integration test on the next branch timed out at the 5s convergence check on blocked nodes. The responder now sends an encrypted Disconnect on the freshly-completed Noise session before tearing down. The initiator decodes it via the existing handle_disconnect path and cleans up the zombie peer within one RTT. DisconnectReason::Other is used instead of SecurityViolation so the wire payload does not name the ACL mechanism; the behavioural signature (explicit reject right after handshake) is the only observable leak. Supporting changes: - Add send_encrypted_link_message_raw in node/mod.rs that operates on a raw NoiseSession + indices without going through self.peers, since the peer has not yet been promoted at the reject site. - Declare the acl test module in node/tests/mod.rs. It had been orphaned since PR #50 so none of its tests ran; two of them no longer compiled against the current next branch. Fixed the two and replaced test_inbound_msg1_denied_by_acl (asserts IK-era behaviour that cannot happen under XX) with test_inbound_msg3_denied_triggers_disconnect, a full end-to-end UDP test covering both the responder cleanup and the initiator's Disconnect-driven cleanup. |
||
|
Johnathan Corgan
|
e5e6054229 |
Merge branch 'master' into next
# Conflicts: # src/bloom/filter.rs # src/node/bloom.rs |
||
|
Johnathan Corgan
|
03f6db58e8 | Merge branch 'maint' | ||
|
Johnathan Corgan
|
db4b32110c |
Validate bloom filter fill ratio on FilterAnnounce ingress
A malformed FilterAnnounce whose fill ratio produces an implausibly high false-positive rate is mostly useless for routing and, once merged into our outgoing filter via bitwise OR, propagates the saturated state to tree peers one hop per announce tick. A saturated filter also made estimated_count() return f64::INFINITY, which compute_mesh_size summed into its cached estimate. handle_filter_announce now rejects inbound FilterAnnounce whose derived FPR exceeds `node.bloom.max_inbound_fpr` (new config field, default 0.05 ≈ fill 0.549 at k=5). Rejection is silent on the wire, logs at WARN, and increments a new `bloom.fill_exceeded` counter. The peer's prior stored filter and filter_sequence are left unchanged so a single rejected announce does not wipe the peer's existing contribution to aggregation. After a successful outgoing FilterAnnounce send, a rate-limited WARN fires if our own filter's FPR exceeds the same cap, surfacing aggregation drift. Limited to once per 60 seconds via a new Node.last_self_warn field. BloomFilter::estimated_count() now takes max_fpr and returns Option<f64>. Returns None for saturated filters (regardless of cap) or when the filter's FPR exceeds max_fpr. Callers updated: debug logs render None as "—", the Debug impl uses f64::INFINITY as "no cap" and prints "saturated" instead of inf, control-socket JSON emits null, and compute_mesh_size propagates None into the already- Option<u64> estimated_mesh_size field. |
||
|
Johnathan Corgan
|
14d2a4f2df |
Merge branch 'master' into next
Integrates PR #50 (peer ACL enforcement) into the XX handshake architecture. ACL enforcement points adapted for XX's deferred identity learning: InboundHandshake check moves from handle_msg1 to handle_msg3 (responder learns initiator identity), OutboundHandshake check remains in handle_msg2 (initiator learns responder identity). Borrow scopes restructured to release connection borrows before authorize_peer calls. |
||
 Alex XieandJohnathan Corgan
|
745b523ac6 |
Add peer ACL enforcement with reloadable allow/deny files (#50)
Implement TCP Wrappers-style peer access control using /etc/fips/peers.allow and /etc/fips/peers.deny files. Evaluation order: allow overrides deny, default permit when no files exist. Three enforcement points: outbound connect (before dialing), inbound handshake (msg1 receipt, after restart/rekey classification), and outbound handshake completion (msg2, before peer promotion). Files support npub, hex pubkey, host alias, and ALL wildcard entries with automatic mtime-based reload. Adds fipsctl acl show query, 954-line acl module with unit tests, and a 6-node Docker integration harness (testing/acl-allowlist/) exercising insider, outsider, and allowed-remote scenarios. CI matrix entry included. Closes #50 Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com> |
||
|
Johnathan Corgan
|
fd9a8d132d | Merge branch 'master' into next | ||
|
Johnathan Corgan
|
5cdcff7386 |
Merge branch 'maint' into master
# Conflicts: # CHANGELOG.md |