mirror of
https://github.com/jmcorgan/fips.git
synced 2026-08-09 08:14:42 +00:00
2eea20a216fc979cb4a379917ce627d7055060f2
162
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Johnathan Corgan
|
2eea20a216 |
tcp: drive inbound connection cap from node.limits.max_connections
The per-transport TCP inbound cap was hardwired to 256 and never read node.limits.max_connections, so raising max_connections was a silent no-op for inbound TCP. Resolve the effective cap with precedence: explicit per-transport max_inbound_connections, then node-wide max_connections, then the built-in default of 256. Established peers remain bounded node-wide by add_connection, so deriving the per-transport raw-accept ceiling from max_connections does not admit more real peers across multiple transports. Add effective_max_inbound on the TCP transport with a node_max_connections setter wired from create_transports, plus a precedence unit test. |
||
|
Johnathan Corgan
|
ea9c7f2d8d |
mesh-size: union all peer filters, not just tree peers
Estimate the OR-union cardinality over self plus every connected peer's inbound filter, dropping the parent/child tree gating in compute_mesh_size. Filter propagation is split-horizon, so cross-links advertise near-complete mesh views; unioning all peers yields the same set as the tree-only union in steady state (OR dedups overlap and no filter can over-count) while damping the node-count flap on parent switches, since dropping the parent no longer collapses the upward leg. This also removes the estimate's dependence on tree-declaration cache freshness. Rename the debug-log child_count to contributor_count, adapt the two membership-invariant tests to all-peers semantics, and add a test that the estimate stays stable across a parent drop when a healthy cross-link is present. |
||
|
Johnathan Corgan
|
d364933ca5 |
node: re-push TreeAnnounce when a peer advertises a worse root
A node with a single tree peer has its periodic parent re-evaluation disabled (it needs at least two peers for a meaningful comparison), so it depends entirely on its peer pushing a TreeAnnounce for it to attach. That push happens once at promotion time plus on the parent's slow periodic no-change re-broadcast. If the one-shot attaching announce is lost, the single-uplink node falls back to self-root and cannot recover until the next periodic re-broadcast (reeval_interval_secs later), stranding it out of the tree and unreachable end-to-end in the interim. Make tree-position exchange self-healing on the receive path: when an accepted TreeAnnounce advertises a root strictly worse (higher NodeAddr, since election is smallest-wins) than our own, echo our current declaration back to that peer. A stranded self-root node's announce now provokes its better-rooted peer to re-push its real position immediately, so the node re-attaches within a round-trip instead of waiting for the periodic cadence. Echo only in that one direction. If the peer's root is lower (better) than ours, we are the stale side: the peer would ignore our worse root anyway and we converge via the parent re-evaluation that follows, so echoing back is pure waste and would double announce traffic in the learning direction during a root change or partition merge. Equal roots are already converged. The echo is bounded by the existing per-peer 500 ms tree-announce rate limiter and is a no-op once the peer adopts our root, so it adds no traffic in a converged mesh. Add a spanning-tree unit test that drives a converged child back to self-root with its peer-ancestry view of the root cleared (modelling the lost attaching announce), and asserts the root re-pushes on the resulting root disagreement and the child re-attaches. |
||
|
Johnathan Corgan
|
974e146bb9 |
node: demote routine per-peer and capacity-cap events from info/warn to debug
On a saturated public-mesh node the connection-lifecycle and capacity-cap events fire continuously and drown out the genuinely notable INFO/WARN lines. Demote them to debug and drop a redundant duplicate: - FMP K-bit cutover promotion (encrypted): info -> debug - "Connection promoted to active peer" (handshake): info -> debug, and remove the duplicate "Inbound peer promoted to active" line that shadowed it on the inbound path - "Peer restart detected" (handshake): info -> debug - "Peer removed and state cleaned up" (dispatch): info -> debug - "Rejecting inbound TCP connection (max_inbound_connections reached)" (tcp): warn -> debug - "Congestion detected, CE flag set on forwarded packet" (forwarding): warn -> debug - "Removing peer: link dead timeout" (mmp): warn -> debug These are expected, high-frequency conditions on a busy public node (new and reconnecting peers, ECN CE marking, the inbound connection cap, and link-dead churn), not operator-actionable signals. |
||
|
Johnathan Corgan
|
180950badf |
node: estimate mesh size by OR-union of filters instead of summing cardinalities
The mesh-size estimator summed the per-filter cardinality of the parent filter and each child filter, which assumes those filters are perfectly disjoint. When they overlap -- a stale or oversized parent filter, or a routing loop -- the sum over-counts and inflates the reported mesh size to as much as several times the true size. Estimate the cardinality of the OR-union of the contributing filters (self + parent + children) once instead. OR is idempotent, so any overlap is deduplicated: the result equals the old sum in the disjoint case and stays correct under overlap. The union is seeded from a clone of a contributing filter so it keeps that filter's size class, and a filter whose size class does not match is skipped rather than panicking. The refuse-to-estimate behavior on a saturated or above-cap filter is preserved. Add a regression test with overlapping parent and child filters where the naive sum over-counts and the union estimate tracks the distinct member count. |
||
|
Johnathan Corgan
|
de327e4527 |
test: run node-level mesh tests over an in-process loopback transport
Add a Loopback variant to TransportHandle backed by an unbounded in-process channel and a shared address-to-receiver registry, so node-level multi-node tests deliver packets directly between nodes instead of over real localhost UDP sockets. This removes the kernel UDP receive-buffer overflow that dropped handshake packets when many tests ran in parallel under CPU contention, and lets the large-network convergence tests run reliably in the default suite again (their parallel-load ignore markers are removed). The new transport and its enum variant are cfg(test)-gated, so the daemon build is unaffected. |
||
|
Johnathan Corgan
|
4af3730be6 | fmp: authenticate inbound frame against pending session before K-bit cutover promotion | ||
|
Johnathan Corgan
|
25fe87ff60 |
fmp: bound rekey msg1 retransmission and make link-dead heartbeat rekey-aware
The FMP rekey msg1 resend driver retransmitted indefinitely with no cap and no abandon, so a rekey that never completed kept resending msg1 forever. Give it a retransmission budget: cap resends at handshake_max_resends with exponential backoff and abandon the rekey cycle cleanly once the budget is exhausted, mirroring the FSP session rekey msg3 driver. With the cap in place the link-dead heartbeat can safely become rekey-aware: check_link_heartbeats now suppresses teardown while a rekey is in progress with msg1 budget remaining, instead of reaping a link that is still actively carrying rekey-handshake traffic. The suppression terminates deterministically (the budget abandons on exhaustion, cutover clears the in-progress flag), so a genuinely dead link is still reaped on the next cycle. Adds a rekey_msg1_resend_count counter on ActivePeer reset at every rekey-clear and cutover site, msg1 resend-budget unit tests, and two-node heartbeat suppression/resume/regression integration tests. |
||
|
Johnathan Corgan
|
e6e2a06879 |
node/tests: stabilize parallel-load flake-class large-network tests
Raise the in-process backpressure headroom in make_test_node_with_mtu (request an 8 MiB recv_buf_size on UdpConfig and grow packet_channel from 256 to 8192) to reduce localhost-UDP receive overflow under parallel-CPU scheduler contention, and mark the large-network convergence tests #[ignore] so cargo test --lib stays green by default. The ignored tests remain runnable on demand with --ignored or --test-threads=1. |
||
|
Johnathan Corgan
|
5b229c03bf |
node: skip Msg1 → Msg2 reply when at max_peers cap
Move the max_peers cap check in handle_msg1 forward, from the late check inside promote_connection (which fires after Msg2 has already been built and put on the wire) to an early position after identity verification but before index allocation and the Msg2 send. When the gate fires for a net-new identity, the Msg1 is silent-dropped — no response goes back to the peer, no AEAD compute or wire bytes are spent. Bypass preserved for known peers (reconnect / cross-connection): if the sender's NodeAddr is already in self.peers, or if a pending outbound connection is in flight to the same identity, the gate is skipped so legitimate maintenance traffic continues to work. The late check inside promote_connection is intentionally retained as defense-in-depth against future call sites or a disconnect racing between the early-gate decision and promotion. Wire-cost rationale: a 45 s tcpdump at saturation observed ~3.6 cap-denials/s steady-state, each previously paying the full Noise IK responder crypto + Msg2 (~104 B) on the wire before being rejected. The bigger value is cleaner peer-side semantics — the peer no longer sees a fake-completed handshake whose data frames subsequently fail decryption locally. Two new unit tests cover the cases: - handle_msg1_silent_drops_at_cap_for_new_peer drives a wire-pumped Msg1 from a fresh identity into a saturated node and asserts no Msg2 reaches the sender socket. Stash-verifies as FAIL on the pre-fix tree (Msg2 hits the wire) and PASS post-fix. - handle_msg1_admits_existing_peer_at_cap drives a Msg1 from an identity already in self.peers and asserts the gate does not evict it. This is a regression check (the no-gate tree behaves the same way here, but the test guards against an accidental future gate that breaks known-peer admit). |
||
|
Johnathan Corgan
|
d4687e5d30 |
node: gate outbound connection initiation on max_peers
node.limits.max_peers was honored only on inbound msg1 admission (handshake.rs handle_msg1 returns PeerLimitExceeded when peers.len is at the cap). Four outbound initiation paths proceeded unconditionally at capacity: auto-reconnect retries (process_pending_retries), Nostr-mediated discovery's BootstrapEvent::Established adoption (poll_nostr_discovery), NAT-traversal punch initiation (the outgoing side of the offer/answer/punch sequence in the Nostr discovery runtime), and NAT-traversal punch response (the incoming side of the same sequence). A saturated node burned CPU, UDP probes, STUN observations, and Nostr relay traffic on connections that the inbound gate would reject the moment they reached msg1. Introduce Node::outbound_admission_check (peers.len < max_peers, or true when max_peers == 0 as the no-cap sentinel) and gate the four paths. The discovery runtime lives in a separate task and does not hold a Node reference; bridge via an Arc<AtomicBool> the runtime reads and Node refreshes once per tick from outbound_admission_check. The atomic granularity is intentionally loose: one-tick lag is acceptable because the inbound msg1 gate continues to be the authoritative cap, and in-flight handshakes started below the cap are allowed to complete. Inbound gate at handshake.rs is unchanged. |
||
|
Johnathan Corgan
|
df43ac79b9 |
node: skip parent explicitly in compute_mesh_size children loop
The mesh-size estimator's children loop relied on the cached peer_declaration(parent_id).parent_id() != my_addr check to exclude the parent. That cached view briefly disagrees with our own latest my_declaration().parent_id() during the window between a local parent-switch and the new parent's next inbound TreeAnnounce: the peer-declaration cache still names us as the parent's parent, so the parent is iterated as if it were a child and its (typically dominant) bloom cardinality is added a second time. Symptom: estimated mesh size displayed in fipsctl show status and fipstop nearly-but-not-exactly doubles during tree rebalancing. Make the invariant structural with an explicit peer_addr == parent_id skip at the head of the children loop. Per-peer 500 ms rate-limiter and overall recompute cadence are unchanged. Adds a regression test that constructs the stale-peer-declaration scenario directly and asserts the parent is not double-counted. |
||
|
Johnathan Corgan
|
ffd78440a8 |
node: periodically re-broadcast TreeAnnounce on no-change in check_periodic_parent_reeval
Closes the eventually-consistent gap in spanning-tree state distribution. Every existing send_tree_announce_to_all call site gates on a local state-change event (parent switch, self-root promotion, ancestry change, peer promotion, parent loss). Once a partition latches — for example a parent-switch announce stranded in the brief cross-init handshake swap window, where the announce arrives on a session-index whose decrypt-worker entry has been unregistered — neither side's state changes again, so neither side ever re-broadcasts. The existing 60 s check_periodic_parent_reeval was a re-evaluation, not a re-broadcast: it short-circuited silently on no-change. Production-side healing depended on incidental link churn; lab harnesses with stable docker-bridge links had no equivalent path. Add a final else branch that fires send_tree_announce_to_all unconditionally on the no-change path, alongside the existing switch and self-promote arms. Receivers coalesce by sequence comparison (ParentDeclaration::is_fresher_than) and short-circuit at the `if !updated` gate in handle_tree_announce; same-sequence repeats drop silently with no cascade. The per-peer 500 ms rate-limiter is well below this 60 s cadence and does not suppress the heartbeat broadcast. The fix is a general protocol-robustness improvement: it addresses any in-flight TreeAnnounce loss class, not only the specific cross-init swap-window drop site. testing/static/scripts/rekey-test.sh BASELINE_CONVERGENCE_TIMEOUT 60 -> 65 so a partition healed by the periodic broadcast at T+60 lands inside the convergence window. wait_for_full_baseline early-exits on PASS, so successful reps see no extra wall-clock. |
||
|
Johnathan Corgan
|
4d5380604a |
node: don't drive connect-on-send from the rx_loop tick path
The tick body's per-peer check_* loops (heartbeats, bloom announces, MMP reports, tree announces) called transport.send for every active peer, which on TCP/Tor fell through to a 5 s connect-on-send wait for any peer whose pool entry was not yet established. That wedged the entire tick body for the full connect_timeout_ms per unreachable peer; under post-restart convergence on a high-peer mesh, this cascaded into multi- second tick stalls. On master, the same mechanism also starved the per-tick control-snapshot republish and pushed fipsctl queries onto an mpsc fallback that was itself queued behind the wedged rx_loop, producing the 5-second fipsctl head-of-line pattern operators observed on loaded nodes. Gate send_encrypted_link_message_with_ce on transport.connection_state before the send: proceed only when Connected; on None, kick off a non-blocking background connect (idempotent — TransportHandle::connect dedupes against the connecting pool and spawns the timeout-bounded TcpStream::connect inside its own tokio task) and fail this send fast with a clear "transport connection not ready" error. A subsequent tick retries once the pool has an entry. The reconnect lifecycle (check_link_heartbeats, process_pending_retries, poll_pending_connects) is unchanged. The connect-on-send branch in transport.send_async itself remains in place for code paths that legitimately need synchronous connect (e.g., explicit operator-driven fipsctl connect). |
||
|
Johnathan Corgan
|
f396d71826 |
node: deterministic tie-breaker for cross-init NAT traversal adoption
When both peers' Nostr-mediated UDP punches complete within the same scheduling window, each side's `BootstrapEvent::Established` event arrives with `is_connecting_to_peer` already true: each side received an inbound msg1 from the peer's pre-punch outbound attempt, which created a connecting-state record. The deduplication skip then fires on both sides, neither installs the fresh traversal socket as canonical, and the peer-adoption budget (45 s) expires. Cross-node wall-clock alignment of the skip log line in observed failures was within ~1 ms — simultaneous dual- fire under contention, the dual-initiation pattern. Apply the deterministic NodeAddr tie-breaker already used at `handlers/handshake.rs:269` for rekey dual-initiation and in `peer::cross_connection_winner` for cross-connection resolution. Smaller NodeAddr wins as adopter: enumerate the in-flight connections whose `expected_identity` points at this peer, tear them down via the canonical `cleanup_stale_connection` helper, and fall through to `adopt_established_traversal`. Larger NodeAddr loses and keeps the existing `continue` semantics; the loser's in-flight outbound is reconciled by `handle_msg1`'s cross- connection logic when the winner's fresh msg1 arrives over the adopted socket. `cleanup_stale_connection` visibility bumped from module-private to `pub(in crate::node)` so it is callable from `lifecycle.rs`. The defensive re-check inside `adopt_established_traversal` itself is left as-is — after the outer cleanup the winner reaches it with `is_connecting_to_peer == false`, so the inner skip won't trip. The `BootstrapEvent::Failed` arm is unchanged: there is no winning outcome on dual failure, and the existing skip + retry-schedule semantics are correct. |
||
|
Johnathan Corgan
|
6e5cb8965f |
Make FSP session rekey hitless under packet loss and reordering
An FSP session rekey could leave the two endpoints holding different key sets for a brief window: if a handshake message was lost in transit, one side rotated to the new keys while the other did not. Traffic sealed in one key epoch then reached a peer still on the other epoch and failed to decrypt, producing bursts of AEAD decryption failures and dropped connectivity until a later rekey cycle reconverged the pair. Choreographing the cutover order cannot close this window: any fixed ordering still leaves a skew that packet reordering widens. Make rekey correctness independent of cutover timing by overlapping the key epochs on the receive path. During a rekey transition the receiver trial-decrypts each frame against every live session it holds: current, the not-yet-promoted pending session, and the draining previous session. The K-bit becomes a hint that orders the trial-decrypt cascade rather than a hard gate, and a frame that authenticates against the pending session is itself the cutover signal. No rotation ordering and no packet reordering can then cause a decryption failure. The pre-rekey Noise session is held in the `previous` slot until the peer has demonstrably moved off it. Its drain deadline is anchored on the most recent frame the peer authenticated against that slot, refreshed each time the trial-decrypt cascade lands there, rather than on a fixed wall-clock timer started unilaterally at the local cutover. A peer that never received the new keys keeps authenticating against `previous` and the slot stays live; without this, a fixed timer would erase the only key set that could decrypt the peer's frames, producing a permanent silent decrypt failure on a live data path. A peer that never catches up is handled by the existing FSP session liveness path rather than by silent decrypt failure. The lost-handshake liveness gap is closed separately by retransmitting the third rekey handshake message until the peer is confirmed on the new keys, with a bounded retry budget after which the rekey cycle is cleanly abandoned and retried on the next timer. Adds unit tests covering the trial-decrypt cascade (epoch selection, promotion on pending decrypt, reordered old-epoch stragglers after cutover, per-slot replay-window integrity), the msg3 retransmission lifecycle, and the peer-progress-aware drain retirement. |
||
|
Johnathan Corgan
|
49bd210480 |
cache: scope coord cache invalidation to entries actually affected by topology change
Replaces the unconditional `CoordCache::clear()` calls at parent-switch, become-root, and loop-detection sites with two targeted invalidation methods scoped to what actually makes an entry stale: - `invalidate_via_node(node_addr)`: drop entries whose cached destination ancestry contains `node_addr`. Used at parent-position- change sites — our prefix changed, so destinations downstream of us have stale-prefix coords. - `invalidate_other_roots(current_root)`: drop entries rooted under a different root than the current one. Used at root-change sites. Under the previous global flush, parent switches blanked the cache across the board, leaving `find_next_hop` returning `None` for every non-direct-peer destination until the cache passively re-warmed via incoming TreeAnnounces / SessionSetup. Surgical invalidation preserves entries that remain correct after the topology change. The cached coord describes a destination's tree position; that position only goes stale relative to our own routing decisions when our own prefix changes (entries we are downstream of) or the root changes (entries in a different tree). Peer removal does not invalidate cached coords: `Node::find_next_hop` recomputes the next-hop decision on every call against the current peer set, bloom filters, and tree state, and Discovery already triggers on `no route to destination` errors when a destination becomes unroutable through us. The peer-removal site retains the original "no cache invalidation" behavior. Each method returns the count of entries removed for observability. Unit tests cover each method against the cases enumerated in the acceptance criterion. |
||
 Martti MalmiandJohnathan Corgan
|
87d1af0269 |
nostr: ignore stale traversal for active peers
Skip BootstrapEvent::Established and BootstrapEvent::Failed dispatch in poll_nostr_discovery for peers that are already connected or actively handshaking. Without these guards, stale traversal events arriving after a peer connected through a different path would either attempt to adopt a redundant socket against the live connection (Established) or poison the per-peer failure-state cooldown and trigger redundant retraversal via schedule_retry / try_peer_addresses (Failed). The four guard sites use a new is_connecting_to_peer helper extracted from the existing closure inside initiate_peer_connection; the helper checks for an in-flight outbound handshake state. adopt_established_traversal gains a defense-in-depth check returning PeerAlreadyExists when called against an already-promoted peer, so the invariant holds if a future caller bypasses the outer dispatch guard. Side benefit: narrows a cooldown-poisoning vector previously available to an attacker injecting stale failure events for an active peer. Test coverage for the new behavior: - test_try_peer_addresses_skips_connected_peer - test_try_peer_addresses_skips_connecting_peer - test_nostr_traversal_failure_skips_connected_peer (Failed-arm event injection) - test_nostr_traversal_established_skips_connected_peer (Established-arm event injection, mirror of the Failed test) - test_adopted_traversal_skips_already_connected_peer (adopt_established_traversal defense-in-depth) CHANGELOG entry under [Unreleased] / Fixed. Closes #87 |
||
|
Johnathan Corgan
|
4f3d2f8471 |
rekey: apply symmetric jitter to desynchronize dual-initiation
Add a per-session signed jitter offset (uniform [-15, +15] seconds) to the rekey timer triggers in check_rekey (FMP) and check_session_rekey (FSP). The configured `node.rekey.after_secs` becomes the nominal interval rather than a floor; mean is preserved. Desynchronizes both endpoints in symmetric-start meshes so the dual-initiation race stops occurring rather than being resolved after the fact by the smaller-NodeAddr tie-breaker. Per-session storage means each rekey cutover reconstructs the session and redraws the jitter naturally — successive cycles get independent offsets, preventing drift back into sync. |
||
|
Martti MalmiandJohnathan Corgan
|
5cda4a9a55 |
noise: switch ChaCha20-Poly1305 backend to ring (BoringSSL asm)
The chacha20 crate (RustCrypto) ships SSE2 + soft backends only — on
aarch64 (Apple Silicon, ARM Linux servers, Docker on M-series Macs) it
falls through to a portable software impl at ~600–800 MB/s/core. ring
0.17 wraps BoringSSL's hand-tuned ChaCha20-Poly1305, which dispatches
to NEON on aarch64 and AVX2/AVX-512 on x86_64 — typically 3-5 GB/s/core
on the same hardware.
Same wire format. ChaCha20-Poly1305 is byte-deterministic for a given
(key, nonce, plaintext, aad), so any correct AEAD implementation
produces identical ciphertext. The full noise test suite covers this
implicitly: IK and XK roundtrip handshakes, replay window correctness,
multi-message nonce sequencing, and 100-message stress all pass at
1129/1129 (the lib's full `cargo test` count) — these only succeed if
ring's output matches what the receiver's existing replay-window
decrypt path expects.
Implementation notes:
* `LessSafeKey` (and `UnboundKey`) deliberately do not implement
Clone for safety. `CipherState`'s manual Clone impl rebuilds it
from the retained 32-byte key — cheap for ChaCha20-Poly1305 since
construction is essentially a key copy + a constant-time check.
* The keyed AEAD is now cached in `CipherState.cipher` instead of
being re-derived per packet. This was already a perf win for the
chacha20poly1305 backend (`new_from_slice` per packet was hot in
profiles); for ring it's a bigger win because `LessSafeKey`
construction also derives the Poly1305 key.
* Public `Vec<u8>`-returning API preserved. New module-private
`seal`/`open` helpers wrap ring's `seal_in_place_append_tag` /
`open_in_place` so the per-packet allocation pattern is local to
one place.
* `EndToEndState::Established` triggers `clippy::large_enum_variant`
after the swap (`NoiseSession` grew from ~600 to ~1.5 KB because
ring precomputes the Poly1305 key state at construction). That
precomputation is the win — boxing the variant would re-add an
indirection per packet and work against it. `#[allow]`'d at the
enum decl with a justifying comment.
ring is widely deployed (rustls, hyper-rustls, AWS SDK, …) and a
pure-Rust crate (uses BoringSSL's asm via a vendored build). It
introduces no new C toolchain requirements that aren't already there
for any rustls user.
Bench data from a downstream consumer of this crate (Docker e2e,
DURATION=10, identical hardware before/after, aarch64 Linux on
Apple Silicon):
2-node direct (A↔B):
TCP 1-stream 437 → 1097 Mbps (2.51×)
TCP 4-stream 439 → 1109 Mbps (2.53×)
TCP 8-stream 445 → 1069 Mbps (2.40×)
UDP @1000 Mbit 599/40% loss → 1000 Mbps lossless
ping under load ~0.6 ms (unchanged)
3-node forced transit (A → C → B):
TCP 1-stream 438 → 1019 Mbps (2.33×)
TCP 4-stream 421 → 982 Mbps (2.33×)
TCP 8-stream 443 → 1031 Mbps (2.33×)
UDP @1000 Mbit 475/52% loss → 1000 Mbps lossless
ping under load 7.68 ms / 215 ms max → 0.72 ms / 3.6 ms max
The relay-path lift is the cleanest tell on the bottleneck: the
transit node was crypto-bound (single-threaded soft chacha couldn't
keep up with offered rate), so the queue accumulated under load. With
NEON the relay isn't crypto-bound and the queue stops accumulating —
the 215ms ping-tail collapses to 3.6ms.
|
||
|
2d18d019d6 |
Evict stale overlay advert when retry hits NoTransportForType
The advert cache inside fetch_advert is read-only on hit — once a peer's overlay advert is cached, every subsequent lookup returns the same endpoints regardless of whether they still work. So when a peer rebinds its NAT (or its STUN-discovered port flaps), connection retries to that peer dial the same dead address forever, even with exponential backoff firing at the right cadence. Observed in deployment: macOS daemon's view of a Linux peer would "regress" — peer marked rch=False after a brief link-dead window, then hours of "Retry connection initiation failed: no operational transport for any of <npub>'s addresses" with no recovery. Manual pause+resume of the daemon (which restarts the FIPS endpoint and forces fresh advert fetches) was the only way out. When initiate_peer_connection / a retry tick returns NodeError::NoTransportForType, fire-and-forget refetch_advert_for_stale_check on the peer's npub. This re-fetches kind 37195 from advert_relays; if the relay has a newer advert it replaces the cached entry, if it has nothing it evicts the cached entry. Either way the next retry tick goes to fresh data instead of looping on the same dead endpoint. Mirrors the existing stale-advert sweep that runs from the BootstrapEvent::Failed (NAT-traversal-streak) path, but covers the direct-UDP-retry path which never crosses that streak threshold. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
64cc30df12 |
Schedule retry on startup peer-init failure
When initiate_peer_connections() runs at boot, address resolution can fail for an entire peer (no operational transport for the configured transport types, all addresses unreachable, NAT rebind invalidated cached endpoints, etc.). Before this change the failure was logged and silently forgotten — the peer entry stayed in a dead state forever, accepting incoming pings but unable to answer them, until the daemon was manually restarted. The retry plumbing (schedule_retry / process_pending_retries with exponential backoff) already exists and is wired into the post-handshake failure paths (BootstrapEvent::Failed, MMP dead-link timeout, handshake timeout). The startup loop just wasn't calling it. Mirror the BootstrapEvent::Failed path: on a startup peer-init error, parse the peer's npub and call schedule_retry so the peer recovers without operator intervention. Includes a regression test that asserts retry_pending is populated when initiate_peer_connections() fails for a peer with no operational transport. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
6ce1406664 |
Refetch overlay advert before every retry, not only on NoTransportForType
The previous fix (6ebca3e) only refetched the advert when retry returned NodeError::NoTransportForType (cache returned no addresses at all). But the much more common stale-cache failure mode is: cache returns an endpoint that LOOKS valid (the address it had last week, before the peer's NAT rebound), the dial succeeds at the IP layer, the handshake times out, MMP fires, schedule_reconnect adds the entry back to retry_pending, next retry hits the same cached endpoint, dials it again, times out again. Loop forever — no NoTransportForType ever fires because the cache has data, just dead data. Move refetch_advert_for_stale_check to before each retry attempt unconditionally. Cheap (one Filter query against advert_relays with a 2s timeout, bounded by the retry backoff cadence), and replaces the cache only if the relay has a newer advert or evicts if the relay has nothing. Keeps the retry loop pinned to relay ground truth instead of whatever the cache happened to learn at startup. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Martti MalmiandJohnathan Corgan
|
e81fd4b477 |
tree: fix TreeAnnounce ancestry when self is smallest visible NodeAddr
When a node was the smallest-NodeAddr peer it could see (no smaller neighbor available as a parent), the spanning-tree state was promoting it to root. But the ancestry it advertised on the next TreeAnnounce still referenced its previous parent's path, so receiving peers rejected the announce with `invalid ancestry: advertised root X is not the minimum path entry Y`, blocking mesh transit on any path that needed to traverse this node. Detect the self-root transition explicitly in `TreeState::become_root` and rebuild the advertised ancestry to start from self. Also surface the same path through the MMP receive handler so a stale ancestry inherited across reconnect is corrected eagerly rather than waiting for the next observation tick. Adds 80 unit tests in `tree::tests` covering self-root transitions, mid-chain ancestor disappearance, and ancestry validation against the new root, plus a regression in `node::tests::spanning_tree` for a 3-node chain where the middle node's only parent (the smallest-addr peer) goes away — previously it would advertise an ancestry rejected by both endpoints; now it self-roots cleanly. |
||
 Martti MalmiandDev
|
fac4450694 |
node: drain packet_rx / tun_outbound_rx in batches in run_rx_loop
The run_rx_loop's `tokio::select!` was costing one full scheduler hop + futex per inbound packet and per outbound TUN packet. Under sustained load that capped throughput at one event per scheduler quantum — independent of CPU (which sat near-idle) because every iteration parked the worker, woke it via futex, processed one event, then parked again. After the await on `packet_rx.recv()` / `tun_outbound_rx.recv()` fires, drain up to 256 additional ready items via `try_recv()` in a tight inner loop before yielding back to `select!`. `biased` ordering gives the data-plane branches priority over tick / control / DNS under sustained load. The 256 cap is empirically tuned to keep the worker on a busy stream between yield points (a contiguous burst of ~256 MTU-sized packets ≈ 400 KB of contiguous traffic) while still bounding the inner loop so a flood on one branch can't starve the periodic tick or control socket. Lower caps (64) left perf on the table; higher caps (1024+) delayed tick handling visibly under stress. Pairs with the recvmmsg(2) change in the previous commit: the kernel UDP queue now hands packets to `packet_rx` in 32-batches, and the rx_loop drains them without a per-packet scheduler hop. |
||
|
Martti MalmiandJohnathan Corgan
|
f0bb29ff6e |
node: inherit primary UDP config when adopting NAT-traversal sockets
`Node::adopt_established_traversal` was constructing the adopted UDP transport with `UdpConfig::default()` — MTU 1280, default recv/send buffer sizes, default accept/advertise flags. If the operator had configured a higher MTU on the primary `[transports.udp]` listener (e.g. 1500 on a path where larger frames are known viable), full-sized tunnel datagrams sent over the NAT-traversed link would exceed the adopted socket's MTU and get dropped at the socket layer with no visibility into why throughput collapsed. Inherit the primary UDP config (MTU + recv/send buffer sizes + accept / advertise flags) and clear the bind / external-address fields since the adopted socket is already bound. Lookup tries `transport_name` first so operators with multiple named `[transports.udp.<name>]` listeners pick up inheritance from the matching listener, and falls back to the unnamed `Single` listener so single-instance configs work unchanged. The previous default of MTU 1280 was deliberately the IPv6 minimum, the only value guaranteed to survive arbitrary middlebox paths. With this change, operators who set their primary listener higher (based on known-clean LAN topology) will have NAT-traversed flows initially attempting that higher MTU and possibly black-holing on tighter paths until reactive `MtuExceeded` recovery kicks in. Documented in the adoption call-site comment so future readers understand why the conservative default went away. Discovered in a downstream consumer where a `MESH_TUNNEL_MTU=1320` / encrypted wire ~1426B produced silent packet drop on every session that had been promoted onto a NAT-traversed link. Adds two sibling tests in `src/node/tests/bootstrap.rs` pinning the new behaviour for the `Single` and `Named` config variants. |
||
|
Johnathan Corgan
|
4cdf382038 |
tree: propagate mid-chain ancestor swaps to leaves
A leaf node's my_coords could go stale after an upstream mid-chain ancestor swap, leaving non-parent destinations with 100% loss until either the parent or the depth also changed. The broadcast gate in handle_tree_announce's `else if !is_root && parent_id == from` branch compared only (root, depth). A swap that altered an interior ancestor without changing root or depth (e.g. A->B->C reorganizing to A->D->C while keeping (A, depth=2)) was silently dropped one hop below the swap node. Downstream nodes' coords paths then drifted from the real tree topology, defeating greedy distance routing for any destination whose path crossed the unrepresented section. Widen the gate to compare the full my_coords.node_addrs() so mid-chain swaps propagate to leaves the same way root/depth changes already did. The gate body's bloom-marking is adjusted in step so the wider gate doesn't generate empty/redundant FilterAnnounces to every peer on every mid-chain swap propagation: mark_changed_peers replaces mark_all_updates_needed in the gate body (parent_id is unchanged in this branch, so outgoing filter content is typically unchanged, and mark_changed_peers correctly marks zero peers in that case), and the unconditional mark_update_needed(*from) at the top of handle_tree_announce is removed (bloom exchange initiation is already handled at handshake completion, and ongoing content changes are picked up naturally by mark_changed_peers in handle_filter_announce when peers send their next filter). Required surface change: peer_inbound_filters in src/node/bloom.rs upgraded from private to pub(super) so the gate body can call it. Verified in a 6-node depth-4 docker reproduction under tc/netem-induced parent flapping: a depth-4 leaf's ancestry_changed counter advances with upstream parent switches while bloom_sent stays at zero matching the pre-change steady-state baseline. |
||
|
Johnathan Corgan
|
a62a0a6cf4 |
nostr: suppress retraversal of cross-FMP-version peers
Open-discovery NAT traversal succeeds at the UDP layer regardless of what FMP-protocol version the peer speaks. When the daemon discovers a peer running a different FMP version (e.g. a v0/v1 mix during a mid-rollout window, or a misconfigured peer in the same advert namespace), the punch sequence completes, the socket is adopted via `Node::adopt_established_traversal`, and we initiate an FMP handshake. The peer drops our msg1 at its own version-gate and we drop their msg1/msg2 at `Unknown FMP version, dropping`. Neither side advances the handshake. Today the bootstrap transport sits idle until the 31s stale- handshake timeout, drops, and the open-discovery sweep ~30s later fires the full STUN+offer+answer+punch sequence again — every minute, indefinitely, against peers the handshake literally cannot complete with. Add a `Node::bootstrap_transport_npubs` map populated alongside `bootstrap_transports` at adopt time. The rx loop reverse-maps the transport_id → npub on version-mismatch and bumps the discovery layer's `failure_state` to a long structural cooldown via the new `NostrDiscovery::record_protocol_mismatch` API. The next sweep skips the npub for `protocol_mismatch_cooldown_secs` (default 86400 = 24h, separate from the 30-min transient-failure `extended_cooldown_secs`). One-shot WARN per fresh observation. Repeat mismatches inside the cooldown window are silent (the failure_state method returns false when an existing comparable cooldown is already in place). The handshake/transport teardown chain is unchanged — the fix is specifically about preventing the *next* sweep cycle from re-traversing. Cleared on `cleanup_bootstrap_transport_if_unused` and on the adopt-failure rollback path so completed handshakes don't leave stale entries behind. Four new unit tests in `failure_state.rs` cover fresh-entry signaling, repeat-suppression inside the window, streak-pin behavior for `show_peers` rendering, and post-cooldown re-arming. |
||
|
Johnathan Corgan
|
7fc890b7a2 |
session: mirror proactive PathMtuNotification into path_mtu_lookup
The TUN-side TCP MSS clamp consults `path_mtu_lookup` (FipsAddress- keyed) when sizing outbound TCP flows. Until now, only the reactive `MtuExceeded` handler mirrored the bottleneck MTU into that store; the proactive end-to-end `PathMtuNotification` echoed by the destination updated only `MmpSessionState.path_mtu`, leaving the TUN mirror stale. On stable long-lived paths, the proactive echo can tighten the session-canonical MTU well before any transit router fires a `MtuExceeded` for those flows (since all current traffic is already sized by the tighter session value). New TCP flows opened during that window get clamped by the discovery-time value rather than the session-canonical one, leading to PMTU-D loss until the reactive path eventually fires. Mirror the post-apply MTU into `path_mtu_lookup` whenever `apply_notification` returns true, with the same tighter-only semantics as the reactive mirror — never loosen the clamp. Gated on the bool return so spurious writes don't happen on rejected increases or no-op same-value notifications. Four new unit tests exercise the empty-lookup write, tighten- existing, keep-tighter-existing, and no-session-no-op paths, parallel to the existing reactive-mirror test trio. |
||
|
Johnathan Corgan
|
2f95929862 |
nostr: public-IP discovery for UDP/TCP advert publication
When a UDP transport had `advertise_on_nostr: true` + `public: true` + `bind_addr: 0.0.0.0:NNNN`, the advert builder previously read the kernel's `local_addr()`, found `0.0.0.0`, filtered it out (correctly — wildcard isn't a valid advertised endpoint), and silently emitted no UDP endpoint in the published advert. Operators on AWS EIP / GCP / Azure setups (where binding to the public IP directly is impossible because 1:1 NAT does the address translation off-host) had no way to advertise UDP without binding to a specific local IP — and no log explaining what was happening. TCP had the same shape, with no `public: true` precondition. Three pieces, layered. UDP gets zero-config autodiscovery via STUN; both UDP and TCP get an explicit operator-supplied override; the fall-through path now logs loudly instead of silently skipping. UDP public-IP autodiscovery (STUN) ---------------------------------- In the UDP `is_public()` + wildcard-bind branch, run a one-shot STUN observation against an ephemeral UDP socket on the daemon's configured `stun_servers`. Take the reflexive IPv4 (the STUN-reported port is the ephemeral source port and is discarded), combine with the configured listener port for the advert (`udp:<reflexive-ip>:<port>`). Works on AWS EIP / GCP / Azure 1:1-NAT setups because STUN sees the public-Internet egress IP and the bind port is preserved through 1:1 NAT. Result is cached per-transport on a new `public_udp_addr_cache` field on `NostrDiscovery` (keyed by `TransportId.as_u32()`). Asymmetric cache TTL: a successful observation is cached for `advert_refresh_secs` (default 30 min) so we don't STUN every refresh tick. A failed observation is cached for only 60s (`PUBLIC_UDP_ADDR_FAILURE_TTL`) so a transient STUN flake at startup retries within ~a minute and the advert grows its UDP endpoint as soon as STUN starts working — rather than waiting the full 30-min cycle. The shared `observe_traversal_addresses` STUN helper had a hard-coded 2s per-server response wait, right for the per-traversal flow (latency-sensitive — 3 STUN servers worst-case = 6s) but too short for the one-shot advert-publish startup discovery. Parameterized `per_server_timeout` on the helper, with two named constants in `stun.rs`: `TRAVERSAL_STUN_TIMEOUT = 2s` (existing call sites) and `ADVERT_STUN_TIMEOUT = 5s` (new public-UDP discovery path). Both use `tokio::time::timeout_at` under the hood, so success returns immediately — the timeout is only the worst case. `external_addr` override (UDP + TCP) ------------------------------------ New `external_addr: Option<String>` field on `transports.udp.*` and `transports.tcp.*` for explicit advertise-as override. Takes precedence over both the bound `local_addr` and (for UDP) the STUN-derived autodiscovery. Required for TCP on cloud-NAT setups (AWS EIP, GCP/Azure external IPs) where binding to the public IP directly fails with `EADDRNOTAVAIL` because the public IP isn't on a host interface — the network fabric does 1:1 NAT off-host. Without this field the operator's only TCP path was "leave advert off" or "find a way to make the public IP locally bindable." For UDP, `external_addr` is optional but useful as a deterministic alternative to STUN. Operators who want to skip STUN egress, whose STUN servers are blocked, or who want the daemon to not depend on external services for advert content can specify it explicitly. The accessor parses two shapes: - Bare IP (`"54.183.70.180"` or `"2001:db8::1"`): combines with the configured `bind_addr` port. - Full host:port (`"54.183.70.180:8443"` or `"[2001:db8::1]:443"`): used verbatim — useful for port-forward setups where the externally-visible port differs from the bind port. Final precedence in `Node::build_overlay_advert` (now async, only caller `refresh_overlay_advert` was already async): - UDP: `external_addr` → non-wildcard `local_addr` → STUN → loud warn - TCP: `external_addr` → non-wildcard `local_addr` → loud warn Loud warns instead of silent skips ---------------------------------- The wildcard-bind fall-through paths now log a `warn!` pointing at the operator-side fixes: - UDP: "set transports.udp.external_addr, bind to a specific public IP, or ensure node.discovery.nostr.stun_servers is reachable" - TCP: "Either set external_addr to the public IP (recommended for cloud 1:1-NAT setups) or bind explicitly to the public IP" Replaces the silent skip that previously cost operators a debugging session when the advert mysteriously contained only the Tor onion endpoint. Tests ----- 11 new unit tests in `src/config/transport.rs` covering the parser (IPv4/IPv6, bare/full, malformed) and the accessor (UDP with default bind, UDP with explicit port override, UDP unset, TCP without bind_addr, TCP with bind_addr, TCP with full socket-addr override, parse_bind_port for IPv4/IPv6/malformed). The 38-test nostr suite still passes. CHANGELOG entries under `[Unreleased]` Fixed. |
||
|
Johnathan Corgan
|
bcc9c525d3 |
nostr: per-peer NAT-traversal failure suppression and clock-skew handling
Public-test daemons with populous open-discovery caches generate
sustained NAT-traversal-failure WARN volume (~140/hour, ~3500/day)
against cache-learned peers that have gone offline — their adverts
are absent from major Nostr relays but cached entries persist until
their advertised `valid_until` expires. The daemon kept publishing
offers indefinitely under exponential backoff with no per-peer
suppression, drowning operator signal and hammering relays. A
parallel concern: the strict freshness check at signal.rs silently
rejected offers under modest clock skew (now_ms() anchors to
SystemTime once at startup, so post-startup NTP step adjustments
don't propagate on long-uptime daemons), indistinguishable from
"peer is offline."
Six independent improvements layered on the existing retry logic.
Per-npub WARN log rate-limit
----------------------------
New `FailureState` struct on `NostrDiscovery` records per-npub
`last_warn_at_ms`. Subsequent failures inside `warn_log_interval_secs`
(default 5 min) emit DEBUG instead of WARN. Each WARN now also
carries `consecutive_failures` and remaining `cooldown_secs` so
operators can read the trajectory without grepping multiple lines.
Per-npub consecutive-failure counter + extended cooldown
--------------------------------------------------------
After `failure_streak_threshold` (default 5) consecutive failures
against a peer, the next `extended_cooldown_secs` (default 1800)
of attempts are suppressed by pushing
`retry_pending[npub].retry_after_ms` past the cooldown wall. The
open-discovery sweep also consults `cooldown_until` and increments
a new `skipped_cooldown` counter so a peer whose `retry_pending`
was cleared by max_retries doesn't get re-enqueued during the
cooldown window. Caps offer-publish rate per dead peer regardless
of how often the sweep tries to re-enqueue.
Stale-advert eviction on streak-threshold transition
----------------------------------------------------
On the threshold-crossing transition (one-shot, not every
subsequent failure), `tokio::spawn` an active re-fetch of the
peer's Kind 37195 advert from `advert_relays`. Three outcomes:
- absent on relays → cache evicted; sweep won't re-enqueue
(peer is genuinely gone).
- newer `created_at` → cache refreshed + streak reset
(peer republished; allowed to retry immediately).
- same → cache untouched; cooldown stands.
Cost: ~one fetch per dead peer per 30-min cooldown cycle, vs
hundreds of offer publishes/hour today.
Clock-skew tolerance on freshness check
---------------------------------------
`signal.rs` `validate_offer_freshness` and
`validate_traversal_answer_for_offer` now allow ±60s grace beyond
strict TTL. Both return a new `FreshnessOutcome` enum so callers
can DEBUG-log when an offer/answer was only accepted via the grace
window. `FRESHNESS_SKEW_TOLERANCE_MS` is hard-coded — loosening
this past minutes erodes the freshness/replay security boundary
and operators tend to tune in the wrong direction.
NTP-style skew estimate (offer_received_at echo)
------------------------------------------------
Added optional `offerReceivedAt: Option<u64>` field to
`TraversalAnswer` payload. Responder fills it with `now_ms()` at
offer-receipt time. Initiator computes the standard NTP offset
formula `((T2-T1) + (T3-T4)) / 2` against the round-trip and
DEBUG-logs when `|skew| ≥ 30s`. Skew is also stashed in
`FailureState` and surfaced in `show_peers`. Non-breaking — older
responders that don't fill the field still produce valid answers,
and `estimate_clock_skew` returns `None`.
Per-peer state in `show_peers` JSON
-----------------------------------
Each peer entry in `show_peers` now carries:
"nostr_traversal": {
"consecutive_failures": <u32>,
"in_cooldown": <bool>,
"cooldown_until_ms": <u64 | null>,
"last_observed_skew_ms": <i64 | null>
}
Always emitted (schema-stable); values populated when discovery is
enabled and the npub has a recorded entry. Required a new public
`Node::nostr_discovery_handle()` accessor and refactored
`FailureState`'s internal Mutex from `tokio::sync` to `std::sync`
(operations never hold across await), which lets the synchronous
`show_peers` handler call `snapshot()` directly without the
dispatcher becoming async.
New config knobs (under `node.discovery.nostr`)
-----------------------------------------------
failure_streak_threshold: 5
extended_cooldown_secs: 1800
warn_log_interval_secs: 300
failure_state_max_entries: 4096
Tests
-----
12 new unit tests:
- 5 in `tests.rs` covering freshness strict / tolerated / rejected
outcomes, NTP skew estimation, and the backward-compat None case
when the responder didn't fill `offer_received_at`.
- 7 in `failure_state.rs` covering streak/warn-rate-limit state
transitions, cooldown active vs expired semantics,
success-resets-streak, observed-skew records, and size-cap
eviction by oldest `last_failure_at`.
CHANGELOG entries added under `[Unreleased]` Fixed.
|
||
|
Johnathan Corgan
|
e08f42e3cc |
Pin discovery state machine: open-discovery sweep + per-attempt lookup timeout
Cover two adjacent runtime behaviors in the discovery state machine
that were previously unpinned at the test level.
1. Open-discovery startup sweep iterate-filter-queue contract.
Cover the runtime sweep behavior: iterate advert cache, apply
skip-filters (own-pubkey, already-connected peers), queue eligible
entries to retry_pending. The config layer was tested but the sweep's
own filtering logic was unpinned.
src/discovery/nostr/runtime.rs: add #[cfg(test)] impl block with
three pub(crate) helpers — new_for_test() builds a minimal
NostrDiscovery with empty cache and no relays/background tasks (uses
fresh nostr::Keys signer + Client::builder().autoconnect(false));
cached_advert_for_test() wraps an OverlayEndpointAdvert into a
CachedOverlayAdvert valid for 1h; insert_advert_for_test() writes
direct to the advert_cache RwLock. All three vanish from release
builds via cfg-gating.
src/node/lifecycle.rs: visibility-only widen on
run_open_discovery_sweep from private async fn to
pub(in crate::node) async fn so the in-tree test can drive it
directly. Same pattern as already-pub(in crate::node) handlers in
src/node/handlers/.
src/node/tests/discovery.rs: add #[tokio::test]
test_open_discovery_sweep_queues_eligible_skips_filtered. Builds
Node + Arc<NostrDiscovery>, injects 3 adverts (eligible, already-
connected peer, own-pubkey), invokes the sweep, asserts retry_pending
contains exactly the eligible entry with matching peer_config npub
and the two filtered entries do NOT appear.
2. Per-attempt timeout state machine in check_pending_lookups.
Cover the central new behavior of
|
||
|
Johnathan Corgan
|
81c0547bdf |
Pin consecutive decrypt-failure counter and threshold-20 force removal
Cover the security-relevant defensive signal: sustained decrypt failures indicate key drift or active probing; threshold-trip force- removes the peer. src/peer/active.rs (+43): two unit tests on the counter struct itself - test_increment_decrypt_failures_monotonic asserts each increment_decrypt_failures() call returns count+1 for at least 25 iterations - test_reset_decrypt_failures_zeroes_counter asserts the reset helper zeroes a non-zero counter and is idempotent src/node/tests/decrypt_failure.rs (new, 93 lines): end-to-end test - Builds a Node + connected peer via existing make_completed_connection / add_connection / promote_connection harness so peers_by_index is exercised, not just peers - Drives the peer to threshold-20 by calling handle_decrypt_failure 20 times; asserts iterations 1..20 leave the peer registered with monotonically increasing counter, then iteration 20 evicts from both peers and peers_by_index src/node/handlers/encrypted.rs: visibility-only widen on handle_decrypt_failure from private to pub(in crate::node) so the in-tree test can drive the threshold logic without re-implementing it. Same pattern as the already-pub(in crate::node) handle_encrypted_frame in the same file. Threshold pinned: DECRYPT_FAILURE_THRESHOLD = 20 at src/node/handlers/encrypted.rs:11. |
||
|
Johnathan Corgan
|
00f4a4c7af |
Pin bloom-not-closer-than-tree-parent fall-through to greedy tree
Add test_routing_bloom_hit_not_closer_falls_through_to_tree to
src/node/tests/routing.rs covering the regression class fixed in
|
||
|
Johnathan Corgan
|
037a965a93 |
Mirror reactive MtuExceeded into path_mtu_lookup
When a transit forwarder drops an oversized data packet and reports the bottleneck back via MtuExceeded, the receive-side handler already updates per-session MmpSessionState::path_mtu (used by PTB synthesis to feed kernel TCP). It did not, however, update path_mtu_lookup — the per-destination map the TUN reader/writer consult at TCP MSS clamp time. So forward-path-asymmetry flows kept clamping at the discovery reverse-path value (too generous for the actual forward-path budget) on every subsequent SYN. Add the missing write at the same point apply_notification runs. Keep the tighter of existing-or-new — the clamp must never loosen. Same write-shape as seed_path_mtu_for_link_peer. Tests: - Three focused unit tests on handle_mtu_exceeded for the empty, tighten, and keep-tighter cases. - Extended test_multihop_pmtud_heterogeneous_mtu to assert the lookup tightens after the wire-level MtuExceeded propagation, alongside its existing PathMtuState assertion. Adds two #[cfg(test)] accessors on Node (path_mtu_lookup_get / path_mtu_lookup_insert) and bumps handle_mtu_exceeded to pub(in crate::node) for direct test invocation. |
||
|
Johnathan Corgan
|
ae607431eb |
Per-destination TCP MSS clamping at the TUN boundary
Adds source-side TCP MSS clamping informed by per-destination path MTU learned via discovery, with a conservative IPv6-minimum-derived ceiling for cold flows where discovery has not yet completed. Closes the multi-hop default-config TCP wedges observed in production where a sender's local-floor MSS exceeds what some intermediate forwarder hop is willing to carry: silent drops, no PTB feedback through the userspace TUN to the kernel TCP stack, retransmits at the same too- large MSS, application connection times out. ## Architecture A new `Arc<RwLock<HashMap<FipsAddress, u16>>>` field `path_mtu_lookup` on Node mirrors the per-destination path MTU in a form accessible from sync TUN reader/writer threads. A new `per_flow_max_mss` helper in `src/upper/tun.rs` reads the lookup at SYN-clamp time and returns the appropriate ceiling for the flow. Three write sites populate `path_mtu_lookup`: 1. **Discovery originator branch** of `handle_lookup_response`: the path MTU bottleneck accumulated through the reverse path lands here when a LookupResponse arrives at the originator. Same value also lands in `coord_cache` per the existing `insert_with_path_mtu` API. 2. **FMP peer-promotion seed** (`seed_path_mtu_for_link_peer`): when an FMP link-layer peer is promoted to active, the local outgoing-link MTU on the peer's transport seeds the lookup. Tighter existing values (learned via discovery) are preserved; the seed only writes when no entry exists or the existing value is looser than the link MTU. Without this seed, directly-configured peers (auto_connect / static peer config) would leave `path_mtu_lookup` empty for their FipsAddress because the FSP session establishes without ever issuing a LookupRequest. 3. **Target-edge fold at `send_lookup_response`**: when a node is the discovery target, it folds its own outgoing-link MTU to the response's next-hop into `path_mtu` before sending. Without this fold, the response leaves the target with `path_mtu = u16::MAX` and only intermediate transits min-fold; the target's first reverse-path hop is never represented in the bottleneck calculation. Refactored the existing transit- side min-fold into a shared `apply_outgoing_link_mtu_to_response` helper called from both sites. ## Read-side: per_flow_max_mss Two TUN call sites consume the lookup: - Outbound `handle_tun_packet` clamps SYN MSS using packet[24..40] (IPv6 destination) as the lookup key. - Inbound `TunWriter::run` clamps SYN-ACK MSS using packet[8..24] (IPv6 source). When the lookup contains a learned value, the helper computes `min(global_max_mss, effective_ipv6_mtu(path_mtu) - 60)` where 60 is IPv6 (40) + TCP (20) headers and `effective_ipv6_mtu` accounts for the FIPS encapsulation overhead. When the lookup is empty for a destination — the cold-flow case — the helper returns `min(global_max_mss, IPv6-minimum-derived ceiling)`. RFC 8200 mandates every IPv6 path accept ≥1280-byte packets, so the IPv6-minimum-derived MSS (1280 - 77 - 60 = 1143) fits any compliant path. Without this conservative ceiling, the first SYN to a destination with no learned path MTU exits the TUN at the kernel-natural MSS (TUN MTU - 60), and the application connection wedges silently before discovery completes for a corrected second SYN to fire. The fix is provably safe: the ceiling is taken with `min` against the local global so operators with even tighter local floors are never loosened upward. Subsequent flows pick up the actual learned per-destination value once discovery (or the FMP-promotion seed for direct peers) populates the lookup. ## Diagnostic logging All write and read sites emit instrumentation suitable for operators bisecting a wedged path: - `debug!` log on every `path_mtu_lookup` write (discovery originator path and FMP-promotion seed path), showing the FipsAddress, written value, prior value, and post-write map size. `warn!` on poisoned-lock failure path. - `trace!` log per `per_flow_max_mss` call covering every fall-through branch (wrong addr_bytes length, non-fd::/8 prefix, lookup poisoned, no entry for destination, empty-lookup conservative ceiling) and the success path. trace level filters out under normal log settings; capture with `RUST_LOG=info,fips::node::handlers::discovery=debug,fips::upper::tun=trace`. ## Tests 15 new unit tests across 3 files: - `per_flow_max_mss` (8 tests in `src/upper/tun.rs::tests`): empty-lookup conservative ceiling, empty-lookup global-smaller floor, learned-value-overrides-conservative, per-destination smaller, per-destination larger capped by global, non-fips addr, short addr slice, per-destination independence. - `seed_path_mtu_for_link_peer` (4 tests in `src/node/tests/unit.rs`): seed when empty, keep tighter existing, tighten looser existing, no-op for unknown transport. - Discovery integration (3 tests in `src/node/tests/discovery.rs`): apply_outgoing_link_mtu_to_response on unknown peer no-op, two-node target-edge fold (path_mtu reflects target-edge link), three-node chain transit min-fold (existing test, updated for target-edge inclusion). Two pre-existing discovery tests had assertions updated to account for the target-edge fold: - `test_response_path_mtu_two_node`: previously asserted `u16::MAX` (no transit to min-fold); now asserts 1280 (the test transport MTU, folded in by send_lookup_response). - `test_response_path_mtu_four_node_chain`: previously asserted 1350 (transit MTUs only); now asserts 1280 (target-edge MTU is the bottleneck). - `test_transit_forwards_when_mtu_sufficient`: previously asserted 1400 (transit MTU only); now asserts 1280 (target- edge MTU is the bottleneck). ## Verification Local CI on this commit: 29/29 suites pass, 1105 lib tests pass, clippy --all-targets --all-features -D warnings clean, cargo fmt clean. Production deploy verified via trace capture across the managed fleet: cold-flow conservative ceiling branch fires on first SYN, learned-lookup branch takes over once discovery completes, both behaviors observable end-to-end at the SYN MSS on the wire. No wire-format change. No config-format change. |
||
|
Johnathan Corgan
|
da5d23ccb7 |
Document nostr-nat ephemeral UDP transport MTU choice
Adopted ephemeral UDP transports created by adopt_established_traversal() default to UdpConfig::default() (MTU=1280, IPv6 minimum) when the bootstrap runtime hands a socket without an explicit transport_config override. This is by design: NAT-traversal middlebox MTU is unpredictable and the IPv6 minimum is the only value guaranteed by spec to survive arbitrary paths. Add an explanatory comment at the call site so future readers find the rationale without spelunking through ISSUE-2026-0013, and so any future change to the inheritance behavior is a deliberate decision rather than an accidental refactor. No behavior change. |
||
|
Johnathan Corgan
|
8448e38510 |
Make Node::transport_mtu() deterministic across restarts (TCP black hole fix)
Default-config TCP flows between fips peers were stalling completely (cwnd-pinned, 0 bps for 10s+) on a non-trivial fraction of restarts. Reproducible with iperf3 between any two peers. Root cause: `Node::transport_mtu()` iterated `self.transports.values()` (HashMap with default RandomState hasher) and returned `handle.mtu()` of the first one whose `is_operational()` returned true. Two stacked sources of non-determinism stacked on each other: HashMap iteration order is randomized per-process via RandomState, and async transport `.start()` completion order races each daemon restart. The returned value drives the TCP MSS clamp ceiling computed once at TUN init (src/upper/tun.rs:501-524) and stored as immutable max_mss in the reader/writer thread state. When the picker landed on a transport with MTU > 1357 (any non-UDP-1280 in the standard fleet defaults), `max_mss > 1220` (kernel's natural fips0-MTU-derived MSS), the daemon's clamp was silently a no-op, and the kernel emitted 1220-byte segments. Those wrap into 1280-byte IPv6 → 1357-byte fips datagrams that exceed UDP-1280 transports at any forwarding hop, causing silent drops with no PTB feedback to the kernel TCP stack. Fix: return min across operational transports instead of first-iterated. With UDP-1280 in the configured set (the common case), `transport_mtu = 1280`, `max_mss = 1143 < 1220`, daemon's clamp engages, MSS=1143 reaches the wire, packets fit, throughput recovers. Empirical green light from a single-UDP-config end-to-end test: iperf3-without-`-M` recovered to ~21 Mbps with no operator-side nft TCPMSS rules. Adds three unit tests: - transport_mtu_returns_min_across_operational: pin selection to smallest MTU when multiple operational transports differ. - transport_mtu_fallback_when_no_operational_transports: 1280 fallback. - transport_mtu_min_with_single_operational: trivial single-transport case. The `effective_ipv6_mtu` field reported by `fipsctl show status` was also racy (consequence of the same bug); fixed by this change as a side effect. |
||
|
Johnathan Corgan
|
ab2edec2c6 |
Add Nostr open-discovery startup sweep with diagnostic logging
Under `node.discovery.nostr.policy: open`, the per-tick auto-dial in `queue_open_discovery_retries` was supposed to pick up adverts cached from the relay subscription backlog at startup, but in practice only adverts arriving live (after the daemon was up) were being dialed. Backlog adverts sat in the in-memory cache until they aged out. Adds a one-shot startup sweep that runs once per daemon start, gated identically to the per-tick sweep (`enabled` && `policy == open`), after a configurable settle delay so the relay subscription backlog has time to populate the advert cache. The sweep iterates the cache with the same skip-filters as the per-tick path (statically-configured peers, already-connected, retry-pending, connecting) plus a tighter age filter: only adverts whose `created_at` is within `startup_sweep_max_age_secs` of now are queued. Two new config fields under `node.discovery.nostr`: - `startup_sweep_delay_secs` (default 5) - `startup_sweep_max_age_secs` (default 3600 = one hour) Both are only consulted when `policy == open`; under any other policy the sweep is a no-op. Adds diagnostic logging to the open-discovery sweep so operators can verify what the auto-dial path is doing on each daemon bring-up: info-level on each retry-queued enqueue (with peer short-npub and advert age), and a one-line summary on every startup sweep and on any per-tick sweep that queues at least one retry. The summary breaks down skipped candidates by reason (age, configured, self, already-connected, retry-pending, connecting, no-endpoints, invalid-npub) — currently the path was silent so there was no operator-visible signal that the cache iteration was running. Refactors the existing `queue_open_discovery_retries` body into a shared `run_open_discovery_sweep(max_age_secs, caller)` helper so the per-tick and startup paths share filter/queue logic and only differ in the age filter and log label. Surfaces `created_at` from `NostrDiscovery::cached_open_discovery_candidates` (return tuple extended) so the age filter has the data it needs. Three new unit tests in `config::node::tests` cover the new defaults, YAML override round-trip, and partial-YAML default fallback. |
||
|
Johnathan Corgan
|
239cbdc4ba |
Fix Tor onion adverts missing port in Nostr overlay discovery
The Nostr overlay advert publisher serialized `transport: tor` endpoints as a bare `<onion>.onion` hostname with no port. The Tor address parser requires `<host>:<port>` form and rejected the bare shape with `expected host:port`. Any peer receiving a Tor-only advert went into a persistent retry-fail loop on jittered backoff until the advert aged out of the discovery cache. The bug had been latent for as long as Tor adverts have been published on Nostr, and was masked in deployments where every node also advertised a non-Tor transport (peers fell through to the working endpoint). Surfaced first on a deployment where Tor was the only advert path. Publisher now emits `<onion>.onion:<port>` using a new `transports.tor.advertised_port` config field that defaults to 443, matching the Tor `HiddenServicePort 443 127.0.0.1:<bind_port>` convention. Operators whose torrc uses a non-default virtual port can override. Adds a unit test that pins the publisher/parser contract: formats the advert exactly as the publisher does and asserts `parse_tor_addr` accepts the result; asserts the bare-onion form (the bug) does not parse, catching any future regression that drops the port again. Parser is unchanged (already correct). |
||
|
Johnathan Corgan
|
96c6b7dea8 |
Admit rekey msg1 from established peers when addr forms differ
Companion to the ethernet `accept_connections: false` rekey-deadlock fix from earlier this release: the same dual-init failure mode shows up over UDP when peers register by hostname, and the existing addr_to_link-only carve-out in `should_admit_msg1` doesn't cover it. The carve-out's first predicate keys `addr_to_link` by the literal `TransportAddr` that `initiate_connection` inserted, which is the hostname-form when a peer config carries a hostname (e.g., `core-vm.tail65015.ts.net:2121`). Inbound packets always arrive with numeric source addrs because `udp_receive_loop` builds the `TransportAddr` from the `SocketAddr` the kernel reports via `recvfrom`. `TransportAddr` equality is byte-exact, so the two forms don't match and the lookup misses. With `udp.accept_connections: false` (or `udp.outbound_only: true`, which forces it false) the gate then rejects the rekey msg1 from an established peer. The dual-init tie-breaker stalls because the loser side never produces msg2; both sides retry indefinitely and the winner side keeps logging "Dual rekey initiation: we win, dropping their msg1" at 1Hz. The earlier ethernet fix didn't generalize to this variant because ethernet TransportAddrs are always numeric MAC bytes — both the config-time form and the inbound-arrival form match identically. Add a second predicate to `should_admit_msg1`: an active peer's `current_addr()` matching `(transport_id, remote_addr)`. `current_addr` is updated and refreshed from inbound encrypted-frame source addrs (`handlers/encrypted.rs`), which are always numeric `SocketAddr`-form, so this catches the established peer regardless of how its `addr_to_link` key was originally inserted. The fast `addr_to_link` check stays first; the iteration over peers is bounded by peer count and only runs when the first predicate misses. Regression coverage in this commit: - Unit test `test_should_admit_msg1_admits_rekey_when_addr_form_differs` in `src/node/tests/handshake.rs`. Constructs the failing scenario in-process: `addr_to_link` populated with hostname-form key, peer's `current_addr` at the resolved numeric form, query with numeric form. Without the new predicate this fails immediately. - New integration topology `rekey-outbound-only` plus matching docker-compose profile. Same 5-node mesh shape as `rekey-accept-off` but `inject-config` sets `udp.outbound_only: true` on node-b and rewrites node-b's peer-c address from the numeric docker IP to the docker hostname (`node-c:2121`), reproducing the production hostname-vs-numeric mismatch. The test asserts no sustained "Dual rekey initiation: we win" log lines on any node (>10 = bug) and the existing rekey health checks catch the connectivity loss the loop produces. - `testing/ci-local.sh` and `.github/workflows/ci.yml` extended to run the new variant in the local sweep and the GitHub CI integration matrix alongside `rekey` and `rekey-accept-off`. Verified locally: full `bash testing/ci-local.sh` sweep passes 29/29 suites (23m 12s) with the new variant green; 1084 unit tests pass. |
||
|
Johnathan Corgan
|
e641eb5b5f |
Drop the nostr-discovery cargo feature flag
Make Nostr-mediated overlay discovery unconditional, mirroring the philosophy of PR #79's collapse of the tui/ble/gateway features in favor of platform cfg gates. nostr / nostr-sdk are pure Rust over WebSockets/TCP, so they build cleanly on every FIPS-supported platform — there is no need for the parallel feature gate. The flag was already in `default = [...]`, so no behavior change for anyone using `cargo build` without `--no-default-features`. Operators who explicitly disabled the feature will now find Nostr code present in the binary; the runtime check `node.discovery.nostr.enabled` still controls whether the runtime starts. Cargo.toml: - Remove the `[features]` table entirely. - Drop `optional = true` from `nostr` and `nostr-sdk`. Source: 27 cfg sites collapsed across 5 files — `src/discovery.rs`, `src/discovery/nostr/mod.rs`, `src/node/handlers/rx_loop.rs`, `src/node/lifecycle.rs`, `src/node/mod.rs`. Two `#[cfg(not(feature = "nostr-discovery"))]` fallback blocks (the udp:nat-without-runtime debug-log path and the "feature not compiled in" warning at startup) were removed as dead code; the always-on path already handles the missing-runtime case via `nostr_discovery: Option<NostrDiscovery>`. Packaging and tooling: - `packaging/openwrt-ipk/Makefile`: drop a stale `--features gateway` flag (the `gateway` feature was already removed in PR #79; this was a leftover that the build path tolerated only because cargo ignored unknown feature names). - `testing/scripts/build.sh`: drop `DEFAULT_CARGO_BUILD_ARGS=(--features nostr-discovery)`; defaults are empty. - `packaging/common/fips.yaml`: drop the "requires the nostr-discovery feature" comment from the discovery section. Bundled cleanup: - Apply `cargo clippy --fix` against three pre-existing warnings in `src/discovery/nostr/runtime.rs` and `src/discovery/nostr/stun.rs` (collapsed `if let Some` chain; two redundant `as i32` casts). These were always present but masked when the feature gate was off; they surface now that the code is unconditionally compiled. - `cargo fmt` settled two minor formatting drift sites in `src/bin/fips-gateway.rs` and `src/config/mod.rs`. Tests: 1083 passed, 0 failed, 4 ignored. clippy clean. fmt clean. |
||
|
Johnathan Corgan
|
37c2973e2f |
Test infrastructure overhaul: gateway robustness + full CI coverage
Single combined commit covering five interlocking pieces of test and CI work that landed during the v0.3.0-prep cycle. ## fips-gateway robustness - src/bin/fips-gateway.rs DNS upstream probe converted from a 3-second hard-fail to a bounded retry loop (5 attempts × 1s timeout, 1s sleep between attempts; ~10s worst case). Covers the cold-boot race where the daemon's TUN is up but the DNS responder at [::1]:5354 is still binding. Each failed attempt logs at INFO. In production the binary's retry is the live recovery mechanism; with retry it recovers silently instead of relying on Restart=on-failure (~5s blip + spurious ERROR per cycle). - packaging/debian/fips-gateway.service `ExecStartPre` now waits up to 30 seconds for the daemon's `fips0` TUN to appear before exec'ing the gateway binary. Eliminates the cold-boot race where the gateway exits with `fips0 interface not found` and recovers via `Restart=on-failure`, producing a 5-second blip and a spurious error log per restart cycle. - testing/docker/entrypoint.sh gateway-mode waits up to 30s for the daemon's DNS responder to bind [::1]:5354 (probes once per second with `dig @::1 -p 5354 ... test.fips`) before exec'ing fips-gateway. Belt-and-suspenders with the binary's own retry: in CI we want deterministic startup ordering. On timeout, fall through so the binary's probe reports the definitive error. ## Test infrastructure DNS bind migration to ::1 After session 359's daemon DNS-bind default flipped from `127.0.0.1` to `::1` (the production fix for ISSUE-2026-0002), the static-test infrastructure was carrying a stale workaround that overrode the default back to IPv4 loopback. The fips-gateway integration test exposed the divergence: the gateway probes its DNS upstream at `[::1]:5354` (production default) while the daemon was binding `127.0.0.1:5354` from the template override — IPv6-explicit sockets do not accept v4-mapped traffic, so the upstream probe exhausted retries and the gateway exited. - Drop the explicit `bind_addr: "127.0.0.1"` line from every test config that emits it: testing/static/configs/node.template.yaml, testing/chaos/configs/node.template.yaml, the sidecar heredoc in testing/docker/entrypoint.sh, testing/acl-allowlist/generate-configs.sh (six per-node blocks), testing/nat/scripts/generate-configs.sh, and the four tor templates under testing/tor/. Daemon picks up its production `::1` default. - Flip the dnsmasq forwarder for `.fips` in testing/docker/Dockerfile from `127.0.0.1#5354` to `::1#5354` so dnsmasq on the shared test image continues to reach the daemon. Template and Dockerfile must move together since most static suites resolve `<npub>.fips` via the test-image dnsmasq. ## rekey-accept-off integration variant + UDP unit test - New `rekey-accept-off` topology and docker-compose profile under testing/static/. 2-node variant where node-b runs with `udp.accept_connections: false`. Pins the regression class that ISSUE-2026-0004 fixed (cross-connection winner's rekey msg1 was being filtered by the accept_connections gate, breaking rekey). - testing/static/scripts/rekey-test.sh accepts REKEY_TOPOLOGY and REKEY_ACCEPT_OFF_NODES env vars; its inject-config subcommand applies the per-node `udp.accept_connections: false` edit, and the test asserts no sustained "Dual rekey initiation" log lines. - New UDP variant of `should_admit_msg1` admit-rekey unit test in src/node/tests/handshake.rs. ## ci-local.sh full integration coverage - New runner functions and dispatcher entries for `acl-allowlist`, `nat-cone` / `nat-symmetric` / `nat-lan`, `rekey-accept-off`, `dns-resolver`, `deb-install`. Each integrates with the existing summary tracking via `record`. - New `--with-tor` flag (off by default) gates `tor-socks5-outbound` and `tor-directory-mode` runners. Tor stays opt-in because both harnesses depend on the live Tor network and would introduce a flake source unrelated to the FIPS code. - New suite arrays (`ACL_SUITES`, `NAT_SUITES`, `DNS_RESOLVER_SUITES`, `DEB_INSTALL_SUITES`, `TOR_SUITES`) drive both the default sweep and `--list` output. - `run_suite` extended to accept the new suite names for `--only` invocations. ## GitHub CI matrix expansions - `gateway` matrix entry runs testing/static/scripts/gateway-test.sh against the existing docker-compose `gateway` profile. - `rekey-accept-off` matrix entry exercises the new topology with REKEY_ACCEPT_OFF_NODES=b. - `deb-install` matrix (debian12 + ubuntu24 + ubuntu26) runs testing/deb-install/test.sh with privileged systemd containers. ~5-7 min cold cache, ~2 min warm per distro. Self-contained: builds its own .deb in a Debian 12 cargo-deb builder image; does not depend on the build job's pre-built artifact. - `dns-resolver` matrix entry runs the full 13-scenario harness (per-distro systemd resolver-backend tests + real-fips end-to-end scenarios) in a single job. Pins the production DNS bind path that ISSUE-2026-0002 lived in. ~7-12 min warm, ~12-15 min cold. Verified locally: full `bash testing/ci-local.sh` sweep passes, including 5/5 deb-install distros and all 13 dns-resolver scenarios. Tor-inclusive sweep (`--with-tor`) verified in a follow-up run. |
||
|
Johnathan Corgan
|
6def31bcf6 |
Admit rekey msg1 from established peers regardless of accept_connections
The accept_connections gate at the top of handle_msg1 was applied unconditionally, so rekey msg1 from a peer with whom an established link already existed was dropped on the same path as fresh handshakes from strangers. Combined with the dual-init tie-breaker, this deadlocked at ~25 minutes when both sides' rekey timers fired near-simultaneously: the smaller-NodeAddr side wins as initiator and expects the larger side to consume its rekey msg1, but if the larger side has accept_connections=false the gate dropped it. Both sides retried at 1 Hz indefinitely; the affected peer fell out of MMP-active rotation. Extract the gate decision into Node::should_admit_msg1, which admits unconditionally when addr_to_link already has an entry for the (transport_id, remote_addr) pair (rekey/restart on an established session) and otherwise consults the transport's accept_connections(). Fresh msg1 from strangers is still rejected before any Noise crypto. Three unit tests pin the truth table: no transport (admit), accept_off no-link (reject, behavior unchanged), accept_off with-link (admit, the carve-out). The fix generalizes for free to BLE, which has the same Node-level gate. TCP and Tor were never subject to this deadlock because their accept condition is runtime state (bind_addr.is_some() / onion_address.is_some()), not a config flag. |
||
|
Johnathan Corgan
|
bf77ececad |
Fix DNS responder silent-drop on systemd-resolved deployments
The previous default configured systemd-resolved with `resolvectl dns fips0 [<fips0_addr>]:5354`, intended to bypass an Ubuntu 22 systemd 249 interface-scoping bug. That target collides with the daemon's mesh-interface filter on Linux: when an IPv6 packet's destination belongs to a non-loopback interface, the kernel attributes the packet to that interface in IPV6_PKTINFO (ipi6_ifindex == fips0) even though loopback delivery is used (tcpdump shows lo). The mesh-interface filter sees arrival_ifindex == mesh_ifindex and silently drops every query at trace level — invisible to operators at the default debug level. Net effect on stock deployments: every .fips query on systemd-resolved hosts was silently dropped. Daemon side ----------- - Default `dns.bind_addr` changes from "::" to "::1" (IPv6 loopback only). The mesh-interface filter is then defanged on the default path because loopback isn't reachable from mesh peers. The filter remains in place defensively for operators who explicitly bind "::" to expose a mesh-reachable responder. fips-dns-setup backend unification ---------------------------------- - New `try_global_drop_in` backend writes /etc/systemd/resolved.conf.d/fips.conf with DNS=[::1]:5354 and Domains=~fips. Inserted ahead of `try_resolvectl` in the dispatch chain. The standard loopback path has no interface scoping, so ipi6_ifindex reports lo and the filter passes. - All other backends now target [::1]:5354 to match the daemon's default IPv6-loopback bind: - try_dns_delegate writes DNS=[::1]:5354 - try_dnsmasq writes server=/fips/::1#5354 - try_nm_dnsmasq writes server=/fips/::1#5354 - Fixed dns-delegate file path: was /etc/systemd/dns-delegate/, must be /etc/systemd/dns-delegate.d/ (with .d suffix). systemd-resolved silently ignored the previous path. - fips-dns-teardown handles the new global-drop-in backend in cleanup. - The legacy resolvectl per-link backend stays as a fallback, documented to require careful daemon bind_addr coordination. fips-gateway upstream pairing ----------------------------- - gateway.dns.upstream default changes from 127.0.0.1:5354 to [::1]:5354 to match the daemon's default bind. Linux IPv6 sockets bound to explicit ::1 do not accept v4-mapped traffic, so the old default would have caused the gateway's startup DNS reachability probe to time out and systemd to restart-loop the service. - Operators who set a non-default daemon `dns.bind_addr` must also set `gateway.dns.upstream` to match — documented inline. Documentation ------------- - packaging/common/fips.yaml and packaging/openwrt-ipk fips.yaml examples updated; rationale for the bind_addr choice and the daemon/gateway pairing recorded inline. Test coverage ------------- - testing/dns-resolver/test.sh: real-fipsd end-to-end scenario added. Builds fipsd in a Debian 12 builder image (cached), runs the daemon with a real TUN in a privileged container, configures DNS via the setup script, and asserts `dig @127.0.0.53 AAAA <npub>.fips` returns AAAA. Refactored as a parameterized helper running across Debian 12/13 and Ubuntu 22/24/26 (5 e2e scenarios). Backend-aware assertions: on systemd >= 258 the expected backend is dns-delegate; on older systemd it's global-drop-in. Strict content checks fail CI on any [::1]:5354 drift. fips-gateway also exercised in the debian12 scenario to lock the gateway-upstream pairing. Renamed all "fipsd" references to "fips" (project convention). - testing/deb-install/ (new harness): builds the actual .deb via cargo-deb in a Debian 12 builder image (cached), installs via apt across each target distro, verifies maintainer scripts, conffile placement, binary placement, and end-to-end .fips resolution after start. Also exercises fips-gateway against the installed daemon to verify the gateway/daemon default pairing on a real .deb path. - This is the test layer that was missing — the previous harness only verified config files were written, never that queries reached the daemon. Verified: dns-resolver 78/78 assertions, deb-install 55/55 assertions across all 5 distros (debian:12, debian:trixie, ubuntu:22.04, ubuntu:24.04, ubuntu:26.04). |
||
 
|
34e00b9f6e |
Add Nostr-mediated overlay discovery and UDP NAT traversal (#53)
Optional peer discovery and NAT hole-punching path gated behind a new
`nostr-discovery` cargo feature. Nodes publish signed overlay endpoint
adverts to public Nostr relays, consume peer adverts to populate
fallback dial addresses, and use STUN-assisted UDP hole punching with
NIP-59 gift-wrap offer/answer signaling to establish direct UDP paths
between NATed peers. Once a punched socket is up, it is handed into
the existing FIPS UDP transport and the standard Noise/FMP session
stack takes over unchanged.
The cargo feature is in the default feature set
(`default = ["nostr-discovery"]`) so stock builds include it; a
build that explicitly disables default features (or selects a
feature set without `nostr-discovery`) does not link the nostr /
nostr-sdk crates and does not emit a no-op poll in the tick loop.
Runtime behavior is independently gated by
`node.discovery.nostr.enabled`, which defaults to false; if the
config enables Nostr on a non-feature build, startup logs a
warning and continues without it.
== Cargo feature and dependencies
- New cargo feature `nostr-discovery = ["dep:nostr", "dep:nostr-sdk"]`.
Not in the default feature set.
- New optional Linux-only dependencies: `nostr 0.44` (features: std,
nip59) and `nostr-sdk 0.44`. Gift-wrap unwrap is hand-rolled in
`src/discovery/nostr/signal.rs` rather than relying on the SDK's
rumor-author check, which FIPS sidesteps by trusting `seal.pubkey`
exclusively.
== Wire format
Overlay advert event: `kind 37195`, parameterized replaceable
(NIP-01 application-defined replaceable range 30000-39999), with
`d = "fips-overlay-v1"`. The digits visually spell FIPS (7=F, 1=I,
9=P, 5=S); a relay survey confirmed the kind is unused.
Advert content carries the version tag, endpoint list
(`udp|tcp|tor` + addr), optional signal-relay and stun-server
metadata, and `issuedAt` / `expiresAt` timestamps. Endpoint
`addr: "nat"` is the sentinel that triggers traversal on the peer
side. NIP-40 `expiration` tag bounds staleness on permanent
shutdown. Lifecycle relies on parameterized-replaceable
supersession; the daemon does not emit NIP-09 kind-5 deletes —
strict relays (Damus, Primal) race delete-against-replace and can
silently drop the replacement.
Gift-wrapped signal event: `kind 21059`. Punch packets carry magic
values `PUNCH_MAGIC` / `PUNCH_ACK_MAGIC`, a sequence number, and a
16-byte session hash.
== Discovery surface
- `src/discovery.rs` (always compiled)
- `EstablishedTraversal`: bound UDP socket + selected remote +
peer npub + optional transport name/config tuning overrides.
- `BootstrapHandoffResult`: returned on successful handoff —
allocated transport id, local/remote addrs, peer NodeAddr,
session id.
- `src/discovery/nostr/` (`#![cfg(feature = "nostr-discovery")]`)
- `types.rs`: wire and control types described above. `ADVERT_KIND`
constant. `BootstrapError` enumerates failure modes (disabled,
missing advert, missing NAT endpoint, no usable relays, invalid
advert, invalid npub, signal timeout, punch timeout, replay,
STUN failure, protocol, nostr, io, serde, event-parse).
- `runtime.rs`: `NostrDiscovery` coordinator. Owns the shared
nostr-sdk `Client`, subscribes to advert + signal event kinds,
maintains a bounded advert cache and a bounded seen-sessions
replay set, drains `BootstrapEvent::{Established, Failed}` for
the node to consume, exposes `update_local_advert`,
`request_connect`, `advert_endpoints_for_peer`,
`cached_open_discovery_candidates`, and `shutdown`.
- `signal.rs`: NIP-59 gift-wrap encode/decode. Outbound wraps are
built against per-attempt ephemeral keys; inbound events are
unwrapped against the node identity.
- `stun.rs`: RFC 5389/8489 Binding Request client with
XOR-MAPPED-ADDRESS parsing for both IPv4 and IPv6; used only to
observe the initiator's own reflexive address against its
locally configured STUN list (peer-advertised STUN is
informational, never an egress target).
- `traversal.rs`: per-attempt candidate-pair punch planner.
Allocates a fresh `0.0.0.0:0` UDP socket per attempt, enumerates
LAN-private and ULA interface addresses alongside the STUN
reflexive address, schedules probe/ack exchanges at the
configured interval for the configured duration, and picks the
first candidate pair that authenticates end-to-end.
Strategy ordering is Reflexive↔Reflexive first, then LAN, then
Mixed. The STUN-observed pair is the only candidate that's reliable
across arbitrary network topologies; trying it first prevents the
planner from latching onto a misleading host-candidate path before
the reflexive path gets a chance. There is no catch-all
Local↔Local strategy: a previous design that paired every local
host candidate from one side with every local host candidate from
the other could declare success on a one-way reachable asymmetric
L3 path (corporate VPN, Tailscale subnet route, overlapping private
address space), only for the FMP handshake to stall because the
return path didn't match. The legitimate `Lan` strategy still pairs
candidates that share a subnet.
== Configuration surface
`node.discovery.nostr.*` (`NostrDiscoveryConfig`), all `serde(default)`
with `deny_unknown_fields`:
- `enabled` (default false), `advertise` (default true)
- `advert_relays`, `dm_relays`, `stun_servers`: defaults are
`wss://relay.damus.io`, `wss://nos.lol`, `wss://offchain.pub`
for both relay lists, and Google / Cloudflare / Twilio for STUN.
Operators are expected to override for production. Other
verified-working public relays for reference:
`nostr.bitcoiner.social`, `nostr-pub.wellorder.net`,
`nostr.oxtr.dev`, `nostr.mom`.
- `app` (default `"fips-overlay-v1"`), `signal_ttl_secs` (120)
- `policy`: `NostrDiscoveryPolicy::{Disabled, ConfiguredOnly (default),
Open}` — controls whether advert-derived endpoints are consumed
only for peers carrying `via_nostr = true`, or also for
non-configured peers within a budget cap.
- `share_local_candidates` (default false) — when false, the offer's
`local_addresses` list is empty and peers see only the reflexive
address. Enable per-node only for genuinely same-LAN deployments;
off-by-default eliminates the misleading-path failure mode for
the common case where peers are not on the same broadcast domain.
- `open_discovery_max_pending` (64) — caps queued open-discovery
retries; bounded by available outbound slots.
- `max_concurrent_incoming_offers` (16) — semaphore against offer
spam; excess offers are debug-logged and dropped.
- `advert_cache_max_entries` (2048) and `seen_sessions_max_entries`
(2048) — bound memory under ambient relay volume; overflow
evictions are debug-logged.
- `attempt_timeout_secs` (10), `replay_window_secs` (300)
- `punch_start_delay_ms` (2000), `punch_interval_ms` (200),
`punch_duration_ms` (10000)
- `advert_ttl_secs` (3600), `advert_refresh_secs` (1800)
Per-peer and per-transport flags:
- `PeerConfig.via_nostr: bool` — when true (and Nostr is enabled),
advert-derived addresses are appended as fallback dial candidates
after static addresses for that peer.
- `PeerConfig.addresses` is now `serde(default)` and may be empty
when `via_nostr: true`; validation requires at least one of the
two to be present per peer, and the error message names the
peer's npub.
- `UdpConfig.advertise_on_nostr: Option<bool>` and
`UdpConfig.public: Option<bool>` — UDP transports can be
advertised either as direct `host:port` (public = true) or as the
`addr: "nat"` sentinel that triggers rendezvous on the peer side.
- `TcpConfig.advertise_on_nostr` and `TorConfig.advertise_on_nostr`
— TCP and Tor onion endpoints can be advertised as directly
reachable.
- A reserved peer address `transport: udp, addr: "nat"` parses without
special-casing in YAML and routes through the bootstrap runtime.
Cross-field validation (`Config::validate`, called from `Node::new`
and `Node::with_identity`):
- Any transport with `advertise_on_nostr = true` requires
`node.discovery.nostr.enabled = true`.
- Any peer with `via_nostr = true` requires
`node.discovery.nostr.enabled = true`.
- A non-public UDP advert (`advertise_on_nostr = true`,
`public = false` — i.e. `udp:nat`) additionally requires at least
one `dm_relay` and at least one `stun_server`.
Surfaced as `ConfigError::Validation`.
== Node integration
`src/node/lifecycle.rs` is the main integration point.
- At node start (after transports are up, before TUN), if Nostr is
enabled and the feature is compiled in, `NostrDiscovery::start` is
invoked, the initial local overlay advert is built from the live
transport set and published, and the runtime handle is stored.
- The rx tick loop calls `poll_nostr_discovery` (feature-gated both
at method definition and call site), which refreshes the local
advert, drains bootstrap events, adopts established traversals,
schedules retries for failed traversals, and — under `policy:
open` — enqueues outbound retries for non-configured peers
visible in the advert cache, bounded by
`open_discovery_max_pending` and the remaining outbound slots.
- Outbound peer dialing is refactored to `try_peer_addresses`, which
first exhausts the static address list in priority order and only
then appends advert-derived fallback addresses; both lists run
through the same `attempt_peer_address_list` code path. The
`udp:nat` sentinel address triggers `NostrDiscovery::request_connect`
for the peer instead of a direct dial and returns `Ok(())`.
- `build_overlay_advert` walks operational transports, consults
per-instance `UdpConfig` / `TcpConfig` / `TorConfig` (matching by
optional transport instance name), and emits an `OverlayAdvert`
including `signalRelays` and `stunServers` when any UDP endpoint
is advertised as NAT.
- `adopt_established_traversal` is the bootstrap handoff API:
allocates a new `TransportId`, constructs a `UdpTransport` with
the user-supplied (or default) `UdpConfig`, calls the new
`adopt_socket_async` to reuse the punched socket verbatim,
registers the transport in the normal transport map, records it
in `bootstrap_transports`, and calls `initiate_connection` so the
normal handshake path runs. On failure, the transport is stopped
and removed cleanly and the set membership is rolled back.
- On clean shutdown, `NostrDiscovery::shutdown` is awaited so
background tasks stop before transports are torn down. (The
advert is not explicitly retracted; NIP-40 expiration plus the
next refresh from any live publisher supersedes it.)
New `Node` fields:
- `nostr_discovery: Option<Arc<NostrDiscovery>>` (feature-gated).
- `bootstrap_transports: HashSet<TransportId>` — per-peer UDP
transports adopted from NAT traversal, cleaned up via
`cleanup_bootstrap_transport_if_unused` whenever the link,
connection, peer, or pending-connect referencing them is removed.
Retry and error surface:
- `RetryState.expires_at_ms: Option<u64>` — optional absolute expiry
for a retry entry. `pump_retries` drops expired entries with an
info log. Used for open-discovery retries, which expire at two
times the advert TTL.
- New `NodeError::BootstrapHandoff(String)` returned from
`adopt_established_traversal` when the underlying transport
adoption fails or local address discovery fails.
- New `ConfigError::Validation(String)`.
- A small refactor extracts `Node::now_ms()` and reuses it across
lifecycle, rx-loop tick, and timeout bookkeeping.
== UDP transport
`src/transport/udp/`:
- `UdpRawSocket::adopt(std::net::UdpSocket, recv_buf, send_buf)`:
adopts an externally bound socket, makes it non-blocking, applies
the configured buffer sizes (warning if the kernel clamps), and
reports the resulting local address. Preserves the NAT mapping —
no rebind.
- `UdpTransport::adopt_socket_async(std::net::UdpSocket)`: the
`start_async` analogue for an already-bound socket, wiring the
async socket and recv task exactly as the fresh-bind path would.
- `Drop` impl for `UdpTransport`: if a transport is dropped while
still holding a recv task or socket (for example on error
teardown), aborts the task, clears the socket, and emits a debug
log so the cleanup is visible in tracing rather than silent.
== Logging and observability
Default `EnvFilter` demotes third-party relay-pool DEBUG output to
TRACE-only: `nostr_relay_pool`, `nostr_sdk`, and `nostr` are pinned
at INFO when our level is anything below TRACE, and at TRACE when
our level is TRACE — so the raw frames are still reachable when
explicitly asked for. RUST_LOG continues to override completely.
Concise one-line DEBUG events are emitted at the meaningful points
in the discovery / hole-punch sequence:
- `advert: published` (event id, relay count, endpoints, ttl)
- `advert: peer cached` (notify-loop ingress for non-self)
- `advert: resolved` (cache hit / relay fetch outcome)
- `traversal: initiator starting`
- `traversal: initiator STUN observed` (reflexive, local count)
- `traversal: offer sent` (session id, relay count, event id)
- `traversal: answer received` (accepted, reflexive, local)
- `traversal: initiator punch succeeded` (remote addr)
- `traversal: offer received` (responder side)
- `traversal: responder STUN observed`
- `traversal: answer sent`
- `traversal: responder punch succeeded`
Npubs are shortened to `npub1<4>..<4>` and event/session ids to
their first 8 hex characters.
Other operator-facing logs:
- `UdpTransport` adoption and drop paths log at info / debug.
- `adopt_established_traversal` logs at debug on entry and info on
successful return, tagged with peer npub, session id, transport
id, and both socket endpoints, so the bootstrap handoff is
traceable end-to-end alongside the `UdpTransport::drop` log.
- `cleanup_bootstrap_transport_if_unused` logs at debug when the
reference-count check drops an adopted transport.
- `connect_peer` tags its entry `debug!` with `peer_npub` so
downstream STUN, punch, and handshake logs for the same peer
correlate for operators.
- Advert-cache and seen-sessions overflow evictions log at debug so
mis-sized caps are visible under ambient relay volume.
- Gift-wrap unwrap failures on `SIGNAL_KIND` events log at trace
(hot path: fires for every unrelated signal event on the same
relay).
- Traversal-offer handler failures log at debug. Expected conditions
such as punch timeout on symmetric NAT are covered there; real
problems are reported upstream via `BootstrapEvent::Failed`.
- Inbound-offer rate-limit messages name the governing config field
(`max_concurrent_incoming_offers`) and state that the offer was
rate-limited rather than failing.
== Tests
- 18 new unit tests in `src/discovery/nostr/tests.rs` covering advert
encoding, signal envelope round-trip, STUN parsing, punch-packet
codec, and replay-window enforcement. Run under the
`nostr-discovery` feature.
- Config-validation tests in `src/config/mod.rs` covering the three
cross-field invariants and YAML parsing of the full
`node.discovery.nostr` block plus `peers[].via_nostr`, empty
`addresses` with `via_nostr: true`, and a `udp: nat` address.
- `src/node/tests/bootstrap.rs` integration tests that drive a
synthetic traversal (bound UDP socket pair + synthetic peer
identity) through `adopt_established_traversal` and assert the
Noise handshake completes over the adopted socket.
- Punch-planner tests assert reflexive-before-LAN ordering and that
same-LAN scenarios still include the LAN target in the plan.
- `testing/nat/` Docker NAT lab harness:
- Local `strfry` relay, local STUN responder, and one or two
router containers performing `iptables` NAT.
- Node LAN interfaces are provisioned with explicit `veth` pairs
injected into the node and router namespaces so every packet
traverses the router namespace (plain Docker bridges are not
used for the LAN).
- `cone` scenario: both peers behind full-cone-emulation NAT
(SNAT with source-port preservation, inbound DNAT back to the
single LAN host regardless of remote source); asserts UDP
traversal succeeds and link remote addresses are on the router
WAN subnet.
- `symmetric` scenario: `MASQUERADE --random-fully`; asserts UDP
traversal fails and TCP fallback converges over router-
published WAN addresses.
- `lan` scenario: both peers share a LAN subnet; asserts LAN
addresses are preferred over reflexive ones.
- Cleanup tears down all profile-gated services
(`--profile cone --profile symmetric --profile lan`) so no
orphan containers survive a run.
- `testing/scripts/build.sh` builds the Docker test image with
`--features "tui nostr-discovery"` by default so NAT-harness
binaries include bootstrap support.
== CI
- Linux release build and nextest unit-test job both use
`--features "gateway nostr-discovery"` so the feature-gated code
and its unit tests compile and run in CI.
- Three new integration matrix entries (`nat-cone`, `nat-symmetric`,
`nat-lan`) invoke `testing/nat/scripts/nat-test.sh`, collect
`docker compose logs` on failure, and always stop containers.
== Packaging and operations
- `packaging/common/fips.yaml` ships a fully commented
`node.discovery.nostr.*` block, plus documented
`advertise_on_nostr` / `public` examples under the UDP transport,
an `advertise_on_nostr` example under TCP, and a `via_nostr: true`
example under the static peer section with both a direct
`host:port` UDP address and a `udp: nat` fallback.
- `.github/workflows/package-openwrt.yml`: NIP-94 release event
publishes target the new default relay set.
== Documentation
- `README.md`: overlay discovery + NAT traversal moved from
"Near-term priorities" into "What works today".
- `docs/design/fips-intro.md`: rewrites the paragraphs that
previously described Nostr discovery and NAT traversal as future
work; describes the shipped mechanism and the feature gate.
- `docs/design/fips-transport-layer.md`: drops the "(future
direction)" qualifier from the Nostr Relay Discovery section,
expands with the `udp:nat` advertisement and bootstrap handoff
description, and updates the Current State callout.
- `docs/design/fips-mesh-layer.md`: notes that mid-session NAT
rebinding (roaming) and initial NAT traversal (Nostr path) are
distinct mechanisms.
- `docs/design/fips-configuration.md`: documents the full
`node.discovery.nostr.*` surface, including the three resource
caps and `share_local_candidates`.
- `docs/design/fips-nostr-discovery.md`: design and configuration
reference for the shipped mechanism, including the empty-
`addresses`-with-`via_nostr` shorthand.
- `docs/proposals/nostr-udp-hole-punch-protocol.md`: adds an
Implemented status callout, clarifies that the punch socket is
per-peer and per-attempt rather than shared with the application
listener, aligns field names with the shipped JSON
(`sessionId`, `issuedAt` / `expiresAt`, `reflexiveAddress`,
`localAddresses`, `stunServer`), sets the `d`-tag to
`fips-overlay-v1`, names the kind as 37195, and notes that
advertised STUN entries are informational.
- `docs/proposals/README.md`: adds a Status column and marks the
hole-punching proposal Implemented.
- `CHANGELOG.md`: Unreleased > Added entry covering the discovery
path, STUN/punch path, configuration surface, and Docker NAT lab.
Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
|
||
|
Johnathan Corgan
|
f16b837a12 |
Tune overly aggressive discovery rate limiting
The default 30s post-failure backoff (300s cap, doubling per consecutive failure) was set to bound traffic from chatty apps looking up unreachable targets, but in practice it dominates cold-start mesh convergence: a single timed-out lookup during initial bloom-filter propagation suppresses any retry for 30s, and the existing reset triggers (parent change, new peer, first RTT, reconnection) don't fire on a stable post-handshake topology. The suppression window winds up dictating the protocol's effective time-to-converge instead of bounding repeat traffic. Replaces the single-lookup-with-internal-retry model (`timeout_secs`/`retry_interval_secs`/`max_attempts`) with a per-attempt timeout sequence in `node.discovery.attempt_timeouts_secs`, defaulting to `[1, 2, 4, 8]`. Each attempt sends a fresh LookupRequest with a new random request_id so successive attempts can take different forwarding paths as the bloom and tree state evolve. The destination is declared unreachable only after the sequence is exhausted (15s total at the default). Disables post-failure suppression by default (`backoff_base_secs`/ `backoff_max_secs` now `0`/`0`). The `DiscoveryBackoff` machinery stays in tree (inert at zero base/cap); operators with chatty apps generating repeat lookups against unreachable destinations can opt back in. `PendingLookup` field shape unchanged so the control-socket `show_routing` JSON (`pending_lookups[].attempt`/`initiated_ms`/ `last_sent_ms`) keeps the same schema for fipstop and external consumers; `last_sent_ms` now means "current-attempt start" under the new state machine. |
||
|
Johnathan CorganandGitHub
|
cbc78091ab |
Rationalize cargo feature and platform-gate surface (#79)
Drop the `tui`, `ble`, and `gateway` cargo features and replace them with platform cfg gates. Plain `cargo build` now produces every subsystem appropriate for the target platform with no feature flags required. Motivation: - `default = ["tui", "ble"]` broke `cargo build` on macOS and Windows because `ble` pulled in `bluer` (BlueZ, Linux-only). Every non-Linux packager needed `--no-default-features`. - The feature flags on `ble` and `gateway` were redundant with their platform-gated deps (`bluer`, `rustables`). The parallel gating was inconsistent and error-prone. - `tui` feature protected against a ratatui binary-size concern that no longer applies in 2026. Cargo.toml: - Remove `tui`, `ble`, `gateway` features; `default = []`. - Promote `ratatui` to a non-optional top-level dependency. - Move `rustables` from top-level optional into the Linux target block, non-optional. - Split `bluer` into its own target block with `cfg(all(target_os = "linux", not(target_env = "musl")))` — BlueZ isn't available on musl router targets and `libdbus-sys` doesn't cross-compile to musl without pkg-config sysroot setup. - Drop `required-features` from the `fipstop` and `fips-gateway` `[[bin]]` entries. build.rs: - Emit a `bluer_available` custom cfg when `target_os == "linux"` and `target_env != "musl"`, for use in place of the verbose full predicate in source cfg gates. Source: - Replace every `#[cfg(feature = "gateway")]` with `#[cfg(target_os = "linux")]`. Gateway code works on both glibc and musl Linux (rustables is fine on musl). - Replace every `#[cfg(feature = "ble")]` with `#[cfg(bluer_available)]`. BLE-specific code (BluerIo module, bluer type conversions, BLE transport instance creation, resolve_ble_addr) is excluded on musl and non-Linux. Generic `BleAddr`, `BleIo` trait, `MockBleIo`, and `BleTransport<I>` still compile on all targets. - `src/bin/fips-gateway.rs`: always compiled, but `main()` is gated to Linux. Non-Linux stub exits 1 with a diagnostic. Existing non-Linux packaging scripts don't ship it, so the stub binary sits unused. Packaging and CI: - Drop `--features` and `--no-default-features` flags from every packaging script and workflow. Defaults now match each platform's capabilities. - AUR `fips-git` automatically aligns with stable `PKGBUILD` (both build with defaults). Verified: `cargo build --release` with no flags produces all four binaries on glibc Linux; all unit and integration tests pass across Linux/macOS/Windows/OpenWrt (musl) in CI. |
||
|
Johnathan Corgan
|
be0708ac9b |
Bring acl test module into the test tree
src/node/tests/acl.rs was added by PR #50 but never declared in src/node/tests/mod.rs, so none of its 4 unit tests ran. The tests themselves still compile and pass against current master code — the fix is a one-line mod declaration. Test count goes from 1031 to 1035. No code under test changes. This only adds previously-dormant coverage of the ACL enforcement call sites (outbound connect, inbound msg1, outbound msg2). |
||
|
Johnathan CorganandGitHub
|
ed312ac6f2 |
Fix DNS resolution on Ubuntu 22 with systemd-resolved (#77)
On Ubuntu 22 (systemd 249), systemd-resolved applies interface-scoped routing to per-link DNS servers. Configuring `resolvectl dns fips0 127.0.0.1:5354` caused resolved to attempt reaching 127.0.0.1 through fips0 (a TUN with only fd00::/8 routes), silently failing. The DNS responder never received queries. Newer systemd versions (250+) have explicit handling for loopback servers on non-loopback interfaces. Changes: - DNS responder default bind_addr changed from "127.0.0.1" to "::" so it listens on all interfaces, including fips0. Bind logic in lifecycle.rs now parses bind_addr as IpAddr and constructs a SocketAddr, handling IPv6 literal formatting. Factored into Node::bind_dns_socket with explicit IPV6_V6ONLY=0 via socket2, so IPv4 clients on 127.0.0.1:5354 still reach the responder regardless of the kernel's net.ipv6.bindv6only sysctl. - fips-dns-setup resolvectl backend now waits for fips0 to have a global IPv6 address, then configures resolved with [<fips0-addr>]:5354. That address is locally delivered by the kernel regardless of which interface resolved tries to route through. The dnsmasq and NetworkManager backends still use 127.0.0.1 (they don't have the interface-scoping issue). - Dropped hardcoded `bind_addr: "127.0.0.1"` from the packaged fips.yaml (Debian + OpenWrt). The shipped config was overriding the new default. - DNS queries are only accepted from the localhost. Verified end-to-end in a privileged Ubuntu 22.04 systemd container: dig @127.0.0.53 AAAA <npub>.fips resolves cleanly through systemd-resolved. The dns-delegate backend (systemd 258+) still uses 127.0.0.1; it has not been verified whether that backend has the same routing issue. |