Commit Graph
111 Commits
Author SHA1 Message Date
Johnathan Corgan bbd4de9e76 Merge master into refactor-hotpath-next
Bring master's runtime peer-list refresh, opt-in mDNS LAN discovery, and
the receive-path reject-reason / reloadable-config refactor into the XX
handshake integration branch. The peer-restart-epoch detection and
stale-FSP-session teardown authored against Noise IK are re-authored onto
the Noise XX rekey path: complete_rekey_msg2 now also surfaces the remote
startup epoch, and the stale session is cleared once the XX rekey
completes (after msg3 is sent). Connect-budget and path-refresh work is
threaded through the XX anonymous-discovery branch.
2026-05-30 02:42:01 +00:00
Johnathan Corgan 3c5d9fd4f2 Merge master into refactor-hotpath
Bring the runtime peer-list refresh and opt-in mDNS LAN discovery work
on master into the receive-path RejectReason / reloadable-config
integration branch. Code files auto-merge clean; the only conflict is
the CHANGELOG Unreleased section, resolved as the union of both sets of
entries.
2026-05-30 01:43:31 +00:00
Martti MalmiandJohnathan Corgan 7d7b551ca1 discovery: add opt-in mDNS LAN discovery
Add scoped mDNS / DNS-SD discovery for peers on the same local link,
giving sub-second pairing without a relay or NAT-traversal roundtrip.
A node advertises its npub, protocol version, and an optional network
scope over link-local multicast, and browses for matching adverts to
initiate Noise handshakes against same-LAN peers.

LAN discovery is disabled by default; operators enable it with
node.discovery.lan.enabled: true. Default-off avoids reintroducing a
per-LAN identity broadcast on nodes that have deliberately disabled
other discovery channels, and avoids any multicast surprise on upgrade.

The startup advertised-port picker now excludes bootstrap transports
and selects a non-bootstrap operational UDP transport with a stable
lowest-id selector, so the advertised port is deterministic across
restarts rather than dependent on HashMap iteration order. This
matches the per-dial transport selection used for discovered peers.

Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
2026-05-30 01:32:06 +00:00
Martti MalmiandJohnathan Corgan da0d9d39a0 node: refresh active peer paths without dropping links
Add Node::update_peers for runtime peer-list refresh. It re-derives the
active peer connections from a new peer configuration, adding newly
configured peers and removing those no longer present, while keeping
links to peers that remain in the set rather than tearing every
connection down. The call returns an UpdatePeersOutcome summarizing the
added, removed, and retained peers.

PeerAddress gains a seen_at_ms recency field (with_seen_at_ms). Active
path selection now sorts address candidates by recency so the most
recently observed address wins when concurrent path probes race.

complete_rekey_msg2 now returns the remote peer's startup epoch
alongside the new Noise session, letting the rekey path detect a peer
restart and clear stale session state. A stale FSP session is cleared
when a peer restart is detected during FMP rekey or cross-connection
promotion, so the session-layer map no longer lingers out of sync with
the freshly promoted peer.

Per-tick work budgets bound the connection churn in a single node tick
(MAX_DISCOVERY_CONNECTS_PER_TICK, MAX_RETRY_CONNECTIONS_PER_TICK,
MAX_PARALLEL_PATH_CANDIDATES_PER_PEER); work beyond a tick's budget is
deferred to the next tick rather than discarded.

Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
2026-05-30 00:46:38 +00:00
Johnathan Corgan fccd760ad7 Merge branch 'refactor-hotpath' into refactor-hotpath-next
# Conflicts:
#	src/node/tests/acl.rs
2026-05-29 02:38:26 +00:00
Johnathan Corgan d672ed865f node: migrate peer ACL to the Reloadable trait and hot-reload the host map
Move PeerAclReloader onto the Reloadable trait: its ACL snapshot is now
published through an arc_swap::ArcSwap so the authorization hot path reads
it without locking, and the former check_reload becomes the trait's
reload(). The node tick calls self.peer_acl.reload().await.

Wire the host map into the tick as well. The host map snapshot was
previously taken once at construction and never polled; it now hot-reloads
on /etc/fips/hosts mtime changes once per tick, alongside the ACL, so
hostname display reflects edits without a restart.

The path_mtu_lookup cache (event-driven, populated from observed traffic)
and the nostr_discovery subsystem (an async spawned task) are deliberately
left off the trait: neither reloads from a backing file, so a no-op reload()
would be misleading. The rationale is documented on the trait module.

The host map and the ACL's embedded alias reloader still stat /etc/fips/hosts
independently each tick. A single small-file stat per tick is cheap, so the
duplicate is left in place; sharing one mtime observation between the two is
a possible future cleanup.

Tests: a node-level test exercises the host-map tick reload end to end
through peer_display_name; the ACL reloader tests are updated to drive the
async reload().
2026-05-29 02:36:00 +00:00
Johnathan Corgan a965b1ec67 Merge branch 'refactor-hotpath' into refactor-hotpath-next 2026-05-29 01:23:41 +00:00
Johnathan Corgan 0bb9ce09c6 node: introduce Reloadable trait and migrate host map to a lock-free snapshot
Add a `Reloadable` trait that normalizes the node's reloadable
configuration/resource pattern onto a single contract built around an
`arc_swap::ArcSwap` snapshot: a lock-free `load()` for the hot read path
and an async `reload()` that re-reads the backing source and atomically
swaps in a fresh snapshot. The trait carries the canonical Arc-wrapper
template documentation (single-writer node tick, many-reader hot path,
whole-snapshot swap so readers never observe a partial update).

Migrate the host map to this trait via a new `HostMapReloadable` that
reuses the existing load/merge/mtime helpers in upper::hosts. The Node
`host_map` field changes from `Arc<HostMap>` to `HostMapReloadable`, and
`peer_display_name` reads through a lock-free guard. The initial snapshot
is byte-identical to the previous construction, so behavior is unchanged.

The host map is still snapshotted once at construction and not polled;
`reload()` is exercised only by unit tests for now. Wiring the periodic
poll into the node tick, and deduplicating the hosts-file stat against
the ACL reloader's embedded copy, is left as a follow-up.

Add `arc-swap` as a dependency. Unit tests cover initial load (base +
file, base only), change/no-change/deletion/creation detection,
base-preserved-on-reload, and equivalence of the initial snapshot to the
pre-migration construction.
2026-05-29 01:23:29 +00:00
Johnathan Corgan 939e2cbd4f Merge branch 'refactor-hotpath' into refactor-hotpath-next
# Conflicts:
#	src/node/bloom.rs
#	src/node/handlers/handshake.rs
2026-05-28 21:21:12 +00:00
Johnathan Corgan 66732e89c1 node: route receive-path silent-rejection sites through typed RejectReason counters
Introduce a typed RejectReason enum and a NodeStats::record_reject
dispatch so every receive-path rejection-and-return site bumps a
machine-readable per-subsystem counter while keeping its operator-facing
log line. The top-level variants mirror the existing NodeStats subsystem
split (Tree, Bloom, Discovery, Forwarding) and add Handshake, Session,
Mmp, and Transport categories; HandshakeStats, SessionStats, and MmpStats
are new sub-stats.

Wired clusters: tree and MMP outbound sign-failure; the FSP session
unknown-session and state-machine cluster; the Noise IK handshake
state-machine cluster (msg1/msg2); and the decode / crypto / cap /
semantic tail across bloom, discovery, forwarding, mmp, and tree. The
TreeStats::ancestry_invalid counter, present since the scaffold but never
incremented, is now bumped from the validate_semantics ancestry rejection.
Several handshake, MMP, tree, and discovery paths that previously had no
counter at all are now counted, including the send_lookup_response
no-route drop (DiscoveryStats::resp_no_route).

Existing direct counters at the bloom / discovery / forwarding sites are
retained alongside the new dispatch while the rollout is in progress (the
bloom_poison tests expect the transitional +2 delta); a later change
collapses the duplicate increment.
2026-05-28 21:03:56 +00:00
Johnathan Corgan 05124367db Merge master into next (outbound admission gate + mesh-size parent skip) 2026-05-26 17:31:47 +00:00
Johnathan Corgan 6991a152e6 Merge maint into master (outbound admission gate + mesh-size parent skip)
Brings two structural fixes landed on maint:
- compute_mesh_size: explicit parent skip in the children loop, so the
  disjoint-subtree invariant no longer depends on peer_declaration cache
  freshness.
- max_peers: outbound connection-initiation gated on the cap (auto-reconnect
  retries, Nostr-mediated discovery established adoption, and both sides
  of the NAT-traversal punch sequence). Inbound msg1 admission gate
  unchanged.
2026-05-26 17:31:47 +00:00
Johnathan Corgan d4687e5d30 node: gate outbound connection initiation on max_peers
node.limits.max_peers was honored only on inbound msg1 admission
(handshake.rs handle_msg1 returns PeerLimitExceeded when peers.len
is at the cap). Four outbound initiation paths proceeded unconditionally
at capacity: auto-reconnect retries (process_pending_retries),
Nostr-mediated discovery's BootstrapEvent::Established adoption
(poll_nostr_discovery), NAT-traversal punch initiation (the outgoing
side of the offer/answer/punch sequence in the Nostr discovery
runtime), and NAT-traversal punch response (the incoming side of the
same sequence). A saturated node burned CPU, UDP probes, STUN
observations, and Nostr relay traffic on connections that the inbound
gate would reject the moment they reached msg1.

Introduce Node::outbound_admission_check (peers.len < max_peers, or
true when max_peers == 0 as the no-cap sentinel) and gate the four
paths. The discovery runtime lives in a separate task and does not
hold a Node reference; bridge via an Arc<AtomicBool> the runtime
reads and Node refreshes once per tick from outbound_admission_check.
The atomic granularity is intentionally loose: one-tick lag is
acceptable because the inbound msg1 gate continues to be the
authoritative cap, and in-flight handshakes started below the cap
are allowed to complete.

Inbound gate at handshake.rs is unchanged.
2026-05-26 17:08:56 +00:00
Johnathan Corgan df43ac79b9 node: skip parent explicitly in compute_mesh_size children loop
The mesh-size estimator's children loop relied on the cached
peer_declaration(parent_id).parent_id() != my_addr check to exclude
the parent. That cached view briefly disagrees with our own latest
my_declaration().parent_id() during the window between a local
parent-switch and the new parent's next inbound TreeAnnounce: the
peer-declaration cache still names us as the parent's parent, so the
parent is iterated as if it were a child and its (typically dominant)
bloom cardinality is added a second time. Symptom: estimated mesh size
displayed in fipsctl show status and fipstop nearly-but-not-exactly
doubles during tree rebalancing.

Make the invariant structural with an explicit peer_addr == parent_id
skip at the head of the children loop. Per-peer 500 ms rate-limiter
and overall recompute cadence are unchanged.

Adds a regression test that constructs the stale-peer-declaration
scenario directly and asserts the parent is not double-counted.
2026-05-26 16:55:11 +00:00
Johnathan Corgan 49735980b8 Merge master into next (cross-init tiebreaker + tick-arm connect-on-send fix) 2026-05-25 05:13:00 +00:00
Johnathan Corgan 18297283ad Merge maint into master (rx_loop tick-arm connect-on-send fix) 2026-05-25 05:02:01 +00:00
Johnathan Corgan 4d5380604a node: don't drive connect-on-send from the rx_loop tick path
The tick body's per-peer check_* loops (heartbeats, bloom
announces, MMP reports, tree announces) called transport.send
for every active peer, which on TCP/Tor fell through to a 5 s
connect-on-send wait for any peer whose pool entry was not yet
established. That wedged the entire tick body for the full
connect_timeout_ms per unreachable peer; under post-restart
convergence on a high-peer mesh, this cascaded into multi-
second tick stalls. On master, the same mechanism also starved
the per-tick control-snapshot republish and pushed fipsctl
queries onto an mpsc fallback that was itself queued behind
the wedged rx_loop, producing the 5-second fipsctl head-of-line
pattern operators observed on loaded nodes.

Gate send_encrypted_link_message_with_ce on
transport.connection_state before the send: proceed only when
Connected; on None, kick off a non-blocking background connect
(idempotent — TransportHandle::connect dedupes against the
connecting pool and spawns the timeout-bounded TcpStream::connect
inside its own tokio task) and fail this send fast with a
clear "transport connection not ready" error. A subsequent
tick retries once the pool has an entry. The reconnect
lifecycle (check_link_heartbeats, process_pending_retries,
poll_pending_connects) is unchanged. The connect-on-send
branch in transport.send_async itself remains in place for
code paths that legitimately need synchronous connect (e.g.,
explicit operator-driven fipsctl connect).
2026-05-25 04:50:30 +00:00
Johnathan Corgan d6729086de Merge master into next (data-plane perf overhaul, packaging fixes)
Forward-merges PR #91 (off-task encrypt/decrypt worker pools, GSO,
connected UDP), the macOS #102 package-integrity fix, and the AUR
#98 fips-dns fix from master.

Six textual conflicts resolved: wire/noise imports, the K-bit-flip
and rekey-cutover handlers, ActivePeer fields, and an advert test
fixture. One semantic conflict: PR #91's worker-pool code on master
referenced FLAG_SP and MmpPeerState.spin_bit, which next removed in
4aded9a2 (spin bit superseded by the MMP receiver-report timestamp
echo; FMP flags bit 2 reclaimed). The worker acceleration and its
fmp_flags round-trip test are kept; the spin-bit references are
dropped to honor next's wire-format change.
2026-05-21 01:54:47 +00:00
Martti MalmiandJohnathan Corgan 0a5c367edc data-plane perf overhaul: off-task encrypt + decrypt, GSO, connected UDP
Moves both AEAD layers (ChaCha20-Poly1305, one round per layer per
packet) plus the sendmsg syscall off the rx_loop task onto a per-shard
worker pool, adds per-peer connect(2)-ed UDP with SO_REUSEPORT, and
uses Linux UDP GSO (sendmsg+UDP_SEGMENT — kernel splits one super-skb
into N on-the-wire datagrams in a single TX-stack walk) when packets
in a batch are uniform-size. Same kernel primitive WireGuard's
in-kernel module and BoringTun use to hit 2.5–3.2 Gbps single-stream.

Single TCP stream on a 5-node docker-bridge mesh, 5 x 15 s x P=1:

  A→D:  1379 → 2708 Mbps  (1.96x, RTT +0.12 ms)
  A→E:  1394 → 2663 Mbps  (1.91x, RTT +0.11 ms)
  E→A:  1406 → 2624 Mbps  (1.87x, RTT +0.19 ms)

Static-peer pairs only — every CoV under 3%, 0 outliers, 0% ICMP
loss. The ~+100 µs RTT is the worker queue handoff cost; AEAD +
sendmmsg now run on a separate core in exchange.

What lands:

- src/node/encrypt_worker.rs: std::thread + crossbeam_channel
  workers; hash-by-destination dispatch pins a TCP flow to one
  worker so wire ordering is preserved; per-worker sendmmsg(2)
  batching up to 32; Linux uses sendmsg(2)+UDP_SEGMENT when
  packets in a group are uniform-size.

- src/node/decrypt_worker.rs: receive-side mirror. Each shard owns
  its session's recv cipher + replay window in a thread-local
  HashMap (no shared RwLock/Mutex). Sessions are handed off at
  promote_connection and re-registered on K-bit flip / rekey
  cutover.

- src/node/handlers/session.rs try_send_session_data_pipelined:
  FSP+FMP both seal in-place in the worker on one wire-buffer
  alloc; no intermediate inner_plaintext / fsp_payload Vecs.

- src/transport/udp/connected_peer.rs + peer_drain.rs: per-peer
  connect(2)-ed UDP socket with SO_REUSEPORT (set on the listen
  socket too — without that, EADDRINUSE on activation and every
  packet falls back to the wildcard path); the worker sends with
  msg_name=NULL and the kernel uses its cached 5-tuple. Tick-
  driven activation in handlers/connected_udp.rs, idempotent.

- src/transport/udp/mod.rs: mem::replace the recvmmsg backing buffer
  instead of buf.to_vec() per packet — single pointer swap, no
  MTU-sized memcpy.

- src/protocol/link.rs SessionDatagramRef: zero-copy borrowed view
  used by handle_session_datagram for the bulk local-delivery
  path; handle_session_payload takes the borrowed payload
  directly (no payload[35..].to_vec()).

- src/transport/mod.rs TransportAddr::from_socket_addr: collapses
  the two-alloc from_string(addr.to_string()) pattern to one.

- src/node/handlers/rx_loop.rs: decrypt-fallback drain promoted
  ahead of packet_rx in the select! (TCP ACK starvation fix);
  interleaved fallback drain every 32 packets inside the rx burst
  loop.

- noise::Session: send_cipher_clone / recv_cipher_clone /
  recv_replay_snapshot_owned / take_send_counter / accept_replay
  so off-task workers can hold a cloned cipher + reserved counter
  while the dispatcher keeps replay/counter sequencing serial.
  CipherState::cipher_clone returns a refcount-bumped LessSafeKey.
  AsyncUdpSocket: AsRawFd so workers issue raw sendmmsg / sendmsg
  without going through the tokio reactor.

- Worker pool sizing: both default to num_cpus, overridable via
  FIPS_ENCRYPT_WORKERS=N / FIPS_DECRYPT_WORKERS=N. Per-peer
  connected UDP can be disabled via FIPS_CONNECTED_UDP=0.

- src/perf_profile.rs: optional per-stage timing reporter under
  FIPS_PERF=1 (or FIPS_PIPELINE_TRACE=1). Off by default; zero
  overhead when disabled.

- All cfg(unix)-gated. Windows continues on the existing tokio-
  based send/recv.

Decrypt worker session lifecycle:

- Node::unregister_decrypt_worker_session mirrors the existing
  register helper. Wired at the two natural sites that already
  iterate peers_by_index: the rekey drain-completion block in
  handlers/rekey.rs (drops the worker entry for the old our_index
  once the drain window has expired and the cache_key is
  unreachable to any in-flight OLD-K packet), and remove_active_peer
  in handlers/dispatch.rs (drops the worker entry for each of the
  four index slots: current, rekey, pending, previous). Only
  our_index is normally registered; unregister_session is fire-
  and-forget for missing entries, so calling unconditionally on
  all four slots is correct and bounds the cleanup without per-
  slot accounting. Without these callers the per-worker sessions
  HashMap and the Node's decrypt_registered_sessions set would
  grow monotonically per rekey on long-lived peers.

Testing:

- testing/static/scripts/bench-multirun.sh: multi-run iperf3 +
  ping bench. N reruns (default 5), median / min / max / CoV % /
  per-run outlier flag, avg ping RTT, ICMP loss %, TCP retransmit
  total. Plain client→dest labels + topology header. Pre-bench
  peer-convergence check (FIPS_BENCH_CONVERGE_SECS, default 15);
  per-path route verification via stats.bytes_sent deltas — fails
  fast if traffic exits via a non-static-peer link.

- testing/static/docker-compose.yml: passes FIPS_ENCRYPT_WORKERS /
  FIPS_DECRYPT_WORKERS / FIPS_PERF through to containers for A/B
  benchmarking without rebuilds.

- testing/static/scripts/iperf-test.sh: same plain client→dest
  labels + topology header (was multihop/direct/N hop, which
  conflated topology distance with on-wire path).

- .config/nextest.toml: synthetic UDP node tests serialized
  through a max-threads=1 test group. Localhost handshakes drop
  on shared CI runners under parallel load; one-at-a-time keeps
  assertions reliable.

- src/node/tests/spanning_tree.rs: repair_missing_edge_handshakes
  — retries up to 5 times for synthetic edges whose msg1 was
  dropped, with a drain after each edge retry instead of after
  each attempt's full burst.

- src/node/decrypt_worker.rs::tests: two unit tests asserting
  WorkerMsg::UnregisterSession removes the worker-thread session
  HashMap entry (handle_msg_unregister_session_removes_entry) and
  is a no-op for never-seen cache_keys
  (handle_msg_unregister_session_idempotent_on_unknown_key), which
  is the safety invariant the unconditional unregister calls at
  the four index slots in remove_active_peer rely on.

- src/node/encrypt_worker.rs::unix_tests
  pipelined_send_wire_layout_roundtrips_canonical_decoders: mirrors
  the encoder geometry of try_send_session_data_pipelined (no
  coords, the common established-session path), runs the worker's
  real seal + send via flush_direct_batch_sync, and decodes the
  resulting wire packet using only canonical receive-side decoders
  (EncryptedHeader::parse, SessionDatagramRef::decode, FSP header
  parse, noise::open). Any divergence between the hand-rolled
  encoder offsets (fsp_aad_offset, fsp_plaintext_offset) and the
  decoders fails at one of the parse / open / decode steps before
  the inner-plaintext assertion fires. Complements the existing
  fsp_preseal_runs_before_outer_fmp_seal test which covers the
  seal-ordering invariant with synthetic headers but does not
  exercise the wire-layout invariant.

CHANGELOG.md [Unreleased] # Changed entry added describing the
worker-pool threading model, hash-by-destination dispatch,
sendmmsg/UDP_GSO, per-peer connected UDP, the operator-facing env
vars, and the bench numbers above.

Cherry-picks from mmalmi/master (paths translated from
crates/fips-core/src/ to src/): 9b7c723, 0deb5cb, 13f7339, e036c0e,
3740a68, 3792f83, 8510193, 4910b07, e53f545, e4e2896, 5fe4af5,
1d01ada, 8c37008, e12469e, 6eb2860.

Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
2026-05-19 20:53:31 +00:00
Johnathan Corgan 6bb17be8b8 rekey: disable timer jitter on next pending XX-path investigation
Set `REKEY_JITTER_SECS` to 0 on next. The symmetric per-session
jitter mechanism added on maint/master works cleanly on the IK
FMP rekey path on those branches, but on next's XX FMP rekey
path it produces reproducible post-cutover routing-convergence
failures: ~50% Phase 5 per-pair-ping loss in the rekey integration
suite across multiple runs, with all crypto, transport, and
link-state log assertions still green (no decrypt failures, no
panics, no link teardowns, no rekey msg2 failures, no FSP
decryption failures during rekey).

The XX rekey path itself is sound — pre-merge it had been passing
18/18 across six consecutive CI runs. Isolation experiment
confirms the trigger: with REKEY_JITTER_SECS=0, the same suite
on next passes 70/70 cleanly with FMP cutover counts (18-21)
in line with master's baseline (24). With the constant at 15
and the test suite's 35s nominal rekey interval, FMP cutover
counts climb to 27-40 and Phase 5 fails consistently.

Two attempted fixes to absorb the jitter-induced state churn on
the XX path (post-msg3 addr_to_link restore; optimistic dampening
at msg1 receipt) were tried and reverted — neither restored
Phase 5 reliability. Diagnosis of the underlying XX cutover
state-cleanup behaviour that fails to absorb variable-interval
rekeys is deferred. The const and surrounding draw/redraw
machinery (peer::active::ActivePeer, node::session::SessionEntry,
draw_rekey_jitter at session.rs:20, redraws at the four cutover
sites) are kept in place; flipping the const back to a positive
value is the re-enable path once the investigation lands.

CHANGELOG entry under `### Changed`.
2026-05-14 21:16:43 +00:00
Johnathan Corgan 74b08b9b6f Merge master into next (rekey jitter, Phase 5 settle, acl-allowlist, CI concurrency)
# Conflicts:
#	CHANGELOG.md
2026-05-14 18:54:05 +00:00
Johnathan Corgan 4f3d2f8471 rekey: apply symmetric jitter to desynchronize dual-initiation
Add a per-session signed jitter offset (uniform [-15, +15] seconds)
to the rekey timer triggers in check_rekey (FMP) and check_session_rekey
(FSP). The configured `node.rekey.after_secs` becomes the nominal
interval rather than a floor; mean is preserved. Desynchronizes
both endpoints in symmetric-start meshes so the dual-initiation
race stops occurring rather than being resolved after the fact by
the smaller-NodeAddr tie-breaker.

Per-session storage means each rekey cutover reconstructs the
session and redraws the jitter naturally — successive cycles get
independent offsets, preventing drift back into sync.
2026-05-14 16:32:07 +00:00
Johnathan Corgan acb750c0d2 Merge branch 'master' into next 2026-05-06 15:03:27 +00:00
Johnathan Corgan a62a0a6cf4 nostr: suppress retraversal of cross-FMP-version peers
Open-discovery NAT traversal succeeds at the UDP layer regardless
of what FMP-protocol version the peer speaks. When the daemon
discovers a peer running a different FMP version (e.g. a v0/v1 mix
during a mid-rollout window, or a misconfigured peer in the same
advert namespace), the punch sequence completes, the socket is
adopted via `Node::adopt_established_traversal`, and we initiate
an FMP handshake. The peer drops our msg1 at its own version-gate
and we drop their msg1/msg2 at `Unknown FMP version, dropping`.
Neither side advances the handshake.

Today the bootstrap transport sits idle until the 31s stale-
handshake timeout, drops, and the open-discovery sweep ~30s later
fires the full STUN+offer+answer+punch sequence again — every
minute, indefinitely, against peers the handshake literally cannot
complete with.

Add a `Node::bootstrap_transport_npubs` map populated alongside
`bootstrap_transports` at adopt time. The rx loop reverse-maps the
transport_id → npub on version-mismatch and bumps the discovery
layer's `failure_state` to a long structural cooldown via the new
`NostrDiscovery::record_protocol_mismatch` API. The next sweep
skips the npub for `protocol_mismatch_cooldown_secs` (default
86400 = 24h, separate from the 30-min transient-failure
`extended_cooldown_secs`).

One-shot WARN per fresh observation. Repeat mismatches inside the
cooldown window are silent (the failure_state method returns false
when an existing comparable cooldown is already in place). The
handshake/transport teardown chain is unchanged — the fix is
specifically about preventing the *next* sweep cycle from
re-traversing.

Cleared on `cleanup_bootstrap_transport_if_unused` and on the
adopt-failure rollback path so completed handshakes don't leave
stale entries behind.

Four new unit tests in `failure_state.rs` cover fresh-entry
signaling, repeat-suppression inside the window, streak-pin
behavior for `show_peers` rendering, and post-cooldown re-arming.
2026-05-06 15:02:26 +00:00
Johnathan Corgan 60a4353661 Merge branch 'master' into next 2026-05-04 14:02:09 +00:00
Johnathan Corgan bcc9c525d3 nostr: per-peer NAT-traversal failure suppression and clock-skew handling
Public-test daemons with populous open-discovery caches generate
sustained NAT-traversal-failure WARN volume (~140/hour, ~3500/day)
against cache-learned peers that have gone offline — their adverts
are absent from major Nostr relays but cached entries persist until
their advertised `valid_until` expires. The daemon kept publishing
offers indefinitely under exponential backoff with no per-peer
suppression, drowning operator signal and hammering relays. A
parallel concern: the strict freshness check at signal.rs silently
rejected offers under modest clock skew (now_ms() anchors to
SystemTime once at startup, so post-startup NTP step adjustments
don't propagate on long-uptime daemons), indistinguishable from
"peer is offline."

Six independent improvements layered on the existing retry logic.

Per-npub WARN log rate-limit
----------------------------
New `FailureState` struct on `NostrDiscovery` records per-npub
`last_warn_at_ms`. Subsequent failures inside `warn_log_interval_secs`
(default 5 min) emit DEBUG instead of WARN. Each WARN now also
carries `consecutive_failures` and remaining `cooldown_secs` so
operators can read the trajectory without grepping multiple lines.

Per-npub consecutive-failure counter + extended cooldown
--------------------------------------------------------
After `failure_streak_threshold` (default 5) consecutive failures
against a peer, the next `extended_cooldown_secs` (default 1800)
of attempts are suppressed by pushing
`retry_pending[npub].retry_after_ms` past the cooldown wall. The
open-discovery sweep also consults `cooldown_until` and increments
a new `skipped_cooldown` counter so a peer whose `retry_pending`
was cleared by max_retries doesn't get re-enqueued during the
cooldown window. Caps offer-publish rate per dead peer regardless
of how often the sweep tries to re-enqueue.

Stale-advert eviction on streak-threshold transition
----------------------------------------------------
On the threshold-crossing transition (one-shot, not every
subsequent failure), `tokio::spawn` an active re-fetch of the
peer's Kind 37195 advert from `advert_relays`. Three outcomes:
- absent on relays → cache evicted; sweep won't re-enqueue
  (peer is genuinely gone).
- newer `created_at` → cache refreshed + streak reset
  (peer republished; allowed to retry immediately).
- same → cache untouched; cooldown stands.

Cost: ~one fetch per dead peer per 30-min cooldown cycle, vs
hundreds of offer publishes/hour today.

Clock-skew tolerance on freshness check
---------------------------------------
`signal.rs` `validate_offer_freshness` and
`validate_traversal_answer_for_offer` now allow ±60s grace beyond
strict TTL. Both return a new `FreshnessOutcome` enum so callers
can DEBUG-log when an offer/answer was only accepted via the grace
window. `FRESHNESS_SKEW_TOLERANCE_MS` is hard-coded — loosening
this past minutes erodes the freshness/replay security boundary
and operators tend to tune in the wrong direction.

NTP-style skew estimate (offer_received_at echo)
------------------------------------------------
Added optional `offerReceivedAt: Option<u64>` field to
`TraversalAnswer` payload. Responder fills it with `now_ms()` at
offer-receipt time. Initiator computes the standard NTP offset
formula `((T2-T1) + (T3-T4)) / 2` against the round-trip and
DEBUG-logs when `|skew| ≥ 30s`. Skew is also stashed in
`FailureState` and surfaced in `show_peers`. Non-breaking — older
responders that don't fill the field still produce valid answers,
and `estimate_clock_skew` returns `None`.

Per-peer state in `show_peers` JSON
-----------------------------------
Each peer entry in `show_peers` now carries:

  "nostr_traversal": {
    "consecutive_failures": <u32>,
    "in_cooldown": <bool>,
    "cooldown_until_ms": <u64 | null>,
    "last_observed_skew_ms": <i64 | null>
  }

Always emitted (schema-stable); values populated when discovery is
enabled and the npub has a recorded entry. Required a new public
`Node::nostr_discovery_handle()` accessor and refactored
`FailureState`'s internal Mutex from `tokio::sync` to `std::sync`
(operations never hold across await), which lets the synchronous
`show_peers` handler call `snapshot()` directly without the
dispatcher becoming async.

New config knobs (under `node.discovery.nostr`)
-----------------------------------------------
  failure_streak_threshold: 5
  extended_cooldown_secs: 1800
  warn_log_interval_secs: 300
  failure_state_max_entries: 4096

Tests
-----
12 new unit tests:
- 5 in `tests.rs` covering freshness strict / tolerated / rejected
  outcomes, NTP skew estimation, and the backward-compat None case
  when the responder didn't fill `offer_received_at`.
- 7 in `failure_state.rs` covering streak/warn-rate-limit state
  transitions, cooldown active vs expired semantics,
  success-resets-streak, observed-skew records, and size-cap
  eviction by oldest `last_failure_at`.

CHANGELOG entries added under `[Unreleased]` Fixed.
2026-05-04 12:39:51 +00:00
Johnathan Corgan 1c8b8d55e8 Merge branch 'master' into next 2026-05-03 12:32:49 +00:00
Johnathan Corgan 037a965a93 Mirror reactive MtuExceeded into path_mtu_lookup
When a transit forwarder drops an oversized data packet and reports
the bottleneck back via MtuExceeded, the receive-side handler
already updates per-session MmpSessionState::path_mtu (used by PTB
synthesis to feed kernel TCP). It did not, however, update
path_mtu_lookup — the per-destination map the TUN reader/writer
consult at TCP MSS clamp time. So forward-path-asymmetry flows kept
clamping at the discovery reverse-path value (too generous for the
actual forward-path budget) on every subsequent SYN.

Add the missing write at the same point apply_notification runs.
Keep the tighter of existing-or-new — the clamp must never loosen.
Same write-shape as seed_path_mtu_for_link_peer.

Tests:

- Three focused unit tests on handle_mtu_exceeded for the empty,
  tighten, and keep-tighter cases.
- Extended test_multihop_pmtud_heterogeneous_mtu to assert the
  lookup tightens after the wire-level MtuExceeded propagation,
  alongside its existing PathMtuState assertion.

Adds two #[cfg(test)] accessors on Node (path_mtu_lookup_get /
path_mtu_lookup_insert) and bumps handle_mtu_exceeded to
pub(in crate::node) for direct test invocation.
2026-05-02 18:30:09 +00:00
Johnathan Corgan b02eaf2c5b Merge branch 'master' into next
Forward-merges the squashed per-destination TCP MSS clamping work
(master `ae60743`) into next.

Auto-merge of source files clean. Two pre-existing test assertions
on the next-side discovery tests were updated as part of merge
resolution to reflect the target-edge MTU fold introduced by the
merged-in commit:

- test_transit_forwards_when_mtu_sufficient: 1400 → 1280 = min(target-edge 1280, transit 1400)
- test_response_path_mtu_four_node_chain: 1350 → 1280 = min(target-edge 1280, transits 1350+1500)

Resulting next tree is bit-for-bit identical to the pre-squash next
state previously verified by full local CI sweep (29 + 1 next-only
suites pass on `6a8b519` in 23m 41s).
2026-05-02 17:39:26 +00:00
Johnathan Corgan ae607431eb Per-destination TCP MSS clamping at the TUN boundary
Adds source-side TCP MSS clamping informed by per-destination path
MTU learned via discovery, with a conservative IPv6-minimum-derived
ceiling for cold flows where discovery has not yet completed. Closes
the multi-hop default-config TCP wedges observed in production where
a sender's local-floor MSS exceeds what some intermediate forwarder
hop is willing to carry: silent drops, no PTB feedback through the
userspace TUN to the kernel TCP stack, retransmits at the same too-
large MSS, application connection times out.

## Architecture

A new `Arc<RwLock<HashMap<FipsAddress, u16>>>` field
`path_mtu_lookup` on Node mirrors the per-destination path MTU in a
form accessible from sync TUN reader/writer threads. A new
`per_flow_max_mss` helper in `src/upper/tun.rs` reads the lookup at
SYN-clamp time and returns the appropriate ceiling for the flow.

Three write sites populate `path_mtu_lookup`:

1. **Discovery originator branch** of `handle_lookup_response`:
   the path MTU bottleneck accumulated through the reverse path
   lands here when a LookupResponse arrives at the originator.
   Same value also lands in `coord_cache` per the existing
   `insert_with_path_mtu` API.

2. **FMP peer-promotion seed** (`seed_path_mtu_for_link_peer`):
   when an FMP link-layer peer is promoted to active, the local
   outgoing-link MTU on the peer's transport seeds the lookup.
   Tighter existing values (learned via discovery) are preserved;
   the seed only writes when no entry exists or the existing
   value is looser than the link MTU. Without this seed,
   directly-configured peers (auto_connect / static peer config)
   would leave `path_mtu_lookup` empty for their FipsAddress
   because the FSP session establishes without ever issuing a
   LookupRequest.

3. **Target-edge fold at `send_lookup_response`**: when a node is
   the discovery target, it folds its own outgoing-link MTU to
   the response's next-hop into `path_mtu` before sending.
   Without this fold, the response leaves the target with
   `path_mtu = u16::MAX` and only intermediate transits min-fold;
   the target's first reverse-path hop is never represented in
   the bottleneck calculation. Refactored the existing transit-
   side min-fold into a shared `apply_outgoing_link_mtu_to_response`
   helper called from both sites.

## Read-side: per_flow_max_mss

Two TUN call sites consume the lookup:

- Outbound `handle_tun_packet` clamps SYN MSS using packet[24..40]
  (IPv6 destination) as the lookup key.
- Inbound `TunWriter::run` clamps SYN-ACK MSS using packet[8..24]
  (IPv6 source).

When the lookup contains a learned value, the helper computes
`min(global_max_mss, effective_ipv6_mtu(path_mtu) - 60)` where 60
is IPv6 (40) + TCP (20) headers and `effective_ipv6_mtu` accounts
for the FIPS encapsulation overhead.

When the lookup is empty for a destination — the cold-flow case —
the helper returns `min(global_max_mss, IPv6-minimum-derived
ceiling)`. RFC 8200 mandates every IPv6 path accept ≥1280-byte
packets, so the IPv6-minimum-derived MSS (1280 - 77 - 60 = 1143)
fits any compliant path. Without this conservative ceiling, the
first SYN to a destination with no learned path MTU exits the TUN
at the kernel-natural MSS (TUN MTU - 60), and the application
connection wedges silently before discovery completes for a
corrected second SYN to fire. The fix is provably safe: the
ceiling is taken with `min` against the local global so operators
with even tighter local floors are never loosened upward.

Subsequent flows pick up the actual learned per-destination value
once discovery (or the FMP-promotion seed for direct peers)
populates the lookup.

## Diagnostic logging

All write and read sites emit instrumentation suitable for
operators bisecting a wedged path:

- `debug!` log on every `path_mtu_lookup` write (discovery
  originator path and FMP-promotion seed path), showing the
  FipsAddress, written value, prior value, and post-write map
  size. `warn!` on poisoned-lock failure path.
- `trace!` log per `per_flow_max_mss` call covering every
  fall-through branch (wrong addr_bytes length, non-fd::/8
  prefix, lookup poisoned, no entry for destination, empty-lookup
  conservative ceiling) and the success path. trace level filters
  out under normal log settings; capture with
  `RUST_LOG=info,fips::node::handlers::discovery=debug,fips::upper::tun=trace`.

## Tests

15 new unit tests across 3 files:

- `per_flow_max_mss` (8 tests in `src/upper/tun.rs::tests`):
  empty-lookup conservative ceiling, empty-lookup global-smaller
  floor, learned-value-overrides-conservative, per-destination
  smaller, per-destination larger capped by global, non-fips
  addr, short addr slice, per-destination independence.
- `seed_path_mtu_for_link_peer` (4 tests in
  `src/node/tests/unit.rs`): seed when empty, keep tighter
  existing, tighten looser existing, no-op for unknown
  transport.
- Discovery integration (3 tests in `src/node/tests/discovery.rs`):
  apply_outgoing_link_mtu_to_response on unknown peer no-op,
  two-node target-edge fold (path_mtu reflects target-edge link),
  three-node chain transit min-fold (existing test, updated for
  target-edge inclusion).

Two pre-existing discovery tests had assertions updated to
account for the target-edge fold:

- `test_response_path_mtu_two_node`: previously asserted
  `u16::MAX` (no transit to min-fold); now asserts 1280 (the
  test transport MTU, folded in by send_lookup_response).
- `test_response_path_mtu_four_node_chain`: previously asserted
  1350 (transit MTUs only); now asserts 1280 (target-edge MTU
  is the bottleneck).
- `test_transit_forwards_when_mtu_sufficient`: previously
  asserted 1400 (transit MTU only); now asserts 1280 (target-
  edge MTU is the bottleneck).

## Verification

Local CI on this commit: 29/29 suites pass, 1105 lib tests pass,
clippy --all-targets --all-features -D warnings clean, cargo fmt
clean. Production deploy verified via trace capture across the
managed fleet: cold-flow conservative ceiling branch fires on
first SYN, learned-lookup branch takes over once discovery
completes, both behaviors observable end-to-end at the SYN MSS
on the wire.

No wire-format change. No config-format change.
2026-05-02 17:37:07 +00:00
Johnathan Corgan e7fc4f7d10 Merge branch 'master' into next 2026-05-02 02:14:36 +00:00
Johnathan Corgan 8448e38510 Make Node::transport_mtu() deterministic across restarts (TCP black hole fix)
Default-config TCP flows between fips peers were stalling completely
(cwnd-pinned, 0 bps for 10s+) on a non-trivial fraction of restarts.
Reproducible with iperf3 between any two peers.

Root cause: `Node::transport_mtu()` iterated `self.transports.values()`
(HashMap with default RandomState hasher) and returned `handle.mtu()`
of the first one whose `is_operational()` returned true. Two stacked
sources of non-determinism stacked on each other: HashMap iteration
order is randomized per-process via RandomState, and async transport
`.start()` completion order races each daemon restart.

The returned value drives the TCP MSS clamp ceiling computed once at
TUN init (src/upper/tun.rs:501-524) and stored as immutable max_mss
in the reader/writer thread state. When the picker landed on a
transport with MTU > 1357 (any non-UDP-1280 in the standard fleet
defaults), `max_mss > 1220` (kernel's natural fips0-MTU-derived MSS),
the daemon's clamp was silently a no-op, and the kernel emitted
1220-byte segments. Those wrap into 1280-byte IPv6 → 1357-byte fips
datagrams that exceed UDP-1280 transports at any forwarding hop,
causing silent drops with no PTB feedback to the kernel TCP stack.

Fix: return min across operational transports instead of first-iterated.
With UDP-1280 in the configured set (the common case),
`transport_mtu = 1280`, `max_mss = 1143 < 1220`, daemon's clamp
engages, MSS=1143 reaches the wire, packets fit, throughput recovers.

Empirical green light from a single-UDP-config end-to-end test:
iperf3-without-`-M` recovered to ~21 Mbps with no operator-side nft
TCPMSS rules.

Adds three unit tests:
- transport_mtu_returns_min_across_operational: pin selection to
  smallest MTU when multiple operational transports differ.
- transport_mtu_fallback_when_no_operational_transports: 1280 fallback.
- transport_mtu_min_with_single_operational: trivial single-transport
  case.

The `effective_ipv6_mtu` field reported by `fipsctl show status` was
also racy (consequence of the same bug); fixed by this change as a
side effect.
2026-05-02 01:27:15 +00:00
Johnathan Corgan ab2edec2c6 Add Nostr open-discovery startup sweep with diagnostic logging
Under `node.discovery.nostr.policy: open`, the per-tick auto-dial in
`queue_open_discovery_retries` was supposed to pick up adverts cached
from the relay subscription backlog at startup, but in practice only
adverts arriving live (after the daemon was up) were being dialed.
Backlog adverts sat in the in-memory cache until they aged out.

Adds a one-shot startup sweep that runs once per daemon start, gated
identically to the per-tick sweep (`enabled` && `policy == open`),
after a configurable settle delay so the relay subscription backlog
has time to populate the advert cache. The sweep iterates the cache
with the same skip-filters as the per-tick path (statically-configured
peers, already-connected, retry-pending, connecting) plus a tighter
age filter: only adverts whose `created_at` is within
`startup_sweep_max_age_secs` of now are queued.

Two new config fields under `node.discovery.nostr`:
- `startup_sweep_delay_secs` (default 5)
- `startup_sweep_max_age_secs` (default 3600 = one hour)

Both are only consulted when `policy == open`; under any other
policy the sweep is a no-op.

Adds diagnostic logging to the open-discovery sweep so operators can
verify what the auto-dial path is doing on each daemon bring-up:
info-level on each retry-queued enqueue (with peer short-npub and
advert age), and a one-line summary on every startup sweep and on
any per-tick sweep that queues at least one retry. The summary
breaks down skipped candidates by reason (age, configured, self,
already-connected, retry-pending, connecting, no-endpoints,
invalid-npub) — currently the path was silent so there was no
operator-visible signal that the cache iteration was running.

Refactors the existing `queue_open_discovery_retries` body into a
shared `run_open_discovery_sweep(max_age_secs, caller)` helper so
the per-tick and startup paths share filter/queue logic and only
differ in the age filter and log label. Surfaces `created_at` from
`NostrDiscovery::cached_open_discovery_candidates` (return tuple
extended) so the age filter has the data it needs.

Three new unit tests in `config::node::tests` cover the new defaults,
YAML override round-trip, and partial-YAML default fallback.
2026-05-01 18:09:50 +00:00
Johnathan Corgan 5317735a91 Merge branch 'master' into next
# Conflicts:
#	src/node/tests/handshake.rs
#	testing/ci-local.sh
2026-04-30 14:05:59 +00:00
Johnathan Corgan e641eb5b5f Drop the nostr-discovery cargo feature flag
Make Nostr-mediated overlay discovery unconditional, mirroring the
philosophy of PR #79's collapse of the tui/ble/gateway features in
favor of platform cfg gates. nostr / nostr-sdk are pure Rust over
WebSockets/TCP, so they build cleanly on every FIPS-supported
platform — there is no need for the parallel feature gate.

The flag was already in `default = [...]`, so no behavior change for
anyone using `cargo build` without `--no-default-features`. Operators
who explicitly disabled the feature will now find Nostr code present
in the binary; the runtime check `node.discovery.nostr.enabled` still
controls whether the runtime starts.

Cargo.toml:
- Remove the `[features]` table entirely.
- Drop `optional = true` from `nostr` and `nostr-sdk`.

Source: 27 cfg sites collapsed across 5 files —
`src/discovery.rs`, `src/discovery/nostr/mod.rs`,
`src/node/handlers/rx_loop.rs`, `src/node/lifecycle.rs`,
`src/node/mod.rs`. Two `#[cfg(not(feature = "nostr-discovery"))]`
fallback blocks (the udp:nat-without-runtime debug-log path and the
"feature not compiled in" warning at startup) were removed as dead
code; the always-on path already handles the missing-runtime case
via `nostr_discovery: Option<NostrDiscovery>`.

Packaging and tooling:
- `packaging/openwrt-ipk/Makefile`: drop a stale `--features gateway`
  flag (the `gateway` feature was already removed in PR #79; this
  was a leftover that the build path tolerated only because cargo
  ignored unknown feature names).
- `testing/scripts/build.sh`: drop `DEFAULT_CARGO_BUILD_ARGS=(--features
  nostr-discovery)`; defaults are empty.
- `packaging/common/fips.yaml`: drop the "requires the
  nostr-discovery feature" comment from the discovery section.

Bundled cleanup:
- Apply `cargo clippy --fix` against three pre-existing warnings in
  `src/discovery/nostr/runtime.rs` and `src/discovery/nostr/stun.rs`
  (collapsed `if let Some` chain; two redundant `as i32` casts).
  These were always present but masked when the feature gate was
  off; they surface now that the code is unconditionally compiled.
- `cargo fmt` settled two minor formatting drift sites in
  `src/bin/fips-gateway.rs` and `src/config/mod.rs`.

Tests: 1083 passed, 0 failed, 4 ignored. clippy clean. fmt clean.
2026-04-30 10:24:32 +00:00
Johnathan Corgan 5a5dbe36da Merge branch 'master' into next
# Conflicts:
#	src/node/lifecycle.rs
#	testing/static/scripts/rekey-test.sh
2026-04-27 16:10:11 +00:00
34e00b9f6e Add Nostr-mediated overlay discovery and UDP NAT traversal (#53)
Optional peer discovery and NAT hole-punching path gated behind a new
`nostr-discovery` cargo feature. Nodes publish signed overlay endpoint
adverts to public Nostr relays, consume peer adverts to populate
fallback dial addresses, and use STUN-assisted UDP hole punching with
NIP-59 gift-wrap offer/answer signaling to establish direct UDP paths
between NATed peers. Once a punched socket is up, it is handed into
the existing FIPS UDP transport and the standard Noise/FMP session
stack takes over unchanged.

The cargo feature is in the default feature set
(`default = ["nostr-discovery"]`) so stock builds include it; a
build that explicitly disables default features (or selects a
feature set without `nostr-discovery`) does not link the nostr /
nostr-sdk crates and does not emit a no-op poll in the tick loop.
Runtime behavior is independently gated by
`node.discovery.nostr.enabled`, which defaults to false; if the
config enables Nostr on a non-feature build, startup logs a
warning and continues without it.

== Cargo feature and dependencies

- New cargo feature `nostr-discovery = ["dep:nostr", "dep:nostr-sdk"]`.
  Not in the default feature set.
- New optional Linux-only dependencies: `nostr 0.44` (features: std,
  nip59) and `nostr-sdk 0.44`. Gift-wrap unwrap is hand-rolled in
  `src/discovery/nostr/signal.rs` rather than relying on the SDK's
  rumor-author check, which FIPS sidesteps by trusting `seal.pubkey`
  exclusively.

== Wire format

Overlay advert event: `kind 37195`, parameterized replaceable
(NIP-01 application-defined replaceable range 30000-39999), with
`d = "fips-overlay-v1"`. The digits visually spell FIPS (7=F, 1=I,
9=P, 5=S); a relay survey confirmed the kind is unused.

Advert content carries the version tag, endpoint list
(`udp|tcp|tor` + addr), optional signal-relay and stun-server
metadata, and `issuedAt` / `expiresAt` timestamps. Endpoint
`addr: "nat"` is the sentinel that triggers traversal on the peer
side. NIP-40 `expiration` tag bounds staleness on permanent
shutdown. Lifecycle relies on parameterized-replaceable
supersession; the daemon does not emit NIP-09 kind-5 deletes —
strict relays (Damus, Primal) race delete-against-replace and can
silently drop the replacement.

Gift-wrapped signal event: `kind 21059`. Punch packets carry magic
values `PUNCH_MAGIC` / `PUNCH_ACK_MAGIC`, a sequence number, and a
16-byte session hash.

== Discovery surface

- `src/discovery.rs` (always compiled)
  - `EstablishedTraversal`: bound UDP socket + selected remote +
    peer npub + optional transport name/config tuning overrides.
  - `BootstrapHandoffResult`: returned on successful handoff —
    allocated transport id, local/remote addrs, peer NodeAddr,
    session id.
- `src/discovery/nostr/` (`#![cfg(feature = "nostr-discovery")]`)
  - `types.rs`: wire and control types described above. `ADVERT_KIND`
    constant. `BootstrapError` enumerates failure modes (disabled,
    missing advert, missing NAT endpoint, no usable relays, invalid
    advert, invalid npub, signal timeout, punch timeout, replay,
    STUN failure, protocol, nostr, io, serde, event-parse).
  - `runtime.rs`: `NostrDiscovery` coordinator. Owns the shared
    nostr-sdk `Client`, subscribes to advert + signal event kinds,
    maintains a bounded advert cache and a bounded seen-sessions
    replay set, drains `BootstrapEvent::{Established, Failed}` for
    the node to consume, exposes `update_local_advert`,
    `request_connect`, `advert_endpoints_for_peer`,
    `cached_open_discovery_candidates`, and `shutdown`.
  - `signal.rs`: NIP-59 gift-wrap encode/decode. Outbound wraps are
    built against per-attempt ephemeral keys; inbound events are
    unwrapped against the node identity.
  - `stun.rs`: RFC 5389/8489 Binding Request client with
    XOR-MAPPED-ADDRESS parsing for both IPv4 and IPv6; used only to
    observe the initiator's own reflexive address against its
    locally configured STUN list (peer-advertised STUN is
    informational, never an egress target).
  - `traversal.rs`: per-attempt candidate-pair punch planner.
    Allocates a fresh `0.0.0.0:0` UDP socket per attempt, enumerates
    LAN-private and ULA interface addresses alongside the STUN
    reflexive address, schedules probe/ack exchanges at the
    configured interval for the configured duration, and picks the
    first candidate pair that authenticates end-to-end.

Strategy ordering is Reflexive↔Reflexive first, then LAN, then
Mixed. The STUN-observed pair is the only candidate that's reliable
across arbitrary network topologies; trying it first prevents the
planner from latching onto a misleading host-candidate path before
the reflexive path gets a chance. There is no catch-all
Local↔Local strategy: a previous design that paired every local
host candidate from one side with every local host candidate from
the other could declare success on a one-way reachable asymmetric
L3 path (corporate VPN, Tailscale subnet route, overlapping private
address space), only for the FMP handshake to stall because the
return path didn't match. The legitimate `Lan` strategy still pairs
candidates that share a subnet.

== Configuration surface

`node.discovery.nostr.*` (`NostrDiscoveryConfig`), all `serde(default)`
with `deny_unknown_fields`:

- `enabled` (default false), `advertise` (default true)
- `advert_relays`, `dm_relays`, `stun_servers`: defaults are
  `wss://relay.damus.io`, `wss://nos.lol`, `wss://offchain.pub`
  for both relay lists, and Google / Cloudflare / Twilio for STUN.
  Operators are expected to override for production. Other
  verified-working public relays for reference:
  `nostr.bitcoiner.social`, `nostr-pub.wellorder.net`,
  `nostr.oxtr.dev`, `nostr.mom`.
- `app` (default `"fips-overlay-v1"`), `signal_ttl_secs` (120)
- `policy`: `NostrDiscoveryPolicy::{Disabled, ConfiguredOnly (default),
  Open}` — controls whether advert-derived endpoints are consumed
  only for peers carrying `via_nostr = true`, or also for
  non-configured peers within a budget cap.
- `share_local_candidates` (default false) — when false, the offer's
  `local_addresses` list is empty and peers see only the reflexive
  address. Enable per-node only for genuinely same-LAN deployments;
  off-by-default eliminates the misleading-path failure mode for
  the common case where peers are not on the same broadcast domain.
- `open_discovery_max_pending` (64) — caps queued open-discovery
  retries; bounded by available outbound slots.
- `max_concurrent_incoming_offers` (16) — semaphore against offer
  spam; excess offers are debug-logged and dropped.
- `advert_cache_max_entries` (2048) and `seen_sessions_max_entries`
  (2048) — bound memory under ambient relay volume; overflow
  evictions are debug-logged.
- `attempt_timeout_secs` (10), `replay_window_secs` (300)
- `punch_start_delay_ms` (2000), `punch_interval_ms` (200),
  `punch_duration_ms` (10000)
- `advert_ttl_secs` (3600), `advert_refresh_secs` (1800)

Per-peer and per-transport flags:

- `PeerConfig.via_nostr: bool` — when true (and Nostr is enabled),
  advert-derived addresses are appended as fallback dial candidates
  after static addresses for that peer.
- `PeerConfig.addresses` is now `serde(default)` and may be empty
  when `via_nostr: true`; validation requires at least one of the
  two to be present per peer, and the error message names the
  peer's npub.
- `UdpConfig.advertise_on_nostr: Option<bool>` and
  `UdpConfig.public: Option<bool>` — UDP transports can be
  advertised either as direct `host:port` (public = true) or as the
  `addr: "nat"` sentinel that triggers rendezvous on the peer side.
- `TcpConfig.advertise_on_nostr` and `TorConfig.advertise_on_nostr`
  — TCP and Tor onion endpoints can be advertised as directly
  reachable.
- A reserved peer address `transport: udp, addr: "nat"` parses without
  special-casing in YAML and routes through the bootstrap runtime.

Cross-field validation (`Config::validate`, called from `Node::new`
and `Node::with_identity`):

- Any transport with `advertise_on_nostr = true` requires
  `node.discovery.nostr.enabled = true`.
- Any peer with `via_nostr = true` requires
  `node.discovery.nostr.enabled = true`.
- A non-public UDP advert (`advertise_on_nostr = true`,
  `public = false` — i.e. `udp:nat`) additionally requires at least
  one `dm_relay` and at least one `stun_server`.
  Surfaced as `ConfigError::Validation`.

== Node integration

`src/node/lifecycle.rs` is the main integration point.

- At node start (after transports are up, before TUN), if Nostr is
  enabled and the feature is compiled in, `NostrDiscovery::start` is
  invoked, the initial local overlay advert is built from the live
  transport set and published, and the runtime handle is stored.
- The rx tick loop calls `poll_nostr_discovery` (feature-gated both
  at method definition and call site), which refreshes the local
  advert, drains bootstrap events, adopts established traversals,
  schedules retries for failed traversals, and — under `policy:
  open` — enqueues outbound retries for non-configured peers
  visible in the advert cache, bounded by
  `open_discovery_max_pending` and the remaining outbound slots.
- Outbound peer dialing is refactored to `try_peer_addresses`, which
  first exhausts the static address list in priority order and only
  then appends advert-derived fallback addresses; both lists run
  through the same `attempt_peer_address_list` code path. The
  `udp:nat` sentinel address triggers `NostrDiscovery::request_connect`
  for the peer instead of a direct dial and returns `Ok(())`.
- `build_overlay_advert` walks operational transports, consults
  per-instance `UdpConfig` / `TcpConfig` / `TorConfig` (matching by
  optional transport instance name), and emits an `OverlayAdvert`
  including `signalRelays` and `stunServers` when any UDP endpoint
  is advertised as NAT.
- `adopt_established_traversal` is the bootstrap handoff API:
  allocates a new `TransportId`, constructs a `UdpTransport` with
  the user-supplied (or default) `UdpConfig`, calls the new
  `adopt_socket_async` to reuse the punched socket verbatim,
  registers the transport in the normal transport map, records it
  in `bootstrap_transports`, and calls `initiate_connection` so the
  normal handshake path runs. On failure, the transport is stopped
  and removed cleanly and the set membership is rolled back.
- On clean shutdown, `NostrDiscovery::shutdown` is awaited so
  background tasks stop before transports are torn down. (The
  advert is not explicitly retracted; NIP-40 expiration plus the
  next refresh from any live publisher supersedes it.)

New `Node` fields:

- `nostr_discovery: Option<Arc<NostrDiscovery>>` (feature-gated).
- `bootstrap_transports: HashSet<TransportId>` — per-peer UDP
  transports adopted from NAT traversal, cleaned up via
  `cleanup_bootstrap_transport_if_unused` whenever the link,
  connection, peer, or pending-connect referencing them is removed.

Retry and error surface:

- `RetryState.expires_at_ms: Option<u64>` — optional absolute expiry
  for a retry entry. `pump_retries` drops expired entries with an
  info log. Used for open-discovery retries, which expire at two
  times the advert TTL.
- New `NodeError::BootstrapHandoff(String)` returned from
  `adopt_established_traversal` when the underlying transport
  adoption fails or local address discovery fails.
- New `ConfigError::Validation(String)`.
- A small refactor extracts `Node::now_ms()` and reuses it across
  lifecycle, rx-loop tick, and timeout bookkeeping.

== UDP transport

`src/transport/udp/`:

- `UdpRawSocket::adopt(std::net::UdpSocket, recv_buf, send_buf)`:
  adopts an externally bound socket, makes it non-blocking, applies
  the configured buffer sizes (warning if the kernel clamps), and
  reports the resulting local address. Preserves the NAT mapping —
  no rebind.
- `UdpTransport::adopt_socket_async(std::net::UdpSocket)`: the
  `start_async` analogue for an already-bound socket, wiring the
  async socket and recv task exactly as the fresh-bind path would.
- `Drop` impl for `UdpTransport`: if a transport is dropped while
  still holding a recv task or socket (for example on error
  teardown), aborts the task, clears the socket, and emits a debug
  log so the cleanup is visible in tracing rather than silent.

== Logging and observability

Default `EnvFilter` demotes third-party relay-pool DEBUG output to
TRACE-only: `nostr_relay_pool`, `nostr_sdk`, and `nostr` are pinned
at INFO when our level is anything below TRACE, and at TRACE when
our level is TRACE — so the raw frames are still reachable when
explicitly asked for. RUST_LOG continues to override completely.

Concise one-line DEBUG events are emitted at the meaningful points
in the discovery / hole-punch sequence:

- `advert: published` (event id, relay count, endpoints, ttl)
- `advert: peer cached` (notify-loop ingress for non-self)
- `advert: resolved` (cache hit / relay fetch outcome)
- `traversal: initiator starting`
- `traversal: initiator STUN observed` (reflexive, local count)
- `traversal: offer sent` (session id, relay count, event id)
- `traversal: answer received` (accepted, reflexive, local)
- `traversal: initiator punch succeeded` (remote addr)
- `traversal: offer received` (responder side)
- `traversal: responder STUN observed`
- `traversal: answer sent`
- `traversal: responder punch succeeded`

Npubs are shortened to `npub1<4>..<4>` and event/session ids to
their first 8 hex characters.

Other operator-facing logs:

- `UdpTransport` adoption and drop paths log at info / debug.
- `adopt_established_traversal` logs at debug on entry and info on
  successful return, tagged with peer npub, session id, transport
  id, and both socket endpoints, so the bootstrap handoff is
  traceable end-to-end alongside the `UdpTransport::drop` log.
- `cleanup_bootstrap_transport_if_unused` logs at debug when the
  reference-count check drops an adopted transport.
- `connect_peer` tags its entry `debug!` with `peer_npub` so
  downstream STUN, punch, and handshake logs for the same peer
  correlate for operators.
- Advert-cache and seen-sessions overflow evictions log at debug so
  mis-sized caps are visible under ambient relay volume.
- Gift-wrap unwrap failures on `SIGNAL_KIND` events log at trace
  (hot path: fires for every unrelated signal event on the same
  relay).
- Traversal-offer handler failures log at debug. Expected conditions
  such as punch timeout on symmetric NAT are covered there; real
  problems are reported upstream via `BootstrapEvent::Failed`.
- Inbound-offer rate-limit messages name the governing config field
  (`max_concurrent_incoming_offers`) and state that the offer was
  rate-limited rather than failing.

== Tests

- 18 new unit tests in `src/discovery/nostr/tests.rs` covering advert
  encoding, signal envelope round-trip, STUN parsing, punch-packet
  codec, and replay-window enforcement. Run under the
  `nostr-discovery` feature.
- Config-validation tests in `src/config/mod.rs` covering the three
  cross-field invariants and YAML parsing of the full
  `node.discovery.nostr` block plus `peers[].via_nostr`, empty
  `addresses` with `via_nostr: true`, and a `udp: nat` address.
- `src/node/tests/bootstrap.rs` integration tests that drive a
  synthetic traversal (bound UDP socket pair + synthetic peer
  identity) through `adopt_established_traversal` and assert the
  Noise handshake completes over the adopted socket.
- Punch-planner tests assert reflexive-before-LAN ordering and that
  same-LAN scenarios still include the LAN target in the plan.
- `testing/nat/` Docker NAT lab harness:
  - Local `strfry` relay, local STUN responder, and one or two
    router containers performing `iptables` NAT.
  - Node LAN interfaces are provisioned with explicit `veth` pairs
    injected into the node and router namespaces so every packet
    traverses the router namespace (plain Docker bridges are not
    used for the LAN).
  - `cone` scenario: both peers behind full-cone-emulation NAT
    (SNAT with source-port preservation, inbound DNAT back to the
    single LAN host regardless of remote source); asserts UDP
    traversal succeeds and link remote addresses are on the router
    WAN subnet.
  - `symmetric` scenario: `MASQUERADE --random-fully`; asserts UDP
    traversal fails and TCP fallback converges over router-
    published WAN addresses.
  - `lan` scenario: both peers share a LAN subnet; asserts LAN
    addresses are preferred over reflexive ones.
  - Cleanup tears down all profile-gated services
    (`--profile cone --profile symmetric --profile lan`) so no
    orphan containers survive a run.
- `testing/scripts/build.sh` builds the Docker test image with
  `--features "tui nostr-discovery"` by default so NAT-harness
  binaries include bootstrap support.

== CI

- Linux release build and nextest unit-test job both use
  `--features "gateway nostr-discovery"` so the feature-gated code
  and its unit tests compile and run in CI.
- Three new integration matrix entries (`nat-cone`, `nat-symmetric`,
  `nat-lan`) invoke `testing/nat/scripts/nat-test.sh`, collect
  `docker compose logs` on failure, and always stop containers.

== Packaging and operations

- `packaging/common/fips.yaml` ships a fully commented
  `node.discovery.nostr.*` block, plus documented
  `advertise_on_nostr` / `public` examples under the UDP transport,
  an `advertise_on_nostr` example under TCP, and a `via_nostr: true`
  example under the static peer section with both a direct
  `host:port` UDP address and a `udp: nat` fallback.
- `.github/workflows/package-openwrt.yml`: NIP-94 release event
  publishes target the new default relay set.

== Documentation

- `README.md`: overlay discovery + NAT traversal moved from
  "Near-term priorities" into "What works today".
- `docs/design/fips-intro.md`: rewrites the paragraphs that
  previously described Nostr discovery and NAT traversal as future
  work; describes the shipped mechanism and the feature gate.
- `docs/design/fips-transport-layer.md`: drops the "(future
  direction)" qualifier from the Nostr Relay Discovery section,
  expands with the `udp:nat` advertisement and bootstrap handoff
  description, and updates the Current State callout.
- `docs/design/fips-mesh-layer.md`: notes that mid-session NAT
  rebinding (roaming) and initial NAT traversal (Nostr path) are
  distinct mechanisms.
- `docs/design/fips-configuration.md`: documents the full
  `node.discovery.nostr.*` surface, including the three resource
  caps and `share_local_candidates`.
- `docs/design/fips-nostr-discovery.md`: design and configuration
  reference for the shipped mechanism, including the empty-
  `addresses`-with-`via_nostr` shorthand.
- `docs/proposals/nostr-udp-hole-punch-protocol.md`: adds an
  Implemented status callout, clarifies that the punch socket is
  per-peer and per-attempt rather than shared with the application
  listener, aligns field names with the shipped JSON
  (`sessionId`, `issuedAt` / `expiresAt`, `reflexiveAddress`,
  `localAddresses`, `stunServer`), sets the `d`-tag to
  `fips-overlay-v1`, names the kind as 37195, and notes that
  advertised STUN entries are informational.
- `docs/proposals/README.md`: adds a Status column and marks the
  hole-punching proposal Implemented.
- `CHANGELOG.md`: Unreleased > Added entry covering the discovery
  path, STUN/punch path, configuration surface, and Docker NAT lab.

Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
2026-04-27 08:15:58 -07:00
Johnathan Corgan f16b837a12 Tune overly aggressive discovery rate limiting
The default 30s post-failure backoff (300s cap, doubling per
consecutive failure) was set to bound traffic from chatty apps
looking up unreachable targets, but in practice it dominates
cold-start mesh convergence: a single timed-out lookup during
initial bloom-filter propagation suppresses any retry for 30s, and
the existing reset triggers (parent change, new peer, first RTT,
reconnection) don't fire on a stable post-handshake topology. The
suppression window winds up dictating the protocol's effective
time-to-converge instead of bounding repeat traffic.

Replaces the single-lookup-with-internal-retry model
(`timeout_secs`/`retry_interval_secs`/`max_attempts`) with a
per-attempt timeout sequence in `node.discovery.attempt_timeouts_secs`,
defaulting to `[1, 2, 4, 8]`. Each attempt sends a fresh LookupRequest
with a new random request_id so successive attempts can take different
forwarding paths as the bloom and tree state evolve. The destination
is declared unreachable only after the sequence is exhausted (15s
total at the default).

Disables post-failure suppression by default (`backoff_base_secs`/
`backoff_max_secs` now `0`/`0`). The `DiscoveryBackoff` machinery
stays in tree (inert at zero base/cap); operators with chatty apps
generating repeat lookups against unreachable destinations can opt
back in.

`PendingLookup` field shape unchanged so the control-socket
`show_routing` JSON (`pending_lookups[].attempt`/`initiated_ms`/
`last_sent_ms`) keeps the same schema for fipstop and external
consumers; `last_sent_ms` now means "current-attempt start" under
the new state machine.
2026-04-26 19:12:19 +00:00
Johnathan CorganandGitHub cbc78091ab Rationalize cargo feature and platform-gate surface (#79)
Drop the `tui`, `ble`, and `gateway` cargo features and replace
them with platform cfg gates. Plain `cargo build` now produces
every subsystem appropriate for the target platform with no
feature flags required.

Motivation:
- `default = ["tui", "ble"]` broke `cargo build` on macOS and
  Windows because `ble` pulled in `bluer` (BlueZ, Linux-only).
  Every non-Linux packager needed `--no-default-features`.
- The feature flags on `ble` and `gateway` were redundant with
  their platform-gated deps (`bluer`, `rustables`). The parallel
  gating was inconsistent and error-prone.
- `tui` feature protected against a ratatui binary-size concern
  that no longer applies in 2026.

Cargo.toml:
- Remove `tui`, `ble`, `gateway` features; `default = []`.
- Promote `ratatui` to a non-optional top-level dependency.
- Move `rustables` from top-level optional into the Linux
  target block, non-optional.
- Split `bluer` into its own target block with
  `cfg(all(target_os = "linux", not(target_env = "musl")))`
  — BlueZ isn't available on musl router targets and
  `libdbus-sys` doesn't cross-compile to musl without pkg-config
  sysroot setup.
- Drop `required-features` from the `fipstop` and `fips-gateway`
  `[[bin]]` entries.

build.rs:
- Emit a `bluer_available` custom cfg when `target_os == "linux"`
  and `target_env != "musl"`, for use in place of the verbose
  full predicate in source cfg gates.

Source:
- Replace every `#[cfg(feature = "gateway")]` with
  `#[cfg(target_os = "linux")]`. Gateway code works on both
  glibc and musl Linux (rustables is fine on musl).
- Replace every `#[cfg(feature = "ble")]` with
  `#[cfg(bluer_available)]`. BLE-specific code (BluerIo module,
  bluer type conversions, BLE transport instance creation,
  resolve_ble_addr) is excluded on musl and non-Linux. Generic
  `BleAddr`, `BleIo` trait, `MockBleIo`, and `BleTransport<I>`
  still compile on all targets.
- `src/bin/fips-gateway.rs`: always compiled, but `main()` is
  gated to Linux. Non-Linux stub exits 1 with a diagnostic.
  Existing non-Linux packaging scripts don't ship it, so the
  stub binary sits unused.

Packaging and CI:
- Drop `--features` and `--no-default-features` flags from every
  packaging script and workflow. Defaults now match each
  platform's capabilities.
- AUR `fips-git` automatically aligns with stable `PKGBUILD`
  (both build with defaults).

Verified: `cargo build --release` with no flags produces all
four binaries on glibc Linux; all unit and integration tests
pass across Linux/macOS/Windows/OpenWrt (musl) in CI.
2026-04-24 13:42:30 -07:00
Johnathan Corgan 3edca4a84f Notify initiator when inbound ACL rejects a Noise XX handshake
Under Noise XX the responder only sees the initiator's static key
in msg3, so the inbound-handshake ACL check cannot fire until msg3
has already been processed. By then the initiator has received
msg2, passed its own outbound ACL check, and promoted its side of
the peering. Silently tearing down the responder's state left the
initiator as a "connected" peer with no traffic until the
link-dead timeout fired tens of seconds later; the acl-allowlist
integration test on the next branch timed out at the 5s
convergence check on blocked nodes.

The responder now sends an encrypted Disconnect on the
freshly-completed Noise session before tearing down. The initiator
decodes it via the existing handle_disconnect path and cleans up
the zombie peer within one RTT. DisconnectReason::Other is used
instead of SecurityViolation so the wire payload does not name the
ACL mechanism; the behavioural signature (explicit reject right
after handshake) is the only observable leak.

Supporting changes:

- Add send_encrypted_link_message_raw in node/mod.rs that operates
  on a raw NoiseSession + indices without going through self.peers,
  since the peer has not yet been promoted at the reject site.
- Declare the acl test module in node/tests/mod.rs. It had been
  orphaned since PR #50 so none of its tests ran; two of them no
  longer compiled against the current next branch. Fixed the two
  and replaced test_inbound_msg1_denied_by_acl (asserts IK-era
  behaviour that cannot happen under XX) with
  test_inbound_msg3_denied_triggers_disconnect, a full end-to-end
  UDP test covering both the responder cleanup and the initiator's
  Disconnect-driven cleanup.
2026-04-22 02:51:17 +00:00
Johnathan Corgan e5e6054229 Merge branch 'master' into next
# Conflicts:
#	src/bloom/filter.rs
#	src/node/bloom.rs
2026-04-21 19:42:35 +00:00
Johnathan Corgan 03f6db58e8 Merge branch 'maint' 2026-04-21 19:33:19 +00:00
Johnathan Corgan db4b32110c Validate bloom filter fill ratio on FilterAnnounce ingress
A malformed FilterAnnounce whose fill ratio produces an implausibly
high false-positive rate is mostly useless for routing and, once
merged into our outgoing filter via bitwise OR, propagates the
saturated state to tree peers one hop per announce tick. A saturated
filter also made estimated_count() return f64::INFINITY, which
compute_mesh_size summed into its cached estimate.

handle_filter_announce now rejects inbound FilterAnnounce whose
derived FPR exceeds `node.bloom.max_inbound_fpr` (new config field,
default 0.05 ≈ fill 0.549 at k=5). Rejection is silent on the wire,
logs at WARN, and increments a new `bloom.fill_exceeded` counter. The
peer's prior stored filter and filter_sequence are left unchanged so
a single rejected announce does not wipe the peer's existing
contribution to aggregation.

After a successful outgoing FilterAnnounce send, a rate-limited WARN
fires if our own filter's FPR exceeds the same cap, surfacing
aggregation drift. Limited to once per 60 seconds via a new
Node.last_self_warn field.

BloomFilter::estimated_count() now takes max_fpr and returns
Option<f64>. Returns None for saturated filters (regardless of cap)
or when the filter's FPR exceeds max_fpr. Callers updated: debug
logs render None as "—", the Debug impl uses f64::INFINITY as "no
cap" and prints "saturated" instead of inf, control-socket JSON
emits null, and compute_mesh_size propagates None into the already-
Option<u64> estimated_mesh_size field.
2026-04-21 19:28:23 +00:00
Johnathan Corgan 14d2a4f2df Merge branch 'master' into next
Integrates PR #50 (peer ACL enforcement) into the XX handshake
architecture. ACL enforcement points adapted for XX's deferred
identity learning: InboundHandshake check moves from handle_msg1
to handle_msg3 (responder learns initiator identity), OutboundHandshake
check remains in handle_msg2 (initiator learns responder identity).
Borrow scopes restructured to release connection borrows before
authorize_peer calls.
2026-04-16 06:11:03 +00:00
Alex XieandJohnathan Corgan 745b523ac6 Add peer ACL enforcement with reloadable allow/deny files (#50)
Implement TCP Wrappers-style peer access control using
/etc/fips/peers.allow and /etc/fips/peers.deny files. Evaluation
order: allow overrides deny, default permit when no files exist.

Three enforcement points: outbound connect (before dialing), inbound
handshake (msg1 receipt, after restart/rekey classification), and
outbound handshake completion (msg2, before peer promotion). Files
support npub, hex pubkey, host alias, and ALL wildcard entries with
automatic mtime-based reload.

Adds fipsctl acl show query, 954-line acl module with unit tests,
and a 6-node Docker integration harness (testing/acl-allowlist/)
exercising insider, outsider, and allowed-remote scenarios. CI
matrix entry included.

Closes #50

Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
2026-04-16 05:13:07 +00:00
Johnathan Corgan 76a7147b17 Merge branch 'master' into next 2026-04-14 09:34:05 +00:00
Johnathan CorganandGitHub 5abae0859e Add historical node and per-peer statistics with btop-style graphs (#64)
In-memory time-series history on the daemon: fast ring (1s × 3600)
plus slow ring (1m × 1440) per metric, covering node-level gauges
(mesh size, tree depth, peer count, active sessions), counters
(parent switches, aggregate bytes/packets in/out), loss rate, and
seven per-peer metrics keyed by NodeAddr (srtt_ms, loss_rate,
bytes_in/out, packets_in/out, ecn_ce). The slow ring is produced by
downsampling the fast ring on minute boundaries with Last / Sum /
Mean aggregation chosen per metric type.

Missing data is first-class. New peers back-fill NaN so every ring
shares a time axis with the node rings; peers absent from a tick
sample NaN (keeps alignment, shows as a visible gap); counter metrics
emit NaN on decrease (new link_stats baseline after reconnect) so
deltas aren't polluted. Peers are evicted 24h after last contact.
Downsampling is NaN-aware: mean skips NaN, all-NaN slow windows stay
NaN. Each history window always returns its full span at the chosen
density (1m / 10m / 1h / 24h), front-padded with NaN when the ring
hasn't yet accumulated enough samples, so switching between windows
feels like zooming in or out rather than clipping to whatever has
arrived. NaN serializes to JSON null via a custom serializer.

Control socket queries:
- show_stats_list enumerates registered metrics plus scope field and
  peer_retention_seconds.
- show_stats_history returns one metric's series for a given window
  and granularity; accepts optional peer (npub) for per-peer metrics.
- show_stats_all_history returns every metric in a single round trip;
  accepts optional peer to fetch all seven per-peer metrics.
- show_stats_peers enumerates tracked peers with lifecycle metadata.
- show_stats_history_all_peers returns one metric across all peers
  for grid rendering.
- show_status carries short sparkline windows for the dashboard so
  the client can render without extra fetches.

fipsctl gains `stats list`, `stats peers`, and `stats history
<metric>` with `--peer` (hostname or npub) and `--plot` for a Unicode
block sparkline. Plot header reports sample count, granularity,
window, and gap count; NaN renders as a blank cell.

fipstop dashboard grows inline sparklines (peer count, mesh size,
aggregate bytes in/out). A new Graphs tab stacks every metric as an
independent mini plot with its own autoscaled range; each plot uses
btop's braille 2×4 filled-area algorithm (25-entry lookup table
packing two samples per character, per-row gradient coloring for the
characteristic btop vertical-band look, rounded borders with embedded
titles). Three modes are cycled with `m`: Node (node-level stack),
MetricByPeer (small-multiples grid, 1 / 2 / 3 columns by terminal
width), PeerByMetric (existing stack scoped to one peer). `n` / `N`
cycles the mode-specific selector (metric or peer), a selector row
shows the current choice, and Graphs-tab refreshes re-fetch
show_stats_peers so selectors track peer churn. Up / Down scrolls
the stack, Left / Right cycles the window, `g` jumps to the tab.

Implements IDEA-0084 (TASK-2026-0062).
2026-04-14 10:24:16 +01:00
Tim O'SheaandJohnathan Corgan 2d342a4e47 Add diagnostic queries for security validation and mesh debugging
Extend the fipsctl control query interface with visibility into internal
state critical for protocol security auditing, mesh troubleshooting, and
operational monitoring.

New command:

  fipsctl show identity-cache

    Lists every node identity cached by the daemon (learned from DNS
    resolution, peer handshakes, sessions, and static config).  Shows
    npub, IPv6 address, display name, and LRU age alongside the
    configured cache capacity.

Extended queries:

  show peers — Noise session counters (send_counter, highest received
    counter) for rekey urgency assessment.  Per-peer replay suppression
    and consecutive decrypt failure counts for active attack detection.
    Session index visibility for hijack analysis.  Rekey lifecycle
    state (in_progress, draining, K-bit epoch).

  show sessions — Handshake resend count during establishment for
    connectivity debugging.  Rekey and session health fields
    (session_start, K-bit, coords warmup, drain state) when
    established.

  show cache — Individual coordinate cache entries with tree
    coordinates, depth, path MTU, and age.  Enables route-level
    debugging by showing exactly which destinations have cached
    routes and via what tree path.  Renames the top-level count
    field from "entries" to "count" for clarity.

  show routing — Pending discovery lookups expanded from count to
    per-target detail (attempt number, age, last sent).  Pending
    TUN packet queue depth for backpressure visibility.  Connection
    retry state per peer (retry count, next attempt, auto-reconnect
    flag).

Updates fipstop to match the revised show_cache and show_routing
response schemas.  Updates README monitoring section with the complete
fipsctl command list.

Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
2026-04-13 17:34:01 +00:00
Johnathan Corgan 9f539ce3c5 Apply cargo fmt to node module and handshake tests
Exposed by the rustfmt toolchain-pin fix now that rustfmt runs under
the pinned 1.94.1 toolchain in CI. Mechanical formatting changes:

- src/node/mod.rs: collapse EthernetTransport::new let-binding onto
  one line (fits within width limit)
- src/node/tests/handshake.rs: reorder imports (super::* after
  specific import) and expand two assert_eq! calls to multi-line form
2026-04-12 09:14:27 +00:00
Johnathan Corgan fe6f9aaa78 Merge branch 'master' into next
# Conflicts:
#	src/node/mod.rs
2026-04-11 17:51:24 +00:00