Guard discovery triggers in handle_path_broken() and
handle_coords_required() with has_cached_identity() check. When the
XK responder receives an error signal before msg3 completes, the
initiator's identity is unknown, making LookupResponse proof
verification impossible. Skip discovery in this case — the handshake
retry mechanism handles recovery.
Downgrade the identity_cache miss log from ERROR to WARN since it's a
known race condition, not a bug.
Cost-based parent selection:
- Replace depth-only parent selection with effective_depth = depth + link_cost
- link_cost computed from locally measured MMP metrics: etx * (1.0 + srtt_ms / 100.0)
- Prevents bottleneck subtrees in heterogeneous networks where a LoRa link
at depth 1 would otherwise always beat fiber at depth 2
- Configurable hysteresis (default 0.2) prevents marginal parent switches
- Configurable hold-down timer (default 30s) suppresses re-evaluation
after parent switch
- Mandatory switches (parent lost, root change) bypass both safeguards
- Link costs passed as HashMap parameter to keep TreeState pure
Periodic re-evaluation:
- evaluate_parent() was only called on TreeAnnounce receipt or parent loss;
after tree stabilization, link degradation went undetected
- Added timer-based re-evaluation (reeval_interval_secs, default 60s) that
calls evaluate_parent() from the tick handler with current MMP link costs
- Respects existing hold-down and hysteresis safeguards
- Short-circuits when disabled or <2 peers
Design documentation:
- Update 7 design docs to reflect cost-based parent selection
- Replace depth-only algorithm descriptions with effective_depth model
- Replace rejected cumulative path cost spec with local-only design rationale
- Rewrite Example 2 (heterogeneous links) for local-only cost model
- Update config docs: parent_switch_threshold replaced by parent_hysteresis,
hold_down_secs, reeval_interval_secs
Chaos simulation enhancements:
- fips_overrides with deep merge for per-scenario FIPS config customization
- Explicit topology algorithm for deterministic test graphs
- Control socket querying via fipsctl for tree/MMP snapshot collection
- Edge existence validation in netem manager
- Per-link netem policy overrides
- 9 new chaos scenarios covering cost avoidance, depth-vs-cost tradeoffs,
stability, mixed topologies, periodic re-evaluation, and bottleneck parent
12 new unit tests, 667 total passing, clippy clean.
Gate peer_inbound_filters() to only collect from tree peers (parent
and children), so outgoing filter computation merges only tree-sourced
information. All peers still receive FilterAnnounce messages and store
filters locally for routing queries — the restriction is only on what
gets merged into outgoing filters.
This prevents bloom filter saturation where mesh shortcuts cause every
node's filter to converge toward the full network. With tree-only
merge, filters contain subtree (from children) + complement (from
parent) + single-hop mesh views.
Implementation:
- Add is_tree_peer() helper to determine tree parent/child relationship
- Gate peer_inbound_filters() to tree peers only (single control point)
- Trigger bloom filter exchange on tree relationship changes
- Add est_entries, set_bits, fill ratio, and tree_peer fields to
FilterAnnounce send/receive debug logs
- Add test_bloom_filter_split_horizon test verifying directional
asymmetry: upward filters contain only the child's subtree, downward
filters contain only the complement
- Add print_filter_cardinality diagnostic helper for test inspection
Design docs:
- fips-bloom-filters.md: Add directional asymmetry and mesh peer filter
subsections, update per-peer filter model, saturation mitigation,
implementation status table
- fips-mesh-operation.md: Update filter propagation description, add
directional asymmetry, tree relationship change trigger
- fips-intro.md: Rewrite bloom propagation paragraph for tree-only merge
Add a new session-layer error signal that transit routers send back to
the source when a forwarded packet exceeds the next-hop transport MTU.
This complements the existing proactive path MTU discovery (min'd at
each hop) by providing immediate feedback when oversized packets are
dropped, closing the transient window before the proactive mechanism
converges.
Wire format: 36-byte payload (msg_type + flags + dest_addr + reporter +
mtu) with FSP phase=0x0 and U flag set, matching the existing
CoordsRequired/PathBroken pattern.
Changes:
- Add SessionMessageType::MtuExceeded (0x22) and MtuExceeded struct with
encode/decode methods to protocol/session.rs
- Add NodeError::MtuExceeded variant to propagate structured MTU info
from TransportError through send_encrypted_link_message()
- Catch MtuExceeded in the forwarding path and send error signal back to
the datagram source via send_mtu_exceeded_error(), rate-limited by the
existing routing_error_rate_limiter
- Handle incoming MtuExceeded at the source by calling
PathMtuState::apply_notification() for immediate MTU decrease
- Add unit tests for encode/decode roundtrip, boundary MTU values, and
too-short payload rejection
Add a Unix domain socket interface for querying node state at runtime.
A spawned tokio task accepts connections and communicates with the main
event loop via mpsc/oneshot channels, keeping all Node access
single-threaded.
Includes:
- src/control/ module with socket lifecycle, JSON protocol, and 11
query handlers (status, peers, links, tree, sessions, bloom, mmp,
cache, connections, transports, routing)
- Separate fipsctl binary for CLI queries (fipsctl show <command>)
- ControlConfig in node configuration (enabled, socket_path)
- Integration into the main select! event loop
Each node generates a random 8-byte startup epoch, encrypted inside
both Noise IK handshake messages (msg1 and msg2). When a peer's msg1
arrives with a different epoch than the stored value, the node tears
down the stale session and processes the msg1 as a new connection,
enabling near-instant restart detection instead of the 30-second
dead timeout.
Wire format impact:
- msg1: 82 -> 106 bytes (added 24-byte encrypted epoch after ss DH)
- msg2: 33 -> 57 bytes (added 24-byte encrypted epoch after se DH)
- Wire msg1: 90 -> 114 bytes, wire msg2: 45 -> 69 bytes
Implement hybrid coordinate cache warming strategy: piggyback coords
via CP flag when they fit within transport MTU, send standalone
CoordsWarmup (0x14) message when they don't. On CoordsRequired or
PathBroken receipt, send CoordsWarmup immediately with source-side
rate limiting (default 2s per destination, configurable).
- Add CoordsWarmup = 0x14 session message type (empty body, CP flag)
- Add send_coords_warmup() following send_session_msg() pattern
- Restructure send_session_data() to send standalone warmup before
data packet when piggybacked coords exceed MTU
- Add immediate CoordsWarmup response in handle_coords_required()
and handle_path_broken() with per-destination rate limiting
- Add coords_response_interval_ms config (node.session)
- Add RoutingErrorRateLimiter::with_interval() constructor
- Zero transit-path changes: existing try_warm_coord_cache() handles
CoordsWarmup identically to CP-flagged data packets
- Update design docs (session layer, wire formats, mesh operation,
configuration)
Replace raw NodeAddr hex strings in log output with human-readable
identifiers using a four-tier lookup: configured alias, active peer
short npub, session endpoint short npub, or truncated hex fallback.
- Add PeerIdentity::short_npub() for compact npub display (npub1xxxx...yyyy)
- Add NodeAddr::short_hex() for compact hex fallback (first 4 bytes + ...)
- Add peer_aliases map populated at startup from peer config
- Add Node::peer_display_name() with four-tier resolution
- Add SessionEntry::remote_pubkey() accessor for session-layer lookups
- Update all 120 log field occurrences across 11 handler/node files
- Pass pre-computed display names to MMP static metric/teardown methods
to work around borrow checker constraints in iterator loops
- PathBroken handler: convert to async, trigger re-discovery via
maybe_initiate_lookup(), reset COORDS_PRESENT warmup counter
(was a stub that only invalidated coord_cache)
- CoordsRequired recovery timing: reset warmup counter in
handle_lookup_response() when discovery completes for an
established session, so COORDS_PRESENT packets fire after
fresh coords are available (not just on CoordsRequired receipt)
- Routing error rate limiting: add RoutingErrorRateLimiter
(100ms per-destination, matching ICMP PTB pattern) to gate
send_routing_error() at transit nodes
- Remove root refresh dead code: the 1800s periodic root
re-announcement in check_tree_state() only propagated to
depth 1 (sequence-only changes don't cascade). Root loss
detection relies on link failure propagation which works
correctly.
Identity cache: remove TTL-based expiry (60s TTL broke active sessions
after expiry since handle_tun_outbound checks identity_cache before
session table). Replace with LRU-only eviction bounded by configurable
identity_size (default 10K). Lookup now touches timestamp for LRU
freshness.
Cache merge: unify coord_cache and route_cache into single coordinate
cache. Both stored NodeAddr→TreeCoordinate; the layer distinction was
conceptual, not functional. Discovery-sourced entries now get the same
TTL+refresh treatment as session-sourced entries. Simplifies
find_next_hop() to single cache lookup.
Parent-change flush: clear coord_cache after recompute_coords() in both
parent-switch paths of handle_tree_announce(). Stale coordinates after
tree reconvergence cause dead-end routing that's more expensive than
re-discovery.
Tested: 493 unit tests passed, clippy clean, Docker mesh 20/20,
Docker chain 6/6.
Add dual-approach MTU handling to prevent TCP connections from hanging
when packets exceed the transport MTU after FIPS encapsulation.
ICMPv6 Packet Too Big:
- Generate RFC 4443 PTB messages for oversized packets at TUN outbound
- Inject back via TUN for local delivery to the application
- Per-source rate limiting (100ms interval, 10s entry expiry)
- MTU check in handle_tun_outbound before session encapsulation
TCP MSS Clamping:
- Intercept SYN packets in run_tun_reader() (outbound)
- Intercept SYN-ACK packets in TunWriter (inbound)
- Clamp MSS option to fit within effective MTU (transport - 127 overhead)
- Recalculate TCP checksum after modification
Code organization:
- ICMP, TCP MSS, and rate limiter modules in upper/ alongside existing
protocol-specific packet handling (dns.rs, tun.rs)
- Shared FIPS_OVERHEAD constant (127 bytes) and effective_ipv6_mtu()
function in upper/icmp.rs
- Node::effective_ipv6_mtu() delegates to the shared function
- run_tun_reader() accepts actual transport MTU from config
Example config corrections:
- UDP transport MTU set to 1472 across all configs (correct max UDP
payload for standard Ethernet: 1500 - 20 IPv4 - 8 UDP)
- Startup logging of effective MTU and max MSS values
Sessions in the Established state that have no activity for 90 seconds
are now automatically removed. This ensures idle sessions are torn down
before transit node coord_cache entries expire (300s TTL), so that when
traffic resumes a fresh SessionSetup re-warms transit node caches with
current coordinates.
The identity cache now stores registration timestamps and expires
entries after 60 seconds via lazy expiry on lookup. This prevents
unbounded growth while allowing natural repopulation through DNS
resolution on next use.
Timer ordering: identity (60s) < session (90s) < coord_cache (300s).
Both timeouts are configurable: node.session.idle_timeout_secs and
node.cache.identity_ttl_secs. Setting idle_timeout_secs to 0 disables
session idle purging.
Changes:
- Add idle_timeout_secs (default 90) to SessionConfig
- Add identity_ttl_secs (default 60) to CacheConfig
- Add timestamp to identity_cache entries, lazy expiry on lookup
- Add purge_idle_sessions() called from tick loop
- Remove #[cfg(test)] from SessionEntry::last_activity()
- 7 new tests covering timeout behavior and edge cases
Transit nodes cache destination coordinates when they forward
SessionSetup messages (via try_warm_coord_cache). These coord_cache
entries have a 5-minute TTL, after which they expire. Once expired,
the transit node can no longer forward data packets for that
destination — find_next_hop returns None and the node sends
CoordsRequired errors back to the source. This creates a permanent
routing failure for any multi-hop path after 5 minutes of the initial
session establishment, even if traffic is actively flowing.
The root cause was that find_next_hop used coord_cache.get(), a
read-only lookup that checks expiry but never extends it. Active
forwarding did not keep the cache warm. Meanwhile, get_and_touch()
existed but only updated last_used without extending expires_at.
Fix:
- find_next_hop now calls coord_cache.get_and_touch() instead of get()
- get_and_touch now calls entry.refresh() instead of entry.touch(),
which extends expires_at by the default TTL on each access
- find_next_hop signature changed from &self to &mut self to allow
the mutable cache access
This ensures that as long as traffic flows through a transit node,
the coord_cache entries stay warm and routing continues to work.
Entries still expire after 5 minutes of inactivity as designed.
Wire the discovery protocol into the data plane path so that
find_next_hop() consults route_cache as a fallback when coord_cache
has no entry. When session initiation fails due to missing routes,
trigger discovery (initiate_lookup) instead of immediately sending
ICMPv6 Destination Unreachable. On discovery completion, retry
session initiation for any pending TUN packets.
Changes:
- find_next_hop() falls back to route_cache when coord_cache misses
- handle_tun_outbound() triggers discovery on session failure
- handle_lookup_response() retries session after discovery completes
- handle_coords_required() triggers discovery for missing coordinates
- Add pending_lookups deduplication with 10-second timeout
- Periodic cleanup of stale lookups in tick handler
- New test: route_cache fallback verification in find_next_hop
Add DNS responder that resolves <npub>.fips queries to FipsAddress IPv6
addresses. Resolution is pure computation (npub → NodeAddr → IPv6) with
identity cache priming as a side effect, enabling subsequent TUN packet
routing to non-peer destinations.
- New dns.rs module: resolve_fips_query(), handle_dns_packet() using
simple-dns crate, run_dns_responder() async UDP server loop
- DnsConfig in config.rs: enabled, bind_addr (127.0.0.1), port (5354)
- Fourth select! arm in RX event loop for DNS identity channel
- DNS task spawn/abort in node lifecycle
- 10 new tests (420 total)
Add examples/two-node-udp/ with standalone walkthrough for testing two
FIPS nodes in Linux network namespaces over a veth pair. Includes
network diagram, config files, DNS routing setup, and troubleshooting.
Wire the TUN reader to the session layer for end-to-end IPv6 packet
delivery through the mesh. TUN reader forwards FIPS-destined packets
(fd::/8) to Node via tokio::sync::mpsc channel. Identity cache maps
FipsAddress prefix bytes to NodeAddr + PublicKey for reverse address
resolution, populated at peer promotion and session creation. Outbound
handler validates IPv6, looks up destination identity, routes through
established session or initiates new one with pending packet queue
(16/dest, 256 dests max). Queue flushed on session establishment.
ICMPv6 Destination Unreachable for unknown destinations. 6 new tests
including 3-node forwarded TUN data and pending queue flush. 410 tests
pass, clean build.
Implement Noise IK session handshake between arbitrary endpoints, carried
inside SessionDatagram envelopes through the mesh. Sessions use a three-state
machine (Initiating → Established for initiator, Responding → Established
for responder on first DataPacket). Includes session initiation API,
encrypted data transfer, simultaneous initiation tie-break, error signal
handlers (CoordsRequired, PathBroken), and local delivery wiring in the
forwarding handler.
New files: node/session.rs (state types), handlers/session.rs (~500 lines,
all session message handlers + send path), tests/session.rs (11 tests).
100-node integration test establishes sessions across random topology,
sends bidirectional encrypted datagrams through injected TUN channels,
and verifies 200/200 deliveries with 100% session establishment. Reports
routing statistics including avg 4.1 link hops per datagram.
404 tests pass (up from 393).
Add src_addr to SessionDatagram envelope (34-byte header: msg_type +
src_addr + dest_addr + hop_limit) so transit routers can route error
signals back to the packet's originator.
Reclassify CoordsRequired/PathBroken as link-layer error signals
(plaintext inside SessionDatagram) rather than e2e encrypted session
messages. Transit routers generate these when forwarding fails and
route them to src_addr; if source is also unreachable, drop silently.
Remove redundant src_addr/dest_addr/hop_limit from SessionSetup,
SessionAck, and DataPacket (now in envelope). DataPacket header
reduced from 36 to 4 bytes. Remove PathBroken.original_src.
Fix routing loop vulnerability: gate bloom filter path on having
cached dest_coords to prevent blind forwarding between peers.
Simplify select_best_candidate() to require coordinates.
Fix gossip protocol type codes (0x11->0x20, 0x12->0x30, 0x13->0x31)
for consistency across all design docs.
All 5 design docs updated and cross-checked for consistency.
335 tests pass, zero warnings.
Add the full next-hop routing algorithm to Node::find_next_hop():
- Local delivery, direct peer, bloom filter candidates, greedy tree
routing fallback, with (link_cost, tree_distance, node_addr) ordering
- select_best_candidate() scores by peer→dest distance (not us→peer)
with self-distance check to prevent routing loops
- TreeState::find_next_hop() for greedy tree routing with progress
guarantee
- ActivePeer::link_cost() placeholder (constant 1.0) for future link
quality metrics
Add routing tests including 100-node all-pairs reachability simulation
(9900/9900 delivered, 0 loops, avg 4.0 hops, max 8).
Update fips-routing.md to reflect bloom filter routing as the primary
forwarding mechanism, with greedy tree routing as fallback during
convergence windows.
Split handlers.rs (986 lines) into handlers/ with 5 subfiles organized
by responsibility: rx_loop, encrypted, handshake, dispatch, timeout.
Split tests.rs (2350 lines) into tests/ with 4 subfiles: unit tests,
handshake integration, spanning tree convergence, and bloom filter tests.
Shared test helpers extracted to tests/mod.rs.
Visibility adjusted from pub(super) to pub(in crate::node) for handler
methods now two levels deep. Unused imports cleaned up in node/mod.rs.
All 316 tests pass, zero warnings.
Fix promote_connection() to detect and clean up pending outbound
handshakes to the same peer, not just already-promoted peers. Previously,
when an inbound handshake completed while an outbound was still pending,
the outbound would linger until the 30s timeout.
Add auto-retry for failed outbound connections to auto-connect peers:
- New RetryState struct and node/retry.rs module
- Exponential backoff (default 5s base, max 5 attempts)
- Config: node.max_retries, node.base_retry_interval_secs
- check_timeouts() schedules retries, rx loop processes them
- promote_connection() clears retry state on success
- Remove unused PeerConnection retry fields (state now at Node level)
Move cleanup_stale_connection() logging to callers for context-appropriate
messages.
289 tests pass, clean build.
Noise IK parity fix:
- Pre-message hash normalizes responder static key to even parity (0x02)
so initiator and responder hash chains match regardless of actual parity
- ECDH uses shared_secret_point() + SHA-256(x-only) instead of
SharedSecret::new() which includes a parity-dependent version byte
- Fixes handshake failure for ~50% of keys when initiator has only npub
Graceful disconnect protocol (link message 0x50):
- DisconnectReason enum with 8 reason codes
- Disconnect struct with encode/decode
- send_encrypted_link_message() reusable helper
- handle_disconnect() with immediate peer removal
- send_disconnect_to_all_peers() called during Node::stop()
Cross-connection fix in handle_msg1():
- addr_to_link check now distinguishes inbound duplicates (reject) from
outbound links (cross-connection, allow and resolve via tie-breaker)
- remove_link() only clears addr_to_link if entry maps to same link_id
- Link cleanup and addr_to_link restoration in cross-connection branches
Handshake timeout cleanup:
- RX loop uses tokio::select! with 1-second interval tick
- check_timeouts() scans for stale (>30s) and failed connections
- cleanup_stale_connection() removes all associated state
Tests: 279 passing (4 new: cross-connection, stale cleanup, failed
cleanup, odd-parity handshake)
Split 2,597-line node.rs into four files under src/node/:
- mod.rs: types, struct definition, constructors, accessors, Debug impl
- lifecycle.rs: start(), stop(), peer connection initiation
- handlers.rs: RX loop, packet dispatch, message handlers, promote_connection
- tests.rs: full test suite (35 tests including integration tests)
No functional changes. All 267 tests pass, no new clippy warnings.
Johnathan Corgan
Origami74andJohnathan Corgan