Add dual-approach MTU handling to prevent TCP connections from hanging
when packets exceed the transport MTU after FIPS encapsulation.
ICMPv6 Packet Too Big:
- Generate RFC 4443 PTB messages for oversized packets at TUN outbound
- Inject back via TUN for local delivery to the application
- Per-source rate limiting (100ms interval, 10s entry expiry)
- MTU check in handle_tun_outbound before session encapsulation
TCP MSS Clamping:
- Intercept SYN packets in run_tun_reader() (outbound)
- Intercept SYN-ACK packets in TunWriter (inbound)
- Clamp MSS option to fit within effective MTU (transport - 127 overhead)
- Recalculate TCP checksum after modification
Code organization:
- ICMP, TCP MSS, and rate limiter modules in upper/ alongside existing
protocol-specific packet handling (dns.rs, tun.rs)
- Shared FIPS_OVERHEAD constant (127 bytes) and effective_ipv6_mtu()
function in upper/icmp.rs
- Node::effective_ipv6_mtu() delegates to the shared function
- run_tun_reader() accepts actual transport MTU from config
Example config corrections:
- UDP transport MTU set to 1472 across all configs (correct max UDP
payload for standard Ethernet: 1500 - 20 IPv4 - 8 UDP)
- Startup logging of effective MTU and max MSS values
Sessions in the Established state that have no activity for 90 seconds
are now automatically removed. This ensures idle sessions are torn down
before transit node coord_cache entries expire (300s TTL), so that when
traffic resumes a fresh SessionSetup re-warms transit node caches with
current coordinates.
The identity cache now stores registration timestamps and expires
entries after 60 seconds via lazy expiry on lookup. This prevents
unbounded growth while allowing natural repopulation through DNS
resolution on next use.
Timer ordering: identity (60s) < session (90s) < coord_cache (300s).
Both timeouts are configurable: node.session.idle_timeout_secs and
node.cache.identity_ttl_secs. Setting idle_timeout_secs to 0 disables
session idle purging.
Changes:
- Add idle_timeout_secs (default 90) to SessionConfig
- Add identity_ttl_secs (default 60) to CacheConfig
- Add timestamp to identity_cache entries, lazy expiry on lookup
- Add purge_idle_sessions() called from tick loop
- Remove #[cfg(test)] from SessionEntry::last_activity()
- 7 new tests covering timeout behavior and edge cases
Transit nodes cache destination coordinates when they forward
SessionSetup messages (via try_warm_coord_cache). These coord_cache
entries have a 5-minute TTL, after which they expire. Once expired,
the transit node can no longer forward data packets for that
destination — find_next_hop returns None and the node sends
CoordsRequired errors back to the source. This creates a permanent
routing failure for any multi-hop path after 5 minutes of the initial
session establishment, even if traffic is actively flowing.
The root cause was that find_next_hop used coord_cache.get(), a
read-only lookup that checks expiry but never extends it. Active
forwarding did not keep the cache warm. Meanwhile, get_and_touch()
existed but only updated last_used without extending expires_at.
Fix:
- find_next_hop now calls coord_cache.get_and_touch() instead of get()
- get_and_touch now calls entry.refresh() instead of entry.touch(),
which extends expires_at by the default TTL on each access
- find_next_hop signature changed from &self to &mut self to allow
the mutable cache access
This ensures that as long as traffic flows through a transit node,
the coord_cache entries stay warm and routing continues to work.
Entries still expire after 5 minutes of inactivity as designed.
FilterAnnounce messages never settled in steady state because
handle_filter_announce unconditionally marked all peers for
re-announcement on every inbound filter, creating a perpetual
ping-pong at ~1 message/sec.
Added last_sent_filters tracking to BloomState. mark_changed_peers()
computes outgoing filters and compares against what was last sent,
only marking peers whose filter content actually differs.
Wire the discovery protocol into the data plane path so that
find_next_hop() consults route_cache as a fallback when coord_cache
has no entry. When session initiation fails due to missing routes,
trigger discovery (initiate_lookup) instead of immediately sending
ICMPv6 Destination Unreachable. On discovery completion, retry
session initiation for any pending TUN packets.
Changes:
- find_next_hop() falls back to route_cache when coord_cache misses
- handle_tun_outbound() triggers discovery on session failure
- handle_lookup_response() retries session after discovery completes
- handle_coords_required() triggers discovery for missing coordinates
- Add pending_lookups deduplication with 10-second timeout
- Periodic cleanup of stale lookups in tick handler
- New test: route_cache fallback verification in find_next_hop
Add DNS responder that resolves <npub>.fips queries to FipsAddress IPv6
addresses. Resolution is pure computation (npub → NodeAddr → IPv6) with
identity cache priming as a side effect, enabling subsequent TUN packet
routing to non-peer destinations.
- New dns.rs module: resolve_fips_query(), handle_dns_packet() using
simple-dns crate, run_dns_responder() async UDP server loop
- DnsConfig in config.rs: enabled, bind_addr (127.0.0.1), port (5354)
- Fourth select! arm in RX event loop for DNS identity channel
- DNS task spawn/abort in node lifecycle
- 10 new tests (420 total)
Add examples/two-node-udp/ with standalone walkthrough for testing two
FIPS nodes in Linux network namespaces over a veth pair. Includes
network diagram, config files, DNS routing setup, and troubleshooting.
Wire the TUN reader to the session layer for end-to-end IPv6 packet
delivery through the mesh. TUN reader forwards FIPS-destined packets
(fd::/8) to Node via tokio::sync::mpsc channel. Identity cache maps
FipsAddress prefix bytes to NodeAddr + PublicKey for reverse address
resolution, populated at peer promotion and session creation. Outbound
handler validates IPv6, looks up destination identity, routes through
established session or initiates new one with pending packet queue
(16/dest, 256 dests max). Queue flushed on session establishment.
ICMPv6 Destination Unreachable for unknown destinations. 6 new tests
including 3-node forwarded TUN data and pending queue flush. 410 tests
pass, clean build.
Implement Noise IK session handshake between arbitrary endpoints, carried
inside SessionDatagram envelopes through the mesh. Sessions use a three-state
machine (Initiating → Established for initiator, Responding → Established
for responder on first DataPacket). Includes session initiation API,
encrypted data transfer, simultaneous initiation tie-break, error signal
handlers (CoordsRequired, PathBroken), and local delivery wiring in the
forwarding handler.
New files: node/session.rs (state types), handlers/session.rs (~500 lines,
all session message handlers + send path), tests/session.rs (11 tests).
100-node integration test establishes sessions across random topology,
sends bidirectional encrypted datagrams through injected TUN channels,
and verifies 200/200 deliveries with 100% session establishment. Reports
routing statistics including avg 4.1 link hops per datagram.
404 tests pass (up from 393).
Add handle_session_datagram handler replacing the 0x40 dispatch stub.
Transit nodes now decode the datagram envelope, enforce hop limits,
warm coordinate caches from SessionSetup/SessionAck/DataPacket payloads,
route via find_next_hop, and generate CoordsRequired/PathBroken error
signals on routing failure.
14 new tests covering decode errors, hop limit enforcement, local
delivery, cache warming for all session message types, single-hop and
multi-hop forwarding through live node chains, error signal generation,
and cache warming enabling subsequent routing. 375 tests pass.
Add src_addr to SessionDatagram envelope (34-byte header: msg_type +
src_addr + dest_addr + hop_limit) so transit routers can route error
signals back to the packet's originator.
Reclassify CoordsRequired/PathBroken as link-layer error signals
(plaintext inside SessionDatagram) rather than e2e encrypted session
messages. Transit routers generate these when forwarding fails and
route them to src_addr; if source is also unreachable, drop silently.
Remove redundant src_addr/dest_addr/hop_limit from SessionSetup,
SessionAck, and DataPacket (now in envelope). DataPacket header
reduced from 36 to 4 bytes. Remove PathBroken.original_src.
Fix routing loop vulnerability: gate bloom filter path on having
cached dest_coords to prevent blind forwarding between peers.
Simplify select_best_candidate() to require coordinates.
Fix gossip protocol type codes (0x11->0x20, 0x12->0x30, 0x13->0x31)
for consistency across all design docs.
All 5 design docs updated and cross-checked for consistency.
335 tests pass, zero warnings.
Add the full next-hop routing algorithm to Node::find_next_hop():
- Local delivery, direct peer, bloom filter candidates, greedy tree
routing fallback, with (link_cost, tree_distance, node_addr) ordering
- select_best_candidate() scores by peer→dest distance (not us→peer)
with self-distance check to prevent routing loops
- TreeState::find_next_hop() for greedy tree routing with progress
guarantee
- ActivePeer::link_cost() placeholder (constant 1.0) for future link
quality metrics
Add routing tests including 100-node all-pairs reachability simulation
(9900/9900 delivered, 0 loops, avg 4.0 hops, max 8).
Update fips-routing.md to reflect bloom filter routing as the primary
forwarding mechanism, with greedy tree routing as fallback during
convergence windows.
Split handlers.rs (986 lines) into handlers/ with 5 subfiles organized
by responsibility: rx_loop, encrypted, handshake, dispatch, timeout.
Split tests.rs (2350 lines) into tests/ with 4 subfiles: unit tests,
handshake integration, spanning tree convergence, and bloom filter tests.
Shared test helpers extracted to tests/mod.rs.
Visibility adjusted from pub(super) to pub(in crate::node) for handler
methods now two levels deep. Unused imports cleaned up in node/mod.rs.
All 316 tests pass, zero warnings.
When both nodes simultaneously initiate connections, each promotes its
inbound handshake first, leaving both sides with mismatched Noise sessions
and session indices. Resolve this in handle_msg2 using the existing
tie-breaker (smaller node_addr's outbound wins):
- Winner (smaller node): swaps ActivePeer to outbound session + indices
via new replace_session() method
- Loser (larger node): keeps inbound session and original their_index
(already references winner's outbound our_index from msg1)
Key insight: the loser must NOT update their_index from msg2, because the
msg2 sender_idx carries the winner's inbound index which becomes stale
after the winner swaps.
Defer outbound connection cleanup from promote_connection() to handle_msg2()
so outbound session state is available for the swap.
Verified with live two-node test over UDP in network namespaces: both nodes
connect, exchange TreeAnnounce, and converge to spanning tree within ~2s.
Multi-node integration test infrastructure with 100-node random topology
convergence test (5 topology variants). Two-phase drain loop processes
Noise IK handshakes and TreeAnnounce cascades over live UDP until
quiescence.
Fix parent ancestry propagation in handle_tree_announce: when a node's
current parent updates its tree state (e.g., learning a new root from
upstream), coordinates must be recomputed even though the parent itself
didn't change. Without this, chains longer than 2 hops fail to converge.
Fix promote_connection() to detect and clean up pending outbound
handshakes to the same peer, not just already-promoted peers. Previously,
when an inbound handshake completed while an outbound was still pending,
the outbound would linger until the 30s timeout.
Add auto-retry for failed outbound connections to auto-connect peers:
- New RetryState struct and node/retry.rs module
- Exponential backoff (default 5s base, max 5 attempts)
- Config: node.max_retries, node.base_retry_interval_secs
- check_timeouts() schedules retries, rx loop processes them
- promote_connection() clears retry state on success
- Remove unused PeerConnection retry fields (state now at Node level)
Move cleanup_stale_connection() logging to callers for context-appropriate
messages.
289 tests pass, clean build.
Noise IK parity fix:
- Pre-message hash normalizes responder static key to even parity (0x02)
so initiator and responder hash chains match regardless of actual parity
- ECDH uses shared_secret_point() + SHA-256(x-only) instead of
SharedSecret::new() which includes a parity-dependent version byte
- Fixes handshake failure for ~50% of keys when initiator has only npub
Graceful disconnect protocol (link message 0x50):
- DisconnectReason enum with 8 reason codes
- Disconnect struct with encode/decode
- send_encrypted_link_message() reusable helper
- handle_disconnect() with immediate peer removal
- send_disconnect_to_all_peers() called during Node::stop()
Cross-connection fix in handle_msg1():
- addr_to_link check now distinguishes inbound duplicates (reject) from
outbound links (cross-connection, allow and resolve via tie-breaker)
- remove_link() only clears addr_to_link if entry maps to same link_id
- Link cleanup and addr_to_link restoration in cross-connection branches
Handshake timeout cleanup:
- RX loop uses tokio::select! with 1-second interval tick
- check_timeouts() scans for stale (>30s) and failed connections
- cleanup_stale_connection() removes all associated state
Tests: 279 passing (4 new: cross-connection, stale cleanup, failed
cleanup, odd-parity handshake)
Split 2,597-line node.rs into four files under src/node/:
- mod.rs: types, struct definition, constructors, accessors, Debug impl
- lifecycle.rs: start(), stop(), peer connection initiation
- handlers.rs: RX loop, packet dispatch, message handlers, promote_connection
- tests.rs: full test suite (35 tests including integration tests)
No functional changes. All 267 tests pass, no new clippy warnings.
Origami74andJohnathan Corgan