Sessions in the Established state that have no activity for 90 seconds
are now automatically removed. This ensures idle sessions are torn down
before transit node coord_cache entries expire (300s TTL), so that when
traffic resumes a fresh SessionSetup re-warms transit node caches with
current coordinates.
The identity cache now stores registration timestamps and expires
entries after 60 seconds via lazy expiry on lookup. This prevents
unbounded growth while allowing natural repopulation through DNS
resolution on next use.
Timer ordering: identity (60s) < session (90s) < coord_cache (300s).
Both timeouts are configurable: node.session.idle_timeout_secs and
node.cache.identity_ttl_secs. Setting idle_timeout_secs to 0 disables
session idle purging.
Changes:
- Add idle_timeout_secs (default 90) to SessionConfig
- Add identity_ttl_secs (default 60) to CacheConfig
- Add timestamp to identity_cache entries, lazy expiry on lookup
- Add purge_idle_sessions() called from tick loop
- Remove #[cfg(test)] from SessionEntry::last_activity()
- 7 new tests covering timeout behavior and edge cases
Transit nodes cache destination coordinates when they forward
SessionSetup messages (via try_warm_coord_cache). These coord_cache
entries have a 5-minute TTL, after which they expire. Once expired,
the transit node can no longer forward data packets for that
destination — find_next_hop returns None and the node sends
CoordsRequired errors back to the source. This creates a permanent
routing failure for any multi-hop path after 5 minutes of the initial
session establishment, even if traffic is actively flowing.
The root cause was that find_next_hop used coord_cache.get(), a
read-only lookup that checks expiry but never extends it. Active
forwarding did not keep the cache warm. Meanwhile, get_and_touch()
existed but only updated last_used without extending expires_at.
Fix:
- find_next_hop now calls coord_cache.get_and_touch() instead of get()
- get_and_touch now calls entry.refresh() instead of entry.touch(),
which extends expires_at by the default TTL on each access
- find_next_hop signature changed from &self to &mut self to allow
the mutable cache access
This ensures that as long as traffic flows through a transit node,
the coord_cache entries stay warm and routing continues to work.
Entries still expire after 5 minutes of inactivity as designed.
FilterAnnounce messages never settled in steady state because
handle_filter_announce unconditionally marked all peers for
re-announcement on every inbound filter, creating a perpetual
ping-pong at ~1 message/sec.
Added last_sent_filters tracking to BloomState. mark_changed_peers()
computes outgoing filters and compares against what was last sent,
only marking peers whose filter content actually differs.
Wire the discovery protocol into the data plane path so that
find_next_hop() consults route_cache as a fallback when coord_cache
has no entry. When session initiation fails due to missing routes,
trigger discovery (initiate_lookup) instead of immediately sending
ICMPv6 Destination Unreachable. On discovery completion, retry
session initiation for any pending TUN packets.
Changes:
- find_next_hop() falls back to route_cache when coord_cache misses
- handle_tun_outbound() triggers discovery on session failure
- handle_lookup_response() retries session after discovery completes
- handle_coords_required() triggers discovery for missing coordinates
- Add pending_lookups deduplication with 10-second timeout
- Periodic cleanup of stale lookups in tick handler
- New test: route_cache fallback verification in find_next_hop
Replace ASCII art diagram in examples/two-node-udp/README.md with
two-node-udp.svg showing namespaces, veth pair, TUN devices, DNS
responders, and transport/session layers.
Add fips-identity-derivation.svg showing the derivation chain from
pubkey to npub, node_addr, and IPv6 address with protocol roles and
privacy boundary. Replaces the code block in Node Address Derivation
with the diagram and consolidated descriptive text.
- fips-architecture.md → fips-software-architecture.md with all
cross-references updated (4 files)
- fips-transport-abstraction.svg → fips-node-architecture.svg, moved
from Transport Abstraction section to Architecture Overview in
fips-intro.md
- Added descriptive paragraph for node architecture diagram covering
three-layer design (application interfaces, router core, transports)
Replace three ASCII art diagrams in fips-intro.md with SVG images:
- Architecture overview: 5-node 4-hop path with layered node boxes
- Mesh topology: 8-node network with spanning tree highlighted
- Transport abstraction: full node stack with 10 transport plugins
Add fd00::/8 route to TUN configure_interface() via rtnetlink - without
this route the kernel had no path to FIPS addresses ("network unreachable").
Update two-node-udp README based on live namespace testing:
- Remove resolvectl steps (doesn't work inside namespaces)
- Remove ICMPv6 Echo Reply mention (kernel handles it natively)
- Add namespace DNS limitation note and kernel ping reply explanation
- Simplify from 8 to 7 steps
Add DNS responder that resolves <npub>.fips queries to FipsAddress IPv6
addresses. Resolution is pure computation (npub → NodeAddr → IPv6) with
identity cache priming as a side effect, enabling subsequent TUN packet
routing to non-peer destinations.
- New dns.rs module: resolve_fips_query(), handle_dns_packet() using
simple-dns crate, run_dns_responder() async UDP server loop
- DnsConfig in config.rs: enabled, bind_addr (127.0.0.1), port (5354)
- Fourth select! arm in RX event loop for DNS identity channel
- DNS task spawn/abort in node lifecycle
- 10 new tests (420 total)
Add examples/two-node-udp/ with standalone walkthrough for testing two
FIPS nodes in Linux network namespaces over a veth pair. Includes
network diagram, config files, DNS routing setup, and troubleshooting.
Wire the TUN reader to the session layer for end-to-end IPv6 packet
delivery through the mesh. TUN reader forwards FIPS-destined packets
(fd::/8) to Node via tokio::sync::mpsc channel. Identity cache maps
FipsAddress prefix bytes to NodeAddr + PublicKey for reverse address
resolution, populated at peer promotion and session creation. Outbound
handler validates IPv6, looks up destination identity, routes through
established session or initiates new one with pending packet queue
(16/dest, 256 dests max). Queue flushed on session establishment.
ICMPv6 Destination Unreachable for unknown destinations. 6 new tests
including 3-node forwarded TUN data and pending queue flush. 410 tests
pass, clean build.
Implement Noise IK session handshake between arbitrary endpoints, carried
inside SessionDatagram envelopes through the mesh. Sessions use a three-state
machine (Initiating → Established for initiator, Responding → Established
for responder on first DataPacket). Includes session initiation API,
encrypted data transfer, simultaneous initiation tie-break, error signal
handlers (CoordsRequired, PathBroken), and local delivery wiring in the
forwarding handler.
New files: node/session.rs (state types), handlers/session.rs (~500 lines,
all session message handlers + send path), tests/session.rs (11 tests).
100-node integration test establishes sessions across random topology,
sends bidirectional encrypted datagrams through injected TUN channels,
and verifies 200/200 deliveries with 100% session establishment. Reports
routing statistics including avg 4.1 link hops per datagram.
404 tests pass (up from 393).
Add handle_session_datagram handler replacing the 0x40 dispatch stub.
Transit nodes now decode the datagram envelope, enforce hop limits,
warm coordinate caches from SessionSetup/SessionAck/DataPacket payloads,
route via find_next_hop, and generate CoordsRequired/PathBroken error
signals on routing failure.
14 new tests covering decode errors, hop limit enforcement, local
delivery, cache warming for all session message types, single-hop and
multi-hop forwarding through live node chains, error signal generation,
and cache warming enabling subsequent routing. 375 tests pass.
Add src_addr to SessionDatagram envelope (34-byte header: msg_type +
src_addr + dest_addr + hop_limit) so transit routers can route error
signals back to the packet's originator.
Reclassify CoordsRequired/PathBroken as link-layer error signals
(plaintext inside SessionDatagram) rather than e2e encrypted session
messages. Transit routers generate these when forwarding fails and
route them to src_addr; if source is also unreachable, drop silently.
Remove redundant src_addr/dest_addr/hop_limit from SessionSetup,
SessionAck, and DataPacket (now in envelope). DataPacket header
reduced from 36 to 4 bytes. Remove PathBroken.original_src.
Fix routing loop vulnerability: gate bloom filter path on having
cached dest_coords to prevent blind forwarding between peers.
Simplify select_best_candidate() to require coordinates.
Fix gossip protocol type codes (0x11->0x20, 0x12->0x30, 0x13->0x31)
for consistency across all design docs.
All 5 design docs updated and cross-checked for consistency.
335 tests pass, zero warnings.
Add the full next-hop routing algorithm to Node::find_next_hop():
- Local delivery, direct peer, bloom filter candidates, greedy tree
routing fallback, with (link_cost, tree_distance, node_addr) ordering
- select_best_candidate() scores by peer→dest distance (not us→peer)
with self-distance check to prevent routing loops
- TreeState::find_next_hop() for greedy tree routing with progress
guarantee
- ActivePeer::link_cost() placeholder (constant 1.0) for future link
quality metrics
Add routing tests including 100-node all-pairs reachability simulation
(9900/9900 delivered, 0 loops, avg 4.0 hops, max 8).
Update fips-routing.md to reflect bloom filter routing as the primary
forwarding mechanism, with greedy tree routing as fallback during
convergence windows.
Split handlers.rs (986 lines) into handlers/ with 5 subfiles organized
by responsibility: rx_loop, encrypted, handshake, dispatch, timeout.
Split tests.rs (2350 lines) into tests/ with 4 subfiles: unit tests,
handshake integration, spanning tree convergence, and bloom filter tests.
Shared test helpers extracted to tests/mod.rs.
Visibility adjusted from pub(super) to pub(in crate::node) for handler
methods now two levels deep. Unused imports cleaned up in node/mod.rs.
All 316 tests pass, zero warnings.
When both nodes simultaneously initiate connections, each promotes its
inbound handshake first, leaving both sides with mismatched Noise sessions
and session indices. Resolve this in handle_msg2 using the existing
tie-breaker (smaller node_addr's outbound wins):
- Winner (smaller node): swaps ActivePeer to outbound session + indices
via new replace_session() method
- Loser (larger node): keeps inbound session and original their_index
(already references winner's outbound our_index from msg1)
Key insight: the loser must NOT update their_index from msg2, because the
msg2 sender_idx carries the winner's inbound index which becomes stale
after the winner swaps.
Defer outbound connection cleanup from promote_connection() to handle_msg2()
so outbound session state is available for the swap.
Verified with live two-node test over UDP in network namespaces: both nodes
connect, exchange TreeAnnounce, and converge to spanning tree within ~2s.
Multi-node integration test infrastructure with 100-node random topology
convergence test (5 topology variants). Two-phase drain loop processes
Noise IK handshakes and TreeAnnounce cascades over live UDP until
quiescence.
Fix parent ancestry propagation in handle_tree_announce: when a node's
current parent updates its tree state (e.g., learning a new root from
upstream), coordinates must be recomputed even though the parent itself
didn't change. Without this, chains longer than 2 hops fail to converge.
Fix promote_connection() to detect and clean up pending outbound
handshakes to the same peer, not just already-promoted peers. Previously,
when an inbound handshake completed while an outbound was still pending,
the outbound would linger until the 30s timeout.
Add auto-retry for failed outbound connections to auto-connect peers:
- New RetryState struct and node/retry.rs module
- Exponential backoff (default 5s base, max 5 attempts)
- Config: node.max_retries, node.base_retry_interval_secs
- check_timeouts() schedules retries, rx loop processes them
- promote_connection() clears retry state on success
- Remove unused PeerConnection retry fields (state now at Node level)
Move cleanup_stale_connection() logging to callers for context-appropriate
messages.
289 tests pass, clean build.
Noise IK parity fix:
- Pre-message hash normalizes responder static key to even parity (0x02)
so initiator and responder hash chains match regardless of actual parity
- ECDH uses shared_secret_point() + SHA-256(x-only) instead of
SharedSecret::new() which includes a parity-dependent version byte
- Fixes handshake failure for ~50% of keys when initiator has only npub
Graceful disconnect protocol (link message 0x50):
- DisconnectReason enum with 8 reason codes
- Disconnect struct with encode/decode
- send_encrypted_link_message() reusable helper
- handle_disconnect() with immediate peer removal
- send_disconnect_to_all_peers() called during Node::stop()
Cross-connection fix in handle_msg1():
- addr_to_link check now distinguishes inbound duplicates (reject) from
outbound links (cross-connection, allow and resolve via tie-breaker)
- remove_link() only clears addr_to_link if entry maps to same link_id
- Link cleanup and addr_to_link restoration in cross-connection branches
Handshake timeout cleanup:
- RX loop uses tokio::select! with 1-second interval tick
- check_timeouts() scans for stale (>30s) and failed connections
- cleanup_stale_connection() removes all associated state
Tests: 279 passing (4 new: cross-connection, stale cleanup, failed
cleanup, odd-parity handshake)
Split 2,597-line node.rs into four files under src/node/:
- mod.rs: types, struct definition, constructors, accessors, Debug impl
- lifecycle.rs: start(), stop(), peer connection initiation
- handlers.rs: RX loop, packet dispatch, message handlers, promote_connection
- tests.rs: full test suite (35 tests including integration tests)
No functional changes. All 267 tests pass, no new clippy warnings.
Add test_run_rx_loop_handshake exercising the full packet dispatch
path (UDP → channel → run_rx_loop → process_packet → handler) using
tokio::select! with timeout to release &mut self for assertions.
Correct session layer documentation: sessions are always established
between communicating nodes regardless of adjacency, not only for
non-adjacent traffic. Add adjacent-peer encryption flow example,
fix table description, rename Link Layer subsection to "Hop-by-Hop".
267 tests pass (266 existing + 1 new).
Fix promote_connection to transfer NoiseSession, session indices,
transport ID, and source address to ActivePeer. Populate peers_by_index
during promotion for O(1) encrypted frame dispatch.
Add responder-side promotion in handle_msg1 — after sending msg2, the
responder immediately promotes to ActivePeer since Noise IK completes
in one step for the responder side.
Wire run_rx_loop into the fips binary using tokio::select with Ctrl+C
shutdown signal, so the daemon actually processes incoming packets.
Add end-to-end integration test: two nodes with real UDP sockets
perform full Noise IK handshake and bidirectional encrypted frame
exchange (266 tests, all passing).
Rename NodeId to NodeAddr and npub to pubkey throughout documentation
and source code for consistency with the established identity model.
Add FIPS API vs IPv6 adapter overview to session protocol document.
Add transport address terminology note to wire protocol document.
Documents the tradeoff between metadata privacy and route error feedback:
- Intermediate routers see src_addr/dest_addr (traffic analysis possible)
- Source address needed for CoordsRequired/PathBroken error messages
- Deliberate choice: explicit feedback over silent drops + app timeouts
- Partial mitigation: addresses are SHA-256(npub), not npubs themselves
- Notes onion routing as alternative with different tradeoffs
Add comprehensive wire format diagrams with byte offsets, field sizes,
and concrete examples to three protocol documents:
fips-wire-protocol.md (Appendix A):
- Encrypted frame (0x00) with plaintext structure breakdown
- Noise IK msg1/msg2 with payload breakdown
- Complete handshake flow diagram
fips-gossip-protocol.md (Appendix A):
- TreeAnnounce with ancestry entry structure and size table
- FilterAnnounce with bloom filter layout
- LookupRequest/LookupResponse with examples
- Message flow showing packet nesting
fips-session-protocol.md (§8):
- SessionSetup, SessionAck, DataPacket, CoordsRequired, PathBroken
- Full nested packet example showing link + session layers
fips-routing.md:
- Refactored Part 4 to "Route Cache Management"
- Moved wire format content to fips-session-protocol.md
Session protocol (fips-session-protocol.md):
- Clarified DNS entry point applies to IP-based apps; native FIPS uses npub
- Updated transport failover examples (UDP → WiFi, WiFi → LTE)
- Added roaming description to session independence section
- Expanded §5 with coords-on-demand mechanism for route cache recovery
- Added Noise IK vs XK privacy tradeoff note to §6
Routing (fips-routing.md):
- DataPacket now has optional coordinates (COORDS_PRESENT flag)
- Updated handle_data_packet to cache coords from packets
- Sender state machine: WARM/COLD based on CoordsRequired errors
- Updated packet type summary table
Architecture (fips-architecture.md):
- Fixed cross-references to session protocol sections
Design change: Session layer now uses Noise IK pattern (previously KK),
matching the link layer. Rationale: initiator knows destination npub,
responder learns initiator identity from handshake - same asymmetry as
link connections.
Document consistency fixes:
- Auth* events → Noise IK msg1/msg2 terminology in state machines
- PeerId → NodeId in routing code examples
- BloomUpdate → FilterAnnounce terminology
- Updated cross-references and tables
Wire format:
- Add HandshakeMessageType enum (NoiseIKMsg1=0x01, NoiseIKMsg2=0x02)
- Define unified TLV framing for handshake and post-handshake messages
- Update fips-design.md and fips-protocol-flow.md with wire format details
Initialization:
- Defer TUN setup until after peer connections initiated
- Order: packet channel → transports → peers → TUN
Logging cleanup:
- Consolidate node/TUN info logs into aligned multi-line format
- TUN shutdown: single start/stop message, remove intermediate noise
- Remove ip link show callout and redundant Starting node message
- Change UDP transport stopped to debug level
- Defer TUN setup until after peer connections initiated
- Split TUN packet/device logs into aligned multi-line format
- Remove ip link show debug callout from fips binary
Config changes:
- Add ConnectPolicy enum (AutoConnect, OnDemand, Manual)
- Add PeerAddress struct (transport, addr, priority)
- Add PeerConfig struct (npub, alias, addresses, connect_policy)
- Add top-level 'peers' section to Config (after transports)
- Add accessor methods: peers(), auto_connect_peers()
Node changes:
- Add find_transport_for_type() to match transports by name
- Add initiate_peer_connections() called after transports start
- Add initiate_peer_connection() creates Link and Peer in Connecting state
- Node::start() now initiates connections to auto-connect peers
This completes the node startup sequence through step 5 (connect
to static peers). Peers are created in Connecting state; the auth
handshake implementation will transition them to Active.
Align terminology and cross-references across all FIPS design documents
based on decisions made in fips-protocol-flow.md (Session 39).
Key changes:
- Distinguish "Routing Session" (hop-by-hop cache) from "Crypto Session"
(end-to-end Noise KK encryption) throughout all docs
- Add handshake_payload field to SessionSetup/SessionAck for combined
establishment of routing and crypto sessions
- Update fips-design.md encryption section to reference Noise KK
- Add SessionSetup (0x06), SessionAck (0x07), CoordsRequired (0x08)
message types
- Add Crypto Session Management config section to fips-architecture.md
- Add cross-references to fips-protocol-flow.md in all design docs
- Update reconciliation tracking in protocol-flow.md §7
- Add name field to UdpTransport for named config instances
- Add name() and local_addr() accessors to TransportHandle
- Update UdpTransport logging to include instance name
- Remove redundant Node-level transport startup logging
- Reorder Node::start() to initialize transports before TUN
- Added TransportHandle enum for polymorphic transport dispatch
- Node now owns transports via HashMap<TransportId, TransportHandle>
- Added packet channel fields (packet_tx, packet_rx) to Node
- Transport initialization in Node::start() with graceful degradation
- Transport shutdown in Node::stop() before TUN cleanup
- Factory method create_transports() instantiates from config
Configuration redesign:
- New transports section with TransportInstances<T> enum
- Single instance: config directly under transport type (no naming overhead)
- Named instances: nested under instance names
- #[serde(deny_unknown_fields)] ensures correct untagged enum matching
- Instance names are Option<&str> - None for single, Some(name) for named
Updated Node transport methods:
- transport_count(), get_transport(), get_transport_mut()
- transport_ids(), packet_rx()
All 189 tests pass (4 new config parsing tests).
- Convert transport.rs to module directory (transport/mod.rs + transport/udp.rs)
- Add packet channel types for transport→Node communication:
- ReceivedPacket struct with transport_id, remote_addr, data, timestamp
- PacketTx/PacketRx type aliases for tokio mpsc channels
- packet_channel() constructor function
- Add UdpConfig to config.rs (enabled, bind_addr, mtu)
- Implement UdpTransport with async lifecycle:
- start_async(): binds socket, spawns receive loop
- stop_async(): aborts receive task, closes socket
- send_async(): sends packet with MTU validation
- discover(): returns empty (peer config is node-level)
- Update lib.rs with new re-exports
- Add tokio net and time features to Cargo.toml
All 185 tests pass.
Spanning tree initialization:
- TreeState now uses sequence=1 and current timestamp per protocol spec
- Added ParentDeclaration::sign() and TreeState::sign_declaration() methods
- Node initialization logs identity (node_id, address)
Node lifecycle refactoring:
- Moved TUN init and thread spawning into async Node::start()
- Moved TUN shutdown and thread cleanup into async Node::stop()
- Node owns I/O infrastructure (thread handles, TunTx channel)
- Removed: init_tun(), take_tun_device(), tun_device(), tun_device_mut()
- Added: tun_tx() accessor for packet sending
tun.rs:
- Added run_tun_reader() function (moved from binary)
fips.rs binary reduced from 230 to 117 lines. All 171 tests pass.
- New icmp.rs module: builds RFC 4443 compliant ICMPv6 Type 1 Code 0
responses with proper checksum and packet validation
- TUN reader/writer separation: fd duplication allows independent I/O
on separate threads
- TunWriter services mpsc queue for multiple future packet sources
- Reader now sends ICMPv6 errors for unroutable packets instead of
silently dropping them
- 171 tests passing
- Add TUN interface detection on startup, delete existing before create
- Add proper interface deletion on shutdown via netlink
- Add TUN packet reader in separate thread with DEBUG logging
- Add graceful shutdown handling for expected "Bad address" error
- Add take_tun_device() to Node for reader ownership transfer
- Add shutdown_tun_interface() public function for cleanup by name
- Add tokio signal feature for Ctrl+C handling
- Add --wait / -w command-line flag for debugger attachment
- When --wait is set, daemon blocks after init with thread::park()
- Add ip link show output after TUN device creation for debugging
Add 7 new modules with all core data types for the mesh routing protocol:
- tree.rs: ParentDeclaration, TreeCoordinate, TreeState
- bloom.rs: BloomFilter (4KB/7 hash), BloomState with debouncing
- transport.rs: TransportId, LinkId, Link, Transport trait, LinkStats
- protocol.rs: Auth messages, TreeAnnounce, FilterAnnounce, LookupRequest/Response, SessionSetup, DataPacket
- cache.rs: CoordCache, RouteCache with LRU eviction and TTL expiry
- peer.rs: Peer lifecycle states, filter tracking, UpstreamPeer
- node.rs: Node container with resource limits
All entities have constructors, error types, and comprehensive tests (161 total).
Stub methods with todo!() for behavior to be implemented later.
No state machine logic, protocol handlers, or async code yet.
Create comprehensive architecture document covering:
- Core entities: Node, Transport, Link, Peer hierarchy
- Transport vs Link distinction (interface vs connection)
- State machines for Transport, Link, and Peer lifecycles
- Reference transport types: UDP/IP, Ethernet, WiFi, Tor
- Spanning tree and Bloom filter state requirements
- Self-healing protocol design (no ack/retry needed)
- Leaf-only operation for constrained devices
- Comprehensive sysctl-style configuration reference
Also add architecture review document capturing identified
issues to resolve before implementation begins.
Create fips-routing.md covering the complete routing architecture:
- Bloom filter design: 4KB filters, K=2 scope, event-driven updates
- Discovery protocol: LookupRequest/Response with signed proofs
- Greedy tree routing using coordinates from discovery
- Session establishment model for minimal data packet overhead
- Router coordinate caching with LRU eviction
Key design decisions:
- Leaf-only mode for constrained devices (single peer handles routing)
- Separation of discovery (find destination) from routing (deliver packets)
- Session setup pays coordinate cost once; data packets carry only addresses
- 36-byte data packet header comparable to IPv6
Update design docs README to include new document.
Move design documents from working directory into source tree:
- fips-design.md - Full protocol design specification
- spanning-tree-dynamics.md - Spanning tree protocol dynamics study
Add README index files for docs/ and docs/design/.
Implements FIPS Identity System (Section 1 of design doc):
- NodeId: 32-byte SHA-256 hash of npub, with Ord for root election
- FipsAddress: 128-bit IPv6-compatible address (0xfd prefix)
- Identity: keypair holder with sign/verify methods
- AuthChallenge/AuthResponse: challenge-response authentication
All 11 tests passing.
Johnathan Corgan