Update 9 documentation files to match current implementation:
- Add missing rekey config section (node.rekey.*) and host mapping
section to fips-configuration.md
- Update Ethernet frame format from [type:1][payload] to
[type:1][length:2 LE][payload] in wire-formats and transport docs
- Fix Ethernet effective MTU from interface-1 to interface-3
- Mark rekey as Implemented in mesh-layer and session-layer status tables
- Change TCP default port examples from 443 to 8443
- Add rekey, persistent identity, host mapping, mesh size estimation to
README features and status sections
- Update chaos scenario count from 16 to 20, add rekey topology to
static test docs
The fips-dns.service unit had three issues:
1. Wants=systemd-resolved.service caused systemd to start systemd-resolved
on systems that weren't using it, breaking existing DNS by rewriting
/etc/resolv.conf to the stub resolver at 127.0.0.53.
2. The ExecStart busy-wait loop for fips0 had no timeout, hanging forever
if fips.service failed to create the TUN device.
3. Installers unconditionally enabled fips-dns.service regardless of whether
systemd-resolved was present.
Fix by replacing Wants= with ConditionPathExists=/run/systemd/resolve (skips
cleanly if resolved isn't running), adding Requires=fips.service (won't start
without the daemon), bounding the fips0 wait loop to 30 seconds, and making
the installers conditional on systemd-resolved being active.
Phase 3 (post-first-rekey connectivity) can see brief disruptions while
links are mid-cutover. Add a 5s settle time after the rekey wait and
allow up to 4 transient pair failures in Phase 3. Phase 5 remains
strict, ensuring full recovery after the second rekey cycle.
fipsctl and fipstop checked XDG_RUNTIME_DIR before /run/fips/, causing
them to look in /run/user/<uid>/fips/ while the systemd-managed daemon
listens on /run/fips/control.sock. Reorder the fallback to check the
system-wide path first. Also improve the permission-denied error in
fipsctl to suggest adding the user to the fips group.
Add full npub column, transport type/address, direction (in/out),
goodput, and LQI-ascending sort to peers table. Colorize spanning
tree parent (magenta) and children (cyan). Add tree relationship
and transport info to show_peers JSON output.
Compute network size estimate by summing bloom filter estimated entry
counts from the spanning tree parent (upward) and children (downward),
leveraging the tree's non-overlapping partition property. Uses the
standard formula n = -(m/k) * ln(1 - X/m) already in BloomFilter.
Surfaces the estimate in three places:
- show_status JSON: "estimated_mesh_size" field
- Periodic info log at MMP log interval (default 30s)
- fipstop Node dashboard State section as "mesh: ~N"
Add a HostMap that resolves human-readable hostnames to npubs,
enabling `gateway.fips` instead of the full `npub1...xyz.fips`.
Two sources populate the map: peer `alias` fields from the YAML
config and an operator-maintained hosts file at /etc/fips/hosts.
The DNS responder auto-reloads the hosts file on each request by
checking the file modification time, so operators can update
mappings without restarting the daemon.
- New src/upper/hosts.rs: HostMap, HostMapReloader, hostname
validation, hosts file parser with auto-reload on mtime change
- DNS resolver checks host map before falling back to direct npub
- Node uses host map for peer display names
- Default hosts file added to both .deb and tarball packaging
- 26 new tests (789 total)
Update the external pub node (vps-chi at 217.77.8.91) to use its
real persistent npub instead of a fabricated test key. Remove the
unnecessary nsec from the external node definition.
Five tests covering the full TCP transport stack at the node level:
two-node handshake, three-node chain convergence with bloom filter
reachability, mixed UDP+TCP transport coexistence, MMP link-dead
detection after connection loss, and reconnection after link death
via connect-on-send.
Debian packaging via cargo-deb:
- Add [package.metadata.deb] to Cargo.toml with assets, dependencies,
and conffiles declarations
- Maintainer scripts: postinst (create fips group, enable services,
restart on upgrade), prerm (stop/disable on remove, stop-only on
upgrade), postrm (purge config, keys, group)
- Systemd units (fips.service, fips-dns.service) with /usr/bin/ paths
for .deb installs
- tmpfiles.d entry for /run/fips/ runtime directory
- build-deb.sh wrapper script for building packages
README rewrite:
- Replace run-from-source Quick Start with Building section
- Add Installation options: Debian .deb (cargo-deb) and generic Linux
(systemd tarball)
- Add full Configuration reference with default fips.yaml
- Add Usage sections: DNS resolution, monitoring (fipsctl + fipstop),
service management, and testing
- Update project structure and features list
Embed git commit hash, dirty flag, and target triple in all binaries
via a zero-dependency build.rs. Wire clap short/long version output
so -V shows "0.1.0 (rev abc1234)" and --version adds the target
triple. Log version at daemon startup.
Add version field to show_status control socket API response. Show
the daemon's version in fipstop's tab bar title and Runtime section.
Add CHANGELOG.md in Keep a Changelog format with the 0.1.0-alpha
release (2026-02-24) and unreleased work since then.
Implement periodic full rekey at both protocol layers using fresh DH
key exchanges. Uses the existing K-bit flag (FLAG_KEY_EPOCH /
FSP_FLAG_K) to coordinate cutover between peers.
FMP layer (IK pattern):
- ActivePeer gains rekey state: pending/previous sessions, K-bit epoch
tracking, drain window, dampening timer
- Handshake state stored on ActivePeer with msg1 sent on existing link
- Encrypted frame handler detects K-bit flips, promotes pending
sessions, falls back to previous session during drain
- Handshake handlers distinguish rekey from new connections using
addr_to_link lookup with identity-based fallback
- Free all session indices (current, rekey, pending, previous) on
peer removal
FSP layer (XK pattern):
- SessionEntry gains parallel rekey fields with XK-specific state
for the 3-message handshake
- Route availability check before FSP rekey initiation
- Encrypted session handler adds K-bit flip detection and dual-session
decrypt fallback
- SessionSetup/Ack/Msg3 handlers extended for rekey paths
Defense-in-depth:
- Consecutive decryption failure detector (threshold=20) triggers
forced peer removal instead of waiting for link-dead timeout
- Identity-based rekey detection as fallback when addr_to_link
doesn't match (e.g., TCP ephemeral ports)
Configuration: RekeyConfig with enabled flag, after_secs (default 120),
and after_messages (default 65536) thresholds.
Logging: info for successful K-bit cutover completions, warn for
failures, debug for intermediate handshake steps, trace for routine
operations (resends, drain cleanup).
Rekey lifecycle:
1. Timer/counter fires -> initiator starts new handshake
2. Old session continues handling traffic during handshake
3. Handshake completes -> initiator cuts over, flips K-bit
4. Responder sees flipped K-bit -> promotes new session
5. Both keep old session for 10s drain window
6. After drain, old session discarded
Integration test: Docker-based multi-phase test exercising both FMP
and FSP rekey with aggressive timers (35s). Verifies connectivity
across all 20 directed pairs survives two consecutive rekey cycles.
Includes rekey topology, docker-compose profile, and CI matrix entry.
Increase ping test convergence wait from 3s to 5s for CI reliability.
Packaging directory structure:
- packaging/common/ — shared config (fips.yaml) used by all formats
- packaging/systemd/ — systemd-specific installer and service units
Systemd packaging includes:
- build-tarball.sh: builds release binaries and creates a self-contained
install tarball with stripped binaries
- fips.service: systemd unit running the daemon with security hardening
- fips-dns.service: oneshot unit configuring resolvectl to route .fips
domain queries to the FIPS DNS shim on 127.0.0.1:5354
- install.sh: deploys binaries to /usr/local/bin, installs systemd units,
creates fips group for non-root control socket access
- uninstall.sh: removes service and binaries, optional --purge for
config and identity key files
- README.install.md: installation and configuration guide
Default config enables UDP (2121), TCP inbound (8443), TUN, and DNS
resolver. Identity is ephemeral by default for privacy; operators can
uncomment persistent: true to maintain a stable npub for static peer
publishing. Ethernet transport is commented out for per-node setup.
Implement identity management for the FIPS daemon with two modes:
Ephemeral (default): A fresh keypair is generated on every start.
Key files (fips.key, fips.pub) are written for operator visibility
but overwritten on each restart. This is privacy-friendly and
requires no configuration.
Persistent (opt-in via `node.identity.persistent: true`): Uses
three-tier identity resolution:
1. Explicit nsec in config file (advanced users)
2. Persistent key file alongside config (reused across restarts)
3. Generate new keypair, persist to key file
Key files follow the SSH id_ed25519/id_ed25519.pub convention:
- fips.key: bare bech32 nsec string, mode 0600
- fips.pub: bare bech32 npub string, mode 0644
The daemon always writes fips.pub on startup so operators can
find their current identity via `cat /etc/fips/fips.pub`.
Identity resolution extracted into testable `resolve_identity()`
function in config module. Graceful degradation: key file write
failure silently falls back to ephemeral identity.
Add `fipsctl keygen` subcommand for manual key generation:
- `-d <dir>` output directory (default: /etc/fips)
- `-f` force overwrite existing files
- `-s` print nsec/npub to stdout instead of writing files
- Warns that `persistent: true` must be set in config
- Works without a running daemon
Includes unit tests for key file read/write roundtrip, file
permissions, whitespace trimming, empty file error, path derivation,
ephemeral-by-default behavior, ephemeral key cycling, persistent
key file loading, and persistent generate-and-reuse lifecycle.
The link-dead check required last_recv_time to be Some, so peers that
completed a handshake but never sent any data back (last_recv_time =
None) were silently skipped and lived forever as zombies. Fall back to
session_start when no frame has ever been received.
Implement hop-by-hop ECN congestion signaling through the FMP layer,
transport-level congestion detection via kernel drop counters, and
chaos harness integration for end-to-end validation.
FMP/session ECN plumbing:
- Thread ce_flag parsed at link layer through dispatch_link_message,
handle_session_datagram, handle_session_payload, and
handle_encrypted_session_msg to session delivery
- Replace hardcoded false in session-layer record_recv() with actual
ce_flag, activating ecn_ce_count tracking in session MMP
ECN congestion detection and CE relay:
- Add EcnConfig (node.ecn.*) with configurable loss_threshold (5%)
and etx_threshold (3.0) for transit congestion detection
- Add send_encrypted_link_message_with_ce() that ORs FLAG_CE into FMP
header flags; original method delegates with ce_flag=false
- Compute outgoing_ce = incoming_ce || local congestion on next-hop
link, enabling hop-by-hop CE relay through transit nodes
IPv6 ECN-CE marking:
- Mark ECN-CE (0b11) in IPv6 Traffic Class on received DataPackets
before TUN delivery when FMP CE flag is set
- Only marks ECN-capable packets (ECT(0)/ECT(1)); Not-ECT packets
unchanged per RFC 3168
Transport congestion abstraction and UDP kernel drop detection:
- Add TransportCongestion struct to transport layer for transport-
agnostic local congestion indicators
- Replace tokio::UdpSocket with AsyncFd<socket2::Socket> using
libc::recvmsg() with ancillary data parsing
- Enable SO_RXQ_OVFL for kernel receive buffer drop counter on every
packet, wiring up previously-stubbed UdpStats.kernel_drops
- Add TransportDropState for per-transport delta tracking with 1s
tick sampling via sample_transport_congestion()
- Extend detect_congestion() with transport kernel drop check
alongside MMP loss metrics
Congestion monitoring and control:
- Add CongestionStats (ce_forwarded, ce_received, congestion_detected,
kernel_drop_events) to NodeStats with snapshot serialization
- Wire counters into forwarding path, session handler, and transport
drop sampling with rate-limited warn logging (5s interval)
- Expose congestion data in show_routing control query and
ecn_ce_count in show_mmp peer entries
- Add congestion counters to fipstop routing tab in two-column layout
Chaos harness integration:
- Add query_routing(), query_transports(), snapshot_all_congestion()
to chaos control module
- Add congestion/kernel-drop log analysis in logs module
- Add congestion-stress scenario: 10-node tree, 1 Mbps bandwidth,
5-10% netem loss, heavy iperf3 traffic
- Add IngressConfig for tc ingress policing with per-peer policer
filters simulating upstream bandwidth bottlenecks
- Add iperf3 JSON result capture to traffic manager for throughput
measurement across scenarios
- Add ECN A/B test scenarios (ecn-ab-on/off.yaml) with ingress
policing and comparison script
- Enable TCP ECN negotiation (tcp_ecn=1 sysctl) in container
entrypoint for end-to-end CE propagation
Tests:
- 10 ECN unit/integration tests: mark_ipv6_ecn_ce variants, CE relay
chain (3-node propagation), EcnConfig serde roundtrip
- 3 transport drop congestion detection unit tests
Documentation:
- Update fips-mesh-layer.md: replace outdated CE Echo stub with full
ECN Congestion Signaling section covering detection logic, CE relay,
IPv6 marking, session tracking, and monitoring counters
- Update fips-configuration.md: add node.ecn.* parameter table and
ecn block in complete reference YAML
- Update fips-transport-layer.md: add Congestion Reporting section
with TransportCongestion struct, congestion() trait method, and
per-transport status; document AsyncFd/recvmsg/SO_RXQ_OVFL in UDP
- Update chaos README: add congestion/ECN scenario docs, ingress
traffic control, and iperf3 JSON capture sections
- Update README.md: add ECN to features list and "What works today";
update transport and tooling entries
Non-FIPS clients (e.g., TLS connections) hitting the TCP listen port
produce misleading "unknown FMP phase" errors because the stream reader
checked phase before version. A TLS ClientHello (byte 0x16) parsed as
version=1, phase=6.
Add UnknownVersion error variant and check the version nibble before
phase dispatch, so non-FIPS connections now report "unknown FMP version: 1"
instead of "unknown FMP phase: 0x06".
Ethernet requires a minimum 46-byte payload (60-byte frame minus
14-byte header). NICs/drivers pad shorter frames with zeros. The
Ethernet transport had no length field, so the receiver included
padding bytes in the ciphertext, causing AEAD (ChaCha20-Poly1305)
authentication tag mismatch on small frames like heartbeats (39 bytes
with prefix, padded to 46).
Add a 2-byte little-endian payload length field after the frame type
byte. Wire format changes from [type:1][payload] to
[type:1][length:2 LE][payload]. The receiver uses the length field to
extract exactly the right number of bytes, ignoring any NIC padding.
Frame overhead increases from 1 to 3 bytes, effective MTU adjusted
accordingly. This is a wire-format breaking change requiring
simultaneous upgrade of all Ethernet peers.
Two bugs prevented Path MTU Discovery from working across heterogeneous
links (e.g., ethernet→UDP boundary):
1. ICMPv6 Packet Too Big wrote the MTU as u16 into a 32-bit field
(RFC 4443 §3.2), causing the kernel to read an inflated value.
Fixed by changing build_packet_too_big() to use u32 and writing
all 4 bytes.
2. handle_tun_outbound() only checked the local transport MTU, not
the per-destination PathMtuState updated by MtuExceeded signals.
Added a PathMtuState check after session lookup so subsequent
oversized packets generate ICMPv6 PTB on TUN instead of being
forwarded and dropped at the bottleneck hop.
Added integration test exercising the full PMTUD loop across a 3-node
chain with heterogeneous MTUs: oversized packet → forwarding failure →
MtuExceeded signal → PathMtuState update → ICMPv6 PTB on TUN.
evaluate_parent() did not check whether a candidate peer's ancestry
path already contained our own node_addr. Two nodes (e.g., sidecar and
VPS) could each select the other as parent, creating an alternating
coordinate loop that grew unbounded on each TreeAnnounce exchange.
Add loop detection in two places:
- evaluate_parent(): skip candidates whose ancestry contains us
- handle_tree_announce(): detect when current parent's updated ancestry
contains us and drop the parent instead of propagating the loop
Tailscale-style sidecar pattern: a FIPS container provides mesh
networking, and a companion app container shares its network namespace
via network_mode: service:fips.
Security model:
- iptables enforces strict isolation — the app container can only
communicate over the FIPS mesh (fd::/8 via fips0)
- No IPv4 access: eth0 restricted to FIPS UDP transport (port 2121)
- No IPv6 on eth0: ip6tables blocks all eth0 IPv6 traffic
- Only fips0 and loopback are reachable from the app container
The sidecar accepts peer configuration via environment variables
(FIPS_NSEC, FIPS_PEER_NPUB, FIPS_PEER_ADDR), so it can be pointed
at any FIPS node without config file generation.
Files:
- testing/sidecar/: Dockerfile, Dockerfile.app, docker-compose.yml,
entrypoint.sh, .env, resolv.conf, scripts/build.sh
- testing/sidecar/README.md: security model, quick-start, architecture,
DNS resolution, troubleshooting, production considerations
- testing/sidecar/scripts/test-sidecar.sh: 3-node chain integration
test verifying link establishment, multi-hop connectivity, and
network isolation on each app container
- .github/workflows/ci.yml: sidecar integration test matrix entry
Chaos harness enhancements:
- transport_mix config: weighted random transport assignment for random
topologies (erdos_renyi, random_geometric) with UDP/Ethernet/TCP
- Replace LoRa with Bluetooth L2CAP in cost-based/mixed-tech scenarios
using realistic netem values (15-40ms delay, 5-15ms jitter, 2-8% loss)
- New churn-20-mixed scenario: 20-node Erdos-Renyi with 60% UDP,
20% Ethernet, 20% TCP, full netem/link-flap/churn/bandwidth config
- Expanded chaos README with full scenario catalog in four categories
Static harness:
- Updated README with topology table and scenario count
Implement TCP transport for FIPS enabling firewall traversal and serving
as the foundation for future Tor transport. This is the first
connection-oriented transport in the system.
Key design decisions:
- FMP header-based framing: reuses existing 4-byte FMP common prefix for
packet boundary recovery with zero framing overhead
- Session survives TCP reconnection: Noise/MMP/FSP state bound to npub,
not TCP connection; MMP liveness is sole authority for peer death
- Connect-on-send: fresh connection on first send, transparent reconnect
- close_connection() trait method for cross-connection deduplication cleanup
New transport files:
- src/transport/tcp/mod.rs: TcpTransport, connection pool, accept loop
- src/transport/tcp/stream.rs: FMP-aware stream reader (shared with Tor)
Modified: transport trait (close_connection), TcpConfig, TransportHandle
match arms, create_transports(), initiate_connection() for connection-
oriented links, cross-connection tie-breaker cleanup, design docs.
Tree announce loop and TCP stability fixes:
- Preserve tree announce rate-limit state across reconnection: carry
forward last_tree_announce_sent_ms when a peer reconnects so the
rate-limit window isn't reset to zero
- Drop oversize TCP packets at sender: pre-send MTU check returns
MtuExceeded instead of writing to the stream, preventing receiver-side
connection teardown and reset-reconnect cycles
Chaos harness:
- TCP transport support: tcp_edges/has_tcp/tcp_peers in SimTopology,
transport-aware config_gen with per-edge transport type, TCP port 443,
pure-TCP node support
- Include all non-Ethernet edges in directed_outbound()
- Fix netem/links log messages to say "IP-based" instead of "UDP"
- Add tcp-chain, tcp-only, and tcp-mesh scenario files
Static harness:
- Transport-aware config generation (get_default_transport, transport_port)
- TCP transport injection via Python post-processing
- Add tcp-chain topology and docker-compose profile
When a peer container is restarted during node churn, the veth pair is
destroyed and recreated. The beacon sender's AF_PACKET socket becomes
stale, producing ENXIO (os error 6) on every send with no recovery.
The beacon sender loop now tracks consecutive send errors and, after 3
consecutive ENXIO failures, attempts to open a fresh AF_PACKET socket
on the same interface. Only the first error in a streak is logged at
warn level to avoid log spam. On successful reopen, beacons resume
normally, allowing peer rediscovery.
Also adds reopen_beacon_socket() helper that creates and wraps a new
PacketSocket without needing access to the EthernetTransport struct.
Update the default UDP bind port from 4000 to 2121 (decimal) and the
default Ethernet EtherType from 0x88B5 to 0x2121 across all source
code, documentation, configuration templates, test fixtures, and
scripts. Remove references to "IEEE 802 experimental range" since
0x2121 is not in that range.
Implement raw Ethernet transport using AF_PACKET SOCK_DGRAM on Linux
with EtherType 0x88B5 (IEEE experimental range) and 1-byte frame type
prefix (0x00=data, 0x01=beacon).
Transport implementation:
- EthernetConfig with interface, ethertype, MTU, buffer sizes, and
four independent discovery knobs (discovery, announce, auto_connect,
accept_connections)
- PacketSocket/AsyncPacketSocket wrappers with ioctl helpers for
interface index, MAC address, and MTU queries
- EthernetTransport with Transport trait impl, async start/stop/send,
receive loop dispatching data frames and discovery beacons
- Discovery beacons (34 bytes: type + version + x-only pubkey) with
DiscoveryBuffer for peer accumulation and dedup
- Atomic statistics counters (frames, bytes, errors, beacons)
- Platform-gated with #[cfg(target_os = "linux")]
Transport-layer discovery integration:
- Promote auto_connect() and accept_connections() to Transport trait
with default implementations and TransportHandle dispatch
- Extract initiate_connection() so both static peer config and
discovery auto-connect share the same handshake initiation path
- Add poll_transport_discovery() to the tick handler to drain
discovery buffers and auto-connect to discovered peers
- Enforce accept_connections() in handle_msg1() — transports with
accept_connections=false silently drop inbound handshakes
Node integration:
- create_transports() handles Ethernet named instances
- resolve_ethernet_addr() parses "interface/mac" address format
- transport_mtu() generalized for multi-transport operation
Test harness:
- VethPair RAII struct for veth pair lifecycle management
- Three #[ignore] integration tests requiring root/CAP_NET_RAW:
two-node handshake, data exchange, mixed transport coexistence
- Chaos harness: transport-aware topology model, VethManager for
veth pairs between Docker containers, Ethernet-aware config gen,
netem split (HTB+u32 for UDP, root netem for veth), transport-aware
link flaps and node churn with veth re-setup
- Container entrypoint waits for configured Ethernet interfaces
before starting FIPS (handles veth creation timing)
- New scenarios: ethernet-only (4-node ring), ethernet-mesh (6-node
mixed UDP+Ethernet with netem and link flaps)
Documentation:
- fips-transport-layer.md: Ethernet section, beacon discovery, WiFi
compatibility, updated discovery state, trait surface additions,
implementation status table
- fips-configuration.md: Ethernet parameter table, named instances,
peer address format, mixed UDP+Ethernet example, complete reference
- fips-wire-formats.md: Ethernet frame type prefix note
Rework top-level README: add badges, status/roadmap section, config
search path docs, minimal two-node example with transports, DNS setup
instructions with systemd-resolved and resolv.conf examples, and
connectivity test walkthrough.
Replace ASCII document-relationship diagram with SVG in design docs.
Change DNS resolver default from disabled to enabled (port 5354).
Update config merge to allow higher-priority configs to disable it.
Bump rand (0.8→0.10), rtnetlink (0.14→0.20), tun (0.7→0.8),
simple-dns (0.9→0.11), socket2 (0.5→0.6), and criterion (0.5→0.8).
Migrate all rand call sites: thread_rng()→rng(), gen()→random(),
gen_range()→random_range(), RngCore→Rng trait. Work around secp256k1
0.30 requiring rand 0.8 by generating random bytes directly and
constructing SecretKey from slice.
Migrate rtnetlink to builder-based API: LinkSetRequest replaced with
LinkUnspec builder + change(), RouteAddRequest replaced with
RouteMessageBuilder.
Remove bloom benchmark (criterion 0.8 incompatible with old harness
config).
Wire format diagrams:
- Add 24 SVG diagrams covering every FMP and FSP wire format: common
prefix, established frame headers, Noise IK handshake messages,
handshake flow, TreeAnnounce, AncestryEntry, FilterAnnounce,
LookupRequest/Response, SessionDatagram, Disconnect, SenderReport,
ReceiverReport, FSP complete message, SessionSetup/Ack/Msg3,
PathMtuNotification, CoordsRequired, PathBroken, and MtuExceeded
- Replace ASCII art in fips-wire-formats.md with SVG references
- Apply text edits to fips-mesh-layer.md, fips-mesh-operation.md,
fips-transport-layer.md, and fips-ipv6-adapter.md
Spanning tree dynamics:
- Add 12 topology SVG diagrams: node join (overview + 3-panel steps),
three-node convergence (4-panel), link addition with depth labels,
link removal, partition formation, and 6 real-world example diagrams
(office, mixed-link, two-site WAN topologies)
- Rewrite all code blocks to narrative prose with diagram references
- Add inline prior art attributions distinguishing Yggdrasil-derived
concepts from FIPS-novel contributions
- Add 3 new references (De Couto ETX, IEEE 802.1D, RFC 2328 OSPF) and
Prior Art summary
- Remove outdated sections: indirect partition note, integration test
gaps, DHT-based lookup reference
- Change "must elect a new root" to "must rediscover its new root"
Spanning tree design review (fips-spanning-tree.md):
- Rename "Root Election" to "Root Discovery" across docs
- Add "What Is a Spanning Tree?" introductory section
- Add parent selection intro explaining self-organization role
- Fix tree distance example: 4 hops, not 2
- Clarify timestamp field as advisory only
- Remove unimplemented ROOT_TIMEOUT and TREE_ENTRY_TTL from timing
parameters and implementation status tables
Bloom filter design review (fips-bloom-filters.md):
- Add "What Is a Bloom Filter?" intro section
- Rewrite Purpose section to frame filters as routing path identification
- Correct FPR analysis (old values were 3-50x overstated)
- Add Filter Occupancy Model based on network size and tree position
- Fix filter expiration to describe actual MMP-based cleanup
- Combine Scale Considerations with Size Classes after Wire Format
- Fix stale FPR values in src/bloom/mod.rs comments
Session layer review (fips-session-layer.md):
- Add inline prior art attributions: Noise Protocol Framework, WireGuard,
DTLS (RFC 6347), IKEv2 (RFC 7296), RFC 1191 PMTUD, Yggdrasil, NIP-44
- Replace warmup state machine ASCII art with SVG diagram
- Convert CoordsWarmup wire format code block to prose
- Add External References section with full citations
Level 5 implementation doc cleanup:
- Delete fips-software-architecture.md (redundant with protocol layer docs)
- Delete fips-state-machines.md (Rust tutorial, not protocol design)
- Add fipsctl command reference to README.md
- Update cross-references in fips-intro.md, docs/design/README.md,
fips-transport-layer.md, fips-configuration.md
Fixes:
- Correct fd::/8 to fd00::/8 in fips-session-layer.md,
fips-identity-derivation.svg, and fips-node-architecture.svg
- Fix config example MTU: 1197 → 1472 in fips-configuration.md
File organization:
- Move all SVG diagrams into docs/design/diagrams/ subdirectory
- Update all diagram references to use new paths
Track parent switch frequency in a sliding window. When switches exceed
a configurable threshold (default 4 in 60s), impose an extended hold-down
period (default 120s) that prevents non-mandatory parent changes.
Mandatory switches (parent loss, root change, shouldn't-be-root) bypass
dampening. The flap counter resets when the window expires naturally.
Implements TASK-2026-0030 / IDEA-0013.
Track consecutive send failures in SenderState. Apply 2^n backoff
multiplier (capped at 32x) to the report interval. Suppress debug
logs after 3 consecutive failures, emit recovery summary on success.
Add per-peer replay suppression counter to ActivePeer. Log the first 3
replay detections at DEBUG, then suppress with a one-time notice. Emit
a summary count on session replacement or peer removal.
Non-replay decryption errors continue to be logged unconditionally.
Guard discovery triggers in handle_path_broken() and
handle_coords_required() with has_cached_identity() check. When the
XK responder receives an error signal before msg3 completes, the
initiator's identity is unknown, making LookupResponse proof
verification impossible. Skip discovery in this case — the handshake
retry mechanism handles recovery.
Downgrade the identity_cache miss log from ERROR to WARN since it's a
known race condition, not a bug.
Cost-based parent selection:
- Replace depth-only parent selection with effective_depth = depth + link_cost
- link_cost computed from locally measured MMP metrics: etx * (1.0 + srtt_ms / 100.0)
- Prevents bottleneck subtrees in heterogeneous networks where a LoRa link
at depth 1 would otherwise always beat fiber at depth 2
- Configurable hysteresis (default 0.2) prevents marginal parent switches
- Configurable hold-down timer (default 30s) suppresses re-evaluation
after parent switch
- Mandatory switches (parent lost, root change) bypass both safeguards
- Link costs passed as HashMap parameter to keep TreeState pure
Periodic re-evaluation:
- evaluate_parent() was only called on TreeAnnounce receipt or parent loss;
after tree stabilization, link degradation went undetected
- Added timer-based re-evaluation (reeval_interval_secs, default 60s) that
calls evaluate_parent() from the tick handler with current MMP link costs
- Respects existing hold-down and hysteresis safeguards
- Short-circuits when disabled or <2 peers
Design documentation:
- Update 7 design docs to reflect cost-based parent selection
- Replace depth-only algorithm descriptions with effective_depth model
- Replace rejected cumulative path cost spec with local-only design rationale
- Rewrite Example 2 (heterogeneous links) for local-only cost model
- Update config docs: parent_switch_threshold replaced by parent_hysteresis,
hold_down_secs, reeval_interval_secs
Chaos simulation enhancements:
- fips_overrides with deep merge for per-scenario FIPS config customization
- Explicit topology algorithm for deterministic test graphs
- Control socket querying via fipsctl for tree/MMP snapshot collection
- Edge existence validation in netem manager
- Per-link netem policy overrides
- 9 new chaos scenarios covering cost avoidance, depth-vs-cost tradeoffs,
stability, mixed topologies, periodic re-evaluation, and bottleneck parent
12 new unit tests, 667 total passing, clippy clean.
Gate peer_inbound_filters() to only collect from tree peers (parent
and children), so outgoing filter computation merges only tree-sourced
information. All peers still receive FilterAnnounce messages and store
filters locally for routing queries — the restriction is only on what
gets merged into outgoing filters.
This prevents bloom filter saturation where mesh shortcuts cause every
node's filter to converge toward the full network. With tree-only
merge, filters contain subtree (from children) + complement (from
parent) + single-hop mesh views.
Implementation:
- Add is_tree_peer() helper to determine tree parent/child relationship
- Gate peer_inbound_filters() to tree peers only (single control point)
- Trigger bloom filter exchange on tree relationship changes
- Add est_entries, set_bits, fill ratio, and tree_peer fields to
FilterAnnounce send/receive debug logs
- Add test_bloom_filter_split_horizon test verifying directional
asymmetry: upward filters contain only the child's subtree, downward
filters contain only the complement
- Add print_filter_cardinality diagnostic helper for test inspection
Design docs:
- fips-bloom-filters.md: Add directional asymmetry and mesh peer filter
subsections, update per-peer filter model, saturation mitigation,
implementation status table
- fips-mesh-operation.md: Update filter propagation description, add
directional asymmetry, tree relationship change trigger
- fips-intro.md: Rewrite bloom propagation paragraph for tree-only merge
The session-layer handshake now uses the 3-message XK pattern instead
of the 2-message IK pattern, providing stronger initiator identity
hiding. The initiator static key is deferred to msg3 and encrypted
under the es+ee DH chain, so eavesdroppers cannot identify the
initiator from the handshake.
XK pattern: -> e, es (msg1) / <- e, ee + epoch (msg2) / -> s, se + epoch (msg3)
Key changes:
- Add XK handshake methods alongside existing IK methods in noise module
- Add SessionMsg3 wire format and FSP_PHASE_MSG3 (0x03) prefix
- Replace Responding state with AwaitingMsg3 in session state machine
- Rewrite session handlers: handle_session_setup defers identity to msg3,
handle_session_ack processes msg2 and sends msg3, new handle_session_msg3
completes the responder handshake and registers identity
- Link-layer (FMP) continues to use Noise IK unchanged
- Add comprehensive XK unit tests and update all integration tests
Add min_mtu (u16) to LookupRequest and path_mtu (u16) to
LookupResponse, enabling the discovery system to report transport
MTU capability along the lookup path.
LookupRequest carries min_mtu (origin's minimum MTU requirement,
default 0 = no requirement). LookupResponse carries path_mtu
(initialized to u16::MAX by the target, reduced by transit nodes
via min(path_mtu, outgoing_link_mtu) on the reverse path).
path_mtu is a transit annotation like SessionDatagram.path_mtu and
is NOT included in the proof signature. The originator stores the
discovered path_mtu in CacheEntry alongside cached coordinates.
Wire format: +2 bytes each for LookupRequest and LookupResponse.
Add a new session-layer error signal that transit routers send back to
the source when a forwarded packet exceeds the next-hop transport MTU.
This complements the existing proactive path MTU discovery (min'd at
each hop) by providing immediate feedback when oversized packets are
dropped, closing the transient window before the proactive mechanism
converges.
Wire format: 36-byte payload (msg_type + flags + dest_addr + reporter +
mtu) with FSP phase=0x0 and U flag set, matching the existing
CoordsRequired/PathBroken pattern.
Changes:
- Add SessionMessageType::MtuExceeded (0x22) and MtuExceeded struct with
encode/decode methods to protocol/session.rs
- Add NodeError::MtuExceeded variant to propagate structured MTU info
from TransportError through send_encrypted_link_message()
- Catch MtuExceeded in the forwarding path and send error signal back to
the datagram source via send_mtu_exceeded_error(), rate-limited by the
existing routing_error_rate_limiter
- Handle incoming MtuExceeded at the source by calling
PathMtuState::apply_notification() for immediate MTU decrease
- Add unit tests for encode/decode roundtrip, boundary MTU values, and
too-short payload rejection
The target node's send_lookup_response() was using greedy tree routing
(find_next_hop) as the primary method to route responses back to the
origin. When the target had the origin's coords cached from a prior
lookup, find_next_hop would route the response to whichever peer was
closest to the origin in tree space -- which might not have been on the
request's forward path. That peer would have no recent_requests entry
for the request_id, causing it to treat the response as if it were the
originator, fail identity_cache lookup, and discard the response.
Fix: prefer the reverse-path (recent_requests.from_peer) as the
primary routing method for the first hop, falling back to
find_next_hop only if no recent_request entry exists.
Also adds diagnostic output on failure to aid future debugging.
Add a Unix domain socket interface for querying node state at runtime.
A spawned tokio task accepts connections and communicates with the main
event loop via mpsc/oneshot channels, keeping all Node access
single-threaded.
Includes:
- src/control/ module with socket lifecycle, JSON protocol, and 11
query handlers (status, peers, links, tree, sessions, bloom, mmp,
cache, connections, transports, routing)
- Separate fipsctl binary for CLI queries (fipsctl show <command>)
- ControlConfig in node configuration (enabled, socket_path)
- Integration into the main select! event loop
Add link_mtu(&TransportAddr) method to the Transport trait with a
default implementation that falls back to the transport-wide mtu().
This enables transports like BLE to report per-connection MTU values
while maintaining backward compatibility for UDP and other transports
that use a single MTU for all links.
Update the forwarding and session send paths to query link_mtu() with
the next-hop peer's current address, falling back to transport-wide
mtu() when no address is available.
Each node generates a random 8-byte startup epoch, encrypted inside
both Noise IK handshake messages (msg1 and msg2). When a peer's msg1
arrives with a different epoch than the stored value, the node tears
down the stale session and processes the msg1 as a new connection,
enabling near-instant restart detection instead of the 30-second
dead timeout.
Wire format impact:
- msg1: 82 -> 106 bytes (added 24-byte encrypted epoch after ss DH)
- msg2: 33 -> 57 bytes (added 24-byte encrypted epoch after se DH)
- Wire msg1: 90 -> 114 bytes, wire msg2: 45 -> 69 bytes
Include target_coords in proof_bytes() signed data to prevent transit
nodes from substituting fake coordinates. Add mandatory signature
verification at the originator using the target's public key from
identity_cache (guaranteed available since lookups are only initiated
from contexts where the key is already cached).
Verification failure discards the response. Identity cache miss (should
never happen) logs an error and discards. Add four new tests covering
verification success, failure, cache miss, and coordinate substitution
detection.
Rename FIPS Link Protocol (FLP) to FIPS Mesh Protocol (FMP)
The "Link Protocol" name understated the layer's scope — spanning tree
construction, bloom filter routing, greedy forwarding, and mesh-wide
coordination go well beyond link-level concerns. Rename fips-link-layer.md
to fips-mesh-layer.md, update FLP→FMP throughout docs and source code
(FLP_VERSION→FMP_VERSION, wire.rs, rx_loop.rs, spanning_tree.rs).
New SVG illustrations
- Protocol stack: color-coded layer diagram replacing ASCII art
- OSI mapping: side-by-side comparison with traditional networking layers
- Bloom filter propagation: 6-node tree with sender-colored filter boxes
showing split-horizon computation per link
- Routing decision flowchart: 5-step priority chain with candidate ranking
by tree distance and link performance
- Coordinate discovery: sequence diagram showing LookupRequest propagation,
response caching, and SessionSetup cache warming
Redesigned existing SVGs
- Architecture overview: uniform node layout, U-shaped encrypted link
connectors, separate end-to-end session line
- Node architecture: split Router Core into FSP and FMP layers, reorganize
transports into Overlay/Shared Medium/Point-to-Point categories
- Identity derivation: wider boxes, visible encode arrow, dashed npub line
fips-intro.md revisions
- Add inline references to prior work: Yggdrasil/Ironwood for coordinate
routing, Noise Protocol Framework for IK handshakes, WireGuard for
index-based session dispatch, Wikipedia for bloom filters, split-horizon,
and greedy embedding
- Add explanatory paragraphs after bloom filter diagram describing
split-horizon filter computation and candidate selection behavior
- Simplify transport abstraction language, remove I2P/LoRa references
- Fix LookupRequest wording ("propagates" not "floods"), note intermediate
node coordinate caching on lookup responses
- Rewrite architecture overview prose to match redesigned diagrams
SessionAck previously only carried the responder's coordinates
(src_coords). When the return path diverged from the forward path
(e.g., after tree reconvergence), transit nodes on the return path
lacked the initiator's coordinates and couldn't route the SessionAck
back, causing handshake timeouts.
Add dest_coords (initiator's coordinates) to the SessionAck wire
format, mirroring SessionSetup's design. Transit nodes now cache both
endpoints' coordinates when forwarding a SessionAck, making the return
path self-sufficient regardless of path asymmetry.
Root cause confirmed by churn-20 sim log analysis: the n04-n14
handshake failure was caused by n15 (return-path transit) lacking
n04's coordinates, not by stale tree routes through a downed node.
fips-link-layer.md:
- Rewrite Liveness Detection: explicit Heartbeat (0x51) with 10s interval
and 30s dead timeout replaces vague gossip-as-heartbeat description
- Add Auto-Reconnect section: MMP dead timeout triggers retry with
unlimited backoff for auto_reconnect peers
- Add Handshake Message Retry section: link + session layer resend with
exponential backoff within timeout window
- Add Heartbeat to Link Message Types table
- Update Implementation Status with three new implemented features
fips-configuration.md:
- Add handshake_resend_interval_ms, handshake_resend_backoff,
handshake_max_resends to rate_limit table
- Add heartbeat_interval_secs, link_dead_timeout_secs to general table
- Add peers[].auto_reconnect to peers table
- Note auto-reconnect bypasses max_retries in retry section
- Update complete reference YAML with all new parameters
fips-wire-formats.md:
- Rename 0x51 from reserved Keepalive to implemented Heartbeat
- Update Disconnect reason 0x07 to Heartbeat liveness timeout
testing/chaos/README.md:
- Add runner.log to output files
- Add Directed Outbound Configs subsection
Auto-reconnect:
- Add per-peer auto_reconnect config (default true) to PeerConfig
- schedule_reconnect() feeds removed peers back into retry system with
unlimited retries and exponential backoff after MMP dead timeout
- RetryState gains reconnect flag to distinguish startup retries
(max_retries-limited) from auto-reconnect (unlimited)
Retry re-fire fix:
- process_pending_retries() now pushes retry_after_ms past the handshake
timeout window after successful initiate_peer_connection(), preventing
retries from firing every tick with no backoff
Chaos sim improvements:
- Directed outbound configs: BFS spanning tree + lower-ID-first assignment
eliminates dual-connect race conditions in simulation
- Save runner log (runner.log) alongside per-node logs for event correlation
- Increase churn-20 traffic aggressiveness and node churn (max_down_nodes
3→5, traffic interval min 0s, duration max 90s, concurrent flows 5→10)
handle_session_ack() had a bug where take_state() followed by
a non-Initiating match would re-insert the entry with state=None,
causing panics on subsequent .state()/.state_mut() calls. Fix by
checking is_initiating() before take_state().
Add safe is_initiating()/is_responding() to SessionEntry (matching
existing is_established()) and convert all production .state() callers
to use these None-safe methods. Gate .state() with #[cfg(test)].
Sends a 1-byte encrypted heartbeat (0x51) to each peer every 10s.
If no frame is received from a peer within 30s, the peer is removed
via remove_active_peer(), triggering tree reconvergence, coord cache
flush, and bloom filter recomputation.
This fixes the critical bug where UDP peers that silently died
(e.g., container stopped) were never detected or removed, leaving
the spanning tree permanently stale.
Both intervals are configurable via node.heartbeat_interval_secs
and node.link_dead_timeout_secs.
Add resend logic for SessionSetup/SessionAck messages routed through
the mesh. Stores the encoded payload on SessionEntry for resend in a
fresh SessionDatagram (so routing can adapt to topology changes).
Uses the same config parameters as link-layer retry.
Also fixes a latent bug: Initiating/Responding sessions previously
had no timeout — a stuck handshake would live forever. Now cleaned up
after handshake_timeout_secs (default 30s).
Responder idempotency: duplicate SessionSetup triggers resend of
stored SessionAck instead of being silently dropped. Initiator-side
duplicate SessionAck already handled safely (entry.take_state() sees
Established, puts it back and returns).
Handshake payload cleared on Established transition at both initiator
(handle_session_ack) and responder (handle_encrypted_session_msg).
Add message-level retry for Noise IK handshake within the 30s timeout
window. Previously, a lost msg1 or msg2 required the full timeout to
expire before cleanup and retry. Under 10% bidirectional loss (~19%
per attempt), this made connection establishment unreliable.
Initiator resends stored msg1 bytes with exponential backoff (1s, 2s,
4s, 8s, 16s — 5 resends). Responder stores msg2 and resends on
duplicate msg1 receipt. Duplicate msg2 at initiator drops silently
via existing pending_outbound cleanup.
Config: handshake_resend_interval_ms (1000), handshake_resend_backoff
(2.0), handshake_max_resends (5) on node.rate_limit.
P(all 6 attempts fail) under 19% loss = 0.19^6 ≈ 0.005%.
Log a warning when the kernel clamps SO_RCVBUF or SO_SNDBUF below
the requested size, directing the operator to increase rmem_max or
wmem_max. Document buffer size defaults and sysctl requirements in
the Docker config template.
Implement hybrid coordinate cache warming strategy: piggyback coords
via CP flag when they fit within transport MTU, send standalone
CoordsWarmup (0x14) message when they don't. On CoordsRequired or
PathBroken receipt, send CoordsWarmup immediately with source-side
rate limiting (default 2s per destination, configurable).
- Add CoordsWarmup = 0x14 session message type (empty body, CP flag)
- Add send_coords_warmup() following send_session_msg() pattern
- Restructure send_session_data() to send standalone warmup before
data packet when piggybacked coords exceed MTU
- Add immediate CoordsWarmup response in handle_coords_required()
and handle_path_broken() with per-destination rate limiting
- Add coords_response_interval_ms config (node.session)
- Add RoutingErrorRateLimiter::with_interval() constructor
- Zero transit-path changes: existing try_warm_coord_cache() handles
CoordsWarmup identically to CP-flagged data packets
- Update design docs (session layer, wire formats, mesh operation,
configuration)
FIPS_OVERHEAD was 150 but had two bugs: the session AEAD tag (16 bytes)
was listed in the comment but missing from the arithmetic, and the
coordinate budget (~60 bytes) was undersized and didn't belong in a
constant representing fixed data path overhead.
Corrected to 106 bytes (the actual fixed overhead without coordinates).
This increases effective_ipv6_mtu from 1322 to 1366 for standard
Ethernet, well above the IPv6 minimum of 1280.
Added a guard in send_session_data() that computes the total wire size
with coordinates before committing to include them. If adding coords
would exceed the transport MTU, the CP flag is skipped and the warmup
counter is not decremented. This prevents silently producing oversized
packets at tree depth 2+.
Updated design docs (ipv6-adapter, wire-formats, mesh-operation) with
corrected overhead values.
Bring all 9 design documents into alignment with the current
implementation. Major additions include MMP coverage at both link and
session layers, wire format tables for SenderReport and ReceiverReport,
UDP socket buffer sizing, PathMtuNotification status updates, and
configuration parameter fixes (default_ttl key name, idle timeout MMP
exclusion semantics).
Under high-throughput forwarding, the kernel default 212KB receive
buffer overflows, silently dropping ~11% of UDP datagrams at transit
nodes. Add configurable recv_buf_size and send_buf_size to UdpConfig
(default 2MB each) using socket2 for pre-bind buffer configuration.
Startup log now reports actual buffer sizes granted by the kernel.
Requires net.core.rmem_max >= 2097152 on the host for the full 2MB
to take effect; otherwise the kernel silently clamps.
Periodic metrics: use long-term EWMA trends for RTT and loss instead
of instantaneous values, add jitter in ms, remove debug-level detail
block. Teardown metrics: add jitter and goodput, use consistent units
(ms for time, % for loss). Session periodic log shows observed MTU as
single 'mtu' field. RTT trend values correctly converted from
microseconds to milliseconds.
Replace raw NodeAddr hex strings in log output with human-readable
identifiers using a four-tier lookup: configured alias, active peer
short npub, session endpoint short npub, or truncated hex fallback.
- Add PeerIdentity::short_npub() for compact npub display (npub1xxxx...yyyy)
- Add NodeAddr::short_hex() for compact hex fallback (first 4 bytes + ...)
- Add peer_aliases map populated at startup from peer config
- Add Node::peer_display_name() with four-tier resolution
- Add SessionEntry::remote_pubkey() accessor for session-layer lookups
- Update all 120 log field occurrences across 11 handler/node files
- Pass pre-computed display names to MMP static metric/teardown methods
to work around borrow checker constraints in iterator loops
Per-packet happy-path events (UDP send/receive, TUN I/O, MMP report
processing, TreeAnnounce/FilterAnnounce sent, RTT samples) moved from
debug to trace. Periodic maintenance and retry scheduling moved from
info to debug. Session state changes (established, initiated, torn
down) and transport stop promoted from debug to info. MMP report send
failures demoted from warn to debug (normal under churn).
MMP reports (SenderReport, ReceiverReport, PathMtuNotification) were
resetting the idle timer on both RX and TX paths, preventing sessions
from ever timing out when MMP traffic kept flowing. Changed touch() to
only be called for DataPacket send/receive and session establishment,
so sessions with no application data tear down after the idle timeout
even with active MMP measurement traffic.
The bash 3.2 compatibility fix (841b376) added set_key/get_key helpers
but didn't update the 6 call sites that still used RESOLVED_*[$key]
associative array syntax. Without declare -A, bash treats string
subscripts as arithmetic (evaluating to 0), so all nodes silently got
the last node's identity. Switch all references to the existing helpers.
Config system overhaul:
- Add mesh-public topology (5 Docker nodes + external pub node at 217.77.8.91)
- Refactor generate-configs.sh to read topology YAML files directly,
replacing hardcoded lookup functions, with multi-char node ID support
- Generate npubs.env with all node npubs; sourced by test scripts and
injected into containers via docker-compose env_file directive
- Add .env with COMPOSE_PROFILES=mesh so docker compose defaults to
mesh topology without requiring --profile
Deterministic mesh identity derivation:
- Add derive-keys.py: pure Python tool (no deps) deriving nsec/npub
from sha256(mesh-name|node-id) via secp256k1 and BIP-173 bech32
- generate-configs.sh and build.sh accept optional mesh-name argument;
Docker node identities are derived while external nodes keep
hardcoded keys from topology YAML
Docker compose improvements:
- Add mesh-public profile service definitions
- build.sh now runs docker compose build automatically, providing a
single command for the full binary+configs+images pipeline
- Ping test script supports mesh-public topology
Container services:
- Add HTTP server on port 8000 (IPv6-bound) serving static page,
accessible over FIPS overlay via npub.fips hostnames
- Add rsync to container packages
Documentation:
- Comprehensive README update covering topology system, identity
derivation, npubs.env, and container background services (SSH,
iperf3, HTTP) with usage examples
Add dnsmasq to forward .fips queries to the FIPS daemon (port 5354)
and all other DNS to Docker's embedded resolver (127.0.0.11). Remove
the port 53 override from the node template so FIPS uses its default
port. Also add curl and python3 to the container image for testing.
Add netem.sh for simulating adverse network conditions (delay, loss,
jitter, duplication, reordering, corruption) on Docker test containers
using tc/netem. Includes three presets (lossy, congested, terrible) and
apply/remove/status actions.
Fix iperf-test.sh bug where all tests connected the client to its own
FIPS npub (loopback) instead of the remote server's npub, meaning
previous iperf results measured loopback performance rather than
cross-node throughput.
Document netem.sh in README.md.
The awk pattern used $(NF-2) which picked up the retransmit count (0)
instead of the bandwidth value, because the retransmit field sits
between the bandwidth unit and "sender". Search for the bits/sec field
by content instead of position.
- PathBroken handler: convert to async, trigger re-discovery via
maybe_initiate_lookup(), reset COORDS_PRESENT warmup counter
(was a stub that only invalidated coord_cache)
- CoordsRequired recovery timing: reset warmup counter in
handle_lookup_response() when discovery completes for an
established session, so COORDS_PRESENT packets fire after
fresh coords are available (not just on CoordsRequired receipt)
- Routing error rate limiting: add RoutingErrorRateLimiter
(100ms per-destination, matching ICMP PTB pattern) to gate
send_routing_error() at transit nodes
- Remove root refresh dead code: the 1800s periodic root
re-announcement in check_tree_state() only propagated to
depth 1 (sequence-only changes don't cascade). Root loss
detection relies on link failure propagation which works
correctly.
Sync fips-configuration.md with code changes from sessions 101-102:
- Replace route_size with identity_size (cache merge)
- Update cache section description (single cache, not dual)
- Add idle_timeout_secs and coords_warmup_packets to session table
- Add both to Complete Reference YAML
- Fix UDP MTU default in Complete Reference (1197 → 1280)
Include source and destination coordinates on the first N DataPackets
of each session (default 5, configurable via
node.session.coords_warmup_packets). This warms transit node
coord_caches so multi-hop forwarding and error signal routing work
after SessionSetup cache entries expire.
On CoordsRequired receipt, reset the counter to re-enable coordinate
inclusion for the next N packets, handling mid-session cache expiry
and path changes.
Changes:
- SessionConfig: add coords_warmup_packets field (default 5)
- SessionEntry: add coords_warmup_remaining counter, initialized on
Established transition (both initiator and responder paths)
- send_session_data(): attach coords via DataPacket::with_coords()
while counter > 0, decrement per send
- handle_coords_required(): reset counter for affected session
- 4 new unit tests for counter lifecycle and config default
Identity cache: remove TTL-based expiry (60s TTL broke active sessions
after expiry since handle_tun_outbound checks identity_cache before
session table). Replace with LRU-only eviction bounded by configurable
identity_size (default 10K). Lookup now touches timestamp for LRU
freshness.
Cache merge: unify coord_cache and route_cache into single coordinate
cache. Both stored NodeAddr→TreeCoordinate; the layer distinction was
conceptual, not functional. Discovery-sourced entries now get the same
TTL+refresh treatment as session-sourced entries. Simplifies
find_next_hop() to single cache lookup.
Parent-change flush: clear coord_cache after recompute_coords() in both
parent-switch paths of handle_tree_announce(). Stale coordinates after
tree reconvergence cause dead-end routing that's more expensive than
re-discovery.
Tested: 493 unit tests passed, clippy clean, Docker mesh 20/20,
Docker chain 6/6.
Sessions in the Established state that have no activity for 90 seconds
are now automatically removed. This ensures idle sessions are torn down
before transit node coord_cache entries expire (300s TTL), so that when
traffic resumes a fresh SessionSetup re-warms transit node caches with
current coordinates.
The identity cache now stores registration timestamps and expires
entries after 60 seconds via lazy expiry on lookup. This prevents
unbounded growth while allowing natural repopulation through DNS
resolution on next use.
Timer ordering: identity (60s) < session (90s) < coord_cache (300s).
Both timeouts are configurable: node.session.idle_timeout_secs and
node.cache.identity_ttl_secs. Setting idle_timeout_secs to 0 disables
session idle purging.
Changes:
- Add idle_timeout_secs (default 90) to SessionConfig
- Add identity_ttl_secs (default 60) to CacheConfig
- Add timestamp to identity_cache entries, lazy expiry on lookup
- Add purge_idle_sessions() called from tick loop
- Remove #[cfg(test)] from SessionEntry::last_activity()
- 7 new tests covering timeout behavior and edge cases
Transit nodes cache destination coordinates when they forward
SessionSetup messages (via try_warm_coord_cache). These coord_cache
entries have a 5-minute TTL, after which they expire. Once expired,
the transit node can no longer forward data packets for that
destination — find_next_hop returns None and the node sends
CoordsRequired errors back to the source. This creates a permanent
routing failure for any multi-hop path after 5 minutes of the initial
session establishment, even if traffic is actively flowing.
The root cause was that find_next_hop used coord_cache.get(), a
read-only lookup that checks expiry but never extends it. Active
forwarding did not keep the cache warm. Meanwhile, get_and_touch()
existed but only updated last_used without extending expires_at.
Fix:
- find_next_hop now calls coord_cache.get_and_touch() instead of get()
- get_and_touch now calls entry.refresh() instead of entry.touch(),
which extends expires_at by the default TTL on each access
- find_next_hop signature changed from &self to &mut self to allow
the mutable cache access
This ensures that as long as traffic flows through a transit node,
the coord_cache entries stay warm and routing continues to work.
Entries still expire after 5 minutes of inactivity as designed.
FilterAnnounce messages never settled in steady state because
handle_filter_announce unconditionally marked all peers for
re-announcement on every inbound filter, creating a perpetual
ping-pong at ~1 message/sec.
Added last_sent_filters tracking to BloomState. mark_changed_peers()
computes outgoing filters and compares against what was last sent,
only marking peers whose filter content actually differs.
Wire the discovery protocol into the data plane path so that
find_next_hop() consults route_cache as a fallback when coord_cache
has no entry. When session initiation fails due to missing routes,
trigger discovery (initiate_lookup) instead of immediately sending
ICMPv6 Destination Unreachable. On discovery completion, retry
session initiation for any pending TUN packets.
Changes:
- find_next_hop() falls back to route_cache when coord_cache misses
- handle_tun_outbound() triggers discovery on session failure
- handle_lookup_response() retries session after discovery completes
- handle_coords_required() triggers discovery for missing coordinates
- Add pending_lookups deduplication with 10-second timeout
- Periodic cleanup of stale lookups in tick handler
- New test: route_cache fallback verification in find_next_hop
Replace ASCII art diagram in examples/two-node-udp/README.md with
two-node-udp.svg showing namespaces, veth pair, TUN devices, DNS
responders, and transport/session layers.
Johnathan Corgan