Rework top-level README: add badges, status/roadmap section, config
search path docs, minimal two-node example with transports, DNS setup
instructions with systemd-resolved and resolv.conf examples, and
connectivity test walkthrough.
Replace ASCII document-relationship diagram with SVG in design docs.
Change DNS resolver default from disabled to enabled (port 5354).
Update config merge to allow higher-priority configs to disable it.
Wire format diagrams:
- Add 24 SVG diagrams covering every FMP and FSP wire format: common
prefix, established frame headers, Noise IK handshake messages,
handshake flow, TreeAnnounce, AncestryEntry, FilterAnnounce,
LookupRequest/Response, SessionDatagram, Disconnect, SenderReport,
ReceiverReport, FSP complete message, SessionSetup/Ack/Msg3,
PathMtuNotification, CoordsRequired, PathBroken, and MtuExceeded
- Replace ASCII art in fips-wire-formats.md with SVG references
- Apply text edits to fips-mesh-layer.md, fips-mesh-operation.md,
fips-transport-layer.md, and fips-ipv6-adapter.md
Spanning tree dynamics:
- Add 12 topology SVG diagrams: node join (overview + 3-panel steps),
three-node convergence (4-panel), link addition with depth labels,
link removal, partition formation, and 6 real-world example diagrams
(office, mixed-link, two-site WAN topologies)
- Rewrite all code blocks to narrative prose with diagram references
- Add inline prior art attributions distinguishing Yggdrasil-derived
concepts from FIPS-novel contributions
- Add 3 new references (De Couto ETX, IEEE 802.1D, RFC 2328 OSPF) and
Prior Art summary
- Remove outdated sections: indirect partition note, integration test
gaps, DHT-based lookup reference
- Change "must elect a new root" to "must rediscover its new root"
Spanning tree design review (fips-spanning-tree.md):
- Rename "Root Election" to "Root Discovery" across docs
- Add "What Is a Spanning Tree?" introductory section
- Add parent selection intro explaining self-organization role
- Fix tree distance example: 4 hops, not 2
- Clarify timestamp field as advisory only
- Remove unimplemented ROOT_TIMEOUT and TREE_ENTRY_TTL from timing
parameters and implementation status tables
Bloom filter design review (fips-bloom-filters.md):
- Add "What Is a Bloom Filter?" intro section
- Rewrite Purpose section to frame filters as routing path identification
- Correct FPR analysis (old values were 3-50x overstated)
- Add Filter Occupancy Model based on network size and tree position
- Fix filter expiration to describe actual MMP-based cleanup
- Combine Scale Considerations with Size Classes after Wire Format
- Fix stale FPR values in src/bloom/mod.rs comments
Session layer review (fips-session-layer.md):
- Add inline prior art attributions: Noise Protocol Framework, WireGuard,
DTLS (RFC 6347), IKEv2 (RFC 7296), RFC 1191 PMTUD, Yggdrasil, NIP-44
- Replace warmup state machine ASCII art with SVG diagram
- Convert CoordsWarmup wire format code block to prose
- Add External References section with full citations
Level 5 implementation doc cleanup:
- Delete fips-software-architecture.md (redundant with protocol layer docs)
- Delete fips-state-machines.md (Rust tutorial, not protocol design)
- Add fipsctl command reference to README.md
- Update cross-references in fips-intro.md, docs/design/README.md,
fips-transport-layer.md, fips-configuration.md
Fixes:
- Correct fd::/8 to fd00::/8 in fips-session-layer.md,
fips-identity-derivation.svg, and fips-node-architecture.svg
- Fix config example MTU: 1197 → 1472 in fips-configuration.md
File organization:
- Move all SVG diagrams into docs/design/diagrams/ subdirectory
- Update all diagram references to use new paths
Cost-based parent selection:
- Replace depth-only parent selection with effective_depth = depth + link_cost
- link_cost computed from locally measured MMP metrics: etx * (1.0 + srtt_ms / 100.0)
- Prevents bottleneck subtrees in heterogeneous networks where a LoRa link
at depth 1 would otherwise always beat fiber at depth 2
- Configurable hysteresis (default 0.2) prevents marginal parent switches
- Configurable hold-down timer (default 30s) suppresses re-evaluation
after parent switch
- Mandatory switches (parent lost, root change) bypass both safeguards
- Link costs passed as HashMap parameter to keep TreeState pure
Periodic re-evaluation:
- evaluate_parent() was only called on TreeAnnounce receipt or parent loss;
after tree stabilization, link degradation went undetected
- Added timer-based re-evaluation (reeval_interval_secs, default 60s) that
calls evaluate_parent() from the tick handler with current MMP link costs
- Respects existing hold-down and hysteresis safeguards
- Short-circuits when disabled or <2 peers
Design documentation:
- Update 7 design docs to reflect cost-based parent selection
- Replace depth-only algorithm descriptions with effective_depth model
- Replace rejected cumulative path cost spec with local-only design rationale
- Rewrite Example 2 (heterogeneous links) for local-only cost model
- Update config docs: parent_switch_threshold replaced by parent_hysteresis,
hold_down_secs, reeval_interval_secs
Chaos simulation enhancements:
- fips_overrides with deep merge for per-scenario FIPS config customization
- Explicit topology algorithm for deterministic test graphs
- Control socket querying via fipsctl for tree/MMP snapshot collection
- Edge existence validation in netem manager
- Per-link netem policy overrides
- 9 new chaos scenarios covering cost avoidance, depth-vs-cost tradeoffs,
stability, mixed topologies, periodic re-evaluation, and bottleneck parent
12 new unit tests, 667 total passing, clippy clean.
Gate peer_inbound_filters() to only collect from tree peers (parent
and children), so outgoing filter computation merges only tree-sourced
information. All peers still receive FilterAnnounce messages and store
filters locally for routing queries — the restriction is only on what
gets merged into outgoing filters.
This prevents bloom filter saturation where mesh shortcuts cause every
node's filter to converge toward the full network. With tree-only
merge, filters contain subtree (from children) + complement (from
parent) + single-hop mesh views.
Implementation:
- Add is_tree_peer() helper to determine tree parent/child relationship
- Gate peer_inbound_filters() to tree peers only (single control point)
- Trigger bloom filter exchange on tree relationship changes
- Add est_entries, set_bits, fill ratio, and tree_peer fields to
FilterAnnounce send/receive debug logs
- Add test_bloom_filter_split_horizon test verifying directional
asymmetry: upward filters contain only the child's subtree, downward
filters contain only the complement
- Add print_filter_cardinality diagnostic helper for test inspection
Design docs:
- fips-bloom-filters.md: Add directional asymmetry and mesh peer filter
subsections, update per-peer filter model, saturation mitigation,
implementation status table
- fips-mesh-operation.md: Update filter propagation description, add
directional asymmetry, tree relationship change trigger
- fips-intro.md: Rewrite bloom propagation paragraph for tree-only merge
Rename FIPS Link Protocol (FLP) to FIPS Mesh Protocol (FMP)
The "Link Protocol" name understated the layer's scope — spanning tree
construction, bloom filter routing, greedy forwarding, and mesh-wide
coordination go well beyond link-level concerns. Rename fips-link-layer.md
to fips-mesh-layer.md, update FLP→FMP throughout docs and source code
(FLP_VERSION→FMP_VERSION, wire.rs, rx_loop.rs, spanning_tree.rs).
New SVG illustrations
- Protocol stack: color-coded layer diagram replacing ASCII art
- OSI mapping: side-by-side comparison with traditional networking layers
- Bloom filter propagation: 6-node tree with sender-colored filter boxes
showing split-horizon computation per link
- Routing decision flowchart: 5-step priority chain with candidate ranking
by tree distance and link performance
- Coordinate discovery: sequence diagram showing LookupRequest propagation,
response caching, and SessionSetup cache warming
Redesigned existing SVGs
- Architecture overview: uniform node layout, U-shaped encrypted link
connectors, separate end-to-end session line
- Node architecture: split Router Core into FSP and FMP layers, reorganize
transports into Overlay/Shared Medium/Point-to-Point categories
- Identity derivation: wider boxes, visible encode arrow, dashed npub line
fips-intro.md revisions
- Add inline references to prior work: Yggdrasil/Ironwood for coordinate
routing, Noise Protocol Framework for IK handshakes, WireGuard for
index-based session dispatch, Wikipedia for bloom filters, split-horizon,
and greedy embedding
- Add explanatory paragraphs after bloom filter diagram describing
split-horizon filter computation and candidate selection behavior
- Simplify transport abstraction language, remove I2P/LoRa references
- Fix LookupRequest wording ("propagates" not "floods"), note intermediate
node coordinate caching on lookup responses
- Rewrite architecture overview prose to match redesigned diagrams
SessionAck previously only carried the responder's coordinates
(src_coords). When the return path diverged from the forward path
(e.g., after tree reconvergence), transit nodes on the return path
lacked the initiator's coordinates and couldn't route the SessionAck
back, causing handshake timeouts.
Add dest_coords (initiator's coordinates) to the SessionAck wire
format, mirroring SessionSetup's design. Transit nodes now cache both
endpoints' coordinates when forwarding a SessionAck, making the return
path self-sufficient regardless of path asymmetry.
Root cause confirmed by churn-20 sim log analysis: the n04-n14
handshake failure was caused by n15 (return-path transit) lacking
n04's coordinates, not by stale tree routes through a downed node.
fips-link-layer.md:
- Rewrite Liveness Detection: explicit Heartbeat (0x51) with 10s interval
and 30s dead timeout replaces vague gossip-as-heartbeat description
- Add Auto-Reconnect section: MMP dead timeout triggers retry with
unlimited backoff for auto_reconnect peers
- Add Handshake Message Retry section: link + session layer resend with
exponential backoff within timeout window
- Add Heartbeat to Link Message Types table
- Update Implementation Status with three new implemented features
fips-configuration.md:
- Add handshake_resend_interval_ms, handshake_resend_backoff,
handshake_max_resends to rate_limit table
- Add heartbeat_interval_secs, link_dead_timeout_secs to general table
- Add peers[].auto_reconnect to peers table
- Note auto-reconnect bypasses max_retries in retry section
- Update complete reference YAML with all new parameters
fips-wire-formats.md:
- Rename 0x51 from reserved Keepalive to implemented Heartbeat
- Update Disconnect reason 0x07 to Heartbeat liveness timeout
testing/chaos/README.md:
- Add runner.log to output files
- Add Directed Outbound Configs subsection
Implement hybrid coordinate cache warming strategy: piggyback coords
via CP flag when they fit within transport MTU, send standalone
CoordsWarmup (0x14) message when they don't. On CoordsRequired or
PathBroken receipt, send CoordsWarmup immediately with source-side
rate limiting (default 2s per destination, configurable).
- Add CoordsWarmup = 0x14 session message type (empty body, CP flag)
- Add send_coords_warmup() following send_session_msg() pattern
- Restructure send_session_data() to send standalone warmup before
data packet when piggybacked coords exceed MTU
- Add immediate CoordsWarmup response in handle_coords_required()
and handle_path_broken() with per-destination rate limiting
- Add coords_response_interval_ms config (node.session)
- Add RoutingErrorRateLimiter::with_interval() constructor
- Zero transit-path changes: existing try_warm_coord_cache() handles
CoordsWarmup identically to CP-flagged data packets
- Update design docs (session layer, wire formats, mesh operation,
configuration)
FIPS_OVERHEAD was 150 but had two bugs: the session AEAD tag (16 bytes)
was listed in the comment but missing from the arithmetic, and the
coordinate budget (~60 bytes) was undersized and didn't belong in a
constant representing fixed data path overhead.
Corrected to 106 bytes (the actual fixed overhead without coordinates).
This increases effective_ipv6_mtu from 1322 to 1366 for standard
Ethernet, well above the IPv6 minimum of 1280.
Added a guard in send_session_data() that computes the total wire size
with coordinates before committing to include them. If adding coords
would exceed the transport MTU, the CP flag is skipped and the warmup
counter is not decremented. This prevents silently producing oversized
packets at tree depth 2+.
Updated design docs (ipv6-adapter, wire-formats, mesh-operation) with
corrected overhead values.
Bring all 9 design documents into alignment with the current
implementation. Major additions include MMP coverage at both link and
session layers, wire format tables for SenderReport and ReceiverReport,
UDP socket buffer sizing, PathMtuNotification status updates, and
configuration parameter fixes (default_ttl key name, idle timeout MMP
exclusion semantics).
- PathBroken handler: convert to async, trigger re-discovery via
maybe_initiate_lookup(), reset COORDS_PRESENT warmup counter
(was a stub that only invalidated coord_cache)
- CoordsRequired recovery timing: reset warmup counter in
handle_lookup_response() when discovery completes for an
established session, so COORDS_PRESENT packets fire after
fresh coords are available (not just on CoordsRequired receipt)
- Routing error rate limiting: add RoutingErrorRateLimiter
(100ms per-destination, matching ICMP PTB pattern) to gate
send_routing_error() at transit nodes
- Remove root refresh dead code: the 1800s periodic root
re-announcement in check_tree_state() only propagated to
depth 1 (sequence-only changes don't cascade). Root loss
detection relies on link failure propagation which works
correctly.
Sync fips-configuration.md with code changes from sessions 101-102:
- Replace route_size with identity_size (cache merge)
- Update cache section description (single cache, not dual)
- Add idle_timeout_secs and coords_warmup_packets to session table
- Add both to Complete Reference YAML
- Fix UDP MTU default in Complete Reference (1197 → 1280)
Add dual-approach MTU handling to prevent TCP connections from hanging
when packets exceed the transport MTU after FIPS encapsulation.
ICMPv6 Packet Too Big:
- Generate RFC 4443 PTB messages for oversized packets at TUN outbound
- Inject back via TUN for local delivery to the application
- Per-source rate limiting (100ms interval, 10s entry expiry)
- MTU check in handle_tun_outbound before session encapsulation
TCP MSS Clamping:
- Intercept SYN packets in run_tun_reader() (outbound)
- Intercept SYN-ACK packets in TunWriter (inbound)
- Clamp MSS option to fit within effective MTU (transport - 127 overhead)
- Recalculate TCP checksum after modification
Code organization:
- ICMP, TCP MSS, and rate limiter modules in upper/ alongside existing
protocol-specific packet handling (dns.rs, tun.rs)
- Shared FIPS_OVERHEAD constant (127 bytes) and effective_ipv6_mtu()
function in upper/icmp.rs
- Node::effective_ipv6_mtu() delegates to the shared function
- run_tun_reader() accepts actual transport MTU from config
Example config corrections:
- UDP transport MTU set to 1472 across all configs (correct max UDP
payload for standard Ethernet: 1500 - 20 IPv4 - 8 UDP)
- Startup logging of effective MTU and max MSS values
Add fips-identity-derivation.svg showing the derivation chain from
pubkey to npub, node_addr, and IPv6 address with protocol roles and
privacy boundary. Replaces the code block in Node Address Derivation
with the diagram and consolidated descriptive text.
- fips-architecture.md → fips-software-architecture.md with all
cross-references updated (4 files)
- fips-transport-abstraction.svg → fips-node-architecture.svg, moved
from Transport Abstraction section to Architecture Overview in
fips-intro.md
- Added descriptive paragraph for node architecture diagram covering
three-layer design (application interfaces, router core, transports)
Replace three ASCII art diagrams in fips-intro.md with SVG images:
- Architecture overview: 5-node 4-hop path with layered node boxes
- Mesh topology: 8-node network with spanning tree highlighted
- Transport abstraction: full node stack with 10 transport plugins
Add src_addr to SessionDatagram envelope (34-byte header: msg_type +
src_addr + dest_addr + hop_limit) so transit routers can route error
signals back to the packet's originator.
Reclassify CoordsRequired/PathBroken as link-layer error signals
(plaintext inside SessionDatagram) rather than e2e encrypted session
messages. Transit routers generate these when forwarding fails and
route them to src_addr; if source is also unreachable, drop silently.
Remove redundant src_addr/dest_addr/hop_limit from SessionSetup,
SessionAck, and DataPacket (now in envelope). DataPacket header
reduced from 36 to 4 bytes. Remove PathBroken.original_src.
Fix routing loop vulnerability: gate bloom filter path on having
cached dest_coords to prevent blind forwarding between peers.
Simplify select_best_candidate() to require coordinates.
Fix gossip protocol type codes (0x11->0x20, 0x12->0x30, 0x13->0x31)
for consistency across all design docs.
All 5 design docs updated and cross-checked for consistency.
335 tests pass, zero warnings.
Add the full next-hop routing algorithm to Node::find_next_hop():
- Local delivery, direct peer, bloom filter candidates, greedy tree
routing fallback, with (link_cost, tree_distance, node_addr) ordering
- select_best_candidate() scores by peer→dest distance (not us→peer)
with self-distance check to prevent routing loops
- TreeState::find_next_hop() for greedy tree routing with progress
guarantee
- ActivePeer::link_cost() placeholder (constant 1.0) for future link
quality metrics
Add routing tests including 100-node all-pairs reachability simulation
(9900/9900 delivered, 0 loops, avg 4.0 hops, max 8).
Update fips-routing.md to reflect bloom filter routing as the primary
forwarding mechanism, with greedy tree routing as fallback during
convergence windows.
Noise IK parity fix:
- Pre-message hash normalizes responder static key to even parity (0x02)
so initiator and responder hash chains match regardless of actual parity
- ECDH uses shared_secret_point() + SHA-256(x-only) instead of
SharedSecret::new() which includes a parity-dependent version byte
- Fixes handshake failure for ~50% of keys when initiator has only npub
Graceful disconnect protocol (link message 0x50):
- DisconnectReason enum with 8 reason codes
- Disconnect struct with encode/decode
- send_encrypted_link_message() reusable helper
- handle_disconnect() with immediate peer removal
- send_disconnect_to_all_peers() called during Node::stop()
Cross-connection fix in handle_msg1():
- addr_to_link check now distinguishes inbound duplicates (reject) from
outbound links (cross-connection, allow and resolve via tie-breaker)
- remove_link() only clears addr_to_link if entry maps to same link_id
- Link cleanup and addr_to_link restoration in cross-connection branches
Handshake timeout cleanup:
- RX loop uses tokio::select! with 1-second interval tick
- check_timeouts() scans for stale (>30s) and failed connections
- cleanup_stale_connection() removes all associated state
Tests: 279 passing (4 new: cross-connection, stale cleanup, failed
cleanup, odd-parity handshake)
Add test_run_rx_loop_handshake exercising the full packet dispatch
path (UDP → channel → run_rx_loop → process_packet → handler) using
tokio::select! with timeout to release &mut self for assertions.
Correct session layer documentation: sessions are always established
between communicating nodes regardless of adjacency, not only for
non-adjacent traffic. Add adjacent-peer encryption flow example,
fix table description, rename Link Layer subsection to "Hop-by-Hop".
267 tests pass (266 existing + 1 new).
Rename NodeId to NodeAddr and npub to pubkey throughout documentation
and source code for consistency with the established identity model.
Add FIPS API vs IPv6 adapter overview to session protocol document.
Add transport address terminology note to wire protocol document.
Documents the tradeoff between metadata privacy and route error feedback:
- Intermediate routers see src_addr/dest_addr (traffic analysis possible)
- Source address needed for CoordsRequired/PathBroken error messages
- Deliberate choice: explicit feedback over silent drops + app timeouts
- Partial mitigation: addresses are SHA-256(npub), not npubs themselves
- Notes onion routing as alternative with different tradeoffs
Add comprehensive wire format diagrams with byte offsets, field sizes,
and concrete examples to three protocol documents:
fips-wire-protocol.md (Appendix A):
- Encrypted frame (0x00) with plaintext structure breakdown
- Noise IK msg1/msg2 with payload breakdown
- Complete handshake flow diagram
fips-gossip-protocol.md (Appendix A):
- TreeAnnounce with ancestry entry structure and size table
- FilterAnnounce with bloom filter layout
- LookupRequest/LookupResponse with examples
- Message flow showing packet nesting
fips-session-protocol.md (§8):
- SessionSetup, SessionAck, DataPacket, CoordsRequired, PathBroken
- Full nested packet example showing link + session layers
fips-routing.md:
- Refactored Part 4 to "Route Cache Management"
- Moved wire format content to fips-session-protocol.md
Session protocol (fips-session-protocol.md):
- Clarified DNS entry point applies to IP-based apps; native FIPS uses npub
- Updated transport failover examples (UDP → WiFi, WiFi → LTE)
- Added roaming description to session independence section
- Expanded §5 with coords-on-demand mechanism for route cache recovery
- Added Noise IK vs XK privacy tradeoff note to §6
Routing (fips-routing.md):
- DataPacket now has optional coordinates (COORDS_PRESENT flag)
- Updated handle_data_packet to cache coords from packets
- Sender state machine: WARM/COLD based on CoordsRequired errors
- Updated packet type summary table
Architecture (fips-architecture.md):
- Fixed cross-references to session protocol sections
Design change: Session layer now uses Noise IK pattern (previously KK),
matching the link layer. Rationale: initiator knows destination npub,
responder learns initiator identity from handshake - same asymmetry as
link connections.
Document consistency fixes:
- Auth* events → Noise IK msg1/msg2 terminology in state machines
- PeerId → NodeId in routing code examples
- BloomUpdate → FilterAnnounce terminology
- Updated cross-references and tables
Wire format:
- Add HandshakeMessageType enum (NoiseIKMsg1=0x01, NoiseIKMsg2=0x02)
- Define unified TLV framing for handshake and post-handshake messages
- Update fips-design.md and fips-protocol-flow.md with wire format details
Initialization:
- Defer TUN setup until after peer connections initiated
- Order: packet channel → transports → peers → TUN
Logging cleanup:
- Consolidate node/TUN info logs into aligned multi-line format
- TUN shutdown: single start/stop message, remove intermediate noise
- Remove ip link show callout and redundant Starting node message
- Change UDP transport stopped to debug level
Align terminology and cross-references across all FIPS design documents
based on decisions made in fips-protocol-flow.md (Session 39).
Key changes:
- Distinguish "Routing Session" (hop-by-hop cache) from "Crypto Session"
(end-to-end Noise KK encryption) throughout all docs
- Add handshake_payload field to SessionSetup/SessionAck for combined
establishment of routing and crypto sessions
- Update fips-design.md encryption section to reference Noise KK
- Add SessionSetup (0x06), SessionAck (0x07), CoordsRequired (0x08)
message types
- Add Crypto Session Management config section to fips-architecture.md
- Add cross-references to fips-protocol-flow.md in all design docs
- Update reconciliation tracking in protocol-flow.md §7
Johnathan Corgan
Origami74andJohnathan Corgan