handle_disconnect() called remove_active_peer without scheduling a
reconnect, orphaning auto-connect peers on a clean upstream shutdown.
Mirror the pattern from the other three peer-removal paths (link-dead,
decrypt failure, peer restart) which all schedule reconnect after
removal.
Adds test_disconnect_schedules_reconnect regression test that verifies
handle_disconnect populates retry_pending for an auto-connect peer.
Visibility of handle_disconnect bumped to pub(in crate::node) for
direct unit-test access.
Fixes#60.
When bloom filter candidates existed but none were strictly closer to
the destination in tree-coordinate distance, find_next_hop returned
None without trying greedy tree routing. This caused NoRoute failures
in topologies where the tree parent was closer but not a bloom
candidate. Fall through to tree routing when no bloom candidate makes
progress.
Replace discovery flooding with bloom-filter-guided tree routing:
lookups sent only to tree peers (parent + children) whose bloom
filter contains the target. If no tree peer matches, fall back to
non-tree peers with bloom matches before dropping the request. This
produces single-path forwarding through the spanning tree (90%
traffic reduction) while recovering from dead ends caused by stale
bloom filters, tree restructuring, or transit node failures.
Remove visited bloom filter from LookupRequest wire format (-257
bytes per request). Tree routing is inherently loop-free; request_id
dedup handles edge cases during tree restructuring. Add response-
forwarded flag to prevent response routing loops from convergent
request paths.
Add originator-side exponential backoff (30s base, 300s cap) after
lookup timeouts and bloom misses. Backoff resets on topology changes
(parent switch, new peer, first RTT, reconnection).
Add transit-side per-target rate limiting (2s minimum interval) for
forwarded lookups as defense-in-depth.
Add discovery retry within the timeout window (default: send at T=0,
retry at T=5s, fail at T=10s) to compensate for single-path fragility.
Lookups with zero eligible tree peers fail immediately.
Improve discovery logging: promote key events to info (initiation,
success, timeout with failure count). Add debug logging for dedup,
pending packet retry, backoff suppression, forward rate limiting,
and backoff reset.
New config: discovery.backoff_base_secs, backoff_max_secs,
forward_min_interval_secs, retry_interval_secs, max_attempts.
New stats: req_backoff_suppressed, req_forward_rate_limited,
req_bloom_miss, req_no_tree_peer, req_fallback_forwarded.
Removed: req_already_visited, visited bloom filter.
Add runtime peer management to the FIPS daemon via control socket
commands, and a new chaos simulation scenario that exercises dynamic
topology mutation with ephemeral node identities.
Daemon (connect/disconnect commands):
- Extend control socket Request with optional params field
- Add commands.rs module for mutating command dispatch, separate from
read-only queries
- Add api_connect() on Node: builds ephemeral PeerConfig (no auto-
reconnect), pre-seeds identity cache, reuses initiate_peer_connection
- Add api_disconnect() on Node: calls remove_active_peer(), clears
retry_pending to suppress reconnection
- Route non-show_* commands to async command dispatch in rx_loop
fipsctl CLI:
- Add Connect and Disconnect subcommands accepting npub or hostname
- Resolve hostnames from /etc/fips/hosts before sending to daemon
- Refactor socket I/O into reusable send_request helper
Chaos simulator (maelstrom scenario):
- Add PeerChurnManager: periodically disconnects a random active link
and connects a random unconnected node pair via control socket
- Add send_command() to control.py using base64-encoded JSON payloads
to avoid shell quoting issues in docker exec
- Add PeerChurnConfig to scenario with interval and ephemeral_fraction
- Ephemeral identity support: nodes configured without nsec generate
fresh keypairs on restart; simulator queries show_status for new
npub and updates its cache via on_node_restart callback
- Add maelstrom.yaml: all chaos dimensions (netem, link flaps, node
churn, peer topology churn, traffic) with 50% ephemeral identity
The PTB source was set to the local FIPS address, which Linux ignores
(loopback PTB security check). Change the source to the original
packet's destination (the remote peer) so the kernel sees it as coming
from a remote router and honors the PMTU update.
Add test coverage for the PTB source address invariant in both the
icmp unit tests and the node integration tests.
The reverse delivery ratio was computed as cumulative_packets_recv /
highest_counter (lifetime values), while the forward ratio correctly
used per-interval deltas. This made ETX increasingly unresponsive over
session lifetime — early loss haunted the ratio forever, late loss
barely moved it.
Replace set_delivery_ratio_reverse() with update_reverse_delivery()
that tracks previous snapshot state and computes deltas, matching the
forward ratio's approach. Fixes GitHub issue #14 (Bug A).
After rekey cutover, frames encrypted with the old session can still
arrive during the 10s drain window. These carry large sender timestamps
from the old session, producing enormous transit deltas that spike the
EWMA jitter estimator (observed 2,000-7,000ms spikes on UDP links).
Add a time-based grace period (DRAIN_WINDOW_SECS + 5s = 15s) after
rekey that suppresses jitter updates until drain-window frames have
flushed. Baselines still update every frame so calculation resumes
cleanly after grace expires.
Fixes#10
Cherry-picked from v0l PR #6 (issue #5):
1. remove_active_peer() now removes the end-to-end session from
self.sessions when evicting a peer. The stale Established entry
caused initiate_session() to silently return Ok(()) via the
is_established() guard, preventing session re-establishment after
link-layer reconnection.
2. initiate_peer_connections() pre-seeds the identity cache from
configured peer npubs at startup, so TUN packets can be dispatched
immediately without waiting for handshake completion.
3. schedule_reconnect() preserves accumulated backoff when a retry
entry already exists, preventing exponential backoff reset on
repeated link-dead cycles.
Includes regression tests for all three fixes.
Rekey dual-initiation race (FMP + FSP):
When both sides' rekey timers fire simultaneously on high-latency links
(Tor ~700ms RTT), both msg1s cross in flight before dampening can
suppress the second initiation. Each node acts as both initiator and
responder, with set_pending_session() from the responder path destroying
the initiator's in-progress state. Each side ends up with a
pending_new_session from a different Noise handshake, causing AEAD
verification failure after cutover and link death. Fix: deterministic
tie-breaker in both FMP msg1 handler and FSP SessionSetup handler
(smaller NodeAddr wins as initiator). Also guard against pending session
overwrite from retransmitted msg1s.
Parent selection SRTT gate:
Peers without MMP RTT data are excluded from parent candidacy via
has_srtt() filter, preventing freshly reconnected peers from appearing
artificially attractive. First-RTT triggers immediate parent re-eval.
The gate in evaluate_parent() skips unmeasured candidates when any peer
has cost data, but falls back to default cost 1.0 during cold start
when no peer has MMP data yet.
Auto-reconnect on all peer removal paths:
The excessive-decryption-failure and peer-restart epoch-mismatch removal
paths were missing schedule_reconnect() calls, causing auto-connect peers
to be permanently abandoned after those events.
Control socket accessibility:
Socket and parent directory chowned to root:fips with mode 0770/0750 so
group members can use fipsctl/fipstop without root.
Log level changes:
Default RUST_LOG changed to debug in systemd service files. "Unknown
session index" log reduced from debug to trace.
Previously, static peers configured with AutoConnect gave up after 6
attempts (1 initial + 5 retries). If the remote peer was offline at
startup, the node permanently abandoned the connection. The reconnect
path (after MMP link-dead) already retried indefinitely but only
activated after a peer had been previously connected.
Remove the reconnect parameter from schedule_retry() and always set
reconnect=true when creating retry entries, since only auto-connect
peers reach this code path. The 300s backoff cap prevents resource waste.
The max_retries=0 config still works as an explicit kill switch.
The FSP XK rekey handshake has a race condition where the initiator
can cut over (K-bit flip) and send data encrypted with the new session
before msg3 reaches the responder. The responder has no pending session
yet, so K-bit detection fails and packets are dropped.
Defer FSP initiator cutover by 2 seconds after handshake completion
(FSP_CUTOVER_DELAY_MS) to give msg3 time to traverse the mesh. FMP
(IK, 2 messages) is unaffected since the responder completes during
msg1 processing.
Also fix MMP metric corruption after rekey cutover: the new session
starts with counter 0 but MMP state carries highest_counter and
GapTracker.expected_next from the old session, producing false
reorder counts, jitter spikes, and invalid OWD trends. Add
reset_for_rekey() methods that clear counter-dependent state while
preserving RTT estimates.
Additional fixes:
- Remove stale peers_by_index entry on abandon_rekey error path
- Replace redundant peers_by_index inserts with debug assertions
verifying the pre-registration invariant
- Tighten rekey integration test to zero tolerance (was 4 failures)
Add TorTransport in src/transport/tor/ supporting three operating modes:
Outbound (socks5 mode):
- Non-blocking SOCKS5 connect via tokio-socks with per-destination
circuit isolation (IsolateSOCKSAuth)
- TorAddr enum for .onion and clearnet address types
- Connection pool with per-connection receive tasks, reuses TCP
stream FMP framing
- connect_async()/connection_state_sync()/promote_connection() follow
the same non-blocking polling pattern as TCP transport
Inbound (directory mode — recommended for production):
- Tor manages the onion service via HiddenServiceDir in torrc
- FIPS reads .onion address from hostname file at startup
- No control port needed — enables Tor Sandbox 1 (seccomp-bpf)
- Accept loop mirrors TCP pattern with DirectoryServiceConfig
Monitoring (control_port mode and optional in directory mode):
- Async control port client supporting TCP and Unix socket connections
via Box<dyn AsyncRead/Write> trait objects
- AUTHENTICATE with cookie or password auth
- 8 GETINFO queries: bootstrap, circuits, traffic, liveness, version,
dormant state, SOCKS listeners
- Background monitoring task polls every 10s, caches TorMonitoringInfo
in Arc<RwLock> for synchronous query access
- Bootstrap milestone logging (25/50/75/100%), stall warning (>60s),
network liveness transitions, dormant mode entry
- Directory mode optionally connects to control port when control_addr
is configured (non-fatal on failure)
Operator visibility:
- show_transports query exposes tor_mode, onion_address, tor_monitoring
(bootstrap, circuit_established, traffic, liveness, version, dormant)
- fipstop transport detail view: Tor mode, onion address, SOCKS5/control
errors, connection stats, Tor daemon status section
- fipstop table view: tor(mode) label with truncated onion address hint
Security hardening:
- Per-destination circuit isolation via IsolateSOCKSAuth
- Unix socket default for control port (/run/tor/control)
- Reference torrc with HiddenServiceDir, VanguardsLiteEnabled,
ConnectionPadding, DoS protections (PoW + intro rate limiting)
Config:
- TorConfig with socks5, control_port, and directory modes
- DirectoryServiceConfig: hostname_file, bind_addr
- control_addr, control_auth, cookie_path, connect_timeout,
max_inbound_connections
Testing:
- 69 unit + integration tests with mock SOCKS5 and control servers
- Docker tests: socks5-outbound (clearnet via Tor) and directory-mode
(HiddenServiceDir onion service)
Documentation:
- Transport layer design doc: Tor architecture, directory mode
- Configuration doc: Tor config tables and examples
TCP (and future Tor) transports previously established connections
synchronously inside send(), blocking the node's RX event loop during
TCP handshake. This is particularly problematic for Tor where SOCKS5
circuit establishment can take 30-120 seconds.
Add a non-blocking connect path:
- ConnectionState enum in transport layer (None/Connecting/Connected/Failed)
- connect_async() on TcpTransport spawns background TCP connect task
- connection_state_sync() polls task completion, promotes to pool
- TransportHandle gains connect() and connection_state() dispatch methods
- Node tracks PendingConnect entries for connection-oriented transports
- initiate_connection() defers handshake for connection-oriented transports
- start_handshake() extracted as separate method for deferred invocation
- poll_pending_connects() in tick handler polls and completes handshakes
- Failed connects trigger retry via schedule_retry()
Connectionless transports (UDP, Ethernet) are unchanged — connect()
is a no-op and connection_state() always returns Connected.
The existing connect-on-send path in send_async() is preserved as
fallback for reconnection after connection drops.
811 tests pass (6 new), clippy clean.
Breaking wire format change: DataPacket payloads inside the AEAD envelope
now carry a 4-byte port header [src_port:2 LE][dst_port:2 LE] before the
service payload. The receiver dispatches by destination port.
Port multiplexing:
- send_session_data() takes src_port/dst_port params, prepends port header
- New send_ipv6_packet() compresses IPv6 header and sends on port 256
- Receive path dispatches DataPackets by port: port 256 decompresses IPv6
header from session context and delivers to TUN, unknown ports dropped
- Port constants: FSP_PORT_HEADER_SIZE (4 bytes), FSP_PORT_IPV6_SHIM (256)
IPv6 header compression:
- New ipv6_shim module with compress_ipv6()/decompress_ipv6() pure functions
- Strips src/dst addresses (32 bytes) and payload length (2 bytes) from each
packet, preserving traffic class, flow label, next header, and hop limit
as 6-byte residual fields
- Addresses reconstructed from session context on receive side
- Net savings: 29 bytes per packet (overhead 106 → 77 bytes)
- FIPS_IPV6_OVERHEAD constant (77 bytes), effective_ipv6_mtu() updated
- 16 unit tests for round-trip fidelity, field preservation, error cases
Documentation:
- fips-wire-formats: DataPacket port header, port registry, IPv6 shim
format tables, updated encapsulation walkthrough and overhead budget
- fips-ipv6-adapter: FIPS_IPV6_OVERHEAD (77 bytes), updated MTU numbers,
TUN reader/writer flow with compression steps, impl status
- fips-session-layer: port-based service dispatch section, data transfer
description, impl status
- fips-intro: IPv6 adapter as port 256 service, node architecture updated
- fips-mesh-operation: packet size summary with compressed overhead
- DataPacket doc updated with port header and dispatch model
- session_wire.rs module doc: DataPacket Port Multiplexing section
Compute network size estimate by summing bloom filter estimated entry
counts from the spanning tree parent (upward) and children (downward),
leveraging the tree's non-overlapping partition property. Uses the
standard formula n = -(m/k) * ln(1 - X/m) already in BloomFilter.
Surfaces the estimate in three places:
- show_status JSON: "estimated_mesh_size" field
- Periodic info log at MMP log interval (default 30s)
- fipstop Node dashboard State section as "mesh: ~N"
Add a HostMap that resolves human-readable hostnames to npubs,
enabling `gateway.fips` instead of the full `npub1...xyz.fips`.
Two sources populate the map: peer `alias` fields from the YAML
config and an operator-maintained hosts file at /etc/fips/hosts.
The DNS responder auto-reloads the hosts file on each request by
checking the file modification time, so operators can update
mappings without restarting the daemon.
- New src/upper/hosts.rs: HostMap, HostMapReloader, hostname
validation, hosts file parser with auto-reload on mtime change
- DNS resolver checks host map before falling back to direct npub
- Node uses host map for peer display names
- Default hosts file added to both .deb and tarball packaging
- 26 new tests (789 total)
Five tests covering the full TCP transport stack at the node level:
two-node handshake, three-node chain convergence with bloom filter
reachability, mixed UDP+TCP transport coexistence, MMP link-dead
detection after connection loss, and reconnection after link death
via connect-on-send.
Implement periodic full rekey at both protocol layers using fresh DH
key exchanges. Uses the existing K-bit flag (FLAG_KEY_EPOCH /
FSP_FLAG_K) to coordinate cutover between peers.
FMP layer (IK pattern):
- ActivePeer gains rekey state: pending/previous sessions, K-bit epoch
tracking, drain window, dampening timer
- Handshake state stored on ActivePeer with msg1 sent on existing link
- Encrypted frame handler detects K-bit flips, promotes pending
sessions, falls back to previous session during drain
- Handshake handlers distinguish rekey from new connections using
addr_to_link lookup with identity-based fallback
- Free all session indices (current, rekey, pending, previous) on
peer removal
FSP layer (XK pattern):
- SessionEntry gains parallel rekey fields with XK-specific state
for the 3-message handshake
- Route availability check before FSP rekey initiation
- Encrypted session handler adds K-bit flip detection and dual-session
decrypt fallback
- SessionSetup/Ack/Msg3 handlers extended for rekey paths
Defense-in-depth:
- Consecutive decryption failure detector (threshold=20) triggers
forced peer removal instead of waiting for link-dead timeout
- Identity-based rekey detection as fallback when addr_to_link
doesn't match (e.g., TCP ephemeral ports)
Configuration: RekeyConfig with enabled flag, after_secs (default 120),
and after_messages (default 65536) thresholds.
Logging: info for successful K-bit cutover completions, warn for
failures, debug for intermediate handshake steps, trace for routine
operations (resends, drain cleanup).
Rekey lifecycle:
1. Timer/counter fires -> initiator starts new handshake
2. Old session continues handling traffic during handshake
3. Handshake completes -> initiator cuts over, flips K-bit
4. Responder sees flipped K-bit -> promotes new session
5. Both keep old session for 10s drain window
6. After drain, old session discarded
Integration test: Docker-based multi-phase test exercising both FMP
and FSP rekey with aggressive timers (35s). Verifies connectivity
across all 20 directed pairs survives two consecutive rekey cycles.
Includes rekey topology, docker-compose profile, and CI matrix entry.
Increase ping test convergence wait from 3s to 5s for CI reliability.
The link-dead check required last_recv_time to be Some, so peers that
completed a handshake but never sent any data back (last_recv_time =
None) were silently skipped and lived forever as zombies. Fall back to
session_start when no frame has ever been received.
Implement hop-by-hop ECN congestion signaling through the FMP layer,
transport-level congestion detection via kernel drop counters, and
chaos harness integration for end-to-end validation.
FMP/session ECN plumbing:
- Thread ce_flag parsed at link layer through dispatch_link_message,
handle_session_datagram, handle_session_payload, and
handle_encrypted_session_msg to session delivery
- Replace hardcoded false in session-layer record_recv() with actual
ce_flag, activating ecn_ce_count tracking in session MMP
ECN congestion detection and CE relay:
- Add EcnConfig (node.ecn.*) with configurable loss_threshold (5%)
and etx_threshold (3.0) for transit congestion detection
- Add send_encrypted_link_message_with_ce() that ORs FLAG_CE into FMP
header flags; original method delegates with ce_flag=false
- Compute outgoing_ce = incoming_ce || local congestion on next-hop
link, enabling hop-by-hop CE relay through transit nodes
IPv6 ECN-CE marking:
- Mark ECN-CE (0b11) in IPv6 Traffic Class on received DataPackets
before TUN delivery when FMP CE flag is set
- Only marks ECN-capable packets (ECT(0)/ECT(1)); Not-ECT packets
unchanged per RFC 3168
Transport congestion abstraction and UDP kernel drop detection:
- Add TransportCongestion struct to transport layer for transport-
agnostic local congestion indicators
- Replace tokio::UdpSocket with AsyncFd<socket2::Socket> using
libc::recvmsg() with ancillary data parsing
- Enable SO_RXQ_OVFL for kernel receive buffer drop counter on every
packet, wiring up previously-stubbed UdpStats.kernel_drops
- Add TransportDropState for per-transport delta tracking with 1s
tick sampling via sample_transport_congestion()
- Extend detect_congestion() with transport kernel drop check
alongside MMP loss metrics
Congestion monitoring and control:
- Add CongestionStats (ce_forwarded, ce_received, congestion_detected,
kernel_drop_events) to NodeStats with snapshot serialization
- Wire counters into forwarding path, session handler, and transport
drop sampling with rate-limited warn logging (5s interval)
- Expose congestion data in show_routing control query and
ecn_ce_count in show_mmp peer entries
- Add congestion counters to fipstop routing tab in two-column layout
Chaos harness integration:
- Add query_routing(), query_transports(), snapshot_all_congestion()
to chaos control module
- Add congestion/kernel-drop log analysis in logs module
- Add congestion-stress scenario: 10-node tree, 1 Mbps bandwidth,
5-10% netem loss, heavy iperf3 traffic
- Add IngressConfig for tc ingress policing with per-peer policer
filters simulating upstream bandwidth bottlenecks
- Add iperf3 JSON result capture to traffic manager for throughput
measurement across scenarios
- Add ECN A/B test scenarios (ecn-ab-on/off.yaml) with ingress
policing and comparison script
- Enable TCP ECN negotiation (tcp_ecn=1 sysctl) in container
entrypoint for end-to-end CE propagation
Tests:
- 10 ECN unit/integration tests: mark_ipv6_ecn_ce variants, CE relay
chain (3-node propagation), EcnConfig serde roundtrip
- 3 transport drop congestion detection unit tests
Documentation:
- Update fips-mesh-layer.md: replace outdated CE Echo stub with full
ECN Congestion Signaling section covering detection logic, CE relay,
IPv6 marking, session tracking, and monitoring counters
- Update fips-configuration.md: add node.ecn.* parameter table and
ecn block in complete reference YAML
- Update fips-transport-layer.md: add Congestion Reporting section
with TransportCongestion struct, congestion() trait method, and
per-transport status; document AsyncFd/recvmsg/SO_RXQ_OVFL in UDP
- Update chaos README: add congestion/ECN scenario docs, ingress
traffic control, and iperf3 JSON capture sections
- Update README.md: add ECN to features list and "What works today";
update transport and tooling entries
Two bugs prevented Path MTU Discovery from working across heterogeneous
links (e.g., ethernet→UDP boundary):
1. ICMPv6 Packet Too Big wrote the MTU as u16 into a 32-bit field
(RFC 4443 §3.2), causing the kernel to read an inflated value.
Fixed by changing build_packet_too_big() to use u32 and writing
all 4 bytes.
2. handle_tun_outbound() only checked the local transport MTU, not
the per-destination PathMtuState updated by MtuExceeded signals.
Added a PathMtuState check after session lookup so subsequent
oversized packets generate ICMPv6 PTB on TUN instead of being
forwarded and dropped at the bottleneck hop.
Added integration test exercising the full PMTUD loop across a 3-node
chain with heterogeneous MTUs: oversized packet → forwarding failure →
MtuExceeded signal → PathMtuState update → ICMPv6 PTB on TUN.
evaluate_parent() did not check whether a candidate peer's ancestry
path already contained our own node_addr. Two nodes (e.g., sidecar and
VPS) could each select the other as parent, creating an alternating
coordinate loop that grew unbounded on each TreeAnnounce exchange.
Add loop detection in two places:
- evaluate_parent(): skip candidates whose ancestry contains us
- handle_tree_announce(): detect when current parent's updated ancestry
contains us and drop the parent instead of propagating the loop
Implement TCP transport for FIPS enabling firewall traversal and serving
as the foundation for future Tor transport. This is the first
connection-oriented transport in the system.
Key design decisions:
- FMP header-based framing: reuses existing 4-byte FMP common prefix for
packet boundary recovery with zero framing overhead
- Session survives TCP reconnection: Noise/MMP/FSP state bound to npub,
not TCP connection; MMP liveness is sole authority for peer death
- Connect-on-send: fresh connection on first send, transparent reconnect
- close_connection() trait method for cross-connection deduplication cleanup
New transport files:
- src/transport/tcp/mod.rs: TcpTransport, connection pool, accept loop
- src/transport/tcp/stream.rs: FMP-aware stream reader (shared with Tor)
Modified: transport trait (close_connection), TcpConfig, TransportHandle
match arms, create_transports(), initiate_connection() for connection-
oriented links, cross-connection tie-breaker cleanup, design docs.
Tree announce loop and TCP stability fixes:
- Preserve tree announce rate-limit state across reconnection: carry
forward last_tree_announce_sent_ms when a peer reconnects so the
rate-limit window isn't reset to zero
- Drop oversize TCP packets at sender: pre-send MTU check returns
MtuExceeded instead of writing to the stream, preventing receiver-side
connection teardown and reset-reconnect cycles
Chaos harness:
- TCP transport support: tcp_edges/has_tcp/tcp_peers in SimTopology,
transport-aware config_gen with per-edge transport type, TCP port 443,
pure-TCP node support
- Include all non-Ethernet edges in directed_outbound()
- Fix netem/links log messages to say "IP-based" instead of "UDP"
- Add tcp-chain, tcp-only, and tcp-mesh scenario files
Static harness:
- Transport-aware config generation (get_default_transport, transport_port)
- TCP transport injection via Python post-processing
- Add tcp-chain topology and docker-compose profile
Update the default UDP bind port from 4000 to 2121 (decimal) and the
default Ethernet EtherType from 0x88B5 to 0x2121 across all source
code, documentation, configuration templates, test fixtures, and
scripts. Remove references to "IEEE 802 experimental range" since
0x2121 is not in that range.
Implement raw Ethernet transport using AF_PACKET SOCK_DGRAM on Linux
with EtherType 0x88B5 (IEEE experimental range) and 1-byte frame type
prefix (0x00=data, 0x01=beacon).
Transport implementation:
- EthernetConfig with interface, ethertype, MTU, buffer sizes, and
four independent discovery knobs (discovery, announce, auto_connect,
accept_connections)
- PacketSocket/AsyncPacketSocket wrappers with ioctl helpers for
interface index, MAC address, and MTU queries
- EthernetTransport with Transport trait impl, async start/stop/send,
receive loop dispatching data frames and discovery beacons
- Discovery beacons (34 bytes: type + version + x-only pubkey) with
DiscoveryBuffer for peer accumulation and dedup
- Atomic statistics counters (frames, bytes, errors, beacons)
- Platform-gated with #[cfg(target_os = "linux")]
Transport-layer discovery integration:
- Promote auto_connect() and accept_connections() to Transport trait
with default implementations and TransportHandle dispatch
- Extract initiate_connection() so both static peer config and
discovery auto-connect share the same handshake initiation path
- Add poll_transport_discovery() to the tick handler to drain
discovery buffers and auto-connect to discovered peers
- Enforce accept_connections() in handle_msg1() — transports with
accept_connections=false silently drop inbound handshakes
Node integration:
- create_transports() handles Ethernet named instances
- resolve_ethernet_addr() parses "interface/mac" address format
- transport_mtu() generalized for multi-transport operation
Test harness:
- VethPair RAII struct for veth pair lifecycle management
- Three #[ignore] integration tests requiring root/CAP_NET_RAW:
two-node handshake, data exchange, mixed transport coexistence
- Chaos harness: transport-aware topology model, VethManager for
veth pairs between Docker containers, Ethernet-aware config gen,
netem split (HTB+u32 for UDP, root netem for veth), transport-aware
link flaps and node churn with veth re-setup
- Container entrypoint waits for configured Ethernet interfaces
before starting FIPS (handles veth creation timing)
- New scenarios: ethernet-only (4-node ring), ethernet-mesh (6-node
mixed UDP+Ethernet with netem and link flaps)
Documentation:
- fips-transport-layer.md: Ethernet section, beacon discovery, WiFi
compatibility, updated discovery state, trait surface additions,
implementation status table
- fips-configuration.md: Ethernet parameter table, named instances,
peer address format, mixed UDP+Ethernet example, complete reference
- fips-wire-formats.md: Ethernet frame type prefix note
Bump rand (0.8→0.10), rtnetlink (0.14→0.20), tun (0.7→0.8),
simple-dns (0.9→0.11), socket2 (0.5→0.6), and criterion (0.5→0.8).
Migrate all rand call sites: thread_rng()→rng(), gen()→random(),
gen_range()→random_range(), RngCore→Rng trait. Work around secp256k1
0.30 requiring rand 0.8 by generating random bytes directly and
constructing SecretKey from slice.
Migrate rtnetlink to builder-based API: LinkSetRequest replaced with
LinkUnspec builder + change(), RouteAddRequest replaced with
RouteMessageBuilder.
Remove bloom benchmark (criterion 0.8 incompatible with old harness
config).
Track parent switch frequency in a sliding window. When switches exceed
a configurable threshold (default 4 in 60s), impose an extended hold-down
period (default 120s) that prevents non-mandatory parent changes.
Mandatory switches (parent loss, root change, shouldn't-be-root) bypass
dampening. The flap counter resets when the window expires naturally.
Implements TASK-2026-0030 / IDEA-0013.
Track consecutive send failures in SenderState. Apply 2^n backoff
multiplier (capped at 32x) to the report interval. Suppress debug
logs after 3 consecutive failures, emit recovery summary on success.
Add per-peer replay suppression counter to ActivePeer. Log the first 3
replay detections at DEBUG, then suppress with a one-time notice. Emit
a summary count on session replacement or peer removal.
Non-replay decryption errors continue to be logged unconditionally.
Guard discovery triggers in handle_path_broken() and
handle_coords_required() with has_cached_identity() check. When the
XK responder receives an error signal before msg3 completes, the
initiator's identity is unknown, making LookupResponse proof
verification impossible. Skip discovery in this case — the handshake
retry mechanism handles recovery.
Downgrade the identity_cache miss log from ERROR to WARN since it's a
known race condition, not a bug.
Cost-based parent selection:
- Replace depth-only parent selection with effective_depth = depth + link_cost
- link_cost computed from locally measured MMP metrics: etx * (1.0 + srtt_ms / 100.0)
- Prevents bottleneck subtrees in heterogeneous networks where a LoRa link
at depth 1 would otherwise always beat fiber at depth 2
- Configurable hysteresis (default 0.2) prevents marginal parent switches
- Configurable hold-down timer (default 30s) suppresses re-evaluation
after parent switch
- Mandatory switches (parent lost, root change) bypass both safeguards
- Link costs passed as HashMap parameter to keep TreeState pure
Periodic re-evaluation:
- evaluate_parent() was only called on TreeAnnounce receipt or parent loss;
after tree stabilization, link degradation went undetected
- Added timer-based re-evaluation (reeval_interval_secs, default 60s) that
calls evaluate_parent() from the tick handler with current MMP link costs
- Respects existing hold-down and hysteresis safeguards
- Short-circuits when disabled or <2 peers
Design documentation:
- Update 7 design docs to reflect cost-based parent selection
- Replace depth-only algorithm descriptions with effective_depth model
- Replace rejected cumulative path cost spec with local-only design rationale
- Rewrite Example 2 (heterogeneous links) for local-only cost model
- Update config docs: parent_switch_threshold replaced by parent_hysteresis,
hold_down_secs, reeval_interval_secs
Chaos simulation enhancements:
- fips_overrides with deep merge for per-scenario FIPS config customization
- Explicit topology algorithm for deterministic test graphs
- Control socket querying via fipsctl for tree/MMP snapshot collection
- Edge existence validation in netem manager
- Per-link netem policy overrides
- 9 new chaos scenarios covering cost avoidance, depth-vs-cost tradeoffs,
stability, mixed topologies, periodic re-evaluation, and bottleneck parent
12 new unit tests, 667 total passing, clippy clean.
Gate peer_inbound_filters() to only collect from tree peers (parent
and children), so outgoing filter computation merges only tree-sourced
information. All peers still receive FilterAnnounce messages and store
filters locally for routing queries — the restriction is only on what
gets merged into outgoing filters.
This prevents bloom filter saturation where mesh shortcuts cause every
node's filter to converge toward the full network. With tree-only
merge, filters contain subtree (from children) + complement (from
parent) + single-hop mesh views.
Implementation:
- Add is_tree_peer() helper to determine tree parent/child relationship
- Gate peer_inbound_filters() to tree peers only (single control point)
- Trigger bloom filter exchange on tree relationship changes
- Add est_entries, set_bits, fill ratio, and tree_peer fields to
FilterAnnounce send/receive debug logs
- Add test_bloom_filter_split_horizon test verifying directional
asymmetry: upward filters contain only the child's subtree, downward
filters contain only the complement
- Add print_filter_cardinality diagnostic helper for test inspection
Design docs:
- fips-bloom-filters.md: Add directional asymmetry and mesh peer filter
subsections, update per-peer filter model, saturation mitigation,
implementation status table
- fips-mesh-operation.md: Update filter propagation description, add
directional asymmetry, tree relationship change trigger
- fips-intro.md: Rewrite bloom propagation paragraph for tree-only merge
The session-layer handshake now uses the 3-message XK pattern instead
of the 2-message IK pattern, providing stronger initiator identity
hiding. The initiator static key is deferred to msg3 and encrypted
under the es+ee DH chain, so eavesdroppers cannot identify the
initiator from the handshake.
XK pattern: -> e, es (msg1) / <- e, ee + epoch (msg2) / -> s, se + epoch (msg3)
Key changes:
- Add XK handshake methods alongside existing IK methods in noise module
- Add SessionMsg3 wire format and FSP_PHASE_MSG3 (0x03) prefix
- Replace Responding state with AwaitingMsg3 in session state machine
- Rewrite session handlers: handle_session_setup defers identity to msg3,
handle_session_ack processes msg2 and sends msg3, new handle_session_msg3
completes the responder handshake and registers identity
- Link-layer (FMP) continues to use Noise IK unchanged
- Add comprehensive XK unit tests and update all integration tests
Add min_mtu (u16) to LookupRequest and path_mtu (u16) to
LookupResponse, enabling the discovery system to report transport
MTU capability along the lookup path.
LookupRequest carries min_mtu (origin's minimum MTU requirement,
default 0 = no requirement). LookupResponse carries path_mtu
(initialized to u16::MAX by the target, reduced by transit nodes
via min(path_mtu, outgoing_link_mtu) on the reverse path).
path_mtu is a transit annotation like SessionDatagram.path_mtu and
is NOT included in the proof signature. The originator stores the
discovered path_mtu in CacheEntry alongside cached coordinates.
Wire format: +2 bytes each for LookupRequest and LookupResponse.
Add a new session-layer error signal that transit routers send back to
the source when a forwarded packet exceeds the next-hop transport MTU.
This complements the existing proactive path MTU discovery (min'd at
each hop) by providing immediate feedback when oversized packets are
dropped, closing the transient window before the proactive mechanism
converges.
Wire format: 36-byte payload (msg_type + flags + dest_addr + reporter +
mtu) with FSP phase=0x0 and U flag set, matching the existing
CoordsRequired/PathBroken pattern.
Changes:
- Add SessionMessageType::MtuExceeded (0x22) and MtuExceeded struct with
encode/decode methods to protocol/session.rs
- Add NodeError::MtuExceeded variant to propagate structured MTU info
from TransportError through send_encrypted_link_message()
- Catch MtuExceeded in the forwarding path and send error signal back to
the datagram source via send_mtu_exceeded_error(), rate-limited by the
existing routing_error_rate_limiter
- Handle incoming MtuExceeded at the source by calling
PathMtuState::apply_notification() for immediate MTU decrease
- Add unit tests for encode/decode roundtrip, boundary MTU values, and
too-short payload rejection
The target node's send_lookup_response() was using greedy tree routing
(find_next_hop) as the primary method to route responses back to the
origin. When the target had the origin's coords cached from a prior
lookup, find_next_hop would route the response to whichever peer was
closest to the origin in tree space -- which might not have been on the
request's forward path. That peer would have no recent_requests entry
for the request_id, causing it to treat the response as if it were the
originator, fail identity_cache lookup, and discard the response.
Fix: prefer the reverse-path (recent_requests.from_peer) as the
primary routing method for the first hop, falling back to
find_next_hop only if no recent_request entry exists.
Also adds diagnostic output on failure to aid future debugging.
Add a Unix domain socket interface for querying node state at runtime.
A spawned tokio task accepts connections and communicates with the main
event loop via mpsc/oneshot channels, keeping all Node access
single-threaded.
Includes:
- src/control/ module with socket lifecycle, JSON protocol, and 11
query handlers (status, peers, links, tree, sessions, bloom, mmp,
cache, connections, transports, routing)
- Separate fipsctl binary for CLI queries (fipsctl show <command>)
- ControlConfig in node configuration (enabled, socket_path)
- Integration into the main select! event loop
Add link_mtu(&TransportAddr) method to the Transport trait with a
default implementation that falls back to the transport-wide mtu().
This enables transports like BLE to report per-connection MTU values
while maintaining backward compatibility for UDP and other transports
that use a single MTU for all links.
Update the forwarding and session send paths to query link_mtu() with
the next-hop peer's current address, falling back to transport-wide
mtu() when no address is available.
Each node generates a random 8-byte startup epoch, encrypted inside
both Noise IK handshake messages (msg1 and msg2). When a peer's msg1
arrives with a different epoch than the stored value, the node tears
down the stale session and processes the msg1 as a new connection,
enabling near-instant restart detection instead of the 30-second
dead timeout.
Wire format impact:
- msg1: 82 -> 106 bytes (added 24-byte encrypted epoch after ss DH)
- msg2: 33 -> 57 bytes (added 24-byte encrypted epoch after se DH)
- Wire msg1: 90 -> 114 bytes, wire msg2: 45 -> 69 bytes
Include target_coords in proof_bytes() signed data to prevent transit
nodes from substituting fake coordinates. Add mandatory signature
verification at the originator using the target's public key from
identity_cache (guaranteed available since lookups are only initiated
from contexts where the key is already cached).
Verification failure discards the response. Identity cache miss (should
never happen) logs an error and discards. Add four new tests covering
verification success, failure, cache miss, and coordinate substitution
detection.
Rename FIPS Link Protocol (FLP) to FIPS Mesh Protocol (FMP)
The "Link Protocol" name understated the layer's scope — spanning tree
construction, bloom filter routing, greedy forwarding, and mesh-wide
coordination go well beyond link-level concerns. Rename fips-link-layer.md
to fips-mesh-layer.md, update FLP→FMP throughout docs and source code
(FLP_VERSION→FMP_VERSION, wire.rs, rx_loop.rs, spanning_tree.rs).
New SVG illustrations
- Protocol stack: color-coded layer diagram replacing ASCII art
- OSI mapping: side-by-side comparison with traditional networking layers
- Bloom filter propagation: 6-node tree with sender-colored filter boxes
showing split-horizon computation per link
- Routing decision flowchart: 5-step priority chain with candidate ranking
by tree distance and link performance
- Coordinate discovery: sequence diagram showing LookupRequest propagation,
response caching, and SessionSetup cache warming
Redesigned existing SVGs
- Architecture overview: uniform node layout, U-shaped encrypted link
connectors, separate end-to-end session line
- Node architecture: split Router Core into FSP and FMP layers, reorganize
transports into Overlay/Shared Medium/Point-to-Point categories
- Identity derivation: wider boxes, visible encode arrow, dashed npub line
fips-intro.md revisions
- Add inline references to prior work: Yggdrasil/Ironwood for coordinate
routing, Noise Protocol Framework for IK handshakes, WireGuard for
index-based session dispatch, Wikipedia for bloom filters, split-horizon,
and greedy embedding
- Add explanatory paragraphs after bloom filter diagram describing
split-horizon filter computation and candidate selection behavior
- Simplify transport abstraction language, remove I2P/LoRa references
- Fix LookupRequest wording ("propagates" not "floods"), note intermediate
node coordinate caching on lookup responses
- Rewrite architecture overview prose to match redesigned diagrams
SessionAck previously only carried the responder's coordinates
(src_coords). When the return path diverged from the forward path
(e.g., after tree reconvergence), transit nodes on the return path
lacked the initiator's coordinates and couldn't route the SessionAck
back, causing handshake timeouts.
Add dest_coords (initiator's coordinates) to the SessionAck wire
format, mirroring SessionSetup's design. Transit nodes now cache both
endpoints' coordinates when forwarding a SessionAck, making the return
path self-sufficient regardless of path asymmetry.
Root cause confirmed by churn-20 sim log analysis: the n04-n14
handshake failure was caused by n15 (return-path transit) lacking
n04's coordinates, not by stale tree routes through a downed node.
Auto-reconnect:
- Add per-peer auto_reconnect config (default true) to PeerConfig
- schedule_reconnect() feeds removed peers back into retry system with
unlimited retries and exponential backoff after MMP dead timeout
- RetryState gains reconnect flag to distinguish startup retries
(max_retries-limited) from auto-reconnect (unlimited)
Retry re-fire fix:
- process_pending_retries() now pushes retry_after_ms past the handshake
timeout window after successful initiate_peer_connection(), preventing
retries from firing every tick with no backoff
Chaos sim improvements:
- Directed outbound configs: BFS spanning tree + lower-ID-first assignment
eliminates dual-connect race conditions in simulation
- Save runner log (runner.log) alongside per-node logs for event correlation
- Increase churn-20 traffic aggressiveness and node churn (max_down_nodes
3→5, traffic interval min 0s, duration max 90s, concurrent flows 5→10)
handle_session_ack() had a bug where take_state() followed by
a non-Initiating match would re-insert the entry with state=None,
causing panics on subsequent .state()/.state_mut() calls. Fix by
checking is_initiating() before take_state().
Add safe is_initiating()/is_responding() to SessionEntry (matching
existing is_established()) and convert all production .state() callers
to use these None-safe methods. Gate .state() with #[cfg(test)].
Johnathan Corgan
KieranandJohnathan Corgan