The entrypoint HTTP server was binding port 80 but gateway-test.sh
curls port 8000. Maint already had 8000; the wrong port was introduced
during a merge to master.
macOS platform:
- Platform-native TUN interface management with shutdown pipe
- Raw Ethernet transport with macOS socket backend (socket_macos.rs)
- EthernetTransport and TransportHandle::Ethernet ungated from Linux-only
- macOS .pkg packaging (build-pkg.sh, launchd plist, uninstall script)
- CI: macOS build and unit test jobs; x86_64 cross-compiled from
macos-latest via rustup target add x86_64-apple-darwin
Gateway feature flag:
- New opt-in `gateway` Cargo feature activates optional `rustables` dep
- `pub mod gateway` and `Config.gateway` gated behind the feature so
macOS builds never pull in Linux-only nftables bindings
- `fips-gateway` bin has `required-features = ["gateway"]`
- All Linux/OpenWrt/AUR packaging passes `--features gateway`
CI / packaging:
- package-linux, package-macos, package-openwrt now trigger on push to
master/maint/next and on pull requests; release uploads remain tag-gated
- Bloom filter routing fix: fall through to tree routing when no candidate
is strictly closer
- MMP intervals: raise MIN to 1s / MAX to 5s with 5-sample cold-start phase
Add fips-gateway binary: a separate daemon that allows unmodified LAN
hosts to reach FIPS mesh destinations via DNS-allocated virtual IPs
and kernel nftables NAT.
Gateway DNS resolver: forwarding proxy on [::]:53 that intercepts
.fips queries, forwards to daemon resolver (localhost:5354), allocates
virtual IPs from pool, returns AAAA records. Always sends AAAA upstream
regardless of client query type, returns proper NODATA for non-AAAA.
Virtual IP pool: fd01::/112 pool with state machine lifecycle
(Allocated → Active → Draining → Free), TTL-based reclamation,
conntrack integration for session tracking.
NAT manager: nftables DNAT/SNAT rules via rustables netlink API,
per-mapping rule lifecycle, fips0 masquerade for LAN client source
address rewriting.
Network setup: local pool route, proxy NDP for virtual IPs on LAN
interface, IPv6 forwarding validation.
Control socket at /run/fips/gateway.sock with show_gateway and
show_mappings queries. fipstop Gateway tab with pool summary gauge
and mappings table.
Gateway config section in fips.yaml with pool CIDR, LAN interface,
DNS upstream, TTL, and grace period settings.
Design doc at docs/design/fips-gateway.md.
Integration test (testing/static/scripts/gateway-test.sh): three
containers verifying DNS resolution, end-to-end HTTP, NAT state,
TTL expiration, SERVFAIL fallback, and clean shutdown.
Wait for all nodes to reach their expected peer counts instead of
only checking a single node. This prevents false failures on slower
CI runners where remote nodes (especially node E in mesh/chain
topologies) take longer to establish all links.
Wait for all nodes to reach their expected peer counts instead of
only checking a single node. This prevents false failures on slower
CI runners where remote nodes (especially node E in mesh/chain
topologies) take longer to establish all links.
BLE transport implementation using L2CAP Connection-Oriented Channels
(SeqPacket mode) via the bluer crate, behind cfg(feature = "ble").
Core transport:
- BleTransport<I> generic over BleIo trait (BluerIo prod, MockBleIo test)
- Connection pool with priority eviction (static > discovered, max 7)
- Connect-on-send via connect_inline() matching TCP behavior
- Per-connection receive loops with pool cleanup on disconnect
Discovery and probing:
- Combined scan_probe_loop using select! over scanner events and a
BinaryHeap delay queue with per-entry random jitter (0-5s) to prevent
herd effects when multiple nodes see the same beacon simultaneously
- Pre-handshake pubkey exchange ([0x00][pubkey:32]) for IK identity
- Cross-probe tie-breaker: smaller NodeAddr's outbound wins (same
convention as FMP/FSP rekey dual-initiation)
- Probed peers reported to DiscoveryBuffer; pool fills through normal
node-layer auto-connect -> send_async -> connect_inline path
Beacon management:
- Periodic advertising: 1s burst every 30s (configurable via
beacon_interval_secs / beacon_duration_secs)
- FIPS service UUID for scan filtering
Configuration (all fields optional with defaults):
- adapter, psm, mtu, max_connections, connect_timeout_ms
- advertise, scan, auto_connect, accept_connections
- beacon_interval_secs (30), beacon_duration_secs (1)
Hardware validated with two BLE nodes:
- 2048-byte MTU, ~60-160ms RTT, zero-config auto-connect
- BLE spike tool at testing/ble/ for standalone adapter validation
42 unit tests + 4 node-level integration tests, all CI-compatible
via MockBleIo (no hardware required). tokio test-util added for
time-dependent scan/probe tests.
Phase 1 pre-rekey baseline was failing intermittently on CI runners
because 5s wasn't enough for multi-hop discovery (B→D requires
B→C→D). The mesh always converged by Phase 3, confirming this was
purely a timing issue.
The rekey test greps container logs for initiator cutover messages that
were demoted to debug level. Override RUST_LOG in the rekey profile to
enable debug for fips::node::handlers::rekey so the test assertions
can find them.
Replace discovery flooding with bloom-filter-guided tree routing:
lookups sent only to tree peers (parent + children) whose bloom
filter contains the target. If no tree peer matches, fall back to
non-tree peers with bloom matches before dropping the request. This
produces single-path forwarding through the spanning tree (90%
traffic reduction) while recovering from dead ends caused by stale
bloom filters, tree restructuring, or transit node failures.
Remove visited bloom filter from LookupRequest wire format (-257
bytes per request). Tree routing is inherently loop-free; request_id
dedup handles edge cases during tree restructuring. Add response-
forwarded flag to prevent response routing loops from convergent
request paths.
Add originator-side exponential backoff (30s base, 300s cap) after
lookup timeouts and bloom misses. Backoff resets on topology changes
(parent switch, new peer, first RTT, reconnection).
Add transit-side per-target rate limiting (2s minimum interval) for
forwarded lookups as defense-in-depth.
Add discovery retry within the timeout window (default: send at T=0,
retry at T=5s, fail at T=10s) to compensate for single-path fragility.
Lookups with zero eligible tree peers fail immediately.
Improve discovery logging: promote key events to info (initiation,
success, timeout with failure count). Add debug logging for dedup,
pending packet retry, backoff suppression, forward rate limiting,
and backoff reset.
New config: discovery.backoff_base_secs, backoff_max_secs,
forward_min_interval_secs, retry_interval_secs, max_attempts.
New stats: req_backoff_suppressed, req_forward_rate_limited,
req_bloom_miss, req_no_tree_peer, req_fallback_forwarded.
Removed: req_already_visited, visited bloom filter.
Three infrastructure bugs fixed:
- DNS resolution never populated the identity cache because Docker
overwrites /etc/resolv.conf at container start. Fix: bind-mount
resolv.conf in the chaos compose template, matching the static and
sidecar test configs.
- Traffic generator used config-time npubs for iperf3 targets, but
ephemeral nodes generate fresh keypairs at startup. Fix: share the
peer churn manager's runtime npub cache with the traffic manager.
- Log analysis had no discovery counters and used wrong patterns.
Fix: add discovery section with correct log message matching.
Also adds timestamped output directories (YYYYMMDD-HHMMSS-scenario),
coord_ttl_secs override in maelstrom.yaml, FIPS_SIM_OUTPUT env var
support, and maelstrom-sparse scenario for sparse topology testing.
Add runtime peer management to the FIPS daemon via control socket
commands, and a new chaos simulation scenario that exercises dynamic
topology mutation with ephemeral node identities.
Daemon (connect/disconnect commands):
- Extend control socket Request with optional params field
- Add commands.rs module for mutating command dispatch, separate from
read-only queries
- Add api_connect() on Node: builds ephemeral PeerConfig (no auto-
reconnect), pre-seeds identity cache, reuses initiate_peer_connection
- Add api_disconnect() on Node: calls remove_active_peer(), clears
retry_pending to suppress reconnection
- Route non-show_* commands to async command dispatch in rx_loop
fipsctl CLI:
- Add Connect and Disconnect subcommands accepting npub or hostname
- Resolve hostnames from /etc/fips/hosts before sending to daemon
- Refactor socket I/O into reusable send_request helper
Chaos simulator (maelstrom scenario):
- Add PeerChurnManager: periodically disconnects a random active link
and connects a random unconnected node pair via control socket
- Add send_command() to control.py using base64-encoded JSON payloads
to avoid shell quoting issues in docker exec
- Add PeerChurnConfig to scenario with interval and ephemeral_fraction
- Ephemeral identity support: nodes configured without nsec generate
fresh keypairs on restart; simulator queries show_status for new
npub and updates its cache via on_node_restart callback
- Add maelstrom.yaml: all chaos dimensions (netem, link flaps, node
churn, peer topology churn, traffic) with 50% ephemeral identity
The K8s sidecar is a standalone example, not a test harness — it has
no CI integration unlike testing/sidecar/. Move it to examples/
alongside sidecar-nostr-relay/ where it belongs. Update internal path
references in Dockerfile, build.sh, and README.
Adds a complete Kubernetes sidecar setup under testing/k8s/ that runs
FIPS as a sidecar container, injecting the mesh TUN interface (fips0)
into any co-located app container via shared pod network namespace.
- Multi-stage Dockerfile builds fips and fipsctl from source (debian
trixie / rust slim-trixie), producing a minimal runtime image with
iproute2, iptables, dnsmasq, and the two binaries
- entrypoint.sh generates fips.yaml from environment variables, rewrites
/etc/resolv.conf to route .fips DNS through dnsmasq, applies optional
iptables isolation, clamps TCP MSS, then starts dnsmasq and execs the
daemon
- pod.yaml annotated example Pod manifest with Secret, sysctl tuning,
readiness/liveness probes, and resource limits
- scripts/build.sh convenience wrapper for docker build
- testing/k8s/.dockerignore keeps the build context small
- README.md usage guide covering quick start, isolation modes, env vars,
multi-peer JSON config, troubleshooting
The FSP XK rekey handshake has a race condition where the initiator
can cut over (K-bit flip) and send data encrypted with the new session
before msg3 reaches the responder. The responder has no pending session
yet, so K-bit detection fails and packets are dropped.
Defer FSP initiator cutover by 2 seconds after handshake completion
(FSP_CUTOVER_DELAY_MS) to give msg3 time to traverse the mesh. FMP
(IK, 2 messages) is unaffected since the responder completes during
msg1 processing.
Also fix MMP metric corruption after rekey cutover: the new session
starts with counter 0 but MMP state carries highest_counter and
GapTracker.expected_next from the old session, producing false
reorder counts, jitter spikes, and invalid OWD trends. Add
reset_for_rekey() methods that clear counter-dependent state while
preserving RTT estimates.
Additional fixes:
- Remove stale peers_by_index entry on abandon_rekey error path
- Replace redundant peers_by_index inserts with debug assertions
verifying the pre-registration invariant
- Tighten rekey integration test to zero tolerance (was 4 failures)
Add TorTransport in src/transport/tor/ supporting three operating modes:
Outbound (socks5 mode):
- Non-blocking SOCKS5 connect via tokio-socks with per-destination
circuit isolation (IsolateSOCKSAuth)
- TorAddr enum for .onion and clearnet address types
- Connection pool with per-connection receive tasks, reuses TCP
stream FMP framing
- connect_async()/connection_state_sync()/promote_connection() follow
the same non-blocking polling pattern as TCP transport
Inbound (directory mode — recommended for production):
- Tor manages the onion service via HiddenServiceDir in torrc
- FIPS reads .onion address from hostname file at startup
- No control port needed — enables Tor Sandbox 1 (seccomp-bpf)
- Accept loop mirrors TCP pattern with DirectoryServiceConfig
Monitoring (control_port mode and optional in directory mode):
- Async control port client supporting TCP and Unix socket connections
via Box<dyn AsyncRead/Write> trait objects
- AUTHENTICATE with cookie or password auth
- 8 GETINFO queries: bootstrap, circuits, traffic, liveness, version,
dormant state, SOCKS listeners
- Background monitoring task polls every 10s, caches TorMonitoringInfo
in Arc<RwLock> for synchronous query access
- Bootstrap milestone logging (25/50/75/100%), stall warning (>60s),
network liveness transitions, dormant mode entry
- Directory mode optionally connects to control port when control_addr
is configured (non-fatal on failure)
Operator visibility:
- show_transports query exposes tor_mode, onion_address, tor_monitoring
(bootstrap, circuit_established, traffic, liveness, version, dormant)
- fipstop transport detail view: Tor mode, onion address, SOCKS5/control
errors, connection stats, Tor daemon status section
- fipstop table view: tor(mode) label with truncated onion address hint
Security hardening:
- Per-destination circuit isolation via IsolateSOCKSAuth
- Unix socket default for control port (/run/tor/control)
- Reference torrc with HiddenServiceDir, VanguardsLiteEnabled,
ConnectionPadding, DoS protections (PoW + intro rate limiting)
Config:
- TorConfig with socks5, control_port, and directory modes
- DirectoryServiceConfig: hostname_file, bind_addr
- control_addr, control_auth, cookie_path, connect_timeout,
max_inbound_connections
Testing:
- 69 unit + integration tests with mock SOCKS5 and control servers
- Docker tests: socks5-outbound (clearnet via Tor) and directory-mode
(HiddenServiceDir onion service)
Documentation:
- Transport layer design doc: Tor architecture, directory mode
- Configuration doc: Tor config tables and examples
Update 9 documentation files to match current implementation:
- Add missing rekey config section (node.rekey.*) and host mapping
section to fips-configuration.md
- Update Ethernet frame format from [type:1][payload] to
[type:1][length:2 LE][payload] in wire-formats and transport docs
- Fix Ethernet effective MTU from interface-1 to interface-3
- Mark rekey as Implemented in mesh-layer and session-layer status tables
- Change TCP default port examples from 443 to 8443
- Add rekey, persistent identity, host mapping, mesh size estimation to
README features and status sections
- Update chaos scenario count from 16 to 20, add rekey topology to
static test docs
Phase 3 (post-first-rekey connectivity) can see brief disruptions while
links are mid-cutover. Add a 5s settle time after the rekey wait and
allow up to 4 transient pair failures in Phase 3. Phase 5 remains
strict, ensuring full recovery after the second rekey cycle.
Update the external pub node (vps-chi at 217.77.8.91) to use its
real persistent npub instead of a fabricated test key. Remove the
unnecessary nsec from the external node definition.
Implement periodic full rekey at both protocol layers using fresh DH
key exchanges. Uses the existing K-bit flag (FLAG_KEY_EPOCH /
FSP_FLAG_K) to coordinate cutover between peers.
FMP layer (IK pattern):
- ActivePeer gains rekey state: pending/previous sessions, K-bit epoch
tracking, drain window, dampening timer
- Handshake state stored on ActivePeer with msg1 sent on existing link
- Encrypted frame handler detects K-bit flips, promotes pending
sessions, falls back to previous session during drain
- Handshake handlers distinguish rekey from new connections using
addr_to_link lookup with identity-based fallback
- Free all session indices (current, rekey, pending, previous) on
peer removal
FSP layer (XK pattern):
- SessionEntry gains parallel rekey fields with XK-specific state
for the 3-message handshake
- Route availability check before FSP rekey initiation
- Encrypted session handler adds K-bit flip detection and dual-session
decrypt fallback
- SessionSetup/Ack/Msg3 handlers extended for rekey paths
Defense-in-depth:
- Consecutive decryption failure detector (threshold=20) triggers
forced peer removal instead of waiting for link-dead timeout
- Identity-based rekey detection as fallback when addr_to_link
doesn't match (e.g., TCP ephemeral ports)
Configuration: RekeyConfig with enabled flag, after_secs (default 120),
and after_messages (default 65536) thresholds.
Logging: info for successful K-bit cutover completions, warn for
failures, debug for intermediate handshake steps, trace for routine
operations (resends, drain cleanup).
Rekey lifecycle:
1. Timer/counter fires -> initiator starts new handshake
2. Old session continues handling traffic during handshake
3. Handshake completes -> initiator cuts over, flips K-bit
4. Responder sees flipped K-bit -> promotes new session
5. Both keep old session for 10s drain window
6. After drain, old session discarded
Integration test: Docker-based multi-phase test exercising both FMP
and FSP rekey with aggressive timers (35s). Verifies connectivity
across all 20 directed pairs survives two consecutive rekey cycles.
Includes rekey topology, docker-compose profile, and CI matrix entry.
Increase ping test convergence wait from 3s to 5s for CI reliability.
Implement hop-by-hop ECN congestion signaling through the FMP layer,
transport-level congestion detection via kernel drop counters, and
chaos harness integration for end-to-end validation.
FMP/session ECN plumbing:
- Thread ce_flag parsed at link layer through dispatch_link_message,
handle_session_datagram, handle_session_payload, and
handle_encrypted_session_msg to session delivery
- Replace hardcoded false in session-layer record_recv() with actual
ce_flag, activating ecn_ce_count tracking in session MMP
ECN congestion detection and CE relay:
- Add EcnConfig (node.ecn.*) with configurable loss_threshold (5%)
and etx_threshold (3.0) for transit congestion detection
- Add send_encrypted_link_message_with_ce() that ORs FLAG_CE into FMP
header flags; original method delegates with ce_flag=false
- Compute outgoing_ce = incoming_ce || local congestion on next-hop
link, enabling hop-by-hop CE relay through transit nodes
IPv6 ECN-CE marking:
- Mark ECN-CE (0b11) in IPv6 Traffic Class on received DataPackets
before TUN delivery when FMP CE flag is set
- Only marks ECN-capable packets (ECT(0)/ECT(1)); Not-ECT packets
unchanged per RFC 3168
Transport congestion abstraction and UDP kernel drop detection:
- Add TransportCongestion struct to transport layer for transport-
agnostic local congestion indicators
- Replace tokio::UdpSocket with AsyncFd<socket2::Socket> using
libc::recvmsg() with ancillary data parsing
- Enable SO_RXQ_OVFL for kernel receive buffer drop counter on every
packet, wiring up previously-stubbed UdpStats.kernel_drops
- Add TransportDropState for per-transport delta tracking with 1s
tick sampling via sample_transport_congestion()
- Extend detect_congestion() with transport kernel drop check
alongside MMP loss metrics
Congestion monitoring and control:
- Add CongestionStats (ce_forwarded, ce_received, congestion_detected,
kernel_drop_events) to NodeStats with snapshot serialization
- Wire counters into forwarding path, session handler, and transport
drop sampling with rate-limited warn logging (5s interval)
- Expose congestion data in show_routing control query and
ecn_ce_count in show_mmp peer entries
- Add congestion counters to fipstop routing tab in two-column layout
Chaos harness integration:
- Add query_routing(), query_transports(), snapshot_all_congestion()
to chaos control module
- Add congestion/kernel-drop log analysis in logs module
- Add congestion-stress scenario: 10-node tree, 1 Mbps bandwidth,
5-10% netem loss, heavy iperf3 traffic
- Add IngressConfig for tc ingress policing with per-peer policer
filters simulating upstream bandwidth bottlenecks
- Add iperf3 JSON result capture to traffic manager for throughput
measurement across scenarios
- Add ECN A/B test scenarios (ecn-ab-on/off.yaml) with ingress
policing and comparison script
- Enable TCP ECN negotiation (tcp_ecn=1 sysctl) in container
entrypoint for end-to-end CE propagation
Tests:
- 10 ECN unit/integration tests: mark_ipv6_ecn_ce variants, CE relay
chain (3-node propagation), EcnConfig serde roundtrip
- 3 transport drop congestion detection unit tests
Documentation:
- Update fips-mesh-layer.md: replace outdated CE Echo stub with full
ECN Congestion Signaling section covering detection logic, CE relay,
IPv6 marking, session tracking, and monitoring counters
- Update fips-configuration.md: add node.ecn.* parameter table and
ecn block in complete reference YAML
- Update fips-transport-layer.md: add Congestion Reporting section
with TransportCongestion struct, congestion() trait method, and
per-transport status; document AsyncFd/recvmsg/SO_RXQ_OVFL in UDP
- Update chaos README: add congestion/ECN scenario docs, ingress
traffic control, and iperf3 JSON capture sections
- Update README.md: add ECN to features list and "What works today";
update transport and tooling entries
Tailscale-style sidecar pattern: a FIPS container provides mesh
networking, and a companion app container shares its network namespace
via network_mode: service:fips.
Security model:
- iptables enforces strict isolation — the app container can only
communicate over the FIPS mesh (fd::/8 via fips0)
- No IPv4 access: eth0 restricted to FIPS UDP transport (port 2121)
- No IPv6 on eth0: ip6tables blocks all eth0 IPv6 traffic
- Only fips0 and loopback are reachable from the app container
The sidecar accepts peer configuration via environment variables
(FIPS_NSEC, FIPS_PEER_NPUB, FIPS_PEER_ADDR), so it can be pointed
at any FIPS node without config file generation.
Files:
- testing/sidecar/: Dockerfile, Dockerfile.app, docker-compose.yml,
entrypoint.sh, .env, resolv.conf, scripts/build.sh
- testing/sidecar/README.md: security model, quick-start, architecture,
DNS resolution, troubleshooting, production considerations
- testing/sidecar/scripts/test-sidecar.sh: 3-node chain integration
test verifying link establishment, multi-hop connectivity, and
network isolation on each app container
- .github/workflows/ci.yml: sidecar integration test matrix entry
Chaos harness enhancements:
- transport_mix config: weighted random transport assignment for random
topologies (erdos_renyi, random_geometric) with UDP/Ethernet/TCP
- Replace LoRa with Bluetooth L2CAP in cost-based/mixed-tech scenarios
using realistic netem values (15-40ms delay, 5-15ms jitter, 2-8% loss)
- New churn-20-mixed scenario: 20-node Erdos-Renyi with 60% UDP,
20% Ethernet, 20% TCP, full netem/link-flap/churn/bandwidth config
- Expanded chaos README with full scenario catalog in four categories
Static harness:
- Updated README with topology table and scenario count
Implement TCP transport for FIPS enabling firewall traversal and serving
as the foundation for future Tor transport. This is the first
connection-oriented transport in the system.
Key design decisions:
- FMP header-based framing: reuses existing 4-byte FMP common prefix for
packet boundary recovery with zero framing overhead
- Session survives TCP reconnection: Noise/MMP/FSP state bound to npub,
not TCP connection; MMP liveness is sole authority for peer death
- Connect-on-send: fresh connection on first send, transparent reconnect
- close_connection() trait method for cross-connection deduplication cleanup
New transport files:
- src/transport/tcp/mod.rs: TcpTransport, connection pool, accept loop
- src/transport/tcp/stream.rs: FMP-aware stream reader (shared with Tor)
Modified: transport trait (close_connection), TcpConfig, TransportHandle
match arms, create_transports(), initiate_connection() for connection-
oriented links, cross-connection tie-breaker cleanup, design docs.
Tree announce loop and TCP stability fixes:
- Preserve tree announce rate-limit state across reconnection: carry
forward last_tree_announce_sent_ms when a peer reconnects so the
rate-limit window isn't reset to zero
- Drop oversize TCP packets at sender: pre-send MTU check returns
MtuExceeded instead of writing to the stream, preventing receiver-side
connection teardown and reset-reconnect cycles
Chaos harness:
- TCP transport support: tcp_edges/has_tcp/tcp_peers in SimTopology,
transport-aware config_gen with per-edge transport type, TCP port 443,
pure-TCP node support
- Include all non-Ethernet edges in directed_outbound()
- Fix netem/links log messages to say "IP-based" instead of "UDP"
- Add tcp-chain, tcp-only, and tcp-mesh scenario files
Static harness:
- Transport-aware config generation (get_default_transport, transport_port)
- TCP transport injection via Python post-processing
- Add tcp-chain topology and docker-compose profile
Update the default UDP bind port from 4000 to 2121 (decimal) and the
default Ethernet EtherType from 0x88B5 to 0x2121 across all source
code, documentation, configuration templates, test fixtures, and
scripts. Remove references to "IEEE 802 experimental range" since
0x2121 is not in that range.
- Ethernet ioctl: cfg-gate the ioctl request parameter type — c_int on
musl, c_ulong on gnu — so the same code compiles on both
x86_64-unknown-linux-gnu and x86_64-unknown-linux-musl targets
- Chaos harness macOS support: replace direct host 'ip link' invocations
with a privileged Docker container helper (--net=host --pid=host) that
shares the Docker VM's namespaces, making veth pair setup work on both
Linux and macOS (where host 'ip' is unavailable and container PIDs
live inside the Docker Desktop VM)
- Python 3.9 compat: add 'from __future__ import annotations' to
docker_exec.py for X|Y union syntax support
- Add deploy/ to .gitignore
Co-authored-by: origami74 <origami74@gmail.com>
Implement raw Ethernet transport using AF_PACKET SOCK_DGRAM on Linux
with EtherType 0x88B5 (IEEE experimental range) and 1-byte frame type
prefix (0x00=data, 0x01=beacon).
Transport implementation:
- EthernetConfig with interface, ethertype, MTU, buffer sizes, and
four independent discovery knobs (discovery, announce, auto_connect,
accept_connections)
- PacketSocket/AsyncPacketSocket wrappers with ioctl helpers for
interface index, MAC address, and MTU queries
- EthernetTransport with Transport trait impl, async start/stop/send,
receive loop dispatching data frames and discovery beacons
- Discovery beacons (34 bytes: type + version + x-only pubkey) with
DiscoveryBuffer for peer accumulation and dedup
- Atomic statistics counters (frames, bytes, errors, beacons)
- Platform-gated with #[cfg(target_os = "linux")]
Transport-layer discovery integration:
- Promote auto_connect() and accept_connections() to Transport trait
with default implementations and TransportHandle dispatch
- Extract initiate_connection() so both static peer config and
discovery auto-connect share the same handshake initiation path
- Add poll_transport_discovery() to the tick handler to drain
discovery buffers and auto-connect to discovered peers
- Enforce accept_connections() in handle_msg1() — transports with
accept_connections=false silently drop inbound handshakes
Node integration:
- create_transports() handles Ethernet named instances
- resolve_ethernet_addr() parses "interface/mac" address format
- transport_mtu() generalized for multi-transport operation
Test harness:
- VethPair RAII struct for veth pair lifecycle management
- Three #[ignore] integration tests requiring root/CAP_NET_RAW:
two-node handshake, data exchange, mixed transport coexistence
- Chaos harness: transport-aware topology model, VethManager for
veth pairs between Docker containers, Ethernet-aware config gen,
netem split (HTB+u32 for UDP, root netem for veth), transport-aware
link flaps and node churn with veth re-setup
- Container entrypoint waits for configured Ethernet interfaces
before starting FIPS (handles veth creation timing)
- New scenarios: ethernet-only (4-node ring), ethernet-mesh (6-node
mixed UDP+Ethernet with netem and link flaps)
Documentation:
- fips-transport-layer.md: Ethernet section, beacon discovery, WiFi
compatibility, updated discovery state, trait surface additions,
implementation status table
- fips-configuration.md: Ethernet parameter table, named instances,
peer address format, mixed UDP+Ethernet example, complete reference
- fips-wire-formats.md: Ethernet frame type prefix note
Cost-based parent selection:
- Replace depth-only parent selection with effective_depth = depth + link_cost
- link_cost computed from locally measured MMP metrics: etx * (1.0 + srtt_ms / 100.0)
- Prevents bottleneck subtrees in heterogeneous networks where a LoRa link
at depth 1 would otherwise always beat fiber at depth 2
- Configurable hysteresis (default 0.2) prevents marginal parent switches
- Configurable hold-down timer (default 30s) suppresses re-evaluation
after parent switch
- Mandatory switches (parent lost, root change) bypass both safeguards
- Link costs passed as HashMap parameter to keep TreeState pure
Periodic re-evaluation:
- evaluate_parent() was only called on TreeAnnounce receipt or parent loss;
after tree stabilization, link degradation went undetected
- Added timer-based re-evaluation (reeval_interval_secs, default 60s) that
calls evaluate_parent() from the tick handler with current MMP link costs
- Respects existing hold-down and hysteresis safeguards
- Short-circuits when disabled or <2 peers
Design documentation:
- Update 7 design docs to reflect cost-based parent selection
- Replace depth-only algorithm descriptions with effective_depth model
- Replace rejected cumulative path cost spec with local-only design rationale
- Rewrite Example 2 (heterogeneous links) for local-only cost model
- Update config docs: parent_switch_threshold replaced by parent_hysteresis,
hold_down_secs, reeval_interval_secs
Chaos simulation enhancements:
- fips_overrides with deep merge for per-scenario FIPS config customization
- Explicit topology algorithm for deterministic test graphs
- Control socket querying via fipsctl for tree/MMP snapshot collection
- Edge existence validation in netem manager
- Per-link netem policy overrides
- 9 new chaos scenarios covering cost avoidance, depth-vs-cost tradeoffs,
stability, mixed topologies, periodic re-evaluation, and bottleneck parent
12 new unit tests, 667 total passing, clippy clean.
fips-link-layer.md:
- Rewrite Liveness Detection: explicit Heartbeat (0x51) with 10s interval
and 30s dead timeout replaces vague gossip-as-heartbeat description
- Add Auto-Reconnect section: MMP dead timeout triggers retry with
unlimited backoff for auto_reconnect peers
- Add Handshake Message Retry section: link + session layer resend with
exponential backoff within timeout window
- Add Heartbeat to Link Message Types table
- Update Implementation Status with three new implemented features
fips-configuration.md:
- Add handshake_resend_interval_ms, handshake_resend_backoff,
handshake_max_resends to rate_limit table
- Add heartbeat_interval_secs, link_dead_timeout_secs to general table
- Add peers[].auto_reconnect to peers table
- Note auto-reconnect bypasses max_retries in retry section
- Update complete reference YAML with all new parameters
fips-wire-formats.md:
- Rename 0x51 from reserved Keepalive to implemented Heartbeat
- Update Disconnect reason 0x07 to Heartbeat liveness timeout
testing/chaos/README.md:
- Add runner.log to output files
- Add Directed Outbound Configs subsection
Auto-reconnect:
- Add per-peer auto_reconnect config (default true) to PeerConfig
- schedule_reconnect() feeds removed peers back into retry system with
unlimited retries and exponential backoff after MMP dead timeout
- RetryState gains reconnect flag to distinguish startup retries
(max_retries-limited) from auto-reconnect (unlimited)
Retry re-fire fix:
- process_pending_retries() now pushes retry_after_ms past the handshake
timeout window after successful initiate_peer_connection(), preventing
retries from firing every tick with no backoff
Chaos sim improvements:
- Directed outbound configs: BFS spanning tree + lower-ID-first assignment
eliminates dual-connect race conditions in simulation
- Save runner log (runner.log) alongside per-node logs for event correlation
- Increase churn-20 traffic aggressiveness and node churn (max_down_nodes
3→5, traffic interval min 0s, duration max 90s, concurrent flows 5→10)
Johnathan Corgan
Origami74
joandJohnathan Corgan
Kieran
