macOS platform:
- Platform-native TUN interface management with shutdown pipe
- Raw Ethernet transport with macOS socket backend (socket_macos.rs)
- EthernetTransport and TransportHandle::Ethernet ungated from Linux-only
- macOS .pkg packaging (build-pkg.sh, launchd plist, uninstall script)
- CI: macOS build and unit test jobs; x86_64 cross-compiled from
macos-latest via rustup target add x86_64-apple-darwin
Gateway feature flag:
- New opt-in `gateway` Cargo feature activates optional `rustables` dep
- `pub mod gateway` and `Config.gateway` gated behind the feature so
macOS builds never pull in Linux-only nftables bindings
- `fips-gateway` bin has `required-features = ["gateway"]`
- All Linux/OpenWrt/AUR packaging passes `--features gateway`
CI / packaging:
- package-linux, package-macos, package-openwrt now trigger on push to
master/maint/next and on pull requests; release uploads remain tag-gated
- Bloom filter routing fix: fall through to tree routing when no candidate
is strictly closer
- MMP intervals: raise MIN to 1s / MAX to 5s with 5-sample cold-start phase
BLE transport implementation using L2CAP Connection-Oriented Channels
(SeqPacket mode) via the bluer crate, behind cfg(feature = "ble").
Core transport:
- BleTransport<I> generic over BleIo trait (BluerIo prod, MockBleIo test)
- Connection pool with priority eviction (static > discovered, max 7)
- Connect-on-send via connect_inline() matching TCP behavior
- Per-connection receive loops with pool cleanup on disconnect
Discovery and probing:
- Combined scan_probe_loop using select! over scanner events and a
BinaryHeap delay queue with per-entry random jitter (0-5s) to prevent
herd effects when multiple nodes see the same beacon simultaneously
- Pre-handshake pubkey exchange ([0x00][pubkey:32]) for IK identity
- Cross-probe tie-breaker: smaller NodeAddr's outbound wins (same
convention as FMP/FSP rekey dual-initiation)
- Probed peers reported to DiscoveryBuffer; pool fills through normal
node-layer auto-connect -> send_async -> connect_inline path
Beacon management:
- Periodic advertising: 1s burst every 30s (configurable via
beacon_interval_secs / beacon_duration_secs)
- FIPS service UUID for scan filtering
Configuration (all fields optional with defaults):
- adapter, psm, mtu, max_connections, connect_timeout_ms
- advertise, scan, auto_connect, accept_connections
- beacon_interval_secs (30), beacon_duration_secs (1)
Hardware validated with two BLE nodes:
- 2048-byte MTU, ~60-160ms RTT, zero-config auto-connect
- BLE spike tool at testing/ble/ for standalone adapter validation
42 unit tests + 4 node-level integration tests, all CI-compatible
via MockBleIo (no hardware required). tokio test-util added for
time-dependent scan/probe tests.
Add runtime peer management to the FIPS daemon via control socket
commands, and a new chaos simulation scenario that exercises dynamic
topology mutation with ephemeral node identities.
Daemon (connect/disconnect commands):
- Extend control socket Request with optional params field
- Add commands.rs module for mutating command dispatch, separate from
read-only queries
- Add api_connect() on Node: builds ephemeral PeerConfig (no auto-
reconnect), pre-seeds identity cache, reuses initiate_peer_connection
- Add api_disconnect() on Node: calls remove_active_peer(), clears
retry_pending to suppress reconnection
- Route non-show_* commands to async command dispatch in rx_loop
fipsctl CLI:
- Add Connect and Disconnect subcommands accepting npub or hostname
- Resolve hostnames from /etc/fips/hosts before sending to daemon
- Refactor socket I/O into reusable send_request helper
Chaos simulator (maelstrom scenario):
- Add PeerChurnManager: periodically disconnects a random active link
and connects a random unconnected node pair via control socket
- Add send_command() to control.py using base64-encoded JSON payloads
to avoid shell quoting issues in docker exec
- Add PeerChurnConfig to scenario with interval and ephemeral_fraction
- Ephemeral identity support: nodes configured without nsec generate
fresh keypairs on restart; simulator queries show_status for new
npub and updates its cache via on_node_restart callback
- Add maelstrom.yaml: all chaos dimensions (netem, link flaps, node
churn, peer topology churn, traffic) with 50% ephemeral identity
Cherry-picked from v0l PR #6 (issue #5):
1. remove_active_peer() now removes the end-to-end session from
self.sessions when evicting a peer. The stale Established entry
caused initiate_session() to silently return Ok(()) via the
is_established() guard, preventing session re-establishment after
link-layer reconnection.
2. initiate_peer_connections() pre-seeds the identity cache from
configured peer npubs at startup, so TUN packets can be dispatched
immediately without waiting for handshake completion.
3. schedule_reconnect() preserves accumulated backoff when a retry
entry already exists, preventing exponential backoff reset on
repeated link-dead cycles.
Includes regression tests for all three fixes.
Previously, static peers configured with AutoConnect gave up after 6
attempts (1 initial + 5 retries). If the remote peer was offline at
startup, the node permanently abandoned the connection. The reconnect
path (after MMP link-dead) already retried indefinitely but only
activated after a peer had been previously connected.
Remove the reconnect parameter from schedule_retry() and always set
reconnect=true when creating retry entries, since only auto-connect
peers reach this code path. The 300s backoff cap prevents resource waste.
The max_retries=0 config still works as an explicit kill switch.
TCP (and future Tor) transports previously established connections
synchronously inside send(), blocking the node's RX event loop during
TCP handshake. This is particularly problematic for Tor where SOCKS5
circuit establishment can take 30-120 seconds.
Add a non-blocking connect path:
- ConnectionState enum in transport layer (None/Connecting/Connected/Failed)
- connect_async() on TcpTransport spawns background TCP connect task
- connection_state_sync() polls task completion, promotes to pool
- TransportHandle gains connect() and connection_state() dispatch methods
- Node tracks PendingConnect entries for connection-oriented transports
- initiate_connection() defers handshake for connection-oriented transports
- start_handshake() extracted as separate method for deferred invocation
- poll_pending_connects() in tick handler polls and completes handshakes
- Failed connects trigger retry via schedule_retry()
Connectionless transports (UDP, Ethernet) are unchanged — connect()
is a no-op and connection_state() always returns Connected.
The existing connect-on-send path in send_async() is preserved as
fallback for reconnection after connection drops.
811 tests pass (6 new), clippy clean.
Add a HostMap that resolves human-readable hostnames to npubs,
enabling `gateway.fips` instead of the full `npub1...xyz.fips`.
Two sources populate the map: peer `alias` fields from the YAML
config and an operator-maintained hosts file at /etc/fips/hosts.
The DNS responder auto-reloads the hosts file on each request by
checking the file modification time, so operators can update
mappings without restarting the daemon.
- New src/upper/hosts.rs: HostMap, HostMapReloader, hostname
validation, hosts file parser with auto-reload on mtime change
- DNS resolver checks host map before falling back to direct npub
- Node uses host map for peer display names
- Default hosts file added to both .deb and tarball packaging
- 26 new tests (789 total)
Implement TCP transport for FIPS enabling firewall traversal and serving
as the foundation for future Tor transport. This is the first
connection-oriented transport in the system.
Key design decisions:
- FMP header-based framing: reuses existing 4-byte FMP common prefix for
packet boundary recovery with zero framing overhead
- Session survives TCP reconnection: Noise/MMP/FSP state bound to npub,
not TCP connection; MMP liveness is sole authority for peer death
- Connect-on-send: fresh connection on first send, transparent reconnect
- close_connection() trait method for cross-connection deduplication cleanup
New transport files:
- src/transport/tcp/mod.rs: TcpTransport, connection pool, accept loop
- src/transport/tcp/stream.rs: FMP-aware stream reader (shared with Tor)
Modified: transport trait (close_connection), TcpConfig, TransportHandle
match arms, create_transports(), initiate_connection() for connection-
oriented links, cross-connection tie-breaker cleanup, design docs.
Tree announce loop and TCP stability fixes:
- Preserve tree announce rate-limit state across reconnection: carry
forward last_tree_announce_sent_ms when a peer reconnects so the
rate-limit window isn't reset to zero
- Drop oversize TCP packets at sender: pre-send MTU check returns
MtuExceeded instead of writing to the stream, preventing receiver-side
connection teardown and reset-reconnect cycles
Chaos harness:
- TCP transport support: tcp_edges/has_tcp/tcp_peers in SimTopology,
transport-aware config_gen with per-edge transport type, TCP port 443,
pure-TCP node support
- Include all non-Ethernet edges in directed_outbound()
- Fix netem/links log messages to say "IP-based" instead of "UDP"
- Add tcp-chain, tcp-only, and tcp-mesh scenario files
Static harness:
- Transport-aware config generation (get_default_transport, transport_port)
- TCP transport injection via Python post-processing
- Add tcp-chain topology and docker-compose profile
Implement raw Ethernet transport using AF_PACKET SOCK_DGRAM on Linux
with EtherType 0x88B5 (IEEE experimental range) and 1-byte frame type
prefix (0x00=data, 0x01=beacon).
Transport implementation:
- EthernetConfig with interface, ethertype, MTU, buffer sizes, and
four independent discovery knobs (discovery, announce, auto_connect,
accept_connections)
- PacketSocket/AsyncPacketSocket wrappers with ioctl helpers for
interface index, MAC address, and MTU queries
- EthernetTransport with Transport trait impl, async start/stop/send,
receive loop dispatching data frames and discovery beacons
- Discovery beacons (34 bytes: type + version + x-only pubkey) with
DiscoveryBuffer for peer accumulation and dedup
- Atomic statistics counters (frames, bytes, errors, beacons)
- Platform-gated with #[cfg(target_os = "linux")]
Transport-layer discovery integration:
- Promote auto_connect() and accept_connections() to Transport trait
with default implementations and TransportHandle dispatch
- Extract initiate_connection() so both static peer config and
discovery auto-connect share the same handshake initiation path
- Add poll_transport_discovery() to the tick handler to drain
discovery buffers and auto-connect to discovered peers
- Enforce accept_connections() in handle_msg1() — transports with
accept_connections=false silently drop inbound handshakes
Node integration:
- create_transports() handles Ethernet named instances
- resolve_ethernet_addr() parses "interface/mac" address format
- transport_mtu() generalized for multi-transport operation
Test harness:
- VethPair RAII struct for veth pair lifecycle management
- Three #[ignore] integration tests requiring root/CAP_NET_RAW:
two-node handshake, data exchange, mixed transport coexistence
- Chaos harness: transport-aware topology model, VethManager for
veth pairs between Docker containers, Ethernet-aware config gen,
netem split (HTB+u32 for UDP, root netem for veth), transport-aware
link flaps and node churn with veth re-setup
- Container entrypoint waits for configured Ethernet interfaces
before starting FIPS (handles veth creation timing)
- New scenarios: ethernet-only (4-node ring), ethernet-mesh (6-node
mixed UDP+Ethernet with netem and link flaps)
Documentation:
- fips-transport-layer.md: Ethernet section, beacon discovery, WiFi
compatibility, updated discovery state, trait surface additions,
implementation status table
- fips-configuration.md: Ethernet parameter table, named instances,
peer address format, mixed UDP+Ethernet example, complete reference
- fips-wire-formats.md: Ethernet frame type prefix note
Each node generates a random 8-byte startup epoch, encrypted inside
both Noise IK handshake messages (msg1 and msg2). When a peer's msg1
arrives with a different epoch than the stored value, the node tears
down the stale session and processes the msg1 as a new connection,
enabling near-instant restart detection instead of the 30-second
dead timeout.
Wire format impact:
- msg1: 82 -> 106 bytes (added 24-byte encrypted epoch after ss DH)
- msg2: 33 -> 57 bytes (added 24-byte encrypted epoch after se DH)
- Wire msg1: 90 -> 114 bytes, wire msg2: 45 -> 69 bytes
Add message-level retry for Noise IK handshake within the 30s timeout
window. Previously, a lost msg1 or msg2 required the full timeout to
expire before cleanup and retry. Under 10% bidirectional loss (~19%
per attempt), this made connection establishment unreliable.
Initiator resends stored msg1 bytes with exponential backoff (1s, 2s,
4s, 8s, 16s — 5 resends). Responder stores msg2 and resends on
duplicate msg1 receipt. Duplicate msg2 at initiator drops silently
via existing pending_outbound cleanup.
Config: handshake_resend_interval_ms (1000), handshake_resend_backoff
(2.0), handshake_max_resends (5) on node.rate_limit.
P(all 6 attempts fail) under 19% loss = 0.19^6 ≈ 0.005%.
Replace raw NodeAddr hex strings in log output with human-readable
identifiers using a four-tier lookup: configured alias, active peer
short npub, session endpoint short npub, or truncated hex fallback.
- Add PeerIdentity::short_npub() for compact npub display (npub1xxxx...yyyy)
- Add NodeAddr::short_hex() for compact hex fallback (first 4 bytes + ...)
- Add peer_aliases map populated at startup from peer config
- Add Node::peer_display_name() with four-tier resolution
- Add SessionEntry::remote_pubkey() accessor for session-layer lookups
- Update all 120 log field occurrences across 11 handler/node files
- Pass pre-computed display names to MMP static metric/teardown methods
to work around borrow checker constraints in iterator loops
Per-packet happy-path events (UDP send/receive, TUN I/O, MMP report
processing, TreeAnnounce/FilterAnnounce sent, RTT samples) moved from
debug to trace. Periodic maintenance and retry scheduling moved from
info to debug. Session state changes (established, initiated, torn
down) and transport stop promoted from debug to info. MMP report send
failures demoted from warn to debug (normal under churn).
Add dual-approach MTU handling to prevent TCP connections from hanging
when packets exceed the transport MTU after FIPS encapsulation.
ICMPv6 Packet Too Big:
- Generate RFC 4443 PTB messages for oversized packets at TUN outbound
- Inject back via TUN for local delivery to the application
- Per-source rate limiting (100ms interval, 10s entry expiry)
- MTU check in handle_tun_outbound before session encapsulation
TCP MSS Clamping:
- Intercept SYN packets in run_tun_reader() (outbound)
- Intercept SYN-ACK packets in TunWriter (inbound)
- Clamp MSS option to fit within effective MTU (transport - 127 overhead)
- Recalculate TCP checksum after modification
Code organization:
- ICMP, TCP MSS, and rate limiter modules in upper/ alongside existing
protocol-specific packet handling (dns.rs, tun.rs)
- Shared FIPS_OVERHEAD constant (127 bytes) and effective_ipv6_mtu()
function in upper/icmp.rs
- Node::effective_ipv6_mtu() delegates to the shared function
- run_tun_reader() accepts actual transport MTU from config
Example config corrections:
- UDP transport MTU set to 1472 across all configs (correct max UDP
payload for standard Ethernet: 1500 - 20 IPv4 - 8 UDP)
- Startup logging of effective MTU and max MSS values
Add DNS responder that resolves <npub>.fips queries to FipsAddress IPv6
addresses. Resolution is pure computation (npub → NodeAddr → IPv6) with
identity cache priming as a side effect, enabling subsequent TUN packet
routing to non-peer destinations.
- New dns.rs module: resolve_fips_query(), handle_dns_packet() using
simple-dns crate, run_dns_responder() async UDP server loop
- DnsConfig in config.rs: enabled, bind_addr (127.0.0.1), port (5354)
- Fourth select! arm in RX event loop for DNS identity channel
- DNS task spawn/abort in node lifecycle
- 10 new tests (420 total)
Add examples/two-node-udp/ with standalone walkthrough for testing two
FIPS nodes in Linux network namespaces over a veth pair. Includes
network diagram, config files, DNS routing setup, and troubleshooting.
Wire the TUN reader to the session layer for end-to-end IPv6 packet
delivery through the mesh. TUN reader forwards FIPS-destined packets
(fd::/8) to Node via tokio::sync::mpsc channel. Identity cache maps
FipsAddress prefix bytes to NodeAddr + PublicKey for reverse address
resolution, populated at peer promotion and session creation. Outbound
handler validates IPv6, looks up destination identity, routes through
established session or initiates new one with pending packet queue
(16/dest, 256 dests max). Queue flushed on session establishment.
ICMPv6 Destination Unreachable for unknown destinations. 6 new tests
including 3-node forwarded TUN data and pending queue flush. 410 tests
pass, clean build.
Fix promote_connection() to detect and clean up pending outbound
handshakes to the same peer, not just already-promoted peers. Previously,
when an inbound handshake completed while an outbound was still pending,
the outbound would linger until the 30s timeout.
Add auto-retry for failed outbound connections to auto-connect peers:
- New RetryState struct and node/retry.rs module
- Exponential backoff (default 5s base, max 5 attempts)
- Config: node.max_retries, node.base_retry_interval_secs
- check_timeouts() schedules retries, rx loop processes them
- promote_connection() clears retry state on success
- Remove unused PeerConnection retry fields (state now at Node level)
Move cleanup_stale_connection() logging to callers for context-appropriate
messages.
289 tests pass, clean build.
Noise IK parity fix:
- Pre-message hash normalizes responder static key to even parity (0x02)
so initiator and responder hash chains match regardless of actual parity
- ECDH uses shared_secret_point() + SHA-256(x-only) instead of
SharedSecret::new() which includes a parity-dependent version byte
- Fixes handshake failure for ~50% of keys when initiator has only npub
Graceful disconnect protocol (link message 0x50):
- DisconnectReason enum with 8 reason codes
- Disconnect struct with encode/decode
- send_encrypted_link_message() reusable helper
- handle_disconnect() with immediate peer removal
- send_disconnect_to_all_peers() called during Node::stop()
Cross-connection fix in handle_msg1():
- addr_to_link check now distinguishes inbound duplicates (reject) from
outbound links (cross-connection, allow and resolve via tie-breaker)
- remove_link() only clears addr_to_link if entry maps to same link_id
- Link cleanup and addr_to_link restoration in cross-connection branches
Handshake timeout cleanup:
- RX loop uses tokio::select! with 1-second interval tick
- check_timeouts() scans for stale (>30s) and failed connections
- cleanup_stale_connection() removes all associated state
Tests: 279 passing (4 new: cross-connection, stale cleanup, failed
cleanup, odd-parity handshake)
Split 2,597-line node.rs into four files under src/node/:
- mod.rs: types, struct definition, constructors, accessors, Debug impl
- lifecycle.rs: start(), stop(), peer connection initiation
- handlers.rs: RX loop, packet dispatch, message handlers, promote_connection
- tests.rs: full test suite (35 tests including integration tests)
No functional changes. All 267 tests pass, no new clippy warnings.
Johnathan Corgan
Origami74
KieranandJohnathan Corgan