On Ubuntu 22 (systemd 249), systemd-resolved applies interface-scoped
routing to per-link DNS servers. Configuring `resolvectl dns fips0
127.0.0.1:5354` caused resolved to attempt reaching 127.0.0.1 through
fips0 (a TUN with only fd00::/8 routes), silently failing. The DNS
responder never received queries. Newer systemd versions (250+) have
explicit handling for loopback servers on non-loopback interfaces.
Changes:
- DNS responder default bind_addr changed from "127.0.0.1" to "::"
so it listens on all interfaces, including fips0. Bind logic in
lifecycle.rs now parses bind_addr as IpAddr and constructs a
SocketAddr, handling IPv6 literal formatting. Factored into
Node::bind_dns_socket with explicit IPV6_V6ONLY=0 via socket2, so
IPv4 clients on 127.0.0.1:5354 still reach the responder
regardless of the kernel's net.ipv6.bindv6only sysctl.
- fips-dns-setup resolvectl backend now waits for fips0 to have a
global IPv6 address, then configures resolved with
[<fips0-addr>]:5354. That address is locally delivered by the
kernel regardless of which interface resolved tries to route
through. The dnsmasq and NetworkManager backends still use
127.0.0.1 (they don't have the interface-scoping issue).
- Dropped hardcoded `bind_addr: "127.0.0.1"` from the packaged
fips.yaml (Debian + OpenWrt). The shipped config was overriding
the new default.
- DNS queries are only accepted from the localhost.
Verified end-to-end in a privileged Ubuntu 22.04 systemd container:
dig @127.0.0.53 AAAA <npub>.fips resolves cleanly through
systemd-resolved.
The dns-delegate backend (systemd 258+) still uses 127.0.0.1; it
has not been verified whether that backend has the same routing
issue.
The test used Identity::generate() for the signing node while pinning
the fixed root to node_addr byte[0]=0x01. About 2/256 random identities
have byte[0] <= 0x01, which made the generated node_addr the path
minimum and triggered AncestryRootNotMinimum. Regenerate the identity
until its node_addr is numerically larger than the fixed parent and
root, so the test matches the preconditions it asserts.
A malformed FilterAnnounce whose fill ratio produces an implausibly
high false-positive rate is mostly useless for routing and, once
merged into our outgoing filter via bitwise OR, propagates the
saturated state to tree peers one hop per announce tick. A saturated
filter also made estimated_count() return f64::INFINITY, which
compute_mesh_size summed into its cached estimate.
handle_filter_announce now rejects inbound FilterAnnounce whose
derived FPR exceeds `node.bloom.max_inbound_fpr` (new config field,
default 0.05 ≈ fill 0.549 at k=5). Rejection is silent on the wire,
logs at WARN, and increments a new `bloom.fill_exceeded` counter. The
peer's prior stored filter and filter_sequence are left unchanged so
a single rejected announce does not wipe the peer's existing
contribution to aggregation.
After a successful outgoing FilterAnnounce send, a rate-limited WARN
fires if our own filter's FPR exceeds the same cap, surfacing
aggregation drift. Limited to once per 60 seconds via a new
Node.last_self_warn field.
BloomFilter::estimated_count() now takes max_fpr and returns
Option<f64>. Returns None for saturated filters (regardless of cap)
or when the filter's FPR exceeds max_fpr. Callers updated: debug
logs render None as "—", the Debug impl uses f64::INFINITY as "no
cap" and prints "saturated" instead of inf, control-socket JSON
emits null, and compute_mesh_size propagates None into the already-
Option<u64> estimated_mesh_size field.
Implement TCP Wrappers-style peer access control using
/etc/fips/peers.allow and /etc/fips/peers.deny files. Evaluation
order: allow overrides deny, default permit when no files exist.
Three enforcement points: outbound connect (before dialing), inbound
handshake (msg1 receipt, after restart/rekey classification), and
outbound handshake completion (msg2, before peer promotion). Files
support npub, hex pubkey, host alias, and ALL wildcard entries with
automatic mtime-based reload.
Adds fipsctl acl show query, 954-line acl module with unit tests,
and a 6-node Docker integration harness (testing/acl-allowlist/)
exercising insider, outsider, and allowed-remote scenarios. CI
matrix entry included.
Closes#50
Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
Adds TreeAnnounce::validate_semantics() called from handle_tree_announce
before any tree-state mutation. Enforces that the ancestry accompanying
a parent declaration conforms to the spanning tree rules:
- first ancestry entry matches the signed sender
- is_root declarations carry a single-entry ancestry
- non-root declarations include the signed parent as the second entry
- the advertised root is the minimum node_addr in the ancestry
Non-conforming announcements are rejected with a warn log and no state
change. Adds unit tests for each rejected shape plus an integration
test covering the full receive path in a two-node tree.
Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
Mesh peers can now reach a configured host:port on the gateway's LAN
via static port-forward rules on fips-gateway. Mirror of the outbound
LAN gateway (IDEA-0079 / TASK-2026-0056).
Config: new gateway.port_forwards list of { listen_port, proto,
target } entries. Targets are SocketAddrV6 — IPv4 is rejected at
parse time. Validation rejects zero listen ports and duplicate
(listen_port, proto) pairs.
NAT: NatManager gains a port_forwards field and set_port_forwards()
setter, rebuilt in the same atomic rustables batch as the address
mappings. Each forward emits a prerouting DNAT rule keyed on
(iifname fips0, nfproto ipv6, l4proto, tcp/udp dport) that rewrites
destination address and port via Nat::with_ip_register +
with_port_register. When any forwards are configured, a single
LAN-side masquerade is installed on (iifname fips0, oifname
lan_interface, nfproto ipv6) so the LAN host sees the gateway as
source and replies flow back through conntrack. This rule is
distinct from the existing oifname fips0 masquerade that serves the
outbound pool.
Binary: fips-gateway validates port_forwards at startup and calls
set_port_forwards after NatManager construction; startup failure
cleans up the nftables table before exiting.
Test: extend testing/static/scripts/gateway-test.sh with Phase 7
that runs a marker HTTP server on the LAN-side client (fd02::20:8080)
and, from the mesh peer, curls the gateway's fips0 address on port
18080 to exercise the full DNAT + LAN masquerade path. The
LAN-side HTTP server is started with 'docker exec -d' plus a
bind-ready poll on ss; 'docker exec bash -c "cmd &"' does not
keep the child alive past the exec session even with nohup.
Test-infra: ci-local.sh now builds/clippies/tests with
--features gateway, matching GitHub CI. Without this the release
fips-gateway binary silently stays stale across runs, since
cargo build --release alone does not compile the gateway bin.
Verified locally: cargo test --features gateway --lib (991 pass),
clippy + fmt clean, full testing/ci-local.sh green (21/21 suites
in 8m36s, including the new gateway Phase 7).
In-memory time-series history on the daemon: fast ring (1s × 3600)
plus slow ring (1m × 1440) per metric, covering node-level gauges
(mesh size, tree depth, peer count, active sessions), counters
(parent switches, aggregate bytes/packets in/out), loss rate, and
seven per-peer metrics keyed by NodeAddr (srtt_ms, loss_rate,
bytes_in/out, packets_in/out, ecn_ce). The slow ring is produced by
downsampling the fast ring on minute boundaries with Last / Sum /
Mean aggregation chosen per metric type.
Missing data is first-class. New peers back-fill NaN so every ring
shares a time axis with the node rings; peers absent from a tick
sample NaN (keeps alignment, shows as a visible gap); counter metrics
emit NaN on decrease (new link_stats baseline after reconnect) so
deltas aren't polluted. Peers are evicted 24h after last contact.
Downsampling is NaN-aware: mean skips NaN, all-NaN slow windows stay
NaN. Each history window always returns its full span at the chosen
density (1m / 10m / 1h / 24h), front-padded with NaN when the ring
hasn't yet accumulated enough samples, so switching between windows
feels like zooming in or out rather than clipping to whatever has
arrived. NaN serializes to JSON null via a custom serializer.
Control socket queries:
- show_stats_list enumerates registered metrics plus scope field and
peer_retention_seconds.
- show_stats_history returns one metric's series for a given window
and granularity; accepts optional peer (npub) for per-peer metrics.
- show_stats_all_history returns every metric in a single round trip;
accepts optional peer to fetch all seven per-peer metrics.
- show_stats_peers enumerates tracked peers with lifecycle metadata.
- show_stats_history_all_peers returns one metric across all peers
for grid rendering.
- show_status carries short sparkline windows for the dashboard so
the client can render without extra fetches.
fipsctl gains `stats list`, `stats peers`, and `stats history
<metric>` with `--peer` (hostname or npub) and `--plot` for a Unicode
block sparkline. Plot header reports sample count, granularity,
window, and gap count; NaN renders as a blank cell.
fipstop dashboard grows inline sparklines (peer count, mesh size,
aggregate bytes in/out). A new Graphs tab stacks every metric as an
independent mini plot with its own autoscaled range; each plot uses
btop's braille 2×4 filled-area algorithm (25-entry lookup table
packing two samples per character, per-row gradient coloring for the
characteristic btop vertical-band look, rounded borders with embedded
titles). Three modes are cycled with `m`: Node (node-level stack),
MetricByPeer (small-multiples grid, 1 / 2 / 3 columns by terminal
width), PeerByMetric (existing stack scoped to one peer). `n` / `N`
cycles the mode-specific selector (metric or peer), a selector row
shows the current choice, and Graphs-tab refreshes re-fetch
show_stats_peers so selectors track peer churn. Up / Down scrolls
the stack, Left / Right cycles the window, `g` jumps to the tab.
Implements IDEA-0084 (TASK-2026-0062).
Extend the fipsctl control query interface with visibility into internal
state critical for protocol security auditing, mesh troubleshooting, and
operational monitoring.
New command:
fipsctl show identity-cache
Lists every node identity cached by the daemon (learned from DNS
resolution, peer handshakes, sessions, and static config). Shows
npub, IPv6 address, display name, and LRU age alongside the
configured cache capacity.
Extended queries:
show peers — Noise session counters (send_counter, highest received
counter) for rekey urgency assessment. Per-peer replay suppression
and consecutive decrypt failure counts for active attack detection.
Session index visibility for hijack analysis. Rekey lifecycle
state (in_progress, draining, K-bit epoch).
show sessions — Handshake resend count during establishment for
connectivity debugging. Rekey and session health fields
(session_start, K-bit, coords warmup, drain state) when
established.
show cache — Individual coordinate cache entries with tree
coordinates, depth, path MTU, and age. Enables route-level
debugging by showing exactly which destinations have cached
routes and via what tree path. Renames the top-level count
field from "entries" to "count" for clarity.
show routing — Pending discovery lookups expanded from count to
per-target detail (attempt number, age, last sent). Pending
TUN packet queue depth for backpressure visibility. Connection
retry state per peer (retry count, next attempt, auto-reconnect
flag).
Updates fipstop to match the revised show_cache and show_routing
response schemas. Updates README monitoring section with the complete
fipsctl command list.
Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
When a user passes an fd00::/8 address as the endpoint for a udp,
tcp, or ethernet transport, the CLI previously echoed success while
the daemon silently failed the bind with EAFNOSUPPORT. Mesh ULAs are
destinations inside the mesh, not reachable transport endpoints.
fipsctl now validates the address up front and prints a clear error
with examples, exiting 1 before the control socket call. Other
transports (tor) are not inspected since they legitimately accept
non-IP endpoints.
Covered by inline tests for bare/bracketed/with-port ULA syntaxes,
non-ULA IPv6, IPv4, hostnames, and the transport filter.
Fixes#61.
handle_disconnect() called remove_active_peer without scheduling a
reconnect, orphaning auto-connect peers on a clean upstream shutdown.
Mirror the pattern from the other three peer-removal paths (link-dead,
decrypt failure, peer restart) which all schedule reconnect after
removal.
Adds test_disconnect_schedules_reconnect regression test that verifies
handle_disconnect populates retry_pending for an auto-connect peer.
Visibility of handle_disconnect bumped to pub(in crate::node) for
direct unit-test access.
Fixes#60.
Exposed by the rustfmt toolchain-pin fix now that rustfmt runs under
the pinned 1.94.1 toolchain in CI. Mechanical formatting changes:
- src/node/mod.rs: collapse EthernetTransport::new let-binding onto
one line (fits within width limit)
- src/node/tests/handshake.rs: reorder imports (super::* after
specific import) and expand two assert_eq! calls to multi-line form
Gate platform-specific code behind cfg attributes and add full Windows
support: TUN device via wintun, TCP control socket on localhost:21210,
Windows Service lifecycle (--install-service/--uninstall-service/--service),
CI build and test matrix, and packaging with ZIP builder and PowerShell
service management scripts.
Key changes:
- Cargo.toml: move tun/libc/rtnetlink behind cfg(unix); add wintun and
windows-service dependencies for Windows
- upper/tun.rs: wintun-based TUN implementation with netsh configuration
for IPv6 address, MTU, and fd00::/8 routing
- control/mod.rs: split into unix_impl/windows_impl; Windows uses TCP on
localhost:21210 with shared connection handler
- bin/fips.rs: refactor main() into run_daemon() accepting a shutdown
signal; add Windows Service support via windows-service crate
- transport/udp/socket.rs: platform-gated modules; Windows uses
tokio::net::UdpSocket (kernel drop count unavailable, returns 0)
- transport/ethernet: gate to cfg(unix); add Windows stub types
- config: platform-conditional default paths (socket, hosts) for Windows
- CI: add windows-latest to build matrix and test-windows job with
cargo-nextest
- packaging/windows: build-zip.ps1, install-service.ps1,
uninstall-service.ps1, and package-windows.yml workflow
- README/docs: Windows build instructions, service management, and
control socket platform differences
Linux and macOS behavior is unchanged.
Replace fixed 1KB bloom filters with variable-size filters (512B-32KB)
that adapt to each node's position in the spanning tree.
Core changes:
- Internal storage: Vec<u8> → Vec<u64> for word-level operations
- Delta compression: XOR diff with word-level RLE, sequence-based
NACK protocol for full retransmit recovery
- Size conversion: fold (large→small) and duplicate (small→large)
with auto-converting merge for mixed-size filter combination
- Native-size storage: peer filters stored at advertised size for
full-resolution routing queries, converted only for outgoing filter
- Adaptive sizing: outgoing fill ratio drives step-up/step-down
between size classes with hysteresis (20%/5% thresholds)
- Filter size decoupled from FMP negotiation: announced dynamically
in filter updates, bit 7 and TLV field 1 removed from handshake
Wire format: FilterAnnounce gains flags byte (delta bit), base_seq
field, RLE-compressed payload. New FilterNack message (0x21) for
out-of-sequence delta recovery.
Redesign shared-media transport framing now that XX replaces IK:
- Ethernet: unified 4-byte header [type][flags][length:2 LE] for all
frame types, effective MTU now if_mtu - 4
- Beacons: strip 32-byte pubkey (34 → 5 bytes), identity learned from
XX handshake msg2/msg3 instead of beacon
- BLE: remove pre-handshake pubkey_exchange() and cross-probe
tie-breaker, both unnecessary with XX
- Discovery: support anonymous connections (pubkey_hint: None) with
address-based dedup for shared-media transports
- Post-handshake identity hooks in handle_msg2/msg3 for self-detection
and future allow/deny list filtering (IDEA-0047)
Drop origin_coords from LookupRequest — unused since reverse-path
routing became primary. Saves 2+16*depth bytes per request.
Wire up min_mtu: populated from TUN MTU config (default 1280) at
origination. Transit nodes skip peers whose link MTU is below the
request's min_mtu requirement. path_mtu on LookupResponse was
already wired (transit min() applied).
Add TLV extension to LookupRequest (after min_mtu) and
LookupResponse (after proof). Uses same TlvEntry format as
negotiation. Transit nodes forward TLV bytes verbatim.
- Delete PROTOCOL_NAME_IK, PROTOCOL_NAME_XK and all IK/XK methods
- Remove Ik/Xk variants from NoisePattern enum
- Rename XX methods to drop xx_ prefix (sole pattern)
- Rename HandshakeMessageType variants from NoiseIKMsg1/NoiseIKMsg2
to Msg1/Msg2, add Msg3 variant (0x03) for XX 3-message flow
- Fix doc comments with correct XX message sizes and descriptions
- Fix stale SessionSetup/SessionAck handshake payload size comments
Replace the 3-message XK handshake with XX for FSP session establishment.
XX requires no prior knowledge of the peer's static key — the responder's
identity is revealed in msg2, the initiator's in msg3.
Key changes:
- session.rs: XX initiator/responder, post-handshake identity verification
using x-only key comparison (parity-independent for npub compatibility),
negotiation payload in msg2/msg3 (FSP version [0,0], features=0)
- Rekey: switched from XK to XX for FSP rekey handshake
- timeout.rs: suppress msg1 resends when target peer is already promoted,
preventing cross-connection session mismatch from duplicate handshakes
- Test template: discovery backoff 3s and handshake timeout 10s for
faster convergence in integration tests
- Integration test timeouts restored to 45s (ping) and 60s (rekey)
Squashed commits:
- Switch FSP handshake from Noise XK to XX
- Fix integration test convergence by reducing discovery backoff
- Fix cross-connection session mismatch from msg1 resend
- Fix FSP identity verification parity mismatch
Replace the 2-message IK handshake with a 3-message XX handshake for
FMP link establishment. XX requires no prior knowledge of the peer's
static key — both identities are revealed during the handshake
(responder in msg2, initiator in msg3). This is the foundation for
the forklift upgrade that enables rolling protocol upgrades.
Changes:
- Noise XX state machine alongside IK/XK (8 unit tests)
- Protocol negotiation payload codec: format byte, packed version
min/max, 64-bit feature bitfield, TLV extensions (11 unit tests)
- FMP wire format version 0→1, msg3 header/builder, TCP stream framing
- FMP handshake switched to XX: PeerConnection 3-message flow,
handle_msg1 simplified (no identity), handle_msg2 sends msg3 and
promotes initiator, new handle_msg3 promotes responder with
restart/rekey/cross-connection detection
- Rekey handshake switched to XX with negotiation payload hash chain
fix (decrypt-and-discard in complete_rekey_msg2/msg3)
- Negotiation payload in msg2/msg3 (FMP version [1,1], features=0)
- Debug logging for handshake promotion paths
- Integration test convergence timeouts adjusted for extra round-trip
Squashed commits:
- Add Noise XX state machine alongside IK/XK
- Add protocol negotiation payload codec
- FMP wire format prep: version 1, msg3 header support
- Switch FMP handshake from Noise IK to XX
- Increase convergence timeouts for XX 3-message handshake
- Fix negotiation hash chain desync in rekey handshake
macOS platform:
- Platform-native TUN interface management with shutdown pipe
- Raw Ethernet transport with macOS socket backend (socket_macos.rs)
- EthernetTransport and TransportHandle::Ethernet ungated from Linux-only
- macOS .pkg packaging (build-pkg.sh, launchd plist, uninstall script)
- CI: macOS build and unit test jobs; x86_64 cross-compiled from
macos-latest via rustup target add x86_64-apple-darwin
Gateway feature flag:
- New opt-in `gateway` Cargo feature activates optional `rustables` dep
- `pub mod gateway` and `Config.gateway` gated behind the feature so
macOS builds never pull in Linux-only nftables bindings
- `fips-gateway` bin has `required-features = ["gateway"]`
- All Linux/OpenWrt/AUR packaging passes `--features gateway`
CI / packaging:
- package-linux, package-macos, package-openwrt now trigger on push to
master/maint/next and on pull requests; release uploads remain tag-gated
- Bloom filter routing fix: fall through to tree routing when no candidate
is strictly closer
- MMP intervals: raise MIN to 1s / MAX to 5s with 5-sample cold-start phase
Add fips-gateway binary: a separate daemon that allows unmodified LAN
hosts to reach FIPS mesh destinations via DNS-allocated virtual IPs
and kernel nftables NAT.
Gateway DNS resolver: forwarding proxy on [::]:53 that intercepts
.fips queries, forwards to daemon resolver (localhost:5354), allocates
virtual IPs from pool, returns AAAA records. Always sends AAAA upstream
regardless of client query type, returns proper NODATA for non-AAAA.
Virtual IP pool: fd01::/112 pool with state machine lifecycle
(Allocated → Active → Draining → Free), TTL-based reclamation,
conntrack integration for session tracking.
NAT manager: nftables DNAT/SNAT rules via rustables netlink API,
per-mapping rule lifecycle, fips0 masquerade for LAN client source
address rewriting.
Network setup: local pool route, proxy NDP for virtual IPs on LAN
interface, IPv6 forwarding validation.
Control socket at /run/fips/gateway.sock with show_gateway and
show_mappings queries. fipstop Gateway tab with pool summary gauge
and mappings table.
Gateway config section in fips.yaml with pool CIDR, LAN interface,
DNS upstream, TTL, and grace period settings.
Design doc at docs/design/fips-gateway.md.
Integration test (testing/static/scripts/gateway-test.sh): three
containers verifying DNS resolution, end-to-end HTTP, NAT state,
TTL expiration, SERVFAIL fallback, and clean shutdown.
The accept_connections gate in handle_msg1() was at the top of the
function, dropping all inbound msg1 packets on transports that don't
accept new connections (e.g. UDP holepunch). This blocked rekey
handshakes on established links, causing repeated "dual rekey
initiation" log floods.
Move the gate below the existing-peer classification so it only
blocks truly new inbound handshakes from unknown addresses. Rekey
and restart msg1s for established peers are now processed normally.
Fixes#47
PR #49
When bloom filter candidates existed but none were strictly closer to
the destination in tree-coordinate distance, find_next_hop returned
None without trying greedy tree routing. This caused NoRoute failures
in topologies where the tree parent was closer but not a bloom
candidate. Fall through to tree routing when no bloom candidate makes
progress.
Raise the report interval floor from 100ms to 1000ms and ceiling from
2000ms to 5000ms. The old 100ms floor produced ~600 reports per 60s
parent evaluation cycle — far more than the ~10 needed for EWMA
convergence. The new floor yields ~60 reports/cycle, still well above
the convergence threshold, while reducing BLE overhead by 10×.
Add cold-start transition: first 5 SRTT samples use the 200ms floor
for fast initial convergence, then switch to the 1000ms steady-state
floor. Session-layer intervals unchanged (500ms–10000ms).
Tailscale (and potentially other routing software) installs a default
IPv6 route in an auxiliary routing table with a policy rule that runs
before the main table. This silently diverts fd00::/8 FIPS traffic
away from the fips0 TUN device.
Add an ip6 rule (priority 5265) during TUN setup that directs fd00::/8
to the main routing table, ensuring the fips0 route is always used.
Tailscale (and potentially other routing software) installs a default
IPv6 route in an auxiliary routing table with a policy rule that runs
before the main table. This silently diverts fd00::/8 FIPS traffic
away from the fips0 TUN device.
Add an ip6 rule (priority 5265) during TUN setup that directs fd00::/8
to the main routing table, ensuring the fips0 route is always used.
- Promote probe connections directly into pool instead of dropping and
reconnecting. Eliminates fragile two-phase connect pattern (probe →
disconnect → reconnect) that caused race conditions on restart.
- send_async fails fast when no connection exists, triggering a
background connect_async instead of blocking the event loop for up
to 10s on inline L2CAP connect. Prevents control socket query
timeouts and MMP processing stalls.
- Add 5-second timeout to pubkey_exchange recv. Without this, a peer
that connects but never sends its pubkey blocks the calling task
forever, killing the scan_probe_loop or accept_loop entirely.
- Add retry timer in scan_probe_loop for addresses that failed probe
but won't get another DeviceAdded from BlueZ (deduplication).
- Clear BlueZ cached devices before starting scan so fresh
advertisements trigger DeviceAdded after daemon restart.
- connect_async now performs pubkey exchange before promoting to pool,
matching the accept_loop's expectation on inbound connections.
- BLE tests updated to pre-establish connections via connect_async
since send_async no longer does inline connect.
- Size receive buffer from negotiated recv_mtu instead of hardcoded 4096
(prevents silent truncation if MTU exceeds buffer size)
- Set advertising interval to 400-600ms for deterministic behavior
instead of depending on BlueZ driver defaults
- Enable power_forced_active on L2CAP sockets to prevent sniff-mode
latency spikes during data transfer (best-effort, logged on failure)
- Log negotiated PHY and MTU at connection establishment for diagnostics
Add log_level field to NodeConfig (case-insensitive, default: info).
The daemon now loads config before initializing tracing so the
configured level takes effect. RUST_LOG env var still overrides if
set.
Remove hardcoded Environment=RUST_LOG=info from systemd units and
OpenWrt procd init script — these prevented config-driven log levels
from working.