Add nft -c -f packaging/common/fips.nft syntax-check to both
testing/ci-local.sh and .github/workflows/ci.yml so a regression in
the 128-line firewall ruleset surfaces in the build gate rather than
when an operator activates fips-firewall.service.
testing/ci-local.sh: first step inside run_build(), before
cargo build --release. Uses command -v nft for prereq detection
mirroring the existing cargo-nextest pattern; records as nft-syntax
in RESULTS. Operator-facing message points at apt install nftables
when nft is absent.
.github/workflows/ci.yml: nftables added to the build job's existing
Linux apt-install step; new Validate fips.nft syntax (Linux only)
step gated on runner.os == 'Linux' (skips macOS/Windows matrix
slots, runs on ubuntu-latest and ubuntu-24.04-arm).
Note: nft -c -f requires netlink cache initialization on modern
nftables even in check mode, so both invocations use sudo (safe in
CI's passwordless sudo, and operator's typical local sudo). Without
sudo, nft fails with "cache initialization failed: Operation not
permitted" before reaching ruleset parse.
Add windows-lint job to .github/workflows/ci.yml running
PSScriptAnalyzer against the three operator-facing PowerShell scripts
(build-zip.ps1, install-service.ps1, uninstall-service.ps1) on the
windows-latest runner. Job runs in parallel with build/test, no
needs:, no tag gating.
Also add packaging/windows/PSScriptAnalyzerSettings.psd1 with two
project-appropriate suppressions:
PSAvoidUsingWriteHost — operator-facing progress in all three
scripts is intentional; Write-Output would conflate progress
with return value.
PSAvoidUsingPositionalParameters — Copy-Item src dst / New-Item
-Path style positional usage is widespread for cp/mv-like
readability.
Severity = @('Error', 'Warning') gates CI on Warnings+ only,
skipping Information-only findings.
The lint job lives in ci.yml rather than package-windows.yml: keeps
the packaging workflow focused, avoids interleaving with the
structural-verification steps in package-windows.yml, and the lint
runs on every push (not just tags).
Extend the gateway integration suite with three previously unexercised
runtime paths. All three share testing/static/scripts/gateway-test.sh
and testing/static/docker-compose.yml so they land as one commit.
6A — UDP port forwarding runtime path. Add udp 18081 -> [fd02::20]:8081
to inject_gateway_config() and a phase-7 case where gw-client runs an
inline Python UDP echo server bound [::]:8081 and gw-server sends a
UDP probe to [GW_MESH_IP]:18081 via inline python3, asserting the
echoed payload prefix. The config layer already accepted proto: udp
(test_port_forwards_same_port_different_proto_ok) but the UDP NAT
rule shape and conntrack handling differ from TCP and were unverified.
Uses Python rather than socat because fips-test:latest does not ship
socat; nc -u IPv6 round-trip semantics are messier than a Python
one-liner.
6B — Second simultaneous TCP forward. Add tcp 18082 -> [fd02::20]:8081
alongside the existing 18080 forward. Phase 7 now greps the daemon's
nft DNAT table for all three rules (18080, 18082, 18081) and runs
HTTP fetches through both TCP forwards with distinct backend payloads
(inbound-forward-ok vs inbound-forward-ok-2) so a misrouted response
fails the assertion.
11A — Concurrent multi-client flows. Add gw-client-2 service to
docker-compose mirroring gw-client (IPv6 fd02::21, IPv4 172.20.1.21).
Phase 3 sets the fd01::/112 route on both. Phase 4 issues DNS lookups
from both, asserts they receive distinct virtual IPs, and queries the
gateway control socket (show_mappings) to confirm exactly 2 active
mappings (5-attempt retry loop tolerates snapshot-publish lag). Phase
5 launches both curl requests concurrently as background processes,
asserts each response. Validates concurrent NAT mappings, pool
contention, proxy NDP under simultaneous LAN-client traffic — all
real-world deployment shape that was not pinned.
Phase 8 reclamation timing unchanged (TTL=5s, grace=5s, 25s wait
covers both mapping ticks generously even with a slight stagger).
Add post-build structural verification steps to the three platform
packaging workflows so structural defects abort before checksum/upload.
The .pkg/.zip/.ipk builds run on tag (or every push for OpenWrt) but
their output layouts were unverified — silent drift could ship missing
binaries, broken plists, or malformed package containers that would
only surface at install time. Field operators rely on manual
validation; CI now catches packaging regressions before release.
macOS (.github/workflows/package-macos.yml):
The pkg is a pkgbuild component flat package; pkgutil --expand yields
the structure but actual files live inside Payload (cpio.gz). The
verification step extracts that with gzip -dc | cpio -i, then asserts:
- usr/local/bin/fips, fipsctl, fipstop all present
- Library/LaunchDaemons/com.fips.daemon.plist present
- plutil -lint on the plist returns zero
Each check prints PASS/FAIL; verifier dumps the full extracted tree on
failure for diagnosis. Codesigning verification deferred (pkgbuild
invocation has no --sign — nothing to verify yet).
Windows (.github/workflows/package-windows.yml):
build-zip.ps1 produces a flat archive (no wrapper directory) via
Compress-Archive -Path "$StagingDir\*". The verifier extracts and
asserts each expected file at the extract root:
fips.exe, fipsctl.exe, fipstop.exe (binaries)
fips.yaml, hosts (config from packaging/common/)
install-service.ps1, uninstall-service.ps1 (from packaging/windows/)
README.txt (generated inline by build script)
Each check prints PASS/FAIL; verifier dumps the full extracted listing
on failure for diagnosis. LICENSE intentionally not asserted: the
build script does not currently ship one, and asserting on it would
false-fail every build.
OpenWrt (.github/workflows/package-openwrt.yml):
Add four post-build verification steps between Build .ipk and SHA-256
hashes. The ipk build runs every push but its contents, shell-script
lint state, and sysctl drop-in syntax were unverified.
Steps:
1. Install shellcheck (conditional — pre-installed on ubuntu-latest;
apt-get install fallback only if missing).
2. Lint init/uci/hotplug/firewall scripts: run shellcheck --shell=sh
--exclude=SC1008,SC2317 against the 5 shipped #!/bin/sh scripts
(fips, fips-gateway, firewall.sh, 99-fips hotplug, 90-fips-setup
uci-default). SC1008 silences the rc.common shebang form; SC2317
silences "unreachable" warnings for externally-invoked rc.common
hooks.
3. Sysctl drop-in syntax: per-line regex check against both
fips-gateway.conf and fips-bridge.conf.
4. Verify ipk structural integrity: extract via tar -xzf (OpenWrt's
actual format despite the misleading "ar archive" comment in
build-ipk.sh), assert all three top-level entries
(control.tar.gz, data.tar.gz, debian-binary), assert 14 file
paths in data.tar.gz (binaries, init scripts, configs, sysctl
drop-ins, hotplug + uci-defaults + sysupgrade keep), and assert
the 4 control-tarball entries plus debian-binary content "2.0".
Full ipk install/runtime test deferred past v0.3.0 scope.
Operator-side note: testing/ci-local.sh NOT modified for the OpenWrt
verifier (operators don't typically have the cross-compile toolchain
locally).
Extend testing/deb-install/test.sh per-distro container test loop with
5 new PASS assertions covering the v0.3.0 nftables firewall baseline
that ships installed-but-disabled (operator opt-in):
1. fips-firewall.service unit present at /lib/systemd/system/
2. fips-firewall.service disabled by default (is-enabled = 'disabled')
3. /etc/fips/fips.nft exists AND is registered as a dpkg conffile
4. /etc/fips/fips.d/ drop-in directory present with mode 755 root:root
5. fips.nft includes the drop-in glob /etc/fips/fips.d/*.nft
Assertions slot in between the existing fips-dns service-state check
and the simulated-boot service-start block. Each runs against every
distro in the existing matrix (debian12 / ubuntu24 / ubuntu26).
The security claim — services on fips0 are not exposed by default —
remains end-to-end-validated separately by the testing/firewall/
suite. This commit specifically pins the static install state so
packaging regressions surface in the per-push deb-install gate
rather than at operator opt-in time.
Cover two adjacent runtime behaviors in the discovery state machine
that were previously unpinned at the test level.
1. Open-discovery startup sweep iterate-filter-queue contract.
Cover the runtime sweep behavior: iterate advert cache, apply
skip-filters (own-pubkey, already-connected peers), queue eligible
entries to retry_pending. The config layer was tested but the sweep's
own filtering logic was unpinned.
src/discovery/nostr/runtime.rs: add #[cfg(test)] impl block with
three pub(crate) helpers — new_for_test() builds a minimal
NostrDiscovery with empty cache and no relays/background tasks (uses
fresh nostr::Keys signer + Client::builder().autoconnect(false));
cached_advert_for_test() wraps an OverlayEndpointAdvert into a
CachedOverlayAdvert valid for 1h; insert_advert_for_test() writes
direct to the advert_cache RwLock. All three vanish from release
builds via cfg-gating.
src/node/lifecycle.rs: visibility-only widen on
run_open_discovery_sweep from private async fn to
pub(in crate::node) async fn so the in-tree test can drive it
directly. Same pattern as already-pub(in crate::node) handlers in
src/node/handlers/.
src/node/tests/discovery.rs: add #[tokio::test]
test_open_discovery_sweep_queues_eligible_skips_filtered. Builds
Node + Arc<NostrDiscovery>, injects 3 adverts (eligible, already-
connected peer, own-pubkey), invokes the sweep, asserts retry_pending
contains exactly the eligible entry with matching peer_config npub
and the two filtered entries do NOT appear.
2. Per-attempt timeout state machine in check_pending_lookups.
Cover the central new behavior of f16b837: the [1, 2, 4, 8] retry
sequence (cumulative deadlines 1100/3100/7100/15100ms), one fresh
LookupRequest per attempt, and final-timeout reaching the unreachable
state. The opt-in DiscoveryBackoff machinery was well-tested but inert
at default config; this pins the state machine that runs by default.
Add test_check_pending_lookups_default_sequence_unreachable to
src/node/tests/discovery.rs. Constructs a Node with a peer that has
the target in its bloom but cannot respond (no Noise session — the
state-machine bookkeeping is independent of wire-send success).
Drives check_pending_lookups deterministically through:
t=1100 → second attempt; entry.attempt advances; req_initiated++
t=3100 → third attempt; entry.attempt advances; req_initiated++
t=7100 → fourth attempt; entry.attempt advances; req_initiated++
t=15099 → no-op (one ms before final deadline)
t=15100 → final timeout
At t=15100 asserts: pending_lookups[target] removed; resp_timed_out
counter +1 (this is the actual counter name); pending_tun_packets
[target] removed (queued packet dropped); a frame on the TUN sender
with IPv6 + next_header=58 + ICMPv6 type=1 (Destination Unreachable).
Fresh-request_id-per-attempt is structurally guaranteed: LookupRequest
::generate() unconditionally calls rand::random::<u64>(), and the
test asserts req_initiated increments by exactly 1 per retry (proving
initiate_lookup runs fresh each time, not a resend of cached state).
The originator's request_id isn't stored on the originator side
(deliberately omitted from recent_requests so the response is
recognized as "ours"), so direct request_id capture is not feasible
and counter-tick is the load-bearing observable.
No production logic touched; no visibility widening needed
(check_pending_lookups was already pub(in crate::node)).
Cover the security-relevant defensive signal: sustained decrypt
failures indicate key drift or active probing; threshold-trip force-
removes the peer.
src/peer/active.rs (+43): two unit tests on the counter struct itself
- test_increment_decrypt_failures_monotonic asserts each
increment_decrypt_failures() call returns count+1 for at least 25
iterations
- test_reset_decrypt_failures_zeroes_counter asserts the reset
helper zeroes a non-zero counter and is idempotent
src/node/tests/decrypt_failure.rs (new, 93 lines): end-to-end test
- Builds a Node + connected peer via existing make_completed_connection
/ add_connection / promote_connection harness so peers_by_index is
exercised, not just peers
- Drives the peer to threshold-20 by calling handle_decrypt_failure 20
times; asserts iterations 1..20 leave the peer registered with
monotonically increasing counter, then iteration 20 evicts from both
peers and peers_by_index
src/node/handlers/encrypted.rs: visibility-only widen on
handle_decrypt_failure from private to pub(in crate::node) so the
in-tree test can drive the threshold logic without re-implementing
it. Same pattern as the already-pub(in crate::node)
handle_encrypted_frame in the same file.
Threshold pinned: DECRYPT_FAILURE_THRESHOLD = 20 at
src/node/handlers/encrypted.rs:11.
Add hand-rolled JSON snapshot harness in src/control/queries.rs to
detect silent schema drift in operator-facing control-socket
responses. Builds a Node with deterministic identity
(Identity::from_secret_bytes(&[0xAB; 32])), invokes each of the 18
show_* handlers, redacts 17 volatile fields (version, pid, exe_path,
control_socket, tun_name, allow_file, deny_file, *_ms / *_secs_ago /
uptime_secs), sorts object keys recursively, and compares against a
fixture in src/control/snapshots/.
First run writes snapshots and passes; subsequent runs enforce.
Future schema changes show as a snapshot diff that operators update
intentionally — not a stability contract, just a tripwire so drift
is never silent.
A 19th meta-test dispatch_covers_all_snapshotted_handlers walks every
name through dispatch() to confirm each returns status: ok and trips
if a 19th handler is added without a matching snapshot.
No new dependencies (insta deliberately not added; Cargo.toml
[dev-dependencies] keeps tempfile + criterion only). 18 fixture
files added, ~544 lines combined; harness is 367 added lines, all
inside #[cfg(test)] mod tests.
Add #[cfg(target_os = "macos")] unit tests catching macOS-specific
regressions before they reach the macos-latest GitHub runner.
src/upper/tun.rs: surgical refactor extracts the inline
AF_INET6_HEADER constant into module-scope helpers
utun_af_inet6_header() (encode) and parse_utun_af_prefix() (decode
inverse for round-trip testability). TunWriter::run now calls the
helper instead of the inline const; behavior unchanged. Six new
tests pin the AF=30 constant matching Darwin, big-endian byte order,
encode/parse round-trip, short-buffer rejection, minimum header
acceptance with trailing payload, and no-panic on garbage bytes.
src/transport/ethernet/socket_macos.rs: existing test mod already
covered bpf_wordalign and 5 parse_next_frame cases. Three new tests
fill genuine gaps: struct layout pin against kernel ABI, caplen-
overrun rejection, full Ethernet header round-trip via parse.
Existing tests pre-exist; only adding to the same #[cfg(test)] block.
Add test_routing_bloom_hit_not_closer_falls_through_to_tree to
src/node/tests/routing.rs covering the regression class fixed in
a859da7: bloom candidates exist BUT none are strictly closer to
destination than the tree parent. Pre-fix find_next_hop returned
None (NoRoute); post-fix it falls through to greedy tree routing.
Existing tests covered bloom-hit-closer, bloom-preferred-over-tree,
no-bloom-hit-tree-fallback, and bloom-hit-without-coords — but the
in-between case was unpinned.
Topology: self is tree root with two children (tree_peer at distance
1, bloom_peer at distance 3); destination is one hop below tree_peer
(self distance 2). Only bloom_peer advertises dest in its filter.
Distance of the only bloom candidate (3) is not strictly less than
self's (2), so select_best_candidate returns None and the call must
fall through to greedy tree routing returning tree_peer. Asserts
explicit identity (assert_eq! tree_peer_addr) and rules out the
wrong-peer regression (assert_ne! bloom_peer_addr).
Add table-driven unit test test_log_level_parser to src/config/node.rs
covering all 5 explicit match arms (trace, debug, warn|warning, error),
the implicit None-and-unknown → INFO default, case-insensitivity via
to_lowercase (TRACE / Debug / Warning / WARN / ERROR / INFO), and
edge cases (empty string, "verbose"). Pins observed behavior: there
is no explicit "info" arm — it falls through the wildcard to INFO,
identical to unknown strings.
Add 6 negative-input unit tests to src/discovery/nostr/stun.rs covering
truncated header (all lengths 0..20), bad magic cookie, unknown attribute
type (skip-not-error), truncated XOR-MAPPED-ADDRESS, length-overflow
attribute, and transaction-ID mismatch. The happy path was exercised by
the 3 NAT scenarios but the parser had no negative-input coverage.
Tests pin observed behavior: parse_stun_binding_success returns
Option<SocketAddr>, so all malformed-response cases assert None rather
than an error variant. Unknown TLVs are silently skipped via the loop's
default arm; length-overflow triggers the value_end > packet.len() guard
and breaks out of the loop without panicking.
When a transit forwarder drops an oversized data packet and reports
the bottleneck back via MtuExceeded, the receive-side handler
already updates per-session MmpSessionState::path_mtu (used by PTB
synthesis to feed kernel TCP). It did not, however, update
path_mtu_lookup — the per-destination map the TUN reader/writer
consult at TCP MSS clamp time. So forward-path-asymmetry flows kept
clamping at the discovery reverse-path value (too generous for the
actual forward-path budget) on every subsequent SYN.
Add the missing write at the same point apply_notification runs.
Keep the tighter of existing-or-new — the clamp must never loosen.
Same write-shape as seed_path_mtu_for_link_peer.
Tests:
- Three focused unit tests on handle_mtu_exceeded for the empty,
tighten, and keep-tighter cases.
- Extended test_multihop_pmtud_heterogeneous_mtu to assert the
lookup tightens after the wire-level MtuExceeded propagation,
alongside its existing PathMtuState assertion.
Adds two #[cfg(test)] accessors on Node (path_mtu_lookup_get /
path_mtu_lookup_insert) and bumps handle_mtu_exceeded to
pub(in crate::node) for direct test invocation.
The B3 path_mtu_lookup plumbing landed without updating the
windows_tun::run_tun_reader signature or its inner handle_tun_packet
call, breaking the Windows build. Linux/macOS variants and
TunWriter (Windows) were already plumbed; this brings the Windows
reader into line.
No behavioral change on any platform.
Adds source-side TCP MSS clamping informed by per-destination path
MTU learned via discovery, with a conservative IPv6-minimum-derived
ceiling for cold flows where discovery has not yet completed. Closes
the multi-hop default-config TCP wedges observed in production where
a sender's local-floor MSS exceeds what some intermediate forwarder
hop is willing to carry: silent drops, no PTB feedback through the
userspace TUN to the kernel TCP stack, retransmits at the same too-
large MSS, application connection times out.
## Architecture
A new `Arc<RwLock<HashMap<FipsAddress, u16>>>` field
`path_mtu_lookup` on Node mirrors the per-destination path MTU in a
form accessible from sync TUN reader/writer threads. A new
`per_flow_max_mss` helper in `src/upper/tun.rs` reads the lookup at
SYN-clamp time and returns the appropriate ceiling for the flow.
Three write sites populate `path_mtu_lookup`:
1. **Discovery originator branch** of `handle_lookup_response`:
the path MTU bottleneck accumulated through the reverse path
lands here when a LookupResponse arrives at the originator.
Same value also lands in `coord_cache` per the existing
`insert_with_path_mtu` API.
2. **FMP peer-promotion seed** (`seed_path_mtu_for_link_peer`):
when an FMP link-layer peer is promoted to active, the local
outgoing-link MTU on the peer's transport seeds the lookup.
Tighter existing values (learned via discovery) are preserved;
the seed only writes when no entry exists or the existing
value is looser than the link MTU. Without this seed,
directly-configured peers (auto_connect / static peer config)
would leave `path_mtu_lookup` empty for their FipsAddress
because the FSP session establishes without ever issuing a
LookupRequest.
3. **Target-edge fold at `send_lookup_response`**: when a node is
the discovery target, it folds its own outgoing-link MTU to
the response's next-hop into `path_mtu` before sending.
Without this fold, the response leaves the target with
`path_mtu = u16::MAX` and only intermediate transits min-fold;
the target's first reverse-path hop is never represented in
the bottleneck calculation. Refactored the existing transit-
side min-fold into a shared `apply_outgoing_link_mtu_to_response`
helper called from both sites.
## Read-side: per_flow_max_mss
Two TUN call sites consume the lookup:
- Outbound `handle_tun_packet` clamps SYN MSS using packet[24..40]
(IPv6 destination) as the lookup key.
- Inbound `TunWriter::run` clamps SYN-ACK MSS using packet[8..24]
(IPv6 source).
When the lookup contains a learned value, the helper computes
`min(global_max_mss, effective_ipv6_mtu(path_mtu) - 60)` where 60
is IPv6 (40) + TCP (20) headers and `effective_ipv6_mtu` accounts
for the FIPS encapsulation overhead.
When the lookup is empty for a destination — the cold-flow case —
the helper returns `min(global_max_mss, IPv6-minimum-derived
ceiling)`. RFC 8200 mandates every IPv6 path accept ≥1280-byte
packets, so the IPv6-minimum-derived MSS (1280 - 77 - 60 = 1143)
fits any compliant path. Without this conservative ceiling, the
first SYN to a destination with no learned path MTU exits the TUN
at the kernel-natural MSS (TUN MTU - 60), and the application
connection wedges silently before discovery completes for a
corrected second SYN to fire. The fix is provably safe: the
ceiling is taken with `min` against the local global so operators
with even tighter local floors are never loosened upward.
Subsequent flows pick up the actual learned per-destination value
once discovery (or the FMP-promotion seed for direct peers)
populates the lookup.
## Diagnostic logging
All write and read sites emit instrumentation suitable for
operators bisecting a wedged path:
- `debug!` log on every `path_mtu_lookup` write (discovery
originator path and FMP-promotion seed path), showing the
FipsAddress, written value, prior value, and post-write map
size. `warn!` on poisoned-lock failure path.
- `trace!` log per `per_flow_max_mss` call covering every
fall-through branch (wrong addr_bytes length, non-fd::/8
prefix, lookup poisoned, no entry for destination, empty-lookup
conservative ceiling) and the success path. trace level filters
out under normal log settings; capture with
`RUST_LOG=info,fips::node::handlers::discovery=debug,fips::upper::tun=trace`.
## Tests
15 new unit tests across 3 files:
- `per_flow_max_mss` (8 tests in `src/upper/tun.rs::tests`):
empty-lookup conservative ceiling, empty-lookup global-smaller
floor, learned-value-overrides-conservative, per-destination
smaller, per-destination larger capped by global, non-fips
addr, short addr slice, per-destination independence.
- `seed_path_mtu_for_link_peer` (4 tests in
`src/node/tests/unit.rs`): seed when empty, keep tighter
existing, tighten looser existing, no-op for unknown
transport.
- Discovery integration (3 tests in `src/node/tests/discovery.rs`):
apply_outgoing_link_mtu_to_response on unknown peer no-op,
two-node target-edge fold (path_mtu reflects target-edge link),
three-node chain transit min-fold (existing test, updated for
target-edge inclusion).
Two pre-existing discovery tests had assertions updated to
account for the target-edge fold:
- `test_response_path_mtu_two_node`: previously asserted
`u16::MAX` (no transit to min-fold); now asserts 1280 (the
test transport MTU, folded in by send_lookup_response).
- `test_response_path_mtu_four_node_chain`: previously asserted
1350 (transit MTUs only); now asserts 1280 (target-edge MTU
is the bottleneck).
- `test_transit_forwards_when_mtu_sufficient`: previously
asserted 1400 (transit MTU only); now asserts 1280 (target-
edge MTU is the bottleneck).
## Verification
Local CI on this commit: 29/29 suites pass, 1105 lib tests pass,
clippy --all-targets --all-features -D warnings clean, cargo fmt
clean. Production deploy verified via trace capture across the
managed fleet: cold-flow conservative ceiling branch fires on
first SYN, learned-lookup branch takes over once discovery
completes, both behaviors observable end-to-end at the SYN MSS
on the wire.
No wire-format change. No config-format change.
The new GitHub Actions Clippy job introduced in a41f80a failed on the
first push because dtolnay/rust-toolchain@stable installed the stable
channel with components: clippy, but the repo's rust-toolchain.toml
overrides to channel "1.94.1" with components ["rustfmt"] only. cargo
clippy then re-resolves the toolchain to 1.94.1 and bombs because the
clippy component isn't installed for that specific version.
Fix: add clippy to the toolchain override's components list. This
makes the rust-toolchain.toml the single source of truth for which
components every cargo invocation needs, and the GitHub Actions
toolchain-installation step picks them up automatically.
Local clippy continues to pass with this change; verified with
cargo clippy --all-targets --all-features -- -D warnings.
Adopted ephemeral UDP transports created by adopt_established_traversal()
default to UdpConfig::default() (MTU=1280, IPv6 minimum) when the
bootstrap runtime hands a socket without an explicit transport_config
override. This is by design: NAT-traversal middlebox MTU is unpredictable
and the IPv6 minimum is the only value guaranteed by spec to survive
arbitrary paths.
Add an explanatory comment at the call site so future readers find the
rationale without spelunking through ISSUE-2026-0013, and so any future
change to the inheritance behavior is a deliberate decision rather than
an accidental refactor.
No behavior change.
Default-config TCP flows between fips peers were stalling completely
(cwnd-pinned, 0 bps for 10s+) on a non-trivial fraction of restarts.
Reproducible with iperf3 between any two peers.
Root cause: `Node::transport_mtu()` iterated `self.transports.values()`
(HashMap with default RandomState hasher) and returned `handle.mtu()`
of the first one whose `is_operational()` returned true. Two stacked
sources of non-determinism stacked on each other: HashMap iteration
order is randomized per-process via RandomState, and async transport
`.start()` completion order races each daemon restart.
The returned value drives the TCP MSS clamp ceiling computed once at
TUN init (src/upper/tun.rs:501-524) and stored as immutable max_mss
in the reader/writer thread state. When the picker landed on a
transport with MTU > 1357 (any non-UDP-1280 in the standard fleet
defaults), `max_mss > 1220` (kernel's natural fips0-MTU-derived MSS),
the daemon's clamp was silently a no-op, and the kernel emitted
1220-byte segments. Those wrap into 1280-byte IPv6 → 1357-byte fips
datagrams that exceed UDP-1280 transports at any forwarding hop,
causing silent drops with no PTB feedback to the kernel TCP stack.
Fix: return min across operational transports instead of first-iterated.
With UDP-1280 in the configured set (the common case),
`transport_mtu = 1280`, `max_mss = 1143 < 1220`, daemon's clamp
engages, MSS=1143 reaches the wire, packets fit, throughput recovers.
Empirical green light from a single-UDP-config end-to-end test:
iperf3-without-`-M` recovered to ~21 Mbps with no operator-side nft
TCPMSS rules.
Adds three unit tests:
- transport_mtu_returns_min_across_operational: pin selection to
smallest MTU when multiple operational transports differ.
- transport_mtu_fallback_when_no_operational_transports: 1280 fallback.
- transport_mtu_min_with_single_operational: trivial single-transport
case.
The `effective_ipv6_mtu` field reported by `fipsctl show status` was
also racy (consequence of the same bug); fixed by this change as a
side effect.
The local ci-local.sh and the GitHub CI clippy invocations both used
`cargo clippy --all -- -D warnings`, which only checks lib + bin
targets. Test code, integration tests, and benches were not lint-gated.
Three pre-existing clippy errors lurked in test modules as a result
(two field_reassign_with_default in config tests, one
items_after_test_module in stun.rs).
Tighten both invocations to `cargo clippy --all-targets --all-features
-- -D warnings` so the gate covers everything cargo can build, and
fix the three exposed errors:
- src/config/mod.rs: rewrite two test-only `Config::default()` +
field-reassign sites to struct-update syntax.
- src/discovery/nostr/stun.rs: move helper `random_txn_id` above the
`#[cfg(test)] mod tests` block.
Also adds a dedicated Clippy job to the GitHub CI workflow so the
strict gate runs on every PR (the workflow had no clippy job before;
clippy ran only via testing/ci-local.sh on operator machines).
No behavior changes; lint hygiene + CI hardening only.
Under `node.discovery.nostr.policy: open`, the per-tick auto-dial in
`queue_open_discovery_retries` was supposed to pick up adverts cached
from the relay subscription backlog at startup, but in practice only
adverts arriving live (after the daemon was up) were being dialed.
Backlog adverts sat in the in-memory cache until they aged out.
Adds a one-shot startup sweep that runs once per daemon start, gated
identically to the per-tick sweep (`enabled` && `policy == open`),
after a configurable settle delay so the relay subscription backlog
has time to populate the advert cache. The sweep iterates the cache
with the same skip-filters as the per-tick path (statically-configured
peers, already-connected, retry-pending, connecting) plus a tighter
age filter: only adverts whose `created_at` is within
`startup_sweep_max_age_secs` of now are queued.
Two new config fields under `node.discovery.nostr`:
- `startup_sweep_delay_secs` (default 5)
- `startup_sweep_max_age_secs` (default 3600 = one hour)
Both are only consulted when `policy == open`; under any other
policy the sweep is a no-op.
Adds diagnostic logging to the open-discovery sweep so operators can
verify what the auto-dial path is doing on each daemon bring-up:
info-level on each retry-queued enqueue (with peer short-npub and
advert age), and a one-line summary on every startup sweep and on
any per-tick sweep that queues at least one retry. The summary
breaks down skipped candidates by reason (age, configured, self,
already-connected, retry-pending, connecting, no-endpoints,
invalid-npub) — currently the path was silent so there was no
operator-visible signal that the cache iteration was running.
Refactors the existing `queue_open_discovery_retries` body into a
shared `run_open_discovery_sweep(max_age_secs, caller)` helper so
the per-tick and startup paths share filter/queue logic and only
differ in the age filter and log label. Surfaces `created_at` from
`NostrDiscovery::cached_open_discovery_candidates` (return tuple
extended) so the age filter has the data it needs.
Three new unit tests in `config::node::tests` cover the new defaults,
YAML override round-trip, and partial-YAML default fallback.
The Nostr overlay advert publisher serialized `transport: tor`
endpoints as a bare `<onion>.onion` hostname with no port. The Tor
address parser requires `<host>:<port>` form and rejected the bare
shape with `expected host:port`. Any peer receiving a Tor-only
advert went into a persistent retry-fail loop on jittered backoff
until the advert aged out of the discovery cache. The bug had been
latent for as long as Tor adverts have been published on Nostr, and
was masked in deployments where every node also advertised a
non-Tor transport (peers fell through to the working endpoint).
Surfaced first on a deployment where Tor was the only advert path.
Publisher now emits `<onion>.onion:<port>` using a new
`transports.tor.advertised_port` config field that defaults to 443,
matching the Tor `HiddenServicePort 443 127.0.0.1:<bind_port>`
convention. Operators whose torrc uses a non-default virtual port
can override.
Adds a unit test that pins the publisher/parser contract: formats
the advert exactly as the publisher does and asserts `parse_tor_addr`
accepts the result; asserts the bare-onion form (the bug) does not
parse, catching any future regression that drops the port again.
Parser is unchanged (already correct).
Companion to the ethernet `accept_connections: false` rekey-deadlock
fix from earlier this release: the same dual-init failure mode shows
up over UDP when peers register by hostname, and the existing
addr_to_link-only carve-out in `should_admit_msg1` doesn't cover it.
The carve-out's first predicate keys `addr_to_link` by the literal
`TransportAddr` that `initiate_connection` inserted, which is the
hostname-form when a peer config carries a hostname (e.g.,
`core-vm.tail65015.ts.net:2121`). Inbound packets always arrive with
numeric source addrs because `udp_receive_loop` builds the
`TransportAddr` from the `SocketAddr` the kernel reports via
`recvfrom`. `TransportAddr` equality is byte-exact, so the two forms
don't match and the lookup misses. With `udp.accept_connections:
false` (or `udp.outbound_only: true`, which forces it false) the
gate then rejects the rekey msg1 from an established peer. The
dual-init tie-breaker stalls because the loser side never produces
msg2; both sides retry indefinitely and the winner side keeps
logging "Dual rekey initiation: we win, dropping their msg1" at 1Hz.
The earlier ethernet fix didn't generalize to this variant because
ethernet TransportAddrs are always numeric MAC bytes — both the
config-time form and the inbound-arrival form match identically.
Add a second predicate to `should_admit_msg1`: an active peer's
`current_addr()` matching `(transport_id, remote_addr)`.
`current_addr` is updated and refreshed from inbound encrypted-frame
source addrs (`handlers/encrypted.rs`), which are always numeric
`SocketAddr`-form, so this catches the established peer regardless
of how its `addr_to_link` key was originally inserted. The fast
`addr_to_link` check stays first; the iteration over peers is
bounded by peer count and only runs when the first predicate misses.
Regression coverage in this commit:
- Unit test `test_should_admit_msg1_admits_rekey_when_addr_form_differs`
in `src/node/tests/handshake.rs`. Constructs the failing scenario
in-process: `addr_to_link` populated with hostname-form key, peer's
`current_addr` at the resolved numeric form, query with numeric form.
Without the new predicate this fails immediately.
- New integration topology `rekey-outbound-only` plus matching
docker-compose profile. Same 5-node mesh shape as `rekey-accept-off`
but `inject-config` sets `udp.outbound_only: true` on node-b and
rewrites node-b's peer-c address from the numeric docker IP to the
docker hostname (`node-c:2121`), reproducing the production
hostname-vs-numeric mismatch. The test asserts no sustained
"Dual rekey initiation: we win" log lines on any node (>10 = bug)
and the existing rekey health checks catch the connectivity loss
the loop produces.
- `testing/ci-local.sh` and `.github/workflows/ci.yml` extended to
run the new variant in the local sweep and the GitHub CI integration
matrix alongside `rekey` and `rekey-accept-off`.
Verified locally: full `bash testing/ci-local.sh` sweep passes 29/29
suites (23m 12s) with the new variant green; 1084 unit tests pass.
Make Nostr-mediated overlay discovery unconditional, mirroring the
philosophy of PR #79's collapse of the tui/ble/gateway features in
favor of platform cfg gates. nostr / nostr-sdk are pure Rust over
WebSockets/TCP, so they build cleanly on every FIPS-supported
platform — there is no need for the parallel feature gate.
The flag was already in `default = [...]`, so no behavior change for
anyone using `cargo build` without `--no-default-features`. Operators
who explicitly disabled the feature will now find Nostr code present
in the binary; the runtime check `node.discovery.nostr.enabled` still
controls whether the runtime starts.
Cargo.toml:
- Remove the `[features]` table entirely.
- Drop `optional = true` from `nostr` and `nostr-sdk`.
Source: 27 cfg sites collapsed across 5 files —
`src/discovery.rs`, `src/discovery/nostr/mod.rs`,
`src/node/handlers/rx_loop.rs`, `src/node/lifecycle.rs`,
`src/node/mod.rs`. Two `#[cfg(not(feature = "nostr-discovery"))]`
fallback blocks (the udp:nat-without-runtime debug-log path and the
"feature not compiled in" warning at startup) were removed as dead
code; the always-on path already handles the missing-runtime case
via `nostr_discovery: Option<NostrDiscovery>`.
Packaging and tooling:
- `packaging/openwrt-ipk/Makefile`: drop a stale `--features gateway`
flag (the `gateway` feature was already removed in PR #79; this
was a leftover that the build path tolerated only because cargo
ignored unknown feature names).
- `testing/scripts/build.sh`: drop `DEFAULT_CARGO_BUILD_ARGS=(--features
nostr-discovery)`; defaults are empty.
- `packaging/common/fips.yaml`: drop the "requires the
nostr-discovery feature" comment from the discovery section.
Bundled cleanup:
- Apply `cargo clippy --fix` against three pre-existing warnings in
`src/discovery/nostr/runtime.rs` and `src/discovery/nostr/stun.rs`
(collapsed `if let Some` chain; two redundant `as i32` casts).
These were always present but masked when the feature gate was
off; they surface now that the code is unconditionally compiled.
- `cargo fmt` settled two minor formatting drift sites in
`src/bin/fips-gateway.rs` and `src/config/mod.rs`.
Tests: 1083 passed, 0 failed, 4 ignored. clippy clean. fmt clean.
Single combined commit covering five interlocking pieces of test and
CI work that landed during the v0.3.0-prep cycle.
## fips-gateway robustness
- src/bin/fips-gateway.rs DNS upstream probe converted from a 3-second
hard-fail to a bounded retry loop (5 attempts × 1s timeout, 1s sleep
between attempts; ~10s worst case). Covers the cold-boot race where
the daemon's TUN is up but the DNS responder at [::1]:5354 is still
binding. Each failed attempt logs at INFO. In production the binary's
retry is the live recovery mechanism; with retry it recovers silently
instead of relying on Restart=on-failure (~5s blip + spurious ERROR
per cycle).
- packaging/debian/fips-gateway.service `ExecStartPre` now waits up to
30 seconds for the daemon's `fips0` TUN to appear before exec'ing
the gateway binary. Eliminates the cold-boot race where the gateway
exits with `fips0 interface not found` and recovers via
`Restart=on-failure`, producing a 5-second blip and a spurious error
log per restart cycle.
- testing/docker/entrypoint.sh gateway-mode waits up to 30s for the
daemon's DNS responder to bind [::1]:5354 (probes once per second
with `dig @::1 -p 5354 ... test.fips`) before exec'ing fips-gateway.
Belt-and-suspenders with the binary's own retry: in CI we want
deterministic startup ordering. On timeout, fall through so the
binary's probe reports the definitive error.
## Test infrastructure DNS bind migration to ::1
After session 359's daemon DNS-bind default flipped from `127.0.0.1`
to `::1` (the production fix for ISSUE-2026-0002), the static-test
infrastructure was carrying a stale workaround that overrode the
default back to IPv4 loopback. The fips-gateway integration test
exposed the divergence: the gateway probes its DNS upstream at
`[::1]:5354` (production default) while the daemon was binding
`127.0.0.1:5354` from the template override — IPv6-explicit sockets
do not accept v4-mapped traffic, so the upstream probe exhausted
retries and the gateway exited.
- Drop the explicit `bind_addr: "127.0.0.1"` line from every test
config that emits it: testing/static/configs/node.template.yaml,
testing/chaos/configs/node.template.yaml, the sidecar heredoc in
testing/docker/entrypoint.sh, testing/acl-allowlist/generate-configs.sh
(six per-node blocks), testing/nat/scripts/generate-configs.sh, and
the four tor templates under testing/tor/. Daemon picks up its
production `::1` default.
- Flip the dnsmasq forwarder for `.fips` in testing/docker/Dockerfile
from `127.0.0.1#5354` to `::1#5354` so dnsmasq on the shared test
image continues to reach the daemon. Template and Dockerfile must
move together since most static suites resolve `<npub>.fips` via
the test-image dnsmasq.
## rekey-accept-off integration variant + UDP unit test
- New `rekey-accept-off` topology and docker-compose profile under
testing/static/. 2-node variant where node-b runs with
`udp.accept_connections: false`. Pins the regression class that
ISSUE-2026-0004 fixed (cross-connection winner's rekey msg1 was
being filtered by the accept_connections gate, breaking rekey).
- testing/static/scripts/rekey-test.sh accepts REKEY_TOPOLOGY and
REKEY_ACCEPT_OFF_NODES env vars; its inject-config subcommand
applies the per-node `udp.accept_connections: false` edit, and
the test asserts no sustained "Dual rekey initiation" log lines.
- New UDP variant of `should_admit_msg1` admit-rekey unit test in
src/node/tests/handshake.rs.
## ci-local.sh full integration coverage
- New runner functions and dispatcher entries for `acl-allowlist`,
`nat-cone` / `nat-symmetric` / `nat-lan`, `rekey-accept-off`,
`dns-resolver`, `deb-install`. Each integrates with the existing
summary tracking via `record`.
- New `--with-tor` flag (off by default) gates `tor-socks5-outbound`
and `tor-directory-mode` runners. Tor stays opt-in because both
harnesses depend on the live Tor network and would introduce a
flake source unrelated to the FIPS code.
- New suite arrays (`ACL_SUITES`, `NAT_SUITES`, `DNS_RESOLVER_SUITES`,
`DEB_INSTALL_SUITES`, `TOR_SUITES`) drive both the default sweep
and `--list` output.
- `run_suite` extended to accept the new suite names for `--only`
invocations.
## GitHub CI matrix expansions
- `gateway` matrix entry runs testing/static/scripts/gateway-test.sh
against the existing docker-compose `gateway` profile.
- `rekey-accept-off` matrix entry exercises the new topology with
REKEY_ACCEPT_OFF_NODES=b.
- `deb-install` matrix (debian12 + ubuntu24 + ubuntu26) runs
testing/deb-install/test.sh with privileged systemd containers.
~5-7 min cold cache, ~2 min warm per distro. Self-contained: builds
its own .deb in a Debian 12 cargo-deb builder image; does not
depend on the build job's pre-built artifact.
- `dns-resolver` matrix entry runs the full 13-scenario harness
(per-distro systemd resolver-backend tests + real-fips end-to-end
scenarios) in a single job. Pins the production DNS bind path that
ISSUE-2026-0002 lived in. ~7-12 min warm, ~12-15 min cold.
Verified locally: full `bash testing/ci-local.sh` sweep passes,
including 5/5 deb-install distros and all 13 dns-resolver scenarios.
Tor-inclusive sweep (`--with-tor`) verified in a follow-up run.
Three related UDP transport changes that together close a real gap in
the v0.2.x "this transport accepts inbound" assumption:
- outbound_only (default false). When true, the transport binds a
kernel-assigned ephemeral port (0.0.0.0:0) regardless of the
configured bind_addr, refuses inbound handshakes (Transport trait's
accept_connections() returns false), and is never advertised on
Nostr regardless of advertise_on_nostr. Lets a node participate in
the mesh as a pure client — initiate outbound links without
exposing an inbound listener on a known port. Also closes the
"loopback bind as outbound-only workaround" trap: a UDP socket
bound to 127.0.0.1 pins 127.0.0.1 as the source IP on outbound
packets, and Linux refuses to deliver such packets out an external
interface — the daemon happily reports "transport started" while
no flow ever reaches an external peer.
- accept_connections (default true). Mirrors the existing
Ethernet/BLE knob. Lets operators run UDP in a "client" posture
(initiate outbound, refuse inbound msg1 from new addresses) without
switching transport. The Node-level handshake gate already carves
out msg1 from peers established on the transport so rekey works
on existing sessions.
- Startup validation: reject `transports.udp[*].bind_addr` set to a
loopback address (127.x.x.x, ::1, localhost) when at least one peer
has a non-loopback UDP address. Replaces the silent "peer link
won't establish" failure mode with a clear error pointing at the
bind misconfiguration. outbound_only is exempt (it overrides
bind_addr to 0.0.0.0:0).
The is_punch_packet-based filter from the previous commit, the
Node-level admission gate landed earlier on master, and these new
config fields together cover the three distinct ways the v0.2.x
"this transport accepts inbound" assumption could break.
Tests: validation truth table (loopback+external rejected,
loopback+loopback ok, outbound_only exempt), is_loopback_addr_str
helper, accept_connections wiring (default, explicit-false,
outbound_only-forces-false), end-to-end ephemeral-bind in the runtime.
1082 tests pass with --features nostr-discovery.
Add a default-deny nftables ruleset for the fips0 mesh interface as
a packaged operator asset, with a companion fips-firewall.service
oneshot unit for systemd hosts. Both are shipped disabled — the
baseline is an operator conffile and the unit is intentionally not
enabled in postinst. Activation is an explicit one-liner:
sudo systemctl enable --now fips-firewall.service
This is deliberate: silently mutating host firewall state on package
install is hostile across the axes that matter (collisions with
existing operator nftables / Docker / OPNsense rulesets, surprise
behaviour for hosts that already filter elsewhere, conversion of an
explicit security decision into an invisible one). The opt-in
posture preserves operator agency.
The baseline closes a real default-exposure gap: any service on a
mesh host bound to a wildcard address (0.0.0.0 or [::]) is
reachable from every authenticated peer in the mesh by default.
Identity on the mesh is the peer's npub but identity is not
authorization, and the mesh is closer to a shared LAN than to the
public internet. With this filter loaded, the surface is closed
unless a drop-in opens it explicitly.
Baseline shape:
- Early-return for non-fips0 traffic (every other firewall left
undisturbed)
- conntrack established/related accept (replies to outbound flows)
- ICMPv6 echo-request accept (ping6 reachability)
- include "/etc/fips/fips.d/*.nft" — operator-supplied allowances
- counter drop default
The accompanying docs/fips-security.md lays out the threat model
(npub-authenticated mesh is closer to a shared LAN than to the
public internet — identity is not authorization), the activation
workflow, drop-in extension recipes (allow inbound SSH from a
specific peer fd97:.../128, allow HTTP from one /64, etc), drop
visibility / debugging via the journal log rule and the drop
counter, coexistence with the runtime-managed `inet fips_gateway`
table, what the baseline does NOT cover (outbound, application
auth, mesh handshake ACL = PR #50 / IDEA-0047 territory), and
future cross-OS work (macOS PF baseline, OpenWrt fw4, gateway
abstraction).
Packaging:
- packaging/common/fips.nft → /etc/fips/fips.nft (conffile)
- packaging/debian/fips-firewall.service → /lib/systemd/system/
- docs/fips-security.md → /usr/share/doc/fips/
- postinst creates /etc/fips/fips.d/ (mode 0755) on configure
- prerm stops/disables fips-firewall.service on remove/purge
OpenWrt fw4 path and macOS PF baseline are deferred — separate
asymmetries, separate work.
When a UDP hole-punch succeeds in only one direction and the local
side adopts the punched socket, the remote end keeps retrying its
own punch attempt for several seconds. Those retries arrive on the
adopted socket and were forwarded to the FMP rx handler, which
parsed the first byte (0x4E from PUNCH_MAGIC's "NPTC" big-endian
encoding) as FMP protocol version 4 and emitted "Unknown FMP
version, dropping" once per probe. The probe stream contaminated
post-adoption handshake logs and added timing pressure during the
handshake window.
Add a transport-level filter in udp_receive_loop that silently
drops any datagram whose first 4 bytes match PUNCH_MAGIC or
PUNCH_ACK_MAGIC. Filter applies to all UDP transports, not just
adopted ones — the magic values cannot collide with valid FMP
frames (FMP version 4 is not assigned, and the protocol's
versioning is wire-format breaking), so universal filtering is
safe and removes any "is this an adopted socket" branching.
Move PUNCH_MAGIC / PUNCH_ACK_MAGIC and the new is_punch_packet()
helper from the `nostr-discovery`-gated submodule up to
crate::discovery (unconditionally compiled) so the UDP transport
can import them without requiring the feature. The
nostr-discovery types module re-exports the constants so the
existing traversal-side imports keep working unchanged.
Test: pushes a probe + ack + real frame through the receive loop
and asserts only the real frame is delivered to packet_tx.
Inject git date + short SHA into the Debian Version field when
Cargo.toml's crate version ends in -dev, so apt-based upgrade
detection works without operator workarounds. Form:
<base>~dev+git<YYYYMMDD>.<sha>[.dirty]-1
e.g. 0.3.0~dev+git20260429.6def31b-1.
Each commit produces a uniquely-comparable Version, so
`apt install ./*.deb` and `ansible.builtin.apt: deb:` stop
silently no-op'ing when one dev .deb is installed on top of
another. The ~dev marker sorts pre-tagged-release so 0.3.0
supersedes any prior dev .deb. Tagged builds (Cargo.toml without
-dev) keep the clean <version>-1 form. --version override still
wins.
Note: legacy 0.3.0-dev-1 dev installs sort ABOVE the new form;
hosts upgrading from a legacy install will need `dpkg -i` once
on the next dev .deb to bypass apt's downgrade refusal.
The accept_connections gate at the top of handle_msg1 was applied
unconditionally, so rekey msg1 from a peer with whom an established
link already existed was dropped on the same path as fresh handshakes
from strangers. Combined with the dual-init tie-breaker, this
deadlocked at ~25 minutes when both sides' rekey timers fired
near-simultaneously: the smaller-NodeAddr side wins as initiator and
expects the larger side to consume its rekey msg1, but if the larger
side has accept_connections=false the gate dropped it. Both sides
retried at 1 Hz indefinitely; the affected peer fell out of MMP-active
rotation.
Extract the gate decision into Node::should_admit_msg1, which admits
unconditionally when addr_to_link already has an entry for the
(transport_id, remote_addr) pair (rekey/restart on an established
session) and otherwise consults the transport's accept_connections().
Fresh msg1 from strangers is still rejected before any Noise crypto.
Three unit tests pin the truth table: no transport (admit), accept_off
no-link (reject, behavior unchanged), accept_off with-link (admit, the
carve-out).
The fix generalizes for free to BLE, which has the same Node-level gate.
TCP and Tor were never subject to this deadlock because their accept
condition is runtime state (bind_addr.is_some() / onion_address.is_some()),
not a config flag.
The previous default configured systemd-resolved with `resolvectl dns
fips0 [<fips0_addr>]:5354`, intended to bypass an Ubuntu 22 systemd 249
interface-scoping bug. That target collides with the daemon's
mesh-interface filter on Linux: when an IPv6 packet's destination
belongs to a non-loopback interface, the kernel attributes the packet
to that interface in IPV6_PKTINFO (ipi6_ifindex == fips0) even though
loopback delivery is used (tcpdump shows lo). The mesh-interface filter
sees arrival_ifindex == mesh_ifindex and silently drops every query at
trace level — invisible to operators at the default debug level.
Net effect on stock deployments: every .fips query on systemd-resolved
hosts was silently dropped.
Daemon side
-----------
- Default `dns.bind_addr` changes from "::" to "::1" (IPv6 loopback
only). The mesh-interface filter is then defanged on the default
path because loopback isn't reachable from mesh peers. The filter
remains in place defensively for operators who explicitly bind "::"
to expose a mesh-reachable responder.
fips-dns-setup backend unification
----------------------------------
- New `try_global_drop_in` backend writes
/etc/systemd/resolved.conf.d/fips.conf with DNS=[::1]:5354 and
Domains=~fips. Inserted ahead of `try_resolvectl` in the dispatch
chain. The standard loopback path has no interface scoping, so
ipi6_ifindex reports lo and the filter passes.
- All other backends now target [::1]:5354 to match the daemon's
default IPv6-loopback bind:
- try_dns_delegate writes DNS=[::1]:5354
- try_dnsmasq writes server=/fips/::1#5354
- try_nm_dnsmasq writes server=/fips/::1#5354
- Fixed dns-delegate file path: was /etc/systemd/dns-delegate/, must
be /etc/systemd/dns-delegate.d/ (with .d suffix). systemd-resolved
silently ignored the previous path.
- fips-dns-teardown handles the new global-drop-in backend in cleanup.
- The legacy resolvectl per-link backend stays as a fallback,
documented to require careful daemon bind_addr coordination.
fips-gateway upstream pairing
-----------------------------
- gateway.dns.upstream default changes from 127.0.0.1:5354 to
[::1]:5354 to match the daemon's default bind. Linux IPv6 sockets
bound to explicit ::1 do not accept v4-mapped traffic, so the old
default would have caused the gateway's startup DNS reachability
probe to time out and systemd to restart-loop the service.
- Operators who set a non-default daemon `dns.bind_addr` must also
set `gateway.dns.upstream` to match — documented inline.
Documentation
-------------
- packaging/common/fips.yaml and packaging/openwrt-ipk fips.yaml
examples updated; rationale for the bind_addr choice and the
daemon/gateway pairing recorded inline.
Test coverage
-------------
- testing/dns-resolver/test.sh: real-fipsd end-to-end scenario added.
Builds fipsd in a Debian 12 builder image (cached), runs the daemon
with a real TUN in a privileged container, configures DNS via the
setup script, and asserts `dig @127.0.0.53 AAAA <npub>.fips` returns
AAAA. Refactored as a parameterized helper running across Debian
12/13 and Ubuntu 22/24/26 (5 e2e scenarios). Backend-aware
assertions: on systemd >= 258 the expected backend is dns-delegate;
on older systemd it's global-drop-in. Strict content checks fail CI
on any [::1]:5354 drift. fips-gateway also exercised in the
debian12 scenario to lock the gateway-upstream pairing. Renamed all
"fipsd" references to "fips" (project convention).
- testing/deb-install/ (new harness): builds the actual .deb via
cargo-deb in a Debian 12 builder image (cached), installs via apt
across each target distro, verifies maintainer scripts, conffile
placement, binary placement, and end-to-end .fips resolution after
start. Also exercises fips-gateway against the installed daemon to
verify the gateway/daemon default pairing on a real .deb path.
- This is the test layer that was missing — the previous harness only
verified config files were written, never that queries reached the
daemon.
Verified: dns-resolver 78/78 assertions, deb-install 55/55 assertions
across all 5 distros (debian:12, debian:trixie, ubuntu:22.04,
ubuntu:24.04, ubuntu:26.04).
- testing/static/scripts/rekey-test.sh: bump REKEY_SETTLE from 5s to
12s so post-rekey ping_all samples are taken after the 10s old-
session drain window has closed (DRAIN_WINDOW_SECS in
handlers/rekey.rs). The 5s sample window straddled the drain,
occasionally catching old-session decrypts mid-cutover and producing
spurious ping failures.
- testing/lib/log_analysis.py: discovery-succeeded match string is
"proof verified, route cached" (the wording in handlers/discovery.rs);
the regex looked for the older "caching route" wording, so the chaos
analysis summary always reported `Succeeded: 0` regardless of how
many successes the raw node logs actually showed.
- testing/scripts/build.sh: drop the second cargo invocation that
passed `--features gateway --bin fips-gateway`. PR #79 removed the
`gateway` cargo feature in favor of platform cfg gates, so this
invocation now fails with `the package 'fips' does not contain this
feature: gateway`. The first cargo build already produces
fips-gateway as a workspace binary on every supported target.
The default 30s post-failure backoff (300s cap, doubling per
consecutive failure) was set to bound traffic from chatty apps
looking up unreachable targets, but in practice it dominates
cold-start mesh convergence: a single timed-out lookup during
initial bloom-filter propagation suppresses any retry for 30s, and
the existing reset triggers (parent change, new peer, first RTT,
reconnection) don't fire on a stable post-handshake topology. The
suppression window winds up dictating the protocol's effective
time-to-converge instead of bounding repeat traffic.
Replaces the single-lookup-with-internal-retry model
(`timeout_secs`/`retry_interval_secs`/`max_attempts`) with a
per-attempt timeout sequence in `node.discovery.attempt_timeouts_secs`,
defaulting to `[1, 2, 4, 8]`. Each attempt sends a fresh LookupRequest
with a new random request_id so successive attempts can take different
forwarding paths as the bloom and tree state evolve. The destination
is declared unreachable only after the sequence is exhausted (15s
total at the default).
Disables post-failure suppression by default (`backoff_base_secs`/
`backoff_max_secs` now `0`/`0`). The `DiscoveryBackoff` machinery
stays in tree (inert at zero base/cap); operators with chatty apps
generating repeat lookups against unreachable destinations can opt
back in.
`PendingLookup` field shape unchanged so the control-socket
`show_routing` JSON (`pending_lookups[].attempt`/`initiated_ms`/
`last_sent_ms`) keeps the same schema for fipstop and external
consumers; `last_sent_ms` now means "current-attempt start" under
the new state machine.
Drop the `tui`, `ble`, and `gateway` cargo features and replace
them with platform cfg gates. Plain `cargo build` now produces
every subsystem appropriate for the target platform with no
feature flags required.
Motivation:
- `default = ["tui", "ble"]` broke `cargo build` on macOS and
Windows because `ble` pulled in `bluer` (BlueZ, Linux-only).
Every non-Linux packager needed `--no-default-features`.
- The feature flags on `ble` and `gateway` were redundant with
their platform-gated deps (`bluer`, `rustables`). The parallel
gating was inconsistent and error-prone.
- `tui` feature protected against a ratatui binary-size concern
that no longer applies in 2026.
Cargo.toml:
- Remove `tui`, `ble`, `gateway` features; `default = []`.
- Promote `ratatui` to a non-optional top-level dependency.
- Move `rustables` from top-level optional into the Linux
target block, non-optional.
- Split `bluer` into its own target block with
`cfg(all(target_os = "linux", not(target_env = "musl")))`
— BlueZ isn't available on musl router targets and
`libdbus-sys` doesn't cross-compile to musl without pkg-config
sysroot setup.
- Drop `required-features` from the `fipstop` and `fips-gateway`
`[[bin]]` entries.
build.rs:
- Emit a `bluer_available` custom cfg when `target_os == "linux"`
and `target_env != "musl"`, for use in place of the verbose
full predicate in source cfg gates.
Source:
- Replace every `#[cfg(feature = "gateway")]` with
`#[cfg(target_os = "linux")]`. Gateway code works on both
glibc and musl Linux (rustables is fine on musl).
- Replace every `#[cfg(feature = "ble")]` with
`#[cfg(bluer_available)]`. BLE-specific code (BluerIo module,
bluer type conversions, BLE transport instance creation,
resolve_ble_addr) is excluded on musl and non-Linux. Generic
`BleAddr`, `BleIo` trait, `MockBleIo`, and `BleTransport<I>`
still compile on all targets.
- `src/bin/fips-gateway.rs`: always compiled, but `main()` is
gated to Linux. Non-Linux stub exits 1 with a diagnostic.
Existing non-Linux packaging scripts don't ship it, so the
stub binary sits unused.
Packaging and CI:
- Drop `--features` and `--no-default-features` flags from every
packaging script and workflow. Defaults now match each
platform's capabilities.
- AUR `fips-git` automatically aligns with stable `PKGBUILD`
(both build with defaults).
Verified: `cargo build --release` with no flags produces all
four binaries on glibc Linux; all unit and integration tests
pass across Linux/macOS/Windows/OpenWrt (musl) in CI.
src/node/tests/acl.rs was added by PR #50 but never declared in
src/node/tests/mod.rs, so none of its 4 unit tests ran. The tests
themselves still compile and pass against current master code —
the fix is a one-line mod declaration. Test count goes from 1031
to 1035.
No code under test changes. This only adds previously-dormant
coverage of the ACL enforcement call sites (outbound connect,
inbound msg1, outbound msg2).
On Ubuntu 22 (systemd 249), systemd-resolved applies interface-scoped
routing to per-link DNS servers. Configuring `resolvectl dns fips0
127.0.0.1:5354` caused resolved to attempt reaching 127.0.0.1 through
fips0 (a TUN with only fd00::/8 routes), silently failing. The DNS
responder never received queries. Newer systemd versions (250+) have
explicit handling for loopback servers on non-loopback interfaces.
Changes:
- DNS responder default bind_addr changed from "127.0.0.1" to "::"
so it listens on all interfaces, including fips0. Bind logic in
lifecycle.rs now parses bind_addr as IpAddr and constructs a
SocketAddr, handling IPv6 literal formatting. Factored into
Node::bind_dns_socket with explicit IPV6_V6ONLY=0 via socket2, so
IPv4 clients on 127.0.0.1:5354 still reach the responder
regardless of the kernel's net.ipv6.bindv6only sysctl.
- fips-dns-setup resolvectl backend now waits for fips0 to have a
global IPv6 address, then configures resolved with
[<fips0-addr>]:5354. That address is locally delivered by the
kernel regardless of which interface resolved tries to route
through. The dnsmasq and NetworkManager backends still use
127.0.0.1 (they don't have the interface-scoping issue).
- Dropped hardcoded `bind_addr: "127.0.0.1"` from the packaged
fips.yaml (Debian + OpenWrt). The shipped config was overriding
the new default.
- DNS queries are only accepted from the localhost.
Verified end-to-end in a privileged Ubuntu 22.04 systemd container:
dig @127.0.0.53 AAAA <npub>.fips resolves cleanly through
systemd-resolved.
The dns-delegate backend (systemd 258+) still uses 127.0.0.1; it
has not been verified whether that backend has the same routing
issue.
The test used Identity::generate() for the signing node while pinning
the fixed root to node_addr byte[0]=0x01. About 2/256 random identities
have byte[0] <= 0x01, which made the generated node_addr the path
minimum and triggered AncestryRootNotMinimum. Regenerate the identity
until its node_addr is numerically larger than the fixed parent and
root, so the test matches the preconditions it asserts.
A malformed FilterAnnounce whose fill ratio produces an implausibly
high false-positive rate is mostly useless for routing and, once
merged into our outgoing filter via bitwise OR, propagates the
saturated state to tree peers one hop per announce tick. A saturated
filter also made estimated_count() return f64::INFINITY, which
compute_mesh_size summed into its cached estimate.
handle_filter_announce now rejects inbound FilterAnnounce whose
derived FPR exceeds `node.bloom.max_inbound_fpr` (new config field,
default 0.05 ≈ fill 0.549 at k=5). Rejection is silent on the wire,
logs at WARN, and increments a new `bloom.fill_exceeded` counter. The
peer's prior stored filter and filter_sequence are left unchanged so
a single rejected announce does not wipe the peer's existing
contribution to aggregation.
After a successful outgoing FilterAnnounce send, a rate-limited WARN
fires if our own filter's FPR exceeds the same cap, surfacing
aggregation drift. Limited to once per 60 seconds via a new
Node.last_self_warn field.
BloomFilter::estimated_count() now takes max_fpr and returns
Option<f64>. Returns None for saturated filters (regardless of cap)
or when the filter's FPR exceeds max_fpr. Callers updated: debug
logs render None as "—", the Debug impl uses f64::INFINITY as "no
cap" and prints "saturated" instead of inf, control-socket JSON
emits null, and compute_mesh_size propagates None into the already-
Option<u64> estimated_mesh_size field.
Mesh peers can now reach a configured host:port on the gateway's LAN
via static port-forward rules on fips-gateway. Mirror of the outbound
LAN gateway (IDEA-0079 / TASK-2026-0056).
Config: new gateway.port_forwards list of { listen_port, proto,
target } entries. Targets are SocketAddrV6 — IPv4 is rejected at
parse time. Validation rejects zero listen ports and duplicate
(listen_port, proto) pairs.
NAT: NatManager gains a port_forwards field and set_port_forwards()
setter, rebuilt in the same atomic rustables batch as the address
mappings. Each forward emits a prerouting DNAT rule keyed on
(iifname fips0, nfproto ipv6, l4proto, tcp/udp dport) that rewrites
destination address and port via Nat::with_ip_register +
with_port_register. When any forwards are configured, a single
LAN-side masquerade is installed on (iifname fips0, oifname
lan_interface, nfproto ipv6) so the LAN host sees the gateway as
source and replies flow back through conntrack. This rule is
distinct from the existing oifname fips0 masquerade that serves the
outbound pool.
Binary: fips-gateway validates port_forwards at startup and calls
set_port_forwards after NatManager construction; startup failure
cleans up the nftables table before exiting.
Test: extend testing/static/scripts/gateway-test.sh with Phase 7
that runs a marker HTTP server on the LAN-side client (fd02::20:8080)
and, from the mesh peer, curls the gateway's fips0 address on port
18080 to exercise the full DNAT + LAN masquerade path. The
LAN-side HTTP server is started with 'docker exec -d' plus a
bind-ready poll on ss; 'docker exec bash -c "cmd &"' does not
keep the child alive past the exec session even with nohup.
Test-infra: ci-local.sh now builds/clippies/tests with
--features gateway, matching GitHub CI. Without this the release
fips-gateway binary silently stays stale across runs, since
cargo build --release alone does not compile the gateway bin.
Verified locally: cargo test --features gateway --lib (991 pass),
clippy + fmt clean, full testing/ci-local.sh green (21/21 suites
in 8m36s, including the new gateway Phase 7).
In-memory time-series history on the daemon: fast ring (1s × 3600)
plus slow ring (1m × 1440) per metric, covering node-level gauges
(mesh size, tree depth, peer count, active sessions), counters
(parent switches, aggregate bytes/packets in/out), loss rate, and
seven per-peer metrics keyed by NodeAddr (srtt_ms, loss_rate,
bytes_in/out, packets_in/out, ecn_ce). The slow ring is produced by
downsampling the fast ring on minute boundaries with Last / Sum /
Mean aggregation chosen per metric type.
Missing data is first-class. New peers back-fill NaN so every ring
shares a time axis with the node rings; peers absent from a tick
sample NaN (keeps alignment, shows as a visible gap); counter metrics
emit NaN on decrease (new link_stats baseline after reconnect) so
deltas aren't polluted. Peers are evicted 24h after last contact.
Downsampling is NaN-aware: mean skips NaN, all-NaN slow windows stay
NaN. Each history window always returns its full span at the chosen
density (1m / 10m / 1h / 24h), front-padded with NaN when the ring
hasn't yet accumulated enough samples, so switching between windows
feels like zooming in or out rather than clipping to whatever has
arrived. NaN serializes to JSON null via a custom serializer.
Control socket queries:
- show_stats_list enumerates registered metrics plus scope field and
peer_retention_seconds.
- show_stats_history returns one metric's series for a given window
and granularity; accepts optional peer (npub) for per-peer metrics.
- show_stats_all_history returns every metric in a single round trip;
accepts optional peer to fetch all seven per-peer metrics.
- show_stats_peers enumerates tracked peers with lifecycle metadata.
- show_stats_history_all_peers returns one metric across all peers
for grid rendering.
- show_status carries short sparkline windows for the dashboard so
the client can render without extra fetches.
fipsctl gains `stats list`, `stats peers`, and `stats history
<metric>` with `--peer` (hostname or npub) and `--plot` for a Unicode
block sparkline. Plot header reports sample count, granularity,
window, and gap count; NaN renders as a blank cell.
fipstop dashboard grows inline sparklines (peer count, mesh size,
aggregate bytes in/out). A new Graphs tab stacks every metric as an
independent mini plot with its own autoscaled range; each plot uses
btop's braille 2×4 filled-area algorithm (25-entry lookup table
packing two samples per character, per-row gradient coloring for the
characteristic btop vertical-band look, rounded borders with embedded
titles). Three modes are cycled with `m`: Node (node-level stack),
MetricByPeer (small-multiples grid, 1 / 2 / 3 columns by terminal
width), PeerByMetric (existing stack scoped to one peer). `n` / `N`
cycles the mode-specific selector (metric or peer), a selector row
shows the current choice, and Graphs-tab refreshes re-fetch
show_stats_peers so selectors track peer churn. Up / Down scrolls
the stack, Left / Right cycles the window, `g` jumps to the tab.
Implements IDEA-0084 (TASK-2026-0062).
- Correct the transport matrix: UDP, TCP, and Tor work on Windows
(previously shown as unsupported). Add an OpenWrt column with BLE
disabled due to missing libdbus on the target.
- Mention the `.fips` DNS resolver and outbound LAN gateway in the
Features list, and add the gateway bullet under What works today.
- Reframe the Linux DNS resolver setup: the `.deb` package now
auto-configures the available backend; the manual resolvectl
snippet is shown for tarball and manual installs.
- Expand the Examples section to list all three example deployments
(Nostr relay sidecar, K8s sidecar, macOS WireGuard sidecar) rather
than only the macOS one.
- Refresh Project Structure to include the `fips-gateway` binary,
the full packaging list (macOS .pkg, Windows ZIP, OpenWrt ipk, AUR
in addition to Debian and systemd tarball), and the examples
directory.
- Mention macOS `.pkg` and Windows ZIP/service packaging in the
packaging line of What works today.
Adds entries for master-only work since 0.2.0 that wasn't captured
yet: Windows and macOS platform support, the outbound LAN gateway
and its packaging, the macOS WireGuard sidecar example, multi-backend
.fips DNS configuration, the node.log_level config, the Nostr UDP
hole punch protocol proposal doc, the MMP report interval retune,
the info-to-debug log demotion, the rekey msg1 gate fix, and the
fipstop ratatui try_init change.
Fixed entries only cover bugs present in 0.2.0. Fixes against
master-only code (BLE reliability work, new sidecar port mapping)
are rolled into their respective Added entries rather than listed
as Fixed, per Keep a Changelog conventions.
Captures three fixes landed on maint since 0.2.0 that were not yet
recorded: the bloom-filter greedy-tree fallback fix, auto-connect
reconnect on graceful disconnect (#60), and fipsctl mesh-address
rejection (#61).
When a user passes an fd00::/8 address as the endpoint for a udp,
tcp, or ethernet transport, the CLI previously echoed success while
the daemon silently failed the bind with EAFNOSUPPORT. Mesh ULAs are
destinations inside the mesh, not reachable transport endpoints.
fipsctl now validates the address up front and prints a clear error
with examples, exiting 1 before the control socket call. Other
transports (tor) are not inspected since they legitimately accept
non-IP endpoints.
Covered by inline tests for bare/bracketed/with-port ULA syntaxes,
non-ULA IPv6, IPv4, hostnames, and the transport filter.
Fixes#61.
handle_disconnect() called remove_active_peer without scheduling a
reconnect, orphaning auto-connect peers on a clean upstream shutdown.
Mirror the pattern from the other three peer-removal paths (link-dead,
decrypt failure, peer restart) which all schedule reconnect after
removal.
Adds test_disconnect_schedules_reconnect regression test that verifies
handle_disconnect populates retry_pending for an auto-connect peer.
Visibility of handle_disconnect bumped to pub(in crate::node) for
direct unit-test access.
Fixes#60.
CI format check was failing because the pinned toolchain
rust-toolchain.toml overrides the `dtolnay/rust-toolchain@stable +
rustfmt` action installation — rustup installs rustfmt onto the
stable channel, but when cargo fmt runs inside the repo, rustup
honors the 1.94.1 pin and does an on-demand install that pulls only
rustc/cargo/rust-std.
Declaring components in rust-toolchain.toml ensures the on-demand
install of the pinned toolchain includes rustfmt.
Replace the resolvectl-only fips-dns.service with a detection script
that configures whichever DNS resolver is available:
1. systemd dns-delegate (systemd >= 258, declarative drop-in)
2. systemd-resolved via resolvectl (most systemd distros)
3. dnsmasq (standalone)
4. NetworkManager with dnsmasq plugin
5. Warning with manual instructions if none found
Service reloads are non-fatal — config is written and the backend
is recorded even if the reload fails, preventing state file cleanup
issues under set -e.
Teardown reads the recorded backend from /run/fips/dns-backend and
reverses the configuration, or cleans up all possible backends if
the state file is missing.
Includes a Docker-based test harness (testing/dns-resolver/test.sh)
covering all five backends across Debian 12, Debian 13, Fedora,
and bare systems.
Fixes#52.
The entrypoint HTTP server was binding port 80 but gateway-test.sh
curls port 8000. Maint already had 8000; the wrong port was introduced
during a merge to master.
Add fips-gateway binary: a separate daemon that allows unmodified LAN
hosts to reach FIPS mesh destinations via DNS-allocated virtual IPs
and kernel nftables NAT.
Gateway DNS resolver: forwarding proxy on [::]:53 that intercepts
.fips queries, forwards to daemon resolver (localhost:5354), allocates
virtual IPs from pool, returns AAAA records. Always sends AAAA upstream
regardless of client query type, returns proper NODATA for non-AAAA.
Virtual IP pool: fd01::/112 pool with state machine lifecycle
(Allocated → Active → Draining → Free), TTL-based reclamation,
conntrack integration for session tracking.
NAT manager: nftables DNAT/SNAT rules via rustables netlink API,
per-mapping rule lifecycle, fips0 masquerade for LAN client source
address rewriting.
Network setup: local pool route, proxy NDP for virtual IPs on LAN
interface, IPv6 forwarding validation.
Control socket at /run/fips/gateway.sock with show_gateway and
show_mappings queries. fipstop Gateway tab with pool summary gauge
and mappings table.
Gateway config section in fips.yaml with pool CIDR, LAN interface,
DNS upstream, TTL, and grace period settings.
Design doc at docs/design/fips-gateway.md.
Integration test (testing/static/scripts/gateway-test.sh): three
containers verifying DNS resolution, end-to-end HTTP, NAT state,
TTL expiration, SERVFAIL fallback, and clean shutdown.
When bloom filter candidates existed but none were strictly closer to
the destination in tree-coordinate distance, find_next_hop returned
None without trying greedy tree routing. This caused NoRoute failures
in topologies where the tree parent was closer but not a bloom
candidate. Fall through to tree routing when no bloom candidate makes
progress.
Raise the report interval floor from 100ms to 1000ms and ceiling from
2000ms to 5000ms. The old 100ms floor produced ~600 reports per 60s
parent evaluation cycle — far more than the ~10 needed for EWMA
convergence. The new floor yields ~60 reports/cycle, still well above
the convergence threshold, while reducing BLE overhead by 10×.
Add cold-start transition: first 5 SRTT samples use the 200ms floor
for fast initial convergence, then switch to the 1000ms steady-state
floor. Session-layer intervals unchanged (500ms–10000ms).
Wait for all nodes to reach their expected peer counts instead of
only checking a single node. This prevents false failures on slower
CI runners where remote nodes (especially node E in mesh/chain
topologies) take longer to establish all links.
Wait for all nodes to reach their expected peer counts instead of
only checking a single node. This prevents false failures on slower
CI runners where remote nodes (especially node E in mesh/chain
topologies) take longer to establish all links.
Tailscale (and potentially other routing software) installs a default
IPv6 route in an auxiliary routing table with a policy rule that runs
before the main table. This silently diverts fd00::/8 FIPS traffic
away from the fips0 TUN device.
Add an ip6 rule (priority 5265) during TUN setup that directs fd00::/8
to the main routing table, ensuring the fips0 route is always used.
Tailscale (and potentially other routing software) installs a default
IPv6 route in an auxiliary routing table with a policy rule that runs
before the main table. This silently diverts fd00::/8 FIPS traffic
away from the fips0 TUN device.
Add an ip6 rule (priority 5265) during TUN setup that directs fd00::/8
to the main routing table, ensuring the fips0 route is always used.
- Promote probe connections directly into pool instead of dropping and
reconnecting. Eliminates fragile two-phase connect pattern (probe →
disconnect → reconnect) that caused race conditions on restart.
- send_async fails fast when no connection exists, triggering a
background connect_async instead of blocking the event loop for up
to 10s on inline L2CAP connect. Prevents control socket query
timeouts and MMP processing stalls.
- Add 5-second timeout to pubkey_exchange recv. Without this, a peer
that connects but never sends its pubkey blocks the calling task
forever, killing the scan_probe_loop or accept_loop entirely.
- Add retry timer in scan_probe_loop for addresses that failed probe
but won't get another DeviceAdded from BlueZ (deduplication).
- Clear BlueZ cached devices before starting scan so fresh
advertisements trigger DeviceAdded after daemon restart.
- connect_async now performs pubkey exchange before promoting to pool,
matching the accept_loop's expectation on inbound connections.
- BLE tests updated to pre-establish connections via connect_async
since send_async no longer does inline connect.
- Size receive buffer from negotiated recv_mtu instead of hardcoded 4096
(prevents silent truncation if MTU exceeds buffer size)
- Set advertising interval to 400-600ms for deterministic behavior
instead of depending on BlueZ driver defaults
- Enable power_forced_active on L2CAP sockets to prevent sniff-mode
latency spikes during data transfer (best-effort, logged on failure)
- Log negotiated PHY and MTU at connection establishment for diagnostics
Add log_level field to NodeConfig (case-insensitive, default: info).
The daemon now loads config before initializing tracing so the
configured level takes effect. RUST_LOG env var still overrides if
set.
Remove hardcoded Environment=RUST_LOG=info from systemd units and
OpenWrt procd init script — these prevented config-driven log levels
from working.
Replace burst beacon pattern (1s on / 30s off) with continuous
advertising. The burst pattern caused L2CAP connect timeouts because
the remote side was no longer connectable when the probe fired after
jitter delay. BLE advertising overhead is negligible (~0.15% duty
cycle on advertising channels).
Replace the seen HashSet + jitter delay queue with a simple cooldown
map. After probing an address (success or failure), suppress re-probe
for 30s (configurable via probe_cooldown_secs). Connected peers are
filtered by pool membership check. This eliminates the bug where
failed probes permanently blacklisted addresses for the session
lifetime.
Remove config fields: scan_interval_secs, beacon_interval_secs,
beacon_duration_secs. Add: probe_cooldown_secs.
BLE transport implementation using L2CAP Connection-Oriented Channels
(SeqPacket mode) via the bluer crate, behind cfg(feature = "ble").
Core transport:
- BleTransport<I> generic over BleIo trait (BluerIo prod, MockBleIo test)
- Connection pool with priority eviction (static > discovered, max 7)
- Connect-on-send via connect_inline() matching TCP behavior
- Per-connection receive loops with pool cleanup on disconnect
Discovery and probing:
- Combined scan_probe_loop using select! over scanner events and a
BinaryHeap delay queue with per-entry random jitter (0-5s) to prevent
herd effects when multiple nodes see the same beacon simultaneously
- Pre-handshake pubkey exchange ([0x00][pubkey:32]) for IK identity
- Cross-probe tie-breaker: smaller NodeAddr's outbound wins (same
convention as FMP/FSP rekey dual-initiation)
- Probed peers reported to DiscoveryBuffer; pool fills through normal
node-layer auto-connect -> send_async -> connect_inline path
Beacon management:
- Periodic advertising: 1s burst every 30s (configurable via
beacon_interval_secs / beacon_duration_secs)
- FIPS service UUID for scan filtering
Configuration (all fields optional with defaults):
- adapter, psm, mtu, max_connections, connect_timeout_ms
- advertise, scan, auto_connect, accept_connections
- beacon_interval_secs (30), beacon_duration_secs (1)
Hardware validated with two BLE nodes:
- 2048-byte MTU, ~60-160ms RTT, zero-config auto-connect
- BLE spike tool at testing/ble/ for standalone adapter validation
42 unit tests + 4 node-level integration tests, all CI-compatible
via MockBleIo (no hardware required). tokio test-util added for
time-dependent scan/probe tests.
The visited_bits field (hash_cnt + 256 bytes) was removed from the
LookupRequest wire format in the discovery-rework (bloom-guided tree
routing replaced the visited filter). Update the diagram to match the
current 46 + 16n byte format.
Phase 1 pre-rekey baseline was failing intermittently on CI runners
because 5s wasn't enough for multi-hop discovery (B→D requires
B→C→D). The mesh always converged by Phase 3, confirming this was
purely a timing issue.
The rekey test greps container logs for initiator cutover messages that
were demoted to debug level. Override RUST_LOG in the rekey profile to
enable debug for fips::node::handlers::rekey so the test assertions
can find them.
Check /run/fips/ directory existence instead of the socket file inside
it. Users not in the fips group can stat the directory but not traverse
it, so the socket file check silently returned false and fell back to
$XDG_RUNTIME_DIR with a misleading "No such file" error.
Bump version to 0.2.0 and finalize changelog with discovery rework,
Tor transport, connect/disconnect commands, reproducible builds, and
12 bug fixes.
Update design documentation for discovery protocol rework:
- fips-wire-formats.md: remove visited bloom filter from LookupRequest,
update size calculations
- fips-mesh-operation.md: replace flooding description with bloom-guided
tree routing, add retry/backoff/rate-limiting subsections
- fips-configuration.md: add 5 new discovery config parameters, update
control socket description for connect/disconnect commands
build-tarball.sh only applied --mtime when SOURCE_DATE_EPOCH was
explicitly set, and did not sort tar entries. This produced different
tarballs across builds due to varying timestamps and filesystem
ordering. Default SOURCE_DATE_EPOCH to the git commit timestamp and
add --sort=name for deterministic output.
Replace discovery flooding with bloom-filter-guided tree routing:
lookups sent only to tree peers (parent + children) whose bloom
filter contains the target. If no tree peer matches, fall back to
non-tree peers with bloom matches before dropping the request. This
produces single-path forwarding through the spanning tree (90%
traffic reduction) while recovering from dead ends caused by stale
bloom filters, tree restructuring, or transit node failures.
Remove visited bloom filter from LookupRequest wire format (-257
bytes per request). Tree routing is inherently loop-free; request_id
dedup handles edge cases during tree restructuring. Add response-
forwarded flag to prevent response routing loops from convergent
request paths.
Add originator-side exponential backoff (30s base, 300s cap) after
lookup timeouts and bloom misses. Backoff resets on topology changes
(parent switch, new peer, first RTT, reconnection).
Add transit-side per-target rate limiting (2s minimum interval) for
forwarded lookups as defense-in-depth.
Add discovery retry within the timeout window (default: send at T=0,
retry at T=5s, fail at T=10s) to compensate for single-path fragility.
Lookups with zero eligible tree peers fail immediately.
Improve discovery logging: promote key events to info (initiation,
success, timeout with failure count). Add debug logging for dedup,
pending packet retry, backoff suppression, forward rate limiting,
and backoff reset.
New config: discovery.backoff_base_secs, backoff_max_secs,
forward_min_interval_secs, retry_interval_secs, max_attempts.
New stats: req_backoff_suppressed, req_forward_rate_limited,
req_bloom_miss, req_no_tree_peer, req_fallback_forwarded.
Removed: req_already_visited, visited bloom filter.
Three infrastructure bugs fixed:
- DNS resolution never populated the identity cache because Docker
overwrites /etc/resolv.conf at container start. Fix: bind-mount
resolv.conf in the chaos compose template, matching the static and
sidecar test configs.
- Traffic generator used config-time npubs for iperf3 targets, but
ephemeral nodes generate fresh keypairs at startup. Fix: share the
peer churn manager's runtime npub cache with the traffic manager.
- Log analysis had no discovery counters and used wrong patterns.
Fix: add discovery section with correct log message matching.
Also adds timestamped output directories (YYYYMMDD-HHMMSS-scenario),
coord_ttl_secs override in maelstrom.yaml, FIPS_SIM_OUTPUT env var
support, and maelstrom-sparse scenario for sparse topology testing.
Add runtime peer management to the FIPS daemon via control socket
commands, and a new chaos simulation scenario that exercises dynamic
topology mutation with ephemeral node identities.
Daemon (connect/disconnect commands):
- Extend control socket Request with optional params field
- Add commands.rs module for mutating command dispatch, separate from
read-only queries
- Add api_connect() on Node: builds ephemeral PeerConfig (no auto-
reconnect), pre-seeds identity cache, reuses initiate_peer_connection
- Add api_disconnect() on Node: calls remove_active_peer(), clears
retry_pending to suppress reconnection
- Route non-show_* commands to async command dispatch in rx_loop
fipsctl CLI:
- Add Connect and Disconnect subcommands accepting npub or hostname
- Resolve hostnames from /etc/fips/hosts before sending to daemon
- Refactor socket I/O into reusable send_request helper
Chaos simulator (maelstrom scenario):
- Add PeerChurnManager: periodically disconnects a random active link
and connects a random unconnected node pair via control socket
- Add send_command() to control.py using base64-encoded JSON payloads
to avoid shell quoting issues in docker exec
- Add PeerChurnConfig to scenario with interval and ephemeral_fraction
- Ephemeral identity support: nodes configured without nsec generate
fresh keypairs on restart; simulator queries show_status for new
npub and updates its cache via on_node_restart callback
- Add maelstrom.yaml: all chaos dimensions (netem, link flaps, node
churn, peer topology churn, traffic) with 50% ephemeral identity
The K8s sidecar is a standalone example, not a test harness — it has
no CI integration unlike testing/sidecar/. Move it to examples/
alongside sidecar-nostr-relay/ where it belongs. Update internal path
references in Dockerfile, build.sh, and README.
The loss and goodput delta guards used prev_rr_highest_counter > 0 and
prev_rr_cum_bytes > 0, which conflates "never received a report" with
"received a report where the value was 0." Since Noise transport
counters start at 0, this is semantically wrong. Use a dedicated
has_prev_rr boolean flag instead. Fixes GitHub issue #14 (Bug B).
Johnathan Corgan
