A CI worker may preempt an in-flight ci-local.sh run (SIGTERM, then SIGKILL
after a grace period) to restart on a newer commit. For that kill to be safe,
the script must clean up after itself and never let a dying run collide with
its restart. It previously had no signal handling, shared the default compose
project name across runs, and tore down each suite only at the suite end.
- Derive a per-run id (honoring FIPS_CI_RUN_ID, else short-sha+random) and
namespace every docker resource to it: a fipsci_<run>_<suite> compose project
per suite and per parallel chaos child, and per-run image tags
(fips-test:<run>, fips-test-app:<run>) retagged to :latest only after both
builds succeed so :latest never points at a half-built image.
- Install a bounded, idempotent teardown trap on SIGTERM/SIGINT (+ EXIT): reap
parallel chaos children, then force-remove this run's docker resources via
the new ci-cleanup.sh, wrapped in timeout so a stuck down cannot wedge it.
- Exit 143 (SIGTERM) / 130 (SIGINT), distinct from 0 (pass) / 1 (failed), so a
preempting worker tells a cancelled run from a real failure.
- Add ci-cleanup.sh (also ci-local.sh --reap): force-removes leftover CI
resources by the com.corganlabs.fips-ci=1 label and the fipsci_ project
prefix, robust to however a prior run died.
- Label every per-suite docker resource so the label sweep reaps it after a
SIGKILL regardless of network name: direct docker run/network resources, the
sidecar compose services, and every per-suite compose network (acl-allowlist,
boringtun, firewall, nat, static, both tor suites, and the chaos generator
template). Parametrize the static/sidecar compose image refs so the per-run
tags are honored.
- Give each parallel chaos child a unique /24 from 10.30.x (a new --subnet
override on the sim CLI, assigned per-child in ci-local.sh) so parallel
children never collide on a shared docker subnet, and a chaos net can never
span a fixed-subnet suite (sidecar/static in 172.20.x). 10.30.x sits outside
docker's default-address-pool range, so an auto-assigned net cannot land on
it either; node IPs derive from the subnet, so no scenario config changes.
Forward-merge the v0.4.0 pre-release content (dependency refresh, CI
deb-install + AUR-build legs, docs refresh, doc-comment fixes, and the
Phase 4 source-content squash: CHANGELOG, release notes, README,
fips.yaml, Cargo.toml metadata, reference docs) up the one-way flow.
Conflict fixups (keep next's identity, fold in master's improvements):
- Version: keep next's 0.5.0-dev (Cargo.toml/lock).
- README status: keep next's v0.5.0-dev / wire-format-breaking framing
and the Breaking-section pointer; fold in the Nym transport and the
"global, public test mesh of thousands of nodes" description.
- docs/reference/cli-fips.md: keep next's 0.5.0-dev version example.
- CHANGELOG: keep next's XX-handshake admission entry (no early cap gate
on XX) and drop master's IK early-cap-at-handle_msg1 entry, which
describes IK-only behavior that does not apply on next; take master's
OR-union mesh-size rewrite (next carries the OR-union code); restore
the Tor connect_refused and MMP receiver-report entries that the
Fixed-section consolidation would otherwise have dropped.
- lifecycle.rs: keep the mDNS/LAN handshake doc-comment as Noise XX
(next unifies on XX), not master's XX-to-IK correction.
Quartet green on the merged tree: fmt, build, clippy -D warnings, and
cargo test --lib (1434 passed).
The GitHub deb-install matrix ran debian12/ubuntu24/ubuntu26, but the
local harness (testing/deb-install/test.sh) runs five distros. Add the
missing debian13 (trixie) and ubuntu22 legs so the cloud gate covers the
same distro set as local CI. Each new leg invokes the existing test.sh
scenario, so per-distro behavior is identical to the local run.
Update the granularity-only parity notes in ci.yml, ci-local.sh, and
check-ci-parity.sh to list the full distro set.
Bring the local runner (testing/ci-local.sh) and GitHub CI into agreement
on two axes.
Suite coverage: the admission-cap integration suite ran only locally, so a
regression in it could never turn the GitHub gate red. Add an admission-cap
leg to the integration matrix, add testing/check-ci-parity.sh (wired as
'ci-local.sh --check-parity') to diff the two suite sets and fail on
unexpected drift, and document the deliberate local-only (live-Tor) and
granularity-only differences in a comment block atop both runners.
Toolchain selection: every CI and packaging job installed its toolchain
with dtolnay/rust-toolchain@stable, but the rust-toolchain.toml channel pin
overrode it for all compilation, wasting an install and printing a
misleading rustc version. Switch the stable call sites to
actions-rust-lang/setup-rust-toolchain, which reads rust-toolchain.toml as
the single source of truth. Keep the explicit cache steps (cache: false on
the new action), set rustflags empty so the action does not impose a global
-D warnings the previous setup never applied, fold the macOS cross-compile
target into the action input, and leave the OpenWrt nightly Tier-3 leg on
dtolnay/rust-toolchain@nightly.
Bring the immutable-state single-store change onto the Noise XX line.
The shared NodeContext is now the sole store; this merge applies the
next-only adaptations the master-side change couldn't carry:
- Remove node_profile from the Node struct (next-only field) so it lives
solely in NodeContext; migrate its readers (tree/bloom/discovery/
handshake negotiation) onto the node_profile() accessor. The two FMP
negotiation sites hoist node_profile() into a local to avoid borrowing
&self while a connection is mutably borrowed.
- leaf_only sets both is_leaf_only and node_profile via the context swap.
- Preserve the XX handshake/rekey structure (no identity-in-msg1; XX/XK
initiator/responder constructors) while applying the startup_epoch()
accessor migration.
- Tests: route profile selection through a make_test_node_with_profile
helper (profile is immutable, set via Config flags) instead of poking
the removed field.
cargo test --lib 1369/0; clippy -D warnings and release build clean.
Remove the duplicated immutable fields (config, identity, startup_epoch,
started_at, is_leaf_only, max_connections/peers/links) from the Node
struct so the Arc<NodeContext> bundle is the single source of truth.
Previously Node owned these fields and a parallel context copy, kept in
lockstep by rebuild_context() at every mutation site — pure overhead that
existed only because of the duplication.
- Replace rebuild_context() with replace_context(): a clone-edit-swap of
the whole Arc. The per-instance context stays immutable; mutation swaps
the Arc. This is the sole runtime mutation path (constructors, leaf_only,
update_peers).
- Add Copy-returning accessors startup_epoch() and max_connections()/
max_peers()/max_links(); migrate the remaining direct field readers onto
the accessors. node_addr()/npub()/Debug now read identity/is_leaf_only
from the context.
- update_peers reads the pre-update peer set from the live context Arc
before building a fresh Config + context and swapping — preserving the
read-before-write ordering its mutation-window test depends on.
- Remove the test-only set_max_* setters; tests set the limits on Config at
construction instead (new make_node_with_max_peers/links helpers).
- Add a ci-local guard that fails if the Node struct re-declares a bundled
field, so the single-store invariant can't silently regress.
cargo test --lib 1291/0; clippy -D warnings and release build clean.
Carries the inbound max_peers admission-cap silent-drop work from
master across the IK→XX structural boundary.
On master (Noise IK), the gate fires at handle_msg1 after the peer's
identity is extracted from the IK msg1 payload, before Msg2 is built
and sent. Wire savings of Msg2 (~104 B) plus the responder crypto
per cap-denied attempt motivate the gate placement.
On next (Noise XX), peer identity is not learned until msg3, by
which point Msg1, Msg2, and Msg3 have all crossed the wire. The IK
gate's wire-savings rationale does not apply on XX. The equivalent
gate is placed in handle_msg3 after the peer's static-key + signature
verification (identity now known) and before promote_connection
(ActivePeer construction, peers_by_index insert, link transition).
The value on XX is local CPU / allocation savings and cleaner peer-
side semantics: no fake-promotion whose subsequent data frames fail
decryption on this side, replaced by a silent drop the peer's
existing auto-reconnect backoff handles cleanly.
Bypass logic is the same as the IK version: known-active peers
(reconnect / cross-connection / restart) and pending-outbound peers
skip the cap so admitting them doesn't grow peers.len(). The late
cap check inside promote_connection is intentionally retained as
defense-in-depth.
Cleanup ordering for the gate's silent-drop path corrects a latent
issue in the adjacent msg3-processing-failure path: our_index is
captured before the connection is removed so the allocator slot is
actually freed (the pre-existing path's get-after-remove returns
None, leaking the index).
The two IK-shaped unit tests at handle_msg1 are marked #[ignore] on
this branch with reason. Their wire-bytes discriminator ("Msg2 must
NOT be sent at cap") is structurally false on XX, and the XX-shaped
equivalents need full three-message-handshake test scaffolding that
isn't authored here. XX-adapted unit + integration test coverage for
the handle_msg3 gate is a follow-up.
CHANGELOG [Unreleased] entry adjusted to describe the handle_msg3
placement on this branch (preserving the master-side framing on the
mainline-merge half of the entry would have been misleading on this
branch).
Three conflicts auto-resolved: the structural handshake.rs span
(~253 lines, IK identity-tied logic discarded on this branch's XX
msg1 path), and three textual ci-local.sh conflicts (admission-cap
suite list / dispatch added alongside the existing mixed-profile
suite list / dispatch).
New integration scenario verifying the early-gate silent-drop behavior
of the inbound max_peers admission check at sustained scale, using the
existing 5-node mesh topology with one node's node.limits.max_peers
lowered to 1. This forces 2 of the cap'd node's 3 configured peers
into a sustained denied state, and asserts via tcpdump that no Msg2
responses go back to those denied peers across a 60s capture window.
A background load-driver restarts the denied peer containers every 15s
to reset their auto-reconnect exponential backoff (5s base / 300s cap),
producing fresh Msg1 bursts each cycle. Without this loop the gate
fires ~3-4 times per denied peer in a 60s window; with restarts the
observed rate is 15 per denied peer (~30 total firings), high enough
that any Msg2 leakage would be caught with strong statistical
confidence.
Local run on this branch: cap'd node-c converged to peer_count=1 with
node-b admitted; nodes d and e sustained-retried as denied; tcpdump
captured 30 inbound Msg1 (len 84) packets from the denied pair and 0
outbound Msg2 (len 104) packets, with final peer_count unchanged.
Files:
testing/static/scripts/admission-cap-test.sh — new test script with
inject-config subcommand (sets node.limits.max_peers) and a
3-phase test driver (converge, capture-with-load, per-peer assert)
testing/ci-local.sh — register admission-cap as a new suite category
(ADMISSION_SUITES), wire run_admission_cap function, add to
run_suite dispatch, list_suites, and the default integration sweep
Together with the existing unit-level coverage in src/node/tests/unit.rs
(handle_msg1_silent_drops_at_cap_for_new_peer with mock-transport Msg2
discriminator, and handle_msg1_admits_existing_peer_at_cap as the
bypass regression guard), the gate's silent-drop behavior is now
verified both at single-firing wire-observable resolution and at
sustained multi-firing cross-process scale.
Forward-merge of 12 master commits past the previous merge
(823b830, master @ 18019bb): dep-audit bumps (rand,
clap, tun, rtnetlink, windows-service, plus the bump-safe
lockfile batch), bloom-storm chaos scenario, control-socket
resolver consolidation, gateway dns.listen default change,
OpenWrt ipk README refresh, gateway tutorial review, Ethernet
MTU rustdoc fix, dead session-variant drop, CHANGELOG prep.
Conflict resolution:
- CHANGELOG.md: both bullets kept under [Unreleased] / Fixed.
Master's spanning-tree internal-path-propagation fix precedes
next's tree-ancestry-test determinism entry and the
responder-Disconnect XX-handshake entry.
- src/protocol/session.rs: kept next's SessionSetup/SessionAck
variants and rustdoc. Master's drop of those variants suits
v0.3.0's FSP phase-byte dispatch but is undone by next's
v0.4.0 wire format, which retains the variants and uses the
inner msg_type byte for handshake identification.
- src/transport/ethernet/mod.rs: kept next's "interface MTU - 4"
comment. Master corrected the v0.3.0 3-byte rustdoc; next
redesigned the framing to a 4-byte header (type/flags/length)
for shared-media beacons, so master's correction does not
apply to next's format.
Auto-merged cleanly: Cargo.toml (next's 0.4.0-dev + the new dep
pins from master), Cargo.lock, all gateway docs,
docs/reference/configuration.md, packaging files,
.github/workflows/ci.yml, testing/chaos/sim/* and
testing/ci-local.sh (bloom-storm additions), src/config/*.
Local verification: cargo build --release, cargo test (1252
passed, 4 ignored), cargo clippy -D warnings, cargo fmt --check
all green.
Six-node depth-4 mesh with an induced upstream parent flap. Asserts a
trailing-window ceiling on per-node `stats.bloom.sent` and a sanity
floor on parent-switch count over a ~3-4 min observation window.
Guards against the regression class where a spanning-tree update that
changes only an internal path edge (no root or depth delta) fails to
be properly contained and instead propagates to leaves as a sustained
bloom-traffic oscillation, visible only at fleet scale and only after
several minutes of uptime.
Adds a new chaos primitive (`link_swap`) for deterministic asymmetric
link-cost flapping and a post-run assertion framework with two
checks:
- `bloom_send_rate.max_per_node`: trailing-window ceiling on the
`show_bloom` stats counter delta. Calibrated against the
post-mortem reproduction harness data (per-variant counter table
against pre-fix vs post-fix binaries).
- `min_parent_switches.min_total`: sanity guard against a
misconfigured harness where the flap inducer fires but the
topology never produces a real parent-switch event (e.g., wrong
root election from a different seed). Without this, the
bloom-rate assertion would trivially pass on any binary
including a regressed one.
The runner exits 3 on assertion failure (alongside 0 success and 2
panic-detected). Threshold derivation is documented in the scenario
README; the seed pin is also documented there since smallest-NodeAddr
root election is sensitive to the pubkey hash ordering.
Wired into ci-local.sh's chaos pool and the GitHub CI chaos matrix.
Cover the previously untested STUN client behavior under server
unreachable, response timeout, and packet loss. The 3 NAT scenarios
test happy paths only; if the STUN client mishandled a fault (panic,
hang, missing log signal), it would silently degrade NAT traversal
without surfacing in CI.
testing/nat/scripts/stun-faults-test.sh (new, 244 lines):
Phase 1 (drop, ~12s): tc prio + netem loss 100% band + u32 filter on
dst 172.31.10.40 udp 3478. Falls back to iptables -j DROP if netem
isn't available. Asserts daemon process alive, no panic, log line
matching stun.*(timed?out|fail|fallback|unreachable|no address)
within the phase window.
Phase 2 (delay then clear, ~17s): tc qdisc add dev eth0 root netem
delay 5000ms for 7s, then deleted. 10s settle. Asserts process alive,
no panic, AND "STUN observation succeeded" log line after clear
(recovery proof).
Phase 3 (kill, ~12s): docker stop fips-nat-stun. Asserts process
alive, no panic, fault evidence in logs.
testing/nat/docker-compose.yml: stun-faults profile adds two
services. stun-fault-node is fips-test:latest on shared-lan at
172.31.10.50. stun-fault-shim is fips-test:latest sharing the
daemon's network namespace via network_mode: service:stun-fault-
node, with cap_add NET_ADMIN, NET_RAW; entrypoint sleep infinity so
the script can docker exec into it. Reuses existing stun
(172.31.10.40:3478) and relay (172.31.10.30:7777) services.
testing/nat/scripts/generate-configs.sh: 3-hunk update so the
generator accepts the new scenario and points its peer config at the
existing relay/STUN. The peer is configured for connect_peer() so
the daemon retries traversal on a loop, repeatedly invoking
observe_traversal_addresses() — which is the fault-injection target.
testing/ci-local.sh: STUN_FAULTS_SUITES=(stun-faults) array,
run_stun_faults runner, list/integration-loop/--only-dispatch hooks.
.github/workflows/ci.yml: matrix row {suite: stun-faults, type:
stun-faults} + 3 steps gated on matrix.type == 'stun-faults' between
nostr-publish-consume and any chaos suite. Reuses fips-linux
artifact + fips-test:latest image.
Approach: script-driven via docker exec stun-fault-shim. Sharing
network namespace means tc rules on the shim's eth0 affect daemon
egress. No timing logic in the shim itself.
End-to-end exercise the v0.3.0 nftables firewall baseline so the
security claim — "services on fips0 are not exposed by default" — is
validated in CI rather than by operator opt-in. The packaging postinst
state is pinned by the deb-install matrix; this suite pins the actual
ruleset behavior.
testing/firewall/ — new directory mirroring the acl-allowlist
precedent (kept in its own directory, not extending testing/static):
docker-compose.yml (52 lines): two FIPS containers on bridge
172.32.0.0/24, peered over UDP/2121. node-b mounts
packaging/common/fips.nft RO at /etc/fips/fips.nft and a generated
drop-in services.nft at /etc/fips/fips.d/.
test.sh (247 lines): four-case asserter
(a) curl from node-a to node-b:8000 → DROP (port not allowlisted;
terminal counter drop fires)
(b) curl from node-b to node-a:8000 → 200 OK (reply traverses
node-b's ct state established,related accept)
(c) ping6 a→b → success (icmpv6 echo-request accept)
(d) nc -z a→b:22 → success (drop-in tcp dport 22 accept honored
via include "/etc/fips/fips.d/*.nft")
Plus drop-counter check after case (a) confirms the dropped
connection actually hit the chain's terminal counter drop.
generate-configs.sh (111 lines): mirrors acl-allowlist generator,
produces the drop-in services.nft + fips configs.
README.md (111 lines): how to run, expected output, design notes.
.gitignore: ignores generated-configs/.
testing/ci-local.sh: FIREWALL_SUITES=(firewall) array + run_firewall
runner; dispatch in run_integration and run_suite mirroring
run_acl_allowlist.
.github/workflows/ci.yml: matrix row {suite: firewall, type:
firewall} + 3 steps gated on matrix.type == 'firewall' between
acl-allowlist and gateway. Reuses fips-linux artifact + fips-test:
latest image.
Activation note: the unified test image does not run systemd, so
test.sh invokes the fips-firewall.service ExecStart
(/usr/sbin/nft -f /etc/fips/fips.nft) directly. The systemd-unit
enablement path is covered by the deb-install matrix; this suite
exercises what the unit configures, not how it gets started.
Cover the previously untested overlay advert publish/relay/consume
round-trip. The bilateral publish/subscribe path was a v0.3.0 release
gap: malformed adverts could panic consumers, broken signatures could
go undetected, and reverse-direction subscription was unverified.
Adds testing/nat/scripts/nostr-relay-test.sh (290 lines):
Phase 1+2 (combined): wait_for_peers on both nodes; pass on
bidirectional advert publish/subscribe round-trip + dial completed;
ping6 both directions confirms TUN-level reachability.
Phase 3 (malformed advert resilience): stdlib-only Python WebSocket
client publishes a syntactically valid Schnorr-signed Kind-37195
event whose `content` is gibberish (cannot deserialize as
OverlayAdvert). The relay enforces BIP-340 signature validity, so the
event reaches the consumers (rather than being dropped at the relay)
— a trivially-junk content payload is the right adversarial input.
Required ~80 lines of stdlib-only secp256k1 + BIP-340 in the script
(no new container deps). Asserts pidof fips on both nodes after the
publish, scans logs for panic markers, re-pings to prove the existing
peer link survives.
testing/nat/docker-compose.yml: new profile nostr-publish-consume
with two daemon services (nostr-pub-a 172.31.10.20, nostr-pub-b
172.31.10.21) on shared-lan, reusing the existing strfry relay
(172.31.10.30:7777) and STUN service (172.31.10.40:3478).
testing/nat/scripts/generate-configs.sh: 2-line allowlist update so
the new scenario flows through the existing config generator (rather
than forking a parallel one). Generated node-{a,b}.yaml + npubs.env
smoke-tested cleanly.
testing/ci-local.sh: NOSTR_RELAY_SUITES=(nostr-publish-consume)
array, run_nostr_publish_consume runner, dispatch in run_integration
and run_suite. Mirrors existing run_nat shape.
.github/workflows/ci.yml: one matrix row + 3 steps in the integration
job, gated on matrix.type == 'nostr-publish-consume'. Consumes the
same fips-linux artifact and fips-test:latest image as the existing
NAT suites.
Tor/TCP transport variants kept out of v0.3.0 scope; the structure
leaves room for nostr-publish-consume-tcp/-tor siblings later without
disturbing this baseline.
Add nft -c -f packaging/common/fips.nft syntax-check to both
testing/ci-local.sh and .github/workflows/ci.yml so a regression in
the 128-line firewall ruleset surfaces in the build gate rather than
when an operator activates fips-firewall.service.
testing/ci-local.sh: first step inside run_build(), before
cargo build --release. Uses command -v nft for prereq detection
mirroring the existing cargo-nextest pattern; records as nft-syntax
in RESULTS. Operator-facing message points at apt install nftables
when nft is absent.
.github/workflows/ci.yml: nftables added to the build job's existing
Linux apt-install step; new Validate fips.nft syntax (Linux only)
step gated on runner.os == 'Linux' (skips macOS/Windows matrix
slots, runs on ubuntu-latest and ubuntu-24.04-arm).
Note: nft -c -f requires netlink cache initialization on modern
nftables even in check mode, so both invocations use sudo (safe in
CI's passwordless sudo, and operator's typical local sudo). Without
sudo, nft fails with "cache initialization failed: Operation not
permitted" before reaching ruleset parse.
The local ci-local.sh and the GitHub CI clippy invocations both used
`cargo clippy --all -- -D warnings`, which only checks lib + bin
targets. Test code, integration tests, and benches were not lint-gated.
Three pre-existing clippy errors lurked in test modules as a result
(two field_reassign_with_default in config tests, one
items_after_test_module in stun.rs).
Tighten both invocations to `cargo clippy --all-targets --all-features
-- -D warnings` so the gate covers everything cargo can build, and
fix the three exposed errors:
- src/config/mod.rs: rewrite two test-only `Config::default()` +
field-reassign sites to struct-update syntax.
- src/discovery/nostr/stun.rs: move helper `random_txn_id` above the
`#[cfg(test)] mod tests` block.
Also adds a dedicated Clippy job to the GitHub CI workflow so the
strict gate runs on every PR (the workflow had no clippy job before;
clippy ran only via testing/ci-local.sh on operator machines).
No behavior changes; lint hygiene + CI hardening only.
Companion to the ethernet `accept_connections: false` rekey-deadlock
fix from earlier this release: the same dual-init failure mode shows
up over UDP when peers register by hostname, and the existing
addr_to_link-only carve-out in `should_admit_msg1` doesn't cover it.
The carve-out's first predicate keys `addr_to_link` by the literal
`TransportAddr` that `initiate_connection` inserted, which is the
hostname-form when a peer config carries a hostname (e.g.,
`core-vm.tail65015.ts.net:2121`). Inbound packets always arrive with
numeric source addrs because `udp_receive_loop` builds the
`TransportAddr` from the `SocketAddr` the kernel reports via
`recvfrom`. `TransportAddr` equality is byte-exact, so the two forms
don't match and the lookup misses. With `udp.accept_connections:
false` (or `udp.outbound_only: true`, which forces it false) the
gate then rejects the rekey msg1 from an established peer. The
dual-init tie-breaker stalls because the loser side never produces
msg2; both sides retry indefinitely and the winner side keeps
logging "Dual rekey initiation: we win, dropping their msg1" at 1Hz.
The earlier ethernet fix didn't generalize to this variant because
ethernet TransportAddrs are always numeric MAC bytes — both the
config-time form and the inbound-arrival form match identically.
Add a second predicate to `should_admit_msg1`: an active peer's
`current_addr()` matching `(transport_id, remote_addr)`.
`current_addr` is updated and refreshed from inbound encrypted-frame
source addrs (`handlers/encrypted.rs`), which are always numeric
`SocketAddr`-form, so this catches the established peer regardless
of how its `addr_to_link` key was originally inserted. The fast
`addr_to_link` check stays first; the iteration over peers is
bounded by peer count and only runs when the first predicate misses.
Regression coverage in this commit:
- Unit test `test_should_admit_msg1_admits_rekey_when_addr_form_differs`
in `src/node/tests/handshake.rs`. Constructs the failing scenario
in-process: `addr_to_link` populated with hostname-form key, peer's
`current_addr` at the resolved numeric form, query with numeric form.
Without the new predicate this fails immediately.
- New integration topology `rekey-outbound-only` plus matching
docker-compose profile. Same 5-node mesh shape as `rekey-accept-off`
but `inject-config` sets `udp.outbound_only: true` on node-b and
rewrites node-b's peer-c address from the numeric docker IP to the
docker hostname (`node-c:2121`), reproducing the production
hostname-vs-numeric mismatch. The test asserts no sustained
"Dual rekey initiation: we win" log lines on any node (>10 = bug)
and the existing rekey health checks catch the connectivity loss
the loop produces.
- `testing/ci-local.sh` and `.github/workflows/ci.yml` extended to
run the new variant in the local sweep and the GitHub CI integration
matrix alongside `rekey` and `rekey-accept-off`.
Verified locally: full `bash testing/ci-local.sh` sweep passes 29/29
suites (23m 12s) with the new variant green; 1084 unit tests pass.
Single combined commit covering five interlocking pieces of test and
CI work that landed during the v0.3.0-prep cycle.
## fips-gateway robustness
- src/bin/fips-gateway.rs DNS upstream probe converted from a 3-second
hard-fail to a bounded retry loop (5 attempts × 1s timeout, 1s sleep
between attempts; ~10s worst case). Covers the cold-boot race where
the daemon's TUN is up but the DNS responder at [::1]:5354 is still
binding. Each failed attempt logs at INFO. In production the binary's
retry is the live recovery mechanism; with retry it recovers silently
instead of relying on Restart=on-failure (~5s blip + spurious ERROR
per cycle).
- packaging/debian/fips-gateway.service `ExecStartPre` now waits up to
30 seconds for the daemon's `fips0` TUN to appear before exec'ing
the gateway binary. Eliminates the cold-boot race where the gateway
exits with `fips0 interface not found` and recovers via
`Restart=on-failure`, producing a 5-second blip and a spurious error
log per restart cycle.
- testing/docker/entrypoint.sh gateway-mode waits up to 30s for the
daemon's DNS responder to bind [::1]:5354 (probes once per second
with `dig @::1 -p 5354 ... test.fips`) before exec'ing fips-gateway.
Belt-and-suspenders with the binary's own retry: in CI we want
deterministic startup ordering. On timeout, fall through so the
binary's probe reports the definitive error.
## Test infrastructure DNS bind migration to ::1
After session 359's daemon DNS-bind default flipped from `127.0.0.1`
to `::1` (the production fix for ISSUE-2026-0002), the static-test
infrastructure was carrying a stale workaround that overrode the
default back to IPv4 loopback. The fips-gateway integration test
exposed the divergence: the gateway probes its DNS upstream at
`[::1]:5354` (production default) while the daemon was binding
`127.0.0.1:5354` from the template override — IPv6-explicit sockets
do not accept v4-mapped traffic, so the upstream probe exhausted
retries and the gateway exited.
- Drop the explicit `bind_addr: "127.0.0.1"` line from every test
config that emits it: testing/static/configs/node.template.yaml,
testing/chaos/configs/node.template.yaml, the sidecar heredoc in
testing/docker/entrypoint.sh, testing/acl-allowlist/generate-configs.sh
(six per-node blocks), testing/nat/scripts/generate-configs.sh, and
the four tor templates under testing/tor/. Daemon picks up its
production `::1` default.
- Flip the dnsmasq forwarder for `.fips` in testing/docker/Dockerfile
from `127.0.0.1#5354` to `::1#5354` so dnsmasq on the shared test
image continues to reach the daemon. Template and Dockerfile must
move together since most static suites resolve `<npub>.fips` via
the test-image dnsmasq.
## rekey-accept-off integration variant + UDP unit test
- New `rekey-accept-off` topology and docker-compose profile under
testing/static/. 2-node variant where node-b runs with
`udp.accept_connections: false`. Pins the regression class that
ISSUE-2026-0004 fixed (cross-connection winner's rekey msg1 was
being filtered by the accept_connections gate, breaking rekey).
- testing/static/scripts/rekey-test.sh accepts REKEY_TOPOLOGY and
REKEY_ACCEPT_OFF_NODES env vars; its inject-config subcommand
applies the per-node `udp.accept_connections: false` edit, and
the test asserts no sustained "Dual rekey initiation" log lines.
- New UDP variant of `should_admit_msg1` admit-rekey unit test in
src/node/tests/handshake.rs.
## ci-local.sh full integration coverage
- New runner functions and dispatcher entries for `acl-allowlist`,
`nat-cone` / `nat-symmetric` / `nat-lan`, `rekey-accept-off`,
`dns-resolver`, `deb-install`. Each integrates with the existing
summary tracking via `record`.
- New `--with-tor` flag (off by default) gates `tor-socks5-outbound`
and `tor-directory-mode` runners. Tor stays opt-in because both
harnesses depend on the live Tor network and would introduce a
flake source unrelated to the FIPS code.
- New suite arrays (`ACL_SUITES`, `NAT_SUITES`, `DNS_RESOLVER_SUITES`,
`DEB_INSTALL_SUITES`, `TOR_SUITES`) drive both the default sweep
and `--list` output.
- `run_suite` extended to accept the new suite names for `--only`
invocations.
## GitHub CI matrix expansions
- `gateway` matrix entry runs testing/static/scripts/gateway-test.sh
against the existing docker-compose `gateway` profile.
- `rekey-accept-off` matrix entry exercises the new topology with
REKEY_ACCEPT_OFF_NODES=b.
- `deb-install` matrix (debian12 + ubuntu24 + ubuntu26) runs
testing/deb-install/test.sh with privileged systemd containers.
~5-7 min cold cache, ~2 min warm per distro. Self-contained: builds
its own .deb in a Debian 12 cargo-deb builder image; does not
depend on the build job's pre-built artifact.
- `dns-resolver` matrix entry runs the full 13-scenario harness
(per-distro systemd resolver-backend tests + real-fips end-to-end
scenarios) in a single job. Pins the production DNS bind path that
ISSUE-2026-0002 lived in. ~7-12 min warm, ~12-15 min cold.
Verified locally: full `bash testing/ci-local.sh` sweep passes,
including 5/5 deb-install distros and all 13 dns-resolver scenarios.
Tor-inclusive sweep (`--with-tor`) verified in a follow-up run.
Drop the `tui`, `ble`, and `gateway` cargo features and replace
them with platform cfg gates. Plain `cargo build` now produces
every subsystem appropriate for the target platform with no
feature flags required.
Motivation:
- `default = ["tui", "ble"]` broke `cargo build` on macOS and
Windows because `ble` pulled in `bluer` (BlueZ, Linux-only).
Every non-Linux packager needed `--no-default-features`.
- The feature flags on `ble` and `gateway` were redundant with
their platform-gated deps (`bluer`, `rustables`). The parallel
gating was inconsistent and error-prone.
- `tui` feature protected against a ratatui binary-size concern
that no longer applies in 2026.
Cargo.toml:
- Remove `tui`, `ble`, `gateway` features; `default = []`.
- Promote `ratatui` to a non-optional top-level dependency.
- Move `rustables` from top-level optional into the Linux
target block, non-optional.
- Split `bluer` into its own target block with
`cfg(all(target_os = "linux", not(target_env = "musl")))`
— BlueZ isn't available on musl router targets and
`libdbus-sys` doesn't cross-compile to musl without pkg-config
sysroot setup.
- Drop `required-features` from the `fipstop` and `fips-gateway`
`[[bin]]` entries.
build.rs:
- Emit a `bluer_available` custom cfg when `target_os == "linux"`
and `target_env != "musl"`, for use in place of the verbose
full predicate in source cfg gates.
Source:
- Replace every `#[cfg(feature = "gateway")]` with
`#[cfg(target_os = "linux")]`. Gateway code works on both
glibc and musl Linux (rustables is fine on musl).
- Replace every `#[cfg(feature = "ble")]` with
`#[cfg(bluer_available)]`. BLE-specific code (BluerIo module,
bluer type conversions, BLE transport instance creation,
resolve_ble_addr) is excluded on musl and non-Linux. Generic
`BleAddr`, `BleIo` trait, `MockBleIo`, and `BleTransport<I>`
still compile on all targets.
- `src/bin/fips-gateway.rs`: always compiled, but `main()` is
gated to Linux. Non-Linux stub exits 1 with a diagnostic.
Existing non-Linux packaging scripts don't ship it, so the
stub binary sits unused.
Packaging and CI:
- Drop `--features` and `--no-default-features` flags from every
packaging script and workflow. Defaults now match each
platform's capabilities.
- AUR `fips-git` automatically aligns with stable `PKGBUILD`
(both build with defaults).
Verified: `cargo build --release` with no flags produces all
four binaries on glibc Linux; all unit and integration tests
pass across Linux/macOS/Windows/OpenWrt (musl) in CI.
Mesh peers can now reach a configured host:port on the gateway's LAN
via static port-forward rules on fips-gateway. Mirror of the outbound
LAN gateway (IDEA-0079 / TASK-2026-0056).
Config: new gateway.port_forwards list of { listen_port, proto,
target } entries. Targets are SocketAddrV6 — IPv4 is rejected at
parse time. Validation rejects zero listen ports and duplicate
(listen_port, proto) pairs.
NAT: NatManager gains a port_forwards field and set_port_forwards()
setter, rebuilt in the same atomic rustables batch as the address
mappings. Each forward emits a prerouting DNAT rule keyed on
(iifname fips0, nfproto ipv6, l4proto, tcp/udp dport) that rewrites
destination address and port via Nat::with_ip_register +
with_port_register. When any forwards are configured, a single
LAN-side masquerade is installed on (iifname fips0, oifname
lan_interface, nfproto ipv6) so the LAN host sees the gateway as
source and replies flow back through conntrack. This rule is
distinct from the existing oifname fips0 masquerade that serves the
outbound pool.
Binary: fips-gateway validates port_forwards at startup and calls
set_port_forwards after NatManager construction; startup failure
cleans up the nftables table before exiting.
Test: extend testing/static/scripts/gateway-test.sh with Phase 7
that runs a marker HTTP server on the LAN-side client (fd02::20:8080)
and, from the mesh peer, curls the gateway's fips0 address on port
18080 to exercise the full DNAT + LAN masquerade path. The
LAN-side HTTP server is started with 'docker exec -d' plus a
bind-ready poll on ss; 'docker exec bash -c "cmd &"' does not
keep the child alive past the exec session even with nohup.
Test-infra: ci-local.sh now builds/clippies/tests with
--features gateway, matching GitHub CI. Without this the release
fips-gateway binary silently stays stale across runs, since
cargo build --release alone does not compile the gateway bin.
Verified locally: cargo test --features gateway --lib (991 pass),
clippy + fmt clean, full testing/ci-local.sh green (21/21 suites
in 8m36s, including the new gateway Phase 7).
Add fips-gateway binary: a separate daemon that allows unmodified LAN
hosts to reach FIPS mesh destinations via DNS-allocated virtual IPs
and kernel nftables NAT.
Gateway DNS resolver: forwarding proxy on [::]:53 that intercepts
.fips queries, forwards to daemon resolver (localhost:5354), allocates
virtual IPs from pool, returns AAAA records. Always sends AAAA upstream
regardless of client query type, returns proper NODATA for non-AAAA.
Virtual IP pool: fd01::/112 pool with state machine lifecycle
(Allocated → Active → Draining → Free), TTL-based reclamation,
conntrack integration for session tracking.
NAT manager: nftables DNAT/SNAT rules via rustables netlink API,
per-mapping rule lifecycle, fips0 masquerade for LAN client source
address rewriting.
Network setup: local pool route, proxy NDP for virtual IPs on LAN
interface, IPv6 forwarding validation.
Control socket at /run/fips/gateway.sock with show_gateway and
show_mappings queries. fipstop Gateway tab with pool summary gauge
and mappings table.
Gateway config section in fips.yaml with pool CIDR, LAN interface,
DNS upstream, TTL, and grace period settings.
Design doc at docs/design/fips-gateway.md.
Integration test (testing/static/scripts/gateway-test.sh): three
containers verifying DNS resolution, end-to-end HTTP, NAT state,
TTL expiration, SERVFAIL fallback, and clean shutdown.
Johnathan Corgan
