laantungir/fips
1
0
Fork 0
mirror of https://github.com/jmcorgan/fips.git synced 2026-08-09 08:14:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

Test infrastructure overhaul: gateway robustness + full CI coverage

Browse Source 
Single combined commit covering five interlocking pieces of test and
CI work that landed during the v0.3.0-prep cycle.

## fips-gateway robustness

- src/bin/fips-gateway.rs DNS upstream probe converted from a 3-second
  hard-fail to a bounded retry loop (5 attempts × 1s timeout, 1s sleep
  between attempts; ~10s worst case). Covers the cold-boot race where
  the daemon's TUN is up but the DNS responder at [::1]:5354 is still
  binding. Each failed attempt logs at INFO. In production the binary's
  retry is the live recovery mechanism; with retry it recovers silently
  instead of relying on Restart=on-failure (~5s blip + spurious ERROR
  per cycle).
- packaging/debian/fips-gateway.service `ExecStartPre` now waits up to
  30 seconds for the daemon's `fips0` TUN to appear before exec'ing
  the gateway binary. Eliminates the cold-boot race where the gateway
  exits with `fips0 interface not found` and recovers via
  `Restart=on-failure`, producing a 5-second blip and a spurious error
  log per restart cycle.
- testing/docker/entrypoint.sh gateway-mode waits up to 30s for the
  daemon's DNS responder to bind [::1]:5354 (probes once per second
  with `dig @::1 -p 5354 ... test.fips`) before exec'ing fips-gateway.
  Belt-and-suspenders with the binary's own retry: in CI we want
  deterministic startup ordering. On timeout, fall through so the
  binary's probe reports the definitive error.

## Test infrastructure DNS bind migration to ::1

After session 359's daemon DNS-bind default flipped from `127.0.0.1`
to `::1` (the production fix for ISSUE-2026-0002), the static-test
infrastructure was carrying a stale workaround that overrode the
default back to IPv4 loopback. The fips-gateway integration test
exposed the divergence: the gateway probes its DNS upstream at
`[::1]:5354` (production default) while the daemon was binding
`127.0.0.1:5354` from the template override — IPv6-explicit sockets
do not accept v4-mapped traffic, so the upstream probe exhausted
retries and the gateway exited.

- Drop the explicit `bind_addr: "127.0.0.1"` line from every test
  config that emits it: testing/static/configs/node.template.yaml,
  testing/chaos/configs/node.template.yaml, the sidecar heredoc in
  testing/docker/entrypoint.sh, testing/acl-allowlist/generate-configs.sh
  (six per-node blocks), testing/nat/scripts/generate-configs.sh, and
  the four tor templates under testing/tor/. Daemon picks up its
  production `::1` default.
- Flip the dnsmasq forwarder for `.fips` in testing/docker/Dockerfile
  from `127.0.0.1#5354` to `::1#5354` so dnsmasq on the shared test
  image continues to reach the daemon. Template and Dockerfile must
  move together since most static suites resolve `<npub>.fips` via
  the test-image dnsmasq.

## rekey-accept-off integration variant + UDP unit test

- New `rekey-accept-off` topology and docker-compose profile under
  testing/static/. 2-node variant where node-b runs with
  `udp.accept_connections: false`. Pins the regression class that
  ISSUE-2026-0004 fixed (cross-connection winner's rekey msg1 was
  being filtered by the accept_connections gate, breaking rekey).
- testing/static/scripts/rekey-test.sh accepts REKEY_TOPOLOGY and
  REKEY_ACCEPT_OFF_NODES env vars; its inject-config subcommand
  applies the per-node `udp.accept_connections: false` edit, and
  the test asserts no sustained "Dual rekey initiation" log lines.
- New UDP variant of `should_admit_msg1` admit-rekey unit test in
  src/node/tests/handshake.rs.

## ci-local.sh full integration coverage

- New runner functions and dispatcher entries for `acl-allowlist`,
  `nat-cone` / `nat-symmetric` / `nat-lan`, `rekey-accept-off`,
  `dns-resolver`, `deb-install`. Each integrates with the existing
  summary tracking via `record`.
- New `--with-tor` flag (off by default) gates `tor-socks5-outbound`
  and `tor-directory-mode` runners. Tor stays opt-in because both
  harnesses depend on the live Tor network and would introduce a
  flake source unrelated to the FIPS code.
- New suite arrays (`ACL_SUITES`, `NAT_SUITES`, `DNS_RESOLVER_SUITES`,
  `DEB_INSTALL_SUITES`, `TOR_SUITES`) drive both the default sweep
  and `--list` output.
- `run_suite` extended to accept the new suite names for `--only`
  invocations.

## GitHub CI matrix expansions

- `gateway` matrix entry runs testing/static/scripts/gateway-test.sh
  against the existing docker-compose `gateway` profile.
- `rekey-accept-off` matrix entry exercises the new topology with
  REKEY_ACCEPT_OFF_NODES=b.
- `deb-install` matrix (debian12 + ubuntu24 + ubuntu26) runs
  testing/deb-install/test.sh with privileged systemd containers.
  ~5-7 min cold cache, ~2 min warm per distro. Self-contained: builds
  its own .deb in a Debian 12 cargo-deb builder image; does not
  depend on the build job's pre-built artifact.
- `dns-resolver` matrix entry runs the full 13-scenario harness
  (per-distro systemd resolver-backend tests + real-fips end-to-end
  scenarios) in a single job. Pins the production DNS bind path that
  ISSUE-2026-0002 lived in. ~7-12 min warm, ~12-15 min cold.

Verified locally: full `bash testing/ci-local.sh` sweep passes,
including 5/5 deb-install distros and all 13 dns-resolver scenarios.
Tor-inclusive sweep (`--with-tor`) verified in a follow-up run.
This commit is contained in:
Johnathan Corgan
2026-04-30 10:24:32 +00:00
parent 674c7fe1ff
commit 37c2973e2f
20 changed files with 640 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files
testing/static/scripts/rekey-test.sh
+55 -3
View File
@@ -20,8 +20,17 @@ set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../../lib/wait-converge.sh"
TOPOLOGY="rekey"
# Selectable topology — defaults to "rekey" but the rekey-accept-off
# variant exercises the auto_connect-initiator-with-accept-off
# regression class (udp.accept_connections=false on a peer that
# also auto-connects).
TOPOLOGY="${REKEY_TOPOLOGY:-rekey}"
NODES="a b c d e"
# Comma-separated list of node IDs to set udp.accept_connections=false
# on during inject-config. Empty (default) leaves all nodes accepting.
# When set, also asserted by the test that no sustained "Dual rekey
# initiation" log lines appear on the affected node.
REKEY_ACCEPT_OFF_NODES="${REKEY_ACCEPT_OFF_NODES:-}"
# Rekey timing configuration
REKEY_AFTER_SECS=35
@@ -30,13 +39,24 @@ REKEY_AFTER_SECS=35
# Inject rekey config into generated node configs. Called separately
# by CI before building Docker images.
if [ "${1:-}" = "inject-config" ]; then
echo "Injecting rekey config (after_secs=$REKEY_AFTER_SECS) into node configs..."
echo "Injecting rekey config (after_secs=$REKEY_AFTER_SECS) into node configs (topology=$TOPOLOGY)..."
if [ -n "$REKEY_ACCEPT_OFF_NODES" ]; then
echo " Setting udp.accept_connections=false on nodes: $REKEY_ACCEPT_OFF_NODES"
fi
for node in $NODES; do
cfg="$SCRIPT_DIR/../generated-configs/$TOPOLOGY/node-$node.yaml"
if [ ! -f "$cfg" ]; then
echo " Error: $cfg not found" >&2
exit 1
fi
accept_off="false"
if [ -n "$REKEY_ACCEPT_OFF_NODES" ]; then
for off_node in ${REKEY_ACCEPT_OFF_NODES//,/ }; do
if [ "$off_node" = "$node" ]; then
accept_off="true"
fi
done
fi
python3 -c "
import yaml
with open('$cfg') as f:
@@ -46,10 +66,22 @@ cfg.setdefault('node', {})['rekey'] = {
'after_secs': $REKEY_AFTER_SECS,
'after_messages': 65536,
}
if '$accept_off' == 'true':
transports = cfg.setdefault('transports', {})
udp = transports.get('udp')
if udp is None:
udp = {'bind_addr': '0.0.0.0:2121'}
transports['udp'] = udp
if isinstance(udp, dict):
udp['accept_connections'] = False
with open('$cfg', 'w') as f:
yaml.dump(cfg, f, default_flow_style=False, sort_keys=False)
"
echo " ✓ node-$node"
if [ "$accept_off" = "true" ]; then
echo " ✓ node-$node (accept_connections=false)"
else
echo " ✓ node-$node"
fi
done
echo "✓ Config injection complete"
exit 0
@@ -316,6 +348,26 @@ assert_zero_count "Rekey msg2 processing failed" "Rekey msg2 failures"
assert_zero_count "Session AEAD decryption failed" \
"FSP decryption failures during rekey"
# Variant-specific: when one or more nodes have udp.accept_connections=false,
# verify the dual-init carve-out keeps the "we win, dropping their msg1"
# log line below the bug threshold. Pre-fix, a 1Hz dual-init loop produced
# ~120 occurrences over the 2-minute test; with the carve-out, the line
# fires at most a handful of times from genuine simultaneous rekeys.
if [ -n "$REKEY_ACCEPT_OFF_NODES" ]; then
DUAL_INIT_THRESHOLD=10
for off_node in ${REKEY_ACCEPT_OFF_NODES//,/ }; do
count=$(docker logs "fips-node-$off_node" 2>&1 \
| grep -cE "Dual rekey initiation: we win" || true)
if [ "${count:-0}" -le "$DUAL_INIT_THRESHOLD" ]; then
echo " PASS: node-$off_node dual-init drops below threshold ($count <= $DUAL_INIT_THRESHOLD)"
PASSED=$((PASSED + 1))
else
echo " FAIL: node-$off_node sustained dual-init drops ($count > $DUAL_INIT_THRESHOLD)"
FAILED=$((FAILED + 1))
fi
done
fi
phase_result "Log analysis"
echo ""