mirror of
https://github.com/jmcorgan/fips.git
synced 2026-08-09 00:04:54 +00:00
Test infrastructure overhaul: gateway robustness + full CI coverage
Single combined commit covering five interlocking pieces of test and CI work that landed during the v0.3.0-prep cycle. ## fips-gateway robustness - src/bin/fips-gateway.rs DNS upstream probe converted from a 3-second hard-fail to a bounded retry loop (5 attempts × 1s timeout, 1s sleep between attempts; ~10s worst case). Covers the cold-boot race where the daemon's TUN is up but the DNS responder at [::1]:5354 is still binding. Each failed attempt logs at INFO. In production the binary's retry is the live recovery mechanism; with retry it recovers silently instead of relying on Restart=on-failure (~5s blip + spurious ERROR per cycle). - packaging/debian/fips-gateway.service `ExecStartPre` now waits up to 30 seconds for the daemon's `fips0` TUN to appear before exec'ing the gateway binary. Eliminates the cold-boot race where the gateway exits with `fips0 interface not found` and recovers via `Restart=on-failure`, producing a 5-second blip and a spurious error log per restart cycle. - testing/docker/entrypoint.sh gateway-mode waits up to 30s for the daemon's DNS responder to bind [::1]:5354 (probes once per second with `dig @::1 -p 5354 ... test.fips`) before exec'ing fips-gateway. Belt-and-suspenders with the binary's own retry: in CI we want deterministic startup ordering. On timeout, fall through so the binary's probe reports the definitive error. ## Test infrastructure DNS bind migration to ::1 After session 359's daemon DNS-bind default flipped from `127.0.0.1` to `::1` (the production fix for ISSUE-2026-0002), the static-test infrastructure was carrying a stale workaround that overrode the default back to IPv4 loopback. The fips-gateway integration test exposed the divergence: the gateway probes its DNS upstream at `[::1]:5354` (production default) while the daemon was binding `127.0.0.1:5354` from the template override — IPv6-explicit sockets do not accept v4-mapped traffic, so the upstream probe exhausted retries and the gateway exited. - Drop the explicit `bind_addr: "127.0.0.1"` line from every test config that emits it: testing/static/configs/node.template.yaml, testing/chaos/configs/node.template.yaml, the sidecar heredoc in testing/docker/entrypoint.sh, testing/acl-allowlist/generate-configs.sh (six per-node blocks), testing/nat/scripts/generate-configs.sh, and the four tor templates under testing/tor/. Daemon picks up its production `::1` default. - Flip the dnsmasq forwarder for `.fips` in testing/docker/Dockerfile from `127.0.0.1#5354` to `::1#5354` so dnsmasq on the shared test image continues to reach the daemon. Template and Dockerfile must move together since most static suites resolve `<npub>.fips` via the test-image dnsmasq. ## rekey-accept-off integration variant + UDP unit test - New `rekey-accept-off` topology and docker-compose profile under testing/static/. 2-node variant where node-b runs with `udp.accept_connections: false`. Pins the regression class that ISSUE-2026-0004 fixed (cross-connection winner's rekey msg1 was being filtered by the accept_connections gate, breaking rekey). - testing/static/scripts/rekey-test.sh accepts REKEY_TOPOLOGY and REKEY_ACCEPT_OFF_NODES env vars; its inject-config subcommand applies the per-node `udp.accept_connections: false` edit, and the test asserts no sustained "Dual rekey initiation" log lines. - New UDP variant of `should_admit_msg1` admit-rekey unit test in src/node/tests/handshake.rs. ## ci-local.sh full integration coverage - New runner functions and dispatcher entries for `acl-allowlist`, `nat-cone` / `nat-symmetric` / `nat-lan`, `rekey-accept-off`, `dns-resolver`, `deb-install`. Each integrates with the existing summary tracking via `record`. - New `--with-tor` flag (off by default) gates `tor-socks5-outbound` and `tor-directory-mode` runners. Tor stays opt-in because both harnesses depend on the live Tor network and would introduce a flake source unrelated to the FIPS code. - New suite arrays (`ACL_SUITES`, `NAT_SUITES`, `DNS_RESOLVER_SUITES`, `DEB_INSTALL_SUITES`, `TOR_SUITES`) drive both the default sweep and `--list` output. - `run_suite` extended to accept the new suite names for `--only` invocations. ## GitHub CI matrix expansions - `gateway` matrix entry runs testing/static/scripts/gateway-test.sh against the existing docker-compose `gateway` profile. - `rekey-accept-off` matrix entry exercises the new topology with REKEY_ACCEPT_OFF_NODES=b. - `deb-install` matrix (debian12 + ubuntu24 + ubuntu26) runs testing/deb-install/test.sh with privileged systemd containers. ~5-7 min cold cache, ~2 min warm per distro. Self-contained: builds its own .deb in a Debian 12 cargo-deb builder image; does not depend on the build job's pre-built artifact. - `dns-resolver` matrix entry runs the full 13-scenario harness (per-distro systemd resolver-backend tests + real-fips end-to-end scenarios) in a single job. Pins the production DNS bind path that ISSUE-2026-0002 lived in. ~7-12 min warm, ~12-15 min cold. Verified locally: full `bash testing/ci-local.sh` sweep passes, including 5/5 deb-install distros and all 13 dns-resolver scenarios. Tor-inclusive sweep (`--with-tor`) verified in a follow-up run.
This commit is contained in:
Johnathan Corgan
+59
-24
@@ -202,36 +202,71 @@ async fn main() {
|
||||
}
|
||||
};
|
||||
|
||||
if let Err(e) = sock.send_to(&query, upstream_addr).await {
|
||||
error!(
|
||||
upstream = %upstream, error = %e,
|
||||
"Failed to send DNS probe — is the FIPS daemon running?"
|
||||
);
|
||||
std::process::exit(1);
|
||||
}
|
||||
// Retry the upstream probe up to MAX_PROBE_ATTEMPTS times with a
|
||||
// 1-second per-attempt timeout and a 1-second sleep between
|
||||
// attempts. Total worst-case wait: ~10 seconds.
|
||||
//
|
||||
// Bounded retry covers the cold-boot race where this gateway and
|
||||
// the fips daemon start at approximately the same time: the
|
||||
// daemon's TUN may be up (the systemd ExecStartPre wait already
|
||||
// gates on that) while its DNS responder is still binding
|
||||
// [::1]:5354. Without this retry, the gateway hard-failed after
|
||||
// a single 3-second probe and relied on Restart=on-failure for
|
||||
// recovery.
|
||||
const MAX_PROBE_ATTEMPTS: u32 = 5;
|
||||
const PROBE_TIMEOUT_SECS: u64 = 1;
|
||||
const PROBE_RETRY_DELAY_SECS: u64 = 1;
|
||||
|
||||
let mut buf = [0u8; 512];
|
||||
match tokio::time::timeout(std::time::Duration::from_secs(3), sock.recv_from(&mut buf))
|
||||
.await
|
||||
{
|
||||
Ok(Ok(_)) => {
|
||||
info!(upstream = %upstream, "DNS upstream is reachable");
|
||||
let mut last_failure: Option<String> = None;
|
||||
let mut succeeded = false;
|
||||
for attempt in 1..=MAX_PROBE_ATTEMPTS {
|
||||
if let Err(e) = sock.send_to(&query, upstream_addr).await {
|
||||
last_failure = Some(format!("send_to failed: {}", e));
|
||||
} else {
|
||||
match tokio::time::timeout(
|
||||
std::time::Duration::from_secs(PROBE_TIMEOUT_SECS),
|
||||
sock.recv_from(&mut buf),
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(Ok(_)) => {
|
||||
info!(
|
||||
upstream = %upstream, attempt = attempt,
|
||||
"DNS upstream is reachable"
|
||||
);
|
||||
succeeded = true;
|
||||
break;
|
||||
}
|
||||
Ok(Err(e)) => {
|
||||
last_failure = Some(format!("recv_from failed: {}", e));
|
||||
}
|
||||
Err(_) => {
|
||||
last_failure = Some(format!(
|
||||
"no response within {}s",
|
||||
PROBE_TIMEOUT_SECS
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(Err(e)) => {
|
||||
error!(
|
||||
upstream = %upstream, error = %e,
|
||||
"DNS upstream recv failed — is the FIPS daemon running?"
|
||||
if attempt < MAX_PROBE_ATTEMPTS {
|
||||
info!(
|
||||
upstream = %upstream, attempt = attempt,
|
||||
last = ?last_failure,
|
||||
"DNS upstream probe attempt failed; retrying"
|
||||
);
|
||||
std::process::exit(1);
|
||||
}
|
||||
Err(_) => {
|
||||
error!(
|
||||
upstream = %upstream,
|
||||
"DNS upstream did not respond within 3s — is the FIPS daemon running?"
|
||||
);
|
||||
std::process::exit(1);
|
||||
tokio::time::sleep(std::time::Duration::from_secs(PROBE_RETRY_DELAY_SECS)).await;
|
||||
}
|
||||
}
|
||||
if !succeeded {
|
||||
error!(
|
||||
upstream = %upstream,
|
||||
attempts = MAX_PROBE_ATTEMPTS,
|
||||
last = ?last_failure,
|
||||
"DNS upstream did not become reachable after exhausting retries — is the FIPS daemon running?"
|
||||
);
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
// --- Initialize components ---
|
||||
|
||||
@@ -1017,3 +1017,42 @@ async fn test_should_admit_msg1_admits_rekey_when_accept_off() {
|
||||
|
||||
assert!(node.should_admit_msg1(transport_id, &addr));
|
||||
}
|
||||
|
||||
/// Same regression coverage as the TCP test above, but exercising the
|
||||
/// UDP transport's new `accept_connections` config field (introduced
|
||||
/// alongside the `outbound_only` mode). Proves the Node-level gate's
|
||||
/// addr_to_link carve-out is transport-agnostic and that the new UDP
|
||||
/// config knob is wired correctly through the Transport trait.
|
||||
#[tokio::test]
|
||||
async fn test_should_admit_msg1_admits_rekey_when_udp_accept_off() {
|
||||
use crate::config::UdpConfig;
|
||||
use crate::transport::udp::UdpTransport;
|
||||
|
||||
let mut node = make_node();
|
||||
let transport_id = TransportId::new(1);
|
||||
|
||||
let cfg = UdpConfig {
|
||||
bind_addr: Some("127.0.0.1:0".to_string()),
|
||||
accept_connections: Some(false),
|
||||
..Default::default()
|
||||
};
|
||||
let (tx, _rx) = packet_channel(64);
|
||||
let udp = UdpTransport::new(transport_id, None, cfg, tx);
|
||||
node.transports
|
||||
.insert(transport_id, TransportHandle::Udp(udp));
|
||||
|
||||
let addr = TransportAddr::from_string("10.0.0.2:2121");
|
||||
|
||||
// Fresh msg1 (no addr_to_link entry) is rejected by the gate when
|
||||
// the transport refuses inbound.
|
||||
assert!(!node.should_admit_msg1(transport_id, &addr));
|
||||
|
||||
// Pre-populate addr_to_link as if a session were established. The
|
||||
// rekey carve-out admits the msg1 even though the transport still
|
||||
// says accept_connections() == false.
|
||||
let link_id = node.allocate_link_id();
|
||||
node.addr_to_link
|
||||
.insert((transport_id, addr.clone()), link_id);
|
||||
|
||||
assert!(node.should_admit_msg1(transport_id, &addr));
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user