mirror of
https://github.com/jmcorgan/fips.git
synced 2026-07-30 19:46:15 +00:00
master
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Johnathan Corgan
|
49163befd5 |
Give each local CI run its own docker build context
Scoping the image tag was never sufficient on its own. Every build read one unscoped directory in the working tree, into which each run copies the binaries it just built, so two concurrent runs raced on the contents of the context as well as on the name of the result — and a run could produce a correctly-per-run-tagged image built from the other run's binaries. The run now copies the context's tracked files into its own directory, installs its binaries there, and builds from it, exporting the path so every other consumer follows. Deliberately not carried over: a previous run's binaries, since inheriting them is the failure this prevents. The path is absolute because compose resolves a relative build context against the compose file's own directory rather than the working directory, which was measured rather than assumed. The chaos entry script gated the whole simulation on a binary in the shared directory by literal path, so it moves in this same commit: left behind, it would have failed on a clean checkout and, worse on a host with leftovers, passed while reading a binary that was not the one under test. Teardown removes the directory on red runs as well as green, since it holds only reproducible content and is never the evidence of a failure. The worker's SIGKILL runs no trap, so the cleanup script also sweeps contexts left by a preempted run, and the ignore rule keeps a concurrent run from showing up as untracked working-tree noise. |
||
|
Johnathan Corgan
|
a718cef8ce |
Have every suite compose file name the caller's test image
Only the static family read FIPS_TEST_IMAGE; the firewall, ACL, nat, sidecar and both Tor compose files named the shared mutable tag directly, which is why the local CI runner has to retag its per-run image to that name. They now take the same defaulted form the static file already uses, so a harness run resolves the image it built and a bare hand run still resolves the shared tag exactly as before. Checked by rendering each file through `docker compose config` with the variables set: every service resolves to the supplied name and none is left naming the shared tag. |
||
|
Johnathan Corgan
|
cbbdf2c13c |
Stop the firewall and ACL suites rebuilding an image their caller supplied
Both suites ran `docker compose up -d --build` unconditionally, and their compose files carry both a build context and an image name, so the flag rebuilt and retagged the shared test image from the shared build context on every run. Their --skip-build flag did not cover it: it guarded only the Rust build above. Under a harness that has already built the image and handed it over, that rebuild can only replace the binaries under test with whatever the shared context currently holds. The nat scripts had the mirror-image problem. Their image guard built a replacement whenever the image was missing, which is right by hand and wrong under a harness: when the caller names an image it built, a miss means something upstream is broken, and manufacturing a substitute hides that. They now fail loudly in that case and keep building only on the hand path. The mesh-lab guard reads the same variable and its build hint moves to the script that still produces the shared tag. |
||
|
Johnathan Corgan
|
611f045d33 |
Let the acl-allowlist suite float its docker subnet
The compose pinned 172.31.0.0/24 with a per-container ipv4_address, so two concurrent runs of this suite asked docker for the same address space and the second failed with a pool-overlap error. Request no subnet instead and let docker assign one from the daemon pool, which leaves nothing for two runs to contend for. Peers now address each other by the docker hostname the compose already sets (host-a through host-f) rather than by literal IP, and docker's embedded DNS is per-network, so the same hostname in two runs resolves inside each run's own subnet. The generated config directory moves under the run suffix in the same change, because it has to: the generator does rm -rf on it, so a shared directory means one run deletes the fixtures another is still using, which fails silently rather than loudly at bring-up. Scoping it requires the generator output path, all five bind-mount sources per service, and the gitignore entry together; scoping only the generator leaves the compose reading the unscoped path. The floating subnet removes one of the two obstacles to running this suite twice at once, not both: the compose project name is still fixed, and nothing scopes it for this suite. Recorded at the site so the comment does not claim more than the change delivers. |
||
|
Johnathan Corgan
|
98560594cf |
Prove ACL rejection per peer by matching a single log line
The four rejection checks were independent greps over the whole log: two npubs, a context and a decision. Together they proved only that node-a mentioned each npub somewhere, rejected somebody on an inbound handshake, and rejected somebody by denylist. Nothing tied a rejection to a named peer, and node-a lists both denied peers as auto_connect peers, so it logs their npubs on the outbound connect path whether or not any rejection ever happened. The actual message was never matched. Replace them with a helper that requires every given string on ONE line, and assert per denied peer: the real message, that peer's npub, and the denylist decision together. Strings match in any order by chaining fixed-string greps over the surviving lines, so the assertion does not depend on how the log formatter orders a message and its fields. A grep over empty input yields the empty string rather than anything a caller could mistake for a match, so an unreadable container times out and fails instead of passing. Two limits are written at the call sites rather than left implied. The decision conjunct discriminates nothing, since the two allowing variants return early and denylist is the only value that can reach that warning. And node-a authorizes before dialing, so it emits a fully formed rejection line for each denied peer on the outbound path; these assertions are satisfiable without the inbound check running at all, and it is the peer-count assertions that would catch that. Verified against the running harness: all three find a real matching line. The defect shape they replace was exercised offline first, with the three strings spread across three lines, where the old checks passed and the new ones fail. |
||
|
Johnathan Corgan
|
291c4312dc |
Stop the ACL peer-count reader from turning "no answer" into zero
wait_for_peers_exact fell back to 0 when it could not read a container's peer count. Two of its six call sites are the ACL denial checks, which expect exactly 0 connected peers — so a failed docker exec satisfied them on the first iteration and the isolation property the suite exists to prove was never observed. Confirmed against a container that does not exist: the old reader returns "0" and the expect-zero comparison passes. The reader now returns the empty string when the daemon does not answer, which no numeric comparison can satisfy, and the timeout message says which of the two happened — never answered, or answered the wrong number. Same shape as admission-cap-test.sh's read_peer_count. |
||
|
Johnathan Corgan
|
e4a854f6b0 |
Stop concurrent local CI runs from clobbering each other's containers
Two runs on one host destroyed each other's containers, producing mid-test "No such container" failures that look like real defects. The automated builder runs a full local CI on the same box every few minutes, so the machine is contended almost always and this has red-ed both a hand run and an automated gate. Two independent causes are fixed here. Container names were hardcoded, and docker names are global rather than scoped by compose project, so two runs collided on the same name; every name now takes an optional suffix that the harness sets from the run id. And the cleanup sweep matched a label shared by every run, so one run's teardown force-removed another's containers; resources now also carry a per-run label and the sweep can be narrowed to it. A third hazard turned up that was not in the original report: the sidecar suite passes explicit compose project names, which override the run-scoped project and put it outside the shared prefix entirely. Its project names, network, and derived container references are now scoped too. Both are default-off. With the suffix unset, names render exactly as they do today and a bare compose invocation is unchanged, which is what keeps the hosted CI and the documentation correct. A cleanup run with no run id still reaps everything, which is what a manual "clear the box" wants. The literal-name sweep was not sufficient: six scripts build container names dynamically from node labels, and two suites create their own containers outside compose. Those are handled at their construction sites. Verified: syntax check on all modified scripts; compose validation on every modified file with the suffix both set and unset; and a synthetic two-run reproduction that shows the old cleanup destroying a bystander run and the new one leaving it alone. Known gap: subnets are still hardcoded, so two concurrent full runs will still collide on address-pool overlap. That fix reaches into topology configs, chaos scenarios, diagrams and production source, so it is left for its own change rather than half-done here. |
||
|
Johnathan Corgan
|
965de26239 |
testing: make ci-local.sh preemption-safe with per-run docker isolation
A CI worker may preempt an in-flight ci-local.sh run (SIGTERM, then SIGKILL after a grace period) to restart on a newer commit. For that kill to be safe, the script must clean up after itself and never let a dying run collide with its restart. It previously had no signal handling, shared the default compose project name across runs, and tore down each suite only at the suite end. - Derive a per-run id (honoring FIPS_CI_RUN_ID, else short-sha+random) and namespace every docker resource to it: a fipsci_<run>_<suite> compose project per suite and per parallel chaos child, and per-run image tags (fips-test:<run>, fips-test-app:<run>) retagged to :latest only after both builds succeed so :latest never points at a half-built image. - Install a bounded, idempotent teardown trap on SIGTERM/SIGINT (+ EXIT): reap parallel chaos children, then force-remove this run's docker resources via the new ci-cleanup.sh, wrapped in timeout so a stuck down cannot wedge it. - Exit 143 (SIGTERM) / 130 (SIGINT), distinct from 0 (pass) / 1 (failed), so a preempting worker tells a cancelled run from a real failure. - Add ci-cleanup.sh (also ci-local.sh --reap): force-removes leftover CI resources by the com.corganlabs.fips-ci=1 label and the fipsci_ project prefix, robust to however a prior run died. - Label every per-suite docker resource so the label sweep reaps it after a SIGKILL regardless of network name: direct docker run/network resources, the sidecar compose services, and every per-suite compose network (acl-allowlist, boringtun, firewall, nat, static, both tor suites, and the chaos generator template). Parametrize the static/sidecar compose image refs so the per-run tags are honored. - Give each parallel chaos child a unique /24 from 10.30.x (a new --subnet override on the sim CLI, assigned per-child in ci-local.sh) so parallel children never collide on a shared docker subnet, and a chaos net can never span a fixed-subnet suite (sidecar/static in 172.20.x). 10.30.x sits outside docker's default-address-pool range, so an auto-assigned net cannot land on it either; node IPs derive from the subnet, so no scenario config changes. |
||
|
Johnathan Corgan
|
e9dd3167f2 |
test(acl-allowlist): poll log assertion to absorb XX-handshake timing race
Convert assert_log_contains from a one-shot grep snapshot into a bounded poll that retries until the pattern appears or the timeout elapses (default 15s). Same wait-with-timeout shape as wait_for_peers_exact above it in the file. The pre-existing flake on next-branch CI is structural: under XX handshake, the cross-connection tie-breaker selects which side reaches its ACL-check point first. When container-a wins the tie-breaker on the first attempt against c and d, only the outbound-handshake-context rejection fires immediately, and the inbound-handshake-context rejection only emits on a later retry when c or d's msg1 lands while a has no pending outbound. On one 2026-05-14 run the inbound rejection appeared 63ms after the test had given up. The race window is small but real. Polling the log instead of one-shot reading absorbs the millisecond-to-second variance without slowing the success path (the helper returns as soon as the pattern appears). |
||
|
Johnathan Corgan
|
37c2973e2f |
Test infrastructure overhaul: gateway robustness + full CI coverage
Single combined commit covering five interlocking pieces of test and CI work that landed during the v0.3.0-prep cycle. ## fips-gateway robustness - src/bin/fips-gateway.rs DNS upstream probe converted from a 3-second hard-fail to a bounded retry loop (5 attempts × 1s timeout, 1s sleep between attempts; ~10s worst case). Covers the cold-boot race where the daemon's TUN is up but the DNS responder at [::1]:5354 is still binding. Each failed attempt logs at INFO. In production the binary's retry is the live recovery mechanism; with retry it recovers silently instead of relying on Restart=on-failure (~5s blip + spurious ERROR per cycle). - packaging/debian/fips-gateway.service `ExecStartPre` now waits up to 30 seconds for the daemon's `fips0` TUN to appear before exec'ing the gateway binary. Eliminates the cold-boot race where the gateway exits with `fips0 interface not found` and recovers via `Restart=on-failure`, producing a 5-second blip and a spurious error log per restart cycle. - testing/docker/entrypoint.sh gateway-mode waits up to 30s for the daemon's DNS responder to bind [::1]:5354 (probes once per second with `dig @::1 -p 5354 ... test.fips`) before exec'ing fips-gateway. Belt-and-suspenders with the binary's own retry: in CI we want deterministic startup ordering. On timeout, fall through so the binary's probe reports the definitive error. ## Test infrastructure DNS bind migration to ::1 After session 359's daemon DNS-bind default flipped from `127.0.0.1` to `::1` (the production fix for ISSUE-2026-0002), the static-test infrastructure was carrying a stale workaround that overrode the default back to IPv4 loopback. The fips-gateway integration test exposed the divergence: the gateway probes its DNS upstream at `[::1]:5354` (production default) while the daemon was binding `127.0.0.1:5354` from the template override — IPv6-explicit sockets do not accept v4-mapped traffic, so the upstream probe exhausted retries and the gateway exited. - Drop the explicit `bind_addr: "127.0.0.1"` line from every test config that emits it: testing/static/configs/node.template.yaml, testing/chaos/configs/node.template.yaml, the sidecar heredoc in testing/docker/entrypoint.sh, testing/acl-allowlist/generate-configs.sh (six per-node blocks), testing/nat/scripts/generate-configs.sh, and the four tor templates under testing/tor/. Daemon picks up its production `::1` default. - Flip the dnsmasq forwarder for `.fips` in testing/docker/Dockerfile from `127.0.0.1#5354` to `::1#5354` so dnsmasq on the shared test image continues to reach the daemon. Template and Dockerfile must move together since most static suites resolve `<npub>.fips` via the test-image dnsmasq. ## rekey-accept-off integration variant + UDP unit test - New `rekey-accept-off` topology and docker-compose profile under testing/static/. 2-node variant where node-b runs with `udp.accept_connections: false`. Pins the regression class that ISSUE-2026-0004 fixed (cross-connection winner's rekey msg1 was being filtered by the accept_connections gate, breaking rekey). - testing/static/scripts/rekey-test.sh accepts REKEY_TOPOLOGY and REKEY_ACCEPT_OFF_NODES env vars; its inject-config subcommand applies the per-node `udp.accept_connections: false` edit, and the test asserts no sustained "Dual rekey initiation" log lines. - New UDP variant of `should_admit_msg1` admit-rekey unit test in src/node/tests/handshake.rs. ## ci-local.sh full integration coverage - New runner functions and dispatcher entries for `acl-allowlist`, `nat-cone` / `nat-symmetric` / `nat-lan`, `rekey-accept-off`, `dns-resolver`, `deb-install`. Each integrates with the existing summary tracking via `record`. - New `--with-tor` flag (off by default) gates `tor-socks5-outbound` and `tor-directory-mode` runners. Tor stays opt-in because both harnesses depend on the live Tor network and would introduce a flake source unrelated to the FIPS code. - New suite arrays (`ACL_SUITES`, `NAT_SUITES`, `DNS_RESOLVER_SUITES`, `DEB_INSTALL_SUITES`, `TOR_SUITES`) drive both the default sweep and `--list` output. - `run_suite` extended to accept the new suite names for `--only` invocations. ## GitHub CI matrix expansions - `gateway` matrix entry runs testing/static/scripts/gateway-test.sh against the existing docker-compose `gateway` profile. - `rekey-accept-off` matrix entry exercises the new topology with REKEY_ACCEPT_OFF_NODES=b. - `deb-install` matrix (debian12 + ubuntu24 + ubuntu26) runs testing/deb-install/test.sh with privileged systemd containers. ~5-7 min cold cache, ~2 min warm per distro. Self-contained: builds its own .deb in a Debian 12 cargo-deb builder image; does not depend on the build job's pre-built artifact. - `dns-resolver` matrix entry runs the full 13-scenario harness (per-distro systemd resolver-backend tests + real-fips end-to-end scenarios) in a single job. Pins the production DNS bind path that ISSUE-2026-0002 lived in. ~7-12 min warm, ~12-15 min cold. Verified locally: full `bash testing/ci-local.sh` sweep passes, including 5/5 deb-install distros and all 13 dns-resolver scenarios. Tor-inclusive sweep (`--with-tor`) verified in a follow-up run. |
||
 Alex XieandJohnathan Corgan
|
745b523ac6 |
Add peer ACL enforcement with reloadable allow/deny files (#50)
Implement TCP Wrappers-style peer access control using /etc/fips/peers.allow and /etc/fips/peers.deny files. Evaluation order: allow overrides deny, default permit when no files exist. Three enforcement points: outbound connect (before dialing), inbound handshake (msg1 receipt, after restart/rekey classification), and outbound handshake completion (msg2, before peer promotion). Files support npub, hex pubkey, host alias, and ALL wildcard entries with automatic mtime-based reload. Adds fipsctl acl show query, 954-line acl module with unit tests, and a 6-node Docker integration harness (testing/acl-allowlist/) exercising insider, outsider, and allowed-remote scenarios. CI matrix entry included. Closes #50 Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com> |