mirror of
https://github.com/jmcorgan/fips.git
synced 2026-07-30 19:46:15 +00:00
2fdc831ce375bee9b75bc2b5492d540ba7120669
44
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Johnathan Corgan
|
2fdc831ce3 |
Claim the NAT lab's two bridges per run and derive every address from them
The NAT lab was the last suite pinning fixed IPv4 subnets, so two overlapping runs collided on the wan and shared-lan bridges. Each run now claims a free /24 for each bridge, scanning candidates and advancing on an overlap while still failing fast on any other network-create error. The claim exports NAT_WAN_PREFIX and NAT_LAN_PREFIX, and every routable address in the compose file and the suite scripts derives from them, with defaults that render exactly what the lab used before. The router-side LANs are deliberately left pinned: they live inside per-container network namespaces, never become docker networks, and cannot collide. An external-network overlay lets the suites attach to the networks the run already claimed instead of creating their own. Two of the three suite scripts had no overlay hook, so they would have requested the claimed range a second time; both now have one. Host veth names are scoped by a short token rather than the full run id, which overruns the fifteen-character interface-name limit at the default run-id length, and the cleanup reaper's pattern is widened to match the new shape. Networks are released inline on every exit path rather than in a trap, since a trap written inside a shell function replaces the script-level handler and would disable the whole run's teardown. |
||
|
Johnathan Corgan
|
fbff52d85e |
Give the NAT lab per-run generated configs
The NAT lab wrote its generated node configs to one shared directory, unlike the static and firewall labs which already scope theirs by run. The directory is bind-mounted by compose and read back by the suite scripts after the containers are up, so two overlapping runs let the second run's generator overwrite the npubs the first is about to ping. Scope the directory with FIPS_CI_NAME_SUFFIX at all three places that have to agree: the generator's output path, the ten compose bind-mounts, and the CONFIG_DIR the suite scripts read back from. An unset suffix renders the plain path the lab has always used, so a bare invocation and the GitHub matrix are unaffected. The run teardown removes the per-run directory alongside the static and firewall ones, and the gitignore is widened to cover the suffixed form. Rendering the compose file with every profile and no suffix set reproduces today's exact ten paths; setting a suffix moves all ten. This lands on its own, ahead of the network and address work, so that a failure of the two-overlapping-runs acceptance test can be bisected between the config fix and the address conversion. |
||
|
Johnathan Corgan
|
93800a503e |
Retire the bloom-storm chaos scenario from both CI runners
The scenario guarded a real regression: a mid-chain tree update that changed neither root nor depth leaking downstream as a sustained bloom announce storm. But it was never once run against the regressed binary, and its per-node ceiling was inferred from a post-mortem harness that no longer exists in the tree. On the only surviving regressed measurement the tail node's rate scales to roughly 7 sends per 30s, well under the scenario's ceiling of 40, so it was never established that the assertion could fire on its own bug class. The ceiling is also uniform per node, calibrated against the flap target that is legitimately busy rather than against the tail where the storm actually shows. This removes coverage rather than relocating it, unlike the two earlier retirements above it, and the comment in ci-local.sh records that gap explicitly. The scenario, its README, the link_swap sim primitive and the mesh-lab dispatch all stay on disk, so it remains runnable by hand. Parity holds at 24 legs a side. The parity guard was break-checked by re-adding the GitHub leg alone, which correctly reported the asymmetry. |
||
|
Johnathan Corgan
|
cdda660f10 |
Stop local CI writing the shared mutable test image tag
The run built per-run images and then retagged them to fips-test:latest as a compatibility bridge for the consumers that had never been migrated. The bridge was the defect: while it existed, two concurrent runs shared one binding, so a suite could start containers from the other run's binaries and the verdict was recorded against a commit whose code never ran. It fails silently by construction — the run is green either way, and nothing compares a running container's binary against the commit under test. Every consumer now reads the run's image, so the retag is deleted rather than kept. That is the point of deleting it: a consumer that was missed fails loudly instead of quietly resolving whichever run wrote the tag last. Two consequences handled here. The cleanup script ran ip(8) inside fips-test:latest to reap simulation interfaces, and nothing writes that tag any more, so it now takes the caller's image, then the run's, then any surviving test image — the last of which is what keeps an unscoped --reap working, since that path execs before the run identity is exported. And a guard checks statically that nothing names the shared tag, because on a host with a hand-built copy lying around a reintroduction would run green; it is break-checked against a reintroduced compose consumer and a reintroduced default. Both runners gate on it, as they do the other guards. |
||
|
Johnathan Corgan
|
49163befd5 |
Give each local CI run its own docker build context
Scoping the image tag was never sufficient on its own. Every build read one unscoped directory in the working tree, into which each run copies the binaries it just built, so two concurrent runs raced on the contents of the context as well as on the name of the result — and a run could produce a correctly-per-run-tagged image built from the other run's binaries. The run now copies the context's tracked files into its own directory, installs its binaries there, and builds from it, exporting the path so every other consumer follows. Deliberately not carried over: a previous run's binaries, since inheriting them is the failure this prevents. The path is absolute because compose resolves a relative build context against the compose file's own directory rather than the working directory, which was measured rather than assumed. The chaos entry script gated the whole simulation on a binary in the shared directory by literal path, so it moves in this same commit: left behind, it would have failed on a clean checkout and, worse on a host with leftovers, passed while reading a binary that was not the one under test. Teardown removes the directory on red runs as well as green, since it holds only reproducible content and is never the evidence of a failure. The worker's SIGKILL runs no trap, so the cleanup script also sweeps contexts left by a preempted run, and the ignore rule keeps a concurrent run from showing up as untracked working-tree noise. |
||
|
Johnathan Corgan
|
0e42c789be |
Let the firewall suite float its docker subnet
The compose pinned 172.32.0.0/24 with a per-container ipv4_address, so two concurrent runs asked docker for the same address space and the second failed with a pool-overlap error. Request no subnet and let docker assign one from the daemon pool. Peers address each other by the docker hostname the compose already sets (host-a, host-b) rather than by literal IP; docker's embedded DNS is per-network, so the same hostname in two runs resolves inside each run's own subnet. Nothing this suite asserts on moves as a result: its checks run over the fips0 overlay, whose addresses are derived from node npubs and are independent of docker addressing, and the packaged nftables ruleset matches on interface rather than on any address. The generated config directory moves under the run suffix for the same reason it did in the acl suite, and the run teardown gains the matching removal so a run no longer leaves its directory behind. Case (b) also wrote curl's output to a fixed path under /tmp. Two concurrent runs shared that one file, and either run's cleanup landing between the other's write and read left an empty read, failing the http_code check for a reason having nothing to do with the firewall. It uses mktemp now. As in the acl suite, the floating subnet removes one of the two obstacles to concurrent runs. The compose project name is still fixed; the local CI runner scopes it externally, a bare hand run does not, and the comment at the site says so rather than claiming the file is self-sufficient. |
||
|
Johnathan Corgan
|
9846e85705 |
Retire the admission-cap Docker suite; the inbound gate is proven in-process
The inbound max_peers early-gate in handle_msg1 (silent-drop a Msg1 from a net-new identity at saturation, send no Msg2, admit no peer) is unit-tested over a real UDP socket by handle_msg1_silent_drops_at_cap_for_new_peer in src/node/tests/unit.rs: it saturates a node, sends a Msg1 from a fresh identity, and polls the sender socket to assert no Msg2 comes back — the same wire-observable discriminator the Docker suite's tcpdump used — with a sibling test for the existing-peer bypass. That is a deterministic superset of the Docker packet-capture assertion. Drop it from both runners in lockstep (its *_SUITES array, run function, default-flow loop, --only dispatch arm, and the GitHub matrix leg with its steps) so the parity guard stays green, and record the retirement in the deliberately-not-run block with a pointer to the standalone runner. The admission-cap-test.sh script stays on disk and runs by hand. |
||
|
Johnathan Corgan
|
41ce64ba82 |
Retire the acl-allowlist Docker suite; the decision is proven in-process
The ACL admission decision is exhaustively unit-tested per npub over real loaded allow/deny files (src/node/acl.rs test module: allow-match-wins, allowlist-miss falls through, deny-only, deny-all, allow_all override, deny-after-allowlist-miss), and the inbound and outbound handshake- admission paths are covered in-process over the loopback transport (src/node/tests/acl.rs: inbound msg1 denial, outbound denial, reload). The Docker suite's only unique coverage was admission over a real UDP transport, which every other real-transport suite already exercises, and its operator-facing log assertion. Drop it from both runners in lockstep (its *_SUITES array, run function, default-flow call, --only dispatch arm, and the GitHub matrix leg with its steps) so the parity guard stays green, and record the retirement in the deliberately-not-run block with a pointer to the standalone runner. The testing/acl-allowlist/ suite stays on disk and runs by hand. |
||
|
Johnathan Corgan
|
7cbe1d3d4e |
Retire the smoke-10 chaos scenario; convergence is covered in-process
smoke-10 was a no-stressor 10-node tree-convergence sanity check (netem off, no ping). Its subject, spanning-tree convergence and root election, is now covered in-process, faster and deterministically, by the loopback spanning-tree harness (src/node/tests/spanning_tree.rs: ring, star, chain, 100-node and disconnected-component convergence) plus end-to-end datagram delivery (src/node/tests/forwarding.rs). Real-UDP convergence smoke still runs via static-mesh and the other scenarios' baseline assertions, so no Docker coverage is lost. Drop it from both runners in lockstep (the CHAOS_SUITES list and the GitHub chaos matrix) so the parity guard stays green, delete the scenario YAML, and update the chaos README. |
||
|
Johnathan Corgan
|
08a226fb63 |
Test cost-based parent selection and kernel-drop detection as unit tests
The cost-selection chaos scenarios (cost-reeval, cost-avoidance, cost-stability, depth-vs-cost, mixed-technology, bottleneck-parent) tested TreeState::evaluate_parent's decision logic through a Docker mesh that could not exercise it reliably: the tree roots at whichever node holds the smallest NodeAddr, MMP link costs take several measurement windows to settle, and the parent hold-down plus hysteresis timing all confound the outcome. A deterministic link-cost flap still produced zero periodic parent switches in a full run. Replace those six scenarios with deterministic unit tests in src/tree/tests.rs that drive evaluate_parent directly: cheaper-link selection at equal depth, switch-on-cost-change, hysteresis suppressing a marginal change while allowing a significant one, and the depth-versus-cost effective-depth tradeoff. Each is constructed so that breaking the cost or hysteresis logic makes it fail. The congestion kernel-drop signal (SO_RXQ_OVFL) cannot be provoked deterministically in Docker: a fresh daemon reader keeps up with container-speed traffic, so the socket receive queue never overflows (an unshaped run with a 4 KB buffer and heavy traffic recorded zero drops on every node). Extract the drop-detection edge -- read the cumulative counter, fire an event only on the transition into a new drop burst -- into TransportDropState::observe_drops and unit-test it directly. congestion-stress keeps its ECN and MMP congestion-signal assertions, which do need the real shaped bottleneck queue. Remove the retired scenarios from both CI runners and update the chaos README. |
||
|
Johnathan Corgan
|
be5deee814 |
Make the gateway integration suite safe for concurrent CI runs
gateway-lan pinned 172.20.1.0/24 and fd02::/64, so two concurrent local CI runs collided on "Pool overlaps" at network creation. The IPv4 subnet has no consumer -- the whole gateway LAN path is IPv6 -- so drop it and let docker auto-assign. The IPv6 side cannot float, because the LAN clients' resolv.conf pins the gateway's address as their nameserver and that must be a literal known before they start, so claim a free /64 per run and thread the prefix through the compose address pins, a generated resolv.conf, and the test via a single exported variable. The claim and the external network ship as a harness-only compose overlay applied by run_gateway; the base compose keeps a normal gateway-lan network, so the GitHub matrix and any standalone bring-up are unaffected. With no claim every address renders exactly as before. |
||
|
Johnathan Corgan
|
9b7a27f219 |
Claim the chaos network range instead of deriving it, ending the collision
Two concurrent ci-local runs could not both run chaos. Each child's subnet came from its position in the suite list, so both runs walked 10.30.0 through 10.30.12 and requested identical ranges; whichever reached the daemon first won and the other died with a pool overlap. A run index or hashed offset would only make that unlikely, and it is precisely the failure being removed. The simulator now claims its range: attempt-create on a candidate, advance on docker's own overlap error, fail loudly on anything else. Docker's address pool becomes the arbiter, so an overlap is impossible rather than improbable. The claim lives in the sim rather than in ci-local because only the process that creates the network can advance on conflict, and because it fixes the bare chaos.sh path too, which a ci-local-only fix would have left broken. The generated compose now declares the network external over the claimed one, and teardown releases the range from both paths, including the setup-failed path where a run that fell over after claiming still holds one. Ordering matters here and is not obvious: node IPs derive from the subnet during topology generation, and traffic shaping keys its filters on those addresses, so the claim happens before the topology exists. Claiming later would give a network on one range and filters on another, which does not fail at bring-up and instead leaves the shaping matching nothing. --subnet survives as an explicit pin for when a known range is wanted, and now fails loudly if that range is taken rather than silently overlapping. ci-local no longer passes it. Validated by running two instances of the same scenario deliberately concurrently: they claimed 10.30.1.0/24 and 10.30.0.0/24, both exited 0 with their assertions passing, and both released their networks. The negative control holds too: with a squatter on a pinned range the run aborts with a message naming the range rather than proceeding. |
||
|
Johnathan Corgan
|
55331a80b5 |
Gate on shell functions whose exit status is a log call's
A bash function returns its last command's status, so one ending in a log
call returns 0 whatever it did, and a caller written as `if func` or
`func || fail` has a gate that cannot fire. This tree found two in one
day: run_chaos ended in record, making every chaos row unconditionally
green since 2026-03-09, and build_fips_for_e2e ended in log, letting five
end-to-end legs test the previous commit's binary and report green. Both
were repaired one at a time. This is the rule that catches the next one.
What it enforces is a contract, not a bug hunt: a function whose status a
caller tests must end in an explicit return rather than leaving its
success value to whatever the last log call produced. That distinction is
deliberate and worth stating, because all three instances found in the
tree were benign — each failure path already returned early, so the
trailing echo reported a real success. Reading the function is the only
way to know that, and the next edit that puts an unguarded command before
the final log converts the benign shape into the defect with nothing to
notice. An explicit return costs a line and makes the class unreachable.
The three are given one here.
Scoped on the call sites rather than the definitions, the same way
check-log-strings.py scopes on what a grep reads rather than what its file
mentions: 315 functions scanned, 42 end in a log call, and only the ones
whose status something consumes are reported. A function that ends by
reporting and is called for its output is idiomatic and is left alone.
Without that scoping the finding list would be 42 long and would stop
being read.
Validated by construction rather than by a green run: injecting a copy of
build_fips_for_e2e's exact shape — unguarded docker cp, then a log as the
last statement, with a `|| { ...; return 1; }` caller — makes the checker
report it and exit 1.
Wired into both runners beside the parity and log-string checks.
|
||
|
Johnathan Corgan
|
2a6039995d |
Retire the ECN A/B test framing and record why suites are excluded
ecn-ab-test.sh was a -test.sh with no path to a failing result: it asserts nothing, applies no threshold, and nothing invokes it. It also could not have worked. It read a fixed sim-results/ecn-ab-on/ path while the runner has written timestamped directories since 2026-03-20, and not one ecn-ab result directory exists on disk, so it has found neither input for months and the "+10.2% recv throughput" figure the README carried is not reproducible from anything available. Rename it to ecn-ab-compare.sh, fix the path resolution to glob the timestamped directories, and stop swallowing a failed simulation with || true so a broken run cannot feed the comparison. Deliberately not adding the threshold assertion that would make it gateable. That needs a calibration corpus, and none exists precisely because the tool has never produced a kept result; a number invented now would assert a guess. The prerequisite is a calibration run set and a decision about what ECN is expected to deliver, which is a protocol question. Written in the script header rather than left implied. Add the exclusion block to ci-local.sh naming every suite and scenario neither runner runs, with the reason for each: interop, boringtun, iperf-test, this comparison tool, mesh-lab, and the four chaos scenarios outside CHAOS_SUITES. Until now "not in the suite list" was indistinguishable from "forgotten". |
||
|
Johnathan Corgan
|
1f149bda32 |
Run the convergence-gate unit tests in both CI runners
wait_until_connected decides whether every static suite proceeds or gives up, so a regression in it turns those suites' verdicts into noise. Its unit tests existed and were invoked by nothing. Wire them into ci-local.sh beside the parity and log-string checks, above the mode branches so --only and --test-only are gated too, and add the matching step to the ci-parity job in ci.yml. Putting them in one runner only would create exactly the drift check-ci-parity.sh exists to catch, and neither placement is visible to that checker since this is not a matrix suite. Verified by breaking what it guards rather than by observing green: setting wait_until_connected's default near-converged slack to 0 fails case4, records FAIL wait-converge, and exits the sweep 1. Restored after. They pass, which nothing had established before — 13 assertions, all green. They also take about 45 seconds rather than the "a few seconds" the header claimed, and nothing had contradicted that claim because no runner had ever invoked it. Header corrected with the measurement. The orphan sweep this called for is done and there are now no zero-reference *-test.sh files under testing/. It also settles a disagreement recorded between the audit and the task file: ecn-ab-test.sh is invoked by nothing, its one reference being a README description rather than a driver, so the audit was right and the earlier sweep wrong to call it a documented manual tool. interop-test.sh does have a driver in interop-stress.sh and iperf-test.sh one in iperf-compare-refs.sh, so those two were correctly classified. |
||
|
Johnathan Corgan
|
3d0a388511 |
Fail the ping test on an unknown topology profile
Every section of ping-test.sh is guarded by the profile name, so an unrecognised profile fell through all of them and the script exited 0 having run no assertion. A typo in the caller's argument produced a green run that tested nothing. Give it an else branch that names the valid profiles and exits 2. No existing leg changes behaviour: the hosted runner gates the ping step on matrix.type == 'static', which only static-mesh and static-chain carry, and ci-local passes only the two names in STATIC_SUITES. Worth noting while confirming that, though: ping-test.sh accepts a mesh-public profile that neither runner ever passes. Also record why the convergence waits are `|| true` — they are settling delays rather than assertions, and the directed-pair pings are what decides the run — and add admission-cap to the suite list in the ci-local header, which ran it without listing it. |
||
|
Johnathan Corgan
|
d21f818659 |
Check that log strings tests match on are still ones the daemon emits
A test that greps the daemon's log for a message the daemon stopped emitting does not fail. It stops observing, and an assertion built on it — especially one expecting a count of zero — then passes because nothing can be seen rather than because nothing happened. Several findings have come from that one class, so it is now checked mechanically. testing/check-log-strings.py extracts the strings test code matches against daemon log text and requires each to exist in src/. It reads four shapes: python `"..." in line` tests, the first argument of the shell log helpers, bash associative-array pattern tables, and greps whose input is daemon log output. Patterns carry regex syntax, so each is reduced to the longest literal run of every alternation branch, with escaping resolved in the same pass — `\.` is a literal dot and `.` is a wildcard, and conflating them would let a pattern match text that is not there. Scoping is by what a grep READS, not by what its file mentions. A suite greps its own analyzer output, fipsctl JSON, Tor's log and ping output, and none of those have to correspond to a string in src/; scoping on the file produced 34 false positives. Strings that legitimately do not come from src/ are named in ALLOWED with a reason, so the exceptions are reviewable rather than invisible. It found six dead strings, three of which were not previously known: a second copy of "Excessive decrypt failures" in the interop pattern table, a dead "Handshake error" branch inside an alternation whose other branches still matched, and "bootstrap failed" in the NAT suite. All six are corrected to what the daemon emits, or dropped where the surviving branch already covers the case. Verified by breaking it two ways and confirming it reports. Also here, since it is the same failure shape one layer down: the shared log library returned an empty string for a container whose logs it could not read, so analysing a missing container reported no panics, no errors and no sessions, and exited 0 — indistinguishable from a clean run. It now reports which containers could not be read and raises, matching the fix already landed in the chaos copy. |
||
|
Johnathan Corgan
|
6c52b0e01e |
Let the static test network float so concurrent CI runs cannot collide
Two local CI runs on one host both asked docker for 172.20.0.0/24 and the second lost its whole static family to "Pool overlaps". Docker honours a fixed subnet request verbatim, so the only robust fix is to stop making one: fips-net now requests no subnet and docker assigns from its own pool, which cannot hand the same range to two runs. That means node addresses are not known before `up`, so peers address each other by container hostname instead. The generator emits node-<id>, or the topology's docker_host where the compose hostname differs — only the gateway profile, whose services are gw-*. External peers keep the address the topology gives them, since it is not ours to assign. The resolv.conf mount stays: dnsmasq is what forwards these names to docker's resolver and .fips to the daemon, so removing it would take out every .fips assertion. generated-configs is now per-run as well. A shared directory let two runs overwrite each other's node configs, which the subnet collision had been hiding by killing runs before that window opened. The generator, the compose bind mounts and env_file, the six scripts that read it, and teardown all follow FIPS_CI_NAME_SUFFIX; unset, every path renders as before. Teardown keeps the directory after a failed run, where it is the evidence of what the failing nodes were configured with. Three things this exposed that were wrong independently: admission-cap built its tcpdump patterns from the topology file's docker_ip literals. Floating the subnet makes those match nothing, which would have left its expect-zero "no Msg2 leaked" assertion passing because it could no longer see anything at all. It now reads addresses from the running containers. Restarting the denied peers together also made them swap addresses, so each peer's counts were really the pair's total; they are restarted one at a time now, and a check fails the suite outright if two denied peers ever share an address, because per-peer attribution is impossible once they do. Attribute lookups in the generator used a fixed ten-line window and read the next node's fields when a node omitted an attribute. An external node followed by an internal one was classified as internal, which under hostname peering would emit a name that resolves nowhere. Lookups are bounded to the node's own block; generated output is byte-identical for all eight topologies. The rekey outbound-only variant used to rewrite peer addresses to hostnames to set up its scenario. The generator now does that everywhere, so the rewrite matched nothing and was silently doing no work. It asserts the premise instead, and fails if a numeric address ever reappears. Verified by running three instances of this compose at once — tcp-chain plus two independent meshes — which drew 10.128.2/3/4.0/24 with no overlap while both meshes passed ping-test 20/20 over the real .fips path. tcp-chain is run by neither CI runner, so it was checked by hand: chain peer counts 1/2/1 and multi-hop .fips reachable both directions over TCP. gateway-lan still pins its own IPv4 and fd02:: ranges and is unchanged here, so the gateway profile is not yet concurrency-safe. |
||
|
Johnathan Corgan
|
56f00058d4 |
Run the CI parity guard in both runners
The guard that keeps "local green" and "GitHub green" meaning the same thing was invoked by nothing. Its only entry point was an exec behind an explicit --check-parity flag, which by exec-ing could not be combined with an actual run, and no workflow step called it at all. A guard nobody runs is a guard that does not exist, which is how the mixed-profile divergence on next survived unnoticed. It now runs as the first stage of every local run and as its own job on GitHub. Locally it reports through the same result-recording path as every other stage, so a divergence sets the run's exit status instead of scrolling past. It is deliberately placed above the mode branches, so a divergence also fails --only, --test-only and --build-only runs: whichever subset was asked for, the claim that a local result means what a GitHub result means is what has broken. The GitHub side installs pyyaml explicitly rather than assuming the runner image carries it. --check-parity keeps working exactly as before. Three comment blocks that described the old folded comparison are corrected here, in the workflow, the local runner and the testing README; they were spread across three files and only two mention the guard by name, so the third is best found by searching for what it claims rather than for the script. |
||
|
Johnathan Corgan
|
4f55a281ac |
Let a failed chaos scenario fail the run
The simulation caught every exception a scenario raised, logged it, and exited 0, so a scenario that died during setup was recorded as a pass. Local CI discarded the exit code in any case: run_chaos ends in a call whose own last statement is an echo, so it returned 0 whatever the scenario did, and the parallel launcher's wait therefore always succeeded and always recorded a pass. Between them no chaos failure of any kind could turn a local run red, and none could since the script was written. Carry the abort out of the run and give it its own exit code, ordered ahead of the two content codes so a run that never produced a mesh cannot report that mesh's panic and assertion counts. Return the scenario's status from run_chaos so the launcher sees it. The documented exit codes were stale before this change and are rewritten in full, including that the argument parser rejects a malformed command line with the same code that reports panics. Expect scenarios that have been reporting green to start reporting red. In the last recorded local run twelve of the thirteen scenarios never got past starting their containers, and all thirteen were counted as passes; those that now execute have not run for months, and their assertions have never been evaluated against a mesh of their own. |
||
|
Johnathan Corgan
|
2ef36c0071 |
Scope chaos host interface names per scenario and reap orphaned pairs
Ethernet edges get a veth pair created in the host namespace and then moved into the containers, named from the node ids alone. ethernet-mesh and ethernet-only both connect n01 to n04 and always run together, so one tore down the other's live link. Hash the scenario suffix to four hex characters and insert those after the vh prefix; 15 characters is far too few to carry the suffix itself. The names are now run-specific and never regenerated, so the delete-before-create no longer reclaims an interface an earlier run abandoned. ci-cleanup.sh takes over, deriving the names from the suffixes a run's teardown hands it. It is the first thing that script removes that is neither a docker object nor labelled, so an unscoped reap can sever a running bare simulation's links; testing/README.md and the --reap help say so. |
||
|
Johnathan Corgan
|
5be7c6d0cb |
Give each chaos scenario its own container names and config directory
Chaos scenarios run four at a time under local CI, but every one of them claimed the container names fips-node-nNN and wrote its generated configs and compose file to the same generated-configs/sim directory. Container names are global in Docker and are not scoped by the compose project, so concurrent scenarios collided on both, and a scenario could start containers from a compose file another had overwritten. Thread the existing FIPS_CI_NAME_SUFFIX into the simulation. run_chaos narrows the run-wide suffix to the scenario, and the sim reads it once when the topology is built, applying it to the container names and to the config directory basename. The compose template renders the name through the topology accessor instead of duplicating the literal, so one expression produces every chaos container name. The suffix is empty when the variable is unset, so a bare chaos.sh run and the hosted CI jobs render byte-identical names and paths. Verified by rendering every scenario's compose file before and after with the variable unset and diffing. |
||
|
Johnathan Corgan
|
e4a854f6b0 |
Stop concurrent local CI runs from clobbering each other's containers
Two runs on one host destroyed each other's containers, producing mid-test "No such container" failures that look like real defects. The automated builder runs a full local CI on the same box every few minutes, so the machine is contended almost always and this has red-ed both a hand run and an automated gate. Two independent causes are fixed here. Container names were hardcoded, and docker names are global rather than scoped by compose project, so two runs collided on the same name; every name now takes an optional suffix that the harness sets from the run id. And the cleanup sweep matched a label shared by every run, so one run's teardown force-removed another's containers; resources now also carry a per-run label and the sweep can be narrowed to it. A third hazard turned up that was not in the original report: the sidecar suite passes explicit compose project names, which override the run-scoped project and put it outside the shared prefix entirely. Its project names, network, and derived container references are now scoped too. Both are default-off. With the suffix unset, names render exactly as they do today and a bare compose invocation is unchanged, which is what keeps the hosted CI and the documentation correct. A cleanup run with no run id still reaps everything, which is what a manual "clear the box" wants. The literal-name sweep was not sufficient: six scripts build container names dynamically from node labels, and two suites create their own containers outside compose. Those are handled at their construction sites. Verified: syntax check on all modified scripts; compose validation on every modified file with the suffix both set and unset; and a synthetic two-run reproduction that shows the old cleanup destroying a bystander run and the new one leaving it alone. Known gap: subnets are still hardcoded, so two concurrent full runs will still collide on address-pool overlap. That fix reaches into topology configs, chaos scenarios, diagrams and production source, so it is left for its own change rather than half-done here. |
||
|
Johnathan Corgan
|
965de26239 |
testing: make ci-local.sh preemption-safe with per-run docker isolation
A CI worker may preempt an in-flight ci-local.sh run (SIGTERM, then SIGKILL after a grace period) to restart on a newer commit. For that kill to be safe, the script must clean up after itself and never let a dying run collide with its restart. It previously had no signal handling, shared the default compose project name across runs, and tore down each suite only at the suite end. - Derive a per-run id (honoring FIPS_CI_RUN_ID, else short-sha+random) and namespace every docker resource to it: a fipsci_<run>_<suite> compose project per suite and per parallel chaos child, and per-run image tags (fips-test:<run>, fips-test-app:<run>) retagged to :latest only after both builds succeed so :latest never points at a half-built image. - Install a bounded, idempotent teardown trap on SIGTERM/SIGINT (+ EXIT): reap parallel chaos children, then force-remove this run's docker resources via the new ci-cleanup.sh, wrapped in timeout so a stuck down cannot wedge it. - Exit 143 (SIGTERM) / 130 (SIGINT), distinct from 0 (pass) / 1 (failed), so a preempting worker tells a cancelled run from a real failure. - Add ci-cleanup.sh (also ci-local.sh --reap): force-removes leftover CI resources by the com.corganlabs.fips-ci=1 label and the fipsci_ project prefix, robust to however a prior run died. - Label every per-suite docker resource so the label sweep reaps it after a SIGKILL regardless of network name: direct docker run/network resources, the sidecar compose services, and every per-suite compose network (acl-allowlist, boringtun, firewall, nat, static, both tor suites, and the chaos generator template). Parametrize the static/sidecar compose image refs so the per-run tags are honored. - Give each parallel chaos child a unique /24 from 10.30.x (a new --subnet override on the sim CLI, assigned per-child in ci-local.sh) so parallel children never collide on a shared docker subnet, and a chaos net can never span a fixed-subnet suite (sidecar/static in 172.20.x). 10.30.x sits outside docker's default-address-pool range, so an auto-assigned net cannot land on it either; node IPs derive from the subnet, so no scenario config changes. |
||
|
Johnathan Corgan
|
4e3890a780 |
ci: cover Debian 13 and Ubuntu 22.04 in the deb-install matrix
The GitHub deb-install matrix ran debian12/ubuntu24/ubuntu26, but the local harness (testing/deb-install/test.sh) runs five distros. Add the missing debian13 (trixie) and ubuntu22 legs so the cloud gate covers the same distro set as local CI. Each new leg invokes the existing test.sh scenario, so per-distro behavior is identical to the local run. Update the granularity-only parity notes in ci.yml, ci-local.sh, and check-ci-parity.sh to list the full distro set. |
||
|
Johnathan Corgan
|
dd4074249c |
Merge branch 'maint'
# Conflicts: # CHANGELOG.md |
||
|
Johnathan Corgan
|
c7218d8486 |
ci: align local/GitHub integration coverage and pin the toolchain source
Bring the local runner (testing/ci-local.sh) and GitHub CI into agreement on two axes. Suite coverage: the admission-cap integration suite ran only locally, so a regression in it could never turn the GitHub gate red. Add an admission-cap leg to the integration matrix, add testing/check-ci-parity.sh (wired as 'ci-local.sh --check-parity') to diff the two suite sets and fail on unexpected drift, and document the deliberate local-only (live-Tor) and granularity-only differences in a comment block atop both runners. Toolchain selection: every CI and packaging job installed its toolchain with dtolnay/rust-toolchain@stable, but the rust-toolchain.toml channel pin overrode it for all compilation, wasting an install and printing a misleading rustc version. Switch the stable call sites to actions-rust-lang/setup-rust-toolchain, which reads rust-toolchain.toml as the single source of truth. Keep the explicit cache steps (cache: false on the new action), set rustflags empty so the action does not impose a global -D warnings the previous setup never applied, fold the macOS cross-compile target into the action input, and leave the OpenWrt nightly Tier-3 leg on dtolnay/rust-toolchain@nightly. |
||
|
Johnathan Corgan
|
d9a4a7807c |
node: make the shared context the sole store of immutable state
Remove the duplicated immutable fields (config, identity, startup_epoch, started_at, is_leaf_only, max_connections/peers/links) from the Node struct so the Arc<NodeContext> bundle is the single source of truth. Previously Node owned these fields and a parallel context copy, kept in lockstep by rebuild_context() at every mutation site — pure overhead that existed only because of the duplication. - Replace rebuild_context() with replace_context(): a clone-edit-swap of the whole Arc. The per-instance context stays immutable; mutation swaps the Arc. This is the sole runtime mutation path (constructors, leaf_only, update_peers). - Add Copy-returning accessors startup_epoch() and max_connections()/ max_peers()/max_links(); migrate the remaining direct field readers onto the accessors. node_addr()/npub()/Debug now read identity/is_leaf_only from the context. - update_peers reads the pre-update peer set from the live context Arc before building a fresh Config + context and swapping — preserving the read-before-write ordering its mutation-window test depends on. - Remove the test-only set_max_* setters; tests set the limits on Config at construction instead (new make_node_with_max_peers/links helpers). - Add a ci-local guard that fails if the Node struct re-declares a bundled field, so the single-store invariant can't silently regress. cargo test --lib 1291/0; clippy -D warnings and release build clean. |
||
|
Johnathan Corgan
|
d575c1f986 |
testing: add admission-cap integration suite for inbound silent-drop gate
New integration scenario verifying the early-gate silent-drop behavior
of the inbound max_peers admission check at sustained scale, using the
existing 5-node mesh topology with one node's node.limits.max_peers
lowered to 1. This forces 2 of the cap'd node's 3 configured peers
into a sustained denied state, and asserts via tcpdump that no Msg2
responses go back to those denied peers across a 60s capture window.
A background load-driver restarts the denied peer containers every 15s
to reset their auto-reconnect exponential backoff (5s base / 300s cap),
producing fresh Msg1 bursts each cycle. Without this loop the gate
fires ~3-4 times per denied peer in a 60s window; with restarts the
observed rate is 15 per denied peer (~30 total firings), high enough
that any Msg2 leakage would be caught with strong statistical
confidence.
Local run on this branch: cap'd node-c converged to peer_count=1 with
node-b admitted; nodes d and e sustained-retried as denied; tcpdump
captured 30 inbound Msg1 (len 84) packets from the denied pair and 0
outbound Msg2 (len 104) packets, with final peer_count unchanged.
Files:
testing/static/scripts/admission-cap-test.sh — new test script with
inject-config subcommand (sets node.limits.max_peers) and a
3-phase test driver (converge, capture-with-load, per-peer assert)
testing/ci-local.sh — register admission-cap as a new suite category
(ADMISSION_SUITES), wire run_admission_cap function, add to
run_suite dispatch, list_suites, and the default integration sweep
Together with the existing unit-level coverage in src/node/tests/unit.rs
(handle_msg1_silent_drops_at_cap_for_new_peer with mock-transport Msg2
discriminator, and handle_msg1_admits_existing_peer_at_cap as the
bypass regression guard), the gate's silent-drop behavior is now
verified both at single-firing wire-observable resolution and at
sustained multi-firing cross-process scale.
|
||
|
Johnathan Corgan
|
b3a1fb464f |
testing: add bloom-storm chaos scenario
Six-node depth-4 mesh with an induced upstream parent flap. Asserts a
trailing-window ceiling on per-node `stats.bloom.sent` and a sanity
floor on parent-switch count over a ~3-4 min observation window.
Guards against the regression class where a spanning-tree update that
changes only an internal path edge (no root or depth delta) fails to
be properly contained and instead propagates to leaves as a sustained
bloom-traffic oscillation, visible only at fleet scale and only after
several minutes of uptime.
Adds a new chaos primitive (`link_swap`) for deterministic asymmetric
link-cost flapping and a post-run assertion framework with two
checks:
- `bloom_send_rate.max_per_node`: trailing-window ceiling on the
`show_bloom` stats counter delta. Calibrated against the
post-mortem reproduction harness data (per-variant counter table
against pre-fix vs post-fix binaries).
- `min_parent_switches.min_total`: sanity guard against a
misconfigured harness where the flap inducer fires but the
topology never produces a real parent-switch event (e.g., wrong
root election from a different seed). Without this, the
bloom-rate assertion would trivially pass on any binary
including a regressed one.
The runner exits 3 on assertion failure (alongside 0 success and 2
panic-detected). Threshold derivation is documented in the scenario
README; the seed pin is also documented there since smallest-NodeAddr
root election is sensitive to the pubkey hash ordering.
Wired into ci-local.sh's chaos pool and the GitHub CI chaos matrix.
|
||
|
Johnathan Corgan
|
43639fecb9 |
Add stun-faults integration suite covering STUN failure/recovery
Cover the previously untested STUN client behavior under server
unreachable, response timeout, and packet loss. The 3 NAT scenarios
test happy paths only; if the STUN client mishandled a fault (panic,
hang, missing log signal), it would silently degrade NAT traversal
without surfacing in CI.
testing/nat/scripts/stun-faults-test.sh (new, 244 lines):
Phase 1 (drop, ~12s): tc prio + netem loss 100% band + u32 filter on
dst 172.31.10.40 udp 3478. Falls back to iptables -j DROP if netem
isn't available. Asserts daemon process alive, no panic, log line
matching stun.*(timed?out|fail|fallback|unreachable|no address)
within the phase window.
Phase 2 (delay then clear, ~17s): tc qdisc add dev eth0 root netem
delay 5000ms for 7s, then deleted. 10s settle. Asserts process alive,
no panic, AND "STUN observation succeeded" log line after clear
(recovery proof).
Phase 3 (kill, ~12s): docker stop fips-nat-stun. Asserts process
alive, no panic, fault evidence in logs.
testing/nat/docker-compose.yml: stun-faults profile adds two
services. stun-fault-node is fips-test:latest on shared-lan at
172.31.10.50. stun-fault-shim is fips-test:latest sharing the
daemon's network namespace via network_mode: service:stun-fault-
node, with cap_add NET_ADMIN, NET_RAW; entrypoint sleep infinity so
the script can docker exec into it. Reuses existing stun
(172.31.10.40:3478) and relay (172.31.10.30:7777) services.
testing/nat/scripts/generate-configs.sh: 3-hunk update so the
generator accepts the new scenario and points its peer config at the
existing relay/STUN. The peer is configured for connect_peer() so
the daemon retries traversal on a loop, repeatedly invoking
observe_traversal_addresses() — which is the fault-injection target.
testing/ci-local.sh: STUN_FAULTS_SUITES=(stun-faults) array,
run_stun_faults runner, list/integration-loop/--only-dispatch hooks.
.github/workflows/ci.yml: matrix row {suite: stun-faults, type:
stun-faults} + 3 steps gated on matrix.type == 'stun-faults' between
nostr-publish-consume and any chaos suite. Reuses fips-linux
artifact + fips-test:latest image.
Approach: script-driven via docker exec stun-fault-shim. Sharing
network namespace means tc rules on the shim's eth0 affect daemon
egress. No timing logic in the shim itself.
|
||
|
Johnathan Corgan
|
33a2063672 |
Add firewall integration suite covering fips0 default-deny baseline
End-to-end exercise the v0.3.0 nftables firewall baseline so the
security claim — "services on fips0 are not exposed by default" — is
validated in CI rather than by operator opt-in. The packaging postinst
state is pinned by the deb-install matrix; this suite pins the actual
ruleset behavior.
testing/firewall/ — new directory mirroring the acl-allowlist
precedent (kept in its own directory, not extending testing/static):
docker-compose.yml (52 lines): two FIPS containers on bridge
172.32.0.0/24, peered over UDP/2121. node-b mounts
packaging/common/fips.nft RO at /etc/fips/fips.nft and a generated
drop-in services.nft at /etc/fips/fips.d/.
test.sh (247 lines): four-case asserter
(a) curl from node-a to node-b:8000 → DROP (port not allowlisted;
terminal counter drop fires)
(b) curl from node-b to node-a:8000 → 200 OK (reply traverses
node-b's ct state established,related accept)
(c) ping6 a→b → success (icmpv6 echo-request accept)
(d) nc -z a→b:22 → success (drop-in tcp dport 22 accept honored
via include "/etc/fips/fips.d/*.nft")
Plus drop-counter check after case (a) confirms the dropped
connection actually hit the chain's terminal counter drop.
generate-configs.sh (111 lines): mirrors acl-allowlist generator,
produces the drop-in services.nft + fips configs.
README.md (111 lines): how to run, expected output, design notes.
.gitignore: ignores generated-configs/.
testing/ci-local.sh: FIREWALL_SUITES=(firewall) array + run_firewall
runner; dispatch in run_integration and run_suite mirroring
run_acl_allowlist.
.github/workflows/ci.yml: matrix row {suite: firewall, type:
firewall} + 3 steps gated on matrix.type == 'firewall' between
acl-allowlist and gateway. Reuses fips-linux artifact + fips-test:
latest image.
Activation note: the unified test image does not run systemd, so
test.sh invokes the fips-firewall.service ExecStart
(/usr/sbin/nft -f /etc/fips/fips.nft) directly. The systemd-unit
enablement path is covered by the deb-install matrix; this suite
exercises what the unit configures, not how it gets started.
|
||
|
Johnathan Corgan
|
5611e976ad |
Add nostr-publish-consume integration suite
Cover the previously untested overlay advert publish/relay/consume
round-trip. The bilateral publish/subscribe path was a v0.3.0 release
gap: malformed adverts could panic consumers, broken signatures could
go undetected, and reverse-direction subscription was unverified.
Adds testing/nat/scripts/nostr-relay-test.sh (290 lines):
Phase 1+2 (combined): wait_for_peers on both nodes; pass on
bidirectional advert publish/subscribe round-trip + dial completed;
ping6 both directions confirms TUN-level reachability.
Phase 3 (malformed advert resilience): stdlib-only Python WebSocket
client publishes a syntactically valid Schnorr-signed Kind-37195
event whose `content` is gibberish (cannot deserialize as
OverlayAdvert). The relay enforces BIP-340 signature validity, so the
event reaches the consumers (rather than being dropped at the relay)
— a trivially-junk content payload is the right adversarial input.
Required ~80 lines of stdlib-only secp256k1 + BIP-340 in the script
(no new container deps). Asserts pidof fips on both nodes after the
publish, scans logs for panic markers, re-pings to prove the existing
peer link survives.
testing/nat/docker-compose.yml: new profile nostr-publish-consume
with two daemon services (nostr-pub-a 172.31.10.20, nostr-pub-b
172.31.10.21) on shared-lan, reusing the existing strfry relay
(172.31.10.30:7777) and STUN service (172.31.10.40:3478).
testing/nat/scripts/generate-configs.sh: 2-line allowlist update so
the new scenario flows through the existing config generator (rather
than forking a parallel one). Generated node-{a,b}.yaml + npubs.env
smoke-tested cleanly.
testing/ci-local.sh: NOSTR_RELAY_SUITES=(nostr-publish-consume)
array, run_nostr_publish_consume runner, dispatch in run_integration
and run_suite. Mirrors existing run_nat shape.
.github/workflows/ci.yml: one matrix row + 3 steps in the integration
job, gated on matrix.type == 'nostr-publish-consume'. Consumes the
same fips-linux artifact and fips-test:latest image as the existing
NAT suites.
Tor/TCP transport variants kept out of v0.3.0 scope; the structure
leaves room for nostr-publish-consume-tcp/-tor siblings later without
disturbing this baseline.
|
||
|
Johnathan Corgan
|
d822ee8b3c |
Validate packaging/common/fips.nft syntax in CI build phase
Add nft -c -f packaging/common/fips.nft syntax-check to both testing/ci-local.sh and .github/workflows/ci.yml so a regression in the 128-line firewall ruleset surfaces in the build gate rather than when an operator activates fips-firewall.service. testing/ci-local.sh: first step inside run_build(), before cargo build --release. Uses command -v nft for prereq detection mirroring the existing cargo-nextest pattern; records as nft-syntax in RESULTS. Operator-facing message points at apt install nftables when nft is absent. .github/workflows/ci.yml: nftables added to the build job's existing Linux apt-install step; new Validate fips.nft syntax (Linux only) step gated on runner.os == 'Linux' (skips macOS/Windows matrix slots, runs on ubuntu-latest and ubuntu-24.04-arm). Note: nft -c -f requires netlink cache initialization on modern nftables even in check mode, so both invocations use sudo (safe in CI's passwordless sudo, and operator's typical local sudo). Without sudo, nft fails with "cache initialization failed: Operation not permitted" before reaching ruleset parse. |
||
|
Johnathan Corgan
|
a41f80a776 |
Tighten clippy gate to --all-targets --all-features and clean up
The local ci-local.sh and the GitHub CI clippy invocations both used `cargo clippy --all -- -D warnings`, which only checks lib + bin targets. Test code, integration tests, and benches were not lint-gated. Three pre-existing clippy errors lurked in test modules as a result (two field_reassign_with_default in config tests, one items_after_test_module in stun.rs). Tighten both invocations to `cargo clippy --all-targets --all-features -- -D warnings` so the gate covers everything cargo can build, and fix the three exposed errors: - src/config/mod.rs: rewrite two test-only `Config::default()` + field-reassign sites to struct-update syntax. - src/discovery/nostr/stun.rs: move helper `random_txn_id` above the `#[cfg(test)] mod tests` block. Also adds a dedicated Clippy job to the GitHub CI workflow so the strict gate runs on every PR (the workflow had no clippy job before; clippy ran only via testing/ci-local.sh on operator machines). No behavior changes; lint hygiene + CI hardening only. |
||
|
Johnathan Corgan
|
96c6b7dea8 |
Admit rekey msg1 from established peers when addr forms differ
Companion to the ethernet `accept_connections: false` rekey-deadlock fix from earlier this release: the same dual-init failure mode shows up over UDP when peers register by hostname, and the existing addr_to_link-only carve-out in `should_admit_msg1` doesn't cover it. The carve-out's first predicate keys `addr_to_link` by the literal `TransportAddr` that `initiate_connection` inserted, which is the hostname-form when a peer config carries a hostname (e.g., `core-vm.tail65015.ts.net:2121`). Inbound packets always arrive with numeric source addrs because `udp_receive_loop` builds the `TransportAddr` from the `SocketAddr` the kernel reports via `recvfrom`. `TransportAddr` equality is byte-exact, so the two forms don't match and the lookup misses. With `udp.accept_connections: false` (or `udp.outbound_only: true`, which forces it false) the gate then rejects the rekey msg1 from an established peer. The dual-init tie-breaker stalls because the loser side never produces msg2; both sides retry indefinitely and the winner side keeps logging "Dual rekey initiation: we win, dropping their msg1" at 1Hz. The earlier ethernet fix didn't generalize to this variant because ethernet TransportAddrs are always numeric MAC bytes — both the config-time form and the inbound-arrival form match identically. Add a second predicate to `should_admit_msg1`: an active peer's `current_addr()` matching `(transport_id, remote_addr)`. `current_addr` is updated and refreshed from inbound encrypted-frame source addrs (`handlers/encrypted.rs`), which are always numeric `SocketAddr`-form, so this catches the established peer regardless of how its `addr_to_link` key was originally inserted. The fast `addr_to_link` check stays first; the iteration over peers is bounded by peer count and only runs when the first predicate misses. Regression coverage in this commit: - Unit test `test_should_admit_msg1_admits_rekey_when_addr_form_differs` in `src/node/tests/handshake.rs`. Constructs the failing scenario in-process: `addr_to_link` populated with hostname-form key, peer's `current_addr` at the resolved numeric form, query with numeric form. Without the new predicate this fails immediately. - New integration topology `rekey-outbound-only` plus matching docker-compose profile. Same 5-node mesh shape as `rekey-accept-off` but `inject-config` sets `udp.outbound_only: true` on node-b and rewrites node-b's peer-c address from the numeric docker IP to the docker hostname (`node-c:2121`), reproducing the production hostname-vs-numeric mismatch. The test asserts no sustained "Dual rekey initiation: we win" log lines on any node (>10 = bug) and the existing rekey health checks catch the connectivity loss the loop produces. - `testing/ci-local.sh` and `.github/workflows/ci.yml` extended to run the new variant in the local sweep and the GitHub CI integration matrix alongside `rekey` and `rekey-accept-off`. Verified locally: full `bash testing/ci-local.sh` sweep passes 29/29 suites (23m 12s) with the new variant green; 1084 unit tests pass. |
||
|
Johnathan Corgan
|
37c2973e2f |
Test infrastructure overhaul: gateway robustness + full CI coverage
Single combined commit covering five interlocking pieces of test and CI work that landed during the v0.3.0-prep cycle. ## fips-gateway robustness - src/bin/fips-gateway.rs DNS upstream probe converted from a 3-second hard-fail to a bounded retry loop (5 attempts × 1s timeout, 1s sleep between attempts; ~10s worst case). Covers the cold-boot race where the daemon's TUN is up but the DNS responder at [::1]:5354 is still binding. Each failed attempt logs at INFO. In production the binary's retry is the live recovery mechanism; with retry it recovers silently instead of relying on Restart=on-failure (~5s blip + spurious ERROR per cycle). - packaging/debian/fips-gateway.service `ExecStartPre` now waits up to 30 seconds for the daemon's `fips0` TUN to appear before exec'ing the gateway binary. Eliminates the cold-boot race where the gateway exits with `fips0 interface not found` and recovers via `Restart=on-failure`, producing a 5-second blip and a spurious error log per restart cycle. - testing/docker/entrypoint.sh gateway-mode waits up to 30s for the daemon's DNS responder to bind [::1]:5354 (probes once per second with `dig @::1 -p 5354 ... test.fips`) before exec'ing fips-gateway. Belt-and-suspenders with the binary's own retry: in CI we want deterministic startup ordering. On timeout, fall through so the binary's probe reports the definitive error. ## Test infrastructure DNS bind migration to ::1 After session 359's daemon DNS-bind default flipped from `127.0.0.1` to `::1` (the production fix for ISSUE-2026-0002), the static-test infrastructure was carrying a stale workaround that overrode the default back to IPv4 loopback. The fips-gateway integration test exposed the divergence: the gateway probes its DNS upstream at `[::1]:5354` (production default) while the daemon was binding `127.0.0.1:5354` from the template override — IPv6-explicit sockets do not accept v4-mapped traffic, so the upstream probe exhausted retries and the gateway exited. - Drop the explicit `bind_addr: "127.0.0.1"` line from every test config that emits it: testing/static/configs/node.template.yaml, testing/chaos/configs/node.template.yaml, the sidecar heredoc in testing/docker/entrypoint.sh, testing/acl-allowlist/generate-configs.sh (six per-node blocks), testing/nat/scripts/generate-configs.sh, and the four tor templates under testing/tor/. Daemon picks up its production `::1` default. - Flip the dnsmasq forwarder for `.fips` in testing/docker/Dockerfile from `127.0.0.1#5354` to `::1#5354` so dnsmasq on the shared test image continues to reach the daemon. Template and Dockerfile must move together since most static suites resolve `<npub>.fips` via the test-image dnsmasq. ## rekey-accept-off integration variant + UDP unit test - New `rekey-accept-off` topology and docker-compose profile under testing/static/. 2-node variant where node-b runs with `udp.accept_connections: false`. Pins the regression class that ISSUE-2026-0004 fixed (cross-connection winner's rekey msg1 was being filtered by the accept_connections gate, breaking rekey). - testing/static/scripts/rekey-test.sh accepts REKEY_TOPOLOGY and REKEY_ACCEPT_OFF_NODES env vars; its inject-config subcommand applies the per-node `udp.accept_connections: false` edit, and the test asserts no sustained "Dual rekey initiation" log lines. - New UDP variant of `should_admit_msg1` admit-rekey unit test in src/node/tests/handshake.rs. ## ci-local.sh full integration coverage - New runner functions and dispatcher entries for `acl-allowlist`, `nat-cone` / `nat-symmetric` / `nat-lan`, `rekey-accept-off`, `dns-resolver`, `deb-install`. Each integrates with the existing summary tracking via `record`. - New `--with-tor` flag (off by default) gates `tor-socks5-outbound` and `tor-directory-mode` runners. Tor stays opt-in because both harnesses depend on the live Tor network and would introduce a flake source unrelated to the FIPS code. - New suite arrays (`ACL_SUITES`, `NAT_SUITES`, `DNS_RESOLVER_SUITES`, `DEB_INSTALL_SUITES`, `TOR_SUITES`) drive both the default sweep and `--list` output. - `run_suite` extended to accept the new suite names for `--only` invocations. ## GitHub CI matrix expansions - `gateway` matrix entry runs testing/static/scripts/gateway-test.sh against the existing docker-compose `gateway` profile. - `rekey-accept-off` matrix entry exercises the new topology with REKEY_ACCEPT_OFF_NODES=b. - `deb-install` matrix (debian12 + ubuntu24 + ubuntu26) runs testing/deb-install/test.sh with privileged systemd containers. ~5-7 min cold cache, ~2 min warm per distro. Self-contained: builds its own .deb in a Debian 12 cargo-deb builder image; does not depend on the build job's pre-built artifact. - `dns-resolver` matrix entry runs the full 13-scenario harness (per-distro systemd resolver-backend tests + real-fips end-to-end scenarios) in a single job. Pins the production DNS bind path that ISSUE-2026-0002 lived in. ~7-12 min warm, ~12-15 min cold. Verified locally: full `bash testing/ci-local.sh` sweep passes, including 5/5 deb-install distros and all 13 dns-resolver scenarios. Tor-inclusive sweep (`--with-tor`) verified in a follow-up run. |
||
 Johnathan CorganandGitHub
|
cbc78091ab |
Rationalize cargo feature and platform-gate surface (#79)
Drop the `tui`, `ble`, and `gateway` cargo features and replace them with platform cfg gates. Plain `cargo build` now produces every subsystem appropriate for the target platform with no feature flags required. Motivation: - `default = ["tui", "ble"]` broke `cargo build` on macOS and Windows because `ble` pulled in `bluer` (BlueZ, Linux-only). Every non-Linux packager needed `--no-default-features`. - The feature flags on `ble` and `gateway` were redundant with their platform-gated deps (`bluer`, `rustables`). The parallel gating was inconsistent and error-prone. - `tui` feature protected against a ratatui binary-size concern that no longer applies in 2026. Cargo.toml: - Remove `tui`, `ble`, `gateway` features; `default = []`. - Promote `ratatui` to a non-optional top-level dependency. - Move `rustables` from top-level optional into the Linux target block, non-optional. - Split `bluer` into its own target block with `cfg(all(target_os = "linux", not(target_env = "musl")))` — BlueZ isn't available on musl router targets and `libdbus-sys` doesn't cross-compile to musl without pkg-config sysroot setup. - Drop `required-features` from the `fipstop` and `fips-gateway` `[[bin]]` entries. build.rs: - Emit a `bluer_available` custom cfg when `target_os == "linux"` and `target_env != "musl"`, for use in place of the verbose full predicate in source cfg gates. Source: - Replace every `#[cfg(feature = "gateway")]` with `#[cfg(target_os = "linux")]`. Gateway code works on both glibc and musl Linux (rustables is fine on musl). - Replace every `#[cfg(feature = "ble")]` with `#[cfg(bluer_available)]`. BLE-specific code (BluerIo module, bluer type conversions, BLE transport instance creation, resolve_ble_addr) is excluded on musl and non-Linux. Generic `BleAddr`, `BleIo` trait, `MockBleIo`, and `BleTransport<I>` still compile on all targets. - `src/bin/fips-gateway.rs`: always compiled, but `main()` is gated to Linux. Non-Linux stub exits 1 with a diagnostic. Existing non-Linux packaging scripts don't ship it, so the stub binary sits unused. Packaging and CI: - Drop `--features` and `--no-default-features` flags from every packaging script and workflow. Defaults now match each platform's capabilities. - AUR `fips-git` automatically aligns with stable `PKGBUILD` (both build with defaults). Verified: `cargo build --release` with no flags produces all four binaries on glibc Linux; all unit and integration tests pass across Linux/macOS/Windows/OpenWrt (musl) in CI. |
||
|
Johnathan Corgan
|
213c0e87c3 |
Implement inbound mesh port forwarding
Mesh peers can now reach a configured host:port on the gateway's LAN
via static port-forward rules on fips-gateway. Mirror of the outbound
LAN gateway (IDEA-0079 / TASK-2026-0056).
Config: new gateway.port_forwards list of { listen_port, proto,
target } entries. Targets are SocketAddrV6 — IPv4 is rejected at
parse time. Validation rejects zero listen ports and duplicate
(listen_port, proto) pairs.
NAT: NatManager gains a port_forwards field and set_port_forwards()
setter, rebuilt in the same atomic rustables batch as the address
mappings. Each forward emits a prerouting DNAT rule keyed on
(iifname fips0, nfproto ipv6, l4proto, tcp/udp dport) that rewrites
destination address and port via Nat::with_ip_register +
with_port_register. When any forwards are configured, a single
LAN-side masquerade is installed on (iifname fips0, oifname
lan_interface, nfproto ipv6) so the LAN host sees the gateway as
source and replies flow back through conntrack. This rule is
distinct from the existing oifname fips0 masquerade that serves the
outbound pool.
Binary: fips-gateway validates port_forwards at startup and calls
set_port_forwards after NatManager construction; startup failure
cleans up the nftables table before exiting.
Test: extend testing/static/scripts/gateway-test.sh with Phase 7
that runs a marker HTTP server on the LAN-side client (fd02::20:8080)
and, from the mesh peer, curls the gateway's fips0 address on port
18080 to exercise the full DNAT + LAN masquerade path. The
LAN-side HTTP server is started with 'docker exec -d' plus a
bind-ready poll on ss; 'docker exec bash -c "cmd &"' does not
keep the child alive past the exec session even with nohup.
Test-infra: ci-local.sh now builds/clippies/tests with
--features gateway, matching GitHub CI. Without this the release
fips-gateway binary silently stays stale across runs, since
cargo build --release alone does not compile the gateway bin.
Verified locally: cargo test --features gateway --lib (991 pass),
clippy + fmt clean, full testing/ci-local.sh green (21/21 suites
in 8m36s, including the new gateway Phase 7).
|
||
|
Johnathan Corgan
|
6196307f0e |
Merge branch 'maint'
# Conflicts: # src/bin/fips.rs # src/bin/fipstop/app.rs # src/config/mod.rs # src/config/node.rs # src/config/transport.rs # src/mmp/receiver.rs # src/mmp/sender.rs # src/node/handlers/handshake.rs # src/node/handlers/rekey.rs # src/node/lifecycle.rs # src/node/mod.rs # src/transport/ethernet/socket.rs # src/transport/mod.rs # src/upper/tun.rs |
||
|
Johnathan Corgan
|
68bdcb2c75 | Add cargo fmt --check to CI and local CI | ||
|
Johnathan Corgan
|
60e5fefb1f |
Implement outbound LAN gateway
Add fips-gateway binary: a separate daemon that allows unmodified LAN hosts to reach FIPS mesh destinations via DNS-allocated virtual IPs and kernel nftables NAT. Gateway DNS resolver: forwarding proxy on [::]:53 that intercepts .fips queries, forwards to daemon resolver (localhost:5354), allocates virtual IPs from pool, returns AAAA records. Always sends AAAA upstream regardless of client query type, returns proper NODATA for non-AAAA. Virtual IP pool: fd01::/112 pool with state machine lifecycle (Allocated → Active → Draining → Free), TTL-based reclamation, conntrack integration for session tracking. NAT manager: nftables DNAT/SNAT rules via rustables netlink API, per-mapping rule lifecycle, fips0 masquerade for LAN client source address rewriting. Network setup: local pool route, proxy NDP for virtual IPs on LAN interface, IPv6 forwarding validation. Control socket at /run/fips/gateway.sock with show_gateway and show_mappings queries. fipstop Gateway tab with pool summary gauge and mappings table. Gateway config section in fips.yaml with pool CIDR, LAN interface, DNS upstream, TTL, and grace period settings. Design doc at docs/design/fips-gateway.md. Integration test (testing/static/scripts/gateway-test.sh): three containers verifying DNS resolution, end-to-end HTTP, NAT state, TTL expiration, SERVFAIL fallback, and clean shutdown. |
||
|
Johnathan Corgan
|
9e63b42bd9 |
Consolidate Docker test harness infrastructure
Replace 4 near-identical per-harness Docker setups with unified shared infrastructure. Net result: -463 lines across 55 files, faster CI (8.5 min vs ~13.5 min), 14 scenarios (down from 21). Unified Docker image (testing/docker/): - Single Dockerfile (trixie-slim) with FIPS_TEST_MODE env var for mode dispatch: default, chaos, sidecar, tor-socks5, tor-directory - Single entrypoint.sh with conditional logic per mode - Replaces 5 Dockerfiles, 3 entrypoints, 4 resolv.conf copies Shared build and libraries (testing/scripts/, testing/lib/): - testing/scripts/build.sh: single build script with macOS zigbuild support, replaces 4 per-harness copies - testing/lib/derive_keys.py: shared key derivation module, replaces 3 copies of derive-keys.py - testing/lib/log_analysis.py: shared log analysis extracted from chaos sim/logs.py, with CLI interface and rekey cutover tracking - testing/lib/wait-converge.sh: shared convergence wait helpers (wait_for_links, wait_for_peers) using fipsctl JSON polling Scenario consolidation: - Remove 7 redundant scenarios: tcp-chain (subsumed by tcp-mesh), tcp-only (subsumed by tcp-mesh), chaos-10 (replaced by churn-mixed --nodes 10), churn-10/churn-20/churn-20-mixed (subsumed by parameterized churn-mixed), cost-mixed-7node (overlaps mixed-technology) - Add churn-mixed scenario with --nodes flag for scale testing - Reduce idle scenario durations: smoke-10 60s→30s, ethernet-only 90s→30s, cost-avoidance 120s→45s, depth-vs-cost 120s→45s, bottleneck-parent 120s→60s, mixed-technology 180s→90s CI updates: - ci-local.sh: unified image build, structured chaos suite entries with per-scenario flags, --skip-build for sidecar - ci.yml: shared binary install + image build step, updated scenario matrix, chaos_flags support for parameterized scenarios - Add tcp-mesh and congestion-stress to CI matrix - Static ping test uses active peer convergence detection instead of hardcoded 5s sleep Chaos infrastructure improvements (from discovery-rework branch): - Pre-built Docker image instead of per-service build at scale - --nodes flag in chaos.sh for runtime topology size override |
||
|
Johnathan Corgan
|
aa53da061f | Add local CI runner script and gitignore fipstop in test dirs |