laantungir/fips
1
0
Fork 0
mirror of https://github.com/jmcorgan/fips.git synced 2026-10-05 11:08:25 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
53a3ec4b2fbaceb7009dd4fbe6974b011e452ae5
fips/testing/README.md
T
Johnathan Corgan 9979826235 Prepare the v0.5.2 release notes, changelog and documentation corrections 
Move the staged changelog entries under a 0.5.2 heading dated 2026-09-28
and leave an empty Unreleased section above it. The date is provisional:
a comment beside the heading says so, and the two release-notes files
carry the same date with the same marker, so the check at the version
bump finds all three.

Add the release notes and mirror them byte for byte to RELEASE-NOTES.md.
Every link is absolute so the Release body resolves them, and each
paragraph and list item is on one line, because the Release page shows
every newline inside a paragraph as a line break; the file exempts
itself from the line-length lint rule. The notes lead with who should
upgrade and with the three defaults that changed: the gateway's DNS
port, the Windows config directory, and an ephemeral node no longer
writing its key file. They say what was measured and what was not,
including the mixed-version interop run against v0.5.1 and v0.5.0, the
Windows installer checks on Windows Server under Windows PowerShell 5.1
and PowerShell 7, and the checks still outstanding. They state that a
link to a v0.5.0 or v0.5.1 node can still drop after a lost rekey reply
until that node is upgraded, since the fix is on the answering side.
The Windows upgrade notes say to stop the service before every run of
the installer, and to move fips.yaml and fips.key from \etc\fips into
C:\ProgramData\fips before upgrading a service that was set up by hand
to read its config from \etc\fips, which otherwise comes up under a new
identity with no warning.

The README's status badge, release-notes link and status paragraph
follow the release.

Correct documentation that no longer matches the gateway, tree, Windows
and packaging behavior:

- The gateway design document, how-to, OpenWrt tutorial and the
  configuration reference describe the NAT rebuild as one transaction,
  the 1000-mapping ceiling and the new-name rate limit in place of the
  pool size as a hard cap, and which DNS queries allocate a mapping.
- The spanning-tree documents describe the periodic re-broadcast and
  the resend of an unconfirmed announce, and the bloom filter update
  triggers include a parent switch and a child joining or leaving.
- The fips, fipsctl and security references cover the restricted
  C:\ProgramData\fips on Windows, the ACL and key paths on macOS,
  FreeBSD and Windows, the legacy peer ACL fallback in \etc\fips, and
  the Debian fips.yaml's actual mode and conffile status.
- The packaging guides no longer list MIPS as supported, the arm64 .deb
  leg is described as also purging the package, and the OpenWrt SDK-feed
  README says a package built from its Makefile carries none of the
  released packages' maintainer scripts.
- The testing README gains a section for the OpenWrt maintainer-script
  suite and says the ACL allowlist suite runs by hand only, and the
  interop README lists the mesh-size check as its eighth phase.


The upgrade notes were then corrected where following them as written
would have left a node worse off:

- Gateway DNS port: a fips.yaml that sets gateway.dns.listen keeps its
  port through the upgrade, and the v0.5.1 example config and deployment
  guide set it to [::1]:5353, so the resolver instruction depends on
  whether the config sets it.
- OpenWrt: an operator who had the gateway disabled must stop it and
  then disable it after the first opkg upgrade.
- FreeBSD: an upgrade step that restarts fips and fips_dns; the command
  comes from pkg's source and is listed as not measured.
- Debian: the upgrade re-enables and starts fips-dns every time;
  `systemctl mask fips-dns` keeps it off. The .deb start bound is 90
  seconds for fips-gateway.
- Arch and the systemd tarball: what to restart or start after the
  upgrade, and that only a .deb upgrade reloads the firewall.
- Ephemeral nodes: set persistent before upgrading to keep a key.
- Windows: one ordered sequence in an elevated PowerShell, with the
  installer run under -ExecutionPolicy Bypass.
- Building from source on glibc Linux also needs libdbus-1-dev and
  pkg-config. README, getting-started and the packaging README install
  the .deb with apt install ./ and point to the packaging README for
  per-format install commands.

The notes record an OpenWrt 24 router test of the gateway DNS port,
and that a gateway that fails to start leaves dnsmasq forwarding .fips
to its port, with how to hand .fips back to the daemon.

Drop the test-us03-next alias from the shipped hosts file and from the
roster in the host-aliases how-to.
2026-09-28 22:16:15 +00:00

245 lines
11 KiB
Markdown
Raw Blame History

# FIPS Testing
Integration and simulation test harnesses for FIPS, using Docker
containers running the full protocol stack.
## Test Harnesses
### [static/](static/) -- Static Docker Network
Fixed topologies with manual scripts for building, config generation,
connectivity tests (ping, iperf), and network impairment (netem).
Useful for deterministic debugging and validating specific topology
configurations.
| Topology | Nodes | Transport | Description |
| ----------- | ----- | --------- | -------------------------------- |
| mesh | 5 | UDP | Sparse mesh, 6 links, multi-hop |
| chain | 5 | UDP | Linear chain, max 4-hop paths |
| rekey | 5 | UDP | Rekey integration test topology |
### [tor/](tor/) -- Tor Transport Integration
End-to-end Tor transport testing with Docker containers running real
Tor daemons. Requires internet access for Tor bootstrapping.
| Scenario | Description |
| -------------- | -------------------------------------------------------- |
| socks5-outbound | Outbound SOCKS5 connections through Tor to clearnet peer |
| directory-mode | Inbound via HiddenServiceDir onion service (co-located) |
### [nat/](nat/) -- NAT Traversal Lab
Real Docker NAT traversal tests for the Nostr/STUN bootstrap path,
using router containers with `iptables`-based NAT, a local Nostr relay,
and a local STUN responder.
| Scenario | Description |
| --------- | ------------------------------------------------------------ |
| cone | Two NATed peers establish a UDP traversal path |
| symmetric | UDP traversal fails under symmetric NAT, TCP fallback wins |
| lan | Peers on the same LAN prefer local addresses over reflexive |
### [chaos/](chaos/) -- Stochastic Simulation
Automated network testing with configurable node counts, topology
algorithms (random geometric, Erdos-Renyi, chain, explicit), and fault
injection (netem mutation, link flaps, traffic generation, node
churn). 10 scenarios covering general stress and node churn, discovery
over sparse topologies, spanning-tree and bloom-propagation regression,
transport-specific validation (UDP, TCP, Ethernet), and ECN/congestion
testing. Scenarios are
defined in YAML and executed via a Python harness that manages the full
lifecycle: topology generation, Docker orchestration, fault scheduling,
log collection, and analysis.
### [interop/](interop/) -- Mixed-Version Interop Harness
On-demand harness that runs an N-node full mesh from a node-spec where
each node can run a different build of the FIPS daemon, then attributes
every FMP/FSP/rekey/connectivity failure to a specific version pair
(same-version vs MIXED). Used to catch interop regressions between
builds, not as a per-commit CI gate; not part of `ci-local.sh`.
### [mesh-lab/](mesh-lab/) -- Mesh Reliability Lab
On-demand harness that runs a chosen integration suite N times under a
configurable host-pressure profile (idle / light / github-runner-
equivalent / heavy via `stress-ng`), per-container netem impairment,
and optional trace-level RUST_LOG, capturing per-rep diagnostics and a
mechanism-match summary across the run. Used for statistical reliability
characterization of known flake classes under calibrated stress, not as
a per-commit gate; not part of `ci-local.sh`.
### [sidecar/](sidecar/) -- Network Sidecar Isolation
FIPS running as a sidecar container that owns the network namespace of
a companion application container, with iptables/ip6tables rules
confining the app to the mesh. `scripts/test-sidecar.sh` boots a
three-node chain of such pairs and asserts both connectivity and
isolation.
### [firewall/](firewall/) -- nftables Baseline
End-to-end exercise of the production `fips0` nftables baseline at
`packaging/common/fips.nft`, covering the default-deny, conntrack and
drop-in semantics.
### [acl-allowlist/](acl-allowlist/) -- Peer ACL Enforcement
Six nodes with per-node allowlist files mounted at the runtime ACL
paths, exercising insiders, outsiders and allowed remotes at once to
check which peer pairs are admitted and which are rejected. Run by hand
only; retired from both CI runners as redundant with the unit and
in-process ACL tests (see `ci-local.sh`).
### [openwrt/](openwrt/) -- OpenWrt Maintainer Scripts
Runs the OpenWrt package scripts and the `fips-gateway` init script
under busybox `ash` against stubbed init scripts and `uci`, and checks
what the real `build-apk.sh` and `build-ipk.sh` package. No router,
opkg or apk-tools is involved. Part of both CI runners as
`openwrt-scripts`.
### [native-api/](native-api/) -- Native Datagram API
Checks the experimental native datagram API: a client process opens a
flow to a remote pubkey over a Unix socket, receives a file descriptor,
and exchanges datagrams on it with no TUN device and no IPv6 emulation.
### [dns-resolver/](dns-resolver/) -- `fips-dns-setup` Backends
Runs `fips-dns-setup` against each supported Linux resolver backend in
systemd containers, verifying backend detection, generated config and
teardown, plus an end-to-end scenario that resolves a `.fips` name
through the configured backend. The end-to-end scenarios run the
binaries from a Debian package: `--deb PATH` supplies one, and without
it the suite builds one through `packaging/debian/build-deb-container.sh`.
### [deb-install/](deb-install/) -- Debian Package Install
Installs the built `.deb` in systemd containers for each
target distro and verifies unit enablement, conffile placement and
end-to-end `.fips` resolution as a user would meet it. GitHub CI also
installs the arm64 package on ubuntu22 on an arm64 runner, a leg the
local run cannot have and the parity check reports as GitHub-only.
### [boringtun/](boringtun/) -- WireGuard Throughput Baseline
Two userspace WireGuard peers running Cloudflare BoringTun, measured
with `iperf3`, as a comparison baseline for FIPS tunnel throughput.
### [ble/](ble/) -- BLE L2CAP Spike
Standalone cargo project (`ble_spike`) that validates the `bluer` API
assumptions behind the `BleIo` trait against real adapters on two
machines. Not a Docker harness.
## Running CI locally (`ci-local.sh`)
[`ci-local.sh`](ci-local.sh) runs the full local CI pipeline — build,
clippy, unit tests, and the integration suites (including the chaos
scenarios) — mirroring the GitHub `ci.yml` integration matrices. Run
`./ci-local.sh --help` for the full option list and `--list` for the
available suites. Every run starts with a parity check that verifies the
local suite set covers the same work as the GitHub matrix, per scenario for
chaos and per distro for deb-install, across every job that carries a
matrix; a divergence fails the run. GitHub
runs the same check as its own `ci-parity` job. `--check-parity` runs it
alone (see [check-ci-parity.sh](check-ci-parity.sh)).
### Per-run isolation and the `FIPS_CI_RUN_ID` override
Every invocation derives a **run id** and scopes all of its Docker
resources to it, so two simultaneous runs on the same host (for example,
one per git worktree, or an operator testing by hand while CI is in
flight) never collide:
- **Compose projects** are named `fipsci_<run-id>_<suite>`, so
container, network, and volume names are all prefixed per run.
- **Build images** are tagged `fips-test:<run-id>` and
`fips-test-app:<run-id>`, exported as `FIPS_TEST_IMAGE` /
`FIPS_TEST_APP_IMAGE`, and **every** compose file and suite script reads
those. The run does not write `fips-test:latest` at all: a bridge back to
that shared mutable name would let a consumer that had been missed keep
working while resolving whichever concurrent run wrote the tag last.
`:latest` stays the hand-build name, produced by
`testing/scripts/build.sh`, and remains the default every consumer falls
back to when the variables are unset.
- **The build context** is a per-run copy at `testing/docker-<run-id>/`,
exported as `FIPS_BUILD_CONTEXT`. It is absolute because compose resolves
a relative build context against the compose file's own directory rather
than the working directory. `testing/docker/` is the hand-run context and
a CI run does not write to it. Without this, two runs race on the contents
of one directory and either can build a correctly-per-run-tagged image
from the other's binaries.
- Each parallel chaos child gets a unique, non-overlapping `/24` in
`10.30.x` (via the sim `--subnet` override). `10.30.x` sits outside
Docker's default address pool and the fixed-subnet suites' `172.x`
ranges, so neither a sibling chaos child nor an auto-assigned network
can swallow a pinned subnet.
By default the run id is `<short-git-sha>-<random>` — the SHA portion
records *what code* a container is testing, the random suffix keeps
simultaneous runs of the same SHA disjoint. Override it for a
reproducible, attach-by-name debug session:
```sh
FIPS_CI_RUN_ID=mydebug ./ci-local.sh --only static-mesh
# containers are named fipsci_mydebug_static_fips-node-a, etc.
```
### Preemption-safety and exit codes
`ci-local.sh` is safe to cancel mid-run. A signal trap tears down *every*
compose project the run started (not just the current suite) and reaps
any in-flight parallel chaos children, bounded by a `timeout` so a stuck
`compose down` cannot wedge the trap. Exit codes distinguish a cancelled
run from a failing one:
| Code | Meaning |
| ---- | ------- |
| `0` | all stages passed |
| `1` | one or more stages failed |
| `130` | interrupted by SIGINT — cancelled, not a failure |
| `143` | terminated by SIGTERM — cancelled, not a failure |
A preempting CI worker (the push-triggered, CI-gated build pipeline that
kills an in-flight run when a newer same-branch tip arrives) maps
`130`/`143` → *cancelled* (discard, do not record a failing commit), `0`
→ green, any other non-zero → red.
### Cleaning up leftover resources
Every CI-created container, network, and volume carries the label
`com.corganlabs.fips-ci=1`. If a run is hard-killed (SIGKILL, OOM, crash)
and leaves resources behind, reap them with:
```sh
./ci-local.sh --reap # or: ./ci-cleanup.sh
```
[`ci-cleanup.sh`](ci-cleanup.sh) force-removes everything bearing the CI
label or a `fipsci_` compose-project prefix; it is safe to run when there
is nothing to reap and safe to run repeatedly. Pass `--project-prefix` to
scope the sweep to a single run.
It also removes the chaos simulation's leftover host-namespace veth
interfaces (`vh…a`/`vh…b`), the one resource it touches that is neither a
docker object nor labelled — a host interface can carry neither a label
nor a compose project, so it is matched by name shape alone. That makes
the reach here asymmetric with everything above, and worth stating
plainly:
- A bare `chaos.sh` run's **containers** survive a broad reap. Its
compose project is not `fipsci_`, and the simulation labels only the
network, not the services.
- A bare `chaos.sh` run's **veth interfaces do not.** An unscoped reap
deletes them while they are in use, severing the Ethernet links of a
live simulation and leaving its containers running.
So do not run a broad `--reap` while a bare simulation is up. Scope the
interface sweep with `--veth-suffixes` (which is what `ci-local.sh`'s own
teardown passes) or wait for the simulation to finish. `--project-prefix`
does not help here: it scopes only the compose-project sweep.
Reference in New Issue View Git Blame Copy Permalink