A CI worker may preempt an in-flight ci-local.sh run (SIGTERM, then SIGKILL
after a grace period) to restart on a newer commit. For that kill to be safe,
the script must clean up after itself and never let a dying run collide with
its restart. It previously had no signal handling, shared the default compose
project name across runs, and tore down each suite only at the suite end.
- Derive a per-run id (honoring FIPS_CI_RUN_ID, else short-sha+random) and
namespace every docker resource to it: a fipsci_<run>_<suite> compose project
per suite and per parallel chaos child, and per-run image tags
(fips-test:<run>, fips-test-app:<run>) retagged to :latest only after both
builds succeed so :latest never points at a half-built image.
- Install a bounded, idempotent teardown trap on SIGTERM/SIGINT (+ EXIT): reap
parallel chaos children, then force-remove this run's docker resources via
the new ci-cleanup.sh, wrapped in timeout so a stuck down cannot wedge it.
- Exit 143 (SIGTERM) / 130 (SIGINT), distinct from 0 (pass) / 1 (failed), so a
preempting worker tells a cancelled run from a real failure.
- Add ci-cleanup.sh (also ci-local.sh --reap): force-removes leftover CI
resources by the com.corganlabs.fips-ci=1 label and the fipsci_ project
prefix, robust to however a prior run died.
- Label every per-suite docker resource so the label sweep reaps it after a
SIGKILL regardless of network name: direct docker run/network resources, the
sidecar compose services, and every per-suite compose network (acl-allowlist,
boringtun, firewall, nat, static, both tor suites, and the chaos generator
template). Parametrize the static/sidecar compose image refs so the per-run
tags are honored.
- Give each parallel chaos child a unique /24 from 10.30.x (a new --subnet
override on the sim CLI, assigned per-child in ci-local.sh) so parallel
children never collide on a shared docker subnet, and a chaos net can never
span a fixed-subnet suite (sidecar/static in 172.20.x). 10.30.x sits outside
docker's default-address-pool range, so an auto-assigned net cannot land on
it either; node IPs derive from the subnet, so no scenario config changes.
Johnathan Corgan