 Johnathan Corgan
|
9e63b42bd9
|
Consolidate Docker test harness infrastructure
Replace 4 near-identical per-harness Docker setups with unified shared
infrastructure. Net result: -463 lines across 55 files, faster CI
(8.5 min vs ~13.5 min), 14 scenarios (down from 21).
Unified Docker image (testing/docker/):
- Single Dockerfile (trixie-slim) with FIPS_TEST_MODE env var for
mode dispatch: default, chaos, sidecar, tor-socks5, tor-directory
- Single entrypoint.sh with conditional logic per mode
- Replaces 5 Dockerfiles, 3 entrypoints, 4 resolv.conf copies
Shared build and libraries (testing/scripts/, testing/lib/):
- testing/scripts/build.sh: single build script with macOS zigbuild
support, replaces 4 per-harness copies
- testing/lib/derive_keys.py: shared key derivation module, replaces
3 copies of derive-keys.py
- testing/lib/log_analysis.py: shared log analysis extracted from
chaos sim/logs.py, with CLI interface and rekey cutover tracking
- testing/lib/wait-converge.sh: shared convergence wait helpers
(wait_for_links, wait_for_peers) using fipsctl JSON polling
Scenario consolidation:
- Remove 7 redundant scenarios: tcp-chain (subsumed by tcp-mesh),
tcp-only (subsumed by tcp-mesh), chaos-10 (replaced by
churn-mixed --nodes 10), churn-10/churn-20/churn-20-mixed
(subsumed by parameterized churn-mixed), cost-mixed-7node
(overlaps mixed-technology)
- Add churn-mixed scenario with --nodes flag for scale testing
- Reduce idle scenario durations: smoke-10 60s→30s,
ethernet-only 90s→30s, cost-avoidance 120s→45s,
depth-vs-cost 120s→45s, bottleneck-parent 120s→60s,
mixed-technology 180s→90s
CI updates:
- ci-local.sh: unified image build, structured chaos suite entries
with per-scenario flags, --skip-build for sidecar
- ci.yml: shared binary install + image build step, updated scenario
matrix, chaos_flags support for parameterized scenarios
- Add tcp-mesh and congestion-stress to CI matrix
- Static ping test uses active peer convergence detection instead
of hardcoded 5s sleep
Chaos infrastructure improvements (from discovery-rework branch):
- Pre-built Docker image instead of per-service build at scale
- --nodes flag in chaos.sh for runtime topology size override
|
2026-03-19 18:08:36 +00:00 |
|
|
Johnathan Corgan
|
66c268a564
|
Add static and stochastic Docker test harnesses
Move examples/docker-network/ to testing/static/ and add testing/chaos/
as a new stochastic simulation harness.
testing/static/ — Static 5-node test harness:
- Fixed mesh, chain, and mesh-public topologies with docker compose
- Manual test scripts (ping, iperf, netem)
- Build script, config generation, key derivation
testing/chaos/ — Stochastic network simulation:
- Python orchestrator generating N-node FIPS meshes with dynamic
network conditions, driven by reproducible YAML scenarios
- Topology generation: random geometric, Erdos-Renyi, or chain
graphs with BFS connectivity guarantee
- Per-link netem: HTB classful qdiscs with u32 filters for per-peer
impairment (delay, loss, jitter), stochastic mutation across
configurable policy profiles
- Per-link bandwidth pacing: HTB rate limiting with configurable
tiers (1/10/100/1000 mbps) randomly assigned per edge
- Link flaps: tc netem 100% loss with graph connectivity protection
- Node churn: docker stop/start with netem re-application on restart,
shared down_nodes tracking across all managers
- Traffic generation: random iperf3 sessions between node pairs
- Down-node guards: all docker exec callers check container liveness,
auto-detect crashed containers via is_container_running() safety net
- Log collection and post-run analysis (panics, errors, sessions,
MMP metrics, tree reconvergence)
- chaos.sh wrapper with --seed, --duration, --verbose, --list options
- Four scenarios: smoke-10, chaos-10, churn-10, churn-20
|
2026-02-20 13:35:57 +00:00 |
|