Johnathan Corgan
cc7f967128
Three deltas to the mesh-lab nat-lan suite for stall characterization: - FIPS_NAT_LAN_CPUSET env-var-driven CPU-pinning sidecar in run_nat_lan, mirroring the bloom-storm pattern. Pinning is needed because the mesh-lab compose-resource-limits.yml override is rekey-family service-name specific (rekey-* / rekey-accept-off-* / rekey-outbound-only-*), so it does not constrain the nat-lan containers. Default cpuset 0,1 mimics a GHA 2-core runner; empty disables the sidecar. - New compose-trace-nat.yml overlay that bumps RUST_LOG to trace on discovery::nostr, transport::udp, node::lifecycle, handlers::handshake, handlers::forwarding — the modules covering the cross-init / adoption / handshake path. Picked up by the nat-test.sh COMPOSE array via a new FIPS_NAT_EXTRA_COMPOSE colon-separated env-var hook. run_nat_lan sets this hook when FIPS_MESH_LAB_TRACE is non-empty; the README env-var section updated to reflect that FIPS_MESH_LAB_TRACE now applies to nat-lan in addition to the rekey-family. - parse_nat_lan extended with a per-node stall_signature emitting last-occurrence timestamps for eight event categories (startup, discovery, adoption, handshake_init, msg2_sent, cross_init_ignore_*, handshake_failed) plus derived last_meaningful_event_ts, last_event_category, silent_gap_s. Top-level stall_class binned as no_timeout / silent / localized / distributed / incomplete from the per-node categories. Aggregation phase consumes the per-rep signatures across a characterization run to classify stall mechanism. Wired support in nat-test.sh: FIPS_NAT_EXTRA_COMPOSE colon-separated list of repo-relative or absolute compose files layered onto the base via the COMPOSE array; FIPS_NAT_SKIP_FINAL_CLEANUP gates the success-path teardown so the mesh-lab harness can capture docker logs before tearing down (failure paths already returned without cleanup, leaving stall-state containers intact for capture). Smoke-tested on idle profile with TRACE on: 1 rep PASS, 32/36 TRACE lines per node, signature.json events all populated with the expected category timestamps.
FIPS Testing
Integration and simulation test harnesses for FIPS, using Docker containers running the full protocol stack.
Test Harnesses
static/ -- Static Docker Network
Fixed topologies with manual scripts for building, config generation, connectivity tests (ping, iperf), and network impairment (netem). Useful for deterministic debugging and validating specific topology configurations.
| Topology | Nodes | Transport | Description |
|---|---|---|---|
| mesh | 5 | UDP | Sparse mesh, 6 links, multi-hop |
| chain | 5 | UDP | Linear chain, max 4-hop paths |
| mesh-public | 5+1 | UDP | Mesh with external public node |
| tcp-chain | 3 | TCP | Linear chain over TCP (port 8443) |
| rekey | 5 | UDP | Rekey integration test topology |
tor/ -- Tor Transport Integration
End-to-end Tor transport testing with Docker containers running real Tor daemons. Requires internet access for Tor bootstrapping.
| Scenario | Description |
|---|---|
| socks5-outbound | Outbound SOCKS5 connections through Tor to clearnet peer |
| directory-mode | Inbound via HiddenServiceDir onion service (co-located) |
nat/ -- NAT Traversal Lab
Real Docker NAT traversal tests for the Nostr/STUN bootstrap path,
using router containers with iptables-based NAT, a local Nostr relay,
and a local STUN responder.
| Scenario | Description |
|---|---|
| cone | Two NATed peers establish a UDP traversal path |
| symmetric | UDP traversal fails under symmetric NAT, TCP fallback wins |
| lan | Peers on the same LAN prefer local addresses over reflexive |
chaos/ -- Stochastic Simulation
Automated network testing with configurable node counts, topology algorithms (random geometric, Erdos-Renyi, chain, explicit), and fault injection (netem mutation, link flaps, traffic generation, node churn). 20 scenarios covering general stress testing, cost-based parent selection, mixed link technologies (fiber/Bluetooth/WiFi), transport-specific validation (UDP, TCP, Ethernet), and ECN/congestion testing. Scenarios are defined in YAML and executed via a Python harness that manages the full lifecycle: topology generation, Docker orchestration, fault scheduling, log collection, and analysis.
interop/ -- Mixed-Version Interop Harness
On-demand harness that runs an N-node full mesh from a node-spec where
each node can run a different build of the FIPS daemon, then attributes
every FMP/FSP/rekey/connectivity failure to a specific version pair
(same-version vs MIXED). Used to catch interop regressions between
builds, not as a per-commit CI gate; not part of ci-local.sh.
mesh-lab/ -- Mesh Reliability Lab
On-demand harness that runs a chosen integration suite N times under a
configurable host-pressure profile (idle / light / github-runner-
equivalent / heavy via stress-ng), per-container netem impairment,
and optional trace-level RUST_LOG, capturing per-rep diagnostics and a
mechanism-match summary across the run. Used for statistical reliability
characterization of known flake classes under calibrated stress, not as
a per-commit gate; not part of ci-local.sh.