Files
fips/testing/mesh-lab/compose-trace-nat.yml
T
Johnathan Corgan cc7f967128 testing/mesh-lab: nat-lan cpu-pinning sidecar + trace overlay + stall_signature
Three deltas to the mesh-lab nat-lan suite for stall characterization:

- FIPS_NAT_LAN_CPUSET env-var-driven CPU-pinning sidecar in
  run_nat_lan, mirroring the bloom-storm pattern. Pinning is needed
  because the mesh-lab compose-resource-limits.yml override is
  rekey-family service-name specific (rekey-* / rekey-accept-off-* /
  rekey-outbound-only-*), so it does not constrain the nat-lan
  containers. Default cpuset 0,1 mimics a GHA 2-core runner; empty
  disables the sidecar.

- New compose-trace-nat.yml overlay that bumps RUST_LOG to trace on
  discovery::nostr, transport::udp, node::lifecycle,
  handlers::handshake, handlers::forwarding — the modules covering
  the cross-init / adoption / handshake path. Picked up by the
  nat-test.sh COMPOSE array via a new FIPS_NAT_EXTRA_COMPOSE
  colon-separated env-var hook. run_nat_lan sets this hook when
  FIPS_MESH_LAB_TRACE is non-empty; the README env-var section
  updated to reflect that FIPS_MESH_LAB_TRACE now applies to nat-lan
  in addition to the rekey-family.

- parse_nat_lan extended with a per-node stall_signature emitting
  last-occurrence timestamps for eight event categories (startup,
  discovery, adoption, handshake_init, msg2_sent, cross_init_ignore_*,
  handshake_failed) plus derived last_meaningful_event_ts,
  last_event_category, silent_gap_s. Top-level stall_class binned as
  no_timeout / silent / localized / distributed / incomplete from
  the per-node categories. Aggregation phase consumes the per-rep
  signatures across a characterization run to classify stall
  mechanism.

Wired support in nat-test.sh: FIPS_NAT_EXTRA_COMPOSE colon-separated
list of repo-relative or absolute compose files layered onto the
base via the COMPOSE array; FIPS_NAT_SKIP_FINAL_CLEANUP gates the
success-path teardown so the mesh-lab harness can capture docker
logs before tearing down (failure paths already returned without
cleanup, leaving stall-state containers intact for capture).

Smoke-tested on idle profile with TRACE on: 1 rep PASS, 32/36 TRACE
lines per node, signature.json events all populated with the
expected category timestamps.
2026-05-24 17:49:37 +00:00

43 lines
2.1 KiB
YAML

# Compose override that bumps RUST_LOG to trace level on the modules
# relevant to NAT-traversal handshake-completion flake evidence
# collection (ISSUE-2026-0027):
#
# - fips::discovery::nostr — overlay advert publish/consume
# (where the cross-init race begins)
# - fips::transport::udp — UDP socket bind/send/recv
# (where the punch packets flow)
# - fips::node::lifecycle — daemon bootstrap, peer state
# machine, adoption transitions
# - fips::node::handlers::handshake — Noise handshake msg1/2/3,
# cross-init tie-breaker
# ("Ignoring established NAT
# traversal..." emits here)
# - fips::node::handlers::forwarding — transit/local datagram routing
# (catches drops post-adoption)
#
# Other modules stay at info to keep log volume manageable. The base
# docker-compose.yml's `*fips-common` env block sets RUST_LOG as a
# list-form env var (`- RUST_LOG=...`); the override below replaces
# it entirely via the YAML map form, which is the docker-compose
# override semantics for the environment field.
#
# Service-name layout: only the two `lan-*` services are FIPS daemons;
# the relay and stun services are not (and don't honor RUST_LOG).
#
# Include this override via the FIPS_NAT_EXTRA_COMPOSE env var that
# testing/nat/scripts/nat-test.sh consults:
# FIPS_NAT_EXTRA_COMPOSE=testing/mesh-lab/compose-trace-nat.yml \
# bash testing/nat/scripts/nat-test.sh lan
#
# The mesh-lab harness sets it automatically when FIPS_MESH_LAB_TRACE
# is set and the suite is nat-lan.
x-trace-rust-log: &trace-rust-log
RUST_LOG: "info,fips::discovery::nostr=trace,fips::transport::udp=trace,fips::node::lifecycle=trace,fips::node::handlers::handshake=trace,fips::node::handlers::forwarding=trace"
services:
lan-a:
environment: *trace-rust-log
lan-b:
environment: *trace-rust-log