Files
fips/testing/chaos/scenarios/congestion-stress.yaml
T
Johnathan Corgan 08547e8dff Merge maint: retire cost/congestion chaos scenarios
Bring up the retirement of the six cost-based parent-selection chaos
scenarios (cost-reeval, cost-avoidance, cost-stability, depth-vs-cost,
mixed-technology, bottleneck-parent), which tested a decision the Docker
harness could not exercise reliably.

master already carries the equivalent sans-IO coverage in
src/proto/stp/tests (effective-depth cost selection, hysteresis, cost
degradation) and its transport-drop tests, so this keeps master's src
unchanged and takes only the scenario removals and the CI-list, README and
scenario-comment updates from maint.
2026-07-23 23:44:31 +00:00

160 lines
5.3 KiB
YAML

# Congestion Stress: exercise both MMP-based and kernel-drop congestion detection
#
# Topology: 10-node tree with all links bandwidth-limited to 1 Mbps
# (egress HTB + ingress policing). Heavy iperf3 traffic (8 concurrent
# sessions, 8 parallel streams) combined with 5-10% netem loss.
#
# Congestion detection signals exercised:
# 1. MMP loss detection: netem loss exceeds the 5% loss_threshold,
# triggering detect_congestion() via MMP metrics on transit nodes.
# 2. Ingress policing: tc policer on the receive side drops excess
# inbound packets, creating bursty arrival patterns.
# 3. ECN CE marking under the shaped bottleneck queue.
#
# The kernel socket-drop signal (SO_RXQ_OVFL) is NOT exercised here. This
# scenario's 1 Mbps cap and ingress policer, which its ECN/MMP signals
# require, make socket overflow impossible. It also cannot be provoked
# deterministically anywhere in Docker — a fresh daemon reader keeps up
# with container-speed traffic — so the FIPS drop-detection logic is
# covered by transport-drop unit tests instead. See the note at the
# assertions block below.
#
# ECN is explicitly enabled via fips_overrides.
#
# Success criteria (verified via post-run congestion snapshot):
# - congestion_detected > 0 on at least one forwarding node
# - ce_forwarded > 0 on transit nodes
# - ce_received > 0 on destination nodes
#
# Topology:
#
# n01 (root)
# / \
# n02 n03
# / \ / \
# n04 n05 n06 n07
# | |
# n08 n09
# |
# n10
scenario:
name: "congestion-stress"
seed: 7
duration_secs: 120
topology:
algorithm: explicit
num_nodes: 10
params:
adjacency:
- [n01, n02]
- [n01, n03]
- [n02, n04]
- [n02, n05]
- [n03, n06]
- [n03, n07]
- [n04, n08]
- [n05, n09]
- [n08, n10]
subnet: "172.20.0.0/24"
ip_start: 10
bandwidth:
enabled: true
tiers_mbps: [1]
ingress:
enabled: true
tiers_kbps: [1000]
burst_bytes: 16000
netem:
enabled: true
default_policy:
delay_ms: [2, 5]
jitter_ms: [0, 1]
loss_pct: [5, 10]
mutation:
interval_secs: {min: 9999, max: 9999}
fraction: 0.0
policies:
baseline:
delay_ms: [2, 5]
loss_pct: [5, 10]
link_flaps:
enabled: false
traffic:
enabled: true
max_concurrent: 8
interval_secs: {min: 2, max: 5}
duration_secs: {min: 30, max: 60}
parallel_streams: 8
node_churn:
enabled: false
# Three of the four success criteria above, encoded. Until now all four
# existed only as that comment and were checked by nothing, so the scenario
# could not fail on any of them.
#
# The floors are 1 node each because that is what the criteria say ("on at
# least one forwarding node", "on transit nodes", "on destination nodes").
# They are deliberately not tightened to the observed counts: the criterion
# is the spec, and a floor invented from six runs would assert something
# nobody wrote down.
#
# What the floors are worth, from the archived corpus. The six runs that
# carry a status.txt -- the only ones provably completed, all between
# 2026-07-22 22:14 and 2026-07-23 02:02 -- meet all three with 5 to 9 nodes
# reporting each signal, so the margin over a floor of 1 is comfortable.
# The 176 older runs meet none of them. Those older runs did reach teardown
# (each wrote an analysis.txt), and ECN landed 2026-03-05, before all but
# one of them, so they are valid runs that observed no congestion rather
# than runs that died early. What changed on 2026-07-22 is not established:
# this file has not been touched since it was created, and neither have
# netem.py, traffic.py or control.py. Worth knowing before trusting a green
# result here, and worth its own investigation.
assertions:
congestion_signals:
min_nodes_detected: 1
min_nodes_ce_forwarded: 1
min_nodes_ce_received: 1
# The kernel socket-drop criterion is NOT asserted here and no longer
# belongs to this scenario. The FIPS drop-DETECTION logic is covered by
# transport-drop unit tests (2026-07-23); the kernel dropping datagrams is a
# kernel behaviour, not FIPS's to test, and could not be provoked in Docker.
#
# Why it could never be met here: across all 182 archived runs of this
# scenario, including the six that meet the three signals above, no node
# ever reported a non-zero kernel_drop_events. SO_RXQ_OVFL counts datagrams
# arriving at a FULL receive queue, and the 1 Mbps cap sets the arrival rate
# to 125 kB/s per link. Linux doubles a requested SO_RCVBUF, so the queue is
# 8192 bytes and filling it would take ~65 ms of reader stall (~22 ms even at
# the busiest node's 3 Mbps aggregate). Traffic volume cannot cause the
# overflow because the volume is capped below the rate the buffer drains, and
# the ingress policer discards excess in tc before the socket ever sees it.
# An unshaped attempt (congestion-drops, 2026-07-23) recorded zero raw drops
# on every node even with a 4 KB buffer and heavy iperf: a fresh daemon
# reader keeps up, so the overflow cannot be provoked deterministically. That
# is why the detection edge is tested as a sans-IO unit test.
#
# The recv_buf_size: 4096 override below is kept because the three asserted
# signals were validated with it in place; it is inert for socket overflow
# under this scenario's cap.
logging:
rust_log: "info"
output_dir: "./sim-results"
fips_overrides:
node:
ecn:
enabled: true
transports:
udp:
recv_buf_size: 4096