mirror of
https://github.com/jmcorgan/fips.git
synced 2026-07-30 19:46:15 +00:00
Johnathan Corgan
d4a2504f99
Found while fixing count_log_pattern: the gate reported that function clean even though it ends in echo and both callers consume its status. Its call-site patterns only matched direct forms -- if fn, while fn, fn ||, fn && -- so a status consumed through command substitution was invisible, because the line begins with the variable rather than the function name. That is not an exotic form. It is how a shell function returns a value, and it is precisely the shape of the swallowed-failure family this gate exists to catch, so the gate was blind to a large part of its own stated class. Three patterns added for the assignment, if-guarded and test-expression forms. The extension finds four real instances, all value-returning helpers whose trailing echo made their exit status unconditionally zero, and each now carries an explicit return 0. Also corrects the finding message, which described the trailing command as a log call; for these it is the return mechanism, and the hazard is that it fixes the status either way. Validated by breaking what it guards: a probe reintroducing the defect shape behind a command substitution is reported and exits 1, where before the extension it would have passed.
167 lines
5.8 KiB
YAML
167 lines
5.8 KiB
YAML
# Congestion Stress: exercise both MMP-based and kernel-drop congestion detection
|
|
#
|
|
# Topology: 10-node tree with all links bandwidth-limited to 1 Mbps
|
|
# (egress HTB + ingress policing). Heavy iperf3 traffic (8 concurrent
|
|
# sessions, 8 parallel streams) combined with 5-10% netem loss.
|
|
#
|
|
# Congestion detection signals exercised:
|
|
# 1. MMP loss detection: netem loss exceeds the 5% loss_threshold,
|
|
# triggering detect_congestion() via MMP metrics on transit nodes.
|
|
# 2. Kernel socket drops: small recv_buf_size (8KB) combined with
|
|
# traffic saturation causes SO_RXQ_OVFL on the UDP socket,
|
|
# triggering the transport drop detection path.
|
|
# 3. Ingress policing: tc policer on the receive side drops excess
|
|
# inbound packets, creating bursty arrival patterns.
|
|
#
|
|
# ECN is explicitly enabled via fips_overrides.
|
|
#
|
|
# Success criteria (verified via post-run congestion snapshot):
|
|
# - congestion_detected > 0 on at least one forwarding node
|
|
# - kernel_drop_events > 0 on at least one node
|
|
# - ce_forwarded > 0 on transit nodes
|
|
# - ce_received > 0 on destination nodes
|
|
#
|
|
# Topology:
|
|
#
|
|
# n01 (root)
|
|
# / \
|
|
# n02 n03
|
|
# / \ / \
|
|
# n04 n05 n06 n07
|
|
# | |
|
|
# n08 n09
|
|
# |
|
|
# n10
|
|
|
|
scenario:
|
|
name: "congestion-stress"
|
|
seed: 7
|
|
duration_secs: 120
|
|
|
|
topology:
|
|
algorithm: explicit
|
|
num_nodes: 10
|
|
params:
|
|
adjacency:
|
|
- [n01, n02]
|
|
- [n01, n03]
|
|
- [n02, n04]
|
|
- [n02, n05]
|
|
- [n03, n06]
|
|
- [n03, n07]
|
|
- [n04, n08]
|
|
- [n05, n09]
|
|
- [n08, n10]
|
|
subnet: "172.20.0.0/24"
|
|
ip_start: 10
|
|
|
|
bandwidth:
|
|
enabled: true
|
|
tiers_mbps: [1]
|
|
|
|
ingress:
|
|
enabled: true
|
|
tiers_kbps: [1000]
|
|
burst_bytes: 16000
|
|
|
|
netem:
|
|
enabled: true
|
|
default_policy:
|
|
delay_ms: [2, 5]
|
|
jitter_ms: [0, 1]
|
|
loss_pct: [5, 10]
|
|
mutation:
|
|
interval_secs: {min: 9999, max: 9999}
|
|
fraction: 0.0
|
|
policies:
|
|
baseline:
|
|
delay_ms: [2, 5]
|
|
loss_pct: [5, 10]
|
|
|
|
link_flaps:
|
|
enabled: false
|
|
|
|
traffic:
|
|
enabled: true
|
|
max_concurrent: 8
|
|
interval_secs: {min: 2, max: 5}
|
|
duration_secs: {min: 30, max: 60}
|
|
parallel_streams: 8
|
|
|
|
node_churn:
|
|
enabled: false
|
|
|
|
# Three of the four success criteria above, encoded. Until now all four
|
|
# existed only as that comment and were checked by nothing, so the scenario
|
|
# could not fail on any of them.
|
|
#
|
|
# The floors are 1 node each because that is what the criteria say ("on at
|
|
# least one forwarding node", "on transit nodes", "on destination nodes").
|
|
# They are deliberately not tightened to the observed counts: the criterion
|
|
# is the spec, and a floor invented from six runs would assert something
|
|
# nobody wrote down.
|
|
#
|
|
# What the floors are worth, from the archived corpus. The six runs that
|
|
# carry a status.txt -- the only ones provably completed, all between
|
|
# 2026-07-22 22:14 and 2026-07-23 02:02 -- meet all three with 5 to 9 nodes
|
|
# reporting each signal, so the margin over a floor of 1 is comfortable.
|
|
# The 176 older runs meet none of them. Those older runs did reach teardown
|
|
# (each wrote an analysis.txt), and ECN landed 2026-03-05, before all but
|
|
# one of them, so they are valid runs that observed no congestion rather
|
|
# than runs that died early. What changed on 2026-07-22 is not established:
|
|
# this file has not been touched since it was created, and neither have
|
|
# netem.py, traffic.py or control.py. Worth knowing before trusting a green
|
|
# result here, and worth its own investigation.
|
|
assertions:
|
|
congestion_signals:
|
|
min_nodes_detected: 1
|
|
min_nodes_ce_forwarded: 1
|
|
min_nodes_ce_received: 1
|
|
|
|
# The fourth criterion, "kernel_drop_events > 0 on at least one node", is
|
|
# NOT asserted, because this scenario has never met it: across all 182
|
|
# archived runs, including the six that meet the other three, no node has
|
|
# reported a non-zero kernel_drop_events. Asserting it would red the scenario
|
|
# permanently, and asserting a weakened version would assert nothing.
|
|
#
|
|
# CORRECTED 2026-07-23. An earlier version of this note concluded that the
|
|
# transport drop-detection path "is exercised by nothing". That is wrong, and
|
|
# the error was one of scope: it looked only inside this scenario. Widening
|
|
# the scan to every scenario's congestion snapshots -- 2317 files, 11778 node
|
|
# records -- finds 51 records reporting kernel_drop_events > 0 across six
|
|
# other scenarios (depth-vs-cost, bottleneck-parent, tcp-mesh, cost-avoidance,
|
|
# maelstrom-sparse, mixed-technology), with raw drop counts up to 41165. The
|
|
# path is live and plumbed through to the snapshot. What is dead is this
|
|
# scenario's ability to provoke it.
|
|
#
|
|
# The inversion is the lead: every scenario that DOES overflow runs with the
|
|
# default 2 MB recv_buf_size, while this one, the only scenario that shrinks
|
|
# it to 4 KB, never does. SO_RXQ_OVFL counts datagrams arriving at a full
|
|
# queue, and the 1 Mbps cap below sets the arrival rate to 125 kB/s per link.
|
|
# Linux doubles a requested SO_RCVBUF, so the queue is 8192 bytes and filling
|
|
# it takes ~65 ms of reader stall -- ~22 ms even at the busiest node's 3 Mbps
|
|
# aggregate. Traffic volume cannot cause the overflow because the volume is
|
|
# capped below the rate the buffer drains. The ingress policer works against
|
|
# the same signal, discarding excess in tc before the socket ever sees it.
|
|
#
|
|
# So signal 3 appears to suppress signal 2, and the two may not belong in one
|
|
# scenario. Tracked with the experiment that would settle it (raise or drop
|
|
# the bandwidth cap for one run and watch the counter) in ISSUE-2026-0084.
|
|
#
|
|
# Note also that the header above says "small recv_buf_size (8KB)" while
|
|
# fips_overrides sets 4096. The 8 KB figure matches what Linux allocates after
|
|
# doubling, so the comment may be describing the effective queue rather than
|
|
# the setting; it reads as the setting and should say which it means.
|
|
|
|
logging:
|
|
rust_log: "info"
|
|
output_dir: "./sim-results"
|
|
|
|
fips_overrides:
|
|
node:
|
|
ecn:
|
|
enabled: true
|
|
transports:
|
|
udp:
|
|
recv_buf_size: 4096
|