mirror of
https://github.com/jmcorgan/fips.git
synced 2026-07-30 19:46:15 +00:00
Johnathan Corgan
24bf4d46d4
The root-count ceiling of 4 was set from six runs whose observed maximum was 3. Fourteen runs that recorded a baseline verdict put the real distribution at 1 to 5 roots, so the ceiling sat at roughly the 93rd percentile of the scenario's own variance: one sample in fourteen exceeded it and four sat at or above it. It therefore reddened a share of runs whatever the daemon did, which is a calibration fault rather than a convergence one. Every sample ran the file's fixed seed, so the spread is container timing and not a differing chaos schedule. The ceiling is now 6, one step beyond the observed maximum, and the parented floor moves to its complement at 4. That still fails a mesh where seven or more of ten nodes are islanded, which is the collapse case the assertion exists to catch; driving evaluate_baseline directly confirms all fourteen samples pass and 7, 8 and 10 roots still fail. min_sessions is left at 10 against an observed minimum of 12. It has never fired, but the comment now records the margin as thin so a future failure there is read as calibration first.
133 lines
4.4 KiB
YAML
133 lines
4.4 KiB
YAML
# Parameterized churn test with mixed transports (UDP, Ethernet, TCP)
|
|
#
|
|
# Default: 20 nodes. Override num_nodes with --nodes flag in chaos.sh,
|
|
# or copy this file and adjust topology.num_nodes directly.
|
|
#
|
|
# Tests spanning tree convergence, link flaps, node churn, and traffic
|
|
# across heterogeneous transports at configurable scale.
|
|
|
|
scenario:
|
|
name: "churn-mixed"
|
|
seed: 42
|
|
duration_secs: 600
|
|
|
|
topology:
|
|
num_nodes: 20
|
|
algorithm: erdos_renyi
|
|
params:
|
|
p: 0.3
|
|
ensure_connected: true
|
|
subnet: "172.20.0.0/16"
|
|
ip_start: 10
|
|
transport_mix:
|
|
udp: 0.6
|
|
ethernet: 0.2
|
|
tcp: 0.2
|
|
|
|
netem:
|
|
enabled: true
|
|
default_policy:
|
|
delay_ms: { min: 5, max: 50 }
|
|
jitter_ms: { min: 1, max: 10 }
|
|
loss_pct: { min: 0, max: 2 }
|
|
mutation:
|
|
interval_secs: { min: 20, max: 45 }
|
|
fraction: 0.3
|
|
policies:
|
|
normal:
|
|
delay_ms: [5, 20]
|
|
loss_pct: [0, 1]
|
|
degraded:
|
|
delay_ms: [50, 100]
|
|
jitter_ms: [10, 30]
|
|
loss_pct: [3, 8]
|
|
|
|
link_flaps:
|
|
enabled: true
|
|
interval_secs: { min: 30, max: 60 }
|
|
max_down_links: 3
|
|
down_duration_secs: { min: 10, max: 30 }
|
|
protect_connectivity: true
|
|
|
|
traffic:
|
|
enabled: true
|
|
max_concurrent: 10
|
|
interval_secs: { min: 0, max: 30 }
|
|
duration_secs: { min: 5, max: 90 }
|
|
parallel_streams: 4
|
|
|
|
node_churn:
|
|
enabled: true
|
|
interval_secs: { min: 60, max: 90 }
|
|
max_down_nodes: 5
|
|
down_duration_secs: { min: 30, max: 90 }
|
|
protect_connectivity: false
|
|
|
|
bandwidth:
|
|
enabled: true
|
|
tiers_mbps: [1, 10, 100, 1000]
|
|
|
|
# Baseline: the mesh came up, agreed on a root, and took parents. This
|
|
# asserts nothing about which nodes survive the churn; it exists so that
|
|
# a run in which the mesh never formed cannot report success, which
|
|
# until now it could, because this scenario carried no assertions at
|
|
# all.
|
|
#
|
|
# This one is calibrated rather than derived, because a scenario that
|
|
# stops and starts nodes on purpose does not hold a single spanning
|
|
# tree.
|
|
#
|
|
# Calibrated 2026-07-26 against fourteen runs that recorded a baseline
|
|
# verdict, read from their assertions.txt files. Every one used this
|
|
# file's fixed seed 42 and therefore an identical chaos schedule, so the
|
|
# spread below is container timing rather than differing scenarios:
|
|
#
|
|
# distinct roots 1 2 3 4 5 -> 2 3 5 3 1 runs
|
|
# nodes parented 9 8 7 6 5 -> the exact complement, in all 14
|
|
# sessions 12 to 20, minimum 12
|
|
#
|
|
# The previous ceiling of 4 roots sat at roughly the 93rd percentile of
|
|
# that distribution: one run in fourteen exceeded it and four sat at or
|
|
# above it, so it reddened a share of runs whatever the daemon did. It
|
|
# was set from six runs whose observed maximum was 3, which is how a
|
|
# threshold one step outside a small sample ends up inside the real one.
|
|
#
|
|
# The ceiling is now 6, one step beyond the observed maximum of 5, and
|
|
# the parented floor is its complement at 4. That still fails a mesh
|
|
# that collapsed — seven or more of ten nodes islanded — which is the
|
|
# only thing this assertion was ever meant to catch. Do not read a pass
|
|
# as convergence: four of the six gating scenarios assert a floor of
|
|
# this kind, and it means the mesh formed and nobody errored.
|
|
#
|
|
# min_sessions stays at 10 against an observed minimum of 12. It has
|
|
# never fired and there is no evidence it is mis-set, but the margin is
|
|
# thin and a future failure there should be read as calibration before
|
|
# it is read as a defect.
|
|
#
|
|
# Tighten any of these only against a larger sample, and against one
|
|
# gathered at the invocation CI actually runs.
|
|
#
|
|
# READ THIS BEFORE RETUNING: the numbers above describe the invocation CI
|
|
# gates on, which is not this file's own defaults. ci-local runs it as
|
|
# "churn-mixed --nodes 10 --duration 120", and --nodes sed-patches
|
|
# num_nodes in a copy of this file before the scenario is loaded, so the
|
|
# gating run is a 10-node 120-second one while a bare `chaos.sh
|
|
# churn-mixed` runs the 20-node 600-second scenario written here. This is
|
|
# the only scenario CI overrides that way.
|
|
#
|
|
# The floors hold for both but are calibrated for the smaller. A bare
|
|
# 20-node run clears them easily: 20 answering, 1 root, 19 parented, 69
|
|
# sessions, measured 2026-07-23. Retuning against numbers like those would
|
|
# put them past what the 10-node gating run can reach, and CI would go red
|
|
# while a manual run stayed green.
|
|
assertions:
|
|
baseline:
|
|
min_nodes_reporting: 10
|
|
max_roots: 6
|
|
min_nodes_parented: 4
|
|
min_sessions: 10
|
|
|
|
logging:
|
|
rust_log: "debug"
|
|
output_dir: "./sim-results"
|