Record that churn-mixed's floors describe the invocation CI overrides

The baseline floors added for this scenario were calibrated from archived
runs, and those runs are all the 10-node 120-second variant ci-local
invokes as "churn-mixed --nodes 10 --duration 120". The file itself
defaults to 20 nodes and 600 seconds, and --nodes sed-patches num_nodes
in a copy before the scenario loads, so the gating run and a bare
chaos.sh run are different scenarios. Nothing at the site said so.

The floors hold for both, but only because the gating run is the smaller
one. Retuning them against a bare 20-node run, which clears them easily
at 20 answering, 1 root, 19 parented and 69 sessions, would put them past
what the 10-node run reaches and turn CI red while a manual run stayed
green. Say that where someone retuning them will read it.

Confirmed by running the gating invocation directly: 10 answered, 2
roots, 8 parented, 18 sessions, all inside the ranges the floors were
built from. churn-mixed is the only scenario CI overrides this way; the
other twelve run their files as written.
This commit is contained in:
Johnathan Corgan
2026-07-23 07:27:16 +00:00
parent 3d0a388511
commit 38a60c61da
+14
View File
@@ -81,6 +81,20 @@ bandwidth:
# so an unlucky run does not go red, which leaves them catching a mesh
# that collapsed rather than one that churned. Tighten them only
# against a larger sample than six runs.
#
# READ THIS BEFORE RETUNING: the numbers above describe the invocation CI
# gates on, which is not this file's own defaults. ci-local runs it as
# "churn-mixed --nodes 10 --duration 120", and --nodes sed-patches
# num_nodes in a copy of this file before the scenario is loaded, so the
# gating run is a 10-node 120-second one while a bare `chaos.sh
# churn-mixed` runs the 20-node 600-second scenario written here. This is
# the only scenario CI overrides that way.
#
# The floors hold for both but are calibrated for the smaller. A bare
# 20-node run clears them easily: 20 answering, 1 root, 19 parented, 69
# sessions, measured 2026-07-23. Retuning against numbers like those would
# put them past what the 10-node gating run can reach, and CI would go red
# while a manual run stayed green.
assertions:
baseline:
min_nodes_reporting: 10