diff --git a/testing/chaos/scenarios/cost-reeval.yaml b/testing/chaos/scenarios/cost-reeval.yaml index 51d2f08..734d8f4 100644 --- a/testing/chaos/scenarios/cost-reeval.yaml +++ b/testing/chaos/scenarios/cost-reeval.yaml @@ -50,10 +50,19 @@ # `topology.pin_root` (default true) now assigns identities in NodeAddr order # so n01 is root and n04 is the depth-1 subject this file describes. # -# STILL TO DO: this scenario has no assertion about its own subject, and that -# was a consequence rather than an omission — none could have passed. With the -# subject now reachable, decide whether the switch is reliable enough across -# seeds to assert, and encode it if so. Tracked in ISSUE-2026-0085. +# MEASURED after the fix, 2026-07-23: the periodic switch now happens, in 2 of +# 3 runs on n04 (0, 1, 1 occurrences). Before the fix it was 0 in 5 of 5, and +# structurally impossible rather than merely rare. So the subject is exercised +# for the first time. +# +# STILL NOT ASSERTED, deliberately. A "at least one periodic switch" assertion +# would red roughly one run in three on this evidence, and a single run reading +# zero is an argument against asserting rather than for it. Two ways forward, +# neither taken here: gather enough runs to know whether 2-in-3 is the real +# rate, or reduce the variance -- the mutation is stochastic with fraction=0.5, +# so a deterministic link-degradation schedule would make the asymmetry this +# scenario needs occur every run and the outcome assertable. +# Tracked in ISSUE-2026-0085. scenario: name: "cost-reeval" diff --git a/testing/chaos/scenarios/mixed-technology.yaml b/testing/chaos/scenarios/mixed-technology.yaml index d6cb1f3..fb9afd5 100644 --- a/testing/chaos/scenarios/mixed-technology.yaml +++ b/testing/chaos/scenarios/mixed-technology.yaml @@ -26,8 +26,21 @@ # Fiber: n03-n08, n06-n10 # # Test subjects: -# - n06 has fiber (n03) and Bluetooth (n02) parents — should pick n03 -# - n08 has fiber (n03) and Bluetooth (n04) parents — should pick n03 +# - n08 has fiber (n03) and Bluetooth (n04) parents — should pick n03. +# ASSERTED below. Holds in 4 of 4 runs under the pinned root. +# - n06 was documented as having "fiber (n03) and Bluetooth (n02) parents". +# CORRECTED 2026-07-23: that is wrong, and the netem block below is the +# evidence rather than the ASCII art above. `n03-n06` is a **WiFi** link +# (5-20ms, 1-3% loss) and `n02-n06` is Bluetooth (15-40ms, 2-8% loss), so +# n06 chooses between two impaired links, not between fiber and Bluetooth. +# n06's only fiber-grade link is `n06-n10`, and n10 sits at depth 3 so it +# is never a good parent. There is therefore no reason to expect n06 to +# prefer n03, and the mutation block (fraction 0.2, degraded 50-100ms) +# moves the two costs run to run: across four runs n06 took n03 twice and +# n02 twice. NOT asserted, and the observed split is the correct outcome +# rather than a defect — in the run checked, n03 measured a link cost of 3 +# against n02's 1, so effective depth 1+3 beat by 1+1 and the daemon chose +# the cheaper path exactly as it should. # # Netem mutation shifts fiber-only links between normal and degraded. # @@ -142,6 +155,19 @@ assertions: max_roots: 1 min_nodes_parented: 9 min_sessions: 3 + # The scenario's own subject, assertable for the first time now that the + # root is pinned to n01 and the topology above describes the tree that + # forms. n08's two candidates sit at equal depth, so the choice is decided + # by link cost alone: n03 over fiber-grade default netem against n04 over + # Bluetooth. That is the fiber-versus-Bluetooth preference this scenario + # exists to test. + # + # The criterion is encoded as written rather than calibrated from runs: + # "should pick n03" is the spec, and four of four runs agree. n06 is + # deliberately absent — see the corrected note in the header for why its + # documented criterion was never founded on this topology. + tree_parents: + n08: n03 logging: rust_log: "info"