Fix FSP rekey cutover race and MMP metric discontinuity

The FSP XK rekey handshake has a race condition where the initiator
can cut over (K-bit flip) and send data encrypted with the new session
before msg3 reaches the responder. The responder has no pending session
yet, so K-bit detection fails and packets are dropped.

Defer FSP initiator cutover by 2 seconds after handshake completion
(FSP_CUTOVER_DELAY_MS) to give msg3 time to traverse the mesh. FMP
(IK, 2 messages) is unaffected since the responder completes during
msg1 processing.

Also fix MMP metric corruption after rekey cutover: the new session
starts with counter 0 but MMP state carries highest_counter and
GapTracker.expected_next from the old session, producing false
reorder counts, jitter spikes, and invalid OWD trends. Add
reset_for_rekey() methods that clear counter-dependent state while
preserving RTT estimates.

Additional fixes:
- Remove stale peers_by_index entry on abandon_rekey error path
- Replace redundant peers_by_index inserts with debug assertions
  verifying the pre-registration invariant
- Tighten rekey integration test to zero tolerance (was 4 failures)
This commit is contained in:
Johnathan Corgan
2026-03-15 17:55:35 +00:00
parent 6c90cf6c02
commit 959134657f
11 changed files with 127 additions and 30 deletions
+4 -14
View File
@@ -63,9 +63,6 @@ FIRST_REKEY_WAIT=40 # > REKEY_AFTER_SECS, allow margin
REKEY_SETTLE=5 # settle time after rekey for cutover to complete
SECOND_REKEY_WAIT=40 # wait for second cycle
# Phase 3 allows transient failures during cutover; Phase 5 must be clean
MAX_PHASE3_FAILURES=4 # allow up to 4 pair failures during first rekey
TIMEOUT=5
PASSED=0
FAILED=0
@@ -199,20 +196,11 @@ assert_min_count "Rekey cutover complete (initiator), K-bit flipped" 1 "FMP reke
phase_result "FMP rekey events"
echo ""
# Verify connectivity after first rekey (allow transient cutover failures)
# Verify connectivity after first rekey (strict — no failures allowed)
echo "Phase 3: Post-rekey connectivity (settling ${REKEY_SETTLE}s)"
sleep "$REKEY_SETTLE"
ping_all
if [ "$FAILED" -le "$MAX_PHASE3_FAILURES" ]; then
if [ "$FAILED" -gt 0 ]; then
echo " (transient failures within threshold: $FAILED <= $MAX_PHASE3_FAILURES)"
fi
TOTAL_PASSED=$((TOTAL_PASSED + PASSED))
# Don't count transient failures toward total
echo " ✓ Post-first-rekey (all 20 pairs): $PASSED passed, $FAILED transient"
else
phase_result "Post-first-rekey (all 20 pairs)"
fi
phase_result "Post-first-rekey (all 20 pairs)"
echo ""
# ── Phase 4: Wait for second rekey cycle ──────────────────────────────
@@ -247,6 +235,8 @@ assert_zero_count "MMP link teardown" "Spurious link teardowns"
assert_zero_count "Excessive decrypt failures" \
"Excessive decrypt failure removals"
assert_zero_count "Rekey msg2 processing failed" "Rekey msg2 failures"
assert_zero_count "Session AEAD decryption failed" \
"FSP decryption failures during rekey"
phase_result "Log analysis"
echo ""