Commit Graph
3 Commits
Author SHA1 Message Date
Johnathan Corgan cfb087a95d Update design docs for heartbeat, auto-reconnect, handshake retry, and sim improvements
fips-link-layer.md:
- Rewrite Liveness Detection: explicit Heartbeat (0x51) with 10s interval
  and 30s dead timeout replaces vague gossip-as-heartbeat description
- Add Auto-Reconnect section: MMP dead timeout triggers retry with
  unlimited backoff for auto_reconnect peers
- Add Handshake Message Retry section: link + session layer resend with
  exponential backoff within timeout window
- Add Heartbeat to Link Message Types table
- Update Implementation Status with three new implemented features

fips-configuration.md:
- Add handshake_resend_interval_ms, handshake_resend_backoff,
  handshake_max_resends to rate_limit table
- Add heartbeat_interval_secs, link_dead_timeout_secs to general table
- Add peers[].auto_reconnect to peers table
- Note auto-reconnect bypasses max_retries in retry section
- Update complete reference YAML with all new parameters

fips-wire-formats.md:
- Rename 0x51 from reserved Keepalive to implemented Heartbeat
- Update Disconnect reason 0x07 to Heartbeat liveness timeout

testing/chaos/README.md:
- Add runner.log to output files
- Add Directed Outbound Configs subsection
2026-02-21 13:10:02 +00:00
Johnathan Corgan 78a73e1749 Auto-reconnect after MMP peer removal, directed outbound configs, sim improvements
Auto-reconnect:
- Add per-peer auto_reconnect config (default true) to PeerConfig
- schedule_reconnect() feeds removed peers back into retry system with
  unlimited retries and exponential backoff after MMP dead timeout
- RetryState gains reconnect flag to distinguish startup retries
  (max_retries-limited) from auto-reconnect (unlimited)

Retry re-fire fix:
- process_pending_retries() now pushes retry_after_ms past the handshake
  timeout window after successful initiate_peer_connection(), preventing
  retries from firing every tick with no backoff

Chaos sim improvements:
- Directed outbound configs: BFS spanning tree + lower-ID-first assignment
  eliminates dual-connect race conditions in simulation
- Save runner log (runner.log) alongside per-node logs for event correlation
- Increase churn-20 traffic aggressiveness and node churn (max_down_nodes
  3→5, traffic interval min 0s, duration max 90s, concurrent flows 5→10)
2026-02-21 13:00:09 +00:00
Johnathan Corgan 66c268a564 Add static and stochastic Docker test harnesses
Move examples/docker-network/ to testing/static/ and add testing/chaos/
as a new stochastic simulation harness.

testing/static/ — Static 5-node test harness:
- Fixed mesh, chain, and mesh-public topologies with docker compose
- Manual test scripts (ping, iperf, netem)
- Build script, config generation, key derivation

testing/chaos/ — Stochastic network simulation:
- Python orchestrator generating N-node FIPS meshes with dynamic
  network conditions, driven by reproducible YAML scenarios
- Topology generation: random geometric, Erdos-Renyi, or chain
  graphs with BFS connectivity guarantee
- Per-link netem: HTB classful qdiscs with u32 filters for per-peer
  impairment (delay, loss, jitter), stochastic mutation across
  configurable policy profiles
- Per-link bandwidth pacing: HTB rate limiting with configurable
  tiers (1/10/100/1000 mbps) randomly assigned per edge
- Link flaps: tc netem 100% loss with graph connectivity protection
- Node churn: docker stop/start with netem re-application on restart,
  shared down_nodes tracking across all managers
- Traffic generation: random iperf3 sessions between node pairs
- Down-node guards: all docker exec callers check container liveness,
  auto-detect crashed containers via is_container_running() safety net
- Log collection and post-run analysis (panics, errors, sessions,
  MMP metrics, tree reconvergence)
- chaos.sh wrapper with --seed, --duration, --verbose, --list options
- Four scenarios: smoke-10, chaos-10, churn-10, churn-20
2026-02-20 13:35:57 +00:00