9 Commits

Include renames

Author	SHA1	Message	Date
Martti MalmiandJohnathan Corgan	0a5c367edc	data-plane perf overhaul: off-task encrypt + decrypt, GSO, connected UDP ... Moves both AEAD layers (ChaCha20-Poly1305, one round per layer per packet) plus the sendmsg syscall off the rx_loop task onto a per-shard worker pool, adds per-peer connect(2)-ed UDP with SO_REUSEPORT, and uses Linux UDP GSO (sendmsg+UDP_SEGMENT — kernel splits one super-skb into N on-the-wire datagrams in a single TX-stack walk) when packets in a batch are uniform-size. Same kernel primitive WireGuard's in-kernel module and BoringTun use to hit 2.5–3.2 Gbps single-stream. Single TCP stream on a 5-node docker-bridge mesh, 5 x 15 s x P=1: A→D: 1379 → 2708 Mbps (1.96x, RTT +0.12 ms) A→E: 1394 → 2663 Mbps (1.91x, RTT +0.11 ms) E→A: 1406 → 2624 Mbps (1.87x, RTT +0.19 ms) Static-peer pairs only — every CoV under 3%, 0 outliers, 0% ICMP loss. The ~+100 µs RTT is the worker queue handoff cost; AEAD + sendmmsg now run on a separate core in exchange. What lands: - src/node/encrypt_worker.rs: std::thread + crossbeam_channel workers; hash-by-destination dispatch pins a TCP flow to one worker so wire ordering is preserved; per-worker sendmmsg(2) batching up to 32; Linux uses sendmsg(2)+UDP_SEGMENT when packets in a group are uniform-size. - src/node/decrypt_worker.rs: receive-side mirror. Each shard owns its session's recv cipher + replay window in a thread-local HashMap (no shared RwLock/Mutex). Sessions are handed off at promote_connection and re-registered on K-bit flip / rekey cutover. - src/node/handlers/session.rs try_send_session_data_pipelined: FSP+FMP both seal in-place in the worker on one wire-buffer alloc; no intermediate inner_plaintext / fsp_payload Vecs. - src/transport/udp/connected_peer.rs + peer_drain.rs: per-peer connect(2)-ed UDP socket with SO_REUSEPORT (set on the listen socket too — without that, EADDRINUSE on activation and every packet falls back to the wildcard path); the worker sends with msg_name=NULL and the kernel uses its cached 5-tuple. Tick- driven activation in handlers/connected_udp.rs, idempotent. - src/transport/udp/mod.rs: mem::replace the recvmmsg backing buffer instead of buf.to_vec() per packet — single pointer swap, no MTU-sized memcpy. - src/protocol/link.rs SessionDatagramRef: zero-copy borrowed view used by handle_session_datagram for the bulk local-delivery path; handle_session_payload takes the borrowed payload directly (no payload[35..].to_vec()). - src/transport/mod.rs TransportAddr::from_socket_addr: collapses the two-alloc from_string(addr.to_string()) pattern to one. - src/node/handlers/rx_loop.rs: decrypt-fallback drain promoted ahead of packet_rx in the select! (TCP ACK starvation fix); interleaved fallback drain every 32 packets inside the rx burst loop. - noise::Session: send_cipher_clone / recv_cipher_clone / recv_replay_snapshot_owned / take_send_counter / accept_replay so off-task workers can hold a cloned cipher + reserved counter while the dispatcher keeps replay/counter sequencing serial. CipherState::cipher_clone returns a refcount-bumped LessSafeKey. AsyncUdpSocket: AsRawFd so workers issue raw sendmmsg / sendmsg without going through the tokio reactor. - Worker pool sizing: both default to num_cpus, overridable via FIPS_ENCRYPT_WORKERS=N / FIPS_DECRYPT_WORKERS=N. Per-peer connected UDP can be disabled via FIPS_CONNECTED_UDP=0. - src/perf_profile.rs: optional per-stage timing reporter under FIPS_PERF=1 (or FIPS_PIPELINE_TRACE=1). Off by default; zero overhead when disabled. - All cfg(unix)-gated. Windows continues on the existing tokio- based send/recv. Decrypt worker session lifecycle: - Node::unregister_decrypt_worker_session mirrors the existing register helper. Wired at the two natural sites that already iterate peers_by_index: the rekey drain-completion block in handlers/rekey.rs (drops the worker entry for the old our_index once the drain window has expired and the cache_key is unreachable to any in-flight OLD-K packet), and remove_active_peer in handlers/dispatch.rs (drops the worker entry for each of the four index slots: current, rekey, pending, previous). Only our_index is normally registered; unregister_session is fire- and-forget for missing entries, so calling unconditionally on all four slots is correct and bounds the cleanup without per- slot accounting. Without these callers the per-worker sessions HashMap and the Node's decrypt_registered_sessions set would grow monotonically per rekey on long-lived peers. Testing: - testing/static/scripts/bench-multirun.sh: multi-run iperf3 + ping bench. N reruns (default 5), median / min / max / CoV % / per-run outlier flag, avg ping RTT, ICMP loss %, TCP retransmit total. Plain client→dest labels + topology header. Pre-bench peer-convergence check (FIPS_BENCH_CONVERGE_SECS, default 15); per-path route verification via stats.bytes_sent deltas — fails fast if traffic exits via a non-static-peer link. - testing/static/docker-compose.yml: passes FIPS_ENCRYPT_WORKERS / FIPS_DECRYPT_WORKERS / FIPS_PERF through to containers for A/B benchmarking without rebuilds. - testing/static/scripts/iperf-test.sh: same plain client→dest labels + topology header (was multihop/direct/N hop, which conflated topology distance with on-wire path). - .config/nextest.toml: synthetic UDP node tests serialized through a max-threads=1 test group. Localhost handshakes drop on shared CI runners under parallel load; one-at-a-time keeps assertions reliable. - src/node/tests/spanning_tree.rs: repair_missing_edge_handshakes — retries up to 5 times for synthetic edges whose msg1 was dropped, with a drain after each edge retry instead of after each attempt's full burst. - src/node/decrypt_worker.rs::tests: two unit tests asserting WorkerMsg::UnregisterSession removes the worker-thread session HashMap entry (handle_msg_unregister_session_removes_entry) and is a no-op for never-seen cache_keys (handle_msg_unregister_session_idempotent_on_unknown_key), which is the safety invariant the unconditional unregister calls at the four index slots in remove_active_peer rely on. - src/node/encrypt_worker.rs::unix_tests pipelined_send_wire_layout_roundtrips_canonical_decoders: mirrors the encoder geometry of try_send_session_data_pipelined (no coords, the common established-session path), runs the worker's real seal + send via flush_direct_batch_sync, and decodes the resulting wire packet using only canonical receive-side decoders (EncryptedHeader::parse, SessionDatagramRef::decode, FSP header parse, noise::open). Any divergence between the hand-rolled encoder offsets (fsp_aad_offset, fsp_plaintext_offset) and the decoders fails at one of the parse / open / decode steps before the inner-plaintext assertion fires. Complements the existing fsp_preseal_runs_before_outer_fmp_seal test which covers the seal-ordering invariant with synthetic headers but does not exercise the wire-layout invariant. CHANGELOG.md [Unreleased] # Changed entry added describing the worker-pool threading model, hash-by-destination dispatch, sendmmsg/UDP_GSO, per-peer connected UDP, the operator-facing env vars, and the bench numbers above. Cherry-picks from mmalmi/master (paths translated from crates/fips-core/src/ to src/): 9b7c723, 0deb5cb, 13f7339, e036c0e, 3740a68, 3792f83, 8510193, 4910b07, e53f545, e4e2896, 5fe4af5, 1d01ada, 8c37008, e12469e, 6eb2860. Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>	2026-05-19 20:53:31 +00:00
Johnathan Corgan	6aff490251	Improve discovery protocol: bloom-guided tree routing with fallback ... Replace discovery flooding with bloom-filter-guided tree routing: lookups sent only to tree peers (parent + children) whose bloom filter contains the target. If no tree peer matches, fall back to non-tree peers with bloom matches before dropping the request. This produces single-path forwarding through the spanning tree (90% traffic reduction) while recovering from dead ends caused by stale bloom filters, tree restructuring, or transit node failures. Remove visited bloom filter from LookupRequest wire format (-257 bytes per request). Tree routing is inherently loop-free; request_id dedup handles edge cases during tree restructuring. Add response- forwarded flag to prevent response routing loops from convergent request paths. Add originator-side exponential backoff (30s base, 300s cap) after lookup timeouts and bloom misses. Backoff resets on topology changes (parent switch, new peer, first RTT, reconnection). Add transit-side per-target rate limiting (2s minimum interval) for forwarded lookups as defense-in-depth. Add discovery retry within the timeout window (default: send at T=0, retry at T=5s, fail at T=10s) to compensate for single-path fragility. Lookups with zero eligible tree peers fail immediately. Improve discovery logging: promote key events to info (initiation, success, timeout with failure count). Add debug logging for dedup, pending packet retry, backoff suppression, forward rate limiting, and backoff reset. New config: discovery.backoff_base_secs, backoff_max_secs, forward_min_interval_secs, retry_interval_secs, max_attempts. New stats: req_backoff_suppressed, req_forward_rate_limited, req_bloom_miss, req_no_tree_peer, req_fallback_forwarded. Removed: req_already_visited, visited bloom filter.	2026-03-22 20:02:00 +00:00
Johnathan Corgan	bf117df0ca	Add periodic Noise rekey with fresh DH for forward secrecy (FMP + FSP) ... Implement periodic full rekey at both protocol layers using fresh DH key exchanges. Uses the existing K-bit flag (FLAG_KEY_EPOCH / FSP_FLAG_K) to coordinate cutover between peers. FMP layer (IK pattern): - ActivePeer gains rekey state: pending/previous sessions, K-bit epoch tracking, drain window, dampening timer - Handshake state stored on ActivePeer with msg1 sent on existing link - Encrypted frame handler detects K-bit flips, promotes pending sessions, falls back to previous session during drain - Handshake handlers distinguish rekey from new connections using addr_to_link lookup with identity-based fallback - Free all session indices (current, rekey, pending, previous) on peer removal FSP layer (XK pattern): - SessionEntry gains parallel rekey fields with XK-specific state for the 3-message handshake - Route availability check before FSP rekey initiation - Encrypted session handler adds K-bit flip detection and dual-session decrypt fallback - SessionSetup/Ack/Msg3 handlers extended for rekey paths Defense-in-depth: - Consecutive decryption failure detector (threshold=20) triggers forced peer removal instead of waiting for link-dead timeout - Identity-based rekey detection as fallback when addr_to_link doesn't match (e.g., TCP ephemeral ports) Configuration: RekeyConfig with enabled flag, after_secs (default 120), and after_messages (default 65536) thresholds. Logging: info for successful K-bit cutover completions, warn for failures, debug for intermediate handshake steps, trace for routine operations (resends, drain cleanup). Rekey lifecycle: 1. Timer/counter fires -> initiator starts new handshake 2. Old session continues handling traffic during handshake 3. Handshake completes -> initiator cuts over, flips K-bit 4. Responder sees flipped K-bit -> promotes new session 5. Both keep old session for 10s drain window 6. After drain, old session discarded Integration test: Docker-based multi-phase test exercising both FMP and FSP rekey with aggressive timers (35s). Verifies connectivity across all 20 directed pairs survives two consecutive rekey cycles. Includes rekey topology, docker-compose profile, and CI matrix entry. Increase ping test convergence wait from 3s to 5s for CI reliability.	2026-03-07 18:33:27 +00:00
Johnathan Corgan	56d39f223b	Add ECN congestion signaling and transport congestion detection ... Implement hop-by-hop ECN congestion signaling through the FMP layer, transport-level congestion detection via kernel drop counters, and chaos harness integration for end-to-end validation. FMP/session ECN plumbing: - Thread ce_flag parsed at link layer through dispatch_link_message, handle_session_datagram, handle_session_payload, and handle_encrypted_session_msg to session delivery - Replace hardcoded false in session-layer record_recv() with actual ce_flag, activating ecn_ce_count tracking in session MMP ECN congestion detection and CE relay: - Add EcnConfig (node.ecn.*) with configurable loss_threshold (5%) and etx_threshold (3.0) for transit congestion detection - Add send_encrypted_link_message_with_ce() that ORs FLAG_CE into FMP header flags; original method delegates with ce_flag=false - Compute outgoing_ce = incoming_ce || local congestion on next-hop link, enabling hop-by-hop CE relay through transit nodes IPv6 ECN-CE marking: - Mark ECN-CE (0b11) in IPv6 Traffic Class on received DataPackets before TUN delivery when FMP CE flag is set - Only marks ECN-capable packets (ECT(0)/ECT(1)); Not-ECT packets unchanged per RFC 3168 Transport congestion abstraction and UDP kernel drop detection: - Add TransportCongestion struct to transport layer for transport- agnostic local congestion indicators - Replace tokio::UdpSocket with AsyncFd<socket2::Socket> using libc::recvmsg() with ancillary data parsing - Enable SO_RXQ_OVFL for kernel receive buffer drop counter on every packet, wiring up previously-stubbed UdpStats.kernel_drops - Add TransportDropState for per-transport delta tracking with 1s tick sampling via sample_transport_congestion() - Extend detect_congestion() with transport kernel drop check alongside MMP loss metrics Congestion monitoring and control: - Add CongestionStats (ce_forwarded, ce_received, congestion_detected, kernel_drop_events) to NodeStats with snapshot serialization - Wire counters into forwarding path, session handler, and transport drop sampling with rate-limited warn logging (5s interval) - Expose congestion data in show_routing control query and ecn_ce_count in show_mmp peer entries - Add congestion counters to fipstop routing tab in two-column layout Chaos harness integration: - Add query_routing(), query_transports(), snapshot_all_congestion() to chaos control module - Add congestion/kernel-drop log analysis in logs module - Add congestion-stress scenario: 10-node tree, 1 Mbps bandwidth, 5-10% netem loss, heavy iperf3 traffic - Add IngressConfig for tc ingress policing with per-peer policer filters simulating upstream bandwidth bottlenecks - Add iperf3 JSON result capture to traffic manager for throughput measurement across scenarios - Add ECN A/B test scenarios (ecn-ab-on/off.yaml) with ingress policing and comparison script - Enable TCP ECN negotiation (tcp_ecn=1 sysctl) in container entrypoint for end-to-end CE propagation Tests: - 10 ECN unit/integration tests: mark_ipv6_ecn_ce variants, CE relay chain (3-node propagation), EcnConfig serde roundtrip - 3 transport drop congestion detection unit tests Documentation: - Update fips-mesh-layer.md: replace outdated CE Echo stub with full ECN Congestion Signaling section covering detection logic, CE relay, IPv6 marking, session tracking, and monitoring counters - Update fips-configuration.md: add node.ecn.* parameter table and ecn block in complete reference YAML - Update fips-transport-layer.md: add Congestion Reporting section with TransportCongestion struct, congestion() trait method, and per-transport status; document AsyncFd/recvmsg/SO_RXQ_OVFL in UDP - Update chaos README: add congestion/ECN scenario docs, ingress traffic control, and iperf3 JSON capture sections - Update README.md: add ECN to features list and "What works today"; update transport and tooling entries	2026-03-05 17:05:57 +00:00
Johnathan Corgan	d8cb4d407e	FLP wire format revision and MMP link-layer measurement protocol ... ## FLP Wire Format Revision Replace the 1-byte discriminator with a structured wire format: - 4-byte common prefix (ver+phase, flags, payload_len) and 16-byte established frame header with AEAD AAD binding - 5-byte encrypted inner header (4-byte session-relative timestamp + 1-byte message type) on all link messages - Phase-based packet dispatch replacing discriminator-based dispatch - SessionDatagram reassigned from type 0x40 to 0x00; add SenderReport (0x01) and ReceiverReport (0x02) message types for MMP - SessionDatagram: rename hop_limit to ttl, add path_mtu field (u16 LE) with min(datagram.path_mtu, transport.mtu()) at forwarding - Updated handshake packets (msg1: 87->90 bytes, msg2: 42->45 bytes) - FIPS_OVERHEAD updated from 135 to 144 bytes ## MMP Link-Layer Measurement Protocol Add the Metrics Measurement Protocol for link quality measurement between FIPS peers. Measures RTT, loss, jitter, throughput, OWD trend, and ETX from periodic sender/receiver reports exchanged over established links. Module layout: - mmp/algorithms.rs: JitterEstimator, SrttEstimator, DualEwma, OwdTrend, SpinBit, ETX computation - mmp/report.rs: SenderReport (48B) and ReceiverReport (68B) wire format - mmp/sender.rs: per-peer TX counters and interval tracking - mmp/receiver.rs: per-peer RX counters, jitter, loss, gap tracking - mmp/metrics.rs: derived metrics from report processing (SRTT, goodput_bps) - mmp/mod.rs: MmpMode (Full/Lightweight/Minimal), MmpConfig, MmpPeerState - node/handlers/mmp.rs: report dispatch, timer-driven generation, operator logging (periodic + teardown) Integration: per-frame TX/RX hooks in encrypted message handling, report dispatch from link message router, timer-driven generation from tick handler, and periodic operator logging with throughput formatting. Three operating modes: Full (sender + receiver reports, spin bit, CE echo), Lightweight (receiver reports only), Minimal (spin bit + CE echo only). ## Design Documentation Updated FLP sections across all design documents to match the implemented wire format, including revised overhead calculations and numeric values. 568 tests pass, clippy clean.	2026-02-18 21:54:21 +00:00
Johnathan Corgan	51597b6a5a	End-to-end session establishment with 100-node bidirectional data test ... Implement Noise IK session handshake between arbitrary endpoints, carried inside SessionDatagram envelopes through the mesh. Sessions use a three-state machine (Initiating → Established for initiator, Responding → Established for responder on first DataPacket). Includes session initiation API, encrypted data transfer, simultaneous initiation tie-break, error signal handlers (CoordsRequired, PathBroken), and local delivery wiring in the forwarding handler. New files: node/session.rs (state types), handlers/session.rs (~500 lines, all session message handlers + send path), tests/session.rs (11 tests). 100-node integration test establishes sessions across random topology, sends bidirectional encrypted datagrams through injected TUN channels, and verifies 200/200 deliveries with 100% session establishment. Reports routing statistics including avg 4.1 link hops per datagram. 404 tests pass (up from 393).	2026-02-13 03:13:21 +00:00
Johnathan Corgan	9a7fa921ab	Discovery protocol: LookupRequest/LookupResponse handlers ... Implement the coordinate discovery protocol with flood-based lookup and reverse-path response routing. Wire format: LookupRequest (0x30) encode/decode with TTL, visited bloom filter, origin coords. LookupResponse (0x31) encode/decode with target coords and Schnorr proof signature. Handler logic: request dedup by request_id, visited filter loop prevention, TTL enforcement, lazy purge of expired entries (10s). Response routing: originator caches route in route_cache, transit nodes reverse-path forward via recent_requests. Node state: RecentRequest struct, route_cache (RouteCache), and recent_requests map for dedup + reverse-path forwarding. 13 handler tests (9 unit + 4 integration) plus 4 protocol tests. 392 tests pass, clean build.	2026-02-12 14:46:33 +00:00
Johnathan Corgan	009101ee4a	SessionDatagram forwarding handler with coordinate cache warming ... Add handle_session_datagram handler replacing the 0x40 dispatch stub. Transit nodes now decode the datagram envelope, enforce hop limits, warm coordinate caches from SessionSetup/SessionAck/DataPacket payloads, route via find_next_hop, and generate CoordsRequired/PathBroken error signals on routing failure. 14 new tests covering decode errors, hop limit enforcement, local delivery, cache warming for all session message types, single-hop and multi-hop forwarding through live node chains, error signal generation, and cache warming enabling subsequent routing. 375 tests pass.	2026-02-12 13:39:34 +00:00
Johnathan Corgan	cc29c51cac	Refactor node/handlers.rs and node/tests.rs into subdirectories ... Split handlers.rs (986 lines) into handlers/ with 5 subfiles organized by responsibility: rx_loop, encrypted, handshake, dispatch, timeout. Split tests.rs (2350 lines) into tests/ with 4 subfiles: unit tests, handshake integration, spanning tree convergence, and bloom filter tests. Shared test helpers extracted to tests/mod.rs. Visibility adjusted from pub(super) to pub(in crate::node) for handler methods now two levels deep. Unused imports cleaned up in node/mod.rs. All 316 tests pass, zero warnings.	2026-02-11 03:49:45 +00:00