From dc89edf60bbd65826feb5d3a35a64f24bb1fc168 Mon Sep 17 00:00:00 2001 From: Johnathan Corgan Date: Sun, 22 Feb 2026 23:03:00 +0000 Subject: [PATCH] Update design docs for session 142 implementation changes Comprehensive documentation review across 12 files to reflect: - Noise XK at FSP (was IK), 3-message handshake, SessionMsg3 wire format - Epoch exchange in Noise handshakes for peer restart detection - Per-link MTU, min_mtu/path_mtu in lookup packets - MtuExceeded (0x22) error signal wire format and behavior - LookupResponse proof now covers target_coords - Discovery reverse-path routing as primary (not greedy) - Control socket architecture and fipsctl binary - FSP handshake state machine (Initiating/AwaitingMsg3/Established) - Root timeout framing updated for heartbeat cascading --- README.md | 4 +- docs/design/fips-configuration.md | 15 ++ docs/design/fips-intro.md | 50 ++++--- docs/design/fips-mesh-layer.md | 22 ++- docs/design/fips-mesh-operation.md | 60 ++++++-- docs/design/fips-session-layer.md | 90 +++++++---- docs/design/fips-software-architecture.md | 67 ++++++++- docs/design/fips-spanning-tree.md | 5 +- docs/design/fips-state-machines.md | 44 ++++++ docs/design/fips-transport-layer.md | 30 ++-- docs/design/fips-wire-formats.md | 174 +++++++++++++++------- docs/design/spanning-tree-dynamics.md | 27 ++-- 12 files changed, 440 insertions(+), 148 deletions(-) diff --git a/README.md b/README.md index 3b7ec94..b1ecfe8 100644 --- a/README.md +++ b/README.md @@ -36,6 +36,8 @@ sessions across the mesh. unmodified IP applications - **Metrics Measurement Protocol** — per-link RTT, loss, jitter, and goodput measurement +- **Operator visibility** — `fipsctl` control socket interface for runtime + inspection of peers, links, sessions, tree state, and metrics - **Zero configuration** — sensible defaults; a node can run with no config file @@ -81,7 +83,7 @@ a layered protocol specification. Start with ## Project Structure ``` -src/ Rust source (library + daemon binary) +src/ Rust source (library + fips/fipsctl binaries) docs/design/ Protocol design specifications testing/ Docker-based integration test harnesses benches/ Criterion benchmarks diff --git a/docs/design/fips-configuration.md b/docs/design/fips-configuration.md index f39ce5e..e53c0b9 100644 --- a/docs/design/fips-configuration.md +++ b/docs/design/fips-configuration.md @@ -49,6 +49,18 @@ transports: # Network transports (UDP, future: TCP, Tor) peers: # Static peer list ``` +### Control Socket (`node.control.*`) + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `node.control.enabled` | bool | `true` | Enable the Unix domain control socket | +| `node.control.socket_path` | string | *(auto)* | Socket file path. Default: `$XDG_RUNTIME_DIR/fips/control.sock` if `XDG_RUNTIME_DIR` is set, otherwise `/tmp/fips-control.sock` | + +The control socket provides read-only access to node state via the +`fipsctl` command-line tool. See +[fips-software-architecture.md](fips-software-architecture.md) for the +protocol and command list. + All tunable protocol parameters live under `node.*`, organized as sysctl-style dotted paths. The top-level sections (`tun`, `dns`, `transports`, `peers`) handle infrastructure concerns only. @@ -352,6 +364,9 @@ node: mode: full # full | lightweight | minimal log_interval_secs: 30 owd_window_size: 32 + control: + enabled: true + socket_path: null # null = auto ($XDG_RUNTIME_DIR/fips/control.sock or /tmp/fips-control.sock) buffers: packet_channel: 1024 tun_channel: 1024 diff --git a/docs/design/fips-intro.md b/docs/design/fips-intro.md index 35cff3a..73272ec 100644 --- a/docs/design/fips-intro.md +++ b/docs/design/fips-intro.md @@ -241,7 +241,7 @@ FIPS uses independent encryption at two protocol layers: | Layer | Scope | Pattern | Purpose | | ----- | ----- | ------- | ------- | | **FMP (Mesh)** | Hop-by-hop | Noise IK | Encrypt all traffic on each peer link | -| **FSP (Session)** | End-to-end | Noise IK | Encrypt application payload between endpoints | +| **FSP (Session)** | End-to-end | Noise XK | Encrypt application payload between endpoints | ### Link Layer (Hop-by-Hop) @@ -258,9 +258,13 @@ from the first handshake message. ### Session Layer (End-to-End) FIPS establishes end-to-end encrypted sessions between any two communicating -nodes using Noise IK, regardless of how many hops separate them. The -initiator knows the destination's npub; the responder learns the initiator's -identity from the handshake — the same asymmetry as link-layer connections. +nodes using Noise XK, regardless of how many hops separate them. The +initiator knows the destination's npub (required for XK's pre-message); +the responder learns the initiator's identity from the third handshake +message. Unlike the link-layer IK pattern where the initiator's identity +is revealed in msg1, XK delays identity disclosure until msg3, providing +stronger initiator identity protection for traffic traversing untrusted +intermediate nodes. A packet from A to D through intermediate nodes B and C: @@ -414,17 +418,19 @@ without individual discovery at each hop. ### Error Recovery -When routing fails — because cached coordinates are stale or a path has -broken — transit nodes signal the source: +When routing fails — because cached coordinates are stale, a path has +broken, or a packet exceeds a link's MTU — transit nodes signal the source: - **CoordsRequired**: A transit node lacks the destination's coordinates. The source re-initiates discovery and resets its coordinate warmup strategy. - **PathBroken**: Greedy routing reached a dead end. The source re-discovers the destination's current coordinates. +- **MtuExceeded**: A transit node cannot forward a packet because it exceeds + the next-hop link MTU. The source adjusts its path MTU estimate. -Both signals trigger active recovery, and are rate-limited to prevent storms -during topology changes. +All three signals trigger active recovery, and are rate-limited to prevent +storms during topology changes. See [fips-mesh-operation.md](fips-mesh-operation.md) for the complete routing and mesh behavior description. @@ -548,7 +554,7 @@ packets. The most important adversary class is the operators of other nodes in the mesh — the peers that forward your traffic. FIPS treats every intermediate router as potentially adversarial. The FSP session layer establishes a -completely independent Noise IK session between the communicating endpoints, +completely independent Noise XK session between the communicating endpoints, so intermediate nodes cannot read application payloads even though they decrypt and re-encrypt the link-layer envelope at each hop. @@ -660,19 +666,21 @@ confidentiality and integrity rather than hiding traffic patterns. ### Noise Protocol Framework -Both FMP link encryption and FSP session encryption use the -[Noise Protocol Framework](https://noiseprotocol.org/) IK handshake pattern. -Noise IK provides mutual authentication with a single round trip, where the -initiator knows the responder's static key in advance. +FIPS uses the [Noise Protocol Framework](https://noiseprotocol.org/) at both +protocol layers, with different handshake patterns chosen for each layer's +threat model. FMP link encryption uses **Noise IK**, providing mutual +authentication with a single round trip where the initiator knows the +responder's static key in advance. [WireGuard](https://www.wireguard.com/) uses the same IK base pattern -(extended with a pre-shared key as IKpsk2) for VPN tunnels. The -[Lightning Network](https://github.com/lightning/bolts/blob/master/08-transport.md) -uses Noise XK, a related pattern where the initiator's static key is -transmitted in a third message rather than the first, providing stronger -initiator identity hiding. FIPS uses pure Noise IK at two independent layers, -sharing the same framework and cryptographic foundation as these deployed -systems while choosing the pattern that best fits its mutual-authentication -requirement. +(extended with a pre-shared key as IKpsk2) for VPN tunnels. FSP session +encryption uses **Noise XK**, the same pattern used by the +[Lightning Network](https://github.com/lightning/bolts/blob/master/08-transport.md), +where the initiator's static key is transmitted in a third message rather +than the first. XK provides stronger initiator identity hiding at the cost +of an additional round trip — a worthwhile tradeoff for session-layer traffic +that traverses untrusted intermediate nodes. At the link layer, where both +peers are configured and directly connected, IK's single round trip is +preferred. ### Index-Based Session Dispatch diff --git a/docs/design/fips-mesh-layer.md b/docs/design/fips-mesh-layer.md index 9df825b..bff1d35 100644 --- a/docs/design/fips-mesh-layer.md +++ b/docs/design/fips-mesh-layer.md @@ -110,6 +110,25 @@ session keys derived from four DH operations (es, ss, ee, se). The handshake provides mutual authentication, forward secrecy, and identity hiding for the initiator. +### Epoch Exchange and Peer Restart Detection + +Both IK handshake messages carry an encrypted epoch payload — an 8-byte +random value generated once at node startup: + +- **msg1**: Ephemeral key (33 bytes) + encrypted static key (49 bytes) + + encrypted epoch (24 bytes) = 106 bytes total +- **msg2**: Ephemeral key (33 bytes) + encrypted epoch (24 bytes) = 57 bytes + total + +The encrypted epoch (EPOCH_ENCRYPTED_SIZE = 24 bytes) consists of the +8-byte epoch value plus a 16-byte AEAD tag. + +On reconnection, each peer compares the received epoch with the previously +stored epoch for that peer. An epoch mismatch indicates the peer has +restarted (generated a new epoch), triggering full link re-establishment +rather than treating the handshake as a simple reconnection. This prevents +stale session state from persisting across restarts. + ### Identity Binding The Noise handshake binds the link to the peer's cryptographic identity. After @@ -493,7 +512,8 @@ an attacker sends invalid packets to elicit responses. | Feature | Status | | ------- | ------ | -| Noise IK handshake | **Implemented** | +| Noise IK handshake (with epoch) | **Implemented** | +| Peer restart detection (epoch mismatch) | **Implemented** | | Link encryption (ChaCha20-Poly1305) | **Implemented** | | Index-based session dispatch | **Implemented** | | Replay protection (sliding window) | **Implemented** | diff --git a/docs/design/fips-mesh-operation.md b/docs/design/fips-mesh-operation.md index cef1c33..779ee04 100644 --- a/docs/design/fips-mesh-operation.md +++ b/docs/design/fips-mesh-operation.md @@ -288,17 +288,26 @@ direct peer), a LookupResponse is created containing: - **request_id**: Echoed from the request - **target**: The target's node_addr - **target_coords**: The target's current tree coordinates -- **proof**: Signature covering `(request_id || target)` — authenticates - that the response is genuine +- **path_mtu**: Minimum MTU along the response path (transit-annotated, + initialized to `u16::MAX` by the target) +- **proof**: Signature covering `(request_id || target || target_coords)` — + authenticates that the response is genuine and the target holds the + claimed tree position -The response routes back to the requester using greedy tree routing toward -the origin_coords from the request. +The response routes back to the requester using reverse-path routing as the +primary mechanism: each transit node looks up the request_id in its +`recent_requests` table to find the peer that forwarded the original request, +and sends the response back through that peer. This ensures the response +follows the same path as the request. Greedy tree routing toward the +origin_coords is used only as a fallback if the reverse-path entry has +expired. -**Security**: Coordinates are intentionally excluded from the signed proof. -Binding coordinates would invalidate signatures whenever the spanning tree -reconverges. Coordinate tampering by transit nodes causes only routing -inefficiency, not a security breach (data integrity is protected by -session-layer encryption). +**Proof verification**: The source verifies the Schnorr proof upon receipt, +confirming that the target actually signed the response. The proof covers +`(request_id || target || target_coords)` — coordinates are included because +verification at the source confirms the target holds the claimed position. +The `path_mtu` field is excluded from the proof because it is a transit +annotation modified at each hop. ### Discovery Outcome @@ -402,9 +411,31 @@ source. 3. Initiate discovery for the destination 4. Reset CP warmup counter +### MtuExceeded + +**Trigger**: A transit node receives a SessionDatagram but the total +packet size exceeds the next-hop link MTU. The packet cannot be forwarded +without fragmentation, which FIPS does not perform at the mesh layer. + +**Transit node action**: + +1. Create a new SessionDatagram addressed back to the original source, + carrying an MtuExceeded payload identifying the destination, the + reporting router, and the bottleneck MTU +2. Route the error via `find_next_hop(src_addr)` +3. Drop the original oversized packet + +**Source recovery**: FSP uses the reported bottleneck MTU to adjust its +session-layer path MTU estimate (immediate decrease). The source can then +reduce payload sizes to fit within the discovered path MTU. MtuExceeded is +the reactive complement to the proactive `path_mtu` field in +SessionDatagram and LookupResponse — the proactive field tracks the +minimum MTU along the forward path, while MtuExceeded signals when an +actual packet exceeds the limit. + ### Error Signal Rate Limiting -Both error types are rate-limited at transit nodes: maximum one error per +All three error types are rate-limited at transit nodes: maximum one error per destination per 100ms. This prevents storms during topology changes when many packets to the same destination hit the same routing failure simultaneously. @@ -414,9 +445,9 @@ per `coords_response_interval_ms` (default 2000ms, configurable). This prevents amplification where a burst of error signals would generate a corresponding burst of warmup messages. -Error signals (CoordsRequired, PathBroken) are handled asynchronously outside -the packet receive path, allowing the RX loop to continue processing without -blocking on discovery or session repair. +Error signals (CoordsRequired, PathBroken, MtuExceeded) are handled +asynchronously outside the packet receive path, allowing the RX loop to +continue processing without blocking on discovery or session repair. ### Error Routing Limitation @@ -577,6 +608,9 @@ recovery). | Hybrid coordinate warmup (CP + CoordsWarmup) | **Implemented** | | CoordsRequired recovery | **Implemented** | | PathBroken recovery | **Implemented** | +| MtuExceeded recovery | **Implemented** | +| LookupResponse proof verification | **Implemented** | +| Discovery reverse-path routing | **Implemented** | | Error signal rate limiting | **Implemented** | | Leaf-only operation | Future direction | | Link cost metrics (ETX) | Future direction | diff --git a/docs/design/fips-session-layer.md b/docs/design/fips-session-layer.md index b7ba536..6cd4ca1 100644 --- a/docs/design/fips-session-layer.md +++ b/docs/design/fips-session-layer.md @@ -89,10 +89,15 @@ FMP signals routing failures asynchronously: - **PathBroken**: Greedy routing reached a dead end. FSP responds by sending a standalone CoordsWarmup (rate-limited), re-discovering the destination's current coordinates, and resetting the warmup counter. +- **MtuExceeded**: A transit node cannot forward a SessionDatagram because + the packet exceeds the next-hop link MTU. FSP uses the reported bottleneck + MTU to adjust its session-layer path MTU estimate. MtuExceeded is the + reactive complement to the proactive `path_mtu` field in SessionDatagram. -Both signals are generated by transit nodes (not the destination) and travel -back to the source inside a new SessionDatagram. They are plaintext (not -end-to-end encrypted) because transit nodes have no session with the source. +All three signals are generated by transit nodes (not the destination) and +travel back to the source inside a new SessionDatagram. They are plaintext +(not end-to-end encrypted) because transit nodes have no session with the +source. ### Local Delivery @@ -106,18 +111,29 @@ node, FMP delivers it to FSP for session-layer processing. Sessions are established on demand when the first datagram needs to be sent to a destination with no existing session. -FSP uses Noise IK for session key agreement. The initiator knows the -destination's npub (from DNS lookup or native API); the responder learns the -initiator's identity from the handshake. This is the same asymmetry as -link-layer peer connections. +FSP uses Noise XK for session key agreement. The initiator knows the +destination's npub (required for XK's pre-message `s` token); the +responder learns the initiator's identity from msg3 (not msg1, unlike +IK at the link layer). This provides stronger initiator identity hiding +— the initiator's static key is encrypted under the established shared +secret rather than under only the responder's static key. -The handshake is carried in SessionSetup and SessionAck messages: +The handshake is a three-message flow carried in SessionSetup, SessionAck, +and SessionMsg3: -1. **Initiator** sends SessionSetup containing Noise IK msg1 and both - parties' tree coordinates -2. **Responder** processes msg1, learns initiator identity, sends SessionAck - containing Noise IK msg2 and both parties' tree coordinates -3. Both parties derive identical symmetric session keys +1. **Initiator** sends SessionSetup containing Noise XK msg1 (ephemeral key + only) and both parties' tree coordinates +2. **Responder** processes msg1, sends SessionAck containing Noise XK msg2 + (ephemeral key + encrypted epoch) and both parties' tree coordinates. + The responder transitions to AwaitingMsg3 state. +3. **Initiator** processes msg2, sends SessionMsg3 containing the encrypted + static key and encrypted epoch. Both parties derive identical symmetric + session keys and the session is established. + +Each side's epoch (an 8-byte random value generated at startup) is +exchanged encrypted in msg2 and msg3. On subsequent handshakes, an epoch +mismatch indicates the peer has restarted, triggering session +re-establishment. Packets that trigger session establishment are queued (with bounded buffer) and transmitted after the session is established. @@ -187,22 +203,30 @@ transport addresses or routing paths. A session survives: ## End-to-End Encryption -### Noise IK Pattern +### Noise XK Pattern -FSP uses the same Noise IK pattern as FMP link encryption, but with -independent keys and sessions. The full Noise descriptor is -`Noise_IK_secp256k1_ChaChaPoly_SHA256`. +FSP uses Noise XK for session encryption, distinct from the Noise IK +pattern used at the link layer. The full Noise descriptor is +`Noise_XK_secp256k1_ChaChaPoly_SHA256`. -The IK pattern: +The XK pattern (pre-message: `← s`): -- **msg1** (`→ e, es, s, ss`): Initiator sends ephemeral key, encrypts static - key to responder. Four DH operations establish session keys. -- **msg2** (`← e, ee, se`): Responder sends ephemeral key. Both parties now - share identical session keys. +- **msg1** (`→ e, es`): Initiator sends ephemeral key only. The initiator's + static identity is not revealed in this message. +- **msg2** (`← e, ee`): Responder sends ephemeral key and encrypted epoch. +- **msg3** (`→ s, se`): Initiator sends encrypted static key and encrypted + epoch. Both parties now share identical session keys. After the handshake, Noise produces two directional symmetric keys (`send_key`, `recv_key`) used with ChaCha20-Poly1305 for all subsequent data. +The XK pattern requires the initiator to know the responder's static key +in advance (the `← s` pre-message), which is satisfied by the discovery +or DNS lookup that precedes session establishment. In exchange, XK +provides stronger initiator identity protection than IK — the initiator's +static key is encrypted under the full shared secret (after three DH +operations) rather than under only the responder's static key. + ### Cryptographic Primitives | Component | Choice | Notes | @@ -218,7 +242,7 @@ These choices prioritize compatibility with the Nostr cryptographic stack. ### secp256k1 Parity Normalization Nostr npubs encode x-only public keys (32 bytes, no y-coordinate parity). The -Noise IK pre-message mixes the responder's static key as a 33-byte compressed +Noise XK pre-message mixes the responder's static key as a 33-byte compressed key, and the default secp256k1 ECDH hash includes a parity-dependent version byte. @@ -229,12 +253,14 @@ actual key parity. ### Privacy Note -Noise IK does not provide initiator anonymity if the responder's static key is -compromised. An attacker who obtains the responder's nsec can decrypt the -initiator's identity from captured handshake messages. Noise XK would protect -initiator identity in this scenario but requires an additional round-trip (3 -handshake messages vs. 2). The privacy/latency tradeoff may be revisited with -deployment experience. +Noise XK provides stronger initiator identity protection than IK. In XK, the +initiator's static key is encrypted in msg3 under the full shared secret +(derived from three DH operations), so an attacker who compromises only the +responder's nsec cannot decrypt the initiator's identity from captured +handshake messages (they would also need the responder's ephemeral key). +This is the primary reason FSP uses XK rather than IK — session-layer +traffic traverses untrusted intermediate nodes, making initiator identity +protection more valuable than at the link layer. ### Data Packet Authentication @@ -460,7 +486,7 @@ feedback: 1. The source sets `path_mtu` in each SessionDatagram envelope to its outbound link MTU. -2. Each transit node applies `min(current, transport.mtu())` before +2. Each transit node applies `min(current, transport.link_mtu(addr))` before forwarding. 3. The destination receives the forward-path minimum and sends a PathMtuNotification (2-byte body: u16 LE path_mtu) back to the source. @@ -492,7 +518,9 @@ MMP session metrics session=npub1tdwa...84le rtt=4.3ms loss=0.6% jitter=0.2ms go | Feature | Status | | ------- | ------ | -| Session establishment (Noise IK) | **Implemented** | +| Session establishment (Noise XK) | **Implemented** | +| Peer restart detection (epoch exchange) | **Implemented** | +| MtuExceeded handling | **Implemented** | | End-to-end encryption (ChaCha20-Poly1305) | **Implemented** | | Explicit counter replay protection | **Implemented** | | Hybrid coordinate warmup (CP + CoordsWarmup) | **Implemented** | diff --git a/docs/design/fips-software-architecture.md b/docs/design/fips-software-architecture.md index dbfe31a..a3510f7 100644 --- a/docs/design/fips-software-architecture.md +++ b/docs/design/fips-software-architecture.md @@ -89,7 +89,7 @@ treatment of this pattern. ## Two-Layer Encryption Rationale -FIPS uses independent Noise IK encryption at two layers: +FIPS uses independent Noise encryption at two layers (IK at FMP, XK at FSP): | Layer | Scope | What It Protects | | ----- | ----- | ---------------- | @@ -108,10 +108,12 @@ FIPS uses independent Noise IK encryption at two layers: means topology changes (a direct peer becomes multi-hop) don't affect sessions. -**Why the same pattern (Noise IK) at both layers**: Both layers need mutual -authentication with identity hiding for the initiator. Reusing the same -cryptographic stack (secp256k1, ChaCha20-Poly1305, SHA-256) simplifies the -implementation and reduces the number of cryptographic dependencies. +**Why different patterns**: FMP uses Noise IK (2-message) because link peers +are configured and directly connected — the single round trip is preferred. +FSP uses Noise XK (3-message) because session traffic traverses untrusted +intermediate nodes — the extra round trip buys stronger initiator identity +hiding. Both layers reuse the same cryptographic stack (secp256k1, +ChaCha20-Poly1305, SHA-256), minimizing cryptographic dependencies. ## Identity Model @@ -297,6 +299,61 @@ while the trait surface remains synchronous for `send()`, `mtu()`, etc. The `TransportHandle` enum provides async dispatch for methods that need it (like `send_async()`) without requiring dyn dispatch. +## Control Socket + +FIPS exposes a read-only observability interface through a Unix domain +socket. The control socket provides runtime visibility into node state +without affecting protocol operation. + +### Architecture + +The control socket is a `UnixListener` integrated into the node's async +runtime. It accepts connections, reads a single JSON request per +connection, dispatches to query handlers that read (but never modify) +node state, and returns a JSON response. + +- **Max request size**: 4,096 bytes +- **I/O timeout**: 5 seconds per connection +- **Socket cleanup**: Automatically removes stale sockets on startup + +### fipsctl Binary + +The `fipsctl` command-line tool communicates with the control socket: + +```text +fipsctl [OPTIONS] + +OPTIONS: + -s, --socket Control socket path override + +COMMANDS: + show status Node status overview + show peers Authenticated peers + show links Active links + show tree Spanning tree state + show sessions End-to-end sessions + show bloom Bloom filter state + show mmp MMP metrics summary + show cache Coordinate cache stats + show connections Pending handshake connections + show transports Transport instances + show routing Routing table summary +``` + +### Protocol + +Request and response are line-delimited JSON: + +```json +{"command":"show_status"} +{"status":"ok","data":{...}} +``` + +Error responses use `{"status":"error","message":"..."}`. + +All queries are read-only — the control socket cannot modify node state, +trigger actions, or shut down the node. + ## References - [fips-intro.md](fips-intro.md) — Protocol overview diff --git a/docs/design/fips-spanning-tree.md b/docs/design/fips-spanning-tree.md index ecc0eb5..ba1126b 100644 --- a/docs/design/fips-spanning-tree.md +++ b/docs/design/fips-spanning-tree.md @@ -170,7 +170,8 @@ for stronger verification. - Type: u64, Unix seconds - Used for stale detection, not versioning - A root declaration is considered stale after `ROOT_TIMEOUT` (60 minutes) - without refresh + without refresh. In practice, heartbeat cascading handles root departure + for the common case (see spanning-tree-dynamics.md §11) ## Reconvergence @@ -229,7 +230,7 @@ Example: In a 1000-node network with depth 10 and 5 peers, a node stores | --------- | ------- | ----------- | | PARENT_SWITCH_THRESHOLD | 1 hop | Minimum depth improvement for same-root switch | | ANNOUNCE_MIN_INTERVAL | 500ms | Minimum between announcements to same peer | -| ROOT_TIMEOUT | 60 min | Root declaration considered stale | +| ROOT_TIMEOUT | 60 min | Root declaration considered stale (not yet enforced; heartbeat cascading covers common case) | | TREE_ENTRY_TTL | 5–10 min | Individual entry expiration | ## Implementation Status diff --git a/docs/design/fips-state-machines.md b/docs/design/fips-state-machines.md index 9c52324..f4f98ec 100644 --- a/docs/design/fips-state-machines.md +++ b/docs/design/fips-state-machines.md @@ -217,6 +217,50 @@ PeerSlot::Active(ActivePeer) - Session indices and transport address (for wire protocol dispatch) - Statistics (`last_seen`, `link_stats`, `authenticated_at`) +### End-to-End Session Lifecycle (FSP) + +```text +EndToEndState::Initiating(HandshakeState) + │ + │ Noise XK msg1 sent (SessionSetup) + │ Awaiting msg2 (SessionAck) + │ + │ msg2 received, processed + │ msg3 sent (SessionMsg3) + ▼ +EndToEndState::Established(NoiseSession) +``` + +```text +[msg1 received from remote initiator] + │ + │ Noise XK msg1 processed, msg2 sent (SessionAck) + ▼ +EndToEndState::AwaitingMsg3(HandshakeState) + │ + │ msg3 received, processed + ▼ +EndToEndState::Established(NoiseSession) +``` + +**EndToEndState** is an enum with three variants: + +- **Initiating(HandshakeState)** — We initiated the session, sent + SessionSetup with XK msg1, awaiting SessionAck (msg2) +- **AwaitingMsg3(HandshakeState)** — We are the XK responder, processed + msg1, sent msg2, awaiting msg3 from initiator +- **Established(NoiseSession)** — Handshake complete, symmetric session + keys derived, ready for application data + +The three-message flow (XK pattern) means the responder has an intermediate +`AwaitingMsg3` state that does not exist in the link-layer IK handshake +(which completes in two messages). The initiator transitions directly from +Initiating to Established upon processing msg2 and sending msg3. + +Each side's epoch (8-byte random startup value) is exchanged encrypted in +msg2 and msg3. Epoch mismatch on subsequent handshakes indicates peer +restart. + ### Link Lifecycle (Connection-Oriented Transports) Links use a simple state enum rather than phase-based structs, since link diff --git a/docs/design/fips-transport-layer.md b/docs/design/fips-transport-layer.md index abfcf51..41196cd 100644 --- a/docs/design/fips-transport-layer.md +++ b/docs/design/fips-transport-layer.md @@ -55,10 +55,15 @@ link — this is the degenerate case. Transports that negotiate MTU per-connection (e.g., BLE ATT_MTU) report the negotiated value for each link individually. -> **Implementation note**: The current transport trait exposes MTU as a -> transport-wide method (`fn mtu(&self) -> u16`). This works for UDP, where -> MTU is a static configuration value. Supporting per-link MTU for future -> transports will require extending this interface. +The transport trait exposes two MTU methods: + +- `fn mtu(&self) -> u16` — Transport-wide default MTU +- `fn link_mtu(&self, addr: &TransportAddr) -> u16` — Per-link MTU for a + specific remote address. The default implementation falls back to + `mtu()`, so transports with uniform MTU (like UDP) need not override it. + +FMP uses `link_mtu()` when computing path MTU for SessionDatagram +forwarding and LookupResponse transit annotation. ### Connection Lifecycle @@ -305,14 +310,15 @@ The transport interface defines what every transport driver must provide. ### Trait Surface ```text -transport_id() → TransportId Unique identifier for this transport instance -transport_type() → &TransportType Static metadata (name, connection-oriented, reliable) -state() → TransportState Current lifecycle state -mtu() → u16 Maximum datagram size -start() → lifecycle Bring transport up (bind socket, open device) -stop() → lifecycle Bring transport down -send(addr, data) → delivery Send datagram to transport address -discover() → Vec Report discovered FIPS endpoints (optional) +transport_id() → TransportId Unique identifier for this transport instance +transport_type() → &TransportType Static metadata (name, connection-oriented, reliable) +state() → TransportState Current lifecycle state +mtu() → u16 Transport-wide default MTU +link_mtu(addr) → u16 Per-link MTU (defaults to mtu()) +start() → lifecycle Bring transport up (bind socket, open device) +stop() → lifecycle Bring transport down +send(addr, data) → delivery Send datagram to transport address +discover() → Vec Report discovered FIPS endpoints (optional) ``` ### Receive Path diff --git a/docs/design/fips-wire-formats.md b/docs/design/fips-wire-formats.md index 13dc674..1cbba6a 100644 --- a/docs/design/fips-wire-formats.md +++ b/docs/design/fips-wire-formats.md @@ -144,29 +144,29 @@ Handshake initiation from connecting party. ```text ┌──────────────────────┬─────────────┬─────────────────────────────────────────┐ │ common prefix │ sender_idx │ Noise IK message 1 │ -│ 4 bytes │ 4 bytes LE │ 82 bytes │ +│ 4 bytes │ 4 bytes LE │ 106 bytes │ └──────────────────────┴─────────────┴─────────────────────────────────────────┘ -Total: 90 bytes +Total: 114 bytes ``` -Common prefix: ver=0, phase=0x1, flags=0, payload_len=86 (4 + 82). +Common prefix: ver=0, phase=0x1, flags=0, payload_len=110 (4 + 106). | Field | Size | Description | | ----- | ---- | ----------- | | common prefix | 4 bytes | ver=0, phase=1, flags=0, payload_len | | sender_idx | 4 bytes LE | Initiator's session index (becomes receiver's `receiver_idx`) | -| noise_msg1 | 82 bytes | Noise IK first message | +| noise_msg1 | 106 bytes | Noise IK first message | -**Noise msg1 breakdown** (82 bytes): +**Noise msg1 breakdown** (106 bytes): | Offset | Field | Size | Description | | ------ | ----- | ---- | ----------- | | 0 | ephemeral_pubkey | 33 bytes | Initiator's ephemeral key (compressed secp256k1) | -| 33 | encrypted_static | 33 bytes | Initiator's static key (encrypted with es key) | -| 66 | tag | 16 bytes | AEAD tag for encrypted_static | +| 33 | encrypted_static | 49 bytes | Initiator's static key (33) + AEAD tag (16) | +| 82 | encrypted_epoch | 24 bytes | Startup epoch (8) + AEAD tag (16) | -Noise pattern: `-> e, es, s, ss` +Noise pattern: `-> e, es, s, ss` with epoch payload ### Noise IK Message 2 (phase 0x2) @@ -175,30 +175,34 @@ Handshake response from responder. ```text ┌──────────────────────┬─────────────┬──────────────┬──────────────────────────┐ │ common prefix │ sender_idx │ receiver_idx │ Noise IK message 2 │ -│ 4 bytes │ 4 bytes LE │ 4 bytes LE │ 33 bytes │ +│ 4 bytes │ 4 bytes LE │ 4 bytes LE │ 57 bytes │ └──────────────────────┴─────────────┴──────────────┴──────────────────────────┘ -Total: 45 bytes +Total: 69 bytes ``` -Common prefix: ver=0, phase=0x2, flags=0, payload_len=41 (4 + 4 + 33). +Common prefix: ver=0, phase=0x2, flags=0, payload_len=65 (4 + 4 + 57). | Field | Size | Description | | ----- | ---- | ----------- | | common prefix | 4 bytes | ver=0, phase=2, flags=0, payload_len | | sender_idx | 4 bytes LE | Responder's session index | | receiver_idx | 4 bytes LE | Echo of initiator's sender_idx from msg1 | -| noise_msg2 | 33 bytes | Noise IK second message | +| noise_msg2 | 57 bytes | Noise IK second message | -**Noise msg2 breakdown** (33 bytes): +**Noise msg2 breakdown** (57 bytes): | Offset | Field | Size | Description | | ------ | ----- | ---- | ----------- | | 0 | ephemeral_pubkey | 33 bytes | Responder's ephemeral key (compressed secp256k1) | +| 33 | encrypted_epoch | 24 bytes | Startup epoch (8) + AEAD tag (16) | -Noise pattern: `<- e, ee, se` +Noise pattern: `<- e, ee, se` with epoch payload -After msg2, both parties derive identical symmetric session keys. +After msg2, both parties derive identical symmetric session keys. The +encrypted epoch in msg1 and msg2 enables peer restart detection — if a +peer's epoch changes, the other side knows it restarted and must +re-establish the link. ### Index Semantics @@ -313,18 +317,19 @@ Coordinate discovery request, flooded through the mesh. | 9 | target | 16 bytes | NodeAddr being sought | | 25 | origin | 16 bytes | Requester's NodeAddr | | 41 | ttl | 1 byte | Remaining hops (default 64) | -| 42 | origin_coords_cnt | 2 bytes LE | Number of coordinate entries | -| 44 | origin_coords | 16 x n bytes | Requester's ancestry (NodeAddr only) | -| 44 + 16n | visited_hash_cnt | 1 byte | Hash count for visited filter | -| 45 + 16n | visited_bits | 256 bytes | Compact bloom of visited nodes | +| 42 | min_mtu | 2 bytes LE | Minimum transport MTU the origin requires (0 = no requirement) | +| 44 | origin_coords_cnt | 2 bytes LE | Number of coordinate entries | +| 46 | origin_coords | 16 x n bytes | Requester's ancestry (NodeAddr only) | +| 46 + 16n | visited_hash_cnt | 1 byte | Hash count for visited filter | +| 47 + 16n | visited_bits | 256 bytes | Compact bloom of visited nodes | -**Size**: `301 + (n x 16)` bytes, where n = origin depth + 1 +**Size**: `303 + (n x 16)` bytes, where n = origin depth + 1 | Origin Depth | Payload | | ------------ | ------- | -| 3 | 349 bytes | -| 5 | 381 bytes | -| 10 | 461 bytes | +| 3 | 351 bytes | +| 5 | 383 bytes | +| 10 | 463 bytes | ### LookupResponse (0x31) @@ -335,21 +340,28 @@ Coordinate discovery response, greedy-routed back to requester. | 0 | msg_type | 1 byte | 0x31 | | 1 | request_id | 8 bytes LE | Echoes the request's ID | | 9 | target | 16 bytes | NodeAddr that was found | -| 25 | target_coords_cnt | 2 bytes LE | Number of coordinate entries | -| 27 | target_coords | 16 x n bytes | Target's ancestry (NodeAddr only) | -| 27 + 16n | proof | 64 bytes | Schnorr signature over `(request_id \|\| target)` | +| 25 | path_mtu | 2 bytes LE | Minimum MTU along response path (transit-annotated) | +| 27 | target_coords_cnt | 2 bytes LE | Number of coordinate entries | +| 29 | target_coords | 16 x n bytes | Target's ancestry (NodeAddr only) | +| 29 + 16n | proof | 64 bytes | Schnorr signature over `(request_id \|\| target \|\| target_coords)` | -**Size**: `91 + (n x 16)` bytes +**Size**: `93 + (n x 16)` bytes | Target Depth | Payload | | ------------ | ------- | -| 3 | 139 bytes | -| 5 | 171 bytes | -| 10 | 251 bytes | +| 3 | 141 bytes | +| 5 | 173 bytes | +| 10 | 253 bytes | -**Proof coverage**: Signs `(request_id || target)` only — coordinates are -excluded so the proof survives tree reconvergence during the lookup -round-trip. +The `path_mtu` field is initialized to `u16::MAX` by the target and each +transit hop applies `min(path_mtu, outgoing_link_mtu)`, giving the +originator an MTU estimate for the discovered path. + +**Proof coverage**: Signs `(request_id || target || target_coords)` — +`path_mtu` is excluded from the proof because it is a transit annotation +modified at each hop. Coordinates are included because proof verification +at the source confirms the target actually holds the claimed tree position. +The source verifies the proof upon receipt. ### SessionDatagram (0x00) @@ -460,8 +472,9 @@ protocol version, session lifecycle phase, per-packet flags, and payload length. | Phase | Type | Description | | ----- | ---- | ----------- | | 0x0 | Established | Post-handshake encrypted traffic or plaintext error signals | -| 0x1 | Handshake msg1 | SessionSetup (Noise IK msg1) | -| 0x2 | Handshake msg2 | SessionAck (Noise IK msg2) | +| 0x1 | Handshake msg1 | SessionSetup (Noise XK msg1) | +| 0x2 | Handshake msg2 | SessionAck (Noise XK msg2) | +| 0x3 | Handshake msg3 | SessionMsg3 (Noise XK msg3) | ### FSP Flags (Established Phase Only) @@ -472,7 +485,7 @@ protocol version, session lifecycle phase, per-packet flags, and payload length. | 2 | U (unencrypted) | Payload is plaintext (error signals) | | 3-7 | — | Reserved (must be zero) | -Flags must be zero in handshake packets (phase 0x1 and 0x2). +Flags must be zero in handshake packets (phase 0x1, 0x2, and 0x3). ### FSP Encrypted Message (phase 0x0, U flag clear) @@ -529,9 +542,10 @@ body. | 0x14 | CoordsWarmup | Standalone coordinate cache warming | | 0x20 | CoordsRequired | Error: transit node lacks destination coordinates | | 0x21 | PathBroken | Error: greedy routing reached dead end | +| 0x22 | MtuExceeded | Error: forwarded packet exceeds next-hop MTU | Message types 0x10-0x14 are carried inside the AEAD ciphertext (dispatched -by the `msg_type` field in the encrypted inner header). Types 0x20-0x21 are +by the `msg_type` field in the encrypted inner header). Types 0x20-0x22 are plaintext error signals (U flag set, no encryption). Session-layer SenderReport (0x11) and ReceiverReport (0x12) use the same @@ -541,7 +555,9 @@ happens at the session level based on the FSP message type. ### SessionSetup (phase 0x1) -Establishes a session and warms transit coordinate caches. +Establishes a session and warms transit coordinate caches. Contains the +first message of the Noise XK handshake (ephemeral key only — the +initiator's static identity is not revealed until msg3). Encoded with FSP prefix: ver=0, phase=0x1, flags=0, payload_len. **Body** (after 4-byte FSP prefix): @@ -554,11 +570,12 @@ Encoded with FSP prefix: ver=0, phase=0x1, flags=0, payload_len. | ... | dest_coords_count | 2 bytes LE | Number of dest coordinate entries | | ... | dest_coords | 16 x m bytes | Destination's ancestry | | ... | handshake_len | 2 bytes LE | Noise payload length | -| ... | handshake_payload | variable | Noise IK msg1 (82 bytes typical) | +| ... | handshake_payload | variable | Noise XK msg1 (33 bytes — ephemeral key only) | ### SessionAck (phase 0x2) -Confirms session establishment, completes the Noise handshake. +Second message of the Noise XK handshake. The responder sends its +ephemeral key and encrypted epoch. Encoded with FSP prefix: ver=0, phase=0x2, flags=0, payload_len. **Body** (after 4-byte FSP prefix): @@ -571,7 +588,32 @@ Encoded with FSP prefix: ver=0, phase=0x2, flags=0, payload_len. | ... | dest_coords_count | 2 bytes LE | Number of initiator coordinate entries | | ... | dest_coords | 16 x m bytes | Initiator's ancestry (for return-path cache warming) | | ... | handshake_len | 2 bytes LE | Noise payload length | -| ... | handshake_payload | variable | Noise IK msg2 (33 bytes typical) | +| ... | handshake_payload | variable | Noise XK msg2 (57 bytes — ephemeral key + encrypted epoch) | + +### SessionMsg3 (phase 0x3) + +Third and final message of the Noise XK handshake. The initiator reveals +its encrypted static identity and epoch. After msg3, both parties derive +identical symmetric session keys and the session is established. +Encoded with FSP prefix: ver=0, phase=0x3, flags=0, payload_len. + +**Body** (after 4-byte FSP prefix): + +| Offset | Field | Size | Description | +| ------ | ----- | ---- | ----------- | +| 0 | flags | 1 byte | Reserved | +| 1 | handshake_len | 2 bytes LE | Noise payload length | +| 3 | handshake_payload | variable | Noise XK msg3 (73 bytes — encrypted static + encrypted epoch) | + +**Noise XK msg3 breakdown** (73 bytes): + +| Offset | Field | Size | Description | +| ------ | ----- | ---- | ----------- | +| 0 | encrypted_static | 49 bytes | Initiator's static key (33) + AEAD tag (16) | +| 49 | encrypted_epoch | 24 bytes | Startup epoch (8) + AEAD tag (16) | + +SessionMsg3 does not carry coordinates — both endpoints already have each +other's coordinates from SessionSetup (msg1) and SessionAck (msg2). ### Data (0x10) @@ -647,6 +689,28 @@ Encoded with FSP prefix: ver=0, phase=0x0, U flag set, payload_len. | 33 | last_coords_count | 2 bytes LE | Number of stale coordinate entries | | 35 | last_known_coords | 16 x n bytes | Stale coordinates that failed | +### MtuExceeded (0x22) + +Plaintext error signal — forwarded packet exceeds the next-hop link MTU. +Sent by a transit router back to the source when a SessionDatagram cannot +be forwarded because its size exceeds the outgoing link's MTU. +Encoded with FSP prefix: ver=0, phase=0x0, U flag set, payload_len. + +**Body** (after 4-byte FSP prefix + 1-byte msg_type): + +| Offset | Field | Size | Description | +| ------ | ----- | ---- | ----------- | +| 0 | flags | 1 byte | Reserved | +| 1 | dest_addr | 16 bytes | NodeAddr of the destination being forwarded to | +| 17 | reporter | 16 bytes | NodeAddr of the router that detected the MTU violation | +| 33 | mtu | 2 bytes LE | Bottleneck MTU at the reporting router | + +**Body size**: 35 bytes. Total with prefix + msg_type: 40 bytes. + +The source uses the reported MTU to adjust its session-layer path MTU +estimate. MtuExceeded is the reactive complement to the proactive +`path_mtu` field in SessionDatagram and LookupResponse. + ## Encapsulation Walkthrough A complete picture of how application data is wrapped through each layer. @@ -709,12 +773,20 @@ endpoint session keys). ## Size Summary -### Handshake Messages +### FMP Handshake Messages (Noise IK) -| Message | Size | -| ------- | ---- | -| Noise IK msg1 | 90 bytes | -| Noise IK msg2 | 45 bytes | +| Message | Raw Noise | Wire Frame | +| ------- | --------- | ---------- | +| IK msg1 (ephemeral + encrypted static + encrypted epoch) | 106 bytes | 114 bytes | +| IK msg2 (ephemeral + encrypted epoch) | 57 bytes | 69 bytes | + +### FSP Handshake Messages (Noise XK) + +| Message | Raw Noise | Notes | +| ------- | --------- | ----- | +| XK msg1 (ephemeral only) | 33 bytes | Carried in SessionSetup | +| XK msg2 (ephemeral + encrypted epoch) | 57 bytes | Carried in SessionAck | +| XK msg3 (encrypted static + encrypted epoch) | 73 bytes | Carried in SessionMsg3 | ### Link-Layer Messages (inside encrypted frame) @@ -722,8 +794,8 @@ endpoint session keys). | ------- | ---- | ----- | | TreeAnnounce | 100 + 32n bytes | n = depth + 1 | | FilterAnnounce | 1,035 bytes | v1 (1KB filter) | -| LookupRequest | 301 + 16n bytes | n = origin depth + 1 | -| LookupResponse | 91 + 16n bytes | n = target depth + 1 | +| LookupRequest | 303 + 16n bytes | n = origin depth + 1 | +| LookupResponse | 93 + 16n bytes | n = target depth + 1 | | SessionDatagram | 36 + payload bytes | Fixed 36-byte header | | Disconnect | 2 bytes | | @@ -731,8 +803,9 @@ endpoint session keys). | Message | Typical Size | Notes | | ------- | ------------ | ----- | -| SessionSetup | ~200 bytes | Depth-dependent | -| SessionAck | ~130 bytes | Depth-dependent (carries both endpoints' coords) | +| SessionSetup | ~170 bytes | Depth-dependent (XK msg1 = 33 bytes) | +| SessionAck | ~190 bytes | Depth-dependent, carries both endpoints' coords (XK msg2 = 57 bytes) | +| SessionMsg3 | ~80 bytes | Fixed (XK msg3 = 73 bytes, no coords) | | Data (minimal) | 12 + 6 + payload + 16 bytes | Steady state | | Data (with coords) | 12 + ~130 + 6 + payload + 16 bytes | Warmup/recovery | | SenderReport | 12 + 6 + 46 + 16 bytes | MMP metrics | @@ -741,6 +814,7 @@ endpoint session keys). | CoordsWarmup | 12 + coords + 6 + 16 bytes | Standalone warmup (empty body) | | CoordsRequired | 38 bytes | Fixed (prefix + msg_type + body) | | PathBroken | 35 + 16n bytes | Includes stale coords | +| MtuExceeded | 40 bytes | Fixed (prefix + msg_type + body) | ### Complete Packet Sizes (link + session) diff --git a/docs/design/spanning-tree-dynamics.md b/docs/design/spanning-tree-dynamics.md index c8608ed..d6ff2ff 100644 --- a/docs/design/spanning-tree-dynamics.md +++ b/docs/design/spanning-tree-dynamics.md @@ -1025,22 +1025,25 @@ The following limitations exist in the current implementation relative to the design described in this document. They are documented here to guide future work. -### Known Limitation: Root Timeout Not Enforced +### Root Timeout Not Enforced The design specifies a 60-minute root timeout (§6 partition detection) after -which nodes should treat the root as departed and re-elect. The current implementation does not track root entry timestamps or -perform staleness checks. +which nodes should treat the root as departed and re-elect. The current +implementation does not track root entry timestamps or perform staleness +checks. -**Impact**: If the root node disappears permanently without a graceful -disconnect, remaining nodes retain stale root state indefinitely. Nodes that -lose their direct parent will re-elect locally (via `handle_parent_lost()`), -but nodes with an intact path to a now-departed root will not detect the -failure. +**Mitigation**: Heartbeat cascading significantly reduces the practical +impact. When the root disappears, its direct children detect the parent loss +(keepalive timeout), re-elect, and announce new coordinates. This cascades +down the tree — each level's children detect their parent's changed state and +re-evaluate. For the common case of root departure, the tree reconverges +without an explicit root timeout. -**Required fix**: Track the timestamp of the most recent root declaration in -`TreeState`. In `check_tree_state()` (called every 1s from the RX loop), -compare against `root_timeout` (default 60 min). On expiration, treat it as -root loss — increment sequence number, become own root, and re-announce. +**Remaining gap**: If an intermediate node maintains a link to the root but +that link silently stops forwarding (no keepalive failure), nodes below it +would retain stale root state. This is an unusual failure mode — most link +failures are detected by keepalive timeouts. An explicit root timeout would +provide defense-in-depth for this edge case. ### Known Limitation: No TTL on Tree Entries