Update design docs for session 142 implementation changes

Comprehensive documentation review across 12 files to reflect:
- Noise XK at FSP (was IK), 3-message handshake, SessionMsg3 wire format
- Epoch exchange in Noise handshakes for peer restart detection
- Per-link MTU, min_mtu/path_mtu in lookup packets
- MtuExceeded (0x22) error signal wire format and behavior
- LookupResponse proof now covers target_coords
- Discovery reverse-path routing as primary (not greedy)
- Control socket architecture and fipsctl binary
- FSP handshake state machine (Initiating/AwaitingMsg3/Established)
- Root timeout framing updated for heartbeat cascading
This commit is contained in:
Johnathan Corgan
2026-02-22 23:03:00 +00:00
parent 2293f7d2d5
commit dc89edf60b
12 changed files with 440 additions and 148 deletions
+15
View File
@@ -49,6 +49,18 @@ transports: # Network transports (UDP, future: TCP, Tor)
peers: # Static peer list
```
### Control Socket (`node.control.*`)
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `node.control.enabled` | bool | `true` | Enable the Unix domain control socket |
| `node.control.socket_path` | string | *(auto)* | Socket file path. Default: `$XDG_RUNTIME_DIR/fips/control.sock` if `XDG_RUNTIME_DIR` is set, otherwise `/tmp/fips-control.sock` |
The control socket provides read-only access to node state via the
`fipsctl` command-line tool. See
[fips-software-architecture.md](fips-software-architecture.md) for the
protocol and command list.
All tunable protocol parameters live under `node.*`, organized as sysctl-style
dotted paths. The top-level sections (`tun`, `dns`, `transports`, `peers`)
handle infrastructure concerns only.
@@ -352,6 +364,9 @@ node:
mode: full # full | lightweight | minimal
log_interval_secs: 30
owd_window_size: 32
control:
enabled: true
socket_path: null # null = auto ($XDG_RUNTIME_DIR/fips/control.sock or /tmp/fips-control.sock)
buffers:
packet_channel: 1024
tun_channel: 1024
+29 -21
View File
@@ -241,7 +241,7 @@ FIPS uses independent encryption at two protocol layers:
| Layer | Scope | Pattern | Purpose |
| ----- | ----- | ------- | ------- |
| **FMP (Mesh)** | Hop-by-hop | Noise IK | Encrypt all traffic on each peer link |
| **FSP (Session)** | End-to-end | Noise IK | Encrypt application payload between endpoints |
| **FSP (Session)** | End-to-end | Noise XK | Encrypt application payload between endpoints |
### Link Layer (Hop-by-Hop)
@@ -258,9 +258,13 @@ from the first handshake message.
### Session Layer (End-to-End)
FIPS establishes end-to-end encrypted sessions between any two communicating
nodes using Noise IK, regardless of how many hops separate them. The
initiator knows the destination's npub; the responder learns the initiator's
identity from the handshake — the same asymmetry as link-layer connections.
nodes using Noise XK, regardless of how many hops separate them. The
initiator knows the destination's npub (required for XK's pre-message);
the responder learns the initiator's identity from the third handshake
message. Unlike the link-layer IK pattern where the initiator's identity
is revealed in msg1, XK delays identity disclosure until msg3, providing
stronger initiator identity protection for traffic traversing untrusted
intermediate nodes.
A packet from A to D through intermediate nodes B and C:
@@ -414,17 +418,19 @@ without individual discovery at each hop.
### Error Recovery
When routing fails — because cached coordinates are stale or a path has
broken — transit nodes signal the source:
When routing fails — because cached coordinates are stale, a path has
broken, or a packet exceeds a link's MTU — transit nodes signal the source:
- **CoordsRequired**: A transit node lacks the destination's coordinates.
The source re-initiates discovery and resets its coordinate warmup
strategy.
- **PathBroken**: Greedy routing reached a dead end. The source re-discovers
the destination's current coordinates.
- **MtuExceeded**: A transit node cannot forward a packet because it exceeds
the next-hop link MTU. The source adjusts its path MTU estimate.
Both signals trigger active recovery, and are rate-limited to prevent storms
during topology changes.
All three signals trigger active recovery, and are rate-limited to prevent
storms during topology changes.
See [fips-mesh-operation.md](fips-mesh-operation.md) for the complete
routing and mesh behavior description.
@@ -548,7 +554,7 @@ packets.
The most important adversary class is the operators of other nodes in the
mesh — the peers that forward your traffic. FIPS treats every intermediate
router as potentially adversarial. The FSP session layer establishes a
completely independent Noise IK session between the communicating endpoints,
completely independent Noise XK session between the communicating endpoints,
so intermediate nodes cannot read application payloads even though they
decrypt and re-encrypt the link-layer envelope at each hop.
@@ -660,19 +666,21 @@ confidentiality and integrity rather than hiding traffic patterns.
### Noise Protocol Framework
Both FMP link encryption and FSP session encryption use the
[Noise Protocol Framework](https://noiseprotocol.org/) IK handshake pattern.
Noise IK provides mutual authentication with a single round trip, where the
initiator knows the responder's static key in advance.
FIPS uses the [Noise Protocol Framework](https://noiseprotocol.org/) at both
protocol layers, with different handshake patterns chosen for each layer's
threat model. FMP link encryption uses **Noise IK**, providing mutual
authentication with a single round trip where the initiator knows the
responder's static key in advance.
[WireGuard](https://www.wireguard.com/) uses the same IK base pattern
(extended with a pre-shared key as IKpsk2) for VPN tunnels. The
[Lightning Network](https://github.com/lightning/bolts/blob/master/08-transport.md)
uses Noise XK, a related pattern where the initiator's static key is
transmitted in a third message rather than the first, providing stronger
initiator identity hiding. FIPS uses pure Noise IK at two independent layers,
sharing the same framework and cryptographic foundation as these deployed
systems while choosing the pattern that best fits its mutual-authentication
requirement.
(extended with a pre-shared key as IKpsk2) for VPN tunnels. FSP session
encryption uses **Noise XK**, the same pattern used by the
[Lightning Network](https://github.com/lightning/bolts/blob/master/08-transport.md),
where the initiator's static key is transmitted in a third message rather
than the first. XK provides stronger initiator identity hiding at the cost
of an additional round trip — a worthwhile tradeoff for session-layer traffic
that traverses untrusted intermediate nodes. At the link layer, where both
peers are configured and directly connected, IK's single round trip is
preferred.
### Index-Based Session Dispatch
+21 -1
View File
@@ -110,6 +110,25 @@ session keys derived from four DH operations (es, ss, ee, se). The handshake
provides mutual authentication, forward secrecy, and identity hiding for the
initiator.
### Epoch Exchange and Peer Restart Detection
Both IK handshake messages carry an encrypted epoch payload — an 8-byte
random value generated once at node startup:
- **msg1**: Ephemeral key (33 bytes) + encrypted static key (49 bytes) +
encrypted epoch (24 bytes) = 106 bytes total
- **msg2**: Ephemeral key (33 bytes) + encrypted epoch (24 bytes) = 57 bytes
total
The encrypted epoch (EPOCH_ENCRYPTED_SIZE = 24 bytes) consists of the
8-byte epoch value plus a 16-byte AEAD tag.
On reconnection, each peer compares the received epoch with the previously
stored epoch for that peer. An epoch mismatch indicates the peer has
restarted (generated a new epoch), triggering full link re-establishment
rather than treating the handshake as a simple reconnection. This prevents
stale session state from persisting across restarts.
### Identity Binding
The Noise handshake binds the link to the peer's cryptographic identity. After
@@ -493,7 +512,8 @@ an attacker sends invalid packets to elicit responses.
| Feature | Status |
| ------- | ------ |
| Noise IK handshake | **Implemented** |
| Noise IK handshake (with epoch) | **Implemented** |
| Peer restart detection (epoch mismatch) | **Implemented** |
| Link encryption (ChaCha20-Poly1305) | **Implemented** |
| Index-based session dispatch | **Implemented** |
| Replay protection (sliding window) | **Implemented** |
+47 -13
View File
@@ -288,17 +288,26 @@ direct peer), a LookupResponse is created containing:
- **request_id**: Echoed from the request
- **target**: The target's node_addr
- **target_coords**: The target's current tree coordinates
- **proof**: Signature covering `(request_id || target)` — authenticates
that the response is genuine
- **path_mtu**: Minimum MTU along the response path (transit-annotated,
initialized to `u16::MAX` by the target)
- **proof**: Signature covering `(request_id || target || target_coords)`
authenticates that the response is genuine and the target holds the
claimed tree position
The response routes back to the requester using greedy tree routing toward
the origin_coords from the request.
The response routes back to the requester using reverse-path routing as the
primary mechanism: each transit node looks up the request_id in its
`recent_requests` table to find the peer that forwarded the original request,
and sends the response back through that peer. This ensures the response
follows the same path as the request. Greedy tree routing toward the
origin_coords is used only as a fallback if the reverse-path entry has
expired.
**Security**: Coordinates are intentionally excluded from the signed proof.
Binding coordinates would invalidate signatures whenever the spanning tree
reconverges. Coordinate tampering by transit nodes causes only routing
inefficiency, not a security breach (data integrity is protected by
session-layer encryption).
**Proof verification**: The source verifies the Schnorr proof upon receipt,
confirming that the target actually signed the response. The proof covers
`(request_id || target || target_coords)` — coordinates are included because
verification at the source confirms the target holds the claimed position.
The `path_mtu` field is excluded from the proof because it is a transit
annotation modified at each hop.
### Discovery Outcome
@@ -402,9 +411,31 @@ source.
3. Initiate discovery for the destination
4. Reset CP warmup counter
### MtuExceeded
**Trigger**: A transit node receives a SessionDatagram but the total
packet size exceeds the next-hop link MTU. The packet cannot be forwarded
without fragmentation, which FIPS does not perform at the mesh layer.
**Transit node action**:
1. Create a new SessionDatagram addressed back to the original source,
carrying an MtuExceeded payload identifying the destination, the
reporting router, and the bottleneck MTU
2. Route the error via `find_next_hop(src_addr)`
3. Drop the original oversized packet
**Source recovery**: FSP uses the reported bottleneck MTU to adjust its
session-layer path MTU estimate (immediate decrease). The source can then
reduce payload sizes to fit within the discovered path MTU. MtuExceeded is
the reactive complement to the proactive `path_mtu` field in
SessionDatagram and LookupResponse — the proactive field tracks the
minimum MTU along the forward path, while MtuExceeded signals when an
actual packet exceeds the limit.
### Error Signal Rate Limiting
Both error types are rate-limited at transit nodes: maximum one error per
All three error types are rate-limited at transit nodes: maximum one error per
destination per 100ms. This prevents storms during topology changes when many
packets to the same destination hit the same routing failure simultaneously.
@@ -414,9 +445,9 @@ per `coords_response_interval_ms` (default 2000ms, configurable). This
prevents amplification where a burst of error signals would generate a
corresponding burst of warmup messages.
Error signals (CoordsRequired, PathBroken) are handled asynchronously outside
the packet receive path, allowing the RX loop to continue processing without
blocking on discovery or session repair.
Error signals (CoordsRequired, PathBroken, MtuExceeded) are handled
asynchronously outside the packet receive path, allowing the RX loop to
continue processing without blocking on discovery or session repair.
### Error Routing Limitation
@@ -577,6 +608,9 @@ recovery).
| Hybrid coordinate warmup (CP + CoordsWarmup) | **Implemented** |
| CoordsRequired recovery | **Implemented** |
| PathBroken recovery | **Implemented** |
| MtuExceeded recovery | **Implemented** |
| LookupResponse proof verification | **Implemented** |
| Discovery reverse-path routing | **Implemented** |
| Error signal rate limiting | **Implemented** |
| Leaf-only operation | Future direction |
| Link cost metrics (ETX) | Future direction |
+59 -31
View File
@@ -89,10 +89,15 @@ FMP signals routing failures asynchronously:
- **PathBroken**: Greedy routing reached a dead end. FSP responds by sending
a standalone CoordsWarmup (rate-limited), re-discovering the destination's
current coordinates, and resetting the warmup counter.
- **MtuExceeded**: A transit node cannot forward a SessionDatagram because
the packet exceeds the next-hop link MTU. FSP uses the reported bottleneck
MTU to adjust its session-layer path MTU estimate. MtuExceeded is the
reactive complement to the proactive `path_mtu` field in SessionDatagram.
Both signals are generated by transit nodes (not the destination) and travel
back to the source inside a new SessionDatagram. They are plaintext (not
end-to-end encrypted) because transit nodes have no session with the source.
All three signals are generated by transit nodes (not the destination) and
travel back to the source inside a new SessionDatagram. They are plaintext
(not end-to-end encrypted) because transit nodes have no session with the
source.
### Local Delivery
@@ -106,18 +111,29 @@ node, FMP delivers it to FSP for session-layer processing.
Sessions are established on demand when the first datagram needs to be sent to
a destination with no existing session.
FSP uses Noise IK for session key agreement. The initiator knows the
destination's npub (from DNS lookup or native API); the responder learns the
initiator's identity from the handshake. This is the same asymmetry as
link-layer peer connections.
FSP uses Noise XK for session key agreement. The initiator knows the
destination's npub (required for XK's pre-message `s` token); the
responder learns the initiator's identity from msg3 (not msg1, unlike
IK at the link layer). This provides stronger initiator identity hiding
— the initiator's static key is encrypted under the established shared
secret rather than under only the responder's static key.
The handshake is carried in SessionSetup and SessionAck messages:
The handshake is a three-message flow carried in SessionSetup, SessionAck,
and SessionMsg3:
1. **Initiator** sends SessionSetup containing Noise IK msg1 and both
parties' tree coordinates
2. **Responder** processes msg1, learns initiator identity, sends SessionAck
containing Noise IK msg2 and both parties' tree coordinates
3. Both parties derive identical symmetric session keys
1. **Initiator** sends SessionSetup containing Noise XK msg1 (ephemeral key
only) and both parties' tree coordinates
2. **Responder** processes msg1, sends SessionAck containing Noise XK msg2
(ephemeral key + encrypted epoch) and both parties' tree coordinates.
The responder transitions to AwaitingMsg3 state.
3. **Initiator** processes msg2, sends SessionMsg3 containing the encrypted
static key and encrypted epoch. Both parties derive identical symmetric
session keys and the session is established.
Each side's epoch (an 8-byte random value generated at startup) is
exchanged encrypted in msg2 and msg3. On subsequent handshakes, an epoch
mismatch indicates the peer has restarted, triggering session
re-establishment.
Packets that trigger session establishment are queued (with bounded buffer)
and transmitted after the session is established.
@@ -187,22 +203,30 @@ transport addresses or routing paths. A session survives:
## End-to-End Encryption
### Noise IK Pattern
### Noise XK Pattern
FSP uses the same Noise IK pattern as FMP link encryption, but with
independent keys and sessions. The full Noise descriptor is
`Noise_IK_secp256k1_ChaChaPoly_SHA256`.
FSP uses Noise XK for session encryption, distinct from the Noise IK
pattern used at the link layer. The full Noise descriptor is
`Noise_XK_secp256k1_ChaChaPoly_SHA256`.
The IK pattern:
The XK pattern (pre-message: `← s`):
- **msg1** (`→ e, es, s, ss`): Initiator sends ephemeral key, encrypts static
key to responder. Four DH operations establish session keys.
- **msg2** (`← e, ee, se`): Responder sends ephemeral key. Both parties now
share identical session keys.
- **msg1** (`→ e, es`): Initiator sends ephemeral key only. The initiator's
static identity is not revealed in this message.
- **msg2** (`← e, ee`): Responder sends ephemeral key and encrypted epoch.
- **msg3** (`→ s, se`): Initiator sends encrypted static key and encrypted
epoch. Both parties now share identical session keys.
After the handshake, Noise produces two directional symmetric keys
(`send_key`, `recv_key`) used with ChaCha20-Poly1305 for all subsequent data.
The XK pattern requires the initiator to know the responder's static key
in advance (the `← s` pre-message), which is satisfied by the discovery
or DNS lookup that precedes session establishment. In exchange, XK
provides stronger initiator identity protection than IK — the initiator's
static key is encrypted under the full shared secret (after three DH
operations) rather than under only the responder's static key.
### Cryptographic Primitives
| Component | Choice | Notes |
@@ -218,7 +242,7 @@ These choices prioritize compatibility with the Nostr cryptographic stack.
### secp256k1 Parity Normalization
Nostr npubs encode x-only public keys (32 bytes, no y-coordinate parity). The
Noise IK pre-message mixes the responder's static key as a 33-byte compressed
Noise XK pre-message mixes the responder's static key as a 33-byte compressed
key, and the default secp256k1 ECDH hash includes a parity-dependent version
byte.
@@ -229,12 +253,14 @@ actual key parity.
### Privacy Note
Noise IK does not provide initiator anonymity if the responder's static key is
compromised. An attacker who obtains the responder's nsec can decrypt the
initiator's identity from captured handshake messages. Noise XK would protect
initiator identity in this scenario but requires an additional round-trip (3
handshake messages vs. 2). The privacy/latency tradeoff may be revisited with
deployment experience.
Noise XK provides stronger initiator identity protection than IK. In XK, the
initiator's static key is encrypted in msg3 under the full shared secret
(derived from three DH operations), so an attacker who compromises only the
responder's nsec cannot decrypt the initiator's identity from captured
handshake messages (they would also need the responder's ephemeral key).
This is the primary reason FSP uses XK rather than IK — session-layer
traffic traverses untrusted intermediate nodes, making initiator identity
protection more valuable than at the link layer.
### Data Packet Authentication
@@ -460,7 +486,7 @@ feedback:
1. The source sets `path_mtu` in each SessionDatagram envelope to its
outbound link MTU.
2. Each transit node applies `min(current, transport.mtu())` before
2. Each transit node applies `min(current, transport.link_mtu(addr))` before
forwarding.
3. The destination receives the forward-path minimum and sends a
PathMtuNotification (2-byte body: u16 LE path_mtu) back to the source.
@@ -492,7 +518,9 @@ MMP session metrics session=npub1tdwa...84le rtt=4.3ms loss=0.6% jitter=0.2ms go
| Feature | Status |
| ------- | ------ |
| Session establishment (Noise IK) | **Implemented** |
| Session establishment (Noise XK) | **Implemented** |
| Peer restart detection (epoch exchange) | **Implemented** |
| MtuExceeded handling | **Implemented** |
| End-to-end encryption (ChaCha20-Poly1305) | **Implemented** |
| Explicit counter replay protection | **Implemented** |
| Hybrid coordinate warmup (CP + CoordsWarmup) | **Implemented** |
+62 -5
View File
@@ -89,7 +89,7 @@ treatment of this pattern.
## Two-Layer Encryption Rationale
FIPS uses independent Noise IK encryption at two layers:
FIPS uses independent Noise encryption at two layers (IK at FMP, XK at FSP):
| Layer | Scope | What It Protects |
| ----- | ----- | ---------------- |
@@ -108,10 +108,12 @@ FIPS uses independent Noise IK encryption at two layers:
means topology changes (a direct peer becomes multi-hop) don't affect
sessions.
**Why the same pattern (Noise IK) at both layers**: Both layers need mutual
authentication with identity hiding for the initiator. Reusing the same
cryptographic stack (secp256k1, ChaCha20-Poly1305, SHA-256) simplifies the
implementation and reduces the number of cryptographic dependencies.
**Why different patterns**: FMP uses Noise IK (2-message) because link peers
are configured and directly connected — the single round trip is preferred.
FSP uses Noise XK (3-message) because session traffic traverses untrusted
intermediate nodes — the extra round trip buys stronger initiator identity
hiding. Both layers reuse the same cryptographic stack (secp256k1,
ChaCha20-Poly1305, SHA-256), minimizing cryptographic dependencies.
## Identity Model
@@ -297,6 +299,61 @@ while the trait surface remains synchronous for `send()`, `mtu()`, etc.
The `TransportHandle` enum provides async dispatch for methods that need it
(like `send_async()`) without requiring dyn dispatch.
## Control Socket
FIPS exposes a read-only observability interface through a Unix domain
socket. The control socket provides runtime visibility into node state
without affecting protocol operation.
### Architecture
The control socket is a `UnixListener` integrated into the node's async
runtime. It accepts connections, reads a single JSON request per
connection, dispatches to query handlers that read (but never modify)
node state, and returns a JSON response.
- **Max request size**: 4,096 bytes
- **I/O timeout**: 5 seconds per connection
- **Socket cleanup**: Automatically removes stale sockets on startup
### fipsctl Binary
The `fipsctl` command-line tool communicates with the control socket:
```text
fipsctl [OPTIONS] <COMMAND>
OPTIONS:
-s, --socket <SOCKET> Control socket path override
COMMANDS:
show status Node status overview
show peers Authenticated peers
show links Active links
show tree Spanning tree state
show sessions End-to-end sessions
show bloom Bloom filter state
show mmp MMP metrics summary
show cache Coordinate cache stats
show connections Pending handshake connections
show transports Transport instances
show routing Routing table summary
```
### Protocol
Request and response are line-delimited JSON:
```json
{"command":"show_status"}
{"status":"ok","data":{...}}
```
Error responses use `{"status":"error","message":"..."}`.
All queries are read-only — the control socket cannot modify node state,
trigger actions, or shut down the node.
## References
- [fips-intro.md](fips-intro.md) — Protocol overview
+3 -2
View File
@@ -170,7 +170,8 @@ for stronger verification.
- Type: u64, Unix seconds
- Used for stale detection, not versioning
- A root declaration is considered stale after `ROOT_TIMEOUT` (60 minutes)
without refresh
without refresh. In practice, heartbeat cascading handles root departure
for the common case (see spanning-tree-dynamics.md §11)
## Reconvergence
@@ -229,7 +230,7 @@ Example: In a 1000-node network with depth 10 and 5 peers, a node stores
| --------- | ------- | ----------- |
| PARENT_SWITCH_THRESHOLD | 1 hop | Minimum depth improvement for same-root switch |
| ANNOUNCE_MIN_INTERVAL | 500ms | Minimum between announcements to same peer |
| ROOT_TIMEOUT | 60 min | Root declaration considered stale |
| ROOT_TIMEOUT | 60 min | Root declaration considered stale (not yet enforced; heartbeat cascading covers common case) |
| TREE_ENTRY_TTL | 510 min | Individual entry expiration |
## Implementation Status
+44
View File
@@ -217,6 +217,50 @@ PeerSlot::Active(ActivePeer)
- Session indices and transport address (for wire protocol dispatch)
- Statistics (`last_seen`, `link_stats`, `authenticated_at`)
### End-to-End Session Lifecycle (FSP)
```text
EndToEndState::Initiating(HandshakeState)
│ Noise XK msg1 sent (SessionSetup)
│ Awaiting msg2 (SessionAck)
│ msg2 received, processed
│ msg3 sent (SessionMsg3)
EndToEndState::Established(NoiseSession)
```
```text
[msg1 received from remote initiator]
│ Noise XK msg1 processed, msg2 sent (SessionAck)
EndToEndState::AwaitingMsg3(HandshakeState)
│ msg3 received, processed
EndToEndState::Established(NoiseSession)
```
**EndToEndState** is an enum with three variants:
- **Initiating(HandshakeState)** — We initiated the session, sent
SessionSetup with XK msg1, awaiting SessionAck (msg2)
- **AwaitingMsg3(HandshakeState)** — We are the XK responder, processed
msg1, sent msg2, awaiting msg3 from initiator
- **Established(NoiseSession)** — Handshake complete, symmetric session
keys derived, ready for application data
The three-message flow (XK pattern) means the responder has an intermediate
`AwaitingMsg3` state that does not exist in the link-layer IK handshake
(which completes in two messages). The initiator transitions directly from
Initiating to Established upon processing msg2 and sending msg3.
Each side's epoch (8-byte random startup value) is exchanged encrypted in
msg2 and msg3. Epoch mismatch on subsequent handshakes indicates peer
restart.
### Link Lifecycle (Connection-Oriented Transports)
Links use a simple state enum rather than phase-based structs, since link
+18 -12
View File
@@ -55,10 +55,15 @@ link — this is the degenerate case. Transports that negotiate MTU
per-connection (e.g., BLE ATT_MTU) report the negotiated value for each
link individually.
> **Implementation note**: The current transport trait exposes MTU as a
> transport-wide method (`fn mtu(&self) -> u16`). This works for UDP, where
> MTU is a static configuration value. Supporting per-link MTU for future
> transports will require extending this interface.
The transport trait exposes two MTU methods:
- `fn mtu(&self) -> u16` — Transport-wide default MTU
- `fn link_mtu(&self, addr: &TransportAddr) -> u16` — Per-link MTU for a
specific remote address. The default implementation falls back to
`mtu()`, so transports with uniform MTU (like UDP) need not override it.
FMP uses `link_mtu()` when computing path MTU for SessionDatagram
forwarding and LookupResponse transit annotation.
### Connection Lifecycle
@@ -305,14 +310,15 @@ The transport interface defines what every transport driver must provide.
### Trait Surface
```text
transport_id() → TransportId Unique identifier for this transport instance
transport_type() → &TransportType Static metadata (name, connection-oriented, reliable)
state() → TransportState Current lifecycle state
mtu() → u16 Maximum datagram size
start() → lifecycle Bring transport up (bind socket, open device)
stop() → lifecycle Bring transport down
send(addr, data)delivery Send datagram to transport address
discover() Vec<DiscoveredPeer> Report discovered FIPS endpoints (optional)
transport_id() → TransportId Unique identifier for this transport instance
transport_type() → &TransportType Static metadata (name, connection-oriented, reliable)
state() → TransportState Current lifecycle state
mtu() → u16 Transport-wide default MTU
link_mtu(addr) → u16 Per-link MTU (defaults to mtu())
start() → lifecycle Bring transport up (bind socket, open device)
stop() lifecycle Bring transport down
send(addr, data)delivery Send datagram to transport address
discover() → Vec<DiscoveredPeer> Report discovered FIPS endpoints (optional)
```
### Receive Path
+124 -50
View File
@@ -144,29 +144,29 @@ Handshake initiation from connecting party.
```text
┌──────────────────────┬─────────────┬─────────────────────────────────────────┐
│ common prefix │ sender_idx │ Noise IK message 1 │
│ 4 bytes │ 4 bytes LE │ 82 bytes
│ 4 bytes │ 4 bytes LE │ 106 bytes │
└──────────────────────┴─────────────┴─────────────────────────────────────────┘
Total: 90 bytes
Total: 114 bytes
```
Common prefix: ver=0, phase=0x1, flags=0, payload_len=86 (4 + 82).
Common prefix: ver=0, phase=0x1, flags=0, payload_len=110 (4 + 106).
| Field | Size | Description |
| ----- | ---- | ----------- |
| common prefix | 4 bytes | ver=0, phase=1, flags=0, payload_len |
| sender_idx | 4 bytes LE | Initiator's session index (becomes receiver's `receiver_idx`) |
| noise_msg1 | 82 bytes | Noise IK first message |
| noise_msg1 | 106 bytes | Noise IK first message |
**Noise msg1 breakdown** (82 bytes):
**Noise msg1 breakdown** (106 bytes):
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | ephemeral_pubkey | 33 bytes | Initiator's ephemeral key (compressed secp256k1) |
| 33 | encrypted_static | 33 bytes | Initiator's static key (encrypted with es key) |
| 66 | tag | 16 bytes | AEAD tag for encrypted_static |
| 33 | encrypted_static | 49 bytes | Initiator's static key (33) + AEAD tag (16) |
| 82 | encrypted_epoch | 24 bytes | Startup epoch (8) + AEAD tag (16) |
Noise pattern: `-> e, es, s, ss`
Noise pattern: `-> e, es, s, ss` with epoch payload
### Noise IK Message 2 (phase 0x2)
@@ -175,30 +175,34 @@ Handshake response from responder.
```text
┌──────────────────────┬─────────────┬──────────────┬──────────────────────────┐
│ common prefix │ sender_idx │ receiver_idx │ Noise IK message 2 │
│ 4 bytes │ 4 bytes LE │ 4 bytes LE │ 33 bytes │
│ 4 bytes │ 4 bytes LE │ 4 bytes LE │ 57 bytes │
└──────────────────────┴─────────────┴──────────────┴──────────────────────────┘
Total: 45 bytes
Total: 69 bytes
```
Common prefix: ver=0, phase=0x2, flags=0, payload_len=41 (4 + 4 + 33).
Common prefix: ver=0, phase=0x2, flags=0, payload_len=65 (4 + 4 + 57).
| Field | Size | Description |
| ----- | ---- | ----------- |
| common prefix | 4 bytes | ver=0, phase=2, flags=0, payload_len |
| sender_idx | 4 bytes LE | Responder's session index |
| receiver_idx | 4 bytes LE | Echo of initiator's sender_idx from msg1 |
| noise_msg2 | 33 bytes | Noise IK second message |
| noise_msg2 | 57 bytes | Noise IK second message |
**Noise msg2 breakdown** (33 bytes):
**Noise msg2 breakdown** (57 bytes):
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | ephemeral_pubkey | 33 bytes | Responder's ephemeral key (compressed secp256k1) |
| 33 | encrypted_epoch | 24 bytes | Startup epoch (8) + AEAD tag (16) |
Noise pattern: `<- e, ee, se`
Noise pattern: `<- e, ee, se` with epoch payload
After msg2, both parties derive identical symmetric session keys.
After msg2, both parties derive identical symmetric session keys. The
encrypted epoch in msg1 and msg2 enables peer restart detection — if a
peer's epoch changes, the other side knows it restarted and must
re-establish the link.
### Index Semantics
@@ -313,18 +317,19 @@ Coordinate discovery request, flooded through the mesh.
| 9 | target | 16 bytes | NodeAddr being sought |
| 25 | origin | 16 bytes | Requester's NodeAddr |
| 41 | ttl | 1 byte | Remaining hops (default 64) |
| 42 | origin_coords_cnt | 2 bytes LE | Number of coordinate entries |
| 44 | origin_coords | 16 x n bytes | Requester's ancestry (NodeAddr only) |
| 44 + 16n | visited_hash_cnt | 1 byte | Hash count for visited filter |
| 45 + 16n | visited_bits | 256 bytes | Compact bloom of visited nodes |
| 42 | min_mtu | 2 bytes LE | Minimum transport MTU the origin requires (0 = no requirement) |
| 44 | origin_coords_cnt | 2 bytes LE | Number of coordinate entries |
| 46 | origin_coords | 16 x n bytes | Requester's ancestry (NodeAddr only) |
| 46 + 16n | visited_hash_cnt | 1 byte | Hash count for visited filter |
| 47 + 16n | visited_bits | 256 bytes | Compact bloom of visited nodes |
**Size**: `301 + (n x 16)` bytes, where n = origin depth + 1
**Size**: `303 + (n x 16)` bytes, where n = origin depth + 1
| Origin Depth | Payload |
| ------------ | ------- |
| 3 | 349 bytes |
| 5 | 381 bytes |
| 10 | 461 bytes |
| 3 | 351 bytes |
| 5 | 383 bytes |
| 10 | 463 bytes |
### LookupResponse (0x31)
@@ -335,21 +340,28 @@ Coordinate discovery response, greedy-routed back to requester.
| 0 | msg_type | 1 byte | 0x31 |
| 1 | request_id | 8 bytes LE | Echoes the request's ID |
| 9 | target | 16 bytes | NodeAddr that was found |
| 25 | target_coords_cnt | 2 bytes LE | Number of coordinate entries |
| 27 | target_coords | 16 x n bytes | Target's ancestry (NodeAddr only) |
| 27 + 16n | proof | 64 bytes | Schnorr signature over `(request_id \|\| target)` |
| 25 | path_mtu | 2 bytes LE | Minimum MTU along response path (transit-annotated) |
| 27 | target_coords_cnt | 2 bytes LE | Number of coordinate entries |
| 29 | target_coords | 16 x n bytes | Target's ancestry (NodeAddr only) |
| 29 + 16n | proof | 64 bytes | Schnorr signature over `(request_id \|\| target \|\| target_coords)` |
**Size**: `91 + (n x 16)` bytes
**Size**: `93 + (n x 16)` bytes
| Target Depth | Payload |
| ------------ | ------- |
| 3 | 139 bytes |
| 5 | 171 bytes |
| 10 | 251 bytes |
| 3 | 141 bytes |
| 5 | 173 bytes |
| 10 | 253 bytes |
**Proof coverage**: Signs `(request_id || target)` only — coordinates are
excluded so the proof survives tree reconvergence during the lookup
round-trip.
The `path_mtu` field is initialized to `u16::MAX` by the target and each
transit hop applies `min(path_mtu, outgoing_link_mtu)`, giving the
originator an MTU estimate for the discovered path.
**Proof coverage**: Signs `(request_id || target || target_coords)`
`path_mtu` is excluded from the proof because it is a transit annotation
modified at each hop. Coordinates are included because proof verification
at the source confirms the target actually holds the claimed tree position.
The source verifies the proof upon receipt.
### SessionDatagram (0x00)
@@ -460,8 +472,9 @@ protocol version, session lifecycle phase, per-packet flags, and payload length.
| Phase | Type | Description |
| ----- | ---- | ----------- |
| 0x0 | Established | Post-handshake encrypted traffic or plaintext error signals |
| 0x1 | Handshake msg1 | SessionSetup (Noise IK msg1) |
| 0x2 | Handshake msg2 | SessionAck (Noise IK msg2) |
| 0x1 | Handshake msg1 | SessionSetup (Noise XK msg1) |
| 0x2 | Handshake msg2 | SessionAck (Noise XK msg2) |
| 0x3 | Handshake msg3 | SessionMsg3 (Noise XK msg3) |
### FSP Flags (Established Phase Only)
@@ -472,7 +485,7 @@ protocol version, session lifecycle phase, per-packet flags, and payload length.
| 2 | U (unencrypted) | Payload is plaintext (error signals) |
| 3-7 | — | Reserved (must be zero) |
Flags must be zero in handshake packets (phase 0x1 and 0x2).
Flags must be zero in handshake packets (phase 0x1, 0x2, and 0x3).
### FSP Encrypted Message (phase 0x0, U flag clear)
@@ -529,9 +542,10 @@ body.
| 0x14 | CoordsWarmup | Standalone coordinate cache warming |
| 0x20 | CoordsRequired | Error: transit node lacks destination coordinates |
| 0x21 | PathBroken | Error: greedy routing reached dead end |
| 0x22 | MtuExceeded | Error: forwarded packet exceeds next-hop MTU |
Message types 0x10-0x14 are carried inside the AEAD ciphertext (dispatched
by the `msg_type` field in the encrypted inner header). Types 0x20-0x21 are
by the `msg_type` field in the encrypted inner header). Types 0x20-0x22 are
plaintext error signals (U flag set, no encryption).
Session-layer SenderReport (0x11) and ReceiverReport (0x12) use the same
@@ -541,7 +555,9 @@ happens at the session level based on the FSP message type.
### SessionSetup (phase 0x1)
Establishes a session and warms transit coordinate caches.
Establishes a session and warms transit coordinate caches. Contains the
first message of the Noise XK handshake (ephemeral key only — the
initiator's static identity is not revealed until msg3).
Encoded with FSP prefix: ver=0, phase=0x1, flags=0, payload_len.
**Body** (after 4-byte FSP prefix):
@@ -554,11 +570,12 @@ Encoded with FSP prefix: ver=0, phase=0x1, flags=0, payload_len.
| ... | dest_coords_count | 2 bytes LE | Number of dest coordinate entries |
| ... | dest_coords | 16 x m bytes | Destination's ancestry |
| ... | handshake_len | 2 bytes LE | Noise payload length |
| ... | handshake_payload | variable | Noise IK msg1 (82 bytes typical) |
| ... | handshake_payload | variable | Noise XK msg1 (33 bytes — ephemeral key only) |
### SessionAck (phase 0x2)
Confirms session establishment, completes the Noise handshake.
Second message of the Noise XK handshake. The responder sends its
ephemeral key and encrypted epoch.
Encoded with FSP prefix: ver=0, phase=0x2, flags=0, payload_len.
**Body** (after 4-byte FSP prefix):
@@ -571,7 +588,32 @@ Encoded with FSP prefix: ver=0, phase=0x2, flags=0, payload_len.
| ... | dest_coords_count | 2 bytes LE | Number of initiator coordinate entries |
| ... | dest_coords | 16 x m bytes | Initiator's ancestry (for return-path cache warming) |
| ... | handshake_len | 2 bytes LE | Noise payload length |
| ... | handshake_payload | variable | Noise IK msg2 (33 bytes typical) |
| ... | handshake_payload | variable | Noise XK msg2 (57 bytes — ephemeral key + encrypted epoch) |
### SessionMsg3 (phase 0x3)
Third and final message of the Noise XK handshake. The initiator reveals
its encrypted static identity and epoch. After msg3, both parties derive
identical symmetric session keys and the session is established.
Encoded with FSP prefix: ver=0, phase=0x3, flags=0, payload_len.
**Body** (after 4-byte FSP prefix):
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | flags | 1 byte | Reserved |
| 1 | handshake_len | 2 bytes LE | Noise payload length |
| 3 | handshake_payload | variable | Noise XK msg3 (73 bytes — encrypted static + encrypted epoch) |
**Noise XK msg3 breakdown** (73 bytes):
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | encrypted_static | 49 bytes | Initiator's static key (33) + AEAD tag (16) |
| 49 | encrypted_epoch | 24 bytes | Startup epoch (8) + AEAD tag (16) |
SessionMsg3 does not carry coordinates — both endpoints already have each
other's coordinates from SessionSetup (msg1) and SessionAck (msg2).
### Data (0x10)
@@ -647,6 +689,28 @@ Encoded with FSP prefix: ver=0, phase=0x0, U flag set, payload_len.
| 33 | last_coords_count | 2 bytes LE | Number of stale coordinate entries |
| 35 | last_known_coords | 16 x n bytes | Stale coordinates that failed |
### MtuExceeded (0x22)
Plaintext error signal — forwarded packet exceeds the next-hop link MTU.
Sent by a transit router back to the source when a SessionDatagram cannot
be forwarded because its size exceeds the outgoing link's MTU.
Encoded with FSP prefix: ver=0, phase=0x0, U flag set, payload_len.
**Body** (after 4-byte FSP prefix + 1-byte msg_type):
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | flags | 1 byte | Reserved |
| 1 | dest_addr | 16 bytes | NodeAddr of the destination being forwarded to |
| 17 | reporter | 16 bytes | NodeAddr of the router that detected the MTU violation |
| 33 | mtu | 2 bytes LE | Bottleneck MTU at the reporting router |
**Body size**: 35 bytes. Total with prefix + msg_type: 40 bytes.
The source uses the reported MTU to adjust its session-layer path MTU
estimate. MtuExceeded is the reactive complement to the proactive
`path_mtu` field in SessionDatagram and LookupResponse.
## Encapsulation Walkthrough
A complete picture of how application data is wrapped through each layer.
@@ -709,12 +773,20 @@ endpoint session keys).
## Size Summary
### Handshake Messages
### FMP Handshake Messages (Noise IK)
| Message | Size |
| ------- | ---- |
| Noise IK msg1 | 90 bytes |
| Noise IK msg2 | 45 bytes |
| Message | Raw Noise | Wire Frame |
| ------- | --------- | ---------- |
| IK msg1 (ephemeral + encrypted static + encrypted epoch) | 106 bytes | 114 bytes |
| IK msg2 (ephemeral + encrypted epoch) | 57 bytes | 69 bytes |
### FSP Handshake Messages (Noise XK)
| Message | Raw Noise | Notes |
| ------- | --------- | ----- |
| XK msg1 (ephemeral only) | 33 bytes | Carried in SessionSetup |
| XK msg2 (ephemeral + encrypted epoch) | 57 bytes | Carried in SessionAck |
| XK msg3 (encrypted static + encrypted epoch) | 73 bytes | Carried in SessionMsg3 |
### Link-Layer Messages (inside encrypted frame)
@@ -722,8 +794,8 @@ endpoint session keys).
| ------- | ---- | ----- |
| TreeAnnounce | 100 + 32n bytes | n = depth + 1 |
| FilterAnnounce | 1,035 bytes | v1 (1KB filter) |
| LookupRequest | 301 + 16n bytes | n = origin depth + 1 |
| LookupResponse | 91 + 16n bytes | n = target depth + 1 |
| LookupRequest | 303 + 16n bytes | n = origin depth + 1 |
| LookupResponse | 93 + 16n bytes | n = target depth + 1 |
| SessionDatagram | 36 + payload bytes | Fixed 36-byte header |
| Disconnect | 2 bytes | |
@@ -731,8 +803,9 @@ endpoint session keys).
| Message | Typical Size | Notes |
| ------- | ------------ | ----- |
| SessionSetup | ~200 bytes | Depth-dependent |
| SessionAck | ~130 bytes | Depth-dependent (carries both endpoints' coords) |
| SessionSetup | ~170 bytes | Depth-dependent (XK msg1 = 33 bytes) |
| SessionAck | ~190 bytes | Depth-dependent, carries both endpoints' coords (XK msg2 = 57 bytes) |
| SessionMsg3 | ~80 bytes | Fixed (XK msg3 = 73 bytes, no coords) |
| Data (minimal) | 12 + 6 + payload + 16 bytes | Steady state |
| Data (with coords) | 12 + ~130 + 6 + payload + 16 bytes | Warmup/recovery |
| SenderReport | 12 + 6 + 46 + 16 bytes | MMP metrics |
@@ -741,6 +814,7 @@ endpoint session keys).
| CoordsWarmup | 12 + coords + 6 + 16 bytes | Standalone warmup (empty body) |
| CoordsRequired | 38 bytes | Fixed (prefix + msg_type + body) |
| PathBroken | 35 + 16n bytes | Includes stale coords |
| MtuExceeded | 40 bytes | Fixed (prefix + msg_type + body) |
### Complete Packet Sizes (link + session)
+15 -12
View File
@@ -1025,22 +1025,25 @@ The following limitations exist in the current implementation relative to the
design described in this document. They are documented here to guide future
work.
### Known Limitation: Root Timeout Not Enforced
### Root Timeout Not Enforced
The design specifies a 60-minute root timeout (§6 partition detection) after
which nodes should treat the root as departed and re-elect. The current implementation does not track root entry timestamps or
perform staleness checks.
which nodes should treat the root as departed and re-elect. The current
implementation does not track root entry timestamps or perform staleness
checks.
**Impact**: If the root node disappears permanently without a graceful
disconnect, remaining nodes retain stale root state indefinitely. Nodes that
lose their direct parent will re-elect locally (via `handle_parent_lost()`),
but nodes with an intact path to a now-departed root will not detect the
failure.
**Mitigation**: Heartbeat cascading significantly reduces the practical
impact. When the root disappears, its direct children detect the parent loss
(keepalive timeout), re-elect, and announce new coordinates. This cascades
down the tree — each level's children detect their parent's changed state and
re-evaluate. For the common case of root departure, the tree reconverges
without an explicit root timeout.
**Required fix**: Track the timestamp of the most recent root declaration in
`TreeState`. In `check_tree_state()` (called every 1s from the RX loop),
compare against `root_timeout` (default 60 min). On expiration, treat it as
root loss — increment sequence number, become own root, and re-announce.
**Remaining gap**: If an intermediate node maintains a link to the root but
that link silently stops forwarding (no keepalive failure), nodes below it
would retain stale root state. This is an unusual failure mode — most link
failures are detected by keepalive timeouts. An explicit root timeout would
provide defense-in-depth for this edge case.
### Known Limitation: No TTL on Tree Entries