mirror of
https://github.com/jmcorgan/fips.git
synced 2026-08-09 00:04:54 +00:00
Update design docs for session 142 implementation changes
Comprehensive documentation review across 12 files to reflect: - Noise XK at FSP (was IK), 3-message handshake, SessionMsg3 wire format - Epoch exchange in Noise handshakes for peer restart detection - Per-link MTU, min_mtu/path_mtu in lookup packets - MtuExceeded (0x22) error signal wire format and behavior - LookupResponse proof now covers target_coords - Discovery reverse-path routing as primary (not greedy) - Control socket architecture and fipsctl binary - FSP handshake state machine (Initiating/AwaitingMsg3/Established) - Root timeout framing updated for heartbeat cascading
This commit is contained in:
@@ -36,6 +36,8 @@ sessions across the mesh.
|
||||
unmodified IP applications
|
||||
- **Metrics Measurement Protocol** — per-link RTT, loss, jitter, and goodput
|
||||
measurement
|
||||
- **Operator visibility** — `fipsctl` control socket interface for runtime
|
||||
inspection of peers, links, sessions, tree state, and metrics
|
||||
- **Zero configuration** — sensible defaults; a node can run with no config
|
||||
file
|
||||
|
||||
@@ -81,7 +83,7 @@ a layered protocol specification. Start with
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
src/ Rust source (library + daemon binary)
|
||||
src/ Rust source (library + fips/fipsctl binaries)
|
||||
docs/design/ Protocol design specifications
|
||||
testing/ Docker-based integration test harnesses
|
||||
benches/ Criterion benchmarks
|
||||
|
||||
@@ -49,6 +49,18 @@ transports: # Network transports (UDP, future: TCP, Tor)
|
||||
peers: # Static peer list
|
||||
```
|
||||
|
||||
### Control Socket (`node.control.*`)
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `node.control.enabled` | bool | `true` | Enable the Unix domain control socket |
|
||||
| `node.control.socket_path` | string | *(auto)* | Socket file path. Default: `$XDG_RUNTIME_DIR/fips/control.sock` if `XDG_RUNTIME_DIR` is set, otherwise `/tmp/fips-control.sock` |
|
||||
|
||||
The control socket provides read-only access to node state via the
|
||||
`fipsctl` command-line tool. See
|
||||
[fips-software-architecture.md](fips-software-architecture.md) for the
|
||||
protocol and command list.
|
||||
|
||||
All tunable protocol parameters live under `node.*`, organized as sysctl-style
|
||||
dotted paths. The top-level sections (`tun`, `dns`, `transports`, `peers`)
|
||||
handle infrastructure concerns only.
|
||||
@@ -352,6 +364,9 @@ node:
|
||||
mode: full # full | lightweight | minimal
|
||||
log_interval_secs: 30
|
||||
owd_window_size: 32
|
||||
control:
|
||||
enabled: true
|
||||
socket_path: null # null = auto ($XDG_RUNTIME_DIR/fips/control.sock or /tmp/fips-control.sock)
|
||||
buffers:
|
||||
packet_channel: 1024
|
||||
tun_channel: 1024
|
||||
|
||||
+29
-21
@@ -241,7 +241,7 @@ FIPS uses independent encryption at two protocol layers:
|
||||
| Layer | Scope | Pattern | Purpose |
|
||||
| ----- | ----- | ------- | ------- |
|
||||
| **FMP (Mesh)** | Hop-by-hop | Noise IK | Encrypt all traffic on each peer link |
|
||||
| **FSP (Session)** | End-to-end | Noise IK | Encrypt application payload between endpoints |
|
||||
| **FSP (Session)** | End-to-end | Noise XK | Encrypt application payload between endpoints |
|
||||
|
||||
### Link Layer (Hop-by-Hop)
|
||||
|
||||
@@ -258,9 +258,13 @@ from the first handshake message.
|
||||
### Session Layer (End-to-End)
|
||||
|
||||
FIPS establishes end-to-end encrypted sessions between any two communicating
|
||||
nodes using Noise IK, regardless of how many hops separate them. The
|
||||
initiator knows the destination's npub; the responder learns the initiator's
|
||||
identity from the handshake — the same asymmetry as link-layer connections.
|
||||
nodes using Noise XK, regardless of how many hops separate them. The
|
||||
initiator knows the destination's npub (required for XK's pre-message);
|
||||
the responder learns the initiator's identity from the third handshake
|
||||
message. Unlike the link-layer IK pattern where the initiator's identity
|
||||
is revealed in msg1, XK delays identity disclosure until msg3, providing
|
||||
stronger initiator identity protection for traffic traversing untrusted
|
||||
intermediate nodes.
|
||||
|
||||
A packet from A to D through intermediate nodes B and C:
|
||||
|
||||
@@ -414,17 +418,19 @@ without individual discovery at each hop.
|
||||
|
||||
### Error Recovery
|
||||
|
||||
When routing fails — because cached coordinates are stale or a path has
|
||||
broken — transit nodes signal the source:
|
||||
When routing fails — because cached coordinates are stale, a path has
|
||||
broken, or a packet exceeds a link's MTU — transit nodes signal the source:
|
||||
|
||||
- **CoordsRequired**: A transit node lacks the destination's coordinates.
|
||||
The source re-initiates discovery and resets its coordinate warmup
|
||||
strategy.
|
||||
- **PathBroken**: Greedy routing reached a dead end. The source re-discovers
|
||||
the destination's current coordinates.
|
||||
- **MtuExceeded**: A transit node cannot forward a packet because it exceeds
|
||||
the next-hop link MTU. The source adjusts its path MTU estimate.
|
||||
|
||||
Both signals trigger active recovery, and are rate-limited to prevent storms
|
||||
during topology changes.
|
||||
All three signals trigger active recovery, and are rate-limited to prevent
|
||||
storms during topology changes.
|
||||
|
||||
See [fips-mesh-operation.md](fips-mesh-operation.md) for the complete
|
||||
routing and mesh behavior description.
|
||||
@@ -548,7 +554,7 @@ packets.
|
||||
The most important adversary class is the operators of other nodes in the
|
||||
mesh — the peers that forward your traffic. FIPS treats every intermediate
|
||||
router as potentially adversarial. The FSP session layer establishes a
|
||||
completely independent Noise IK session between the communicating endpoints,
|
||||
completely independent Noise XK session between the communicating endpoints,
|
||||
so intermediate nodes cannot read application payloads even though they
|
||||
decrypt and re-encrypt the link-layer envelope at each hop.
|
||||
|
||||
@@ -660,19 +666,21 @@ confidentiality and integrity rather than hiding traffic patterns.
|
||||
|
||||
### Noise Protocol Framework
|
||||
|
||||
Both FMP link encryption and FSP session encryption use the
|
||||
[Noise Protocol Framework](https://noiseprotocol.org/) IK handshake pattern.
|
||||
Noise IK provides mutual authentication with a single round trip, where the
|
||||
initiator knows the responder's static key in advance.
|
||||
FIPS uses the [Noise Protocol Framework](https://noiseprotocol.org/) at both
|
||||
protocol layers, with different handshake patterns chosen for each layer's
|
||||
threat model. FMP link encryption uses **Noise IK**, providing mutual
|
||||
authentication with a single round trip where the initiator knows the
|
||||
responder's static key in advance.
|
||||
[WireGuard](https://www.wireguard.com/) uses the same IK base pattern
|
||||
(extended with a pre-shared key as IKpsk2) for VPN tunnels. The
|
||||
[Lightning Network](https://github.com/lightning/bolts/blob/master/08-transport.md)
|
||||
uses Noise XK, a related pattern where the initiator's static key is
|
||||
transmitted in a third message rather than the first, providing stronger
|
||||
initiator identity hiding. FIPS uses pure Noise IK at two independent layers,
|
||||
sharing the same framework and cryptographic foundation as these deployed
|
||||
systems while choosing the pattern that best fits its mutual-authentication
|
||||
requirement.
|
||||
(extended with a pre-shared key as IKpsk2) for VPN tunnels. FSP session
|
||||
encryption uses **Noise XK**, the same pattern used by the
|
||||
[Lightning Network](https://github.com/lightning/bolts/blob/master/08-transport.md),
|
||||
where the initiator's static key is transmitted in a third message rather
|
||||
than the first. XK provides stronger initiator identity hiding at the cost
|
||||
of an additional round trip — a worthwhile tradeoff for session-layer traffic
|
||||
that traverses untrusted intermediate nodes. At the link layer, where both
|
||||
peers are configured and directly connected, IK's single round trip is
|
||||
preferred.
|
||||
|
||||
### Index-Based Session Dispatch
|
||||
|
||||
|
||||
@@ -110,6 +110,25 @@ session keys derived from four DH operations (es, ss, ee, se). The handshake
|
||||
provides mutual authentication, forward secrecy, and identity hiding for the
|
||||
initiator.
|
||||
|
||||
### Epoch Exchange and Peer Restart Detection
|
||||
|
||||
Both IK handshake messages carry an encrypted epoch payload — an 8-byte
|
||||
random value generated once at node startup:
|
||||
|
||||
- **msg1**: Ephemeral key (33 bytes) + encrypted static key (49 bytes) +
|
||||
encrypted epoch (24 bytes) = 106 bytes total
|
||||
- **msg2**: Ephemeral key (33 bytes) + encrypted epoch (24 bytes) = 57 bytes
|
||||
total
|
||||
|
||||
The encrypted epoch (EPOCH_ENCRYPTED_SIZE = 24 bytes) consists of the
|
||||
8-byte epoch value plus a 16-byte AEAD tag.
|
||||
|
||||
On reconnection, each peer compares the received epoch with the previously
|
||||
stored epoch for that peer. An epoch mismatch indicates the peer has
|
||||
restarted (generated a new epoch), triggering full link re-establishment
|
||||
rather than treating the handshake as a simple reconnection. This prevents
|
||||
stale session state from persisting across restarts.
|
||||
|
||||
### Identity Binding
|
||||
|
||||
The Noise handshake binds the link to the peer's cryptographic identity. After
|
||||
@@ -493,7 +512,8 @@ an attacker sends invalid packets to elicit responses.
|
||||
|
||||
| Feature | Status |
|
||||
| ------- | ------ |
|
||||
| Noise IK handshake | **Implemented** |
|
||||
| Noise IK handshake (with epoch) | **Implemented** |
|
||||
| Peer restart detection (epoch mismatch) | **Implemented** |
|
||||
| Link encryption (ChaCha20-Poly1305) | **Implemented** |
|
||||
| Index-based session dispatch | **Implemented** |
|
||||
| Replay protection (sliding window) | **Implemented** |
|
||||
|
||||
@@ -288,17 +288,26 @@ direct peer), a LookupResponse is created containing:
|
||||
- **request_id**: Echoed from the request
|
||||
- **target**: The target's node_addr
|
||||
- **target_coords**: The target's current tree coordinates
|
||||
- **proof**: Signature covering `(request_id || target)` — authenticates
|
||||
that the response is genuine
|
||||
- **path_mtu**: Minimum MTU along the response path (transit-annotated,
|
||||
initialized to `u16::MAX` by the target)
|
||||
- **proof**: Signature covering `(request_id || target || target_coords)` —
|
||||
authenticates that the response is genuine and the target holds the
|
||||
claimed tree position
|
||||
|
||||
The response routes back to the requester using greedy tree routing toward
|
||||
the origin_coords from the request.
|
||||
The response routes back to the requester using reverse-path routing as the
|
||||
primary mechanism: each transit node looks up the request_id in its
|
||||
`recent_requests` table to find the peer that forwarded the original request,
|
||||
and sends the response back through that peer. This ensures the response
|
||||
follows the same path as the request. Greedy tree routing toward the
|
||||
origin_coords is used only as a fallback if the reverse-path entry has
|
||||
expired.
|
||||
|
||||
**Security**: Coordinates are intentionally excluded from the signed proof.
|
||||
Binding coordinates would invalidate signatures whenever the spanning tree
|
||||
reconverges. Coordinate tampering by transit nodes causes only routing
|
||||
inefficiency, not a security breach (data integrity is protected by
|
||||
session-layer encryption).
|
||||
**Proof verification**: The source verifies the Schnorr proof upon receipt,
|
||||
confirming that the target actually signed the response. The proof covers
|
||||
`(request_id || target || target_coords)` — coordinates are included because
|
||||
verification at the source confirms the target holds the claimed position.
|
||||
The `path_mtu` field is excluded from the proof because it is a transit
|
||||
annotation modified at each hop.
|
||||
|
||||
### Discovery Outcome
|
||||
|
||||
@@ -402,9 +411,31 @@ source.
|
||||
3. Initiate discovery for the destination
|
||||
4. Reset CP warmup counter
|
||||
|
||||
### MtuExceeded
|
||||
|
||||
**Trigger**: A transit node receives a SessionDatagram but the total
|
||||
packet size exceeds the next-hop link MTU. The packet cannot be forwarded
|
||||
without fragmentation, which FIPS does not perform at the mesh layer.
|
||||
|
||||
**Transit node action**:
|
||||
|
||||
1. Create a new SessionDatagram addressed back to the original source,
|
||||
carrying an MtuExceeded payload identifying the destination, the
|
||||
reporting router, and the bottleneck MTU
|
||||
2. Route the error via `find_next_hop(src_addr)`
|
||||
3. Drop the original oversized packet
|
||||
|
||||
**Source recovery**: FSP uses the reported bottleneck MTU to adjust its
|
||||
session-layer path MTU estimate (immediate decrease). The source can then
|
||||
reduce payload sizes to fit within the discovered path MTU. MtuExceeded is
|
||||
the reactive complement to the proactive `path_mtu` field in
|
||||
SessionDatagram and LookupResponse — the proactive field tracks the
|
||||
minimum MTU along the forward path, while MtuExceeded signals when an
|
||||
actual packet exceeds the limit.
|
||||
|
||||
### Error Signal Rate Limiting
|
||||
|
||||
Both error types are rate-limited at transit nodes: maximum one error per
|
||||
All three error types are rate-limited at transit nodes: maximum one error per
|
||||
destination per 100ms. This prevents storms during topology changes when many
|
||||
packets to the same destination hit the same routing failure simultaneously.
|
||||
|
||||
@@ -414,9 +445,9 @@ per `coords_response_interval_ms` (default 2000ms, configurable). This
|
||||
prevents amplification where a burst of error signals would generate a
|
||||
corresponding burst of warmup messages.
|
||||
|
||||
Error signals (CoordsRequired, PathBroken) are handled asynchronously outside
|
||||
the packet receive path, allowing the RX loop to continue processing without
|
||||
blocking on discovery or session repair.
|
||||
Error signals (CoordsRequired, PathBroken, MtuExceeded) are handled
|
||||
asynchronously outside the packet receive path, allowing the RX loop to
|
||||
continue processing without blocking on discovery or session repair.
|
||||
|
||||
### Error Routing Limitation
|
||||
|
||||
@@ -577,6 +608,9 @@ recovery).
|
||||
| Hybrid coordinate warmup (CP + CoordsWarmup) | **Implemented** |
|
||||
| CoordsRequired recovery | **Implemented** |
|
||||
| PathBroken recovery | **Implemented** |
|
||||
| MtuExceeded recovery | **Implemented** |
|
||||
| LookupResponse proof verification | **Implemented** |
|
||||
| Discovery reverse-path routing | **Implemented** |
|
||||
| Error signal rate limiting | **Implemented** |
|
||||
| Leaf-only operation | Future direction |
|
||||
| Link cost metrics (ETX) | Future direction |
|
||||
|
||||
@@ -89,10 +89,15 @@ FMP signals routing failures asynchronously:
|
||||
- **PathBroken**: Greedy routing reached a dead end. FSP responds by sending
|
||||
a standalone CoordsWarmup (rate-limited), re-discovering the destination's
|
||||
current coordinates, and resetting the warmup counter.
|
||||
- **MtuExceeded**: A transit node cannot forward a SessionDatagram because
|
||||
the packet exceeds the next-hop link MTU. FSP uses the reported bottleneck
|
||||
MTU to adjust its session-layer path MTU estimate. MtuExceeded is the
|
||||
reactive complement to the proactive `path_mtu` field in SessionDatagram.
|
||||
|
||||
Both signals are generated by transit nodes (not the destination) and travel
|
||||
back to the source inside a new SessionDatagram. They are plaintext (not
|
||||
end-to-end encrypted) because transit nodes have no session with the source.
|
||||
All three signals are generated by transit nodes (not the destination) and
|
||||
travel back to the source inside a new SessionDatagram. They are plaintext
|
||||
(not end-to-end encrypted) because transit nodes have no session with the
|
||||
source.
|
||||
|
||||
### Local Delivery
|
||||
|
||||
@@ -106,18 +111,29 @@ node, FMP delivers it to FSP for session-layer processing.
|
||||
Sessions are established on demand when the first datagram needs to be sent to
|
||||
a destination with no existing session.
|
||||
|
||||
FSP uses Noise IK for session key agreement. The initiator knows the
|
||||
destination's npub (from DNS lookup or native API); the responder learns the
|
||||
initiator's identity from the handshake. This is the same asymmetry as
|
||||
link-layer peer connections.
|
||||
FSP uses Noise XK for session key agreement. The initiator knows the
|
||||
destination's npub (required for XK's pre-message `s` token); the
|
||||
responder learns the initiator's identity from msg3 (not msg1, unlike
|
||||
IK at the link layer). This provides stronger initiator identity hiding
|
||||
— the initiator's static key is encrypted under the established shared
|
||||
secret rather than under only the responder's static key.
|
||||
|
||||
The handshake is carried in SessionSetup and SessionAck messages:
|
||||
The handshake is a three-message flow carried in SessionSetup, SessionAck,
|
||||
and SessionMsg3:
|
||||
|
||||
1. **Initiator** sends SessionSetup containing Noise IK msg1 and both
|
||||
parties' tree coordinates
|
||||
2. **Responder** processes msg1, learns initiator identity, sends SessionAck
|
||||
containing Noise IK msg2 and both parties' tree coordinates
|
||||
3. Both parties derive identical symmetric session keys
|
||||
1. **Initiator** sends SessionSetup containing Noise XK msg1 (ephemeral key
|
||||
only) and both parties' tree coordinates
|
||||
2. **Responder** processes msg1, sends SessionAck containing Noise XK msg2
|
||||
(ephemeral key + encrypted epoch) and both parties' tree coordinates.
|
||||
The responder transitions to AwaitingMsg3 state.
|
||||
3. **Initiator** processes msg2, sends SessionMsg3 containing the encrypted
|
||||
static key and encrypted epoch. Both parties derive identical symmetric
|
||||
session keys and the session is established.
|
||||
|
||||
Each side's epoch (an 8-byte random value generated at startup) is
|
||||
exchanged encrypted in msg2 and msg3. On subsequent handshakes, an epoch
|
||||
mismatch indicates the peer has restarted, triggering session
|
||||
re-establishment.
|
||||
|
||||
Packets that trigger session establishment are queued (with bounded buffer)
|
||||
and transmitted after the session is established.
|
||||
@@ -187,22 +203,30 @@ transport addresses or routing paths. A session survives:
|
||||
|
||||
## End-to-End Encryption
|
||||
|
||||
### Noise IK Pattern
|
||||
### Noise XK Pattern
|
||||
|
||||
FSP uses the same Noise IK pattern as FMP link encryption, but with
|
||||
independent keys and sessions. The full Noise descriptor is
|
||||
`Noise_IK_secp256k1_ChaChaPoly_SHA256`.
|
||||
FSP uses Noise XK for session encryption, distinct from the Noise IK
|
||||
pattern used at the link layer. The full Noise descriptor is
|
||||
`Noise_XK_secp256k1_ChaChaPoly_SHA256`.
|
||||
|
||||
The IK pattern:
|
||||
The XK pattern (pre-message: `← s`):
|
||||
|
||||
- **msg1** (`→ e, es, s, ss`): Initiator sends ephemeral key, encrypts static
|
||||
key to responder. Four DH operations establish session keys.
|
||||
- **msg2** (`← e, ee, se`): Responder sends ephemeral key. Both parties now
|
||||
share identical session keys.
|
||||
- **msg1** (`→ e, es`): Initiator sends ephemeral key only. The initiator's
|
||||
static identity is not revealed in this message.
|
||||
- **msg2** (`← e, ee`): Responder sends ephemeral key and encrypted epoch.
|
||||
- **msg3** (`→ s, se`): Initiator sends encrypted static key and encrypted
|
||||
epoch. Both parties now share identical session keys.
|
||||
|
||||
After the handshake, Noise produces two directional symmetric keys
|
||||
(`send_key`, `recv_key`) used with ChaCha20-Poly1305 for all subsequent data.
|
||||
|
||||
The XK pattern requires the initiator to know the responder's static key
|
||||
in advance (the `← s` pre-message), which is satisfied by the discovery
|
||||
or DNS lookup that precedes session establishment. In exchange, XK
|
||||
provides stronger initiator identity protection than IK — the initiator's
|
||||
static key is encrypted under the full shared secret (after three DH
|
||||
operations) rather than under only the responder's static key.
|
||||
|
||||
### Cryptographic Primitives
|
||||
|
||||
| Component | Choice | Notes |
|
||||
@@ -218,7 +242,7 @@ These choices prioritize compatibility with the Nostr cryptographic stack.
|
||||
### secp256k1 Parity Normalization
|
||||
|
||||
Nostr npubs encode x-only public keys (32 bytes, no y-coordinate parity). The
|
||||
Noise IK pre-message mixes the responder's static key as a 33-byte compressed
|
||||
Noise XK pre-message mixes the responder's static key as a 33-byte compressed
|
||||
key, and the default secp256k1 ECDH hash includes a parity-dependent version
|
||||
byte.
|
||||
|
||||
@@ -229,12 +253,14 @@ actual key parity.
|
||||
|
||||
### Privacy Note
|
||||
|
||||
Noise IK does not provide initiator anonymity if the responder's static key is
|
||||
compromised. An attacker who obtains the responder's nsec can decrypt the
|
||||
initiator's identity from captured handshake messages. Noise XK would protect
|
||||
initiator identity in this scenario but requires an additional round-trip (3
|
||||
handshake messages vs. 2). The privacy/latency tradeoff may be revisited with
|
||||
deployment experience.
|
||||
Noise XK provides stronger initiator identity protection than IK. In XK, the
|
||||
initiator's static key is encrypted in msg3 under the full shared secret
|
||||
(derived from three DH operations), so an attacker who compromises only the
|
||||
responder's nsec cannot decrypt the initiator's identity from captured
|
||||
handshake messages (they would also need the responder's ephemeral key).
|
||||
This is the primary reason FSP uses XK rather than IK — session-layer
|
||||
traffic traverses untrusted intermediate nodes, making initiator identity
|
||||
protection more valuable than at the link layer.
|
||||
|
||||
### Data Packet Authentication
|
||||
|
||||
@@ -460,7 +486,7 @@ feedback:
|
||||
|
||||
1. The source sets `path_mtu` in each SessionDatagram envelope to its
|
||||
outbound link MTU.
|
||||
2. Each transit node applies `min(current, transport.mtu())` before
|
||||
2. Each transit node applies `min(current, transport.link_mtu(addr))` before
|
||||
forwarding.
|
||||
3. The destination receives the forward-path minimum and sends a
|
||||
PathMtuNotification (2-byte body: u16 LE path_mtu) back to the source.
|
||||
@@ -492,7 +518,9 @@ MMP session metrics session=npub1tdwa...84le rtt=4.3ms loss=0.6% jitter=0.2ms go
|
||||
|
||||
| Feature | Status |
|
||||
| ------- | ------ |
|
||||
| Session establishment (Noise IK) | **Implemented** |
|
||||
| Session establishment (Noise XK) | **Implemented** |
|
||||
| Peer restart detection (epoch exchange) | **Implemented** |
|
||||
| MtuExceeded handling | **Implemented** |
|
||||
| End-to-end encryption (ChaCha20-Poly1305) | **Implemented** |
|
||||
| Explicit counter replay protection | **Implemented** |
|
||||
| Hybrid coordinate warmup (CP + CoordsWarmup) | **Implemented** |
|
||||
|
||||
@@ -89,7 +89,7 @@ treatment of this pattern.
|
||||
|
||||
## Two-Layer Encryption Rationale
|
||||
|
||||
FIPS uses independent Noise IK encryption at two layers:
|
||||
FIPS uses independent Noise encryption at two layers (IK at FMP, XK at FSP):
|
||||
|
||||
| Layer | Scope | What It Protects |
|
||||
| ----- | ----- | ---------------- |
|
||||
@@ -108,10 +108,12 @@ FIPS uses independent Noise IK encryption at two layers:
|
||||
means topology changes (a direct peer becomes multi-hop) don't affect
|
||||
sessions.
|
||||
|
||||
**Why the same pattern (Noise IK) at both layers**: Both layers need mutual
|
||||
authentication with identity hiding for the initiator. Reusing the same
|
||||
cryptographic stack (secp256k1, ChaCha20-Poly1305, SHA-256) simplifies the
|
||||
implementation and reduces the number of cryptographic dependencies.
|
||||
**Why different patterns**: FMP uses Noise IK (2-message) because link peers
|
||||
are configured and directly connected — the single round trip is preferred.
|
||||
FSP uses Noise XK (3-message) because session traffic traverses untrusted
|
||||
intermediate nodes — the extra round trip buys stronger initiator identity
|
||||
hiding. Both layers reuse the same cryptographic stack (secp256k1,
|
||||
ChaCha20-Poly1305, SHA-256), minimizing cryptographic dependencies.
|
||||
|
||||
## Identity Model
|
||||
|
||||
@@ -297,6 +299,61 @@ while the trait surface remains synchronous for `send()`, `mtu()`, etc.
|
||||
The `TransportHandle` enum provides async dispatch for methods that need it
|
||||
(like `send_async()`) without requiring dyn dispatch.
|
||||
|
||||
## Control Socket
|
||||
|
||||
FIPS exposes a read-only observability interface through a Unix domain
|
||||
socket. The control socket provides runtime visibility into node state
|
||||
without affecting protocol operation.
|
||||
|
||||
### Architecture
|
||||
|
||||
The control socket is a `UnixListener` integrated into the node's async
|
||||
runtime. It accepts connections, reads a single JSON request per
|
||||
connection, dispatches to query handlers that read (but never modify)
|
||||
node state, and returns a JSON response.
|
||||
|
||||
- **Max request size**: 4,096 bytes
|
||||
- **I/O timeout**: 5 seconds per connection
|
||||
- **Socket cleanup**: Automatically removes stale sockets on startup
|
||||
|
||||
### fipsctl Binary
|
||||
|
||||
The `fipsctl` command-line tool communicates with the control socket:
|
||||
|
||||
```text
|
||||
fipsctl [OPTIONS] <COMMAND>
|
||||
|
||||
OPTIONS:
|
||||
-s, --socket <SOCKET> Control socket path override
|
||||
|
||||
COMMANDS:
|
||||
show status Node status overview
|
||||
show peers Authenticated peers
|
||||
show links Active links
|
||||
show tree Spanning tree state
|
||||
show sessions End-to-end sessions
|
||||
show bloom Bloom filter state
|
||||
show mmp MMP metrics summary
|
||||
show cache Coordinate cache stats
|
||||
show connections Pending handshake connections
|
||||
show transports Transport instances
|
||||
show routing Routing table summary
|
||||
```
|
||||
|
||||
### Protocol
|
||||
|
||||
Request and response are line-delimited JSON:
|
||||
|
||||
```json
|
||||
{"command":"show_status"}
|
||||
{"status":"ok","data":{...}}
|
||||
```
|
||||
|
||||
Error responses use `{"status":"error","message":"..."}`.
|
||||
|
||||
All queries are read-only — the control socket cannot modify node state,
|
||||
trigger actions, or shut down the node.
|
||||
|
||||
## References
|
||||
|
||||
- [fips-intro.md](fips-intro.md) — Protocol overview
|
||||
|
||||
@@ -170,7 +170,8 @@ for stronger verification.
|
||||
- Type: u64, Unix seconds
|
||||
- Used for stale detection, not versioning
|
||||
- A root declaration is considered stale after `ROOT_TIMEOUT` (60 minutes)
|
||||
without refresh
|
||||
without refresh. In practice, heartbeat cascading handles root departure
|
||||
for the common case (see spanning-tree-dynamics.md §11)
|
||||
|
||||
## Reconvergence
|
||||
|
||||
@@ -229,7 +230,7 @@ Example: In a 1000-node network with depth 10 and 5 peers, a node stores
|
||||
| --------- | ------- | ----------- |
|
||||
| PARENT_SWITCH_THRESHOLD | 1 hop | Minimum depth improvement for same-root switch |
|
||||
| ANNOUNCE_MIN_INTERVAL | 500ms | Minimum between announcements to same peer |
|
||||
| ROOT_TIMEOUT | 60 min | Root declaration considered stale |
|
||||
| ROOT_TIMEOUT | 60 min | Root declaration considered stale (not yet enforced; heartbeat cascading covers common case) |
|
||||
| TREE_ENTRY_TTL | 5–10 min | Individual entry expiration |
|
||||
|
||||
## Implementation Status
|
||||
|
||||
@@ -217,6 +217,50 @@ PeerSlot::Active(ActivePeer)
|
||||
- Session indices and transport address (for wire protocol dispatch)
|
||||
- Statistics (`last_seen`, `link_stats`, `authenticated_at`)
|
||||
|
||||
### End-to-End Session Lifecycle (FSP)
|
||||
|
||||
```text
|
||||
EndToEndState::Initiating(HandshakeState)
|
||||
│
|
||||
│ Noise XK msg1 sent (SessionSetup)
|
||||
│ Awaiting msg2 (SessionAck)
|
||||
│
|
||||
│ msg2 received, processed
|
||||
│ msg3 sent (SessionMsg3)
|
||||
▼
|
||||
EndToEndState::Established(NoiseSession)
|
||||
```
|
||||
|
||||
```text
|
||||
[msg1 received from remote initiator]
|
||||
│
|
||||
│ Noise XK msg1 processed, msg2 sent (SessionAck)
|
||||
▼
|
||||
EndToEndState::AwaitingMsg3(HandshakeState)
|
||||
│
|
||||
│ msg3 received, processed
|
||||
▼
|
||||
EndToEndState::Established(NoiseSession)
|
||||
```
|
||||
|
||||
**EndToEndState** is an enum with three variants:
|
||||
|
||||
- **Initiating(HandshakeState)** — We initiated the session, sent
|
||||
SessionSetup with XK msg1, awaiting SessionAck (msg2)
|
||||
- **AwaitingMsg3(HandshakeState)** — We are the XK responder, processed
|
||||
msg1, sent msg2, awaiting msg3 from initiator
|
||||
- **Established(NoiseSession)** — Handshake complete, symmetric session
|
||||
keys derived, ready for application data
|
||||
|
||||
The three-message flow (XK pattern) means the responder has an intermediate
|
||||
`AwaitingMsg3` state that does not exist in the link-layer IK handshake
|
||||
(which completes in two messages). The initiator transitions directly from
|
||||
Initiating to Established upon processing msg2 and sending msg3.
|
||||
|
||||
Each side's epoch (8-byte random startup value) is exchanged encrypted in
|
||||
msg2 and msg3. Epoch mismatch on subsequent handshakes indicates peer
|
||||
restart.
|
||||
|
||||
### Link Lifecycle (Connection-Oriented Transports)
|
||||
|
||||
Links use a simple state enum rather than phase-based structs, since link
|
||||
|
||||
@@ -55,10 +55,15 @@ link — this is the degenerate case. Transports that negotiate MTU
|
||||
per-connection (e.g., BLE ATT_MTU) report the negotiated value for each
|
||||
link individually.
|
||||
|
||||
> **Implementation note**: The current transport trait exposes MTU as a
|
||||
> transport-wide method (`fn mtu(&self) -> u16`). This works for UDP, where
|
||||
> MTU is a static configuration value. Supporting per-link MTU for future
|
||||
> transports will require extending this interface.
|
||||
The transport trait exposes two MTU methods:
|
||||
|
||||
- `fn mtu(&self) -> u16` — Transport-wide default MTU
|
||||
- `fn link_mtu(&self, addr: &TransportAddr) -> u16` — Per-link MTU for a
|
||||
specific remote address. The default implementation falls back to
|
||||
`mtu()`, so transports with uniform MTU (like UDP) need not override it.
|
||||
|
||||
FMP uses `link_mtu()` when computing path MTU for SessionDatagram
|
||||
forwarding and LookupResponse transit annotation.
|
||||
|
||||
### Connection Lifecycle
|
||||
|
||||
@@ -305,14 +310,15 @@ The transport interface defines what every transport driver must provide.
|
||||
### Trait Surface
|
||||
|
||||
```text
|
||||
transport_id() → TransportId Unique identifier for this transport instance
|
||||
transport_type() → &TransportType Static metadata (name, connection-oriented, reliable)
|
||||
state() → TransportState Current lifecycle state
|
||||
mtu() → u16 Maximum datagram size
|
||||
start() → lifecycle Bring transport up (bind socket, open device)
|
||||
stop() → lifecycle Bring transport down
|
||||
send(addr, data) → delivery Send datagram to transport address
|
||||
discover() → Vec<DiscoveredPeer> Report discovered FIPS endpoints (optional)
|
||||
transport_id() → TransportId Unique identifier for this transport instance
|
||||
transport_type() → &TransportType Static metadata (name, connection-oriented, reliable)
|
||||
state() → TransportState Current lifecycle state
|
||||
mtu() → u16 Transport-wide default MTU
|
||||
link_mtu(addr) → u16 Per-link MTU (defaults to mtu())
|
||||
start() → lifecycle Bring transport up (bind socket, open device)
|
||||
stop() → lifecycle Bring transport down
|
||||
send(addr, data) → delivery Send datagram to transport address
|
||||
discover() → Vec<DiscoveredPeer> Report discovered FIPS endpoints (optional)
|
||||
```
|
||||
|
||||
### Receive Path
|
||||
|
||||
@@ -144,29 +144,29 @@ Handshake initiation from connecting party.
|
||||
```text
|
||||
┌──────────────────────┬─────────────┬─────────────────────────────────────────┐
|
||||
│ common prefix │ sender_idx │ Noise IK message 1 │
|
||||
│ 4 bytes │ 4 bytes LE │ 82 bytes │
|
||||
│ 4 bytes │ 4 bytes LE │ 106 bytes │
|
||||
└──────────────────────┴─────────────┴─────────────────────────────────────────┘
|
||||
|
||||
Total: 90 bytes
|
||||
Total: 114 bytes
|
||||
```
|
||||
|
||||
Common prefix: ver=0, phase=0x1, flags=0, payload_len=86 (4 + 82).
|
||||
Common prefix: ver=0, phase=0x1, flags=0, payload_len=110 (4 + 106).
|
||||
|
||||
| Field | Size | Description |
|
||||
| ----- | ---- | ----------- |
|
||||
| common prefix | 4 bytes | ver=0, phase=1, flags=0, payload_len |
|
||||
| sender_idx | 4 bytes LE | Initiator's session index (becomes receiver's `receiver_idx`) |
|
||||
| noise_msg1 | 82 bytes | Noise IK first message |
|
||||
| noise_msg1 | 106 bytes | Noise IK first message |
|
||||
|
||||
**Noise msg1 breakdown** (82 bytes):
|
||||
**Noise msg1 breakdown** (106 bytes):
|
||||
|
||||
| Offset | Field | Size | Description |
|
||||
| ------ | ----- | ---- | ----------- |
|
||||
| 0 | ephemeral_pubkey | 33 bytes | Initiator's ephemeral key (compressed secp256k1) |
|
||||
| 33 | encrypted_static | 33 bytes | Initiator's static key (encrypted with es key) |
|
||||
| 66 | tag | 16 bytes | AEAD tag for encrypted_static |
|
||||
| 33 | encrypted_static | 49 bytes | Initiator's static key (33) + AEAD tag (16) |
|
||||
| 82 | encrypted_epoch | 24 bytes | Startup epoch (8) + AEAD tag (16) |
|
||||
|
||||
Noise pattern: `-> e, es, s, ss`
|
||||
Noise pattern: `-> e, es, s, ss` with epoch payload
|
||||
|
||||
### Noise IK Message 2 (phase 0x2)
|
||||
|
||||
@@ -175,30 +175,34 @@ Handshake response from responder.
|
||||
```text
|
||||
┌──────────────────────┬─────────────┬──────────────┬──────────────────────────┐
|
||||
│ common prefix │ sender_idx │ receiver_idx │ Noise IK message 2 │
|
||||
│ 4 bytes │ 4 bytes LE │ 4 bytes LE │ 33 bytes │
|
||||
│ 4 bytes │ 4 bytes LE │ 4 bytes LE │ 57 bytes │
|
||||
└──────────────────────┴─────────────┴──────────────┴──────────────────────────┘
|
||||
|
||||
Total: 45 bytes
|
||||
Total: 69 bytes
|
||||
```
|
||||
|
||||
Common prefix: ver=0, phase=0x2, flags=0, payload_len=41 (4 + 4 + 33).
|
||||
Common prefix: ver=0, phase=0x2, flags=0, payload_len=65 (4 + 4 + 57).
|
||||
|
||||
| Field | Size | Description |
|
||||
| ----- | ---- | ----------- |
|
||||
| common prefix | 4 bytes | ver=0, phase=2, flags=0, payload_len |
|
||||
| sender_idx | 4 bytes LE | Responder's session index |
|
||||
| receiver_idx | 4 bytes LE | Echo of initiator's sender_idx from msg1 |
|
||||
| noise_msg2 | 33 bytes | Noise IK second message |
|
||||
| noise_msg2 | 57 bytes | Noise IK second message |
|
||||
|
||||
**Noise msg2 breakdown** (33 bytes):
|
||||
**Noise msg2 breakdown** (57 bytes):
|
||||
|
||||
| Offset | Field | Size | Description |
|
||||
| ------ | ----- | ---- | ----------- |
|
||||
| 0 | ephemeral_pubkey | 33 bytes | Responder's ephemeral key (compressed secp256k1) |
|
||||
| 33 | encrypted_epoch | 24 bytes | Startup epoch (8) + AEAD tag (16) |
|
||||
|
||||
Noise pattern: `<- e, ee, se`
|
||||
Noise pattern: `<- e, ee, se` with epoch payload
|
||||
|
||||
After msg2, both parties derive identical symmetric session keys.
|
||||
After msg2, both parties derive identical symmetric session keys. The
|
||||
encrypted epoch in msg1 and msg2 enables peer restart detection — if a
|
||||
peer's epoch changes, the other side knows it restarted and must
|
||||
re-establish the link.
|
||||
|
||||
### Index Semantics
|
||||
|
||||
@@ -313,18 +317,19 @@ Coordinate discovery request, flooded through the mesh.
|
||||
| 9 | target | 16 bytes | NodeAddr being sought |
|
||||
| 25 | origin | 16 bytes | Requester's NodeAddr |
|
||||
| 41 | ttl | 1 byte | Remaining hops (default 64) |
|
||||
| 42 | origin_coords_cnt | 2 bytes LE | Number of coordinate entries |
|
||||
| 44 | origin_coords | 16 x n bytes | Requester's ancestry (NodeAddr only) |
|
||||
| 44 + 16n | visited_hash_cnt | 1 byte | Hash count for visited filter |
|
||||
| 45 + 16n | visited_bits | 256 bytes | Compact bloom of visited nodes |
|
||||
| 42 | min_mtu | 2 bytes LE | Minimum transport MTU the origin requires (0 = no requirement) |
|
||||
| 44 | origin_coords_cnt | 2 bytes LE | Number of coordinate entries |
|
||||
| 46 | origin_coords | 16 x n bytes | Requester's ancestry (NodeAddr only) |
|
||||
| 46 + 16n | visited_hash_cnt | 1 byte | Hash count for visited filter |
|
||||
| 47 + 16n | visited_bits | 256 bytes | Compact bloom of visited nodes |
|
||||
|
||||
**Size**: `301 + (n x 16)` bytes, where n = origin depth + 1
|
||||
**Size**: `303 + (n x 16)` bytes, where n = origin depth + 1
|
||||
|
||||
| Origin Depth | Payload |
|
||||
| ------------ | ------- |
|
||||
| 3 | 349 bytes |
|
||||
| 5 | 381 bytes |
|
||||
| 10 | 461 bytes |
|
||||
| 3 | 351 bytes |
|
||||
| 5 | 383 bytes |
|
||||
| 10 | 463 bytes |
|
||||
|
||||
### LookupResponse (0x31)
|
||||
|
||||
@@ -335,21 +340,28 @@ Coordinate discovery response, greedy-routed back to requester.
|
||||
| 0 | msg_type | 1 byte | 0x31 |
|
||||
| 1 | request_id | 8 bytes LE | Echoes the request's ID |
|
||||
| 9 | target | 16 bytes | NodeAddr that was found |
|
||||
| 25 | target_coords_cnt | 2 bytes LE | Number of coordinate entries |
|
||||
| 27 | target_coords | 16 x n bytes | Target's ancestry (NodeAddr only) |
|
||||
| 27 + 16n | proof | 64 bytes | Schnorr signature over `(request_id \|\| target)` |
|
||||
| 25 | path_mtu | 2 bytes LE | Minimum MTU along response path (transit-annotated) |
|
||||
| 27 | target_coords_cnt | 2 bytes LE | Number of coordinate entries |
|
||||
| 29 | target_coords | 16 x n bytes | Target's ancestry (NodeAddr only) |
|
||||
| 29 + 16n | proof | 64 bytes | Schnorr signature over `(request_id \|\| target \|\| target_coords)` |
|
||||
|
||||
**Size**: `91 + (n x 16)` bytes
|
||||
**Size**: `93 + (n x 16)` bytes
|
||||
|
||||
| Target Depth | Payload |
|
||||
| ------------ | ------- |
|
||||
| 3 | 139 bytes |
|
||||
| 5 | 171 bytes |
|
||||
| 10 | 251 bytes |
|
||||
| 3 | 141 bytes |
|
||||
| 5 | 173 bytes |
|
||||
| 10 | 253 bytes |
|
||||
|
||||
**Proof coverage**: Signs `(request_id || target)` only — coordinates are
|
||||
excluded so the proof survives tree reconvergence during the lookup
|
||||
round-trip.
|
||||
The `path_mtu` field is initialized to `u16::MAX` by the target and each
|
||||
transit hop applies `min(path_mtu, outgoing_link_mtu)`, giving the
|
||||
originator an MTU estimate for the discovered path.
|
||||
|
||||
**Proof coverage**: Signs `(request_id || target || target_coords)` —
|
||||
`path_mtu` is excluded from the proof because it is a transit annotation
|
||||
modified at each hop. Coordinates are included because proof verification
|
||||
at the source confirms the target actually holds the claimed tree position.
|
||||
The source verifies the proof upon receipt.
|
||||
|
||||
### SessionDatagram (0x00)
|
||||
|
||||
@@ -460,8 +472,9 @@ protocol version, session lifecycle phase, per-packet flags, and payload length.
|
||||
| Phase | Type | Description |
|
||||
| ----- | ---- | ----------- |
|
||||
| 0x0 | Established | Post-handshake encrypted traffic or plaintext error signals |
|
||||
| 0x1 | Handshake msg1 | SessionSetup (Noise IK msg1) |
|
||||
| 0x2 | Handshake msg2 | SessionAck (Noise IK msg2) |
|
||||
| 0x1 | Handshake msg1 | SessionSetup (Noise XK msg1) |
|
||||
| 0x2 | Handshake msg2 | SessionAck (Noise XK msg2) |
|
||||
| 0x3 | Handshake msg3 | SessionMsg3 (Noise XK msg3) |
|
||||
|
||||
### FSP Flags (Established Phase Only)
|
||||
|
||||
@@ -472,7 +485,7 @@ protocol version, session lifecycle phase, per-packet flags, and payload length.
|
||||
| 2 | U (unencrypted) | Payload is plaintext (error signals) |
|
||||
| 3-7 | — | Reserved (must be zero) |
|
||||
|
||||
Flags must be zero in handshake packets (phase 0x1 and 0x2).
|
||||
Flags must be zero in handshake packets (phase 0x1, 0x2, and 0x3).
|
||||
|
||||
### FSP Encrypted Message (phase 0x0, U flag clear)
|
||||
|
||||
@@ -529,9 +542,10 @@ body.
|
||||
| 0x14 | CoordsWarmup | Standalone coordinate cache warming |
|
||||
| 0x20 | CoordsRequired | Error: transit node lacks destination coordinates |
|
||||
| 0x21 | PathBroken | Error: greedy routing reached dead end |
|
||||
| 0x22 | MtuExceeded | Error: forwarded packet exceeds next-hop MTU |
|
||||
|
||||
Message types 0x10-0x14 are carried inside the AEAD ciphertext (dispatched
|
||||
by the `msg_type` field in the encrypted inner header). Types 0x20-0x21 are
|
||||
by the `msg_type` field in the encrypted inner header). Types 0x20-0x22 are
|
||||
plaintext error signals (U flag set, no encryption).
|
||||
|
||||
Session-layer SenderReport (0x11) and ReceiverReport (0x12) use the same
|
||||
@@ -541,7 +555,9 @@ happens at the session level based on the FSP message type.
|
||||
|
||||
### SessionSetup (phase 0x1)
|
||||
|
||||
Establishes a session and warms transit coordinate caches.
|
||||
Establishes a session and warms transit coordinate caches. Contains the
|
||||
first message of the Noise XK handshake (ephemeral key only — the
|
||||
initiator's static identity is not revealed until msg3).
|
||||
Encoded with FSP prefix: ver=0, phase=0x1, flags=0, payload_len.
|
||||
|
||||
**Body** (after 4-byte FSP prefix):
|
||||
@@ -554,11 +570,12 @@ Encoded with FSP prefix: ver=0, phase=0x1, flags=0, payload_len.
|
||||
| ... | dest_coords_count | 2 bytes LE | Number of dest coordinate entries |
|
||||
| ... | dest_coords | 16 x m bytes | Destination's ancestry |
|
||||
| ... | handshake_len | 2 bytes LE | Noise payload length |
|
||||
| ... | handshake_payload | variable | Noise IK msg1 (82 bytes typical) |
|
||||
| ... | handshake_payload | variable | Noise XK msg1 (33 bytes — ephemeral key only) |
|
||||
|
||||
### SessionAck (phase 0x2)
|
||||
|
||||
Confirms session establishment, completes the Noise handshake.
|
||||
Second message of the Noise XK handshake. The responder sends its
|
||||
ephemeral key and encrypted epoch.
|
||||
Encoded with FSP prefix: ver=0, phase=0x2, flags=0, payload_len.
|
||||
|
||||
**Body** (after 4-byte FSP prefix):
|
||||
@@ -571,7 +588,32 @@ Encoded with FSP prefix: ver=0, phase=0x2, flags=0, payload_len.
|
||||
| ... | dest_coords_count | 2 bytes LE | Number of initiator coordinate entries |
|
||||
| ... | dest_coords | 16 x m bytes | Initiator's ancestry (for return-path cache warming) |
|
||||
| ... | handshake_len | 2 bytes LE | Noise payload length |
|
||||
| ... | handshake_payload | variable | Noise IK msg2 (33 bytes typical) |
|
||||
| ... | handshake_payload | variable | Noise XK msg2 (57 bytes — ephemeral key + encrypted epoch) |
|
||||
|
||||
### SessionMsg3 (phase 0x3)
|
||||
|
||||
Third and final message of the Noise XK handshake. The initiator reveals
|
||||
its encrypted static identity and epoch. After msg3, both parties derive
|
||||
identical symmetric session keys and the session is established.
|
||||
Encoded with FSP prefix: ver=0, phase=0x3, flags=0, payload_len.
|
||||
|
||||
**Body** (after 4-byte FSP prefix):
|
||||
|
||||
| Offset | Field | Size | Description |
|
||||
| ------ | ----- | ---- | ----------- |
|
||||
| 0 | flags | 1 byte | Reserved |
|
||||
| 1 | handshake_len | 2 bytes LE | Noise payload length |
|
||||
| 3 | handshake_payload | variable | Noise XK msg3 (73 bytes — encrypted static + encrypted epoch) |
|
||||
|
||||
**Noise XK msg3 breakdown** (73 bytes):
|
||||
|
||||
| Offset | Field | Size | Description |
|
||||
| ------ | ----- | ---- | ----------- |
|
||||
| 0 | encrypted_static | 49 bytes | Initiator's static key (33) + AEAD tag (16) |
|
||||
| 49 | encrypted_epoch | 24 bytes | Startup epoch (8) + AEAD tag (16) |
|
||||
|
||||
SessionMsg3 does not carry coordinates — both endpoints already have each
|
||||
other's coordinates from SessionSetup (msg1) and SessionAck (msg2).
|
||||
|
||||
### Data (0x10)
|
||||
|
||||
@@ -647,6 +689,28 @@ Encoded with FSP prefix: ver=0, phase=0x0, U flag set, payload_len.
|
||||
| 33 | last_coords_count | 2 bytes LE | Number of stale coordinate entries |
|
||||
| 35 | last_known_coords | 16 x n bytes | Stale coordinates that failed |
|
||||
|
||||
### MtuExceeded (0x22)
|
||||
|
||||
Plaintext error signal — forwarded packet exceeds the next-hop link MTU.
|
||||
Sent by a transit router back to the source when a SessionDatagram cannot
|
||||
be forwarded because its size exceeds the outgoing link's MTU.
|
||||
Encoded with FSP prefix: ver=0, phase=0x0, U flag set, payload_len.
|
||||
|
||||
**Body** (after 4-byte FSP prefix + 1-byte msg_type):
|
||||
|
||||
| Offset | Field | Size | Description |
|
||||
| ------ | ----- | ---- | ----------- |
|
||||
| 0 | flags | 1 byte | Reserved |
|
||||
| 1 | dest_addr | 16 bytes | NodeAddr of the destination being forwarded to |
|
||||
| 17 | reporter | 16 bytes | NodeAddr of the router that detected the MTU violation |
|
||||
| 33 | mtu | 2 bytes LE | Bottleneck MTU at the reporting router |
|
||||
|
||||
**Body size**: 35 bytes. Total with prefix + msg_type: 40 bytes.
|
||||
|
||||
The source uses the reported MTU to adjust its session-layer path MTU
|
||||
estimate. MtuExceeded is the reactive complement to the proactive
|
||||
`path_mtu` field in SessionDatagram and LookupResponse.
|
||||
|
||||
## Encapsulation Walkthrough
|
||||
|
||||
A complete picture of how application data is wrapped through each layer.
|
||||
@@ -709,12 +773,20 @@ endpoint session keys).
|
||||
|
||||
## Size Summary
|
||||
|
||||
### Handshake Messages
|
||||
### FMP Handshake Messages (Noise IK)
|
||||
|
||||
| Message | Size |
|
||||
| ------- | ---- |
|
||||
| Noise IK msg1 | 90 bytes |
|
||||
| Noise IK msg2 | 45 bytes |
|
||||
| Message | Raw Noise | Wire Frame |
|
||||
| ------- | --------- | ---------- |
|
||||
| IK msg1 (ephemeral + encrypted static + encrypted epoch) | 106 bytes | 114 bytes |
|
||||
| IK msg2 (ephemeral + encrypted epoch) | 57 bytes | 69 bytes |
|
||||
|
||||
### FSP Handshake Messages (Noise XK)
|
||||
|
||||
| Message | Raw Noise | Notes |
|
||||
| ------- | --------- | ----- |
|
||||
| XK msg1 (ephemeral only) | 33 bytes | Carried in SessionSetup |
|
||||
| XK msg2 (ephemeral + encrypted epoch) | 57 bytes | Carried in SessionAck |
|
||||
| XK msg3 (encrypted static + encrypted epoch) | 73 bytes | Carried in SessionMsg3 |
|
||||
|
||||
### Link-Layer Messages (inside encrypted frame)
|
||||
|
||||
@@ -722,8 +794,8 @@ endpoint session keys).
|
||||
| ------- | ---- | ----- |
|
||||
| TreeAnnounce | 100 + 32n bytes | n = depth + 1 |
|
||||
| FilterAnnounce | 1,035 bytes | v1 (1KB filter) |
|
||||
| LookupRequest | 301 + 16n bytes | n = origin depth + 1 |
|
||||
| LookupResponse | 91 + 16n bytes | n = target depth + 1 |
|
||||
| LookupRequest | 303 + 16n bytes | n = origin depth + 1 |
|
||||
| LookupResponse | 93 + 16n bytes | n = target depth + 1 |
|
||||
| SessionDatagram | 36 + payload bytes | Fixed 36-byte header |
|
||||
| Disconnect | 2 bytes | |
|
||||
|
||||
@@ -731,8 +803,9 @@ endpoint session keys).
|
||||
|
||||
| Message | Typical Size | Notes |
|
||||
| ------- | ------------ | ----- |
|
||||
| SessionSetup | ~200 bytes | Depth-dependent |
|
||||
| SessionAck | ~130 bytes | Depth-dependent (carries both endpoints' coords) |
|
||||
| SessionSetup | ~170 bytes | Depth-dependent (XK msg1 = 33 bytes) |
|
||||
| SessionAck | ~190 bytes | Depth-dependent, carries both endpoints' coords (XK msg2 = 57 bytes) |
|
||||
| SessionMsg3 | ~80 bytes | Fixed (XK msg3 = 73 bytes, no coords) |
|
||||
| Data (minimal) | 12 + 6 + payload + 16 bytes | Steady state |
|
||||
| Data (with coords) | 12 + ~130 + 6 + payload + 16 bytes | Warmup/recovery |
|
||||
| SenderReport | 12 + 6 + 46 + 16 bytes | MMP metrics |
|
||||
@@ -741,6 +814,7 @@ endpoint session keys).
|
||||
| CoordsWarmup | 12 + coords + 6 + 16 bytes | Standalone warmup (empty body) |
|
||||
| CoordsRequired | 38 bytes | Fixed (prefix + msg_type + body) |
|
||||
| PathBroken | 35 + 16n bytes | Includes stale coords |
|
||||
| MtuExceeded | 40 bytes | Fixed (prefix + msg_type + body) |
|
||||
|
||||
### Complete Packet Sizes (link + session)
|
||||
|
||||
|
||||
@@ -1025,22 +1025,25 @@ The following limitations exist in the current implementation relative to the
|
||||
design described in this document. They are documented here to guide future
|
||||
work.
|
||||
|
||||
### Known Limitation: Root Timeout Not Enforced
|
||||
### Root Timeout Not Enforced
|
||||
|
||||
The design specifies a 60-minute root timeout (§6 partition detection) after
|
||||
which nodes should treat the root as departed and re-elect. The current implementation does not track root entry timestamps or
|
||||
perform staleness checks.
|
||||
which nodes should treat the root as departed and re-elect. The current
|
||||
implementation does not track root entry timestamps or perform staleness
|
||||
checks.
|
||||
|
||||
**Impact**: If the root node disappears permanently without a graceful
|
||||
disconnect, remaining nodes retain stale root state indefinitely. Nodes that
|
||||
lose their direct parent will re-elect locally (via `handle_parent_lost()`),
|
||||
but nodes with an intact path to a now-departed root will not detect the
|
||||
failure.
|
||||
**Mitigation**: Heartbeat cascading significantly reduces the practical
|
||||
impact. When the root disappears, its direct children detect the parent loss
|
||||
(keepalive timeout), re-elect, and announce new coordinates. This cascades
|
||||
down the tree — each level's children detect their parent's changed state and
|
||||
re-evaluate. For the common case of root departure, the tree reconverges
|
||||
without an explicit root timeout.
|
||||
|
||||
**Required fix**: Track the timestamp of the most recent root declaration in
|
||||
`TreeState`. In `check_tree_state()` (called every 1s from the RX loop),
|
||||
compare against `root_timeout` (default 60 min). On expiration, treat it as
|
||||
root loss — increment sequence number, become own root, and re-announce.
|
||||
**Remaining gap**: If an intermediate node maintains a link to the root but
|
||||
that link silently stops forwarding (no keepalive failure), nodes below it
|
||||
would retain stale root state. This is an unusual failure mode — most link
|
||||
failures are detected by keepalive timeouts. An explicit root timeout would
|
||||
provide defense-in-depth for this edge case.
|
||||
|
||||
### Known Limitation: No TTL on Tree Entries
|
||||
|
||||
|
||||
Reference in New Issue
Block a user