mirror of
https://github.com/jmcorgan/fips.git
synced 2026-07-30 19:46:15 +00:00
Hybrid coordinate warmup: CoordsWarmup message and proactive fallback
Implement hybrid coordinate cache warming strategy: piggyback coords via CP flag when they fit within transport MTU, send standalone CoordsWarmup (0x14) message when they don't. On CoordsRequired or PathBroken receipt, send CoordsWarmup immediately with source-side rate limiting (default 2s per destination, configurable). - Add CoordsWarmup = 0x14 session message type (empty body, CP flag) - Add send_coords_warmup() following send_session_msg() pattern - Restructure send_session_data() to send standalone warmup before data packet when piggybacked coords exceed MTU - Add immediate CoordsWarmup response in handle_coords_required() and handle_path_broken() with per-destination rate limiting - Add coords_response_interval_ms config (node.session) - Add RoutingErrorRateLimiter::with_interval() constructor - Zero transit-path changes: existing try_warm_coord_cache() handles CoordsWarmup identically to CP-flagged data packets - Update design docs (session layer, wire formats, mesh operation, configuration)
This commit is contained in:
Johnathan Corgan
@@ -148,7 +148,8 @@ Controls end-to-end session behavior and packet queuing.
|
||||
| `node.session.pending_packets_per_dest` | usize | `16` | Queue depth per destination during session establishment |
|
||||
| `node.session.pending_max_destinations` | usize | `256` | Max destinations with pending packets |
|
||||
| `node.session.idle_timeout_secs` | u64 | `90` | Idle session timeout; established sessions with no application data for this duration are removed. MMP reports (SenderReport, ReceiverReport, PathMtuNotification) do not count as activity |
|
||||
| `node.session.coords_warmup_packets` | u8 | `5` | Number of initial data packets per session that include the CP flag for transit cache warmup; also the reset count on CoordsRequired receipt |
|
||||
| `node.session.coords_warmup_packets` | u8 | `5` | Number of initial data packets per session that include the CP flag for transit cache warmup; also the reset count on CoordsRequired/PathBroken receipt |
|
||||
| `node.session.coords_response_interval_ms` | u64 | `2000` | Minimum interval (ms) between standalone CoordsWarmup responses to CoordsRequired/PathBroken signals per destination |
|
||||
|
||||
The anti-replay window size (2048 packets) is a compile-time constant and not
|
||||
configurable.
|
||||
@@ -327,6 +328,7 @@ node:
|
||||
pending_max_destinations: 256
|
||||
idle_timeout_secs: 90
|
||||
coords_warmup_packets: 5
|
||||
coords_response_interval_ms: 2000
|
||||
mmp:
|
||||
mode: full # full | lightweight | minimal
|
||||
log_interval_secs: 30
|
||||
|
||||
@@ -338,17 +338,19 @@ After the handshake completes, the entire forward and reverse paths have
|
||||
cached coordinates for both endpoints. Subsequent data packets use minimal
|
||||
headers (no coordinates) and route efficiently through the warmed caches.
|
||||
|
||||
## CP (Coords Present) Warmup
|
||||
## Hybrid Coordinate Warmup (CP + CoordsWarmup)
|
||||
|
||||
The CP flag in the FSP common prefix provides a secondary cache-warming
|
||||
mechanism that complements SessionSetup. See
|
||||
[fips-session-layer.md](fips-session-layer.md) for the warmup-then-reactive
|
||||
strategy.
|
||||
The CP flag in the FSP common prefix and the standalone CoordsWarmup message
|
||||
(0x14) together provide a hybrid cache-warming mechanism that complements
|
||||
SessionSetup. See [fips-session-layer.md](fips-session-layer.md) for the
|
||||
full warmup strategy.
|
||||
|
||||
Transit nodes parse the CP flag from the FSP header and extract source and
|
||||
destination coordinates from the cleartext section between the header and
|
||||
ciphertext — no decryption needed. This is the same caching operation
|
||||
performed for SessionSetup coordinates.
|
||||
performed for SessionSetup coordinates. CoordsWarmup messages use the same
|
||||
CP-flag format and are handled identically by transit nodes via the existing
|
||||
`try_warm_coord_cache()` path.
|
||||
|
||||
## Error Recovery
|
||||
|
||||
@@ -361,15 +363,22 @@ corrective action.
|
||||
coordinates for the destination. It cannot make a forwarding decision.
|
||||
|
||||
**Transit node action**:
|
||||
|
||||
1. Create a new SessionDatagram addressed back to the original source,
|
||||
carrying a CoordsRequired payload identifying the unreachable destination
|
||||
2. Route the error via `find_next_hop(src_addr)`
|
||||
3. If the source is also unreachable, drop silently (no cascading errors)
|
||||
|
||||
**Source recovery**:
|
||||
1. Initiate discovery (LookupRequest flood) for the destination
|
||||
2. Reset CP warmup counter — subsequent data packets include coordinates
|
||||
3. When discovery completes, warmup counter resets again (covers timing gap)
|
||||
|
||||
1. Immediately send a standalone CoordsWarmup (0x14) message to re-warm
|
||||
transit caches along the path (rate-limited: at most one per destination
|
||||
per configurable interval, default 2s)
|
||||
2. Reset CP warmup counter — subsequent data packets piggyback coordinates
|
||||
when possible, or trigger additional CoordsWarmup messages when
|
||||
piggybacking would exceed the transport MTU
|
||||
3. Initiate discovery (LookupRequest flood) for the destination
|
||||
4. When discovery completes, warmup counter resets again (covers timing gap)
|
||||
|
||||
The crypto session remains active throughout — only routing state is
|
||||
refreshed.
|
||||
@@ -384,9 +393,12 @@ tree distance metric). The cached coordinates may be stale.
|
||||
source.
|
||||
|
||||
**Source recovery**:
|
||||
1. Remove stale coordinates from cache
|
||||
2. Initiate discovery for the destination
|
||||
3. Reset CP warmup counter
|
||||
|
||||
1. Immediately send a standalone CoordsWarmup (0x14) message (rate-limited,
|
||||
same per-destination interval as CoordsRequired response)
|
||||
2. Remove stale coordinates from cache
|
||||
3. Initiate discovery for the destination
|
||||
4. Reset CP warmup counter
|
||||
|
||||
### Error Signal Rate Limiting
|
||||
|
||||
@@ -394,6 +406,12 @@ Both error types are rate-limited at transit nodes: maximum one error per
|
||||
destination per 100ms. This prevents storms during topology changes when many
|
||||
packets to the same destination hit the same routing failure simultaneously.
|
||||
|
||||
At the source side, CoordsWarmup responses to CoordsRequired/PathBroken are
|
||||
independently rate-limited: at most one standalone CoordsWarmup per destination
|
||||
per `coords_response_interval_ms` (default 2000ms, configurable). This
|
||||
prevents amplification where a burst of error signals would generate a
|
||||
corresponding burst of warmup messages.
|
||||
|
||||
Error signals (CoordsRequired, PathBroken) are handled asynchronously outside
|
||||
the packet receive path, allowing the RX loop to continue processing without
|
||||
blocking on discovery or session repair.
|
||||
@@ -554,7 +572,7 @@ recovery).
|
||||
| Flush coord cache on parent change | **Implemented** |
|
||||
| LookupRequest/LookupResponse discovery | **Implemented** |
|
||||
| SessionSetup self-bootstrapping | **Implemented** |
|
||||
| CP warmup-then-reactive | **Implemented** |
|
||||
| Hybrid coordinate warmup (CP + CoordsWarmup) | **Implemented** |
|
||||
| CoordsRequired recovery | **Implemented** |
|
||||
| PathBroken recovery | **Implemented** |
|
||||
| Error signal rate limiting | **Implemented** |
|
||||
|
||||
@@ -83,10 +83,12 @@ may traverse multiple hops, each with independent link encryption.
|
||||
FLP signals routing failures asynchronously:
|
||||
|
||||
- **CoordsRequired**: A transit node lacks the destination's tree coordinates.
|
||||
FSP responds by re-initiating discovery and resetting the coordinate warmup
|
||||
strategy.
|
||||
- **PathBroken**: Greedy routing reached a dead end. FSP responds by
|
||||
re-discovering the destination's current coordinates and resetting warmup.
|
||||
FSP responds by sending a standalone CoordsWarmup (0x14) message
|
||||
(rate-limited), re-initiating discovery, and resetting the coordinate warmup
|
||||
counter.
|
||||
- **PathBroken**: Greedy routing reached a dead end. FSP responds by sending
|
||||
a standalone CoordsWarmup (rate-limited), re-discovering the destination's
|
||||
current coordinates, and resetting the warmup counter.
|
||||
|
||||
Both signals are generated by transit nodes (not the destination) and travel
|
||||
back to the source inside a new SessionDatagram. They are plaintext (not
|
||||
@@ -190,6 +192,7 @@ independent keys and sessions. The full Noise descriptor is
|
||||
`Noise_IK_secp256k1_ChaChaPoly_SHA256`.
|
||||
|
||||
The IK pattern:
|
||||
|
||||
- **msg1** (`→ e, es, s, ss`): Initiator sends ephemeral key, encrypts static
|
||||
key to responder. Four DH operations establish session keys.
|
||||
- **msg2** (`← e, ee, se`): Responder sends ephemeral key. Both parties now
|
||||
@@ -262,7 +265,7 @@ regardless of what packets were lost or reordered.
|
||||
The same `ReplayWindow` and `decrypt_with_replay_check()` implementation is
|
||||
used at both the link and session layers.
|
||||
|
||||
## CP (Coords Present) Warmup Strategy
|
||||
## Hybrid Coordinate Warmup Strategy
|
||||
|
||||
Session establishment (SessionSetup/SessionAck) warms transit node coordinate
|
||||
caches along the path. But coordinate caches have a finite TTL (default 300s),
|
||||
@@ -270,14 +273,43 @@ and entries may be evicted under memory pressure. When a transit node's cache
|
||||
entry expires, it cannot forward data packets (which carry only addresses, not
|
||||
coordinates) and sends a CoordsRequired error.
|
||||
|
||||
FSP uses a warmup-then-reactive strategy to keep transit caches populated:
|
||||
FSP uses a hybrid warmup strategy combining proactive piggybacking with
|
||||
reactive standalone messages to keep transit caches populated:
|
||||
|
||||
### Warmup Phase
|
||||
### Proactive Warmup Phase
|
||||
|
||||
After session establishment, the first N data packets (configurable, default 5)
|
||||
include both source and destination coordinates via the CP flag in the FSP
|
||||
common prefix. The coordinates appear in cleartext between the 12-byte header
|
||||
and the ciphertext, allowing transit nodes to cache them without decryption.
|
||||
per session attempt to piggyback source and destination coordinates via the CP
|
||||
flag in the FSP common prefix. The coordinates appear in cleartext between the
|
||||
12-byte header and the ciphertext, allowing transit nodes to cache them without
|
||||
decryption.
|
||||
|
||||
If piggybacking coordinates would cause the total packet to exceed the
|
||||
transport MTU, the source sends a standalone **CoordsWarmup** message (0x14)
|
||||
first, followed by the data packet without the CP flag. This ensures transit
|
||||
caches are warmed even when data packets are near the MTU limit.
|
||||
|
||||
### CoordsWarmup Message (0x14)
|
||||
|
||||
CoordsWarmup is an encrypted FSP message (phase 0x0) with the CP flag set and
|
||||
inner msg_type 0x14. It carries cleartext source and destination coordinates
|
||||
between the FSP header and the AEAD ciphertext, using the same format as
|
||||
CP-flagged data packets. The inner body is empty — the message exists solely
|
||||
to deliver coordinates to transit nodes.
|
||||
|
||||
Transit nodes extract coordinates via the existing `try_warm_coord_cache()`
|
||||
code path with zero changes to the transit forwarding logic. CoordsWarmup is
|
||||
indistinguishable from any other CP-flagged message at the transit layer.
|
||||
|
||||
Wire format:
|
||||
|
||||
```text
|
||||
FSP header (12 bytes, AAD): ver=0, phase=0, flags=CP, counter, payload_len
|
||||
Cleartext coords: src_coords + dst_coords (same encoding as CP flag)
|
||||
AEAD ciphertext: inner_header(6) + Poly1305 tag(16) = 22 bytes
|
||||
|
||||
Total FSP payload: 12 + coords + 22
|
||||
```
|
||||
|
||||
### Steady State
|
||||
|
||||
@@ -285,31 +317,32 @@ After the warmup count is reached, FSP clears the CP flag and sends minimal
|
||||
data packets (12-byte header + ciphertext). Transit nodes serve from their
|
||||
coordinate caches.
|
||||
|
||||
### Reactive Recovery
|
||||
### Reactive Re-Warm
|
||||
|
||||
When FSP receives a CoordsRequired signal:
|
||||
When FSP receives a CoordsRequired or PathBroken signal:
|
||||
|
||||
1. The warmup counter resets — subsequent data packets include coordinates again
|
||||
2. A new LookupRequest may be initiated to rediscover the destination's
|
||||
current coordinates
|
||||
3. When the LookupResponse arrives for an established session, the warmup
|
||||
1. A standalone CoordsWarmup message is sent immediately to re-warm transit
|
||||
caches, rate-limited at one per destination per configurable interval
|
||||
(default 2000ms, `node.session.coords_response_interval_ms`)
|
||||
2. The warmup counter resets — subsequent data packets piggyback coordinates
|
||||
again when possible (or send additional CoordsWarmup messages when the
|
||||
data packet would exceed the MTU)
|
||||
3. A new LookupRequest may be initiated to rediscover the destination's
|
||||
current coordinates (always for PathBroken; optionally for CoordsRequired)
|
||||
4. When the LookupResponse arrives for an established session, the warmup
|
||||
counter resets again (handling the timing gap where warmup packets might
|
||||
fire before transit caches are repopulated by discovery)
|
||||
|
||||
When FSP receives a PathBroken signal:
|
||||
|
||||
1. A LookupRequest is initiated to discover the destination's current
|
||||
coordinates (which may have changed due to topology change)
|
||||
2. The warmup counter resets
|
||||
|
||||
Both signals are rate-limited at transit nodes (100ms per destination) to
|
||||
prevent storms during topology changes.
|
||||
The source-side rate limiting prevents amplification: at most one standalone
|
||||
CoordsWarmup response per destination per `coords_response_interval_ms`
|
||||
(default 2s). This is independent of the transit-node rate limiting on error
|
||||
signal generation (100ms per destination).
|
||||
|
||||
### Warmup State Machine
|
||||
|
||||
```text
|
||||
┌──────────────┐
|
||||
│ WARMUP │ ◄── Send first N packets with coords
|
||||
│ WARMUP │ ◄── Send first N packets with coords (CP or CoordsWarmup)
|
||||
└──────┬───────┘
|
||||
│ N packets sent without CoordsRequired
|
||||
▼
|
||||
@@ -318,8 +351,13 @@ prevent storms during topology changes.
|
||||
└──────┬───────┘
|
||||
│ CoordsRequired or PathBroken received
|
||||
▼
|
||||
┌──────────────────────────┐
|
||||
│ SEND CoordsWarmup (0x14) │ ◄── Immediate standalone warmup (rate-limited)
|
||||
└──────────┬───────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────┐
|
||||
│ WARMUP │ ◄── Counter reset, send coords again
|
||||
│ WARMUP │ ◄── Counter reset, piggyback coords again
|
||||
└──────────────┘
|
||||
```
|
||||
|
||||
@@ -336,6 +374,7 @@ configurable size (default 10K entries). There is no TTL — entries are evicted
|
||||
only when the cache is full and space is needed for a new entry.
|
||||
|
||||
Cache population mechanisms:
|
||||
|
||||
- **DNS lookup**: The primary path. Resolving `npub1xxx...xxx.fips` derives
|
||||
the IPv6 address and populates the identity cache.
|
||||
- **Inbound traffic**: Authenticated sessions from other nodes populate the
|
||||
@@ -454,7 +493,7 @@ MMP session metrics session=npub1tdwa...84le rtt=4.3ms loss=0.6% jitter=0.2ms go
|
||||
| Session establishment (Noise IK) | **Implemented** |
|
||||
| End-to-end encryption (ChaCha20-Poly1305) | **Implemented** |
|
||||
| Explicit counter replay protection | **Implemented** |
|
||||
| CP warmup-then-reactive | **Implemented** |
|
||||
| Hybrid coordinate warmup (CP + CoordsWarmup) | **Implemented** |
|
||||
| FSP wire format (prefix, AAD, inner header) | **Implemented** |
|
||||
| Session-layer MMP | **Implemented** |
|
||||
| Identity cache (LRU-only) | **Implemented** |
|
||||
|
||||
@@ -526,10 +526,11 @@ body.
|
||||
| 0x11 | SenderReport | MMP sender-side metrics report |
|
||||
| 0x12 | ReceiverReport | MMP receiver-side metrics report |
|
||||
| 0x13 | PathMtuNotification | End-to-end path MTU echo |
|
||||
| 0x14 | CoordsWarmup | Standalone coordinate cache warming |
|
||||
| 0x20 | CoordsRequired | Error: transit node lacks destination coordinates |
|
||||
| 0x21 | PathBroken | Error: greedy routing reached dead end |
|
||||
|
||||
Message types 0x10-0x13 are carried inside the AEAD ciphertext (dispatched
|
||||
Message types 0x10-0x14 are carried inside the AEAD ciphertext (dispatched
|
||||
by the `msg_type` field in the encrypted inner header). Types 0x20-0x21 are
|
||||
plaintext error signals (U flag set, no encryption).
|
||||
|
||||
@@ -586,6 +587,34 @@ Sent by the destination to report the observed forward-path MTU.
|
||||
|
||||
**Total body: 2 bytes** (plus FSP common prefix + encrypted header + AEAD tag).
|
||||
|
||||
### CoordsWarmup (0x14)
|
||||
|
||||
Standalone coordinate cache warming message. Sent when piggybacking coordinates
|
||||
via the CP flag on a data packet would exceed the transport MTU, or as an
|
||||
immediate response to CoordsRequired/PathBroken signals (rate-limited).
|
||||
|
||||
CoordsWarmup is an encrypted FSP message with the CP flag set and an empty
|
||||
body. Transit nodes extract coordinates via the existing CP-flag parsing
|
||||
path — no transit-side changes required.
|
||||
|
||||
**Wire format**:
|
||||
|
||||
```text
|
||||
FSP header (12 bytes, AAD): ver=0, phase=0, flags=CP, counter, payload_len
|
||||
Cleartext coords: src_coords + dst_coords (same encoding as CP flag)
|
||||
AEAD ciphertext: inner_header(6) + Poly1305 tag(16) = 22 bytes
|
||||
|
||||
Total FSP payload: 12 + coords + 22
|
||||
```
|
||||
|
||||
The cleartext coords section uses the same variable-length encoding as any
|
||||
CP-flagged message: `src_coords_count(2) + src_coords(16×n) +
|
||||
dest_coords_count(2) + dest_coords(16×m)`.
|
||||
|
||||
**Typical size** (depth-3 tree): 12 + (2+64+2+64) + 22 = **166 bytes** FSP
|
||||
payload. With SessionDatagram + link overhead: 166 + 36 + 37 = **239 bytes**
|
||||
on the wire.
|
||||
|
||||
### CoordsRequired (0x20)
|
||||
|
||||
Plaintext error signal — transit node lacks coordinates for destination.
|
||||
@@ -707,6 +736,7 @@ endpoint session keys).
|
||||
| SenderReport | 12 + 6 + 46 + 16 bytes | MMP metrics |
|
||||
| ReceiverReport | 12 + 6 + 66 + 16 bytes | MMP metrics |
|
||||
| PathMtuNotification | 12 + 6 + 2 + 16 bytes | MTU signal |
|
||||
| CoordsWarmup | 12 + coords + 6 + 16 bytes | Standalone warmup (empty body) |
|
||||
| CoordsRequired | 38 bytes | Fixed (prefix + msg_type + body) |
|
||||
| PathBroken | 35 + 16n bytes | Includes stale coords |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user