Session 53: Route cache warming and session protocol refinements

Session protocol (fips-session-protocol.md): - Clarified DNS entry point applies to IP-based apps; native FIPS uses npub - Updated transport failover examples (UDP → WiFi, WiFi → LTE) - Added roaming description to session independence section - Expanded §5 with coords-on-demand mechanism for route cache recovery - Added Noise IK vs XK privacy tradeoff note to §6 Routing (fips-routing.md): - DataPacket now has optional coordinates (COORDS_PRESENT flag) - Updated handle_data_packet to cache coords from packets - Sender state machine: WARM/COLD based on CoordsRequired errors - Updated packet type summary table Architecture (fips-architecture.md): - Fixed cross-references to session protocol sections
2026-07-30 19:46:15 +00:00 · 2026-02-02 03:39:13 +00:00
parent a8115f7622
commit 7dc0b7fc52
3 changed files with 126 additions and 69 deletions
@@ -931,7 +931,8 @@ this node wants to reach. This is the primary cache for endpoint nodes.
 > **Terminology note**: These parameters configure *routing sessions*—hop-by-hop
 > cached state at intermediate routers. For *crypto session* (end-to-end
 > encryption) parameters, see the Crypto Session section below. See
-> [fips-session-protocol.md](fips-session-protocol.md) §5 for the distinction.
+> [fips-session-protocol.md](fips-session-protocol.md) §3 for crypto sessions
+> and §5 for route cache warming.

 | Parameter | Type | Default | Description |
 |-----------|------|---------|-------------|
@@ -298,8 +298,8 @@ No global routing tables. Each node makes purely local decisions.
 > **Terminology note**: This section describes *routing sessions*—hop-by-hop
 > cached state at intermediate routers. FIPS also has *crypto sessions*—end-to-end
 > authenticated encryption between source and destination. See
-> [fips-session-protocol.md](fips-session-protocol.md) §5 for the distinction and §6
-> for crypto session details.
+> [fips-session-protocol.md](fips-session-protocol.md) §3 for crypto session details
+> and §5 for route cache warming.

 ### Routing Session Purpose

@@ -349,13 +349,18 @@ struct SessionAck {
    handshake_payload: Option<Vec<u8>>,  // Noise IK message 2
 }

-/// Minimal data packet
+/// Data packet with optional coordinates
 struct DataPacket {
-    flags: u8,
+    flags: u8,                 // Bit 0: COORDS_PRESENT
    hop_limit: u8,
    payload_length: u16,
    src_addr: Ipv6Addr,        // 16 bytes
    dest_addr: Ipv6Addr,       // 16 bytes
+
+    // Optional: present only if COORDS_PRESENT flag is set
+    src_coords: Option<Vec<NodeId>>,
+    dest_coords: Option<Vec<NodeId>>,
+
    payload: Vec<u8>,
 }

@@ -375,9 +380,10 @@ struct CoordsRequired {
 | payload_length | 2 bytes |
 | src_addr | 16 bytes |
 | dest_addr | 16 bytes |
-| **Total header** | **36 bytes** |
+| **Minimal header** | **36 bytes** |

-Comparable to IPv6 (40 bytes). No coordinates in data packets.
+Comparable to IPv6 (40 bytes). When COORDS_PRESENT flag is set, coordinates
+add variable overhead based on tree depth (typically 200-400 bytes).

 ### Routing Session Setup Flow

@@ -419,6 +425,23 @@ impl Router {
    }

    fn handle_data_packet(&mut self, packet: DataPacket, from: PeerId) {
+        // If packet carries coordinates, cache them
+        if packet.flags & COORDS_PRESENT != 0 {
+            if let (Some(src_coords), Some(dest_coords)) =
+                (&packet.src_coords, &packet.dest_coords)
+            {
+                self.coord_cache.insert(packet.dest_addr, CacheEntry {
+                    coords: dest_coords.clone(),
+                    expires: now() + CACHE_TTL,
+                });
+                self.coord_cache.insert(packet.src_addr, CacheEntry {
+                    coords: src_coords.clone(),
+                    expires: now() + CACHE_TTL,
+                });
+            }
+        }
+
+        // Route using cache (now populated if coords were present)
        match self.coord_cache.get(&packet.dest_addr) {
            Some(entry) => {
                entry.last_used = now();
@@ -467,13 +490,16 @@ by:
 When a router's cache entry is evicted mid-session:

 ```text
-1. Data packet arrives, cache miss
+1. Data packet arrives (minimal header), cache miss
 2. Router sends CoordsRequired to packet source
-3. Source receives error, re-sends SessionSetup (or packet with coords)
-4. Path warms again, data flow resumes
+3. Source marks route as cold
+4. Source resends with COORDS_PRESENT flag set
+5. Router caches coordinates from packet, forwards
+6. After N successful packets, source clears flag
 ```

-From application perspective: brief latency spike, transparent recovery.
+The crypto session remains active throughout—only routing state is refreshed.
+From application perspective: one packet delayed, transparent recovery.

 ### Sender Behavior

@@ -481,21 +507,29 @@ From application perspective: brief latency spike, transparent recovery.
 impl Sender {
    fn send(&mut self, dest: Ipv6Addr, data: &[u8]) {
        if !self.session_established(dest) {
-            // Need to establish session first
+            // Need to establish crypto session first
            let dest_coords = self.discover_or_cached(dest)?;
            self.send_session_setup(dest, &dest_coords);
            self.await_session_ack(dest)?;
        }

-        self.send_data_packet(dest, data);
+        // Check route state
+        let include_coords = self.route_state(dest) == RouteCold;
+        self.send_data_packet(dest, data, include_coords);
    }

    fn handle_coords_required(&mut self, err: CoordsRequired) {
-        // Path went cold, re-establish
-        self.mark_session_cold(err.dest_addr);
-        self.send_session_setup(err.dest_addr, &self.cached_coords(err.dest_addr));
+        // Route cache expired at intermediate router
+        // Crypto session still valid - just need to re-warm route
+        self.mark_route_cold(err.dest_addr);
+        // Next send() will include coordinates
    }
 }
+
+enum RouteState {
+    RouteWarm,  // Send minimal headers
+    RouteCold,  // Include coordinates until warm
+}
 ```

 ---
@@ -509,12 +543,13 @@ impl Sender {
 | LookupResponse | Return coordinates | ~400 bytes | Reply to discovery |
 | SessionSetup | Warm router caches + crypto init | ~400-700 bytes | Before data transfer |
 | SessionAck | Confirm session + crypto response | ~300-500 bytes | Session confirmation |
-| DataPacket | Application data | 36 bytes + payload | Bulk of traffic |
-| CoordsRequired | Request retransmit | ~50 bytes | Cache miss recovery |
+| DataPacket | Application data | 36 bytes + payload (minimal) | Bulk of traffic |
+| DataPacket | With coordinates | ~300-500 bytes + payload | After CoordsRequired |
+| CoordsRequired | Request coords in next packet | ~50 bytes | Cache miss recovery |

 > **Note**: SessionSetup/SessionAck sizes vary based on coordinate depth and
 > whether they carry crypto handshake payloads (combined establishment per
-> [fips-session-protocol.md](fips-session-protocol.md) §5.5).
+> [fips-session-protocol.md](fips-session-protocol.md) §3.4 and §5.1).

 ---

@@ -9,6 +9,10 @@ including peer discovery, authentication, tree announcements, and data routing.

 ## 1. Application-Initiated Traffic Flow

+> **Note**: This section applies to traditional IP-based applications using the
+> TUN interface. Applications using the native FIPS datagram service address
+> destinations directly by npub, and routing proceeds from there without DNS.
+
 Traffic flow begins at the application layer with a DNS query, which triggers
 a cascade of events through the FIPS stack.

@@ -17,10 +21,10 @@ a cascade of events through the FIPS stack.
 An application wants to send IPv6 traffic to another FIPS node, identified by
 an npub. The flow:

-1. **DNS Query**: Application queries a local FIPS DNS server for the npub
-   (format TBD - perhaps `npub1xxx...xxx.fips` or similar)
+1. **DNS Query**: Application queries the local FIPS DNS service for the npub
+   mapping to an IPv6 address

-2. **FIPS DNS Server** performs two functions:
+2. **FIPS DNS service** performs two functions:
   - **Address derivation**: Converts the npub to an identity and derives the
     corresponding `fd::/8` IPv6 address
   - **Cache priming**: Stores the identity mapping (IPv6 address ↔ npub ↔ node_id)
@@ -34,6 +38,8 @@ an npub. The flow:
 5. **TUN Processing**: When the packet arrives at the TUN, FIPS already has the
   cached mapping from the DNS lookup, enabling immediate routing decisions

+6. **Note**: This identity cache is only necessary when using the FIPS IPv6 shim
+
 ### 1.2 Design Rationale

 Using DNS as the trigger point ensures the routing cache is populated *before*
@@ -194,12 +200,13 @@ management) and transmitted after the session is established.
 FIPS sessions exist above the routing layer. A session between two npubs
 survives:

- Transport failover (UDP → Tor → back to UDP)
+- Transport failover (UDP → WiFi → back to UDP)
 - Route changes (different intermediate hops)
- Transport address changes on either end
+- Transport address changes on either end (WiFi → LTE → WiFi)

 The session is bound to **npub identities**, not transport addresses or routing
-paths.
+paths. This allows FIPS endpoints to roam over their transports as needed while
+maintaining an established session.

 ### 3.4 Session Establishment Flow

@@ -207,7 +214,7 @@ FIPS uses Noise IK for session establishment. The initiator knows the
 destination's npub; the responder learns the initiator's identity from the
 handshake. This is the same asymmetry as link-layer connections.

-The handshake is carried inside SessionSetup/SessionAck messages (see §5.5),
+The handshake is carried inside SessionSetup/SessionAck messages (see §5.1),
 which also establish routing session state at intermediate nodes.

 ### 3.5 Simultaneous Session Initiation (Crossing Hellos)
@@ -255,61 +262,67 @@ Route cache entries:

 ---

-## 5. Session Terminology
+## 5. Route Cache Warming

-FIPS uses two distinct session concepts at different layers:
+### 5.1 Initial Warming via Handshake

-| Term                | Layer       | Purpose                      | Endpoints              |
-|---------------------|-------------|------------------------------|------------------------|
-| **Crypto Session**  | End-to-end  | Authentication + encryption  | Source ↔ Destination   |
-| **Routing Session** | Hop-by-hop  | Cache coordinates at routers | Along the path         |
+The crypto session handshake (SessionSetup/SessionAck) warms route caches at
+intermediate routers as it transits. Each message carries the sender's
+coordinates; routers extract and cache `(src_addr, dest_addr) → next_hop` for
+both directions. After the handshake completes, data packets use minimal
+36-byte headers and routers forward based on cached routes.

-### 5.1 Crypto Session
+### 5.2 Cache Miss Recovery

- Established between two npub identities
- Provides confidentiality and authenticity via Noise IK
- Survives route changes and transport failover
- Keyed by: `(local_npub, remote_npub)`
+When an intermediate router's cache entry expires or is evicted, it cannot
+forward data packets (which carry only addresses, not coordinates). The router
+returns a CoordsRequired error to the sender.

-### 5.2 Routing Session
+The crypto session remains valid—only the routing state is lost. Recovery uses
+coords-on-demand: data packets include an optional coordinates field. When the
+sender receives CoordsRequired:

- Warms coordinate caches at intermediate routers
- Enables minimal 36-byte data packet headers
- Must be re-established when router caches expire
- Keyed by: `(src_addr, dest_addr)` at each router
+1. Sender marks the route as "cold"
+2. Subsequent data packets include coordinates (flag bit set)
+3. Routers along the path cache coordinates as packets transit
+4. Once route is warm again, sender clears the flag and resumes minimal headers

-### 5.3 Combined Establishment
+This avoids a full SessionSetup round-trip for what is purely a routing cache
+refresh.

-Both sessions are established together: the routing session setup carries the
-crypto handshake, minimizing round-trips.
+### 5.3 DataPacket Coordinate Flag
+
+The DataPacket `flags` field includes a `COORDS_PRESENT` bit:
+
+| Bit | Meaning                                              |
+|-----|------------------------------------------------------|
+| 0   | COORDS_PRESENT - coordinates follow the fixed header |
+
+When set, the packet includes `src_coords` and `dest_coords` after the standard
+header fields. Routers process these coordinates the same way as SessionSetup:
+cache both directions and forward using greedy routing.
+
+### 5.4 Sender State Machine

 ```text
-1. Route discovery (if needed)
-   └─► LookupRequest/Response → obtain destination coordinates
-
-2. SessionSetup + Crypto Init
-   └─► Source sends SessionSetup containing:
-       - src/dest coordinates (for router caching)
-       - Noise IK handshake initiation (for destination)
-   └─► Routers cache coordinates as packet transits
-   └─► Destination receives crypto init, begins handshake
-
-3. SessionAck + Crypto Response
-   └─► Destination sends SessionAck containing:
-       - Its coordinates (for reverse path caching)
-       - Noise IK handshake response
-   └─► Routers cache reverse path
-   └─► Source completes crypto handshake
-
-4. Data flow
-   └─► Encrypted payloads with minimal 36-byte headers
-   └─► Both crypto session and routing session now active
+        ┌──────────────┐
+        │    WARM      │ ◄── Normal: send minimal headers
+        └──────┬───────┘
+               │ CoordsRequired received
+               ▼
+        ┌──────────────┐
+        │    COLD      │ ◄── Send packets with coords
+        └──────┬───────┘
+               │ N packets sent successfully
+               ▼
+        ┌──────────────┐
+        │    WARM      │
+        └──────────────┘
 ```

-SessionSetup and SessionAck messages carry both routing information (coordinates
-for router caching) and crypto payload (handshake messages, opaque to routers).
-Routers process the routing portion and forward; only endpoints process the
-crypto portion.
+The sender transitions back to WARM after sending a configurable number of
+packets with coordinates (e.g., 3) without receiving CoordsRequired. This
+provides confidence that caches along the path are populated.

 ---

@@ -325,6 +338,14 @@ FIPS uses two independent Noise Protocol handshakes at different layers:
 Both use `Noise_IK_secp256k1_ChaChaPoly_SHA256` with the same cryptographic
 primitives, but with separate keys and sessions.

+> **Privacy note**: Noise IK does not provide initiator anonymity if the
+> responder's static key is compromised—an attacker who obtains the responder's
+> nsec can decrypt the initiator's identity from captured handshake messages.
+> Noise XK would protect initiator identity in this scenario, but requires an
+> additional round-trip (3 handshake messages vs 2), increasing session setup
+> from 3 packets to 4. Further deployment experience is needed to evaluate
+> whether the privacy benefit justifies the latency cost.
+
 ### 6.1 Why Two Layers?

 **Link encryption** protects against passive observers on each hop but allows