Add FIPS routing design document

Create fips-routing.md covering the complete routing architecture: - Bloom filter design: 4KB filters, K=2 scope, event-driven updates - Discovery protocol: LookupRequest/Response with signed proofs - Greedy tree routing using coordinates from discovery - Session establishment model for minimal data packet overhead - Router coordinate caching with LRU eviction Key design decisions: - Leaf-only mode for constrained devices (single peer handles routing) - Separation of discovery (find destination) from routing (deliver packets) - Session setup pays coordinate cost once; data packets carry only addresses - 36-byte data packet header comparable to IPv6 Update design docs README to include new document.
2026-07-30 19:46:15 +00:00 · 2026-01-26 00:12:47 +00:00
parent d8cba510a7
commit dc0acf32ea
2 changed files with 609 additions and 0 deletions
@@ -6,4 +6,5 @@ Protocol design specifications and analysis.
 |--------------------------------------------------------|-------------------------------------------------|
 | [fips-design.md](fips-design.md)                       | Full protocol design specification              |
 | [fips-links.md](fips-links.md)                         | Link protocol requirements and characteristics  |
+| [fips-routing.md](fips-routing.md)                     | Routing, Bloom filters, discovery, sessions     |
 | [spanning-tree-dynamics.md](spanning-tree-dynamics.md) | Spanning tree protocol dynamics study           |
@@ -0,0 +1,608 @@
+# FIPS Routing Design
+
+**Status**: Work in Progress
+
+This document describes the routing architecture for FIPS, including Bloom
+filter reachability, discovery protocol, greedy tree routing, and session
+establishment.
+
+## Overview
+
+FIPS routing combines three mechanisms:
+
+1. **Bloom filters**: Fast reachability lookup for nearby destinations (within
+   K-hop scope)
+2. **Discovery protocol**: Query-based lookup for distant destinations
+3. **Greedy tree routing**: Coordinate-based forwarding using spanning tree
+   position
+
+The design separates discovery (finding where a destination is) from routing
+(getting packets there). Bloom filters and discovery handle the former; tree
+coordinates handle the latter.
+
+## Design Goals
+
+- Minimize per-packet overhead for data transfer
+- Bounded state at each node (independent of network size)
+- Efficient routing without global knowledge
+- Graceful degradation for constrained devices
+- Fast convergence on topology changes
+
+## Network Scale Assumptions
+
+| Scale | Nodes | Bloom Filter Role |
+|-------|-------|-------------------|
+| Small private network | 100-1,000 | Covers entire network |
+| Modest public network | ~1,000,000 | Covers K-hop neighborhood |
+| Internet-scale | Billions | Out of scope (requires different architecture) |
+
+The primary design target is networks up to ~1M nodes.
+
+## Node Participation Modes
+
+### Full Participant
+
+- Maintains Bloom filters for peer reachability
+- Participates in spanning tree (can be selected as parent)
+- Routes packets for other nodes
+- Minimum viable device: ESP32-class (~500KB RAM)
+
+### Leaf-Only
+
+- Single peer handles all routing on its behalf
+- No Bloom filter storage or processing
+- Does not participate in spanning tree as potential parent
+- Suitable for highly constrained devices (sensors, battery-powered nodes)
+
+Leaf-only nodes appear as a single entry in their peer's Bloom filter. All
+traffic tunnels through that peer.
+
+---
+
+## Part 1: Bloom Filter Design
+
+### Parameters
+
+| Parameter | Value | Rationale |
+|-----------|-------|-----------|
+| Filter size | 4 KB (32,768 bits) | Balances accuracy vs. memory |
+| Hash functions | 7 | Near-optimal for expected fill ratio |
+| Scope (K) | 2 | Effective ~4-hop reach with TTL propagation |
+
+### False Positive Rates
+
+| Nodes in Filter | FPR |
+|-----------------|-----|
+| 1,000 | ~0.05% |
+| 2,000 | ~0.5% |
+| 5,000 | ~1.3% |
+| 10,000 | ~8% |
+
+With K=2 and average degree d=8, expected nodes in scope ≈ d^(2K) ≈ 4,096.
+
+### Filter Contents
+
+Each node's filter contains Node IDs (and optionally gateway /64 prefixes)
+that are reachable through that node. A Node ID is the SHA-256 hash of the
+node's npub, truncated or used directly as the filter key.
+
+### Per-Peer Filters
+
+Each node maintains a Bloom filter for each peer direction:
+
+```rust
+peer_filters: HashMap<PeerId, BloomFilter>
+```
+
+The filter for peer P answers: "Which destinations are reachable through P?"
+
+### Update Mechanism: Event-Driven
+
+Filters are updated on events rather than periodic refresh:
+
+**Triggering events:**
+
+1. Peer connects — exchange current filters
+2. Peer disconnects — remove their filter, recompute, notify other peers
+3. Received filter changes outgoing filter — recompute, send updates
+4. Local state change — new leaf dependent, become gateway, etc.
+
+**Rate limiting:**
+
+To prevent update storms, rate-limit or debounce updates:
+
+```rust
+const MIN_UPDATE_INTERVAL: Duration = Duration::from_millis(500);
+
+fn maybe_send_update(&mut self, peer: PeerId) {
+    if self.last_send_time[peer].elapsed() < MIN_UPDATE_INTERVAL {
+        self.pending_update[peer] = true;
+        return;
+    }
+    // ... send update
+}
+```
+
+### Filter Announcement Message
+
+```rust
+struct FilterAnnounce {
+    filter: BloomFilter,  // 4 KB
+    ttl: u8,              // Remaining propagation hops
+    sequence: u64,        // Freshness / deduplication
+}
+```
+
+### Propagation Rules
+
+When node N sends a FilterAnnounce to peer Q:
+
+1. Include N's own Node ID
+2. Include N's leaf-only dependents
+3. Include entries from filters received from other peers (not Q) with TTL > 0
+
+When node N receives FilterAnnounce from peer P:
+
+1. Store: `peer_filters[P] = received.filter`
+2. If `received.ttl > 0`: include P's filter contents in N's next announcement
+   to other peers, with TTL decremented
+
+### K-Hop Scope Emergence
+
+With TTL starting at K=2:
+
+- Entries propagate ~2K hops before stopping
+- Each node's filter contains destinations within ~4-hop effective range
+- Bounded by O(d^2K) entries regardless of total network size
+
+### Expiration
+
+Bloom filters cannot remove individual entries. Expiration is handled via:
+
+- **Peer disconnect**: Remove that peer's filter entirely, recompute
+- **Filter replacement**: Each FilterAnnounce replaces the previous one
+- **Implicit timeout**: If no updates received from peer within threshold,
+  consider their filter stale
+
+---
+
+## Part 2: Discovery Protocol
+
+### Purpose
+
+Discover the tree coordinates of distant destinations not covered by local
+Bloom filters.
+
+### When Used
+
+- Destination not found in any peer's Bloom filter
+- Route cache miss
+- After cached route failure
+
+### Message Formats
+
+```rust
+struct LookupRequest {
+    request_id: u64,
+    target: NodeId,              // Who we're looking for
+    origin: NodeId,              // Who's asking
+    origin_coords: Vec<NodeId>,  // Origin's ancestry (for return path)
+    ttl: u8,                     // Propagation limit
+    visited: BloomFilter,        // Prevent loops (compact, ~256 bytes)
+}
+
+struct LookupResponse {
+    request_id: u64,
+    target: NodeId,
+    target_coords: Vec<NodeId>,  // Target's ancestry — the key payload
+    proof: Signature,            // Target signs to prove existence
+}
+```
+
+### Discovery Flow
+
+```text
+1. S wants to reach D, D not in any local filter
+2. S checks route cache — miss
+3. S creates LookupRequest with own coordinates, floods to peers
+4. Request propagates (Bloom filters may help direct it)
+5. Request reaches D (or node with D in filter)
+6. D creates LookupResponse with its coordinates, signs it
+7. Response routes back to S using S's coordinates (greedy)
+8. S caches D's coordinates
+9. S can now route to D using greedy tree routing
+```
+
+### Request Propagation
+
+**Flood with TTL and visited filter:**
+
+- Send to all peers not in `visited` filter
+- Each hop decrements TTL, adds self to `visited`
+- At TTL=0, stop propagating
+- `visited` filter prevents redundant processing
+
+**Bloom filter assistance (optional optimization):**
+
+If a node's peer filter indicates "maybe" for the target, prioritize that
+direction. Reduces flood scope when target is partially in range.
+
+### Response Routing
+
+Response uses greedy tree routing based on `origin_coords` from the request.
+Each router forwards toward the origin using tree distance.
+
+### Security
+
+**Target signs response:**
+
+```rust
+struct LookupResponse {
+    // ...
+    proof: Signature,  // Sign(request_id || target || target_coords)
+}
+```
+
+Without this, a malicious node could claim reachability for any target and
+blackhole traffic. The signature proves the target authorized the route.
+
+### Caching
+
+Discovered coordinates are cached:
+
+```rust
+struct RouteCache {
+    entries: HashMap<NodeId, CachedCoords>,
+}
+
+struct CachedCoords {
+    coords: Vec<NodeId>,
+    discovered_at: Timestamp,
+    last_used: Timestamp,
+}
+```
+
+- **Eviction**: LRU when cache full
+- **Expiration**: TTL-based (coordinates may go stale if target moves in tree)
+- **Invalidation**: On route failure, evict and re-discover
+
+---
+
+## Part 3: Tree Coordinates and Greedy Routing
+
+### Tree Coordinates
+
+A node's coordinates are its ancestry path from self to root:
+
+```text
+coords(N) = [N, Parent(N), Parent(Parent(N)), ..., Root]
+```
+
+Example: Node D at depth 4 has coordinates `[D, P1, P2, P3, Root]`.
+
+### Tree Distance
+
+Distance between two nodes is hops through their lowest common ancestor (LCA):
+
+```rust
+fn tree_distance(a_coords: &[NodeId], b_coords: &[NodeId]) -> usize {
+    let lca_depth = longest_common_suffix_length(a_coords, b_coords);
+    let a_to_lca = a_coords.len() - lca_depth;
+    let b_to_lca = b_coords.len() - lca_depth;
+    a_to_lca + b_to_lca
+}
+```
+
+Note: Coordinates are ordered self-to-root, so common ancestry is a suffix.
+
+### Greedy Routing Algorithm
+
+```rust
+fn greedy_next_hop(&self, dest_coords: &[NodeId]) -> PeerId {
+    // Check if we are the destination
+    if dest_coords[0] == self.node_id {
+        return LOCAL_DELIVERY;
+    }
+
+    // Check if destination is a direct peer
+    for peer in &self.peers {
+        if peer.node_id == dest_coords[0] {
+            return peer.id;
+        }
+    }
+
+    // Forward to peer closest to destination
+    self.peers
+        .iter()
+        .min_by_key(|p| tree_distance(&p.coords, dest_coords))
+        .map(|p| p.id)
+        .expect("no peers")
+}
+```
+
+### Guaranteed Progress
+
+Greedy routing makes progress as long as:
+
+1. Tree is connected
+2. Destination's coordinates are accurate
+3. Current node is not the destination
+
+Unlike DHT routing, greedy tree routing cannot get stuck in local minima if
+the tree is properly formed.
+
+### What Each Node Knows
+
+| Information | Source |
+|-------------|--------|
+| Own coordinates | Spanning tree protocol (ancestry to root) |
+| Each peer's coordinates | Exchanged on peering |
+| Destination coordinates | From packet header (established via session) |
+
+No global routing tables. Each node makes purely local decisions.
+
+---
+
+## Part 4: Session Establishment
+
+### Session Purpose
+
+Establish cached coordinate state along a path so that subsequent data packets
+can omit coordinates, minimizing per-packet overhead.
+
+### Session Lifecycle
+
+```text
+┌─────────────────────────────────────────────────────────────────┐
+│  1. Discovery: S queries for D's coordinates                    │
+│  2. Setup: S sends SessionSetup, routers cache coordinates      │
+│  3. Data: Packets carry only addresses, routers use cache       │
+│  4. Refresh: Periodic or on-demand to prevent cache expiry      │
+│  5. Teardown: Implicit (cache expires) or explicit              │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+### Session Message Formats
+
+```rust
+/// Establishes cached state along path
+struct SessionSetup {
+    src_addr: Ipv6Addr,
+    dest_addr: Ipv6Addr,
+    src_coords: Vec<NodeId>,   // For return path caching
+    dest_coords: Vec<NodeId>,  // For forward path routing
+    flags: SessionFlags,
+}
+
+struct SessionFlags {
+    request_ack: bool,         // Ask destination to confirm
+    bidirectional: bool,       // Set up both directions
+}
+
+/// Confirms session establishment
+struct SessionAck {
+    src_addr: Ipv6Addr,
+    dest_addr: Ipv6Addr,
+    src_coords: Vec<NodeId>,   // Acknowledger's coords (for return caching)
+}
+
+/// Minimal data packet
+struct DataPacket {
+    flags: u8,
+    hop_limit: u8,
+    payload_length: u16,
+    src_addr: Ipv6Addr,        // 16 bytes
+    dest_addr: Ipv6Addr,       // 16 bytes
+    payload: Vec<u8>,
+}
+
+/// Error when router cannot route (cache miss)
+struct CoordsRequired {
+    dest_addr: Ipv6Addr,
+    reporter: NodeId,          // Which router had the miss
+}
+```
+
+### Data Packet Overhead
+
+| Field | Size |
+|-------|------|
+| flags | 1 byte |
+| hop_limit | 1 byte |
+| payload_length | 2 bytes |
+| src_addr | 16 bytes |
+| dest_addr | 16 bytes |
+| **Total header** | **36 bytes** |
+
+Comparable to IPv6 (40 bytes). No coordinates in data packets.
+
+### Session Setup Flow
+
+```text
+S                       R1                      R2                      D
+│                        │                       │                       │
+│──SessionSetup─────────>│                       │                       │
+│  (src_coords,          │──SessionSetup────────>│                       │
+│   dest_coords)         │                       │──SessionSetup────────>│
+│                        │                       │                       │
+│                        │  cache:               │  cache:               │
+│                        │  dest_addr→dest_coords│  dest_addr→dest_coords│
+│                        │  src_addr→src_coords  │  src_addr→src_coords  │
+│                        │                       │                       │
+│<─────────────────────────────────────────────────────────SessionAck───│
+│                        │                       │                       │
+│══DataPacket═══════════>│══════════════════════>│══════════════════════>│
+│  (addresses only)      │  (use cached coords)  │  (use cached coords)  │
+```
+
+### Router Behavior
+
+```rust
+impl Router {
+    fn handle_session_setup(&mut self, setup: SessionSetup, from: PeerId) {
+        // Cache coordinates for both directions
+        self.coord_cache.insert(setup.dest_addr, CacheEntry {
+            coords: setup.dest_coords.clone(),
+            expires: now() + CACHE_TTL,
+        });
+        self.coord_cache.insert(setup.src_addr, CacheEntry {
+            coords: setup.src_coords.clone(),
+            expires: now() + CACHE_TTL,
+        });
+
+        // Forward toward destination
+        let next = self.greedy_next_hop(&setup.dest_coords);
+        self.forward(next, setup);
+    }
+
+    fn handle_data_packet(&mut self, packet: DataPacket, from: PeerId) {
+        match self.coord_cache.get(&packet.dest_addr) {
+            Some(entry) => {
+                entry.last_used = now();
+                let next = self.greedy_next_hop(&entry.coords);
+                self.forward(next, packet);
+            }
+            None => {
+                // Cache miss — request coordinates
+                self.send_error(from, CoordsRequired {
+                    dest_addr: packet.dest_addr,
+                    reporter: self.node_id,
+                });
+            }
+        }
+    }
+}
+```
+
+### Cache Management
+
+```rust
+struct CoordCache {
+    entries: HashMap<Ipv6Addr, CacheEntry>,
+    max_entries: usize,
+}
+
+struct CacheEntry {
+    coords: Vec<NodeId>,
+    created: Timestamp,
+    last_used: Timestamp,
+    expires: Timestamp,
+}
+```
+
+**Eviction policy**: LRU (least recently used) when cache exceeds max_entries.
+
+**Expiration**: Entries expire after TTL (e.g., 300 seconds). Can be refreshed
+by:
+
+- Subsequent SessionSetup
+- SessionRefresh message (lightweight, just touches expiry)
+- Data packet transit (optional: refresh on use)
+
+### Handling Cache Eviction
+
+When a router's cache entry is evicted mid-session:
+
+```text
+1. Data packet arrives, cache miss
+2. Router sends CoordsRequired to packet source
+3. Source receives error, re-sends SessionSetup (or packet with coords)
+4. Path warms again, data flow resumes
+```
+
+From application perspective: brief latency spike, transparent recovery.
+
+### Sender Behavior
+
+```rust
+impl Sender {
+    fn send(&mut self, dest: Ipv6Addr, data: &[u8]) {
+        if !self.session_established(dest) {
+            // Need to establish session first
+            let dest_coords = self.discover_or_cached(dest)?;
+            self.send_session_setup(dest, &dest_coords);
+            self.await_session_ack(dest)?;
+        }
+
+        self.send_data_packet(dest, data);
+    }
+
+    fn handle_coords_required(&mut self, err: CoordsRequired) {
+        // Path went cold, re-establish
+        self.mark_session_cold(err.dest_addr);
+        self.send_session_setup(err.dest_addr, &self.cached_coords(err.dest_addr));
+    }
+}
+```
+
+---
+
+## Part 5: Packet Type Summary
+
+| Type | Purpose | Size | When Used |
+|------|---------|------|-----------|
+| FilterAnnounce | Bloom filter propagation | ~4.1 KB | Topology changes |
+| LookupRequest | Discover coordinates | ~300 bytes | First contact with distant node |
+| LookupResponse | Return coordinates | ~400 bytes | Reply to discovery |
+| SessionSetup | Warm router caches | ~400-600 bytes | Before data transfer |
+| SessionAck | Confirm session | ~300 bytes | Optional confirmation |
+| DataPacket | Application data | 36 bytes + payload | Bulk of traffic |
+| CoordsRequired | Request retransmit | ~50 bytes | Cache miss recovery |
+
+---
+
+## Part 6: Traffic Analysis
+
+### Steady State (Stable Network)
+
+- **Bloom filter traffic**: Near zero (event-driven, no changes)
+- **Discovery traffic**: Rare (warm caches)
+- **Session traffic**: Rare (established sessions)
+- **Data traffic**: Minimal overhead (36-byte header)
+
+### Network Churn
+
+When nodes join/leave:
+
+- Bloom filter updates propagate (bounded by K-hop scope)
+- Affected sessions may need re-establishment
+- Discovery queries for newly-joined nodes
+
+### Per-Node Resource Requirements
+
+| Resource | Full Participant | Leaf-Only |
+|----------|------------------|-----------|
+| Bloom filter storage | d × 4 KB (d = peer count) | None |
+| Coordinate cache | 10K-100K entries | None |
+| Route cache | 1K-10K entries | Minimal |
+| Bandwidth (idle) | < 1 KB/sec | Near zero |
+
+---
+
+## Open Questions
+
+1. **Coordinate compression**: Can tree coordinates be compressed for smaller
+   SessionSetup messages? (e.g., delta encoding, shorter node ID representation)
+
+2. **Multi-path routing**: How to handle multiple valid paths? Load balancing?
+   Failover?
+
+3. **Asymmetric paths**: S→D and D→S may traverse different routers. Is this
+   acceptable or should paths be symmetric?
+
+4. **Gateway /64 prefixes**: How do subnet prefixes interact with Bloom filters
+   and discovery? One filter entry per gateway regardless of devices behind it?
+
+5. **Cache sizing**: What's the right cache size for different node roles?
+   Core nodes vs. edge nodes?
+
+6. **Mobility**: When a node changes tree position (new parent), how quickly
+   do sessions recover? Should nodes announce position changes?
+
+---
+
+## References
+
+- [fips-design.md](fips-design.md) — Overall FIPS architecture
+- [fips-links.md](fips-links.md) — Link layer requirements
+- [spanning-tree-dynamics.md](spanning-tree-dynamics.md) — Tree protocol details