Implement greedy routing with bloom filter priority

Add the full next-hop routing algorithm to Node::find_next_hop():
- Local delivery, direct peer, bloom filter candidates, greedy tree
  routing fallback, with (link_cost, tree_distance, node_addr) ordering
- select_best_candidate() scores by peer→dest distance (not us→peer)
  with self-distance check to prevent routing loops
- TreeState::find_next_hop() for greedy tree routing with progress
  guarantee
- ActivePeer::link_cost() placeholder (constant 1.0) for future link
  quality metrics

Add routing tests including 100-node all-pairs reachability simulation
(9900/9900 delivered, 0 loops, avg 4.0 hops, max 8).

Update fips-routing.md to reflect bloom filter routing as the primary
forwarding mechanism, with greedy tree routing as fallback during
convergence windows.
This commit is contained in:
Johnathan Corgan
2026-02-11 16:55:14 +00:00
parent 066865ddd7
commit 2f8e97c0ab
6 changed files with 942 additions and 74 deletions
+123 -62
View File
@@ -1,25 +1,46 @@
# FIPS Routing Design
This document describes the routing architecture for FIPS, including Bloom
filter reachability, discovery protocol, greedy tree routing, and routing
session establishment.
filter routing, greedy tree routing, discovery protocol, and routing session
establishment.
For wire formats and exchange rules, see [fips-gossip-protocol.md](fips-gossip-protocol.md).
For spanning tree dynamics and convergence, see [spanning-tree-dynamics.md](spanning-tree-dynamics.md).
## Overview
FIPS routing combines three mechanisms:
FIPS uses a layered routing strategy where each mechanism handles different
situations. In steady state, bloom filter routing handles the vast majority
of forwarding decisions.
1. **Bloom filters**: Fast reachability lookup for destinations reachable
through peers
2. **Discovery protocol**: Query-based lookup for distant destinations
3. **Greedy tree routing**: Coordinate-based forwarding using spanning tree
position
### Next-Hop Selection (in priority order)
The design separates discovery (finding where a destination is) from routing
(getting packets there). Bloom filters and discovery handle the former; tree
coordinates handle the latter.
1. **Local delivery** destination is self
2. **Direct peer** — destination is an authenticated peer
3. **Bloom filter routing** — one or more peers' bloom filters contain the
destination; select the best candidate by `(link_cost, tree_distance,
node_addr)`. Since filters propagate unboundedly through the network,
every reachable destination eventually appears in at least one peer's
filter. This is the **primary routing path** for most traffic.
4. **Greedy tree routing** — fallback when bloom filters haven't yet
converged (transient condition during topology changes). Requires the
destination's tree coordinates to be in the local coordinate cache,
populated by a prior SessionSetup or LookupResponse.
5. **No route** — destination unreachable
### Role of Each Mechanism
- **Bloom filters**: Primary forwarding — tell each node which peer to
send through for a given destination. Propagate unboundedly via
split-horizon merge, so they cover the entire reachable network at
steady state.
- **Greedy tree routing**: Fallback forwarding during convergence windows
when bloom filters are incomplete. Also used for routing LookupResponse
messages back to the origin.
- **Discovery protocol**: Populates the coordinate cache to enable greedy
tree routing and to provide intermediate routers with coordinate data
for more efficient path selection. Not required for basic reachability
once bloom filters have converged.
## Design Goals
@@ -33,11 +54,15 @@ coordinates handle the latter.
| Scale | Nodes | Bloom Filter Role |
|-------|-------|-------------------|
| Small private network | 100-1,000 | Covers entire network |
| Modest public network | ~1,000,000 | Covers transitive peer neighborhood |
| Small private network | 100-1,000 | Covers entire network with low FPR |
| Modest public network | ~1,000,000 | Covers entire network but FPR increases at hub nodes due to filter saturation |
| Internet-scale | Billions | Out of scope (requires different architecture) |
The primary design target is networks up to ~1M nodes.
The primary design target is networks up to ~1M nodes. Since bloom filters
propagate unboundedly (no TTL), they converge to represent the entire
reachable network. At large scale, the fixed 1KB filter size means higher
false positive rates at well-connected hub nodes, which may trigger
unnecessary discovery queries but does not affect correctness.
## Node Participation Modes
@@ -127,9 +152,12 @@ the node's reachable neighborhood.
| 1,200 | 7.5% | Hub node |
| 1,600 | 15% | Heavily loaded hub |
FPR above 5% triggers more LookupRequests but the discovery protocol handles
this gracefully. Hub nodes may benefit from larger filters in future protocol
versions (see §1.6).
Since filters propagate unboundedly, hub nodes with many peers will have
higher occupancy (more entries merged from more peers). FPR above 5% means
bloom filter routing may occasionally select a peer that can't actually
reach the destination (false positive), requiring fallback to greedy tree
routing or error recovery. Hub nodes may benefit from larger filters in
future protocol versions (see §1.6).
### Size Classes (Forward Compatibility)
@@ -220,14 +248,16 @@ Bloom filters cannot remove individual entries. Expiration is handled via:
### Purpose
Discover the tree coordinates of distant destinations not covered by local
Bloom filters.
Discover the tree coordinates of a destination to enable greedy tree routing
and to populate coordinate caches at intermediate routers for more efficient
forwarding. In steady state, bloom filters handle reachability; discovery
provides the coordinate information that improves path selection quality.
### When Used
- Destination not found in any peer's Bloom filter
- Route cache miss
- After cached route failure
- During bloom filter convergence (destination not yet in any peer's filter)
- To populate coordinate caches for greedy tree routing (optimization)
- After cached route failure (coordinates may be stale)
For wire formats, see [fips-gossip-protocol.md](fips-gossip-protocol.md) §4-5.
@@ -299,6 +329,13 @@ struct CachedCoords {
## Part 3: Tree Coordinates and Greedy Routing
Tree coordinates and greedy routing serve two roles:
1. **Fallback forwarding** during bloom filter convergence windows
2. **Tie-breaking** among bloom filter candidates — tree distance between
a candidate peer and the destination helps select the best path when
multiple peers advertise reachability
### Tree Coordinates
A node's coordinates are its ancestry path from self to root:
@@ -326,39 +363,48 @@ Note: Coordinates are ordered self-to-root, so common ancestry is a suffix.
### Greedy Routing Algorithm
When used as a fallback (no bloom filter hits), greedy routing forwards to
the peer that minimizes tree distance to the destination. A self-distance
check ensures progress — the packet is only forwarded if the chosen peer is
strictly closer than the current node.
```rust
fn greedy_next_hop(&self, dest_coords: &[NodeAddr]) -> NodeAddr {
// Check if we are the destination
if dest_coords[0] == self.node_addr {
return LOCAL_DELIVERY;
fn greedy_next_hop(&self, dest_coords: &TreeCoordinate) -> Option<NodeAddr> {
if self.my_coords.root_id() != dest_coords.root_id() {
return None; // different tree
}
// Check if destination is a direct peer
for peer in &self.peers {
if peer.node_addr == dest_coords[0] {
return peer.node_addr;
let my_distance = self.my_coords.distance_to(dest_coords);
// Find peer with minimum distance, tie-break by smallest node_addr
let best = self.peer_ancestry.iter()
.min_by(|(id_a, coords_a), (id_b, coords_b)| {
coords_a.distance_to(dest_coords)
.cmp(&coords_b.distance_to(dest_coords))
.then_with(|| id_a.cmp(id_b))
});
match best {
Some((peer_id, coords)) if coords.distance_to(dest_coords) < my_distance => {
Some(*peer_id)
}
_ => None, // no peer is closer (local minimum)
}
// Forward to peer closest to destination
self.peers
.iter()
.min_by_key(|p| tree_distance(&p.coords, dest_coords))
.map(|p| p.node_addr)
.expect("no peers")
}
```
### Guaranteed Progress
### Progress Guarantee
Greedy routing makes progress as long as:
When the coordinate cache is populated, greedy routing makes progress as
long as:
1. Tree is connected
2. Destination's coordinates are accurate
3. Current node is not the destination
3. A peer is closer to the destination than the current node
Unlike DHT routing, greedy tree routing cannot get stuck in local minima if
the tree is properly formed.
If no peer is closer (local minimum), routing returns `None` and the caller
generates a PathBroken error. In a properly formed tree this should not
occur, but the self-distance check provides a safety net.
### What Each Node Knows
@@ -404,9 +450,13 @@ with explicit error signaling over metadata privacy.
### Route Cache Purpose
Intermediate routers cache coordinate mappings so that data packets can use
minimal headers (addresses only, no coordinates). This reduces per-packet
overhead from ~300 bytes to 38 bytes.
The coordinate cache serves two functions:
1. **Greedy routing fallback** — when bloom filters haven't converged,
cached coordinates enable tree-distance-based forwarding.
2. **Reduced packet overhead** — once coordinates are cached at intermediate
routers, data packets can carry addresses only (38 bytes) rather than
full coordinates (~300 bytes).
### Cache Lifecycle
@@ -476,7 +526,15 @@ impl Router {
}
}
// Route using cache (now populated if coords were present)
// Primary: bloom filter routing
let candidates = self.destination_in_filters(&packet.dest_addr);
if !candidates.is_empty() {
let next = self.select_best_candidate(&candidates);
self.forward(next, packet);
return;
}
// Fallback: greedy tree routing via cached coordinates
match self.coord_cache.get(&packet.dest_addr) {
Some(entry) => {
entry.last_used = now();
@@ -484,7 +542,7 @@ impl Router {
self.forward(next, packet);
}
None => {
// Cache miss — request coordinates
// No bloom filter hit and no cached coordinates
self.send_error(from, CoordsRequired {
dest_addr: packet.dest_addr,
reporter: self.node_addr,
@@ -638,25 +696,28 @@ consulting the visited filter. This is referenced in the gossip protocol spec
(section 4.4, rate limiting) but needs to be elevated to a protocol
requirement, not an optimization.
### Known Limitation: Capacity-Blind Greedy Routing
### Known Limitation: Capacity-Blind Routing
Greedy routing selects the next hop by minimizing tree distance, which is
purely topological (hop count through the LCA). It does not account for link
capacity, latency, or loss.
Both bloom filter candidate selection and greedy tree routing currently
select next hops without considering link quality. When multiple peers
can reach a destination, the selection is based on tree distance and
node address tie-breaking — purely topological metrics that ignore link
capacity, latency, and loss.
This creates a problem when a topologically short but low-capacity link exists
alongside a longer but high-capacity path. Greedy routing will prefer the short
path, potentially saturating the slow link while the high-capacity path goes
underutilized.
This creates a problem when a topologically short but low-capacity link
exists alongside a longer but high-capacity path. The routing algorithm
will prefer the topologically closer peer, potentially saturating a slow
link while a higher-capacity path goes underutilized.
**Proposed mitigation**: Each node locally measures the quality of its direct
peer links (RTT, bandwidth, loss) and applies a cost adjustment to the
forwarding decision:
**Proposed mitigation**: Each node locally measures the quality of its
direct peer links (RTT, bandwidth, loss) and incorporates this into a
`link_cost()` metric. The next-hop selection uses a composite ordering
of `(link_cost, tree_distance, node_addr)` — link quality takes priority
over topological distance.
```text
effective_distance(peer, dest) =
tree_distance(peer, dest) × local_link_cost(peer)
```
The `link_cost()` interface is implemented (currently returning a constant),
ready to be populated with real measurements using an established link
quality algorithm (ETX, Babel composite metric, etc.).
This requires no protocol changes — link quality is measured locally, not
advertised. Self-reported cost claims are intentionally excluded from the
+114 -6
View File
@@ -706,14 +706,122 @@ impl Node {
self.peers.values().filter(|p| p.can_send()).count()
}
// === Routing (stubs) ===
// === Routing ===
/// Find next hop for a destination (stub).
/// Find next hop for a destination node address.
///
/// Returns the peer that minimizes tree distance to the destination.
pub fn find_next_hop(&self, _dest_node_addr: &NodeAddr) -> Option<&ActivePeer> {
// Stub: would implement greedy tree routing
None
/// Routing priority:
/// 1. Destination is self → `None` (local delivery)
/// 2. Destination is a direct peer → that peer
/// 3. Bloom filter candidates + greedy tree routing → among peers whose
/// bloom filter contains the destination, pick the one that minimizes
/// tree distance to the destination (if dest coords are cached), with
/// `(link_cost, tree_distance_to_dest, node_addr)` tie-breaking.
/// Falls back to greedy tree routing if no bloom filter hits.
/// 4. No route → `None`
///
/// The self-distance check from greedy routing also applies to bloom
/// filter candidates: a peer is only selected if it is strictly closer
/// to the destination than we are (prevents routing loops).
pub fn find_next_hop(&self, dest_node_addr: &NodeAddr) -> Option<&ActivePeer> {
// 1. Local delivery
if dest_node_addr == self.node_addr() {
return None;
}
// 2. Direct peer
if let Some(peer) = self.peers.get(dest_node_addr) {
if peer.can_send() {
return Some(peer);
}
}
// Look up destination coords (used by both bloom and tree paths)
let now_ms = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
let dest_coords = self.coord_cache.get(dest_node_addr, now_ms);
// 3. Bloom filter candidates, scored by tree distance to dest
let candidates: Vec<&ActivePeer> = self.destination_in_filters(dest_node_addr);
if !candidates.is_empty() {
return self.select_best_candidate(&candidates, dest_coords);
}
// 4. Greedy tree routing fallback
let dest_coords = dest_coords?;
let next_hop_id = self.tree_state.find_next_hop(dest_coords)?;
self.peers.get(&next_hop_id).filter(|p| p.can_send())
}
/// Select the best peer from a set of bloom filter candidates.
///
/// When dest_coords are available, uses distance from each candidate's
/// coordinates to the destination as the primary metric (after link_cost).
/// Only selects peers that are strictly closer to the destination than
/// we are (self-distance check prevents loops).
///
/// When dest_coords are not available, falls back to distance from us
/// to the candidate peer (a weaker heuristic — prefers closer peers on
/// the theory that shorter paths are better).
///
/// Ordering: `(link_cost, distance_to_dest, node_addr)`.
fn select_best_candidate<'a>(
&'a self,
candidates: &[&'a ActivePeer],
dest_coords: Option<&crate::tree::TreeCoordinate>,
) -> Option<&'a ActivePeer> {
let my_distance = dest_coords.map(|dc| self.tree_state.my_coords().distance_to(dc));
let mut best: Option<(&ActivePeer, f64, usize)> = None;
for &candidate in candidates {
if !candidate.can_send() {
continue;
}
let cost = candidate.link_cost();
// Compute distance: peer→dest if coords available, else us→peer
let dist = match dest_coords {
Some(dc) => self
.tree_state
.peer_coords(candidate.node_addr())
.map(|pc| pc.distance_to(dc))
.unwrap_or(usize::MAX),
None => self
.tree_state
.distance_to_peer(candidate.node_addr())
.unwrap_or(usize::MAX),
};
// Self-distance check: when dest coords are available,
// only consider peers that are strictly closer than us
if let Some(my_dist) = my_distance {
if dist >= my_dist {
continue;
}
}
let dominated = match &best {
None => true,
Some((_, best_cost, best_dist)) => {
cost < *best_cost
|| (cost == *best_cost && dist < *best_dist)
|| (cost == *best_cost
&& dist == *best_dist
&& candidate.node_addr() < best.as_ref().unwrap().0.node_addr())
}
};
if dominated {
best = Some((candidate, cost, dist));
}
}
best.map(|(peer, _, _)| peer)
}
/// Check if a destination is in any peer's bloom filter.
+1
View File
@@ -6,6 +6,7 @@ use std::time::Duration;
mod bloom;
mod handshake;
mod routing;
mod spanning_tree;
mod unit;
+518
View File
@@ -0,0 +1,518 @@
//! Routing integration tests.
//!
//! Tests the full Node::find_next_hop() routing logic including bloom
//! filter priority, greedy tree routing, and tie-breaking.
use super::*;
use crate::bloom::BloomFilter;
use crate::tree::{ParentDeclaration, TreeCoordinate};
use spanning_tree::{
cleanup_nodes, drain_all_packets, generate_random_edges, initiate_handshake, make_test_node,
run_tree_test, verify_tree_convergence, TestNode,
};
use std::collections::HashSet;
// === Local delivery ===
#[test]
fn test_routing_local_delivery() {
let node = make_node();
let my_addr = *node.node_addr();
assert!(node.find_next_hop(&my_addr).is_none());
}
// === Direct peer ===
#[test]
fn test_routing_direct_peer() {
let mut node = make_node();
let transport_id = TransportId::new(1);
let link_id = LinkId::new(1);
let (conn, identity) = make_completed_connection(&mut node, link_id, transport_id, 1000);
let peer_addr = *identity.node_addr();
node.add_connection(conn).unwrap();
node.promote_connection(link_id, identity, 2000).unwrap();
let result = node.find_next_hop(&peer_addr);
assert!(result.is_some());
assert_eq!(result.unwrap().node_addr(), &peer_addr);
}
// === No route ===
#[test]
fn test_routing_unknown_destination() {
let node = make_node();
let unknown = make_node_addr(99);
assert!(node.find_next_hop(&unknown).is_none());
}
// === Bloom filter priority ===
#[test]
fn test_routing_bloom_filter_hit() {
let mut node = make_node();
let transport_id = TransportId::new(1);
// Create two peers
let link_id1 = LinkId::new(1);
let (conn1, id1) = make_completed_connection(&mut node, link_id1, transport_id, 1000);
let peer1_addr = *id1.node_addr();
node.add_connection(conn1).unwrap();
node.promote_connection(link_id1, id1, 2000).unwrap();
let link_id2 = LinkId::new(2);
let (conn2, id2) = make_completed_connection(&mut node, link_id2, transport_id, 1000);
let peer2_addr = *id2.node_addr();
node.add_connection(conn2).unwrap();
node.promote_connection(link_id2, id2, 2000).unwrap();
// Destination not directly connected
let dest = make_node_addr(99);
// Add dest to peer1's bloom filter only
let peer1 = node.get_peer_mut(&peer1_addr).unwrap();
let mut filter = BloomFilter::new();
filter.insert(&dest);
peer1.update_filter(filter, 1, 3000);
// Should route through peer1 (bloom filter hit)
let result = node.find_next_hop(&dest);
assert!(result.is_some());
assert_eq!(result.unwrap().node_addr(), &peer1_addr);
// Peer2 should NOT be selected (no filter hit)
assert_ne!(result.unwrap().node_addr(), &peer2_addr);
}
#[test]
fn test_routing_bloom_filter_multiple_hits_tiebreak() {
let mut node = make_node();
let transport_id = TransportId::new(1);
// Create three peers
let mut peer_addrs = Vec::new();
for i in 1..=3 {
let link_id = LinkId::new(i);
let (conn, id) = make_completed_connection(&mut node, link_id, transport_id, 1000);
let addr = *id.node_addr();
peer_addrs.push(addr);
node.add_connection(conn).unwrap();
node.promote_connection(link_id, id, 2000).unwrap();
}
let dest = make_node_addr(99);
// Add dest to ALL peers' bloom filters
for &addr in &peer_addrs {
let peer = node.get_peer_mut(&addr).unwrap();
let mut filter = BloomFilter::new();
filter.insert(&dest);
peer.update_filter(filter, 1, 3000);
}
// All peers have equal link_cost (1.0) and no tree coords set,
// so tree distance is usize::MAX for all. Tie-break by smallest node_addr.
let result = node.find_next_hop(&dest);
assert!(result.is_some());
let smallest_addr = peer_addrs.iter().min().unwrap();
assert_eq!(result.unwrap().node_addr(), smallest_addr);
}
// === Greedy tree routing ===
#[test]
fn test_routing_tree_fallback() {
let mut node = make_node();
let transport_id = TransportId::new(1);
let my_addr = *node.node_addr();
// Create a peer
let link_id = LinkId::new(1);
let (conn, id) = make_completed_connection(&mut node, link_id, transport_id, 1000);
let peer_addr = *id.node_addr();
node.add_connection(conn).unwrap();
node.promote_connection(link_id, id, 2000).unwrap();
// Set up tree state through the public API.
// We're root, peer is our child. The peer has a subtree below it.
// TreeState::new() already makes us the root with coords [my_addr].
// Add peer as child of us.
let peer_coords = TreeCoordinate::from_addrs(vec![peer_addr, my_addr]).unwrap();
node.tree_state_mut().update_peer(
ParentDeclaration::new(peer_addr, my_addr, 1, 1000),
peer_coords,
);
// Destination: a node under our peer in the tree
let dest = make_node_addr(99);
let dest_coords =
TreeCoordinate::from_addrs(vec![dest, peer_addr, my_addr]).unwrap();
// Put dest coords in the cache
let now_ms = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
node.coord_cache_mut().insert(dest, dest_coords, now_ms);
// No bloom filter hit — should fall back to tree routing.
// Our distance to dest: 2 (root → peer → dest)
// Peer's distance to dest: 1 (peer → dest)
// Peer is closer, so it's the next hop.
let result = node.find_next_hop(&dest);
assert!(result.is_some());
assert_eq!(result.unwrap().node_addr(), &peer_addr);
}
#[test]
fn test_routing_tree_no_coords_in_cache() {
let mut node = make_node();
let transport_id = TransportId::new(1);
// Create a peer
let link_id = LinkId::new(1);
let (conn, id) = make_completed_connection(&mut node, link_id, transport_id, 1000);
node.add_connection(conn).unwrap();
node.promote_connection(link_id, id, 2000).unwrap();
// Destination not in bloom filters and not in coord cache
let dest = make_node_addr(99);
assert!(node.find_next_hop(&dest).is_none());
}
// === Integration: converged network ===
#[tokio::test]
async fn test_routing_chain_topology() {
// Build a 4-node chain: 0 -- 1 -- 2 -- 3
let mut nodes = vec![
make_test_node().await,
make_test_node().await,
make_test_node().await,
make_test_node().await,
];
// Connect the chain
initiate_handshake(&mut nodes, 0, 1).await;
initiate_handshake(&mut nodes, 1, 2).await;
initiate_handshake(&mut nodes, 2, 3).await;
// Converge tree and bloom filters
drain_all_packets(&mut nodes, false).await;
// Verify tree convergence
let root = nodes.iter().map(|n| *n.node.node_addr()).min().unwrap();
for tn in &nodes {
assert_eq!(
*tn.node.tree_state().root(),
root,
"Tree not converged"
);
}
// Populate coord caches: each node caches the far-end node's coords
let now_ms = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
let node3_addr = *nodes[3].node.node_addr();
let node3_coords = nodes[3].node.tree_state().my_coords().clone();
nodes[0]
.node
.coord_cache_mut()
.insert(node3_addr, node3_coords, now_ms);
let node0_addr = *nodes[0].node.node_addr();
let node0_coords = nodes[0].node.tree_state().my_coords().clone();
nodes[3]
.node
.coord_cache_mut()
.insert(node0_addr, node0_coords, now_ms);
// Node 0 should be able to route toward node 3.
// The next hop should be node 1 (only peer of node 0).
let hop = nodes[0].node.find_next_hop(&node3_addr);
assert!(hop.is_some(), "Node 0 should find route to node 3");
let node1_addr = *nodes[1].node.node_addr();
assert_eq!(
hop.unwrap().node_addr(),
&node1_addr,
"Node 0's next hop to node 3 should be node 1"
);
// Node 3 should route toward node 0 via node 2.
let hop = nodes[3].node.find_next_hop(&node0_addr);
assert!(hop.is_some(), "Node 3 should find route to node 0");
let node2_addr = *nodes[2].node.node_addr();
assert_eq!(
hop.unwrap().node_addr(),
&node2_addr,
"Node 3's next hop to node 0 should be node 2"
);
}
#[tokio::test]
async fn test_routing_bloom_preferred_over_tree() {
// Build a 3-node triangle: 0 -- 1, 0 -- 2, 1 -- 2
let mut nodes = vec![
make_test_node().await,
make_test_node().await,
make_test_node().await,
];
initiate_handshake(&mut nodes, 0, 1).await;
initiate_handshake(&mut nodes, 0, 2).await;
initiate_handshake(&mut nodes, 1, 2).await;
drain_all_packets(&mut nodes, false).await;
// Create a destination beyond the network
let dest = make_node_addr(99);
// Add dest to peer 2's bloom filter (from node 0's perspective)
let peer2_addr = *nodes[2].node.node_addr();
let peer2 = nodes[0].node.get_peer_mut(&peer2_addr).unwrap();
let mut filter = BloomFilter::new();
filter.insert(&dest);
peer2.update_filter(filter, 100, 50000);
// Even though we could use tree routing (if coords were cached),
// the bloom filter hit should be preferred.
let hop = nodes[0].node.find_next_hop(&dest);
assert!(hop.is_some(), "Should route via bloom filter");
assert_eq!(
hop.unwrap().node_addr(),
&peer2_addr,
"Should pick peer with bloom filter hit"
);
}
// === Multi-hop forwarding simulation ===
/// Result of simulating multi-hop packet forwarding.
#[derive(Debug)]
enum ForwardResult {
/// Packet reached the destination in the given number of hops.
Delivered(usize),
/// Routing returned None at the given node index (no route).
NoRoute { at_node: usize, hops: usize },
/// Routing loop detected (visited the same node twice).
Loop { at_node: usize, hops: usize },
}
/// Build a NodeAddr → node index lookup table.
fn build_addr_index(nodes: &[TestNode]) -> std::collections::HashMap<NodeAddr, usize> {
nodes
.iter()
.enumerate()
.map(|(i, tn)| (*tn.node.node_addr(), i))
.collect()
}
/// Simulate multi-hop forwarding from source to destination.
///
/// At each hop, calls `find_next_hop` on the current node and follows
/// the result to the next node. Terminates on delivery, routing failure,
/// or loop detection.
fn simulate_forwarding(
nodes: &[TestNode],
addr_index: &std::collections::HashMap<NodeAddr, usize>,
src: usize,
dst: usize,
) -> ForwardResult {
let dest_addr = *nodes[dst].node.node_addr();
let max_hops = nodes.len(); // can't take more hops than nodes
let mut current = src;
let mut visited = HashSet::new();
visited.insert(current);
for hop in 0..max_hops {
let next = nodes[current].node.find_next_hop(&dest_addr);
match next {
None => {
// find_next_hop returns None for local delivery (dest == self)
if *nodes[current].node.node_addr() == dest_addr {
return ForwardResult::Delivered(hop);
}
return ForwardResult::NoRoute {
at_node: current,
hops: hop,
};
}
Some(peer) => {
let next_addr = *peer.node_addr();
// Is next hop the destination?
if next_addr == dest_addr {
return ForwardResult::Delivered(hop + 1);
}
// Find the node index for the next hop
let next_idx = match addr_index.get(&next_addr) {
Some(&idx) => idx,
None => {
return ForwardResult::NoRoute {
at_node: current,
hops: hop,
};
}
};
// Loop detection
if visited.contains(&next_idx) {
return ForwardResult::Loop {
at_node: next_idx,
hops: hop + 1,
};
}
visited.insert(next_idx);
current = next_idx;
}
}
}
ForwardResult::NoRoute {
at_node: current,
hops: max_hops,
}
}
/// 100-node random graph: verify all-pairs routing reachability.
///
/// After tree and bloom filter convergence, simulates multi-hop packet
/// forwarding between every pair of nodes. Every packet must be delivered
/// without loops.
#[tokio::test]
async fn test_routing_reachability_100_nodes() {
const NUM_NODES: usize = 100;
const TARGET_EDGES: usize = 250;
const SEED: u64 = 42;
let edges = generate_random_edges(NUM_NODES, TARGET_EDGES, SEED);
let mut nodes = run_tree_test(NUM_NODES, &edges, false).await;
verify_tree_convergence(&nodes);
// Populate coord caches: every node learns every other node's coordinates.
// In production this happens via SessionSetup/LookupResponse; here we
// inject them directly so routing can make progress-based decisions.
let now_ms = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
// Collect all (addr, coords) pairs first to avoid borrow issues
let all_coords: Vec<(NodeAddr, TreeCoordinate)> = nodes
.iter()
.map(|tn| (*tn.node.node_addr(), tn.node.tree_state().my_coords().clone()))
.collect();
for node in &mut nodes {
for &(ref addr, ref coords) in &all_coords {
if addr != node.node.node_addr() {
node.node.coord_cache_mut().insert(*addr, coords.clone(), now_ms);
}
}
}
let addr_index = build_addr_index(&nodes);
let mut total_pairs = 0;
let mut total_hops = 0usize;
let mut max_hops = 0usize;
let mut failures = Vec::new();
let mut loops = Vec::new();
// Test all pairs
for src in 0..NUM_NODES {
for dst in 0..NUM_NODES {
if src == dst {
continue;
}
total_pairs += 1;
match simulate_forwarding(&nodes, &addr_index, src, dst) {
ForwardResult::Delivered(hops) => {
total_hops += hops;
if hops > max_hops {
max_hops = hops;
}
}
ForwardResult::NoRoute { at_node, hops } => {
failures.push((src, dst, at_node, hops));
}
ForwardResult::Loop { at_node, hops } => {
loops.push((src, dst, at_node, hops));
}
}
}
}
let delivered = total_pairs - failures.len() - loops.len();
let avg_hops = if delivered > 0 {
total_hops as f64 / delivered as f64
} else {
0.0
};
eprintln!(
"\n === Routing Reachability ({} nodes) ===",
NUM_NODES
);
eprintln!(
" Pairs tested: {} | Delivered: {} | Failed: {} | Loops: {}",
total_pairs,
delivered,
failures.len(),
loops.len()
);
eprintln!(
" Hops: avg={:.1} max={}",
avg_hops, max_hops
);
if !failures.is_empty() {
let show = failures.len().min(10);
eprintln!(" First {} failures:", show);
for &(src, dst, at_node, hops) in &failures[..show] {
eprintln!(
" {} -> {}: stuck at node {} after {} hops",
src, dst, at_node, hops
);
}
}
if !loops.is_empty() {
let show = loops.len().min(10);
eprintln!(" First {} loops:", show);
for &(src, dst, at_node, hops) in &loops[..show] {
eprintln!(
" {} -> {}: loop at node {} after {} hops",
src, dst, at_node, hops
);
}
}
assert!(
loops.is_empty(),
"Detected {} routing loops out of {} pairs",
loops.len(),
total_pairs
);
assert!(
failures.is_empty(),
"Detected {} routing failures out of {} pairs",
failures.len(),
total_pairs
);
cleanup_nodes(&mut nodes).await;
}
+9
View File
@@ -391,6 +391,15 @@ impl ActivePeer {
&mut self.link_stats
}
/// Link cost for routing decisions.
///
/// Returns a scalar cost where lower is better. Currently returns a
/// constant (all links equal). Future versions will compute from RTT,
/// loss rate, and throughput measurements.
pub fn link_cost(&self) -> f64 {
1.0
}
/// When this peer was authenticated.
pub fn authenticated_at(&self) -> u64 {
self.authenticated_at
+177 -6
View File
@@ -589,13 +589,46 @@ impl TreeState {
.map(|coords| self.my_coords.distance_to(coords))
}
/// Find the best next hop toward a destination.
/// Find the best next hop toward a destination using greedy tree routing.
///
/// Returns the peer that minimizes tree distance to the destination.
/// This is a stub - full implementation requires greedy routing logic.
pub fn find_next_hop(&self, _dest_coords: &TreeCoordinate) -> Option<NodeAddr> {
// Stub: would implement greedy tree routing
None
/// Returns the peer that minimizes tree distance to the destination,
/// but only if that peer is strictly closer than we are (prevents
/// routing loops at local minima). Tie-breaks equal distance by
/// smallest node_addr.
///
/// Returns `None` if:
/// - No peers have coordinates
/// - Destination is in a different tree (different root)
/// - No peer is closer to the destination than we are
pub fn find_next_hop(&self, dest_coords: &TreeCoordinate) -> Option<NodeAddr> {
if self.my_coords.root_id() != dest_coords.root_id() {
return None;
}
let my_distance = self.my_coords.distance_to(dest_coords);
let mut best: Option<(NodeAddr, usize)> = None;
for (peer_id, peer_coords) in &self.peer_ancestry {
let distance = peer_coords.distance_to(dest_coords);
let dominated = match &best {
None => true,
Some((best_id, best_dist)) => {
distance < *best_dist
|| (distance == *best_dist && peer_id < best_id)
}
};
if dominated {
best = Some((*peer_id, distance));
}
}
match best {
Some((peer_id, distance)) if distance < my_distance => Some(peer_id),
_ => None,
}
}
/// Minimum depth improvement required to switch parents (same root).
@@ -1301,4 +1334,142 @@ mod tests {
assert!(state.my_declaration().sequence() > seq_before);
assert_eq!(state.root(), &my_node);
}
// === find_next_hop tests ===
/// Build a TreeState with our own coordinates set.
fn make_tree_state(my_addr: u8, coord_path: &[u8]) -> TreeState {
let my_node = make_node_addr(my_addr);
let mut state = TreeState::new(my_node);
let coords = make_coords(coord_path);
state.root = *coords.root_id();
state.my_coords = coords;
state
}
/// Add a peer with given coordinates to the tree state.
fn add_peer(state: &mut TreeState, peer_addr: u8, coord_path: &[u8]) {
let peer = make_node_addr(peer_addr);
let parent = make_node_addr(coord_path[1]);
state.update_peer(
ParentDeclaration::new(peer, parent, 1, 1000),
make_coords(coord_path),
);
}
#[test]
fn test_find_next_hop_chain() {
// Chain: 0 (root) <- 5 (us) <- 1 <- 2
// Both peers 1 and 2 are in our peer_ancestry. Peer 2 IS the
// destination (distance 0), so it's the best next hop.
let mut state = make_tree_state(5, &[5, 0]);
add_peer(&mut state, 1, &[1, 5, 0]);
add_peer(&mut state, 2, &[2, 1, 5, 0]);
let dest = make_coords(&[2, 1, 5, 0]);
assert_eq!(state.find_next_hop(&dest), Some(make_node_addr(2)));
}
#[test]
fn test_find_next_hop_chain_indirect() {
// Chain: 0 (root) <- 5 (us) <- 1
// Dest is node 2 at [2, 1, 5, 0] but peer 2 is NOT in our peer
// list — only peer 1 is. So we route via peer 1 (distance 1).
let mut state = make_tree_state(5, &[5, 0]);
add_peer(&mut state, 1, &[1, 5, 0]);
let dest = make_coords(&[2, 1, 5, 0]);
assert_eq!(state.find_next_hop(&dest), Some(make_node_addr(1)));
}
#[test]
fn test_find_next_hop_toward_root() {
// Tree: 0 (root) <- 1 <- 5 (us)
// Routing toward root should pick node 1 (our parent).
let mut state = make_tree_state(5, &[5, 1, 0]);
add_peer(&mut state, 1, &[1, 0]);
let dest = make_coords(&[0]);
assert_eq!(state.find_next_hop(&dest), Some(make_node_addr(1)));
}
#[test]
fn test_find_next_hop_sibling() {
// Tree: 0 (root) <- 5 (us), 0 <- 3
// Routing to sibling 3: should go through parent 0... but 0 is
// the root and not in our peer list. Our only peer is 3 itself.
// But 3 is not a "closer" peer in tree distance — distance from
// us to 3 is 2 (up to root, down to 3), and distance from 3 to
// 3 is 0, so 3 IS closer. Should pick 3.
let mut state = make_tree_state(5, &[5, 0]);
add_peer(&mut state, 3, &[3, 0]);
let dest = make_coords(&[3, 0]);
assert_eq!(state.find_next_hop(&dest), Some(make_node_addr(3)));
}
#[test]
fn test_find_next_hop_tie_breaking() {
// Tree: 0 (root) <- 5 (us), 0 <- 3, 0 <- 2
// Both peers are siblings at depth 1, equidistant to a dest
// at [4, 0]. Should pick node 2 (smaller node_addr).
let mut state = make_tree_state(5, &[5, 0]);
add_peer(&mut state, 3, &[3, 0]);
add_peer(&mut state, 2, &[2, 0]);
let dest = make_coords(&[4, 0]);
// Our distance: 2 (up to root, down to 4)
// Peer 3 distance: 2 (up to root, down to 4)
// Peer 2 distance: 2 (up to root, down to 4)
// All equal to our distance — no peer is strictly closer.
assert_eq!(state.find_next_hop(&dest), None);
}
#[test]
fn test_find_next_hop_different_root() {
let mut state = make_tree_state(5, &[5, 0]);
add_peer(&mut state, 1, &[1, 0]);
// Destination in a different tree (root = 9)
let dest = make_coords(&[3, 9]);
assert_eq!(state.find_next_hop(&dest), None);
}
#[test]
fn test_find_next_hop_no_peers() {
let state = make_tree_state(5, &[5, 0]);
let dest = make_coords(&[3, 0]);
assert_eq!(state.find_next_hop(&dest), None);
}
#[test]
fn test_find_next_hop_local_minimum() {
// Tree: 0 (root) <- 5 (us), 5 <- 8
// Routing to node 3 at [3, 0]. Our distance = 2.
// Peer 8's distance = 4 (8→5→0→3 but via coords: [8,5,0] to [3,0] = 3).
// Actually: lca of [8,5,0] and [3,0] is root 0 at depth 0.
// dist = (2-0) + (1-0) = 3. Our dist = (1-0) + (1-0) = 2.
// Peer is farther, so no hop.
let mut state = make_tree_state(5, &[5, 0]);
add_peer(&mut state, 8, &[8, 5, 0]);
let dest = make_coords(&[3, 0]);
assert_eq!(state.find_next_hop(&dest), None);
}
#[test]
fn test_find_next_hop_best_of_multiple() {
// Tree: 0 (root) <- 1 <- 5 (us), 1 <- 3 <- 7
// Dest is node 7 at [7, 3, 1, 0].
// Peer 1 coords [1, 0]: dist to dest = 0 + 2 = 2
// Peer 3 coords [3, 1, 0]: dist to dest = 0 + 1 = 1
// Our coords [5, 1, 0]: dist to dest = 1 + 2 = 3
// Peer 3 is closest. Should pick 3.
let mut state = make_tree_state(5, &[5, 1, 0]);
add_peer(&mut state, 1, &[1, 0]);
add_peer(&mut state, 3, &[3, 1, 0]);
let dest = make_coords(&[7, 3, 1, 0]);
assert_eq!(state.find_next_hop(&dest), Some(make_node_addr(3)));
}
}