Improve discovery protocol: bloom-guided tree routing with fallback

Replace discovery flooding with bloom-filter-guided tree routing:
lookups sent only to tree peers (parent + children) whose bloom
filter contains the target. If no tree peer matches, fall back to
non-tree peers with bloom matches before dropping the request. This
produces single-path forwarding through the spanning tree (90%
traffic reduction) while recovering from dead ends caused by stale
bloom filters, tree restructuring, or transit node failures.

Remove visited bloom filter from LookupRequest wire format (-257
bytes per request). Tree routing is inherently loop-free; request_id
dedup handles edge cases during tree restructuring. Add response-
forwarded flag to prevent response routing loops from convergent
request paths.

Add originator-side exponential backoff (30s base, 300s cap) after
lookup timeouts and bloom misses. Backoff resets on topology changes
(parent switch, new peer, first RTT, reconnection).

Add transit-side per-target rate limiting (2s minimum interval) for
forwarded lookups as defense-in-depth.

Add discovery retry within the timeout window (default: send at T=0,
retry at T=5s, fail at T=10s) to compensate for single-path fragility.
Lookups with zero eligible tree peers fail immediately.

Improve discovery logging: promote key events to info (initiation,
success, timeout with failure count). Add debug logging for dedup,
pending packet retry, backoff suppression, forward rate limiting,
and backoff reset.

New config: discovery.backoff_base_secs, backoff_max_secs,
forward_min_interval_secs, retry_interval_secs, max_attempts.
New stats: req_backoff_suppressed, req_forward_rate_limited,
req_bloom_miss, req_no_tree_peer, req_fallback_forwarded.
Removed: req_already_visited, visited bloom filter.
This commit is contained in:
Johnathan Corgan
2026-03-22 20:02:00 +00:00
parent 7a643a9ac3
commit 6aff490251
14 changed files with 805 additions and 206 deletions
+4
View File
@@ -232,6 +232,7 @@ impl Node {
}
self.tree_state.recompute_coords();
self.coord_cache.clear();
self.reset_discovery_backoff();
self.stats_mut().tree.parent_switched += 1;
self.stats_mut().tree.parent_switches += 1;
@@ -275,6 +276,7 @@ impl Node {
return;
}
self.coord_cache.clear();
self.reset_discovery_backoff();
self.send_tree_announce_to_all().await;
}
return;
@@ -299,6 +301,7 @@ impl Node {
}
self.tree_state.recompute_coords();
self.coord_cache.clear();
self.reset_discovery_backoff();
let new_root = *self.tree_state.root();
let new_depth = self.tree_state.my_coords().depth();
@@ -376,6 +379,7 @@ impl Node {
}
self.tree_state.recompute_coords();
self.coord_cache.clear();
self.reset_discovery_backoff();
self.stats_mut().tree.parent_switched += 1;
self.stats_mut().tree.parent_switches += 1;