Improve discovery protocol: bloom-guided tree routing with fallback

Replace discovery flooding with bloom-filter-guided tree routing:
lookups sent only to tree peers (parent + children) whose bloom
filter contains the target. If no tree peer matches, fall back to
non-tree peers with bloom matches before dropping the request. This
produces single-path forwarding through the spanning tree (90%
traffic reduction) while recovering from dead ends caused by stale
bloom filters, tree restructuring, or transit node failures.

Remove visited bloom filter from LookupRequest wire format (-257
bytes per request). Tree routing is inherently loop-free; request_id
dedup handles edge cases during tree restructuring. Add response-
forwarded flag to prevent response routing loops from convergent
request paths.

Add originator-side exponential backoff (30s base, 300s cap) after
lookup timeouts and bloom misses. Backoff resets on topology changes
(parent switch, new peer, first RTT, reconnection).

Add transit-side per-target rate limiting (2s minimum interval) for
forwarded lookups as defense-in-depth.

Add discovery retry within the timeout window (default: send at T=0,
retry at T=5s, fail at T=10s) to compensate for single-path fragility.
Lookups with zero eligible tree peers fail immediately.

Improve discovery logging: promote key events to info (initiation,
success, timeout with failure count). Add debug logging for dedup,
pending packet retry, backoff suppression, forward rate limiting,
and backoff reset.

New config: discovery.backoff_base_secs, backoff_max_secs,
forward_min_interval_secs, retry_interval_secs, max_attempts.
New stats: req_backoff_suppressed, req_forward_rate_limited,
req_bloom_miss, req_no_tree_peer, req_fallback_forwarded.
Removed: req_already_visited, visited bloom filter.
This commit is contained in:
Johnathan Corgan
2026-03-22 20:02:00 +00:00
parent 7a643a9ac3
commit 6aff490251
14 changed files with 805 additions and 206 deletions
+6 -2
View File
@@ -55,6 +55,7 @@ class AnalysisResult:
discovery_timeout: list[tuple[str, str]] = field(default_factory=list)
discovery_retry: list[tuple[str, str]] = field(default_factory=list)
discovery_no_tree_peer: list[tuple[str, str]] = field(default_factory=list)
discovery_fallback: list[tuple[str, str]] = field(default_factory=list)
discovery_trigger: list[tuple[str, str]] = field(default_factory=list)
def summary(self) -> str:
@@ -85,6 +86,7 @@ class AnalysisResult:
f"Backoff suppressed: {len(self.discovery_backoff)}",
f"Deduplicated: {len(self.discovery_dedup)}",
f"No tree peer: {len(self.discovery_no_tree_peer)}",
f"Non-tree fallback: {len(self.discovery_fallback)}",
f"Timed out: {len(self.discovery_timeout)}",
]
@@ -172,7 +174,7 @@ def _analyze_lines(result: AnalysisResult, source: str, log_text: str):
if "Rekey cutover complete" in line or "FSP rekey cutover complete" in line:
result.rekey_cutovers.append((source, line))
# Discovery
if "Initiating LookupRequest" in line:
if "Initiating LookupRequest" in line or "Discovery lookup initiated" in line:
result.discovery_initiated.append((source, line))
if "proof verified, caching route" in line:
result.discovery_succeeded.append((source, line))
@@ -186,8 +188,10 @@ def _analyze_lines(result: AnalysisResult, source: str, log_text: str):
result.discovery_timeout.append((source, line))
if "Discovery retry sent" in line:
result.discovery_retry.append((source, line))
if "no tree peers with bloom match" in line:
if "no tree peers with bloom match" in line or "No eligible peers to forward" in line:
result.discovery_no_tree_peer.append((source, line))
if "non-tree fallback" in line:
result.discovery_fallback.append((source, line))
if "Failed to initiate session, trying discovery" in line:
result.discovery_trigger.append((source, line))