Files
amethyst/quartz/src
Claude 54ad837559 feat(graperank): TCP reachability pre-probe + .onion skip to cull the dead graveyard
At hop-8 the crawl dials into thousands of dead relay hints from old accounts.
Most fail slowly: a silently-dropping host has no RST to receive, so the WS
connect just hangs to the 7s connectTimeout. First-strike eviction pays that once
per host, but with ~3,000 dead hosts that's ~80s of connect-setup serialized
through the dispatcher.

Add a background reachability culler: a cheap raw TCP connect (one round trip, 2s
timeout) over the learned relays COLD-TAIL FIRST, dropping the unreachable ones
into deadHosts before the WS path pays its 7s. The key property is that a tight
TCP timeout is safe where a tight WS timeout is not — a busy-but-alive relay
accepts the SYN instantly at the kernel level and only stalls at the app layer, so
the probe separates "unreachable" from "slow" and never false-kills the busy. It
only ever marks dead and probes each authority once; a host the WS path already
resolved (isDead) is skipped, and a live host passes the probe, so the WS verdict
always wins. Injected as an optional Config.reachabilityProbe (JVM: java.net.Socket
in the CLI; --no-probe disables); writeRelayFreq becomes concurrent so the culler
can read it while routeByOutbox writes.

Also: when there's no Tor transport (Config.torEnabled=false), isDead skips every
.onion relay on sight — no socket, no wasted connect.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
2026-07-08 22:38:46 +00:00
..