feat(graperank): log slow/timed-out relays with their query under --diagnose

Add a --diagnose slow-relay log: every content drain that reaches its terminal
(EOSE or timeout) slower than SLOW_DRAIN_LOG_MS, or times out entirely, is
recorded with the offending relay URL, the failure/EOSE reason, elapsed ms, and
the exact filter shape (kinds + author count + first authors). This lets a human
replay that precise REQ later to understand why the relay lags. Gated on
--diagnose so there is no per-group timing/collection overhead otherwise.

Make the content-drain fan-out configurable via a new --drain-concurrency flag
(Config.drainConcurrency), replacing the DRAIN_CONCURRENCY constant. Default
stays at the validated 24: an A/B at 64 ran ~2x slower with more dead relays
(a higher global fan-out re-floods busy hubs faster than the per-relay demotion
catches up), so the flag is a probe knob, not a speedup. Client WebSocket pings
were also tried and reverted — busy-but-alive relays don't reliably pong while
their query handler runs, so pinging just cut them as dead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWk2ZMrGBSr4WenKgwqmbB
This commit is contained in:
Claude
2026-07-07 19:10:33 +00:00
parent 9eec6323c8
commit c9ee79beba
2 changed files with 37 additions and 9 deletions
@@ -127,6 +127,11 @@ object GrapeRankCommand {
// forces the per-event insert path (baseline); higher amortizes the SQLite
// transaction + writer-mutex cost across the batch.
val insertBatch = args.intFlag("insert-batch", 500)
// How many outbox batches drain in parallel (the worker-pool size). 24 is the
// validated default; higher fan-out re-floods busy hubs faster than the
// per-relay demotion catches up (an A/B at 64 was ~2x slower with MORE dead
// relays), so raise it only to probe specific slow relays.
val drainConcurrency = args.intFlag("drain-concurrency", 24)
val doPublish = args.bool("publish")
// Publish cutoff: only cards with rank >= this are published; existing
// cards for targets below it (or gone from the graph) are retracted. Rank
@@ -185,6 +190,7 @@ object GrapeRankCommand {
timeoutMs = timeoutMs,
diagnose = diagnose,
insertBatchSize = insertBatch,
drainConcurrency = drainConcurrency,
),
log = { System.err.println(it) },
)