mirror of
https://github.com/vitorpamplona/amethyst.git
synced 2026-08-09 08:04:45 +00:00
perf(cli): widen OkHttp dispatcher + tighten connect timeout for the crawl
The crawl's client ran on OkHttp defaults: Dispatcher.maxRequests=64 and a 10s connectTimeout. Every relay WS-upgrade handshake is an async call through that shared dispatcher, so 64 caps the connection-ramp width — and a dead relay squats on a slot for the full connectTimeout, starving live relays queued behind it (observed: only ~150 sockets open at once during an active wave touching hundreds of relays). Raise maxRequests to 256 / maxRequestsPerHost to 16 and drop connectTimeout to 5s so unreachable relays release their slot fast. This is orthogonal to REQ concurrency (bounded per-relay by AdaptiveRelayLimiter on already-open sockets), so it can't trip a relay's REQ rate-limit — it only speeds connection setup. The dispatcher's executor pool grows threads on demand, and FD headroom is ample (4096 limit vs ~150 in use), so the wider cap just lets more short-lived handshakes run at once.
This commit is contained in:
@@ -78,8 +78,10 @@ import kotlinx.coroutines.joinAll
|
||||
import kotlinx.coroutines.launch
|
||||
import kotlinx.coroutines.selects.select
|
||||
import kotlinx.coroutines.withTimeoutOrNull
|
||||
import okhttp3.Dispatcher
|
||||
import okhttp3.OkHttpClient
|
||||
import java.util.concurrent.ConcurrentHashMap
|
||||
import java.util.concurrent.TimeUnit
|
||||
|
||||
/**
|
||||
* Per-invocation wiring. Each CLI run constructs a Context, does its work,
|
||||
@@ -118,7 +120,28 @@ class Context(
|
||||
val identity: Identity,
|
||||
val state: RunState,
|
||||
) : AutoCloseable {
|
||||
private val okhttp = OkHttpClient.Builder().socketFactory(TcpNoDelaySocketFactory).build()
|
||||
private val okhttp =
|
||||
OkHttpClient
|
||||
.Builder()
|
||||
.socketFactory(TcpNoDelaySocketFactory)
|
||||
// The crawl opens WebSockets to thousands of relays. Each WS-upgrade
|
||||
// handshake is an async call through OkHttp's shared Dispatcher, whose
|
||||
// default cap (maxRequests=64) throttles the connection ramp — worse,
|
||||
// a dead relay holds a slot for the whole connectTimeout, starving live
|
||||
// relays queued behind it. Widen the dispatcher so handshakes fan out,
|
||||
// and tighten connectTimeout so an unreachable relay frees its slot
|
||||
// fast. This is orthogonal to REQ concurrency (that runs on already-open
|
||||
// sockets, bounded by AdaptiveRelayLimiter), so it can't trip a relay's
|
||||
// REQ rate-limit — it only speeds connection setup. The executor thread
|
||||
// pool is unbounded on demand, so raising maxRequests just lets more of
|
||||
// those short-lived handshakes proceed at once.
|
||||
.connectTimeout(5, TimeUnit.SECONDS)
|
||||
.dispatcher(
|
||||
Dispatcher().apply {
|
||||
maxRequests = 256
|
||||
maxRequestsPerHost = 16
|
||||
},
|
||||
).build()
|
||||
|
||||
val client: NostrClient =
|
||||
NostrClient(
|
||||
|
||||
Reference in New Issue
Block a user