The speedup assertion depends on AVAILABLE parallelism, which a
full-suite run (the pre-push hook) can eat — it flaked at 1.14x under
load. Now retries up to 3 measurement passes and enforces a hard 1.05x
floor that a genuine serialization regression (the per-event-async
version measured 0.94x) can never pass, warning instead of failing in
the noise band between floor and the 1.5x target.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Replaces the decoder's spin lock with lock-free concurrent maps so the
hot per-frame duplicate check never serializes across the pool's relay
consumer coroutines. New minimal ConcurrentHashCache expect/actual
(get/put/size/clear) following LargeCache's per-platform choices:
ConcurrentHashMap on JVM/Android, CacheMap on Apple, copy-on-write on
the CI-only Linux target. Counters become atomics; the generational
rotation keeps its deliberately-tolerated benign races, now documented
per failure mode (each is at worst a redundant re-parse, never a wrong
message).
CachingEventDecoderConcurrencyTest hammers one shared decoder from 8
threads with 80k duplicate-heavy frames and capacity 256 (rotations
fire constantly): zero wrong messages, exact parsed+reused accounting.
The 7 scan-safety tests and DedupDecodeBenchmark's enforced speedup
pass unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Deliberate design decision reversing the 4096-frame receive bound:
1. The remote infrastructure isn't ours — TCP backpressure parks the
backlog in the RELAY's outbound buffers. A client should release the
relay from its duties as fast as it can send and own the buffering
itself.
2. The app holds 2000+ simultaneous relay connections; a bounded buffer
under a slow consumer blocks OkHttp reader threads, and at that
connection count blocked readers are a thread-starvation hazard far
worse than the heap growth they prevent.
The UNLIMITED channels now carry an explicit do-not-bound comment with
this rationale, and the slow-consumer risk is addressed from the other
side: keep the consumer faster than any relay's send rate
(CachingEventDecoder, ParallelEventVerifier, PoolRequests sharding).
BoundedReceiveBufferTest removed with the bound it tested; plan doc
records the decision.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Two server bugs masking each other made a single giant REQ crawl and
then wedge:
LiveEventStore's historical-replay dedup used an immutable Set under an
AtomicReference with copy-on-add — set + id copies the whole set per
streamed event, so large replays were accidentally O(n²) (100k-event
REQ: ~700 events/s, degrading as the response grew). Replaced with a
spin-lock-guarded mutable HashSet (same threads, single contains/add
per critical section).
Fixing that unmasked WebSocketSessionPump's slow-client policy: a fast
replay instantly overflowed the 8192-frame cap — which conflated 'slow
client' with 'replay outruns the socket writer', normal for bulk — and
the 'drop' only closed the internal queue, leaving the socket half-dead
(no EOSE, no close frame, tail silently missing: the likely cause of
the benchmark's 99,998/100,000). Producers are now paced against a full
backlog (bounded blocking wait, consistent with the documented ingest
fanout behavior) and only a client still behind after 30s is dropped,
by actually cancelling the socket.
GiantReqStreamTest guards the regression: 20k-event REQ pre-fix 8.4s
(~2.4k events/s), post-fix 0.7s (~27k events/s), all events + EOSE
delivered. 96 geode tests and the quartz relay/server suites pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
The per-connection-wall test prints the negotiated
Sec-WebSocket-Extensions header; against strfry OkHttp negotiates
permessage-deflate (client_no_context_takeover) out of the box, closing
the 'is compression actually on?' question from the optimization list —
it is, no change needed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
One reconcile feeds by-id download batches to N clients (one socket
each) x reqsPerClient workers; reconcile windows also round-robin
across the connections, since a single connection produced need-ids at
only ~9k/s on a 2.6M corpus and starved the downloads. Events funnel
through a bounded channel to a single consumer (exact maxEvents,
single-threaded onEvent); all stages backpressure; localEntries diffing
and have-counting match negentropyReconcile. reconcileWindows became
multi-client internally; single-client paths pass listOf(this).
Production shootout (same-run pairs, 100k cap): +18% to +64% over the
tuned single client, capped by the relay's server-side reconcile id
production rather than download parallelism (which the by-id matrix
shows scales 2.7x with connections). 4 in-process multi-client tests;
18 negentropy tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Client-side mirror of IngestQueue.parallelVerify: submit() is a cheap
bounded-channel send from the relay consumer coroutine; a drain loop
batches greedily (up to 256) and fans each batch across
Dispatchers.Default in core-sized chunks, dispatching callbacks in
submission order. preVerified short-circuits already-trusted ids; the
bounded channel backpressures the socket instead of growing heap.
Batch/chunk sizing is measurement-driven: per-event async cost ~40us of
scheduling each (swallowing the gain), and the per-batch join barrier
at 64 still cost half (64 -> 1.2x, 256 -> 2.2x, 1024 -> 3.2x on 4
cores). ParallelVerifyBenchmark (fresh signed events per pass — Event
caches derived state after first verify, so passes must not share
instances) measures 1.8x vs sequential with an enforced >=1.5x
assertion. 4 correctness tests cover valid/tampered routing, ordering,
preVerified and callback-crash resilience.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
The reader-thread-to-consumer channel in BasicOkHttpWebSocket and the
app's OkHttpWebSocket was UNLIMITED: a consumer slower than the socket
accumulated frame Strings without bound (gigabytes over a multi-million
event download). Now capped at 4096 frames — when full, OkHttp's reader
thread blocks and TCP flow control pushes back on the relay instead of
the heap. The trade-off (a blocked reader delays PING/PONG handling) is
documented on the constant; the bound is deep enough that only a
pathologically slow consumer hits it.
BoundedReceiveBufferTest forces sustained backpressure (8-frame buffer,
sleeping consumer, real socket to a local geode relay) and asserts all
events plus EOSE arrive in order with no drops or deadlock.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
BasicRelayClient's decode step becomes a pluggable MessageDecoder
(default unchanged). The opt-in CachingEventDecoder scans EVENT frames
for their id (~0.3us, JSON-escape-safe so embedded event JSON in repost
content cannot confuse it; any irregularity falls back to full parse)
and on a cache hit synthesizes the EventMessage from the already-parsed
Event with the frame's own subId — every subscription still gets its
delivery and per-relay bookkeeping is unchanged; only the redundant
parse is skipped. Production traffic measured 14-57% duplicate frames.
DedupDecodeBenchmark (60k frames, 67% dups): 10.0us/frame full parse vs
1.5us/frame cached — 6.5x, with an in-benchmark assertion so the gain
is enforced. 7 scan-safety unit tests in CachingEventDecoderTest.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
The single global spin lock serialized every EVENT frame from every
relay and measured negative scaling (4 concurrent relay consumers
pushed 3.6M deliveries/s aggregate vs 11.1M for one thread alone). The
lock now lives in RequestSubscriptionState — one per subscription —
since all compound mutations are per-subId and different subs share no
wire state. decideCommandLocked takes the state instance to avoid
re-entering the non-reentrant lock; all-subs iterations lock one sub at
a time; withLock is inline to keep the hot path allocation-free.
DispatchStageBenchmark (PoolRequests-only, 1 -> 4 feeders): scaling
flips from 0.33x to 3.4-7.3x across runs. PoolRequests concurrency,
NostrClient and negentropy suites pass unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Splits the reconcile out of negentropySync so callers decide how to
load: negentropyReconcile streams needIds (relay has, local lacks —
download) and haveIds (local has, relay lacks — publish) in batchSize
chunks with back-pressure, taking local state as List<IdAndTime> and
slicing it per created_at window on overflow splits; the accumulating
negentropyReconcileIds convenience returns both lists. negentropySync
now delegates to the same window engine.
NegentropySession's primary constructor takes List<IdAndTime> (JVM
erasure forbids a List<Event> overload); the event-list form moved to
NegentropySession.fromEvents, mirroring NegentropyServerSession, with
all call sites migrated.
Adds NostrClientNegentropyReconcileTest (empty local set, partial
overlap both directions, identical sets, batch streaming, since/until
window slicing) — 49 negentropy tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Restructures the sync around the measured bottlenecks: one download
worker pool now spans the whole sync (windows no longer join before the
next reconcile starts), overflow-split windows are reconciled by a
caller-set number of concurrent NEG sessions (reconcileConcurrency,
default 1) from a shared work queue, and the reconcile-to-download
buffer depth is exposed (idBufferBatches). No NIP-11 auto-detection:
peak subscription usage (maxConcurrentReqs + reconcileConcurrency + 1)
is documented and budgeting it against the relay's max_subscriptions is
the caller's call.
All 44 negentropy tests pass unchanged. Production shootout on a 2.6M
corpus, single connection: 3.6k events/s with old-equivalent params,
4.5k/s tuned (12 reqs + 4 reconcilers) — ~82% of the measured ~5.5k/s
per-connection by-id ceiling, vs 1.6k/s for the old implementation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
The relay was backfilled from 33k to 2.6M kind-30382 events between
runs, and the picture changed: on the big corpus one connection
saturates at ~8 in-flight REQs (~5.5k events/s) and the relay tops out
at ~15k events/s around 40 total in-flight — past that, added
concurrency only inflates per-REQ latency. Axis 1 now stops at the
relay's NIP-11 max_subscriptions (20; exceeding it wedged the
connection in the first extended run), cells carry a hard 120s budget
with one retry per batch, and per-REQ latency is computed over
completed batches so partial cells report honestly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Downloads the full 33k kind-30382 corpus from nip85.nosfabrica.com per
cell over a (connections x concurrent REQs) matrix with pre-connected
sockets. One connection scales near-linearly to 16 in-flight REQs
(1,970 -> 30,735 events/s) with no wall at 4, and 1x16 matches 4x4 —
total in-flight REQs is the real variable. Every slow path measured so
far (serial page cursors ~2-3.7k/s, negentropySync 1.6k/s) is a
pipelining/serialization problem: negentropySync starves its download
workers on reconcile cadence and recurses overflow windows
sequentially. Findings and implied negentropySync improvements in the
plan doc.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
A raw no-parse socket and the full quartz stack page the same
production query on one connection at the same ~3.4-3.8k events/s,
proving the single-connection ceiling reported by a user is the
relay's per-connection response cadence rather than the client's
serial parse (which is ~1% busy at that rate). Findings and the
assessment of the proposed parallel-parse + ordered-dispatch pipeline
are in the plan doc.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Measures how to download millions of events from a single relay as fast
as possible, download+parse only: local geode ceilings (giant REQ vs
until-cursor paging vs created_at-sharded connections, quartz stack vs
raw frames), offline per-frame strategies (full parse, parallel parse,
id-scan for raw archiving), and a production case syncing kind 30382
from nip85.nosfabrica.com via NIP-77 negentropy against plain paging.
Headline results in the plan doc: paging beats giant REQs 26x (which
also dropped frames), created_at sharding stacks ~2x on top, parse is
2-5% of the budget, and negentropy is 2.4x slower than paging for a
cold download (its win is incremental re-sync).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Offline microbenchmark of NostrClient's dispatch stage — PoolRequests
state machine, listener fan-out, id dedup and handoff to a verify
stage — under 1 and 4 concurrent relay feeders. Key results: ~100ns per
message uncontended, but negative scaling under concurrency (the
PoolRequests busy-wait spin lock makes 4 feeders slower in aggregate
than 1), per-event channel handoff costs ~180ns vs a free 64-batch,
and early dedup before the locked path gives 2.7-4x aggregate
throughput at production duplicate factors. Findings appended to the
receiver-perf plan doc.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Gated JVM test (-PprodRelayBench=1) that connects to live relays with
realistic filters and measures per-relay queue delay, processing time,
consumer busy fraction, EOSE latency and duplicate rates, comparing the
current inline verification against a parallel verify stage, plus
offline single-thread parse/verify ceilings on captured frames.
Findings from a first run are written up in
quartz/plans/2026-07-02-nostrclient-receiver-perf.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018saXqYfAa3RvSJoDXK591R
Part C of the dispatchers/thread-caps audit. Both LnurlEndpointCache and
DesktopCachedRichTextParser were bounded caches backed by a LinkedHashMap
behind a single monitor (@Synchronized / Collections.synchronizedMap with
accessOrder). An access-order map structurally mutates on get, so every
read took the lock — serializing all readers on paths that are hot
(kind-9735 zap-receipt validation; feed rich-text rendering).
Add ConcurrentLruCache<K, V> in quartz utils: storage is a
ConcurrentHashMap so get is lock-free; writes + eviction run under a small
write lock that is off the read path. Eviction is least-recently-put order
(get does not refresh recency) — exactly what LnurlEndpointCache already
did, and fine for the deterministic rich-text parse cache.
Point both caches at the shared helper. Covered by a new
ConcurrentLruCacheTest (round-trip, eviction order, re-put recency
refresh, get-does-not-refresh, clear, and a concurrent size-bound smoke
test); the existing LnurlEndpointCacheTest still passes unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ANuUziXKRafSTBxbh4SMoq
Part of the dispatchers/thread-caps audit. Two low-risk fixes that remove
thread-blocking work from paths hit on every event / every packet:
- LargeCache (iOS actual): drop the runBlocking wrapper around
createIfAbsent. The block contained only synchronous CacheMap ops (the
same get/put getOrCreate already calls without runBlocking), so it was
pure dispatcher-blocking overhead on the per-event ingest path. Now
mirrors the JVM actual's plain-function shape.
- QUIC JCA AEADs (AES-GCM + ChaCha20-Poly1305): split the single
`synchronized(this)` monitor into disjoint encryptLock / decryptLock.
seal-family touches only encryptCipher + recentEncryptNonces; open-family
touches only decryptCipher, so a connection's send loop and read loop no
longer serialize against each other through crypto on every packet. The
documented defence-in-depth against cross-coroutine Cipher corruption is
preserved per direction.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ANuUziXKRafSTBxbh4SMoq
printStackTrace() dumps straight to stderr, bypassing both Log.minLevel
and the consumer's Log.sink — the very thing the LogSink work exists to
control. Migrate the five production call sites:
- Lud06: drop two printStackTrace() calls that sat directly above an
existing Log.w(..., t) carrying the same throwable (pure duplication).
- ElectrumXClient: the swallowed-lookup catch said "Log but don't crash"
yet used printStackTrace(); route it through Log.w with context.
- OpenTimestamps: log the swallowed merge failure via Log.w; drop the
print-then-rethrow (the rethrown exception already carries the trace).
Socket-protocol writer.println(...) and README/KDoc println examples are
left as-is — they are wire I/O and documentation, not logging.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EK3TrDkP1EXj1d62oKJMdc
Quartz already funnels every diagnostic through the `Log` facade, but the
sink was hardcoded per platform (android.util.Log / System.err / NSLog /
println), so a consuming app couldn't route Quartz logs into its own stack
(Timber, SLF4J, Crashlytics, a file, a test buffer, or /dev/null).
Add a `LogSink` fun interface and a replaceable `Log.sink`, defaulting to
`PlatformLogSink` which reproduces the historical per-platform behavior.
All ~225 call sites and the `Log.*` signatures are unchanged; the lazy
`() -> String` overloads still short-circuit on `minLevel` before the
lambda runs, preserving the allocation-free fast path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EK3TrDkP1EXj1d62oKJMdc
Fold the full-text-search on/off switch into `IndexingStrategy` as
`indexFullTextSearch` (default `true`) instead of a separate top-level
`enableFullTextSearch` constructor param on `EventStore`/`SQLiteEventStore`.
`IndexingStrategy` is already the single place that decides which indexes
the store builds — every field is a per-index toggle with a size/speed
tradeoff, and `QueryBuilder` already receives it — so FTS, being just
another index, belongs there rather than split across two config surfaces.
Behaviour is unchanged: the module's no-op path and the QueryBuilder
"search matches nothing" guards now read the flag via the strategy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjzUpY8H31c7ux669zytWg
Add an `enableFullTextSearch` flag (default `true`) to `EventStore` and
`SQLiteEventStore` so deployments that never serve NIP-50 search from
SQLite — e.g. a relay that offloads search to an external engine like
Vespa — can skip the full-text-search write cost.
When disabled:
- `FullTextSearchModule` becomes an inert no-op: the `event_fts` virtual
table and its `fts_foreign_key` delete trigger are never created,
inserts skip `indexableContent()` + tokenization, and both reindex
entry points return immediately.
- `QueryBuilder` short-circuits any query/count/delete filter carrying a
non-empty `search` term to a "matches nothing" result (an empty-string
search still imposes no constraint), so no SQL ever references the
absent `event_fts` table. In a multi-filter union the search branch
contributes nothing while the other filters resolve normally.
Everything else (replaceable/addressable handling, deletions,
expirations, right-to-vanish, negentropy) is unchanged, and the default
keeps FTS on for existing callers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjzUpY8H31c7ux669zytWg
Kotlin/Native (the iOS test target) rejects commas in backtick function
names, so test-quartz-ios failed to compile even though jvmTest — which
allows them — passed. Rename the two offending tests. No behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JGa1EM5KWyDo1o5Yr6sS18
Relay operators wiring up a Quartz-based relay had to hand-write the
NIP-11 document as a large JSON string, which is error-prone and drifts
from the model. Add a DSL builder so they can describe the document in
Kotlin instead:
val info = relayInformation {
name = "sot"
description = "NIP-50 profile search ranked by Nostr web-of-trust"
software = "https://github.com/vitorpamplona/sot"
version = "0.1"
supports(1, 11, 42, 50)
}
call.respondText(info.toJson(), ContentType.parse(Nip11RelayInformation.CONTENT_TYPE))
The builder covers every field, with nested `limitation { }` / `fees { }`
DSLs, repeatable list helpers, a `retention(...)` entry adder, and a
`limitation(RelayLimits)` overload that advertises exactly the limits the
relay enforces so the two can't drift.
Also fix FlexibleIntListSerializer to emit numeric `supported_nips` as
JSON integers ([1,11,42,50]) instead of quoted strings, matching the
NIP-11 spec; non-numeric ids still fall back to strings. Geode's default
document now builds via the DSL.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C64Yy2d3na7Y28u7GRrHMX
Introduce OptionalAuthPolicy, a relay-server policy that runs the full
NIP-42 challenge/verify handshake — emitting the AUTH challenge on connect
and recording verified pubkeys into the connection scope — but never
requires it: EVENT, REQ, and COUNT are always accepted, so clients that
ignore the challenge keep working.
It subclasses FullAuthPolicy and only relaxes the EVENT/REQ/COUNT gates, so
the authorize() hook and per-connection authenticatedUsers set behave
identically; downstream policies can still gate or rewrite on caller
identity. Wire it into geode via an optional_auth config option and a
--optional-auth CLI flag (ignored when require_auth/--auth is set, since
mandatory AUTH already sends the challenge).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BJWy4dBBYLbqthhLvbcZh7
Four Kotlin sources embedded literal NUL (0x00) bytes, used as string
separators and as literal characters in KDoc comments. A NUL is valid
UTF-8 (U+0000) so decoders accept it, but SonarScanner flags an embedded
NUL in a text source as a file-encoding problem, and git tracked these
files as binary.
New KDoc blocks on isHex/isHex64 already state the "~47ns" and
"~30% faster" perf notes, so the trailing EOL comments between the
KDoc and the function now trip ktlint's standard:no-consecutive-comments
rule ("an EOL comment may not be preceded by a KDoc"). Remove them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BawgXifvcPidMMqJ719Ka2
When a Podcasting-2.0 person's href points at an npub/nprofile (bare,
nostr: URI, or an njump-style link), upgrade the free-text credit to a real
Nostr profile: the standard ClickableUserPicture + UsernameDisplay, tappable
through to the profile. Plain web links keep the free-text card with the
default profile-image loader.
- quartz: PodcastPerson.nostrPubKey() resolves href → pubkey via Nip19Parser
(npub/nprofile only). Covered by PodcastPersonSoundbiteTest.
- UI: PodcastPeople branches per person — LoadUser + standard profile
components for nostr identities, free-text card otherwise — sharing one
card scaffold so both look identical in the Hosts & Guests strip.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JGa1EM5KWyDo1o5Yr6sS18
Same discoverability treatment as the Hex helpers: KDoc on the reusable,
heavily-used primitives external integrators and AIs kept missing, plus
skill coverage.
- TimeUtils: object + key-fn KDoc emphasizing Unix *seconds* (created_at)
vs the millisecond nowMillis() exception.
- RandomInstance: object + per-fn KDoc (secure random; use over kotlin.random).
- sha256(): KDoc pointing event-id work to EventHasher.
- EventHasher: object + fn KDoc on canonical id serialization / verification.
- StringUtils: KDoc on the allocation-free case-insensitive matchers + DualCase.
- Bech32: "use NIP-19 helpers unless you need a custom prefix" pointer; KDoc
on bechToBytes.
- quartz-integration skill: new "Everyday utilities" section (time/random/
hashing/bech32/base64) + quick-reference rows.
- nostr-expert skill: "Core Utilities" section.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011HM5uueF4a17umGpjC8wcz
AI agents integrating Quartz were re-implementing hex encoding or pulling
in third-party codecs because the built-in helpers weren't discoverable.
- Add KDoc to the `HexKey` typealias, its extension functions
(`toHexKey`/`hexToByteArray`/`hexToByteArrayOrNull`/`isValid`) and the
`PUBKEY_LENGTH`/`EVENT_ID_LENGTH` constants.
- Add KDoc to the `Hex` object and its public API
(`isHex`/`isHex64`/`decode`/`encode`/`isEqual`).
- Add a dedicated "Hex utilities" section + quick-reference rows to the
quartz-integration skill, and fix the wrong `HexKey.decodeHex(hex)`
snippet (no such API) to the real `hex.hexToByteArray()`.
- Add a "Hex Encoding" section to the nostr-expert skill.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011HM5uueF4a17umGpjC8wcz
Adds two Podcasting-2.0 features to the podcast stack:
Persons (podcast:person) — hosts/guests as free-text credits with role,
avatar, and link (not necessarily Nostr users):
- quartz: PodcastPerson model; PersonTag ["person", name, role, img, href]
on kind 30054 episodes; a persons[] array in the kind 30078 show JSON.
Exposed via PodcastEpisode.episodePersons() / PodcastShow.showPersons().
- UI: PodcastPeople — a "Hosts & Guests" avatar strip (robohash fallback,
tap opens href), shown on the episode card and the show header.
Soundbites (podcast:soundbite) — highlight clips:
- quartz: PodcastSoundbite model; SoundbiteTag ["soundbite", start, dur,
title?] on episodes; PodcastEpisode.episodeSoundbites().
- UI: PodcastSoundbites — "jump to the good part" chips under the audio
player that seek the live media controller to the clip's start.
Both parse leniently, round-trip through build(), and are covered by
PodcastPersonSoundbiteTest. NIP-F4 returns empty for both (no such tags),
so nothing renders there.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JGa1EM5KWyDo1o5Yr6sS18
IdleClock.bump() is called for every message the relay sends — the connection
listener bumps it per event, so a multi-million-event download bumped it millions
of times. It stored a ValueTimeMark into an AtomicReference, and since the value
class boxes when used as a generic type argument, every bump allocated a heap
object. That is needless GC pressure on the hottest path (and battery/jank on
Android).
Replace it with a single monotonic base mark taken once (stored unboxed) plus a
@Volatile Long of nanos-since-start updated on each bump — zero allocation per
bump, and only visibility (not atomicity) is needed since each relay's bumps come
from its single reader thread and the driver only reads. Behavior is unchanged;
negentropy + concurrency suites pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JmSyzdmKyiz3pPxUZ8Mg8Z
The fixed `timeoutMs` (default 30s) applied to every reconcile round, but the
FIRST round on a large relay is a legitimate long silence while the relay builds
its whole negentropy fingerprint — observed at ~63-68s for a 3.5M-event kind:0
set on a real relay. So the old default spuriously failed big syncs with
UNAVAILABLE ("reconcile round timed out"), even though the relay was working fine
and would have answered seconds later.
Replace it with `idleTimeoutMs` (default 120s): the maximum time the relay may go
COMPLETELY SILENT before giving up. It resets on every message the relay sends —
each NIP-77 round and every download EOSE/event — and on connect, so a genuinely
slow but progressing sync runs for as long as it needs; only true silence trips
it. Because the watchdog is fed by a connection-level listener that sees all of
the relay's traffic, download activity extends the reconcile deadline and vice
versa. `idleTimeoutMs = 0` disables it entirely (run until the socket drops); the
initial connect and each download batch keep their own finite bounds so an
unreachable relay or a single stuck batch still can't hang the pipeline.
Liveness of a dead/half-open socket does not depend on this: the WebSocket
keep-alive (ping/pong) detects it and the disconnect is already turned into a
clean NEG-ERR abort.
Verified against wss://wot.grapevine.network: the first round took 68.4s (the old
30s default would have thrown UNAVAILABLE) and the sync sailed through it under
the 120s idle window. Negentropy unit suite passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JmSyzdmKyiz3pPxUZ8Mg8Z
Revert the fresh-subId-per-page workaround (ed5c25e2) now that the underlying
double-REQ race is fixed at the root in PoolRequests. Relays cap the number of
concurrent subscriptions per connection, so a single reused id — opened per page
with the page's `until`, closed before the next page — keeps the whole download
to one subscription slot instead of churning through a distinct id each page.
Safe because the pool now serializes the "send a REQ" decision: after a page's
EOSE, the auto-resend and the loop's unsubscribe+resubscribe can no longer both
fire a REQ for the same id (guarded by PoolRequestsConcurrencyTest). Unit suites
(negentropy paging scenarios, subscriptions) pass; full-scale real-relay
verification to follow before push.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JmSyzdmKyiz3pPxUZ8Mg8Z
A subscription id is driven from two threads at once: the app thread (the
subscribe/unsubscribe path) and every relay's socket-reader thread (an EOSE
that triggers an auto-resend). Both read the subscription state and both can
decide "the filters changed, send a REQ", but the decision (read state) and the
send (mark state SENT in onSent) were not atomic. So the reader could observe
the pre-send state — filters still on the previous value — while the app had
already moved the desired filters forward, and both would send a REQ for the
same sub id.
Two REQs on one id race on the wire: the relay answers with duplicate EOSEs and
events, or — if a CLOSE interleaves — an empty result that silently truncates a
paged download. This is what intermittently broke fetchAllPages on large sets
(fixed at the call site in ed5c25e2 by using a fresh sub id per page); this
commit fixes the underlying race in the relay-client layer, which could equally
corrupt any subscription that spans multiple relays (several reader threads
mutate the same RequestSubscriptionState maps concurrently).
The fix:
- Add a tiny non-reentrant spin lock (withStateLock, same AtomicBoolean
primitive BasicRelayClient uses) guarding every access to the subscription
state machine. Listener callbacks and socket sends stay OUTSIDE the lock —
they re-enter this class via onSent, so holding it across them would deadlock.
- Fold the send decision into decideCommandLocked, which runs under the lock and
pre-marks the state SENT (+ filters) the moment it decides to send a REQ. A
concurrent decider then sees SENT/updated filters and declines, so exactly one
REQ is ever produced.
Verified with a deterministic A/B repro that pins the exact interleaving open:
pre-fix 300/300 episodes produced a duplicate REQ; post-fix 0/300 (max one REQ
per episode). Kept as PoolRequestsConcurrencyTest. Full relay test suite passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JmSyzdmKyiz3pPxUZ8Mg8Z
fetchAllPages reused a single subscription id across all pages
(unsubscribe + immediately re-subscribe the same id). On a real relay that
caps REQ results, the rapid same-id CLOSE→REQ races on the wire: in-flight
events from the previous page's REQ bleed into the next page's listener.
Those stale events carry a created_at above the freshly-lowered `until`, so
`match()` rejects them, the page ends with pageCount == 0, and the whole
loop breaks — silently truncating the download.
Observed against wss://wot.grapevine.network: a full kind:0 download (~3.55M
events, per a concurrent negentropy sync) stopped at 89,500. A controlled
diagnosis paging the same data with a fresh subId per page vs a shared subId
reproduced it exactly: shared stalled at ~95k with in-page duplicates and
events above `until`; fresh advanced cleanly with no duplicates. After the
fix, the real-relay fetchAllPages sails past the old stall (100k+ and
counting).
Fix: allocate the subId inside the paging loop so each page is an
independent subscription.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JmSyzdmKyiz3pPxUZ8Mg8Z
Rework the negentropy download path from "reconcile fully → then download"
into a single back-pressured streaming pipeline so peak memory is independent
of the window size — built for multi-million-event syncs.
- reconcileStreaming drives the NIP-77 rounds directly (instead of via
NegentropyManager) and hands each round's ids to a bounded id-queue *before*
acking the next round, so the relay's id stream is paced to the downloader.
- Ids flow id-queue → bounded download worker pool → bounded delivery channel;
a slow consumer back-pressures the whole chain. The full id list is never
materialised.
- Drop the global event-dedup set (was O(set) ~ hundreds of MB at 4M): NIP-77
yields a distinct id set, so each event is requested once. Keep only a tiny
per-batch dedup (bounded by fetchBatch) to absorb a relay replaying a REQ.
- Pin the relay with a never-matching keep-alive subscription for the sync's
duration: a NEG-OPEN isn't a REQ, so during a reconcile round the pool would
otherwise see the relay as unwanted and disconnect it mid-sync.
- Document that timeoutMs must accommodate the relay's first-frame snapshot
latency on huge sets (a real strfry took ~73s for an unbounded kind:0 set).
Validated against wss://wot.grapevine.network: streamed 30k kind:0 events with
heap bounded at ~30-90 MB (not growing with the set) and zero duplicates. New
unit test forces many small reconcile frames to exercise the multi-round /
back-pressure path; existing windowing/cap/fallback tests still green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JmSyzdmKyiz3pPxUZ8Mg8Z
Vitor PamplonaandGitHub
Claude
davotoula