From ed5c25e2b1d2b1768ead2d3a56810a2a23d49162 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 1 Jul 2026 00:42:33 +0000 Subject: [PATCH] fix(quartz): fresh subId per page in fetchAllPages (was truncating large results) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit fetchAllPages reused a single subscription id across all pages (unsubscribe + immediately re-subscribe the same id). On a real relay that caps REQ results, the rapid same-id CLOSE→REQ races on the wire: in-flight events from the previous page's REQ bleed into the next page's listener. Those stale events carry a created_at above the freshly-lowered `until`, so `match()` rejects them, the page ends with pageCount == 0, and the whole loop breaks — silently truncating the download. Observed against wss://wot.grapevine.network: a full kind:0 download (~3.55M events, per a concurrent negentropy sync) stopped at 89,500. A controlled diagnosis paging the same data with a fresh subId per page vs a shared subId reproduced it exactly: shared stalled at ~95k with in-page duplicates and events above `until`; fresh advanced cleanly with no duplicates. After the fix, the real-relay fetchAllPages sails past the old stall (100k+ and counting). Fix: allocate the subId inside the paging loop so each page is an independent subscription. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JmSyzdmKyiz3pPxUZ8Mg8Z --- .../client/accessories/NostrClientFetchAllPagesExt.kt | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip01Core/relay/client/accessories/NostrClientFetchAllPagesExt.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip01Core/relay/client/accessories/NostrClientFetchAllPagesExt.kt index 8bacb7b383..22cbcdd495 100644 --- a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip01Core/relay/client/accessories/NostrClientFetchAllPagesExt.kt +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip01Core/relay/client/accessories/NostrClientFetchAllPagesExt.kt @@ -64,11 +64,17 @@ suspend fun INostrClient.fetchAllPages( // Track how many matching events each filter has received so far. val matchCountPerFilter = IntArray(filters.size) - val subId = newSubId() - while (true) { coroutineContext.ensureActive() + // A fresh subscription id per page. Reusing one id across pages + // (unsubscribe + immediately re-subscribe the same id) races on the wire: + // in-flight events from the previous page's REQ bleed into the next page's + // listener. Those stale events carry a `created_at` above the new `until`, + // so `match()` rejects them, the page ends with `pageCount == 0`, and the + // whole download terminates early — silently truncating large results. + val subId = newSubId() + val pagedFilters = if (until == null) { filters