Per-relay sub cap 32 made the hop-3 crawl 52% slower with slightly fewer
lists (27 relays demoted us vs 5 at cap 16) - the Phase-B plateau is
relay-side service rate, so 16 stays the default and AMY_RELAY_SUB_CAP
remains an experiment hatch. Final validation: cold hop-5 completed in
85.4 min with 203,903 contact lists / 391,549 users / 626,599 events -
the workload that never finished before this branch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zEYRGKF943RgLaHTViJaB
Phase-B throughput plateaus at ~100-150 users/s at every crawl scale while
CPU/FDs/network sit idle, pointing at the per-relay 16-permit gates on the
hot backbone/fallback relays every batch touches. The 16-vs-100 benchmark
that chose the default predates the multithreaded-crawl fix, so expose the
starting cap for A/B runs; the NOTICE/CLOSED demotion ladder still protects
relays that push back.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zEYRGKF943RgLaHTViJaB
Cold hop-3 A/B (fresh store per leg, same observer): 48 workers beat 24
twice at identical contact-list counts (579s->559s and 474s->396s). The
old result that made 64 look 2x slower predated the multithreaded
dispatcher fix - more coroutines on one starved event-loop thread. Plan
doc gains the full A/B table (kept: background-park degating, sweep
dedup, tail overlap, dc48; rejected: 5s fast window).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zEYRGKF943RgLaHTViJaB
Timestamped hop-3 logs split the 373s tail three ways: 222s blocking
convergence on parkedInFlight (whose late lists provably cannot fold once
pending is empty - parked filters only ever asked for in-budget users and
they are all done; the events still stream into the store), 14s of
aggregator recovery, and 137s of report-deletion fetching serialized
behind it. Convergence now breaks as soon as the frontier is empty; the
aggregator pass and report deletions run concurrently; and the aggregator
folds the lateHarvest trickle until one full park window of silence
instead of inheriting every background park's lifetime.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zEYRGKF943RgLaHTViJaB
A saved crawl log couldn't attribute wall time to the finishing tail
(aggregator recovery vs report deletions vs parked drains) without
external timestamps. Prefix every crawl progress line with [t+SSSs].
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zEYRGKF943RgLaHTViJaB
Straggler rounds re-offered the identical missing set to the identical
top-relay backbone each round, re-paying the rotation barriers (~20s of
Phase A per round) for ~zero new lists. Track swept users and only sweep
newcomers - but only once the backbone is at full shard width, so users
swept against round 2's proto-backbone still get a real sweep when the
top-10 exists.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zEYRGKF943RgLaHTViJaB
A hop-3 crawl spent ~500s of its 594s total in the finishing tail waiting
for parkedInFlight to hit zero - and the spike (1791 parked units) was the
fire-and-forget Tier-2 relay-list sweeps, whose whole point was not to
block the crawl. They only gated it because a parked unit persisted its
events at park END, so cancelling early would have lost them.
Fix in two parts: the park loop now persists incrementally (drains + stores
the queued chunk on every activity ping, plus a NonCancellable final drain),
so cancelling a park loses nothing already delivered; and drainGated grew a
background flag - Tier-2 sweep parks (and their capped-page paginations)
no longer count toward parkedInFlight and are simply cancelled at crawl
end. Round-critical parks (outbox drains, sharded sweep, aggregator
recovery) still gate convergence exactly as before. paginateIfCapped takes
(pageSize, oldest) since the incremental path no longer retains the page.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zEYRGKF943RgLaHTViJaB
Once the crawl went multithreaded, the store became the visible Phase-A
bottleneck: harvestFromStore, shardedSweep's missing-check, the Phase-B
consumer, and the aggregator straggler scan each issued one contactsOf
point query per user (~8ms each on a multi-GB store under concurrent
writers) - 100k-user folds burned minutes serially, and the aggregator
pass over 300k discovered users would have burned ~40 on its own. Add
latestContactsFor (one chunked author query per 300 users, newest
created_at per author wins) and route all four paths through it;
harvestFromStore folds chunk-by-chunk so peak memory stays one chunk.
Phase A now folds the store first and shardedSweep trusts `done` instead
of re-checking per author.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zEYRGKF943RgLaHTViJaB
The CLI calls crawl() from runBlocking's single-threaded event loop, so
Phase B's thousands of concurrent drain-unit coroutines (timeout timers,
channels, REQ JSON encoding, signature verifies, SQLite writes) all queued
on ONE thread: timers fired late, batch walls inflated ~5x, and the
--diagnose ticker showed 24/24 workers pinned while completing ~1 user/s
with one core pegged and three idle. It also explains why the old
64-worker A/B ran slower - more coroutines on the same thread. Hop the
whole run onto Dispatchers.IO inside the crawler so every front end gets
real parallelism; IO (not Default) because the store's blocking SQLite
calls must not starve the cores-sized pool.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zEYRGKF943RgLaHTViJaB
Any graperank crawl/probe now creates ~/.amy/operator/ (the reachability
cache signs with the operator monitor key), and listAccounts counted it as
an account - so a single-account user's very first crawl left every later
command failing with "multiple accounts (operator, <name>)". Add it to
RESERVED_NAMES alongside shared/current. Hit for real while network-testing
the connect-storm changes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zEYRGKF943RgLaHTViJaB
Most of a from-scratch crawl's wall clock was connection setup, re-paid
serially: the relay pool tears down a relay's socket ~300ms after its last
drain unsubscribes, OkHttp allowed only 256 concurrent WS handshakes, and
every dial re-ran an uncached blocking getaddrinfo (10-30s per dead domain,
once per per-user path URL of the same host). Open the connections once, in
parallel, and keep them:
- GrapeRankCrawler: the warm-pool trick (never-matching REQ that only holds
the socket) now covers the whole candidate universe instead of the top 20 -
seeded at crawl start from the reachability cache's live set (one parallel
connection storm, Config.knownLiveRelays) and refreshed each round with
newly learned outbox relays, capped by Config.preconnectCap (FD-budget
aware, --preconnect-cap / --no-preconnect).
- CachingDns (quartz jvmAndroid): 10-min positive + negative DNS cache with
in-flight per-host dedup; dead domains fail in microseconds instead of
re-burning resolver timeouts, path URLs of one host resolve once.
- cli Context: dispatcher and pre-connect caps derived from the process's
open-files limit (UnixOperatingSystemMXBean), warning when ulimit is low.
- amy graperank probe: relay census - mass-connects every relay the store
knows (kind:10002 universe deduped per authority + cached verdicts) in
waves, records live/dead with real measured rtt-open into the NIP-66
reachability cache (RelayProber + RelayReachabilityStore.recordProbed),
so the next crawl skips dead relays and waits once for the slow-but-alive.
Single-server limits (FDs, ephemeral ports, DNS, threads, conntrack) and the
design are documented in quartz/plans/2026-07-10-graperank-connect-storm.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zEYRGKF943RgLaHTViJaB
Relay targeting is fully distributed: every feed, loader, finder and
broadcast path builds its own relay set and hands it to the shared
INostrClient. Only the follow-outbox flows and the top-nav feed filters
subtracted the NIP-51 kind:10006 blocked list, so blocked relays still
leaked in through the event/thread loaders (FilterMissingEvents /
FilterMissingAddressables), the user-metadata finder
(pickRelaysToLoadUsers), channel finder, DM targeting, the one-shot
fetch helpers, and the publish path (Account.computeRelayListToBroadcast)
— none of which consulted the blocked set.
Add BlockedRelayFilteringClient, a thin INostrClient decorator that
strips the active account's blocked relays from subscribe, count and
publish right before they reach the pool. Because the one-shot fetch
helpers route through subscribe/count, wrapping the client covers them
too. The blocked set is read per-call so account switches and list
edits apply with nothing to invalidate.
Wire it around the shared app client (blocked set from the logged-in
account) and around the per-account crawl client used by Event Sync and
Cashu discovery. Add commonTest coverage for the filtering, pass-through,
fully-blocked, and per-call-read behaviors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JNMPdC2eGUwTrt3XkQefuf
Activity.dispatchKeyEvent is a public framework hook; lint flags the
override only because androidx.core's intermediate override carries a
library-group @RestrictTo. Scoped to the method so the check stays live
for genuine restricted-API use. Makes :nappletHost:lintDebug pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HLfwhTdf72qFPnmPRYnzqu
Addresses correctness/perf issues found in the crawler + reachability audit:
- deadHosts permanent eviction (#1): an authority that accrued timeoutEvictStrikes
before its first EOSE was evicted forever — clearTimeoutStrikes only zeroed the
counter and could not un-evict, contradicting the "a host that ever produces is
never evicted" invariant. Add a producedHosts set that isDead() consults, so a
proven-productive authority is never treated as dead even if a concurrent strike
from the 24-worker fan-out raced it into deadHosts.
- Parking-disabled event loss (#2): when parking is off (no bgScope, or
parkTimeoutMs <= timeoutMs), a relay that streamed events but didn't EOSE in the
fast window had its buffer dropped without persist() and reported count 0. Drain,
persist, and return those events like the other two branches; strike only when
nothing was delivered.
- Wide-sweep over-narrowing (#4): relayListDiscoverySwept excluded an already-swept
straggler from the wide pass even though the wide net grows each round, so a 10002
hosted only on a later-learned relay was never fetched. Gate the wide pass on the
asked-relay set (wideRelaysSwept) instead: new users get the full net, older
stragglers get only newly-appeared relays, no (user, relay) pair asked twice.
- Onion detection (#10): replace loose relay.url.contains(".onion") with
RelayUrlNormalizer.isOnion() in isDead() and networkTypeOf(), fixing the
foo.onionfake.com false positive and the store/crawler disagreement.
- rtt-open=0 semantics (#9): document that the crawler's reachable records use
rtt-open purely as a liveness flag (0 = latency not probed), not a real 0 ms
measurement, and must not be published as authoritative latency data.
deadHosts is deliberately still NOT persisted to the 24h reachability cache (#8):
a timeout eviction means "too slow under our fan-out this run", not "proven
unreachable", so persisting it would blacklist slow-but-live hubs across runs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
FrameDispatchStats stamped a ValueTimeMark on every relay frame and recorded a
contended atomic per frame in BasicOkHttpWebSocket — the WebSocket layer used by
the whole app, unconditionally, forever — to answer a one-time question that only
graperank --diagnose read. It served its purpose (proved the our-side dispatch
lag is ~200ms mean and the EOSE-wait is dominantly relay-side, so the crawler is
network-bound), but the ongoing per-frame Pair allocation + atomic contention on
every client's relay traffic isn't worth carrying. Revert the channel back to
Channel<String> and delete the stats holder. The diagnose-gated saturation ticker
and per-drain latency breakdown stay — they're crawler-local, off the hot path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
Measured: relays deliver their events in ~0.6s then sit ~4.6s (86% of drain wall)
before sending EOSE — mostly relay-side (our pipeline adds only ~200ms). So instead
of waiting the full 10s fast window then parking, close a drain that has delivered
>=1 event and then gone silent for eoseIdleMs, treating it as complete ("eose-idle").
awaitTerminalOrQuiescent: the idle timer arms only AFTER the first event, so a relay
merely slow to answer still gets the full timeoutMs and is never cut prematurely; a
still-streaming relay keeps resetting the window. eose-idle paginates if the page was
capped and clears timeout strikes (it delivered), but joins notAnswered (no clean
EOSE, so its missing authors are retried elsewhere). Off by default (eoseIdleMs=0),
CLI --eose-idle-ms, so it can be A/B'd against the plain fast window.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
The dedicated frame-dispatch pool (ea1093ad) made dispatch lag WORSE, not better:
mean 200ms→460ms, max 3.5s→5.5s, frames>1s 43k→76k. The pool was sized cores*2
(=8 here) vs Dispatchers.IO's 64 threads, so it cut frame-processing parallelism
~8x. Lesson: the our-side lag is dominated by per-connection serial decode
throughput / thread count, NOT cross-contention with the store's IO writes — the
experiment ruled that hypothesis out. Reverting to shared IO; keep FrameDispatchStats.
EOSE-wait is confirmed dominantly relay-side (200ms our-mean vs ~5s eose-wait).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
Measured on a GrapeRank crawl, frame decode/dispatch (per-connection consumer
coroutines) ran on the shared Dispatchers.IO — the same pool that runs the store's
blocking SQLite inserts. During event floods, frame coroutines queued behind those
inserts: mean 200ms and up to 3.5s of dispatch lag, with 43k frames waiting >1s in
our pipeline. That lag also skews the relay-idle/EOSE timing the crawler reads.
Give frame processing its own daemon thread pool (sized to a small multiple of
cores; decode is light + CPU-bound), shared across all connections. Frame delivery
stays prompt regardless of what the IO pool is doing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
Adds FrameDispatchStats: the lag between a relay frame arriving on the OkHttp
reader thread and our per-connection consumer coroutine (on shared Dispatchers.IO)
pulling it off the channel — pure our-side pipeline delay, relay send-timing
excluded. BasicOkHttpWebSocket stamps arrival before enqueue and records the lag
on dequeue; the crawler resets it at start and dumps it in the --diagnose summary.
Answers whether a drain's 5s gap between the relay's last event and its EOSE is
the relay being slow to SEND eose (low dispatch-lag) or our IO pipeline backing up
so the already-arrived eose frame sits queued (high dispatch-lag).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
Answers "are we resource-bound or waiting on relays" without a profiler:
- progress ticker gains "Nw/CAPw" (drain workers busy vs drainConcurrency) and
"N rl" (rate-limit responses so far) — a rarely-full pool means the producer or
the relays are the limit, not concurrency; a climbing rl count is the external
ceiling that made concurrency 60 backfire.
- crawl-end "latency breakdown": splits each drain's wall into time-to-first-event
vs EOSE-wait-AFTER-the-relay's-last-event, and reports the % of drain wall spent
waiting for EOSE after the relay was already done, how many drains blew the fast
window and parked, and total rate-limit hits. A high EOSE-wait % is the direct
case for a shorter/adaptive fast window over more concurrency.
All gated on config.diagnose; zero cost on a normal run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
Two changes:
1. The reachability flush re-wrote the SEEDED known-dead relays with a fresh
created_at every run, refreshing their TTL without a re-probe — so a relay
marked dead once (and thereafter skipped, never re-dialed) would stay
blacklisted forever as long as crawls kept running, defeating the TTL's
re-probe. Stats.deadRelays now reports only relays actually dialed this run
(deadRelays - knownDeadRelays); seeded records keep their original timestamp
and age out on schedule so the next run re-probes them.
2. Rename GrapeRankDataCrawler -> GrapeRankCrawler (file + all references).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
Wire RelayReachabilityStore into the crawler and the WoT updater so liveness is
shared across procedures and runs instead of each rediscovering dead relays.
- OperatorKeys.monitorKey(): a dedicated machine monitor identity derived from the
operator master (domain "relay-monitor:"), independent of any account — the
30166 records are published under this, not the observer key.
- Context.reachability: a RelayReachabilityStore over the shared store, signed by
the monitor key.
- Crawler: Config.knownDeadRelays seeds deadRelays before the run; Stats now
returns the final dead/live sets. GrapeRankCommand seeds from snapshot().dead
and flushes the crawl's verdicts back via reachability.record().
- Updater: Config.knownDead skips proven-dead relays from the reconcile plan — a
dead relay cannot serve its authors, so reconciling it only burns a timeout.
Live author-advertised relays are always synced.
All behind --no-reachability-cache. TTL'd (24h), so a recovered relay is retried
once its record ages out — a "skip for now", never a permanent ignore, keeping
the outbox rule that every live advertised relay is tried.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
A durable, shareable relay-reachability cache backed by the EventStore as NIP-66
kind:30166 Relay Discovery events, so the crawler, the WoT updater, and future
runs share liveness knowledge instead of each rediscovering dead relays from an
in-memory set wiped at process exit.
- 30166 is addressable by its d-tag (relay URL) → one replaceable status slot per
(monitor, relay), with created_at giving a free TTL.
- Reachable → 30166 with rtt-open; dead → 30166 without (NIP-66 has no explicit
offline field; liveness is inferred from a fresh successful open). Live wins
over dead within the TTL, so third-party monitors' 30166 can be ingested.
- snapshot() loads the fresh set once (not a per-request hot-path query); record()
flushes a run's findings. A relay is only skipped for the TTL, never permanently
— consistent with the outbox rule that every advertised write relay is tried.
Reuses the existing RelayDiscoveryEvent. jvmTest covers record/reload,
live-overrides-dead, TTL expiry, and .onion→Tor network tagging.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
The synchronized intrinsic is only available on JVM/Android, breaking the
iOS (compileKotlinIosSimulatorArm64) build in commonMain. Replace the Any()
lock with a kotlinx.coroutines Mutex + withLock, which is KMP-common and
safe here since all three store methods are already suspend functions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A918G2Ks9i9LXRuVoNgm7R
- Correct the AUTH-approval persistence check: this JVM uses
MacOSXPreferences (~/Library/Preferences/com.vitorpamplona.amethyst.plist),
not ~/.java/.userPrefs. Updated T3.b/c/d to read it via plutil.
- Record session results (T1,T2,T3a,T3c,T6,T6b,T8,T12 pass) and the three
bugs found+fixed during the run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The relay-hint plumbing existed end to end in quartz (NIP17Factory ->
GiftWrapEvent.create(recipientRelayHint), with GiftWrapRelayHintTest), but
DesktopIAccount called createMessageNIP17/createEncryptedFileNIP17 without
passing recipientRelayHints. The default {null} lambda meant every outgoing
gift wrap shipped a 2-element ["p", pubkey] tag — the hint feature was dead
in production. Manual testing (T8) caught this: wraps on the recipient's
inbox relay had no third element.
Pre-resolve each recipient's primary DM inbox relay (first entry of their
kind:10050, order-preserving) and pass it as the hint, yielding
["p", pubkey, "wss://primary-relay/"]. Recipients with no resolvable
kind:10050 map to null and keep the 2-element shape.
Adds resolveDmInboxRelaysStrictOrdered (order-preserving) as the basis for
both the target-relay set and the primary-relay hint.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The new-conversation picker rendered a pasted npub as a non-clickable
Surface whenever getUserIfExists returned null — i.e. for any recipient
whose kind:0 metadata the local cache hadn't seen. The npub showed in the
results list but couldn't be selected, so you couldn't start a DM to
anyone new by npub.
A DM recipient is identified purely by pubkey; metadata is not required to
open a conversation. Use getOrCreateUser so a valid npub always resolves to
a selectable UserSearchCard, keeping the non-clickable fallback only for
keys that can't be resolved at all.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-8 profiling showed the crawl re-querying the same never-had-a-10002
users' outboxes every round they recirculated — ~144k slow kind:10002 drains
(p50 17.4s) against a static discovery set, dragging the round to ~18 users/s.
1. ensureRelayLists guards with `relayListDiscoverySwept`: each user's outbox
discovery runs once. The discovery relay set is static, so a second sweep of
a user still lacking a 10002 cannot find one the first missed.
2. The discovery REQ to the bounded INDEXER set co-fetches [10002, 3]: the
outbox lookup already pays the round-trip and an indexer holding a user's
10002 often holds their kind:3, so we harvest the contact list as a cheap
byproduct. The wide "every live relay" completeness sweep stays 10002-ONLY —
co-fetching kind:3 across thousands of relays downloaded the same big contact
lists repeatedly and inflated the fire-and-forget bgScope sweep the finishing
drain waits on (measured +260s at hop-3; the indexer-only co-fetch keeps
coverage flat at baseline speed).
3. harvestFromStore folds any already-stored kind:3 into the graph at Phase-A
time so Phase B never re-drains a list we hold (also speeds re-runs).
Verified same-session hop-3: pre-fix 685s / narrowed 690s / wide-co-fetch 945s,
coverage 91.74% across all.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
Covers the review fix — recipient with NIP-65 read relays but no
kind:10050 must be treated as unreachable, not routed to the read relays.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The indexer fan-out already used lists.dmInbox (strict), but the
LocalCache fast-path in DmInboxRelayResolver.resolve() — and the
no-resolver fallback in DesktopIAccount — went through the lenient
User.dmInboxRelays(), which falls back to NIP-65 read relays (kind:10002)
when the recipient has no kind:10050.
Because that fast-path returns first and short-circuits the strict
indexer lookup, a recipient with NIP-65 read relays but no published
DM-inbox would get gift wraps published to relays they never designated
for DMs — re-introducing the metadata leak (recipient pubkey + send
timing) the P0 fix was meant to close. LocalCache commonly holds
kind:10002 but not kind:10050, so this path fired often.
Switch both LocalCache lookups to dmInboxRelaysStrict().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The per-relay pending maps in RelayLatencyTracker are
Collections.synchronizedMap(LinkedHashMap): individual read/write
ops are thread-safe, but per the synchronizedMap javadoc iteration
is NOT — callers MUST hold the map's monitor while walking its
views. sweep() was iterating directly, so any network-dispatcher
mutation (adding a pending REQ, receiving an OK) during a sweep
would throw ConcurrentModificationException on AWT-EventQueue-0,
killing the Compose renderer while coroutine work kept running.
Pre-existing bug, documented in memory
desktop_relay_health_cme_crash. Ordinarily "not our problem", but
it's actively blocking manual T3 testing of this branch's AUTH
approval banner: adding any new relay triggers a
RelayHealthStore.reclassify sweep, so testers can't get a banner
render in without hitting the crash. Fix it here so the branch is
actually testable end-to-end.
Wrap both iteration loops in synchronized(pending) blocks. Sweep is
O(pending) with typically single-digit entries per relay, so the
hold time is negligible and the network dispatcher just briefly
waits.
Reproduced during manual T3 testing 2026-07-06 when adding
wss://pyramid.fiatjaf.com. Stack: RelayLatencyTracker.sweep:182 →
RelayHealthStore$reclassify$flagged$1.invokeSuspend:268.
DesktopAuthCoordinator.selfApprovedRelaysFor was calling the lenient
User.dmInboxRelays() helper, whose NIP-65-read fallback silently
expanded the tier-1 auto-allow set to include every relay in the
user's read markers. That defeated the tier-2 prompt for any AUTH-
required relay the user happened to have in NIP-65 — such as
wss://pyramid.fiatjaf.com, which never surfaced a banner during
manual testing because the coordinator was auto-signing it as
tier-1.
Switch to dmInboxRelaysStrict() (added in the earlier pre-send
alignment fix) so tier-1 is truly kind:10050 only. The KDoc already
promised strictness ("write/read relays are NOT included") — this
just makes the code match.
Surfaced during manual T3 testing 2026-07-06 with an account whose
NIP-65 outbox included pyramid.fiatjaf.com.
Rewrites the manual testing sheet so every step is an executable action
or a specific observation:
- Numbered steps within each test — no jumping between reference
sections. "Click X", "run Y in a terminal", "watch for Z".
- Each observation records YES/NO/SKIPPED for the sign-off matrix.
- Setup section spells out the wipe-preferences command and the
post-restart smoke check.
- T3 broken into T3.a/b/c/d for each button and its persistence
check separately (previously bundled T3.1–T3.10 was too dense).
- T6/T7 include exact tcpdump/tshark commands for the security
observations that require packet capture.
- T13 shortened to a sanity re-check (already verified).
- Sign-off table lists every test with a checkbox.
Purpose: give the tester a self-contained document they can follow
top to bottom in ~40 min without cross-referencing other sheets.
Updates the manual walkthrough to reflect the full state of the branch
after rebase + banner padding fix + pre-send validation fix:
- Marks what's already verified (T1 startup wiring, T5 lifecycle,
T6 P0 security via upstream UI, T13 pre-send fix).
- Adds T13 covering the alignment + resolver-probe fix from
a139c0fb17 with three sub-scenarios (strict alignment, resolver
unblock, and the real 2-user bootstrap flow).
- Documents the known pre-existing issues surfaced during testing
(RelayLatencyTracker CME, CompressionQuality stale-daemon) so
they don't get filed as regressions of ours.
- Adds a summary table pointing to the automated sheet for the
Phase A greens.
- Explicit out-of-scope list mirroring the deepening synthesis.
No code changes — documentation only.
Two related fixes that surface the same class of bug: the pre-send
"Recipient has no DM relay list" warning could disagree with the
actual send path, causing sends that either fail after the user
clicks Send, or block sends the user would have expected to work.
(a) STRICT ALIGNMENT — User.dmInboxRelays() (commons) is the lenient
"give me a delivery target for a DM" helper: it returns kind:10050
if present else the NIP-65 read marker (kind:10002). The send path
(DesktopIAccount.resolveDmInboxRelaysStrict → DmInboxRelayResolver)
uses the strict variant that returns kind:10050 only, because NIP-17
§Publishing mandates delivery to the recipient's kind:10050
exclusively — routing a wrap through a NIP-65 read relay leaks the
conversation metadata to a relay the recipient did not designate
for DMs.
Add User.dmInboxRelaysStrict() as the kind:10050-only accessor and
switch ChatNewMessageState.updateRecipientRelayStatus() to it so the
UI's "can we deliver" check matches what the send path actually
enforces.
(b) RESOLVER PROBE — pre-send validation was cache-only: if a peer's
kind:10050 hadn't landed in LocalCache yet (e.g. the peer just
published, or their event sits on an indexer relay we don't
subscribe to), the UI reported them unreachable and blocked send
even though the send path's DmInboxRelayResolver would have found
them via indexer fan-out.
Add an optional `dmInboxResolver: suspend (HexKey) -> List<...>?`
callback to ChatNewMessageState. On a synchronous cache miss for any
peer, the state optimistically blocks (preserving the "don't
silent-fail" invariant) and launches a probe. If any peer's relays
turn up, unblock immediately without requiring the user to reopen
the conversation.
Wired at both ChatNewMessageState construction sites in
DesktopMessagesScreen to DesktopIAccount.dmInboxResolver (which
Main.kt injected in the earlier P4 wire-up commit). Android's
ChatNewMessageViewModel is a separate class and keeps
cache-only behaviour — Android UI parity is deferred.
Surfaced during manual testing 2026-07-06 with two accounts where one
had a kind:10050 and one didn't: the UI correctly blocked send, but
the block persisted even after publishing kind:10050 for the missing
account until the conversation was reopened.
Amethyst Desktop uses apple.awt.fullWindowContent = true (see
applyNativeWindowChrome), which draws content edge-to-edge under
the title bar so the macOS traffic-light buttons overlap whatever
sits in the top-left of the App content column.
The AuthApprovalBanner mounts at (0, 0) of the content column,
which put its lock icon + "pyramid.fiatjaf.com" text directly
under the red/yellow/green window buttons. Screenshotted in
manual testing 2026-07-06.
Two-part fix:
1. Bump the row's own padding from horizontal 12dp / vertical 8dp
to horizontal 16dp / vertical 10dp for better breathing room
in general.
2. At the mount site in Main.kt, wrap the banner in a
platform-aware Modifier: on macOS pad 80dp from start (clears
3 traffic lights at 14pt each + spacing) plus 8dp top / bottom
4dp; on other platforms just an 8dp horizontal / 4dp vertical
margin. Non-mac users see the banner flush-ish since their
window chrome doesn't overlap.
Padding lives at the mount site so the banner composable itself
remains reusable inside chat panes or other contexts where
traffic-light clearance isn't needed.
Adds a Progress(current, total, label?) variant to SigningOpState so
multi-step signing operations (NIP-17 group sends via remote signer,
batched zaps) can show "Encrypting via remote signer (3 of 5)" rather
than an opaque indeterminate spinner.
Backwards compatible:
- Pending stays a data object — existing callers' `is Pending` checks
unaffected.
- New helper `isPending()` returns true for both Pending and Progress;
SigningState.execute uses it so a second execute() during Progress
returns null (matching the old single-flight semantics).
- SigningAwareButton renders both Pending and Progress as a spinner;
callers wanting the counter must read the state directly.
- SigningStatusBar adds a Progress branch that shows "<label>
(<current> of <total>)" — uses "Signing" as default label.
New `SigningState.updateProgress(current, total, label?)` lets the
in-flight block emit progress updates between Pending start and
finish. No-op when state is Idle/Error so callers don't have to gate.
Wire-up for NIP-17 bunker sends (publishing per-recipient progress
during NIP17Factory.createWraps' mapNotNullAsync) is a follow-up
that depends on threading the SigningState reference into the
factory's signing lambda; the substrate is here.
Four tests covering the lambda shape that DesktopAuthCoordinator's
signWithAllLoggedInUsers calls into for every NIP-42 challenge:
build RelayAuthEvent template → classify via policy → sign or
block → return List<RelayAuthEvent> for RelayAuthenticator
- tier-1 own-inbox auto-signs a valid kind:22242 event with the
right challenge + relay tags
- tier-2 unknown surfaces a PendingAuthApproval; ONCE resolution
produces a signed event (no persistence)
- tier-2 BLOCKED returns null AND persists the rejection
- tier-2 ALWAYS persists and skips the prompt on subsequent calls
Concurrency: the policy.classify call inside the lambda suspends on
the CompletableDeferred when prompting; tests use coroutineScope +
async + yieldUntilNotNull to model the banner-resolving-from-outside
pattern, mirroring how DesktopAuthCoordinator.resolve() drives the
deferred from a UI click.
Together with the existing PoolEventOutboxStateTest (auth-required
carve-out), AuthApprovalPolicyTest (classifier), and
GiftWrapRelayHintTest (NIP-17 hint placement), this completes
unit-level coverage of the AUTH pipeline. The websocket-level
round-trip stays covered by geode/.../KtorRelayTest.kt against a
real Ktor mock relay; that infra is reusable for a future
desktopApp integration test that combines mock relay + this stack.
Completes Phase 4 end-to-end. DesktopIAccount.resolveDmInboxRelaysStrict
now uses the resolver injected from Main.kt instead of the
LocalCache-only fast path. Three-layer lookup at every call:
1. LocalCache hit (kind:10050 already observed via feed pipeline)
2. Resolver's 1h LRU cache
3. Indexer fan-out via the dedicated unauthenticated NostrClient
The unauthenticated NostrClient is constructed in App() alongside
relayManager and connects on creation; DisposableEffect disconnects
on the App-level dispose. Critically NO RelayAuthenticator is
attached to this client — only the primary relayManager.client has
one (via DesktopAuthCoordinator). This closes security review F-01:
indexer queries no longer extract identity-key signatures during
kind:10050 probes against curated indexers.
resolveDmInboxRelaysStrict is converted from sync to suspend; the
three send paths (sendNip17PrivateMessage, sendNip17EncryptedFile,
sendGiftWraps) already run in suspend context inside DmSendTracker
batches, so the conversion is local. Resolver is plumbed through
MainContent as a new parameter rather than a CompositionLocal —
explicit threading matches the existing pattern for accountRelays
and relayManager.
The legacy LocalCache-only fallback inside resolveDmInboxRelaysStrict
is preserved for the constructor-default case (tests, CLI). When
dmInboxResolver is null, behaviour matches the pre-this-commit
strict-fix from 5293dae65.
Eight tests covering the resolver contract:
- localLookup hit short-circuits indexer fan-out
- empty indexer set returns empty
- empty local + empty indexer (no events arrive) yields empty
- cache hit within TTL skips indexer
- cache expiry triggers fresh indexer call
- clear() wipes all entries
- invalidate(pubkey) removes only the named entry
- localLookup returning an EMPTY list falls through to cache/indexer
(the takeIf { isNotEmpty() } guard — emptyList from localLookup
means "I don't know", not "I know they have nothing")
Uses EmptyNostrClient so RecipientRelayFetcher.fetchRelayLists returns
no events — covers the canonical "indexer found nothing" path without
needing a real mock relay. Tests for the populated-indexer path will
land with the Phase 4 wire-up commit when a Ktor-based mock relay is
plumbed through.
Claude
Vitor PamplonaandGitHub
davotoulaandClaude Fable 5
nrobi144andClaude Opus 4.8