Most of a from-scratch crawl's wall clock was connection setup, re-paid
serially: the relay pool tears down a relay's socket ~300ms after its last
drain unsubscribes, OkHttp allowed only 256 concurrent WS handshakes, and
every dial re-ran an uncached blocking getaddrinfo (10-30s per dead domain,
once per per-user path URL of the same host). Open the connections once, in
parallel, and keep them:
- GrapeRankCrawler: the warm-pool trick (never-matching REQ that only holds
the socket) now covers the whole candidate universe instead of the top 20 -
seeded at crawl start from the reachability cache's live set (one parallel
connection storm, Config.knownLiveRelays) and refreshed each round with
newly learned outbox relays, capped by Config.preconnectCap (FD-budget
aware, --preconnect-cap / --no-preconnect).
- CachingDns (quartz jvmAndroid): 10-min positive + negative DNS cache with
in-flight per-host dedup; dead domains fail in microseconds instead of
re-burning resolver timeouts, path URLs of one host resolve once.
- cli Context: dispatcher and pre-connect caps derived from the process's
open-files limit (UnixOperatingSystemMXBean), warning when ulimit is low.
- amy graperank probe: relay census - mass-connects every relay the store
knows (kind:10002 universe deduped per authority + cached verdicts) in
waves, records live/dead with real measured rtt-open into the NIP-66
reachability cache (RelayProber + RelayReachabilityStore.recordProbed),
so the next crawl skips dead relays and waits once for the slow-but-alive.
Single-server limits (FDs, ephemeral ports, DNS, threads, conntrack) and the
design are documented in quartz/plans/2026-07-10-graperank-connect-storm.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zEYRGKF943RgLaHTViJaB
Relay targeting is fully distributed: every feed, loader, finder and
broadcast path builds its own relay set and hands it to the shared
INostrClient. Only the follow-outbox flows and the top-nav feed filters
subtracted the NIP-51 kind:10006 blocked list, so blocked relays still
leaked in through the event/thread loaders (FilterMissingEvents /
FilterMissingAddressables), the user-metadata finder
(pickRelaysToLoadUsers), channel finder, DM targeting, the one-shot
fetch helpers, and the publish path (Account.computeRelayListToBroadcast)
— none of which consulted the blocked set.
Add BlockedRelayFilteringClient, a thin INostrClient decorator that
strips the active account's blocked relays from subscribe, count and
publish right before they reach the pool. Because the one-shot fetch
helpers route through subscribe/count, wrapping the client covers them
too. The blocked set is read per-call so account switches and list
edits apply with nothing to invalidate.
Wire it around the shared app client (blocked set from the logged-in
account) and around the per-account crawl client used by Event Sync and
Cashu discovery. Add commonTest coverage for the filtering, pass-through,
fully-blocked, and per-call-read behaviors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JNMPdC2eGUwTrt3XkQefuf
Activity.dispatchKeyEvent is a public framework hook; lint flags the
override only because androidx.core's intermediate override carries a
library-group @RestrictTo. Scoped to the method so the check stays live
for genuine restricted-API use. Makes :nappletHost:lintDebug pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HLfwhTdf72qFPnmPRYnzqu
Addresses correctness/perf issues found in the crawler + reachability audit:
- deadHosts permanent eviction (#1): an authority that accrued timeoutEvictStrikes
before its first EOSE was evicted forever — clearTimeoutStrikes only zeroed the
counter and could not un-evict, contradicting the "a host that ever produces is
never evicted" invariant. Add a producedHosts set that isDead() consults, so a
proven-productive authority is never treated as dead even if a concurrent strike
from the 24-worker fan-out raced it into deadHosts.
- Parking-disabled event loss (#2): when parking is off (no bgScope, or
parkTimeoutMs <= timeoutMs), a relay that streamed events but didn't EOSE in the
fast window had its buffer dropped without persist() and reported count 0. Drain,
persist, and return those events like the other two branches; strike only when
nothing was delivered.
- Wide-sweep over-narrowing (#4): relayListDiscoverySwept excluded an already-swept
straggler from the wide pass even though the wide net grows each round, so a 10002
hosted only on a later-learned relay was never fetched. Gate the wide pass on the
asked-relay set (wideRelaysSwept) instead: new users get the full net, older
stragglers get only newly-appeared relays, no (user, relay) pair asked twice.
- Onion detection (#10): replace loose relay.url.contains(".onion") with
RelayUrlNormalizer.isOnion() in isDead() and networkTypeOf(), fixing the
foo.onionfake.com false positive and the store/crawler disagreement.
- rtt-open=0 semantics (#9): document that the crawler's reachable records use
rtt-open purely as a liveness flag (0 = latency not probed), not a real 0 ms
measurement, and must not be published as authoritative latency data.
deadHosts is deliberately still NOT persisted to the 24h reachability cache (#8):
a timeout eviction means "too slow under our fan-out this run", not "proven
unreachable", so persisting it would blacklist slow-but-live hubs across runs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
FrameDispatchStats stamped a ValueTimeMark on every relay frame and recorded a
contended atomic per frame in BasicOkHttpWebSocket — the WebSocket layer used by
the whole app, unconditionally, forever — to answer a one-time question that only
graperank --diagnose read. It served its purpose (proved the our-side dispatch
lag is ~200ms mean and the EOSE-wait is dominantly relay-side, so the crawler is
network-bound), but the ongoing per-frame Pair allocation + atomic contention on
every client's relay traffic isn't worth carrying. Revert the channel back to
Channel<String> and delete the stats holder. The diagnose-gated saturation ticker
and per-drain latency breakdown stay — they're crawler-local, off the hot path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
Measured: relays deliver their events in ~0.6s then sit ~4.6s (86% of drain wall)
before sending EOSE — mostly relay-side (our pipeline adds only ~200ms). So instead
of waiting the full 10s fast window then parking, close a drain that has delivered
>=1 event and then gone silent for eoseIdleMs, treating it as complete ("eose-idle").
awaitTerminalOrQuiescent: the idle timer arms only AFTER the first event, so a relay
merely slow to answer still gets the full timeoutMs and is never cut prematurely; a
still-streaming relay keeps resetting the window. eose-idle paginates if the page was
capped and clears timeout strikes (it delivered), but joins notAnswered (no clean
EOSE, so its missing authors are retried elsewhere). Off by default (eoseIdleMs=0),
CLI --eose-idle-ms, so it can be A/B'd against the plain fast window.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
The dedicated frame-dispatch pool (ea1093ad) made dispatch lag WORSE, not better:
mean 200ms→460ms, max 3.5s→5.5s, frames>1s 43k→76k. The pool was sized cores*2
(=8 here) vs Dispatchers.IO's 64 threads, so it cut frame-processing parallelism
~8x. Lesson: the our-side lag is dominated by per-connection serial decode
throughput / thread count, NOT cross-contention with the store's IO writes — the
experiment ruled that hypothesis out. Reverting to shared IO; keep FrameDispatchStats.
EOSE-wait is confirmed dominantly relay-side (200ms our-mean vs ~5s eose-wait).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
Measured on a GrapeRank crawl, frame decode/dispatch (per-connection consumer
coroutines) ran on the shared Dispatchers.IO — the same pool that runs the store's
blocking SQLite inserts. During event floods, frame coroutines queued behind those
inserts: mean 200ms and up to 3.5s of dispatch lag, with 43k frames waiting >1s in
our pipeline. That lag also skews the relay-idle/EOSE timing the crawler reads.
Give frame processing its own daemon thread pool (sized to a small multiple of
cores; decode is light + CPU-bound), shared across all connections. Frame delivery
stays prompt regardless of what the IO pool is doing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
Adds FrameDispatchStats: the lag between a relay frame arriving on the OkHttp
reader thread and our per-connection consumer coroutine (on shared Dispatchers.IO)
pulling it off the channel — pure our-side pipeline delay, relay send-timing
excluded. BasicOkHttpWebSocket stamps arrival before enqueue and records the lag
on dequeue; the crawler resets it at start and dumps it in the --diagnose summary.
Answers whether a drain's 5s gap between the relay's last event and its EOSE is
the relay being slow to SEND eose (low dispatch-lag) or our IO pipeline backing up
so the already-arrived eose frame sits queued (high dispatch-lag).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
Answers "are we resource-bound or waiting on relays" without a profiler:
- progress ticker gains "Nw/CAPw" (drain workers busy vs drainConcurrency) and
"N rl" (rate-limit responses so far) — a rarely-full pool means the producer or
the relays are the limit, not concurrency; a climbing rl count is the external
ceiling that made concurrency 60 backfire.
- crawl-end "latency breakdown": splits each drain's wall into time-to-first-event
vs EOSE-wait-AFTER-the-relay's-last-event, and reports the % of drain wall spent
waiting for EOSE after the relay was already done, how many drains blew the fast
window and parked, and total rate-limit hits. A high EOSE-wait % is the direct
case for a shorter/adaptive fast window over more concurrency.
All gated on config.diagnose; zero cost on a normal run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
Two changes:
1. The reachability flush re-wrote the SEEDED known-dead relays with a fresh
created_at every run, refreshing their TTL without a re-probe — so a relay
marked dead once (and thereafter skipped, never re-dialed) would stay
blacklisted forever as long as crawls kept running, defeating the TTL's
re-probe. Stats.deadRelays now reports only relays actually dialed this run
(deadRelays - knownDeadRelays); seeded records keep their original timestamp
and age out on schedule so the next run re-probes them.
2. Rename GrapeRankDataCrawler -> GrapeRankCrawler (file + all references).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
Wire RelayReachabilityStore into the crawler and the WoT updater so liveness is
shared across procedures and runs instead of each rediscovering dead relays.
- OperatorKeys.monitorKey(): a dedicated machine monitor identity derived from the
operator master (domain "relay-monitor:"), independent of any account — the
30166 records are published under this, not the observer key.
- Context.reachability: a RelayReachabilityStore over the shared store, signed by
the monitor key.
- Crawler: Config.knownDeadRelays seeds deadRelays before the run; Stats now
returns the final dead/live sets. GrapeRankCommand seeds from snapshot().dead
and flushes the crawl's verdicts back via reachability.record().
- Updater: Config.knownDead skips proven-dead relays from the reconcile plan — a
dead relay cannot serve its authors, so reconciling it only burns a timeout.
Live author-advertised relays are always synced.
All behind --no-reachability-cache. TTL'd (24h), so a recovered relay is retried
once its record ages out — a "skip for now", never a permanent ignore, keeping
the outbox rule that every live advertised relay is tried.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
A durable, shareable relay-reachability cache backed by the EventStore as NIP-66
kind:30166 Relay Discovery events, so the crawler, the WoT updater, and future
runs share liveness knowledge instead of each rediscovering dead relays from an
in-memory set wiped at process exit.
- 30166 is addressable by its d-tag (relay URL) → one replaceable status slot per
(monitor, relay), with created_at giving a free TTL.
- Reachable → 30166 with rtt-open; dead → 30166 without (NIP-66 has no explicit
offline field; liveness is inferred from a fresh successful open). Live wins
over dead within the TTL, so third-party monitors' 30166 can be ingested.
- snapshot() loads the fresh set once (not a per-request hot-path query); record()
flushes a run's findings. A relay is only skipped for the TTL, never permanently
— consistent with the outbox rule that every advertised write relay is tried.
Reuses the existing RelayDiscoveryEvent. jvmTest covers record/reload,
live-overrides-dead, TTL expiry, and .onion→Tor network tagging.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
The synchronized intrinsic is only available on JVM/Android, breaking the
iOS (compileKotlinIosSimulatorArm64) build in commonMain. Replace the Any()
lock with a kotlinx.coroutines Mutex + withLock, which is KMP-common and
safe here since all three store methods are already suspend functions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A918G2Ks9i9LXRuVoNgm7R
- Correct the AUTH-approval persistence check: this JVM uses
MacOSXPreferences (~/Library/Preferences/com.vitorpamplona.amethyst.plist),
not ~/.java/.userPrefs. Updated T3.b/c/d to read it via plutil.
- Record session results (T1,T2,T3a,T3c,T6,T6b,T8,T12 pass) and the three
bugs found+fixed during the run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The relay-hint plumbing existed end to end in quartz (NIP17Factory ->
GiftWrapEvent.create(recipientRelayHint), with GiftWrapRelayHintTest), but
DesktopIAccount called createMessageNIP17/createEncryptedFileNIP17 without
passing recipientRelayHints. The default {null} lambda meant every outgoing
gift wrap shipped a 2-element ["p", pubkey] tag — the hint feature was dead
in production. Manual testing (T8) caught this: wraps on the recipient's
inbox relay had no third element.
Pre-resolve each recipient's primary DM inbox relay (first entry of their
kind:10050, order-preserving) and pass it as the hint, yielding
["p", pubkey, "wss://primary-relay/"]. Recipients with no resolvable
kind:10050 map to null and keep the 2-element shape.
Adds resolveDmInboxRelaysStrictOrdered (order-preserving) as the basis for
both the target-relay set and the primary-relay hint.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The new-conversation picker rendered a pasted npub as a non-clickable
Surface whenever getUserIfExists returned null — i.e. for any recipient
whose kind:0 metadata the local cache hadn't seen. The npub showed in the
results list but couldn't be selected, so you couldn't start a DM to
anyone new by npub.
A DM recipient is identified purely by pubkey; metadata is not required to
open a conversation. Use getOrCreateUser so a valid npub always resolves to
a selectable UserSearchCard, keeping the non-clickable fallback only for
keys that can't be resolved at all.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-8 profiling showed the crawl re-querying the same never-had-a-10002
users' outboxes every round they recirculated — ~144k slow kind:10002 drains
(p50 17.4s) against a static discovery set, dragging the round to ~18 users/s.
1. ensureRelayLists guards with `relayListDiscoverySwept`: each user's outbox
discovery runs once. The discovery relay set is static, so a second sweep of
a user still lacking a 10002 cannot find one the first missed.
2. The discovery REQ to the bounded INDEXER set co-fetches [10002, 3]: the
outbox lookup already pays the round-trip and an indexer holding a user's
10002 often holds their kind:3, so we harvest the contact list as a cheap
byproduct. The wide "every live relay" completeness sweep stays 10002-ONLY —
co-fetching kind:3 across thousands of relays downloaded the same big contact
lists repeatedly and inflated the fire-and-forget bgScope sweep the finishing
drain waits on (measured +260s at hop-3; the indexer-only co-fetch keeps
coverage flat at baseline speed).
3. harvestFromStore folds any already-stored kind:3 into the graph at Phase-A
time so Phase B never re-drains a list we hold (also speeds re-runs).
Verified same-session hop-3: pre-fix 685s / narrowed 690s / wide-co-fetch 945s,
coverage 91.74% across all.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
Covers the review fix — recipient with NIP-65 read relays but no
kind:10050 must be treated as unreachable, not routed to the read relays.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The indexer fan-out already used lists.dmInbox (strict), but the
LocalCache fast-path in DmInboxRelayResolver.resolve() — and the
no-resolver fallback in DesktopIAccount — went through the lenient
User.dmInboxRelays(), which falls back to NIP-65 read relays (kind:10002)
when the recipient has no kind:10050.
Because that fast-path returns first and short-circuits the strict
indexer lookup, a recipient with NIP-65 read relays but no published
DM-inbox would get gift wraps published to relays they never designated
for DMs — re-introducing the metadata leak (recipient pubkey + send
timing) the P0 fix was meant to close. LocalCache commonly holds
kind:10002 but not kind:10050, so this path fired often.
Switch both LocalCache lookups to dmInboxRelaysStrict().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The per-relay pending maps in RelayLatencyTracker are
Collections.synchronizedMap(LinkedHashMap): individual read/write
ops are thread-safe, but per the synchronizedMap javadoc iteration
is NOT — callers MUST hold the map's monitor while walking its
views. sweep() was iterating directly, so any network-dispatcher
mutation (adding a pending REQ, receiving an OK) during a sweep
would throw ConcurrentModificationException on AWT-EventQueue-0,
killing the Compose renderer while coroutine work kept running.
Pre-existing bug, documented in memory
desktop_relay_health_cme_crash. Ordinarily "not our problem", but
it's actively blocking manual T3 testing of this branch's AUTH
approval banner: adding any new relay triggers a
RelayHealthStore.reclassify sweep, so testers can't get a banner
render in without hitting the crash. Fix it here so the branch is
actually testable end-to-end.
Wrap both iteration loops in synchronized(pending) blocks. Sweep is
O(pending) with typically single-digit entries per relay, so the
hold time is negligible and the network dispatcher just briefly
waits.
Reproduced during manual T3 testing 2026-07-06 when adding
wss://pyramid.fiatjaf.com. Stack: RelayLatencyTracker.sweep:182 →
RelayHealthStore$reclassify$flagged$1.invokeSuspend:268.
DesktopAuthCoordinator.selfApprovedRelaysFor was calling the lenient
User.dmInboxRelays() helper, whose NIP-65-read fallback silently
expanded the tier-1 auto-allow set to include every relay in the
user's read markers. That defeated the tier-2 prompt for any AUTH-
required relay the user happened to have in NIP-65 — such as
wss://pyramid.fiatjaf.com, which never surfaced a banner during
manual testing because the coordinator was auto-signing it as
tier-1.
Switch to dmInboxRelaysStrict() (added in the earlier pre-send
alignment fix) so tier-1 is truly kind:10050 only. The KDoc already
promised strictness ("write/read relays are NOT included") — this
just makes the code match.
Surfaced during manual T3 testing 2026-07-06 with an account whose
NIP-65 outbox included pyramid.fiatjaf.com.
Rewrites the manual testing sheet so every step is an executable action
or a specific observation:
- Numbered steps within each test — no jumping between reference
sections. "Click X", "run Y in a terminal", "watch for Z".
- Each observation records YES/NO/SKIPPED for the sign-off matrix.
- Setup section spells out the wipe-preferences command and the
post-restart smoke check.
- T3 broken into T3.a/b/c/d for each button and its persistence
check separately (previously bundled T3.1–T3.10 was too dense).
- T6/T7 include exact tcpdump/tshark commands for the security
observations that require packet capture.
- T13 shortened to a sanity re-check (already verified).
- Sign-off table lists every test with a checkbox.
Purpose: give the tester a self-contained document they can follow
top to bottom in ~40 min without cross-referencing other sheets.
Updates the manual walkthrough to reflect the full state of the branch
after rebase + banner padding fix + pre-send validation fix:
- Marks what's already verified (T1 startup wiring, T5 lifecycle,
T6 P0 security via upstream UI, T13 pre-send fix).
- Adds T13 covering the alignment + resolver-probe fix from
a139c0fb17 with three sub-scenarios (strict alignment, resolver
unblock, and the real 2-user bootstrap flow).
- Documents the known pre-existing issues surfaced during testing
(RelayLatencyTracker CME, CompressionQuality stale-daemon) so
they don't get filed as regressions of ours.
- Adds a summary table pointing to the automated sheet for the
Phase A greens.
- Explicit out-of-scope list mirroring the deepening synthesis.
No code changes — documentation only.
Two related fixes that surface the same class of bug: the pre-send
"Recipient has no DM relay list" warning could disagree with the
actual send path, causing sends that either fail after the user
clicks Send, or block sends the user would have expected to work.
(a) STRICT ALIGNMENT — User.dmInboxRelays() (commons) is the lenient
"give me a delivery target for a DM" helper: it returns kind:10050
if present else the NIP-65 read marker (kind:10002). The send path
(DesktopIAccount.resolveDmInboxRelaysStrict → DmInboxRelayResolver)
uses the strict variant that returns kind:10050 only, because NIP-17
§Publishing mandates delivery to the recipient's kind:10050
exclusively — routing a wrap through a NIP-65 read relay leaks the
conversation metadata to a relay the recipient did not designate
for DMs.
Add User.dmInboxRelaysStrict() as the kind:10050-only accessor and
switch ChatNewMessageState.updateRecipientRelayStatus() to it so the
UI's "can we deliver" check matches what the send path actually
enforces.
(b) RESOLVER PROBE — pre-send validation was cache-only: if a peer's
kind:10050 hadn't landed in LocalCache yet (e.g. the peer just
published, or their event sits on an indexer relay we don't
subscribe to), the UI reported them unreachable and blocked send
even though the send path's DmInboxRelayResolver would have found
them via indexer fan-out.
Add an optional `dmInboxResolver: suspend (HexKey) -> List<...>?`
callback to ChatNewMessageState. On a synchronous cache miss for any
peer, the state optimistically blocks (preserving the "don't
silent-fail" invariant) and launches a probe. If any peer's relays
turn up, unblock immediately without requiring the user to reopen
the conversation.
Wired at both ChatNewMessageState construction sites in
DesktopMessagesScreen to DesktopIAccount.dmInboxResolver (which
Main.kt injected in the earlier P4 wire-up commit). Android's
ChatNewMessageViewModel is a separate class and keeps
cache-only behaviour — Android UI parity is deferred.
Surfaced during manual testing 2026-07-06 with two accounts where one
had a kind:10050 and one didn't: the UI correctly blocked send, but
the block persisted even after publishing kind:10050 for the missing
account until the conversation was reopened.
Amethyst Desktop uses apple.awt.fullWindowContent = true (see
applyNativeWindowChrome), which draws content edge-to-edge under
the title bar so the macOS traffic-light buttons overlap whatever
sits in the top-left of the App content column.
The AuthApprovalBanner mounts at (0, 0) of the content column,
which put its lock icon + "pyramid.fiatjaf.com" text directly
under the red/yellow/green window buttons. Screenshotted in
manual testing 2026-07-06.
Two-part fix:
1. Bump the row's own padding from horizontal 12dp / vertical 8dp
to horizontal 16dp / vertical 10dp for better breathing room
in general.
2. At the mount site in Main.kt, wrap the banner in a
platform-aware Modifier: on macOS pad 80dp from start (clears
3 traffic lights at 14pt each + spacing) plus 8dp top / bottom
4dp; on other platforms just an 8dp horizontal / 4dp vertical
margin. Non-mac users see the banner flush-ish since their
window chrome doesn't overlap.
Padding lives at the mount site so the banner composable itself
remains reusable inside chat panes or other contexts where
traffic-light clearance isn't needed.
Adds a Progress(current, total, label?) variant to SigningOpState so
multi-step signing operations (NIP-17 group sends via remote signer,
batched zaps) can show "Encrypting via remote signer (3 of 5)" rather
than an opaque indeterminate spinner.
Backwards compatible:
- Pending stays a data object — existing callers' `is Pending` checks
unaffected.
- New helper `isPending()` returns true for both Pending and Progress;
SigningState.execute uses it so a second execute() during Progress
returns null (matching the old single-flight semantics).
- SigningAwareButton renders both Pending and Progress as a spinner;
callers wanting the counter must read the state directly.
- SigningStatusBar adds a Progress branch that shows "<label>
(<current> of <total>)" — uses "Signing" as default label.
New `SigningState.updateProgress(current, total, label?)` lets the
in-flight block emit progress updates between Pending start and
finish. No-op when state is Idle/Error so callers don't have to gate.
Wire-up for NIP-17 bunker sends (publishing per-recipient progress
during NIP17Factory.createWraps' mapNotNullAsync) is a follow-up
that depends on threading the SigningState reference into the
factory's signing lambda; the substrate is here.
Four tests covering the lambda shape that DesktopAuthCoordinator's
signWithAllLoggedInUsers calls into for every NIP-42 challenge:
build RelayAuthEvent template → classify via policy → sign or
block → return List<RelayAuthEvent> for RelayAuthenticator
- tier-1 own-inbox auto-signs a valid kind:22242 event with the
right challenge + relay tags
- tier-2 unknown surfaces a PendingAuthApproval; ONCE resolution
produces a signed event (no persistence)
- tier-2 BLOCKED returns null AND persists the rejection
- tier-2 ALWAYS persists and skips the prompt on subsequent calls
Concurrency: the policy.classify call inside the lambda suspends on
the CompletableDeferred when prompting; tests use coroutineScope +
async + yieldUntilNotNull to model the banner-resolving-from-outside
pattern, mirroring how DesktopAuthCoordinator.resolve() drives the
deferred from a UI click.
Together with the existing PoolEventOutboxStateTest (auth-required
carve-out), AuthApprovalPolicyTest (classifier), and
GiftWrapRelayHintTest (NIP-17 hint placement), this completes
unit-level coverage of the AUTH pipeline. The websocket-level
round-trip stays covered by geode/.../KtorRelayTest.kt against a
real Ktor mock relay; that infra is reusable for a future
desktopApp integration test that combines mock relay + this stack.
Completes Phase 4 end-to-end. DesktopIAccount.resolveDmInboxRelaysStrict
now uses the resolver injected from Main.kt instead of the
LocalCache-only fast path. Three-layer lookup at every call:
1. LocalCache hit (kind:10050 already observed via feed pipeline)
2. Resolver's 1h LRU cache
3. Indexer fan-out via the dedicated unauthenticated NostrClient
The unauthenticated NostrClient is constructed in App() alongside
relayManager and connects on creation; DisposableEffect disconnects
on the App-level dispose. Critically NO RelayAuthenticator is
attached to this client — only the primary relayManager.client has
one (via DesktopAuthCoordinator). This closes security review F-01:
indexer queries no longer extract identity-key signatures during
kind:10050 probes against curated indexers.
resolveDmInboxRelaysStrict is converted from sync to suspend; the
three send paths (sendNip17PrivateMessage, sendNip17EncryptedFile,
sendGiftWraps) already run in suspend context inside DmSendTracker
batches, so the conversion is local. Resolver is plumbed through
MainContent as a new parameter rather than a CompositionLocal —
explicit threading matches the existing pattern for accountRelays
and relayManager.
The legacy LocalCache-only fallback inside resolveDmInboxRelaysStrict
is preserved for the constructor-default case (tests, CLI). When
dmInboxResolver is null, behaviour matches the pre-this-commit
strict-fix from 5293dae65.
Eight tests covering the resolver contract:
- localLookup hit short-circuits indexer fan-out
- empty indexer set returns empty
- empty local + empty indexer (no events arrive) yields empty
- cache hit within TTL skips indexer
- cache expiry triggers fresh indexer call
- clear() wipes all entries
- invalidate(pubkey) removes only the named entry
- localLookup returning an EMPTY list falls through to cache/indexer
(the takeIf { isNotEmpty() } guard — emptyList from localLookup
means "I don't know", not "I know they have nothing")
Uses EmptyNostrClient so RecipientRelayFetcher.fetchRelayLists returns
no events — covers the canonical "indexer found nothing" path without
needing a real mock relay. Tests for the populated-indexer path will
land with the Phase 4 wire-up commit when a Ktor-based mock relay is
plumbed through.
Three-layer resolver for "where do I publish this NIP-17 gift wrap":
1. LocalCache hit — if the caller already saw the user's kind:10050
via the regular feed pipeline, skip I/O entirely.
2. In-memory LRU cache — TTL 1h, 100 entries; avoids re-querying
indexers when opening several conversations in sequence.
3. Indexer fan-out — RecipientRelayFetcher against a curated set
(DefaultDmIndexerRelays: relay.nos.social, relay.damus.io,
nos.lol, relay.nostr.band, purplerelay.com — purplepag.es
deliberately excluded for poor kind:10050 coverage).
Strictness vs. the existing User.dmInboxRelays():
- filters to kind:10050 ONLY; NEVER falls back to NIP-65 read
marker (kind:10002). User.dmInboxRelays() silently substitutes
that, which is the same metadata-leak class fixed by 5293dae65.
- empty list = canonical "unreachable" signal; caller refuses to
publish (DesktopIAccount.resolveDmInboxRelaysStrict already
does this).
Security: the NostrClient passed in MUST be a dedicated
unauthenticated instance — no RelayAuthenticator attached. An
authenticated indexer fan-out (the current state with the primary
client) would extract identity-key signatures during the kind:10050
probe, escalating "indexer learns we want to DM pubkey X" into
"indexer learns user U wants to DM pubkey X". KDoc warning is
explicit; Phase 4 follow-up creates the unauth client in Main.kt
and injects it.
LocalLookup callback is plugged via lambda so CLI / headless
callers (amy) can use this without a Compose LocalCache.
Not yet wired into DesktopIAccount.resolveDmInboxRelaysStrict —
that wire-up is the next commit and converts the sync helper to
suspend, threading through sendNip17* batch construction.
Adds AuthApprovalBanner in commons.relayClient.auth — a Compose-
Multiplatform composable that renders one row per pending tier-2
NIP-42 AUTH challenge with three actions matching the AuthApprovalScope:
[Once] — sign this challenge, don't persist
[Always] — sign + persist ALWAYS via the store
[Never] — drop + persist BLOCKED via the store
Wired into desktop Main.kt as a global top-of-content banner reading
authCoordinator.pendingApprovals and calling authCoordinator.resolve.
Now tier-2 challenges actually have a UI to resolve — desktop AUTH is
end-to-end usable.
Up to 3 rows stack inline; the rest collapse into a "+N more pending"
row (click-to-expand can come later). Each row shows the relay's
display URL plus message-count when multiple challenges from the same
relay have coalesced.
The composable itself is in commons so Android picks it up free when
its AccountAuthApprovals VM wire-up lands — only the Main.kt-level
wiring (where to mount the banner in the layout) is platform-specific.
Lifecycle:
- Banner subscribes to pendingApprovals via collectAsState; recomposes
only when the PersistentMap identity changes (per the substrate
built in earlier commits).
- onResolve calls authCoordinator.resolve(url, scope), which completes
the underlying CompletableDeferred + removes the entry from the
pending map; the suspended signer wakes up and signs (or doesn't).
Until now desktop had no NIP-42 AUTH wiring at all — relays demanding
AUTH from desktop users got silently ignored. This commit closes the
gap, but does it the security-conscious way using the
AuthApprovalPolicy substrate from earlier commits.
DesktopAuthCoordinator binds to AccountState transitions in Main.kt
and per logged-in account:
- constructs a PreferencesAuthApprovalStore scoped by pubkey
- constructs an AuthApprovalPolicy with self-approved relays sourced
from the active account's NIP-17 DM-inbox (kind:10050) cache
- constructs a RelayAuthenticator whose signWithAllLoggedInUsers
lambda routes every AUTH challenge through the policy
Tier 1 (own DM-inbox + persisted ALWAYS) signs automatically. Tier 2
challenges hand back a CompletableDeferred surfaced on
authCoordinator.pendingApprovals. Until the inline banner UI lands
(P2.5 follow-up), tier-2 pending stays unresolved — which means
tier-2 relays don't get an AUTH response, same outcome as the
pre-this-commit world. The improvement here is tier-1: own DM
inbox relays now AUTH automatically without any prompt.
Lifecycle: onLogin attaches the authenticator; onLogout and account-
switch tear it down and complete any pending deferreds with BLOCKED
so suspended signers don't dangle.
Self-approved relays are deliberately scoped to kind:10050 (DM
inbox) only, NOT NIP-65 write/read relays. A user may follow read-
only relays they don't want to AUTH-identify themselves on — and the
common case where AUTH matters most is the user's own DM inbox.
NIP17Factory.createWraps launches all per-recipient seal builds via
mapNotNullAsync, which today runs them fully parallel. Each seal
needs nip44_encrypt + sign — for a NIP-46 (bunker) signer that means
two round-trips per recipient. A 5-recipient group send launches 10
concurrent in-flight requests against the bunker socket, and nsec.app
/ Amber / Keychat typically serialize past ~10 in-flight, so some
requests queue past the 65s timeout and silently fail.
Cap at 4 concurrent when signer is NostrSignerRemote. Local signers
(NostrSignerInternal, NostrSignerSync) bypass the semaphore and stay
fully parallel — no overhead, no behaviour change for nsec users.
The real fix is the batched nip44_get_conversation_keys NIP-46 RPC
(separate spec PR + plan) which collapses N×2 round-trips into ~2.
This commit is the interim throttle until that lands.
Per NIP-17 §Publishing, gift wraps MUST only be published to the relays
advertised in the recipient's kind:10050. Today three send paths in
DesktopIAccount fall through to relayManager.connectedRelays.value
when the recipient has no kind:10050 cached:
sendNip17PrivateMessage (line 200)
sendNip17EncryptedFile (line 231)
sendGiftWraps (line 253)
This is the security-review F-04 metadata leak: at best the wrap never
reaches the recipient (their other clients don't read those relays);
at worst the recipient pubkey + send timestamp leak to general/feed
relays outside their chosen inbox. Same class of bug as the relay-
power-tools work explicitly closed for the relay picker on
2026-04-20 ("block DM fallback to all relays — metadata leak").
Replace the fallback with strict resolution: if the recipient has no
kind:10050 in the cache, return an empty target set. DmSendTracker
already handles total relay count == 0 with a "No relays available"
failure state, so the user gets a visible error instead of a silent
leak.
Indexer fan-out + a UI dialog for the missing-10050 case is the
permanent fix, scoped to Phase 4 (DmInboxRelayResolver). This commit
is the conservative pre-Phase-4 plug — better to fail visibly than
leak silently.
NIP-04 send is unchanged: that path is pre-NIP-17, the encrypted
content sits next to other public events on the sender's outbox by
design.
Three regression tests covering the NIP-17 relay-hint contract just
introduced on GiftWrapEvent.create:
- default (no hint) emits the historical two-element ["p", pubkey]
shape — guards every existing caller against a wire-format
regression.
- with-hint emits ["p", pubkey, relay-url] — the canonical NIP-17
shape with the hint on the public wrap (NOT inside the seal, which
is the encrypted envelope and would hide routing info).
- null-hint must NOT produce ["p", pubkey, ""] — that would broadcast
"this user has no canonical inbox" as a metadata leak.
Per NIP-17 §Publishing, a gift wrap (kind 1059) MAY carry the
recipient's primary DM inbox relay as a third element of the p tag.
Other clients the recipient runs (or relays acting as inbox routers)
can then locate the wrap without performing their own kind:10050
lookup — handy when the recipient is multi-device and the second
device's 10050 cache is cold.
GiftWrapEvent.create gains an optional `recipientRelayHint:
NormalizedRelayUrl?` parameter that flows into PTag.assemble (which
already accepts a relay hint). NIP17Factory.createWraps and the four
public createMessageNIP17 / createEncryptedFileNIP17 /
createReactionWithinGroup entry points gain a matching
`recipientRelayHints: (HexKey) -> NormalizedRelayUrl?` lambda so
multi-recipient sends can pass per-recipient hints in one shot.
All new parameters default to null / { null }, so every existing
caller compiles unchanged and still emits the historical
two-element ["p", recipientPubKey] shape. Callers that resolve
kind:10050 via the (forthcoming) DmInboxRelayResolver can wire the
result through to populate the hint.
While here, document the existing — but undocumented — invariant
that shared rumor created_at falls out naturally because the
rumor is signed once before the per-recipient mapNotNullAsync loop.
This is what anchors cross-recipient reaction/receipt dedupe.
Desktop persistence for the AuthApprovalPolicy in commons. Backs the
`auth_approvals` use case from the plan using java.util.prefs.Preferences
instead of the originally proposed sibling outbox.db SQLite table.
Trade-off rationale: the AUTH approval set per account is small
(typically < 50 relays for any user) and the read pattern is bounded
(one lookup per relay per session, easily cached in memory by the
policy layer). java.util.prefs is already in use elsewhere on desktop
(SearchHistoryStore, DesktopPreferences) and adds zero new
dependencies or schema migrations.
The retry_queue table from the same outbox.db proposal needs the
higher-throughput characteristics SQLite gives us; it remains scoped
to P3 (send visibility), which can introduce a proper sibling DB at
that point.
Per-account scoping by Preferences node — logout/account-delete calls
clear() which removeNode()s the subtree. ONCE scope is never written
to disk, enforced explicitly here in addition to the interface
contract.
Not yet wired into a DesktopAuthCoordinator (today desktop has NO
AUTH wiring at all). That wiring lands in P2.5 alongside the banner
UI.
The current Android-only AuthCoordinator signs every NIP-42 AUTH
challenge from every relay unconditionally (and across every logged-in
account). For desktop there is no AUTH wiring at all — challenges are
ignored, so AUTH-walled relays silently drop DMs.
Both behaviours fail the security review: unconditional signing lets
any relay the user reads (or any malicious relay they touch) extract an
identity-key signature with timestamp, and signing across all accounts
links them under one relay observer.
This commit adds the substrate for a tiered classifier — wire-up will
follow with the desktop AuthCoordinator (P2.5) and SQLite-backed
persistence (P2.4). The policy itself is platform-agnostic and lives in
commons so Android can adopt the same design later.
Two tiers, no third silent-drop path:
- auto-allow when the relay is in the user's own outbox/DM-inbox set,
or has a persisted ALWAYS grant (subject to BLOCKED override)
- prompt-and-suspend via CompletableDeferred for everything else, with
the user's `[Once] [Always] [Never]` choice driving the deferred
Includes InMemoryAuthApprovalStore for tests + the ONCE session cache;
SqliteAuthApprovalStore lands in P2.4 with the sibling outbox.db.
Eight unit tests cover tier-1, persisted ALWAYS, persisted BLOCKED
(including BLOCKED overriding tier-1), unknown-prompt-then-cache,
re-eval of selfApprovedRelays on Account changes, and store.clear().
RelayAuthStatus has to stay mutable — it holds LruCaches addressable from
the per-relay OkHttp dispatcher thread, and replacing the whole holder
on every mutation would be wasteful. But its mutability also makes it
useless as a StateFlow value: mutating an entry doesn't change map
identity, so distinct-until-changed downstream swallows the update and
Compose never recomposes.
Add an immutable view alongside: RelayAuthSnapshot (phase +
lastAuthSuccessAt). RelayAuthStatus.snapshot() derives it from the LRU.
RelayAuthenticator publishes a PersistentMap<NormalizedRelayUrl,
RelayAuthSnapshot> via authStateFlow on every mutation (connect,
disconnect, AUTH-submitted, AUTH-OK, AUTH-fail). PersistentMap gives
O(log32 n) updates and a fresh identity per put, so both StateFlow
equality and Compose strong-skipping work.
This is the substrate for downstream consumers — the AUTH approval
banner, the retry-queue wake on authCompleted, the indexer-fan-out gate
— none of which are wired yet. They will read authStateFlow rather than
querying RelayAuthStatus directly.
Claude
Vitor PamplonaandGitHub
davotoulaandClaude Fable 5
nrobi144andClaude Opus 4.8