Clients in the wild publish structurally wrong values in profile
metadata — e.g. "nip05":{} — and the strict field serializers made
JsonMapper throw, so contactMetaData() returned null and the entire
profile (name, picture, about…) was discarded.
Generalize the BirthdayTolerantSerializer precedent: string fields now
use TolerantStringSerializer (accepts any JSON primitive, ignores
objects/arrays/null) and the bot flag uses TolerantBooleanSerializer,
so a single malformed field is skipped instead of being fatal.
Non-JSON content (e.g. "Relay initialized") remains unrecoverable and
still parses to null; a test pins that it does so without throwing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KSCGp6KgcnaepEMbxNroYe
The miner enumerates the nonce space deterministically (the random base is
overwritten before the first hash), so naively racing N copies of the search
duplicates the exact same candidate sequence N times. PoWMiner.mine() now
races workers over disjoint slices instead: each worker's nonce carries a
distinct fixed prefix while only the bytes after it are enumerated, making the
aggregate hash rate scale with cores (~3.8x on a 4-core box).
The hot loop also switches from sha256() to sha256Into() with a reused
32-byte buffer, so hashing no longer allocates per attempt.
amy wiring:
- `pow mine` and `post --pow` mine on all cores by default; `pow mine
--threads N` overrides.
- `pow bench` measures the all-cores rate (what mining now uses, also the
basis for expected_seconds) alongside a new hashes_per_second_single_core.
- PoWEstimator benchmarks with sha256Into to match the miner, and gains a
workers overload that prices in cross-core contention.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UhaF5scnAvhP9wr9R7bTWM
Two bugs found in a pre-merge audit:
- quartz PoolEventOutboxState: auth-required NAKs only spared the `responses`
budget, but `tries` (grown by every send/re-pump and NOT auth-aware) still
accumulated across reconnects, so a slow/flapping AUTH handshake could exhaust
Tries.isDone() and drop the event — with a spurious onEventGaveUp — before AUTH
landed. Now an auth-required NAK resets the relay's retry budget (it responded,
so it's up and just wants auth). Regression test added.
- RelayAuthPromptBus used a replay=0 SharedFlow, so a challenge that resolved to
ASK before RelayAuthPromptHost subscribed (cold start / account switch) was
dropped and the auth coroutine stalled the full timeout then DISMISSed. Add
replay so late subscribers recover pending prompts (the host already filters
resolved ones). Regression test added.
Also record the as-built design (Always/Never/Custom + toggles, venues, give-up
toast, known deny-relay-outbox limitation) in the plan doc.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EZjmYpgHP4pf79Sav5QT8a
Instead of silently dropping an event once its per-relay retry budget is
spent, PoolEventOutbox now reports it: PoolEventOutboxState.newTry returns
whether the attempt gave up on the relay, PoolEventOutbox.onSent surfaces
the dropped event, and NostrClient notifies a new (default no-op, so
non-breaking) RelayConnectionListener.onEventGaveUp(relay, event). Lets a
host surface a failed delivery rather than lose it silently; the event may
still be pending on other relays. Unit-tested via the outbox try budget.
Note: timed retry backoff (the other half of this item) is intentionally
deferred — applied in the shared syncState path it would also delay the
post-auth resend and regress the auth-required fix, so it needs
trigger-aware handling designed separately.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EZjmYpgHP4pf79Sav5QT8a
Add INostrClient.activeOutboxEvents(url) (backed by
PoolEventOutbox.activeOutboxEventsFor) returning the full events still
pending delivery to a relay, not just their ids like activeOutboxCache.
This lets a host explain *why* a relay is being authenticated with —
e.g. a pending kind-1059 gift wrap means we're sending a DM to its
recipient — by inspecting kind/tags. Combined with the existing
activeRequests(url) filters, it is the generic challenge context the
NIP-42 decision hook needs. Updates the INostrClient test fakes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EZjmYpgHP4pf79Sav5QT8a
PoolEventOutboxState treated an `auth-required` OK-false the same as any
transient failure, recording it against the per-relay Tries budget
(responses > 2 drops the event on the next send attempt). Relays that
NAK every unauthenticated EVENT could therefore exhaust the budget and
drop the message before the AUTH handshake completed — the event was
gone by the time syncFilters re-sent it after the auth OK.
Treat `auth-required` as a deferred state instead: keep the relay in
relaysRemaining and record no failure, so the existing
syncFilters-after-auth path redelivers it. Mirrors the behavior already
present in StandaloneRelayClient. Terminal rejections (invalid/pow/
replaced/deleted) and ordinary transient errors are unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EZjmYpgHP4pf79Sav5QT8a
Findings from a full NIP-13 review:
- The spec recommends updating created_at while mining. Queue jobs now
re-stamp the template to "now" when a worker picks them up (a post can
wait behind other jobs, and a job restored after process death could be
hours old); the restorer does the same. Scheduled posts are exempt —
their future created_at is intentional. Anonymous posts re-stamp before
mining against the throwaway key.
- PoWTag.assemble(nonce, null) serialized the literal string "null" as
the third tag entry; a missing commitment now omits the entry entirely.
- New tests: PoWTagTest pins the NIP-13 example tag shape and the
no-commitment round trip; a queue test asserts the created_at
re-stamp at mining start.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ADb3dez9jPk6QqyQ1rTx4V
Implements user-facing PoW publishing (#3317) on top of the existing
quartz miner:
- quartz: PoWMiner.run gains a cooperative isActive cancellation hook
(checked every ~VALID_BYTES^2 hashes); PoWNostrSigner decorator mines
kind-scoped templates pre-signature so it composes with every signer;
GiftWrapEvent.create/NIP17Factory can mine the outer ephemeral-key
wrap (never the seal/rumor); NostrSignerWithClientTag exposes
prepareTags so mining runs over the final tag set.
- commons: PoWPublishQueue (FIFO, capped worker pool on
Dispatchers.Default, per-job cancel, in-memory only — unmined posts
are lost on process death, logged) and PoWPolicy (kind-group
categories with a hardcoded NEVER list: auth, zap requests, NWC and
bunker RPC, HTTP/Blossom auth, drafts, metadata and lists, OTS).
- amethyst: per-account synced settings (difficulty Off/16/20/24/28 or
custom, per-category checklist) in Compose Settings; Post enqueues the
template and returns immediately; reactions, reposts, reports, private
notes, DMs and long-form route through the same shouldMine gate at
their existing choke points; per-post PoW override chip in the
composer options row; "Mining proof of work… (N in queue)" phase with
per-job cancel in the broadcast banner.
Scheduled posts and anonymous posts mine against the correct key
(scheduled posts skip mining in v1); the client tag is applied to the
template before mining so signing never invalidates the nonce.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ADb3dez9jPk6QqyQ1rTx4V
Most of a from-scratch crawl's wall clock was connection setup, re-paid
serially: the relay pool tears down a relay's socket ~300ms after its last
drain unsubscribes, OkHttp allowed only 256 concurrent WS handshakes, and
every dial re-ran an uncached blocking getaddrinfo (10-30s per dead domain,
once per per-user path URL of the same host). Open the connections once, in
parallel, and keep them:
- GrapeRankCrawler: the warm-pool trick (never-matching REQ that only holds
the socket) now covers the whole candidate universe instead of the top 20 -
seeded at crawl start from the reachability cache's live set (one parallel
connection storm, Config.knownLiveRelays) and refreshed each round with
newly learned outbox relays, capped by Config.preconnectCap (FD-budget
aware, --preconnect-cap / --no-preconnect).
- CachingDns (quartz jvmAndroid): 10-min positive + negative DNS cache with
in-flight per-host dedup; dead domains fail in microseconds instead of
re-burning resolver timeouts, path URLs of one host resolve once.
- cli Context: dispatcher and pre-connect caps derived from the process's
open-files limit (UnixOperatingSystemMXBean), warning when ulimit is low.
- amy graperank probe: relay census - mass-connects every relay the store
knows (kind:10002 universe deduped per authority + cached verdicts) in
waves, records live/dead with real measured rtt-open into the NIP-66
reachability cache (RelayProber + RelayReachabilityStore.recordProbed),
so the next crawl skips dead relays and waits once for the slow-but-alive.
Single-server limits (FDs, ephemeral ports, DNS, threads, conntrack) and the
design are documented in quartz/plans/2026-07-10-graperank-connect-storm.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zEYRGKF943RgLaHTViJaB
A Nostr pubkey is x-only, exactly 32 bytes, but NPub.parse/NProfile.parse never
checked the length — they hex-encoded whatever bytes the bech32/TLV carried. A
malformed npub/nprofile that some clients encode with the full 33-byte COMPRESSED
secp256k1 key (0x02/0x03 prefix) therefore round-tripped its 66-char hex straight
into a `p`/`q` tag via the quote/mention path, and a strict relay (relay29 /
pyramid.fiatjaf.com) rejected the whole group message:
blocked: schema validation failed: tag[..]: invalid pubkey value
'02977dcf…c3402' ... pubkey should be 64-char hex
We never generate compressed keys ourselves (Nip01Crypto.pubKeyCreate strips the
prefix byte); this is purely inbound malformed input. Enforce the 32-byte length
at the decode boundary so the bad entity never becomes a mention/quote tag.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B5MLY4hq5LXJ2D5WeLRyXj
Two changes:
1. The reachability flush re-wrote the SEEDED known-dead relays with a fresh
created_at every run, refreshing their TTL without a re-probe — so a relay
marked dead once (and thereafter skipped, never re-dialed) would stay
blacklisted forever as long as crawls kept running, defeating the TTL's
re-probe. Stats.deadRelays now reports only relays actually dialed this run
(deadRelays - knownDeadRelays); seeded records keep their original timestamp
and age out on schedule so the next run re-probes them.
2. Rename GrapeRankDataCrawler -> GrapeRankCrawler (file + all references).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
Three regression tests covering the NIP-17 relay-hint contract just
introduced on GiftWrapEvent.create:
- default (no hint) emits the historical two-element ["p", pubkey]
shape — guards every existing caller against a wire-format
regression.
- with-hint emits ["p", pubkey, relay-url] — the canonical NIP-17
shape with the hint on the public wrap (NOT inside the seal, which
is the encrypted envelope and would hide routing info).
- null-hint must NOT produce ["p", pubkey, ""] — that would broadcast
"this user has no canonical inbox" as a metadata leak.
NIP-42 AUTH challenges arrive as `auth-required:` OK responses. Today
they accumulate via PoolEventOutboxState.newResponse → Tries.addResponse,
and after three of them the relay is silently dropped from the outbox on
the next newTry — even though RelayAuthenticator is concurrently signing
the AUTH event and the relay would have accepted the original publish
once authenticated.
Carve `auth-required:` out of the failure path: it's a "wait, AUTH in
flight" signal, not a rejection. The existing
RelayAuthenticator.checkAuthResults → client.syncFilters hook re-pumps
the outbox after AUTH-OK, so the original event is retried naturally.
Adds PoolEventOutboxStateTest covering the carve-out plus regressions
for regular rejections, terminal rejections, and success.
Audit of the authorsMissingOutbox anti-join surfaced one correctness bug and
one performance win:
- Bug (semantic): kind-1059 giftwraps store a random one-time key in
event_headers.pubkey (the real recipient is only a hash), so the query
returned an unbounded set of ephemeral keys that can never own a 10002 —
junk for the outbox model this feeds. Both the SQLite path and the generic
default now exclude kind 1059 from the "authors" set.
- Performance: replaced the DISTINCT + correlated NOT EXISTS scan with an
index-only EXCEPT (all authors minus 10002 owners). Both sides ride the
unconditional query_by_kind_pubkey_created covering index — so it does NOT
depend on the optional pubkey-alone index — and measured ~3x faster
(44ms vs 137ms at 152k events / 20k authors); the gap widens with author
count, since the old form paid one seek per distinct author. A loose-index
skip-scan was rejected: it needs the pubkey-alone index and degrades to a
full scan per author without it.
Also: corrected the KDocs (the old text implied an efficient index-only
distinct that wasn't guaranteed), added a giftwrap-exclusion test, and added
FsAuthorsMissingOutboxTest — the only coverage of the IEventStore DEFAULT
implementation, which EventStore always overrides.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CuLzfXyVZ16ozG8oJ7hBBc
Adds a whole-store query returning every distinct author with at least
one stored event that has NO NIP-65 relay list (kind 10002 / outbox).
This is a set-difference the positive-only nostr Filter grammar can't
express (there is no "NOT kind 10002"), so it lives as a dedicated
IEventStore method rather than a query(Filter). The interface carries a
correct default (collect authors-with-outbox, then stream events keeping
the rest — O(events)); SQLiteEventStore overrides it with a single
SELECT DISTINCT ... NOT EXISTS that seeks the outbox check on the
(kind, pubkey, created_at) index.
"Missing" is relative to what the store holds: an author whose only
10002 was deleted (NIP-09) or expired (NIP-40) is reported as missing
again, since no row remains.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CuLzfXyVZ16ozG8oJ7hBBc
Relay-signed kind-39000 has no author-of-a-follow, but the people dimension
still exists: a follow may be the relay signing key, a group admin (39001), or
a member (39002). Discovery now resolves each top-nav filter into a per-relay
GroupDiscoveryConstraint instead of collapsing every filter to the same REQ.
quartz:
- GroupMetadataEvent / EditMetadataEvent: build + read #t (topics) and #g
(geohash, mip-mapped so a coarser followed geohash still matches). Interop
tests for parse/build round-trips.
amethyst:
- dal/RelayGroupDiscoveryFeedFilter: sealed GroupDiscoveryConstraint
(AllGroups / ByPeople / ByHashtags / ByGeohashes / AnyOf) + toGroupConstraints()
mapping each IFeedTopNavPerRelayFilterSet to per-relay constraints, with
matches() covering the relay-key/admin/member people paths and topic/geo tags.
Unit tests.
- Directory REQ narrows to 39000 #t/#g for topic/geo filters, broad directory
otherwise (people match needs the rosters).
- ViewModel keys the feed on the constraint map and re-scans on any directory
event (metadata OR roster) so late-arriving admins/members surface groups.
- Create/edit form gains a Discovery section (topics + geohash) threaded through
Account.createRelayGroup/editRelayGroupMetadata and EditMetadata.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B5MLY4hq5LXJ2D5WeLRyXj
The drain classifier had two "act on it" verdicts, HARD (drop now) and TRANSIENT
(strike a few times, might clear). Re-probing hop-8's failed relays fresh showed
the TRANSIENT bucket almost never clears: 503 Service Unavailable 0/12 reachable,
502 Bad Gateway 3/15, connection-establishment failures 0/30; the codes that were
alive (402/403) are gated and will never serve us, and 200 isn't a relay. So the
extra dials TRANSIENT bought were spent on hosts that stay dead for the run.
Collapse to a single DEAD verdict, dropped on the first strike, and carve out the
only two connect failures that genuinely recover so they stay retryable (null):
a READ timeout (relay answered the handshake, slow — 67% reachable fresh, kept on
the clear-on-success authority-strike path) and an HTTP 429 rate-limit (alive,
4/4 reachable — retrying spaced by the limiter is how we get its data). Removes
the now-unused relayStrikes map, MAX_DEAD_STRIKES, and the HARD/TRANSIENT merge.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
A connect failure and a read timeout were both treated as "busy, retry" and
took three strikes to drop. Re-probing hop-8's failed relays fresh, outside the
crawl, showed the two are not alike: relays that failed to ESTABLISH a
connection (connect timed out, refused, unroutable, or the proxy couldn't tunnel
the CONNECT) were 0/30 reachable — genuinely dead — while relays that hit a READ
timeout were 12/18 (67%) reachable, alive but overloaded by the crawl's fan-out
(user.kindpag.es among them).
So classifyDrainFailure now returns HARD for connection-establishment failures
(one strike drops them instead of burning two more dials on a dead host), while
a read/generic timeout still returns null and stays on the patient,
clear-on-success timeout-strike path so live-but-slow relays we need are not
wrongly evicted. Mid-stream resets stay TRANSIENT. Adds DrainFailureTest, which
the classifier previously had none of.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
Reject a single-character group id (except the default `_`) so a possessive
glued to a bare relay URL — `wss://relay.damus.io's uptime` — no longer
linkifies group "s". Real ids (relay29/Wisp/0xchat) are all longer.
Adds coverage proving only genuine ws/wss relay URLs are peeked: apostrophes
after http, nostr:, blossom:, email and bech32 tokens never become group links;
plus ws:// (insecure), second-apostrophe boundary, and multi-link cases.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B5MLY4hq5LXJ2D5WeLRyXj
Recognise the de-facto `<relay>'<groupId>[?code=<code>]` NIP-29 group invite
link format used by Wisp and 0xchat, both inside rendered note content and as
an external deep link, so tapping one opens the group.
The URL detector correctly stops a host at the apostrophe (host names can't
contain `'`), so the group id is torn off before classification. Rather than
loosen the shared URL grammar — which would swallow prose possessives like
`example.com's` — group links are recovered by peeking just past each relay
URL the detector already found. This is cache-miss-only and costs nothing on
notes without a `wss://` link.
- quartz: GroupInviteLink.parse / suffixLength (+ tests)
- commons: Urls.groupLinks, UrlParser peek, RichTextParser plumbing
(atomic span through fixMissingSpaces, new RelayGroupLinkSegment) (+ tests)
- amethyst: ClickableRelayGroupLink renderer + uriToRoute deep-link branch
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B5MLY4hq5LXJ2D5WeLRyXj
classifyDrainFailure deliberately treats every timeout — connect timeout or
park idle-cut — as "busy, retry" and never dead, so a relay that connects but
never answers a REQ gets re-routed through every straggler's outbox, every
round, each visit burning the full timeout + park window for zero data. The
outbox model makes this worse: one dead server (e.g. filter.nostr.wine) is
advertised as hundreds of distinct per-user path URLs, so a per-URL counter
never reaches a threshold on any single one.
Count unproductive-timeout strikes per relay AUTHORITY (host[:port]) and evict
the whole host after Config.timeoutEvictStrikes (default 3; CLI --timeout-evict,
0 disables). Any clean EOSE or delivered event clears the authority, so only
never-productive hosts are evicted; a multi-path relay where some paths are
slow but others deliver stays live. Authority is host-only and never folds a
filter. subdomain into its parent, so an open bare host is untouched when its
sibling filter host is shed. Purely behavior-driven — no NIP-11.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSW59hJtP4Yn8fnRUxc7F5
Two compatibility gaps found analyzing nostrord (a NIP-29 client):
- Thread titles: NIP-7D (and Amethyst) use a `title` tag, but nostrord
writes/reads `subject`, so neither showed the other's thread titles.
ThreadEvent.title() now reads `title` OR `subject`; we still emit only the
spec-correct `title`.
- Joined-groups list (kind 10009): Amethyst wrote memberships as NIP-44
private items, but both reference clients (Flotilla, nostrord) store — and
nostrord only READS — public `["group", id, relay]` tags, so an Amethyst
user's groups were invisible to them. follow() (and the amy CLI) now write
public tags. NIP-29 membership is already public via the relay's kind-39002
list, so this loses no real privacy; reads still merge any legacy private
items so existing lists keep working.
Tests: read title from title/subject (title wins; we emit title only); public
group is a plain tag and still read through the cache; a mixed public+private
list reads as both.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B5MLY4hq5LXJ2D5WeLRyXj
Replying to a NIP-29 group thread went through the shared NIP-22 comment
composer, which built a plain kind-1111 comment with no group `h` tag and
broadcast it to the author's outbox. Such a reply is not group content: the
host relay rejects it and no other member — or other NIP-29 client like
Flotilla — ever sees it, and for a private/closed group it leaks to
unrelated relays.
Both fixes are tightly guarded on the replied-to event being group-scoped,
so ordinary comments are untouched:
- CommentPostViewModel inherits the group's `h` tag from the event being
replied to (covers replies to the kind-11 root and to nested 1111
comments — both route here).
- The reply is published only to the group's host relay (the relay the
thread was seen on) via signAndSendPrivatelyOrBroadcast, instead of the
outbox-computing broadcast — so it reaches the group and never leaks.
Verified: quartz test builds the reply as the composer does
(CommentEvent.replyBuilder { hTag } over a kind-11 root) and asserts it is a
1111 carrying the group `h` tag and referencing the thread.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B5MLY4hq5LXJ2D5WeLRyXj
Fixes found in a full audit of the NIP-29 relay-groups feature across
quartz/commons/amethyst/cli.
Correctness (app):
- Own group messages never appeared in the timeline until an app restart:
the optimistic send is consumed with a null relay, so attachToRelayGroup
bailed on the relay==null guard, and the host relay's echo (new==false)
was skipped by the "only attach when newly consumed" gate. Attach now runs
on every arrival, gated on the note being loaded, and the null-relay case
attaches to the already-open channel(s) for that group id. Also avoids the
wrong-relay phantom by only fabricating a channel from real provenance.
- Roster subscription was frozen after an in-place join/leave (state keyed
on the stable account, never re-derived); it now invalidates on every
liveRelayGroupList change, so a fresh join's 39002 admission is fetched.
- membershipOf demoted a 39001 admin with an empty/unknown role to MEMBER,
hiding moderation; presence in the admins list now means at least MODERATOR.
- Members roster showed permanent truncated-hex names (one-shot
getUserIfExists cached null); uses checkGetOrCreateUser so UsernameDisplay
fills in when kind:0 arrives.
Protocol / data:
- GroupTag had no value equality → joined-group Sets never deduped and the
StateFlow re-emitted on every identical re-arrival. Equality is now the
(id, relay) pair, excluding the cosmetic name.
- create/edit emitted non-canonical ["public"]/["open"] status tags; NIP-29
flags are presence-only, so only private/closed are emitted when set.
- Metadata/member/admin supersede guards use <= so an equal-createdAt
duplicate isn't reprocessed (first-arrival wins); updatedMetadataAt is now
private-set. Relay-group channels are now included in the prune loops.
CLI:
- join/leave/create updated the kind:10009 list from a network-only drain;
a slow/empty fetch could publish a fresh list containing ONLY the new
group, wiping the rest. Now reads the local store (source of truth) too.
- edit re-asserted both visibility axes from flag presence, so --closed on a
private group leaked it public. It now reads current 39000 and merges,
with --public/--open counter-flags; only the specified axis changes.
- create now tracks the new group in kind:10009 (parity with join/Android).
UI polish:
- Invite dialog no longer mints a 9009 for open groups and won't copy a code
it never displayed. Browse "popular" list normalizes URLs before filtering.
Tests: GroupTag identity, unknown-role-admin-moderates, equal-createdAt
no-resupersede added; all quartz+commons NIP-29 suites and the amy
relaygroup harness pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B5MLY4hq5LXJ2D5WeLRyXj
Audited Quartz's nip29RelayGroups package against every event and tag
Armada (gitlab.com/soapbox-pub/armada) creates and reads, and closed the
gaps so Quartz can round-trip all of them.
- GroupMetadataEvent (39000): fix flag parsing. `private`/`restricted`/
`hidden`/`closed`/`livekit` are single-element presence tags, but the old
accessors used hasTagWithContent (requires a value) AND conflated
restricted<->closed and hidden<->private with wrong defaults (an open
public group parsed as private+restricted). Now each flag is a plain
presence check. Add hasLivekit() and supportedKinds(), extend GroupStatus
with RESTRICTED/HIDDEN/LIVEKIT, and support supported_kinds in build().
- Add GroupParticipantsEvent (kind 39004, LiveKit AV presence) and register
it in EventFactory + KindNames.
- Add TagArray.hasTagName() presence helper in nip01Core.
- Add GroupScope.kt: hTag() builder + Event.groupId()/isGroupScoped()
readers so kind 9 (chat), 11 (thread), 7 (reaction), 1068, 1111 etc. can
be group-scoped via the `h` tag and read back — NIP-29 reuses these
carriers rather than defining new content kinds.
- Add Nip29ArmadaInteropTest covering parse + build for every event with
Armada's exact tag shapes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B5MLY4hq5LXJ2D5WeLRyXj
The web-of-trust crawl (~400 lines: outbox routing, sharded backbone sweep,
Phase-B worker pool, relay-list discovery, report-deletion fetch, warm pool)
was making the CLI's GrapeRankCommand unmaintainably large. Move it into a
reusable, KMP-portable GrapeRankDataCrawler in quartz commonMain.
The crawler takes a NostrClient + IEventStore + AdaptiveRelayLimiter, injected
relay policy (discovery + content-fallback sets, since those defaults live in
app code, not the protocol library), and a log callback; it streams contact
lists into a TrustGraphBuilder and returns crawl Stats. GrapeRankCommand shrinks
to arg-parsing + offline load + scoring + publish + sub-verbs, delegating the
online path to the crawler.
To reach commonMain (portable to every target, incl. iOS):
- Add ConcurrentMap / ConcurrentSet expect classes under utils/concurrent, with
jvmAndroid actuals (java.util.concurrent) and native actuals (copy-on-write
over kotlin.concurrent.atomics.AtomicReference, mirroring ConcurrentHashCache).
commonMain has no ConcurrentHashMap, and the crawl's producer/consumer/drain-
worker state needs atomic getOrPut/merge plus a concurrent set.
- Move AdaptiveRelayLimiter and DrainFailure/classifyDrainFailure from cli to
quartz commonMain (java atomics -> kotlin.concurrent.atomics, ConcurrentHashMap
-> ConcurrentMap, System.currentTimeMillis -> TimeUtils.nowMillis, stderr -> Log).
- The gated drain (REQ-size splitting, per-relay permits, verify+store) moves into
the crawler; Context.drain loses its now-unused gatePerRelay path.
Net: cli -1077 lines; the crawler + relay machinery are now reusable by the
Android app. Adds ConcurrentCollectionsTest; verified via JVM + commonMain
metadata compile, the wot/graperank suites, and a bounded live crawl.
BasicRelayClient collapsed a connection failure into a message string built
from the throwable's text alone. Message text is localized and inconsistent
across platforms, so a listener can't reliably tell a busy relay (a connect
timeout) from a dead one (bad domain / TLS misconfig) from it. Always append
the exception class name (SocketTimeoutException / UnknownHostException /
SSLHandshakeException / ConnectException …), which is stable, so listeners
can classify the failure by type. Message text is preserved; the type is
added in parentheses. Updated the one test that pinned the old format.
The GrapeRank engine, TrustGraph (compact int-CSR) and TrustGraphBuilder
are pure Nostr-social-graph computation over HexKeys — no UI, no Compose,
and no commons-only dependency. They're a utility for implementing the
NIP-85 rank assertions quartz already models, so they belong in quartz
rather than commons. Move commons/wot -> quartz experimental/graperank
(package com.vitorpamplona.quartz.experimental.graperank), including both
commonTest suites, and repoint the CLI import. TrustGraphBuilder was already
protocol-agnostic (takes HexKey lists; the caller does the event->edge
extraction), so nothing had to change but the package. Makes the algorithm
reusable by the Android app for spam/trust filtering without pulling in
commons.
Add Hex.toLong64/toLong128/toLong256 (plus the shared readLong helper) to
pack the first 64, 128 or 256 bits of a hex string into a single Long, two
Longs or four Longs. Big-endian, allocation-light, branch-free — 16 table
lookups and shifts per word. Useful as cheap map/set keys or bucket hashes
for 32-byte event ids and pubkeys without decoding to a ByteArray.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019CU1wR6NvQmdmNNsPe9GuN
- Use the app's cached stringRes helper instead of raw stringResource,
matching the dominant convention in ui/note/types
- Drive the icon animation from the Compose frame clock (withFrameMillis)
instead of a delay loop, so the ticker suspends whenever the composition
stops drawing rather than waking the main dispatcher 4x/sec from the
back stack
- Drop the unconsumed memoryCardId/blockState/blockHash accessors; the
tag schema stays documented in the class KDoc
- Document the frames arrays as frozen: mutating them in place would
silently break the @Immutable skip contract; build a new instance
to change pixels
- Replace the API-29-deprecated Bitmap.createBitmap(IntArray, ...)
overload with createBitmap(w, h, config) + setPixels
Bulk NDJSON import/export as first-class geode subcommands, mirroring
`strfry import` / `strfry export` (one JSON event per line — the interchange
format for seeding a relay, migrating between relays, or taking a backup):
geode import [--db …] [--no-verify] [FILE…] # files, or stdin when none
geode export [--db …] # NDJSON to stdout
Both stream — memory is bounded to one batch (import) / one event (export), so
a multi-million-event corpus round-trips in roughly constant memory. `import`
verifies signatures by default (same `Event.verify()` the relay's VerifyPolicy
uses), upholding the relay's verify-by-default stance rather than trusting the
file; `--no-verify` is the trusted-input escape hatch. Verb dispatch is
backward-compatible: a bare `geode --port …` (no verb) still serves.
This makes the benchmark-only `CorpusServerMain` redundant — a corpus source is
now just `geode import` into a DB, then a normal `geode` serve — so it's
deleted, removing benchmark-only code from the production geode artifact (the
question that started this). The 1M sync-throughput plan is updated to describe
sources via `geode import` + serve.
Also fixes a native-target CI break: MergeQueryCorrectnessTest used the
deprecated `String(CharArray)` (error-level on Kotlin/Native) — switched to
`CharArray.concatToString()`.
Verified end-to-end through the packaged `geode` binary: import (file + stdin,
--no-verify), export round-trip, and verify-on rejecting bad signatures.
ImportExportTest covers the counts, duplicate handling, malformed-line
skipping, and verify accepting a freshly-signed event while rejecting bad sigs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
MergeQueryExecutor's winner-picker tie-breaks equal created_at by id ASC,
but each per-stream cursor sorted by created_at DESC only, and repeated
authors/kinds opened duplicate cursors:
- id tie-break: thread the IndexingStrategy through run()/prepareStreams
and append ", id ASC" to the per-stream ORDER BY when
useAndIndexIdOnOrderBy is set — matching every sibling query in
QueryBuilder. The id-indexed order comes straight off the index (no
extra sort, lazy cursor preserved), so the merge now matches the
single-SQL path byte-for-byte on same-second same-author events. Without
the id index the tie stays in rowid order (a valid NIP-01 newest-N);
documented on the class.
- dedup: streamCount/prepareStreams now operate on distinct authors and
kinds, so a filter with a repeated pubkey can no longer open two
identical cursors and emit each matching event twice (the single-SQL
IN(…) path already dedups).
Adds two MergeQueryCorrectnessTest cases the suite was missing: a
within-stream same-second tie sliced by the limit, and duplicate authors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Op instances key Timestamp.ops (MutableMap<Op, Timestamp>), so contract
violations corrupt hash-map behavior:
- OpKECCAK256 defined equals without hashCode, so equal instances hashed
by identity — two equal keys could land in different buckets, producing
duplicate branches or failed lookups in keccak256 timestamp trees. Add
hashCode = TAG, mirroring OpSHA1/OpSHA256/OpRIPEMD160.
- OpBinary defined hashCode without equals — and its TAG referenced
Op.TAG (0x00), a no-op XOR. Define the equals/hashCode pair once on
OpBinary using tag() and drop the duplicated overrides from
OpAppend/OpPrepend (behavior unchanged: same tag + same arg content).
- VerifyResult.equals cast without a type test (ClassCastException on
foreign types instead of false) and hashCode force-cast the nullable
timestamp (NPE for null-timestamp results). Convert to a data class;
the custom toString and compareTo stay.
The `rawQueryPathMatchesDecodedQuery` case called `store.rawQuery(filter)`, but
`EventStore` only exposes the streaming `rawQuery(filters, onEach)` — the
list-returning overload lives on the inner `SQLiteEventStore`. Point the check
at `store.store.rawQuery(filter)` so the zero-decode path is actually exercised.
Record the shipped k-way merge result in the plan doc: a fresh 1M relayBench run
has geode `follow-feed` at 18.8 ms vs strfry 17.7 ms (down from 97.7 ms, now at
parity) and 46,258 ev/s vs strfry 15,365 @8conn, both returning the same 500.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
The home-feed REQ (`authors=[…] (+ kinds=[…]) [+ since/until] limit=N`,
newest-first) is one of the most common relay queries. SQLite serves it by
seeking every `(kind, pubkey)` combo and feeding *all* matching rows through a
LIMIT-bounded sorter, so it reads O(the followed set's whole matching history)
— on a cold on-disk 1M corpus that was the `follow-feed` regression (relayBench:
97.7 ms vs strfry 17.6 ms).
Add `MergeQueryExecutor`, an app-level k-way merge that opens one lazy
newest-first cursor per stream off the existing composite indexes
(`query_by_kind_pubkey_created`, or `query_by_pubkey_created` for authors-only),
merges their heads `(created_at DESC, id ASC)` and stops at the limit — reading
only O(limit + streams) rows regardless of how much history the authors have.
It reuses indexes that already exist, so write throughput and on-disk size are
untouched. Eligibility is narrow (2..2048 streams, simple filter, explicit
limit, no ids/d-tags); everything else falls through to the single-SQL plan.
Wired into both `query` and the zero-decode `rawQuery` paths (the relay REQ hot
path) and the single-element filter-list variants, so `LiveEventStore` REQs go
through it.
`MergeQueryCorrectnessTest` proves the merge returns exactly the single-SQL
top-N — vs an independent Kotlin reference and vs the SQL path — across
distinct/tied created_at, since/until windows, authors-only, fewer-than-limit,
streaming onEach, and the raw path. `FollowFeedReadBenchmark` gains a `merge`
variant: at 1.05M events it's flat ~10-12 ms across both prolific-recent and
sparse-old, where `scan` is catastrophic on sparse follows (1995 ms) and
`current` is disk-bound on prolific ones.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
- only parse commonName when the alt tag has the Birdstar prefix
- summary() on both Birdstar events now uses the canonical NIP-31
tags.alt() helper instead of a raw firstTagValue("alt") lookup
- speciesReference() only returns http(s) URLs since UIs render it as
a clickable link (rejects e.g. javascript: schemes), with a test
- Detection card: parse tags once into a single remember slot, drop
the near-dead '?: summary' title fallback, stop rebuilding the
italic TextStyle every recomposition
- Hoist the duplicated bird-emoji literal into a shared BIRD_PREFIX
- Trim the redundant factory test to the assertIs idiom
- fetch Birdex life lists in home and profile relay REQs
- richer Birdstar cards — common name title, Wikidata link, bird emoji
- surface Birdstar bird detections in home and profile feeds
ingest() bypasses the per-connection policy chain, which is where
VerifyPolicy lives. The IngestQueue verify hook only exists when
parallelVerify is true, so with parallelVerify = false and
skipVerify = false, ingest() previously verified nothing — an untrusted
mirror upstream on a relay running the legacy in-policy verify path could
inject forgeries. ingest() now verifies inline in that configuration
(same rejection reason as the queue), so the documented "default keeps
verify-everything semantics" holds regardless of parallelVerify. KDoc
also spells out that ingest() skips the entire policy chain (blacklists,
size limits), which callers must screen for themselves.
Test covers the parallelVerify = false server: forged rejected, trusted
skip and valid still land.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Audit follow-ups on the live NIP-77 index, all with the store/scan
equivalence test extended to cover them:
- Same-batch replaceable displacement left a dead id in the index.
applyAfterCommit applied all removes before all adds, so when a later
row in one transaction displaced an earlier row of the same batch (two
versions of one replaceable — the mirror-backfill hot path), the
displaced row's remove no-op'd against an index that hadn't taken its
add yet, then the add re-inserted it: the index advertised an id the
trigger had already deleted. recordAccepted now cancels the pending add
instead of queueing a remove (added is a LinkedHashSet for O(1)
cancel).
- A kind-5 that deleted nothing (a delete broadcast for events this relay
never stored — the common case) still invalidated the whole index,
forcing a full-scan rebuild under the writer mutex on the next
NEG-OPEN. DeletionRequestModule.insert now returns the rows it deleted;
recordAccepted only invalidates when that count is > 0, else records the
kind-5 as a plain row.
- The first NEG-OPEN over a corpus larger than the serve cap scanned the
whole table uncapped, built a full index that could never produce a
snapshot, and then maintained it forever for zero benefit.
liveNegentropySnapshot now caps the rebuild scan at maxEntries + 1 and
leaves the index unpopulated when the corpus is over-cap (the scan path
answers NEG-ERR, as before).
- delete/deleteExpired/clearDB invalidate() moved inside the writer mutex
so no NEG-OPEN can seal a snapshot of just-deleted rows, and no
concurrent rebuild can be discarded by a late invalidate.
Also corrects the ~40 B/event heap figure to ~140 B (IdAndTime keeps the
id as a 64-char hex string, not 32 bytes) in the strategy/plan docs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Backlog items 4-5 plumbing, config-gated and off by default so quartz
library defaults stay untouched for the app-side stores:
- quartz: SQLiteEventStore/EventStore accept extraPragmas (applied on
every pooled connection AFTER the built-in configuration, so they
can override it) and expose optimize() — an analysis_limit-bounded
PRAGMA optimize for incremental planner-stats refresh.
- geode: [database] readers / mmap_size / temp_store_memory map onto
the store; optimize_interval_seconds drives a maintenance coroutine
(cancelled first in the shutdown hook, before the store closes).
The A/B verdict on whether the example config should RECOMMEND any of
these on container-class hardware follows in the next commit — the
knobs themselves are operator tools worth having either way, since
mmap/temp-store value is hardware-dependent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Reverts the queryRawInline fast path (fb29d655, 1b786f31) per the
keep-only-winners rule. Three relayBench runs at 50k (baseline, cap-256
where the path never engaged, cap-512 where author-archive/by-ids/
500-limit feeds genuinely took it) showed no movement outside the
container drift band — strfry's own numbers drifted ±30% between runs
and inline-eligible scenarios moved the same as ineligible ones.
The in-process win was real but small (~17%, 0.60 -> 0.50 ms per
~21-row REQ); the wire-level p50 is 1.2-1.7 ms, so the missing ~1 ms
per REQ sits in the transport (Ktor frame send path + client round
trip) — backlog item 6 territory, not dispatch. Findings, numbers, and
the do-not-retry note live in quartz/plans/2026-07-04-small-req-floor.md;
SmallReqFloorBenchmark stays as the measurement tool.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Claude
Vitor PamplonaandGitHub
nrobi144
davotoula