A kind-0, limit-1 REQ works everywhere: purpose relays (purplepag.es)
reject kind-less filters outright, and practically every relay stores
some profile. Verified against production — purplepag.es's read side now
measures instead of going unobserved. Pass a different kinds list to
probe a specific shelf, or null for a kind-less query on relays known to
allow one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9sSyh1QLJD3PZ18tVPPVK
Verified the new probe surface end-to-end against production relays
(probeFlow streaming, readWriteCheck, signed 30166 templates). One
compatibility finding: purpose relays like purplepag.es reject any REQ
that names no kind ('blocked: filters must specify at least one kind'),
leaving their read side unobserved. readTestFilter/readWriteCheck now
take an optional kinds list for those; the default stays kind-less
because naming kinds also narrows the query on every other relay.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9sSyh1QLJD3PZ18tVPPVK
Audit findings across the branch, each verified with a failing test or
measurement before the fix:
- publishAndCollectResults counted an OK from a relay OUTSIDE relayList
(same event id — a probe-wave straggler, or any republish of the same
event to a different relay set) toward its confirmation window, ending
the wait loop early and misreporting still-pending listed relays as
NO_RESPONSE. The OK branch now carries the same relayList guard the
onCannotConnect/onDisconnected branches always had. Regression test
proves the failure without the guard. readWriteCheck additionally
varies the probe event content per wave so wave N's confirmation window
can never match wave N-1's event id at all.
- RelayUrlNormalizer.fix() called trimEnd('%','2','0') unconditionally,
allocating a full string copy for ANY url merely ending in '%', '2' or
'0' — which includes every relay port ending in zero (wss://host:3030).
Now gated on endsWith("%20"), keeping the hot path allocation-free;
semantics unchanged (test pins both the trim and the untouched-port
cases).
- amy relay probe --file: unreadable file is now a clean bad_args error
instead of a stack trace, and skipped onion urls are counted and
reported (file_onion_skipped) instead of vanishing from the tally.
- probeFlow KDoc now states that a slow collector eats into the current
wave's absolute deadline (answers are still recorded; silent relays get
less listening time), not just that it delays the next wave.
Verified non-issues: androidx.collection LruCache is internally locked
(safe for CachedNip11Fetcher/normalizer concurrency); probeWave's
per-terminal emission cannot lose or double-emit verdicts (remaining-set
guard, data maps read at emission time); existing publish callers all
benefit from the OK guard rather than depending on the old behavior.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9sSyh1QLJD3PZ18tVPPVK
`needsToResendRequest(List, List)` used a non-local `return` inside
`forEachIndexed`, so the loop always returned on iteration 0 and only
`filters[0]` was ever compared. A subscription whose first filter happened
to be unchanged reported "no resend needed" however much the rest had
changed, leaving the relay serving a stale filter set and the app silently
missing events. Only the size check offered any protection, so the bug was
invisible whenever the filter count stayed constant.
Replaces the loop with an indexed scan over all filters, which also drops
the lambda allocation and matches the hot-path style in this package.
Adds FiltersChangedTest. 3 of its 9 cases fail on the unfixed code — all of
them changes beyond index 0 — while the other 6 pass both before and after,
pinning the blast radius to exactly the buggy behaviour. Coverage includes
the deliberate `since`-moves-forward exemption, which must not trigger a
resend on any index.
Note for reviewers: PoolRequests.kt:490 and :528 use this inverted as a
"same as last" refusal check, so those become stricter — filter sets that
differ only beyond index 0 were previously treated as identical and will
now correctly be treated as changed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
RelayProber.readWriteCheck(relays, signer) is the deeper check pair for
relays already proven live (warm sockets from a probe that just ran):
- READ: a real limit-1 REQ the relay must query its store for, timed
REQ→first answer (honest rtt-read on an open socket).
- WRITE: one ephemeral RelayProbeWriteTest event signed by the monitor
key, timed publish→OK (honest rtt-write). An OK false is a measured
policy answer, kept with its NIP-01 machine-readable reason; only
silence leaves the write side unobserved (writeAccepted = null).
publishAndCollectResults now stamps each OK with its elapsedMs (a
rejection is still a round trip; -1 when the relay never answered), so
any caller gets write latency for free.
toDiscoveryEventTemplate(readWrite = ...) folds the pair into the 30166
template: rtt-read/rtt-write when measured, R auth / R pow when the
write was refused with auth-required:/pow:. NIP-11-derived tags (N
supported NIPs, k kinds, T type) are deliberately NOT emitted — those
are relay self-claims, and publishing them under a monitor signature
without per-NIP compliance tests would launder claims into
measurements. Per-NIP/per-kind compliance suites can come later as
opt-in checks; open/read/write is the default surface.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9sSyh1QLJD3PZ18tVPPVK
RelayProber.probe()/probeFlow() take a filters option choosing the check:
LIVENESS_FILTERS (default, impossible-id REQ — EOSE proves liveness with no
payload) or readTestFilter(limit = 1) — a REQ the relay must actually work
for, querying and streaming real events, making Verdict.rttEoseMs a genuine
read test.
RelayProbeWriteTest.build() creates the write-check event: ephemeral kind
20166 (never stored by compliant relays) carrying a NIP-40 expiration tag
60s out as belt-and-braces for relays that store unknown ephemeral kinds.
Publish it under the monitor key, time the OK for rtt-write, map rejection
prefixes to R requirement tags — an OK false still proves the write path.
Nip11Fetcher is the missing fetch seam for relay information documents,
mirroring Nip05Fetcher: the interface lives in commonMain,
OkHttpNip11Fetcher (jvmAndroid) does the Accept: application/nostr+json
GET, and CachedNip11Fetcher wraps any implementation with a TTL cache —
successes trusted for a day, failures remembered for five minutes so a
census doesn't hammer hosts that just refused.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9sSyh1QLJD3PZ18tVPPVK
RelayProber.probeFlow(urls) is a cold Flow that emits each relay's Verdict
the moment the relay resolves (EOSE, CLOSED or connect failure) instead of
at the end of the whole census; only silent relays wait for their wave's
deadline. probeWave now resolves verdicts per-terminal, so the batch
probe() shares the same path.
Verdict.toDiscoveryEventTemplate() renders a verdict as an UNSIGNED
kind:30166 template (d = normalized url, n network type, rtt-open when
reachable, R auth when the probe hit a NIP-42 auth-required CLOSED) so an
external consumer signs with its own monitor key:
prober.probeFlow(urls).map { it.toDiscoveryEventTemplate() }
.collect { publish(signer.sign(it)) }
rtt-eose is deliberately never published as rtt-read: it is measured from
the wave start (dial + TLS + queueing + read), and aggregators rank on
rtt values. The RelayObserver/RelayMonitor path supplies honest
rtt-read/rtt-write from real traffic.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9sSyh1QLJD3PZ18tVPPVK
Validated against a 45k-entry corpus of relay-url hints exported from real
events (317k tag occurrences). The normalizer was converting ~30k distinct
https:// urls with paths (Mastodon/bridge actor urls from proxy tags, web
pages, images) into wss:// addresses that can never answer, wasting
connection attempts and relay-pool slots.
- http(s) → ws(s) scheme swap now only applies to bare hosts
(host[:port] plus optional trailing slash); an http url with a path,
query or fragment is a web resource, not a mistyped relay.
- Authority validation for all schemes: rejects empty hosts, userinfo
(@), percent-encoding and commas in the host, and paths that start
with // (the signature of a second pasted url, e.g. wss://https//host).
- Interior whitespace and backslashes reject the whole string (multiple
urls or prose in one field).
- Zero-width characters (U+200B..D, U+2060, BOM) are stripped instead of
corrupting the parse (wss://\u200Bnos.lol previously normalized to the
scheme-less //nos.lol/).
- Schemeless candidates must look like host[:port] (single colon, numeric
port), rejecting addressable pointers (31990:pubkey:dtag) and bare
scheme leftovers (wss:) before the expensive RFC 3986 parse.
- Protocol-relative //host/ inputs normalize as wss:// instead of
resolving to https://.
- normalizeOrNull now double-checks the parser output still starts with
ws(s):// and rejects otherwise.
Corpus impact: 30,014 garbage urls (30,333 events) now rejected, 0 real
relays lost (all 15,162 kept urls normalize byte-identically), 6 broken
outputs fixed. fix() itself stays allocation-free on the happy path
(~357ns vs ~318ns per call on the garbage-heavy corpus).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9sSyh1QLJD3PZ18tVPPVK
quartz:
- SQLiteEventStore: classify per-row savepoint errors — policy refusals
(blocked:/constraint/not allowed) stay Rejected, everything else is now
Failed, so disk-full no longer masquerades as 2M duplicate rejections
- IEventStore.batchInsert default: rethrow CancellationException and map
unknown throws to Failed (re-offering a duplicate is idempotent;
dropping a good event on a transient store error is not)
- IngestQueue: rethrow CancellationException instead of stamping a
cancelled batch Failed and continuing
- HostStrikes: make the eviction verdict exactly-once under concurrency
(deadHosts.add is the atomic gate) and re-check produced before
publishing
- SyncCoverage: bound the identity fingerprint cache (a caller minting
fresh Filter instances per cycle could grow it forever); legs() gains a
floor parameter so a complete band re-opens its older span when the
caller's window deepens; coveringWindow no longer treats a fully
covered relay as needing the whole filter
- PagingWindowProgress: accept single-second windows (a band's re-read
edge leg is exactly that shape)
geode:
- MirrorWorker: cap reconciledThrough at the leg's own ceiling — the
older leg of a resumed catch-up no longer stamps the band complete
through 'now' before the newer leg has run (silent event loss for up
to fullResyncSeconds if that leg failed)
- MirrorWorker: run negentropy and the paged fallback by hand instead of
negentropySyncOrFetch: drops the O(delivered-ids) dedup set from the
mirror path, and a fallback resets the observed span so a band never
claims interior ranges only a half-finished reconcile scattered over
- MirrorWorker: clamp a paged band's ceiling to the snapshot instant so
one future-dated event cannot suppress the next boot's newer leg
- MirrorWorker.close(): join the workers (bounded) so the final coverage
flush carries the last records
- Main: gate the coverage file on the store actually being persistent —
database.file with in_memory=true (the default) persisted bands over a
volatile store, and the next boot skipped the backfill over an empty
database; honor --db overrides
- SyncCoverageFile: request ATOMIC_MOVE explicitly; fix the restore/dirty
comment
- Import summary now prints the failed count; document
mirror_sync_state_file in config.example.toml
Three semantics rules each existed as multiple independent copies:
- Event.owner() (gift-wrap recipient controls the wrap, else the
author — NIP-09/62 authority) was derived inline in
EventIndexesModule and again in EventStoreProjection.ownerOf.
- The NIP-01 replaceable tiebreak (newest created_at, ties to the
lexically smallest id) lived in EventStoreProjection.supersedes,
in SQL, and downstream.
- "Indexable tag name" was spelled `length == 1` in four places,
which admits "5" and "#" — names the NIP-01 #x filter space
(single a-zA-Z letters) cannot address, letting stores disagree
about which tags filters reach. isIndexableTagName encodes the
NIP-01 rule; converging FilterIndex and the SQLite
IndexingStrategy on it deliberately tightens single-char
non-letter tag names out of the index.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MfwV3xgMSmfxy16ujGPxGW
An all-dash search token ("--") stripped to an empty exclusion that
toSearchString/stripExtensions round-tripped into a REQUIRED "-" term
reaching SQLite FTS; it is now dropped at parse. IngestQueue's
batchInsert fallback was the one site still hand-writing "insert
failed" (unprefixed) while every other path emits
RejectionReason.INSERT_FAILED. RejectionReason no longer duplicates
the NIP-01 prefixes MachineReadablePrefix already owns, and the
expiration trigger now rejects with the same words as the Kotlin
pre-check instead of its own spelling. Tests pin the "--" drop,
consecutive quoted spans, text after a closing quote, the extractor's
fallback tier, blank-content normalization, and the
unparseable-buzz-content hashtag seam; extractor KDoc now states
where the trimmed/non-empty and never-empty-Profile guarantees
actually live.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MfwV3xgMSmfxy16ujGPxGW
SearchableEvent.indexableContent() flattens a kind's searchable text
into one blob, so a weighted full-text backend (title above summary
above body) had to re-derive the decomposition itself — and drift
every time a kind's parsing changed here. SearchFieldExtractor now
lives beside the kinds it decomposes: title-like accessors primary,
summary/description secondary, body tertiary, kind-0-shaped metadata
in profile roles, with an indexableContent() fallback so every
searchable kind, current or future, is covered.
IndexableFields is a sealed shape — Profile or Tiered — so a kind
cannot mix identity fields with content tiers, and each shape
declares its own website role (a profile's homepage; a content kind's
affiliation URLs). Multi-valued roles are carried UNJOINED, as lists:
hashtag and location tags ride raw beside the tiers (filled by the
one tiers() funnel every content branch uses, so no branch can forget
them and profile shapes never see them), and separator or weighting
choices — hashtags at summary weight, "\n" vs " ", arrays vs joined
columns — belong to the backend, not the library. Empty extractions
always normalize to None.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MfwV3xgMSmfxy16ujGPxGW
Renames from review: SyncBands -> SyncCoverage ("sync" reads negentropy-ish
in quartz, and coverage is the role — the bands are the records), and
PagingProgress moves into relay.client.paging as PagingWindowProgress,
beside RelayLoadingCursors and RelayPagingProgress, with its docs swept from
"walk" dialect to quartz's pagination vocabulary and cross-references
delineating the three: cursors are in-memory positions for demand-driven UI
paging, the window progress is fraction/ETA for a bulk pagination over a
known window, coverage is persistent intervals that license skipping work.
geode adopts both halves of the new contract. MirrorWorker counts
InsertOutcome.Failed in its own `failed` counter instead of folding it into
`rejected`, and the down catch-up gains resume memory: SyncCoverageFile
persists SyncCoverage next to the event database (admin state-file
convention, temp-file + atomic move, daemon flush), and runCatchUpDown asks
only for the legs outside the covered band. Bands are keyed on the stable
scoped filter — never the boot window, whose since/until change every start
— and clamped to the window, which only slides forward, so an old band can
never license skipping a range an earlier boot could not ask about. A clean
reconcile records completeness through its snapshot instant; a paged
fallback earns only the span it saw. For an upstream without NIP-77 this
turns the every-boot full re-download of the backfill window into a
resumed walk.
Off unless wired: MirrorWorker's coverage parameter defaults to null and
in-memory stores keep no state file, so existing tests and setups are
unchanged. Full :quartz:jvmTest and :geode:test pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y4Pi9YYMhdzTFxRiV2jF9R
Bisecting existed to reconstruct per-event attribution after a store threw a
batch-wide exception. Better to never lose the attribution: batchInsert's
contract now requires per-row isolation, with a third outcome telling whose
fault a miss was. Rejected is the EVENT's fault (duplicate, expired, invalid,
blocked) and is final; Failed is the STORE's fault (schema drift, a failed
feed, a resource error) — the event was good, it is lost unless re-offered,
and a rising Failed count means the store is broken rather than that
upstreams send junk. Throwing is reserved for failures with no per-event
answer (engine unreachable, transaction never started), readable as "nothing
in this batch was written".
Consumers updated: RelaySession maps Failed to OK false with NIP-01's
"error:" prefix; IngestQueue converts a thrown batch and a missing outcome to
Failed instead of Rejected; NdjsonImportExport counts failed apart from
rejected; geode's MirrorWorker logs store failures at warn instead of
folding them into debug-level rejections. BisectingInsert and its test are
removed — with attribution guaranteed by the contract, retry-by-splitting
has nothing left to do.
Full :quartz:jvmTest passes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y4Pi9YYMhdzTFxRiV2jF9R
NIP-50 search strings in the wild carry Google-style syntax the
extension tokenizer could not see: "exact phrase" requirements,
-"phrase" and -word exclusions. SearchQuery now lifts quoted spans
BEFORE the extension pass — the order is load-bearing: the extension
pass is quote-blind, so a span ending in an extension-shaped token
would lose its closing quote, and lifting first also lets quotes
protect extension-shaped tokens ("include:spam" is a phrase, not an
extension). toSearchString() and stripExtensions() reassemble the
full grammar; existing parses are unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MfwV3xgMSmfxy16ujGPxGW
Every store spelled its rejection reasons inline — the expired-event
string was duplicated four times across ObservableEventStore and
SQLiteEventStore. RejectionReason now carries the NIP-01 OK
machine-readable prefixes plus the standard store reasons, so
InsertOutcome.Rejected tallies and OK-frame building see one
vocabulary no matter which store produced the outcome.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MfwV3xgMSmfxy16ujGPxGW
Four pieces extracted from a production Nostr mirror (NosFabrica/vespa-relay),
generalized to quartz's multiplatform primitives, with their test suites:
- nip01Core.store.insertBisecting: batchInsert fails as a unit, so one bad
event costs its whole batch (999 good events per bad one at a 1000-event
batch). Bisecting isolates the offender in ~2*log2(n) extra writes, and a
fixed write budget keeps a store-wide failure (full disk, dead engine) from
turning one failed write into ~2n.
- accessories.SyncBands: resume memory for fetchAllPages. Remembers the
created_at band covered per (relay, filter) and asks only for the legs
outside it, with inclusive edges so a page boundary cannot strand a run of
same-second events. A finished negentropy reconcile records completeness
through its start instant; a periodic full re-walk keeps stale claims from
narrowing forever. Persistence is the caller's, via export/restore and an
onChange hook.
- accessories.PagingProgress: progress for paged walks measured on the time
axis, the only axis whose end is known in advance - count-based percentages
degenerate to downloaded/downloaded = 100%. Needs no COUNT support.
- nip66RelayMonitor.reachability.HostStrikes: per-authority strike counting
for outbox-scale fan-outs, where a filtering relay mints one url per user
and per-url counters never converge. Ever-delivered overrides eviction in
both race orders, and eviction surfaces exactly once for publishing.
reachability.Unreachability (jvmAndroid): which failures may be published
as a signed NIP-66 unreachable record - connection-level only, so a relay
that answered the handshake and hung up mid-page is never libelled, and a
caller's own bug is never the relay's fault.
57 tests pass on the JVM target; the common code uses quartz's ConcurrentMap,
ConcurrentSet and TimeUtils only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y4Pi9YYMhdzTFxRiV2jF9R
A production ANR on a Pixel 8 (Amethyst 1.13.1, anr_2026-08-03-12-55-26-256)
showed the app burning 596% CPU — 6 of 9 cores — with the main thread stuck in
WaitingForGcToComplete. 37 of 52 runnable DefaultDispatcher workers sat at ONE
program point inside PoolRequests.onIncomingMessage and 12 more at one point in
syncState, all state=R, while the single thread actually holding the lock was
itself parked in GC.
Root cause: RequestSubscriptionState.withLock was a raw busy-wait
(`while (lock.exchange(true)) { while (lock.load()) {} }`) with no yield or
backoff, and being `inline` it disappeared into its callers' frames. The lock is
per subId, but one subId spans every relay it runs on — 191 live sockets on that
device — so dozens of relay-dispatch threads piled onto a single AtomicBoolean.
Spinning is only correct when the holder cannot be descheduled; on Android it
always can.
The fix stripes the lock per (subId, relay) rather than making waiting cheaper.
All 11 withLock bodies in PoolRequests are already scoped to exactly one relay,
and every field of RequestSubscriptionState is keyed by relay, so the sharing was
purely an artifact of mutableMapOf not being thread-safe. State moves into a
ConcurrentMap<T, RelayState>; locks live in a fixed 32-entry stripe array that is
never mutated, so lock identity stays stable — if locks lived inside the map
values, a thread holding one while another dropped and re-created that entry
would leave both inside the critical section excluding nothing.
A suspending Mutex was measured and rejected: it needs 262 method overrides and
110 call sites to become suspend, and ran at 0.35-0.63x the current throughput.
Measured (LockDesignComparisonBenchmark, 191 relays / 64 dispatcher threads):
striped vs per-sub lock 1.5-2.8x throughput, bystander p50 halved
On device (SM-T220, playBenchmark, same account, n=3 per design):
DefaultDispatcher CPU -35% mean / -30% median vs the spin lock,
with non-overlapping ranges; GC -18%
Plus 10 min of driven UI stress (feed, profiles, chat, notifications,
communities): no ANRs, no crashes, thread pools stable.
Also here:
- PlatformLock: new expect/actual parking lock (ReentrantLock on jvmAndroid,
NSRecursiveLock on Apple, spin only on linuxX64 which is a CI target). quartz
cannot use commons' equivalent KmpLock because commons depends on quartz.
- LiveNegentropyIndex had the identical busy-wait with a full list SORT inside
the critical section; switched to PlatformLock.
- ConcurrentMap.remove (+ tests), with a caution that a removable value must not
own a lock callers acquire.
- SpinLockConvoyBenchmark: regression guard asserting contended waiters PARK
rather than spin (fails-before / passes-after). Pure benchmarks are gated
behind -PprodRelayBench=1, so CI cost is 0.3s rather than 51.5s.
Analysis and measurements: quartz/plans/2026-08-03-poolrequests-lock-contention.md
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Audit follow-up on the timeout work.
- fetchAllPages matched each event with `for ((i, f) in activeFilters)`.
That destructuring form allocates an Iterator on every event, on the
relay's reader thread, for the whole download -- millions of short-lived
objects in a bulk walk. Switched to an indexed loop, which is why quartz
uses the fast* operators elsewhere in hot event paths (those only cover
Array, so a List needs the index form).
- count() and fetchFirst() drain their channels in an inner loop that only
suspends when the channel is empty, so a backlog was consumed with no
cancellation check: neither the idle window expiring nor the caller
giving up could interrupt it mid-drain. Added an explicit ensureActive(),
matching the check fetchAllPages already does per page.
- count() could also lose a result that arrived after the last window
closed but before unsubscribe, understating the returned map. Added the
post-loop tryReceive drain that fetchAllWithHooks and fetchFirst have.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KZe1Pq2ejoHehdufhPDRgb
Every wait in the accessories package is an idle window measured from the
relay's most recent progress, so the parameter now says so. The name is
the contract: a caller reading timeoutMs reasonably expects a deadline,
which is exactly the misreading that made fetchAllPages' hard per-page
cap look correct for so long.
Renamed across the public surface -- fetchAll (7 overloads),
fetchAllWithHooks, fetchAllPages (2), fetchAllPagesFromPool,
fetchAllPagesFromPoolWithHooks, fetchFirst, count (2), countMerged -- plus
the quartz wrappers whose own parameter is a pure pass-through of that
window (KeyPackageFetcher, RecipientRelayFetcher, FollowerCrawler.Config)
and every call site across commons, cli, desktopApp and amethyst.
Deliberately NOT renamed, because these are genuine wall-clock bounds and
the differing name is the tell:
- publishAndConfirm's timeoutInSeconds -- one fixed window to collect
the OKs, a bounded confirmation round-trip rather than a stream.
- GrapeRankCrawler.Config.timeoutMs -- a hard per-drain gate
(withTimeoutOrNull(config.timeoutMs)) that also drives parking.
- Context.awaitReply's timeoutMs, Context.syncIncoming, and the
non-accessory app-layer helpers (RelayProber, FeedMetadataCoordinator,
RelayAuthPromptBus).
This is a source-breaking change for named-argument callers of quartz.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KZe1Pq2ejoHehdufhPDRgb
Follow-up audit on the timeout normalization. Three findings.
1. The maxTotalMs I added to fetchFirst and multi-relay count was the
wrong shape twice over. A hard wall-clock bound already composes at the
call site -- withTimeoutOrNull(ms) { fetchFirst(...) } -- so putting it
in the signature duplicates what the caller has for free. And it was
papering over the real defect: repeat chatter from a relay already
accounted for (a CLOSED/reconnect loop, a duplicate COUNT) was treated
as activity and restarted the idle window, so a flapping relay could
hold the call open indefinitely. Both now reset the window only on
genuine progress -- an event, or the first terminal signal from a relay
still being waited on -- which is the rule the negentropy watchdog
already applies to NOTICE/CLOSED chatter, and which makes both calls
self-bounding at one window per relay. Ceiling params removed; the
overflow guard they needed goes with them.
2. fetchFirst could drop a match: an event landing after the last terminal
signal but before unsubscribe was left unread in the channel and the
fetch reported nothing found. Added the post-loop drain that
fetchAllWithHooks already does. Covered by a test.
3. fetchAllPages published its per-page counters across threads without a
barrier on the idle path. received/delivered/pageMinTs/idsAtPageMin are
written on the relay reader thread and read by the driver once the wait
ends; the EOSE path gets happens-before from the channel, the idle path
had none, so the driver could read a stale pageMinTs (ending the walk
early) or an unsafely published idsAtPageMin. The volatile IdleClock
bump now runs in a finally, so it covers every event including the
early-returning duplicate and orders after the counters.
Also folded the single-relay count channel close into its finally, so a
throw mid-wait cleans up like every sibling accessory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KZe1Pq2ejoHehdufhPDRgb
The per-page wall-clock ceiling added alongside the idle-window change did
not do what it claimed. fetchAllPages waits inside a while(true): when a
page's wait ends, the loop advances the until cursor and issues the next
REQ rather than returning, so a ceiling bounds one page, never the call.
Measured against a relay trickling events forever with an unbounded
filter, maxPageMs=400 produced 8 REQs and no return -- the walk ran until
the caller cancelled, exactly as it would with no ceiling at all.
It also made truncation unsafe. Cutting a page mid-stream advances until
to the oldest event received so far, which only preserves the set if the
relay streams strictly newest-first -- NIP-01 recommends that but does not
require it -- so a ceiling firing on an out-of-order relay can skip the
not-yet-sent events above that cursor. That is the same class of silent
gap the idle window was introduced to close.
What actually bounds a paged download is the filter's limit (already the
documented way) or cancelling the caller, which the ensureActive() at the
top of each page honors. Removed from fetchAllPages, fetchAllPagesFromPool
and fetchAllPagesFromPoolWithHooks; a regression test pins the real
contract so nobody re-adds a ceiling believing it caps the walk.
maxTotalMs stays on fetchAll/fetchAllWithHooks, fetchFirst and multi-relay
count: those wait in a single loop and return, so there the ceiling
genuinely ends the call (each is covered by a test asserting it does).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KZe1Pq2ejoHehdufhPDRgb
Audit follow-ups on the idle-window normalization:
- maxTotalMs/maxPageMs defaults are timeoutMs * 10, which silently
overflows to a negative Long for an effectively-infinite idle window
(Long.MAX_VALUE * 10 wraps to -10). withTimeoutOrNull then expired
immediately, inverting 'wait forever' into 'never wait'. All three
ceilings (fetchAllPages, fetchFirst, fetchAllWithHooks) now treat a
non-positive ceiling as uncapped, matching the idle window's <= 0
convention. Regression-tested via fetchFirst.
- count(filters) leaked its RelayConnectionListener and left COUNT subs
open if the caller cancelled mid-wait or a listener threw: the cleanup
ran as straight-line code with no try/finally, unlike every sibling
accessory. Wrapped so unsubscribe + removeConnectionListener + channel
close always run.
- New FetchFirstIdleTimeoutTest pins fetchFirst's idle-window semantics
(signals restart the window, silence costs exactly one window, the
ceiling stops endless terminal chatter, overflow means uncapped).
- README: note publishAndConfirm's fixed window as the deliberate
exception, and correct the fetchAllPagesFromPool row, which claimed
cross-relay dedup the function explicitly does not do.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KZe1Pq2ejoHehdufhPDRgb
fetchAllPages waited for each page's EOSE under a hard wall-clock
withTimeoutOrNull, unlike fetchAll / fetchAllWithHooks / the negentropy
sync, whose timeouts are idle windows reset by every arriving message.
A slow-but-streaming page could be silently truncated, and a relay
slower than timeoutMs to first byte was mistaken for a drained set.
- Extract IdleClock + receiveWithinIdle out of NostrClientNegentropySyncExt
into a shared internal IdleWatchdog.kt and document the package-wide
convention in the accessories README.
- fetchAllPages (+ pool/hooks variants): the per-page timeout is now an
idle window bumped by every event; a maxPageMs ceiling (10x, matching
fetchAllWithHooks' maxTotalMs) backstops a never-EOSE trickle. A
non-positive ceiling means uncapped, mirroring the idle <= 0 convention.
- fetchFirst: the wait restarts on every arriving signal instead of one
hard deadline across all relays; maxTotalMs ceiling added.
- count (multi-relay): each arriving COUNT result restarts the window;
maxTotalMs ceiling added. Single-relay count is trivially idle already.
- NegentropyStoreSync page fallback clamps a disabled watchdog (0) to
DEFAULT_DOWNLOAD_IDLE_MS like fetchByIds, instead of paging unbounded.
- New NostrClientFetchAllPagesIdleTimeoutTest pins the semantics
(verified to fail against the old hard-deadline wait).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KZe1Pq2ejoHehdufhPDRgb
A relay holding 12,289,614 profiles answers
["COUNT","c1",{"kinds":[0]}] -> {"count":500}
LimitsPolicy ran the same clampLimits over CountCmd as over ReqCmd, so
an unbounded COUNT was given RelayLimits.defaultLimit and the store then
counted at most that many.
defaultLimit answers "how many events should a REQ return when the client
names none". A COUNT returns no events, so the question has no meaning
for it, and applying the answer anyway turns every unbounded COUNT into
min(matches, defaultLimit).
The failure is quiet, which is what let it survive: 500 is a plausible
number, and the kinds under the default answered correctly. On the relay
that surfaced it, kinds 1 and 10040 were right while 0, 10002 and 30382
were all exactly 500.
maxLimit still applies — a client asking to count at most N is asking
something a relay may bound. Only the invented default is dropped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
It was measured from onConnecting to onConnected, which includes the time
the call sat in the client's dispatcher queue. Under a 16,507-relay
fan-out that queue dominates everything else: published records showed a
median rtt-open of 33.5 SECONDS and a max of 90, against a true minimum
of 140ms.
That is the field aggregators rank relays by, so it was worse than
publishing nothing — a signed claim that healthy relays are slow, when
the slowness was ours.
pingMillis already carried the right number and was being handed to the
listener unused: BasicOkHttpWebSocket computes it as
receivedResponseAtMillis - sentRequestAtMillis, so it starts when the
upgrade request actually goes out and excludes everything before it.
Zero or negative means the transport could not time the handshake, and
then no timing is published rather than a fabricated one — the same rule
the rest of this class already followed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Audit finding on the StoreQueryContext seam: RelaySession kept
authenticatedUsers as a plain mutable LinkedHashSet and handed out a
live view through RequestContext. A store (or EventSource) reading the
set during a long REQ replay — exactly what StoreQueryContext invites —
could race a concurrent NIP-42 AUTH commit on another coroutine:
iteration vs. add on an unsynchronized set is a
ConcurrentModificationException or a torn read.
The engine now swaps an immutable Set behind a @Volatile field on each
AUTH (the only writer), so every read is a consistent snapshot and
holding one across a replay is safe. AUTH is rare; the copy is off the
hot path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hH4RY2AUwfMZ54RkMiT45
The IEventStore contract now defines the seams a store implementation
needs instead of having the relay layer decide for it:
- NIP-50 search extensions: LiveEventStore no longer strips key:value
tokens before the store sees them. Filter.search reaches every store
verbatim, and each implementation decides which extensions it
supports — the store, not the middleware, is the only component that
knows whether sort:rank is a directive or noise. The built-in SQLite
and filesystem stores strip at their own boundary (their FTS engines
would otherwise error on / literally match the tokens), preserving
the NIP-50 'ignore unsupported extensions' behavior end to end.
Extension-aware stores need no side channel to recover the raw
string anymore. Fs delete now checks emptiness on the stripped
filter so an extensions-only search cannot wipe the store.
- Caller identity: new StoreQueryContext coroutine-context element,
installed by LiveEventStore around every REQ/COUNT store call when
the connection has NIP-42-authenticated pubkeys. Observer-relative
stores (web-of-trust ranking, for-you relevance) read it off the
coroutine context; ranking context only, never match-set changes.
- Negentropy liveness: snapshotIdsForNegentropy gains an optional
onProgress hook so mirrors syncing huge corpora get a running count
through the interface type instead of a concrete-class overload.
SQLite reports from its row loop; the interface default streams and
reports too.
- FtsReindexProgress.cursor documented as opaque and store-defined so
resumable-reindex callers never assume id semantics.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hH4RY2AUwfMZ54RkMiT45
RelayObserver is a RelayConnectionListener, so on its own it can only
report on relays something opened a websocket to. On a large fan-out that
is a small minority, and it is the wrong minority: the cheap checks that
decide NOT to dial — a TCP probe, a DNS failure, a host struck out after
repeated silence — are precisely the ones that learn a relay is gone, and
their findings had nowhere to go.
Measured on a 16,507-relay list: 104 records published. Everything else
was ruled out before the client ever saw it, so the monitor had nothing
to say about 99% of the relays it had just formed an opinion on.
record() takes those findings. Same rules as the connection path — a
relay that answered is not demoted by one failed probe, and a reachable
relay with no measured time is published with no time rather than a
fabricated zero.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nip42AuthDmDeliveryTest stalled for its full 10s timeout on CI while
passing locally. The stall is a race in InProcessWebSocket.connect():
server.connect() runs the session's connect-time policies synchronously,
so FullAuthPolicy's AUTH challenge reached the client's listener before
the socket assigned its `incoming` channel and before onOpen fired —
breaking the WebSocketListener contract (no onMessage before onOpen).
RelayAuthenticator answers that challenge on its own coroutine. When the
signed AUTH reply hit send() before the connect thread reached the
`incoming` assignment, send() returned false and the reply was silently
dropped. Nothing recovers from that: the challenge is already dedup'd as
answered, and an EVENT rejected with OK-false `auth-required:` never
re-triggers auth (only a CLOSED does), so the pending gift wrap was
never resent — exactly the CI signature (10.011s, no auth activity
between the authenticator's Init and Destroy logs).
Server->client frames now go through an outbound channel drained by a
coroutine started only after onOpen, so every connect-time frame reaches
the listener with the socket fully wired. Order is preserved by the
single drainer, same as the existing inbound path.
Both new InProcessWebSocketTest cases fail deterministically without the
reorder (the challenge always outran onOpen; a reply sent from the first
onMessage was always rejected) and pass with it, on top of the full
:geode:test and :quartz:jvmTest suites.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bf1Y91sfjxwi2ig4ymGTA9
Native targets reject a comma inside a backticked name, so five tests
that read fine on JVM broke every Kotlin/Native build. Renamed without
them.
The override parameters now match RelayConnectionListener — pingMillis,
compressed, cmdStr, cmd, msg, errorMessage — which silences six warnings
and, more to the point, fixes a misreading: onConnected's second and
third parameters are the connection's ping and whether it is compressed,
and I had them named attempt and success.
Both were missed the same way: jvmTest passes without ever compiling the
native TEST sources. All five targets now compile, main and test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
commonMain, so it compiled on JVM and broke every native target. Sorted
into a LinkedHashMap instead, which is the same output everywhere.
Found by CI on iosSimulatorArm64 because I had only compiled the JVM
target locally; all five now build.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A monitor normally probes — opens connections purely to measure, then
throws them away. A client that is already subscribing, fetching and
publishing has better data for free: measured under real load, against
the relays it actually uses, at the concurrency it actually runs.
RelayObserver is a RelayConnectionListener, so it sees every connection
whichever code path opened it and none of them has to report anything:
rtt-open onConnecting to onConnected
rtt-read first REQ to its EOSE
rtt-write first EVENT to its OK
reachable it opened, or served something
auth-required it sent AUTH, or CLOSED saying so
the error, verbatim, when it never opened
Everything is OBSERVED. Nothing is copied from a relay's NIP-11: that is
the relay's own claim, available to anyone who asks, and republishing it
under a monitor's signature adds nothing but a chance to go stale. Where
the two disagree — a relay advertising open reads that then challenges
us — the observation is the half worth having, and copying the claim
would erase it. It also keeps quartz free of an HTTP dependency.
RelayMonitor is the whole wiring: construct one and connections are
measured, signed as 30166s on an interval, and folded into a cheap
in-memory isKnownDead for picking relays. That read has to be cheap — an
outbox picker runs per event — so it answers from a snapshot refreshed on
an interval, never a store query.
The signer is required. Measuring relay quality and letting others check
it IS NIP-66, and an optional signer would just add the failure mode this
library keeps designing out: configured, silent, doing nothing. A client
that should not publish does not construct one.
RelayObserver also replaces the CLI's RelayDiagnostics, which was the
same listener minus the timings. Porting it surfaced a bug both shared:
substringBefore(':') returns the WHOLE string when there is no colon, so
a relay's free-form CLOSED prose became its own tally key and the map
grew with the number of distinct sentences relays wrote. The colon is
now required.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
javap showed the hexToByte/byteToHex field re-loaded on every use inside
these methods (16 times per readLong call) — the JVM/ART doesn't reliably
prove the load loop-invariant. Hoisting it into a local measured ~25%
faster for decode and ~10% for isEqual and readLong on the JVM
(4096 random 32-byte ids, best-of-150 rounds, 3 repeats); encode was
neutral on HotSpot but is hoisted too since ART is historically worse
at this (see the internalIsHex comment).
Branchless variants of isHex/isHex64 were also measured and were a
wash-to-slightly-worse than the branchy early-exit versions on valid
input, so those keep their current implementations.
Also adds the new exact-size codecs to the on-device HexBenchmark
(decode64, decode64OrNull, encode64, decode128, encode128, toLong256,
and the old isHex64+decode two-pass for comparison) so ART numbers can
be collected with the existing benchmark harness.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hwv6XwT9mwGUQc57zH4ky4
javap on the previous version showed the hexToByte field re-loaded
twice per iteration, the trip count as a runtime parameter, and the
whole loop wrapped in an exception table (only needed because chars
above 0xFF overflow the 256-entry lookup table).
Now the table is hoisted into a local, the function is inline so the
32/64-byte length becomes a compile-time constant at each call site,
and out-of-range chars are rejected branchlessly: the index is masked
with 'and 0xFF' so it cannot overflow, while '255 - code' goes negative
for any char above 0xFF and is folded into the same sign-bit
accumulator that already catches invalid hex digits. No try/catch, no
exception table, no branches in the loop.
Measured on the JVM (4096 random ids, best-of-200 rounds, two runs
with variant order reversed to rule out JIT profile artifacts):
~25% faster than the previous version and ~2x faster than the
isHex64 + decode two-pass combination.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hwv6XwT9mwGUQc57zH4ky4
Adds exact-size codec entry points to the Hex utility so 32-byte
pubkeys/event ids (64 chars) and 64-byte signatures (128 chars) with the
wrong size or invalid characters are rejected instead of silently
decoded:
- decode64 / decode128 throw IllegalArgumentException; the OrNull
variants return null for untrusted input.
- encode64 / encode128 require exactly 32 / 64 input bytes.
The decode is single-pass: character validation is folded into the
decode loop via a sign-bit OR-accumulator (the lookup table yields -1
for invalid chars), so it is faster than the isHex64 + decode
two-pass combination.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hwv6XwT9mwGUQc57zH4ky4
Replaces the hex-string cache key with a small CacheKey holding the raw
(privateKey, pubKey) references — no byte copy and no ~400-byte hex String per
lookup. The precomputed Int hash is only a bucket selector; equals() does the
authoritative full-content comparison, so collisions share a bucket and are
disambiguated rather than returning the wrong peer's secret. Same value-type
immutability contract already relied on by X25519KeyPair as a map key.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PY7kpiTcFiDshYqBsQ5KVC
Two hardening fixes in the NIP-44/NIP-04 encryption primitives:
1. SharedKeyCache keyed conversation/shared secrets by a 32-bit polynomial
hashCode of (privateKey || pubKey). Distinct peers whose bytes hash to the
same Int collided on one LRU slot, so get() could return one peer's key for
a message meant for another — a silent wrong-key encrypt/decrypt. The hash
collides independently of the private key, so a pubkey aliasing a victim's
contact is grindable. Now keyed on the full (priv || pub) byte content.
Added SharedKeyCacheTest, which fails on the old hashCode key and passes now.
2. NIP-44 MAC verification (Hkdf.fastExpand and Nip44v2.checkHMacAad) used
ByteArray.contentEquals, which short-circuits on the first mismatching byte
and leaks a timing side channel on the HMAC tag. Switched to a shared
equalsConstantTime helper (utils/ConstantTime.kt), matching the constant-time
tag checks already used in ChaCha20Poly1305/XChaCha20Poly1305/MLS.
Also corrects the X25519 KDoc, which claimed the JVM/Android actual delegates to
java.security XDH; it is in fact the same pure-Kotlin Montgomery ladder as the
other targets.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PY7kpiTcFiDshYqBsQ5KVC
ServiceType.parse destructured the result of split(":", limit = 2) into
two components, so any value without a colon (e.g. the "client" of a
["client", "nostria"] tag) threw IndexOutOfBoundsException instead of
returning null. It also accepted "30382:" as an empty service type.
ServiceType.isOfKind had the same class of bug: it read
serviceType[kind.length] after startsWith, which is out of bounds when
the value equals the kind exactly ("30382" vs "30382").
Parse the separator by index and check the length before indexing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0112ysMEzSaezH9mXFteNt13
contactMetaData() decodes kind-0 content with the lenient JsonMapper, but
contactMetadataJson() ran the strict default parser. Profiles that sit in
that gap (bare keys, unquoted values) rendered fine yet read back as null,
so updateFromPast() started from an empty map and silently dropped every
field Amethyst does not edit itself the next time the owner touched their
profile — the exact thing its "tries to not delete any existing attribute
that we do not work with" contract promises not to do.
Both accessors now use JsonMapper.jsonInstance. The lenient reader tags
unquoted tokens as strings, so re-encoding still emits valid JSON.
Also pins the behaviour of two malformed kind-0s seen in the wild — a
JavaScript object literal and a value truncated by stray quotes. Neither
is recoverable by any parser; the test records that they are dropped with
a warning and never throw.
The always-on notification says "connected to N relays" and nothing more. That
count is emergent, not curated: NotificationRelayService owns no subscriptions,
and nothing closes a socket when it stops being useful, so a relay stays
connected exactly as long as some filter still references it. Neither a user nor
a developer can tell whether those N relays are carrying DMs or re-dialling a
stale outbox hint.
The client already tracks every in-flight REQ per relay with its filters
(INostrClient.activeRequests, which the Connected Relays screen reads). A filter
says *what* is matched — kinds, authors, tags — but not *who asked*, and subIds
are random. This adds the missing half.
`Filter` becomes `open`, with `copy()` open too, and commons gets
`ExplainedFilter` carrying a `SubPurpose`. The purpose rides on the filter, so it
arrives with the data the relay screens already read — no parallel registry to
keep in sync or leak on teardown.
**It never reaches a relay.** FilterSerializer is registered against Filter and
writes an explicit protocol field list, and Jackson applies a serializer
registered for a class to its subclasses — so an ExplainedFilter serializes to
byte-identical JSON. That is the point: telling relays what each REQ is *for*
would hand them a ready-made fingerprint of client intent and correlate
subscriptions that are deliberately kept apart.
`copy()` is overridden because filters are copied on the live path — assemblers
call copy(since = …) after every EOSE. Inheriting the base implementation would
downgrade to a plain Filter on the first window refresh, so the purpose would
survive the opening REQ and vanish seconds later.
Both invariants are pinned by tests, and both were mutation-checked rather than
assumed:
- removing the copy() override -> `copy preserves the purpose` fails
- unregistering FilterSerializer -> 3 tests fail, one reporting the literal
leak: `purpose leaked to the wire: {…}`
A test also asserts FiltersChanged does not see the new field, so tagging a
filter cannot trigger a re-REQ storm across ~400 relays.
Wired three assemblers as proof (metadata, reactions, outbox finder) and surfaced
the derived set on BasicRelaySetupInfo.purposes for the Connected Relays screen.
Verified on device: 8 relays attributed RELAY_LISTS through
client.activeRequests() — purpleplag.es, user.kindpag.es, indexer.coracle.social,
directory.yabu.me and friends, which is semantically right — across 2,709 sent
REQs containing zero occurrences of "purpose" or any SubPurpose name.
The remaining assemblers are untagged, which is why `purposes` is documented as
"not yet attributed" rather than "idle". Notification-popup copy is deliberately
not built yet: it should describe the measured background grouping, not the
intended one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four kind-0 events collected from relays exposed two content-parsing
problems. Both `MetadataEvent.contactMetaData()` and `contactMetadataJson()`
are shared code, so Jackson (JVM/Android) and kotlinx (native) were equally
affected — verified against both mappers.
Empty content is a valid, empty profile — someone wiping their metadata —
not a parse failure. It logged "Content Parse Error" and returned null,
which made `LocalCache.consume` drop the event, so the stale profile stayed
in place forever even though a newer replaceable event had arrived. Blank
content now decodes to a blank `UserMetadata` (and an empty `JsonObject`).
A string field wrapped in a one-element array (`"nip05":["a@b.com"]`) has
only one possible reading, so `TolerantStringSerializer` now unwraps it
rather than silently dropping the user's NIP-05 verification. Empty,
multi-element, and non-primitive arrays stay ignored.
The other two events already behaved correctly and are pinned as
regressions: `"nip05":{}` with foreign client keys, and Ditto's ambiguous
`"birthday":"10-24"` — both drop just the offending field.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XGYcqfBKsP9SCvRdPzdkSx
An entity edited after a CORD-06 Refounding was frozen at its pre-Refounding
version and its newer state silently discarded on every refold. Observed on
device: entity e83ee182 of a real community held at v1 across 21 consecutive
refolds while the current epoch offered v2 citing v1's own hash.
`EditionFold.foldEntity` anchored the walk by requiring the offered set to
*contain* the floor edition. But a Refounding re-wraps ONE edition per entity
(CORD-06 §3, "the last Control Plane state is simply rewrapped"), so an entity
edited afterwards has only its successor on the new epoch while the floor
edition stays behind on the old one. `prev == floor.hash` is a stronger proof
the chain connects than mere presence, and it was being ignored. Worse, the
gap made `admissible` — the pre-filter every derived fold shares — keep only
`version < floor.version`, so the newer edition was dropped every refold: a
role edit, channel rename or banlist entry reverting itself for that user.
Checked against Armada (semantics only, AGPLv3) and the canonical spec before
changing consensus behaviour. Armada selects an arm per entity, and Amethyst
was missing both halves:
1. Its chain-walk arm has three anchor branches to our one; the missing
`versions[0] === floor + 1n -> bytesEq(lo.prevHash, floorHash)` is now
implemented.
2. Once an entity has been re-wrapped into the epoch being folded it does not
chain-walk at all — it anchors on version alone (`bootstrapHead`), because
behind a compaction dangling `prev`s are normal and, since seal signatures
survive re-wrap, any group-key holder can re-serve a genuine old edition
under the current group. A re-wrap cannot raise the version inside the
signed seal, so version is what bounds that. This is the half that fixes
the observed pin; branch 1 alone would still refuse a floor-v1 entity whose
only served edition is v3.
Implemented as an optional `snapshot` (the rumor ids of the epoch being
folded), defaulting to null = pure chain walk for every other caller.
`ConcordCommunityState.fold`/`authorizedHeads` capture it from their own
editions argument before `admissible` re-seats older-epoch heads — Amethyst
folds exactly one epoch per call, so the argument is the snapshot. `admissible`
and `candidates` now ask `foldEntity` whether it gapped rather than
re-deriving the test, so the three sites cannot drift apart again.
`LOG_GAP` also dedups on (entity, floor, offered): the same refusal was
re-reported on every refold, 22 byte-identical warnings a boot, which read as
22 attacks rather than one unchanged state. Deduping is what made the two
genuinely distinct refusals visible and led here. Fails open above 4096
distinct refusals — a flood is when the warnings matter most.
Anti-rollback is unchanged: all 12 pre-existing floor tests still pass, and
three new negative tests cover the fork, the below-floor re-serve, and an
entity absent from the snapshot. Device gaps for the pinned entity: 21 -> 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Vitor PamplonaandGitHub
Claude
Vitor PamplonaandClaude Opus 5