Kotlin/Native marks the stdlib `assert()` with `@ExperimentalNativeApi`, so
the iOS test target (`compileTestKotlinIosSimulatorArm64`) failed to compile
without an opt-in — while JVM/Android were fine. Swap it for `assertTrue`
from `kotlin.test`, which needs no opt-in on any target and matches the
assertions already used in this file.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
- docs(dm): note the profile Reports tab also reads reportsNamingUser
- refactor(dm): push report-tag typing to quartz and simplify the warning stack
- fix(dm): narrow report indexing and address final review findings
- perf(dm): resolve a 1:1 chat row's counterpart once per row
Reworks the Buzz relay's Community view to mirror Buzz's own sidebar
ordering instead of stacking the DM + Console entries on top of the
channels:
- Channels and Forums are now separate sections, split by the relay-signed
39000 `channel_type` (stream vs forum); DM-typed channels are excluded
from both (they belong to the DM section). Adds isBuzzForum() and the
forum/stream type constants alongside the existing DM reader.
- Direct Messages moves below the channels and renders inline: the most
recent conversations (avatar + name + last-activity time), a New-message
action in the section header, and a "See all N" row into the full inbox
when there are more than fit.
- Agent Console drops to a single footer card at the very bottom.
- Section headers get a consistent modern style (primary-colored labels
with optional trailing actions), replacing the old top-stacked action
cards. BuzzImportRow gains an onOpen tap target so a channel row opens
the chat while keeping its Add-to-list affordance.
Vanilla NIP-29 relays are unchanged (flat channel directory, no
forums/DMs/console).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
Kind 20001 is claimed by both BitChat's GeohashPresenceEvent and Buzz's
PresenceUpdateEvent. EventFactory's flat when(kind) could only route one, so
Buzz presence never materialized (parsed as a geohash event) in either
direction — this was the deferred "EventFactory collision".
Disambiguate inside the shared 20001 branch by BitChat's required `g` (geohash)
tag: present -> GeohashPresenceEvent, absent -> PresenceUpdateEvent. Verified
against the Buzz Rust ground truth (buzz-sdk build_presence_update + the relay's
synthesize_presence read form): Buzz presence carries the status in content plus
a `status` tag (client) or a `p` tag (relay-synthesized), never a `g` tag, and
BitChat presence always carries `g` with empty content. Both inbound parse
(EventDeserializer) and outbound signing (EventAssembler) route through this
factory, so one guard fixes both.
Make it usable, not just parseable: add BuzzPresenceState (process-wide latest
online/away/offline per subject, mirroring BuzzTypingState), a
PresenceUpdateEvent.subjectPubKey() accessor (the `p` tag or the author), and a
LocalCache branch that records presence and drops the ephemeral without storing
it. Tests cover both Buzz wire shapes, the BitChat guard, and latest-wins.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
The NIP-89 client tag says "this app composed this event", so it belongs only
on templates Amethyst authored. NostrSignerWithClientTag was applied at the
account level, which meant it also fired on every event we sign on behalf of
an external client.
That is wrong twice over. It misattributes the event, and — because the tag is
appended before signing — it rewrites the exact bytes the caller is about to
have hashed into an id. NIP-07 callers routinely re-check the returned event
against the template they submitted, and block/buzz compares tags outright
(web/src/shared/lib/nostr-signer.ts):
JSON.stringify(actual.tags) === JSON.stringify(expected.tags)
so joining a Buzz community from the in-app browser failed with "The NIP-07
extension returned an invalid signed event". Probed live over the WebView
devtools protocol: kind, created_at, content and pubkey all round-tripped
intact and only tags differed, by exactly the ["client","Amethyst"] we append.
Add NostrSigner.withoutClientTag() and use it at the two boundaries where the
template belongs to someone else:
- the napplet broker, covering napplets, nSites and web apps over NIP-07
- Nip46SignerState, where we act as another client's bunker — the same defect,
and quieter, since that client never learns why its event changed underneath
it
Amethyst's own events are untouched and still carry the tag. Unwrapping keeps
everything layered below (metering, NIP-13 mining) and leaves pubKey alone.
There is no NIP-55 provider surface to fix; we are only ever the client there.
Verified against Buzz's own four acceptance conditions after the change:
pubkey matches, sameUnsignedEvent true, id and sig present — and the invite
join then succeeded.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The earlier discovery layer read the NIP-29 joined list (kind-10009) and
kind-41001 — neither of which the deployed relay uses, so a joined workspace
rendered nothing. Rework it to the model live testing confirmed.
Enabling layer — persist joined workspaces:
- commons BuzzWorkspaces: process-wide set of joined workspace relays (Buzz
membership is server-side, so there's no join event to rebuild from). Joining
also marks the relay a Buzz dialect. Unit-tested.
- BuzzWorkspacePreferences: device-global DataStore that restores the set at
startup (so the app connects + authenticates + discovers on cold start) and
mirrors changes. Eager init in AppModules. BuzzInviteScreen now `join`s.
Quartz:
- BuzzChannelMetadata: read the relay's `t` channel-type tag ("stream"/"forum"/
"dm") and a DM's inlined `p` participants off kind-39000.
Discovery (both hubs now source from the relay's real signals):
- BuzzWorkspacesViewModel: fetch + live-subscribe kind-44100 member-added
notifications (#p=me) across joined relays → my channels; fetch each channel's
39000 metadata; keep the non-DM ones. BuzzWorkspacesScreen unions this with the
NIP-29 joined list.
- BuzzDmListViewModel: same 44100 discovery, kept where 39000 `t`=dm (participants
from the metadata `p` tags), minus the 30622 hidden set — replaces the dead
kind-41001 path.
- Account.openBuzzDm returns the relay-assigned channel id from the OK response
(`response:{channel_id}`); BuzzNewDmViewModel opens the chat from it instead of
polling 41001.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
Lower the quartz module's Kotlin jvmTarget for both the JVM and Android
compilations from JVM_21 to JVM_17, broadening the range of runtimes that
can consume the published library.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1Hn61gQJ1joUznESzUHU4
Validated against the production relay wss://amethyst.communities.buzz.xyz
by joining and running a full DM round-trip. Adds the join primitive and
fixes the interop gaps that live testing surfaced.
- quartz: BuzzInviteLink — parse `https://<host>/invite/<token>` (relay-signed
base64url payload → community/role/expiry). A Buzz invite is NOT a NIP-29
code; it is redeemed over HTTP against the tenant host. Unit-tested with a
real token; rejects the Concord `/invite/<naddr>#…` shape (no collision).
- cli: `amy buzz join <invite-url>` — the real 3-step claim: GET /api/join-policy,
POST /api/invites/accept-policy, then NIP-98-signed POST /api/invites/claim.
Proven live (status: joined, role: member).
- cli: Context.publish now authenticates-then-retries on an `auth-required`
relay (warm the connection with a pendingOnAuthRequired REQ, then re-publish)
— the write path had no NIP-42 handling, so every Buzz write was rejected.
- cli: Buzz reads (dm list / read / console / personas) use the auth-aware
drain (pendingOnAuthRequired).
- cli: `dm open` surfaces the relay's synchronous OK `response:{channel_id}` —
the authoritative DM channel id (the relay assigns it; it is not polled).
- cli: `dm list` rewritten to the relay's actual discovery — kind-44100
member-added notifications (#p=me) filtered to the kind-40099 `dm_created`
channels. The deployed relay does NOT emit kind-41001.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
A Buzz DM is a relay-authoritative NIP-29 group whose h/id is a
relay-generated UUID, so its timeline reuses the whole relay-group chat
stack unchanged. This adds the missing discovery + product layer:
- commons: BuzzDmRegistry — process-wide registry fed by LocalCache from
the relay-signed DmCreatedEvent (41001) and per-viewer DmVisibilityEvent
(30622); tracks conversations (channel id -> participants/relay) and the
viewer's hidden set. Unit-tested.
- LocalCache: record 41001/30622 into the registry on consume (was
store-only).
- Account: openBuzzDm (41010), hideBuzzDm (41012), addBuzzDmMember (41011).
The relay assigns the channel UUID and confirms via 41001 — we never
mint it.
- Android: BuzzDmListViewModel (two-phase fetch: discover 41001/30622 #p=me,
then fetch each DM's 39000-39003 roster so the shared composer's member
gate passes), BuzzDmListScreen (inbox), BuzzNewDmScreen (publish 41010,
await the 41001, jump into the shared RelayGroupChatScreen). Reached from
a Direct Messages card on the Workspaces tab.
- CLI: amy buzz dm list/open/hide/add-member, mirroring buzz-cli.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
- Workspace shell: BuzzWorkspacesScreen now groups joined Buzz-dialect groups
BY RELAY into workspace→channels (a Buzz workspace IS a relay/tenant, per
buzz-core relay_url_authority), each an expandable section with its channels
and a + to the relay's directory (Concord-style community→channels).
- Forum composer (45001): the Threads-tab FAB opens BuzzForumPostScreen on a
Buzz relay, publishing a ForumPostEvent (mirrors build_forum_post) instead of
the vanilla kind-11 thread a NIP-29 relay uses.
- Held-attestation persistence: BuzzAttestationPreferences mirrors
BuzzHeldAttestations to the device DataStore and reloads at startup,
re-verifying each credential against its agent key (drops tampered entries).
Deliberately NOT built: job/huddle composers — jobs (43xxx) and huddles (48xxx)
have no builder in buzz-sdk (reserved kinds), so a composer would encode an
unconfirmed schema. Presence (20001) skipped per request (EventFactory collision
with GeohashPresenceEvent).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
Gives Buzz a first-class navigation identity instead of hiding inside the generic
Relay Groups list. NavBarItem.BUZZ (Route.BuzzWorkspaces) is a pinnable bottom-nav
destination (added to NavBarCatalog + BottomBarCategories + the preloader when).
BuzzWorkspacesScreen is a modern hub: an Agent-Console hero card up top, then the
user's joined groups filtered to Buzz-dialect relays as cards with a colored
monogram avatar, live name/host/member-count and an unread dot, plus an inviting
empty state that routes to Browse groups. It reuses the always-on relay-group state
subscription (no per-screen fetch) and the same RelayGroupChannel the chat opens.
Also fixes the open-chat-tail test for the added 20002 typing kind
(RELAY_GROUP_OPEN_TAIL_KINDS), and documents typing + the hub in the buzz README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
Earlier text wrongly said Buzz has no reaction kind. buzz-core/src/kind.rs ALL_KINDS
includes KIND_REACTION(7), KIND_DELETION(5), KIND_NIP29_DELETE_EVENT(9005)/GROUP(9008),
KIND_REPORT(1984), and emoji list/set (10030/30030); requires_h_channel_scope(7) is
false. So the shared chat action sheet's react/delete/report already work against Buzz.
Only zaps (no 9734/9735) aren't relay-stored (payment still works). Typing/presence are
accepted by Buzz but not yet rendered by Amethyst.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
Tier 1 connect path. BuzzHeldAttestations (commons) stores the OwnerAttestations
this device received, keyed by the agent pubkey each authorizes (verify-passing
only). AuthCoordinator.buzzAugmented appends the owner-signed auth tag to an
account's NIP-42 AUTH event when that account's key has a held attestation and
the relay speaks the Buzz dialect — and only that account's AUTH, never the
Concord stream-key AUTHs sharing the template, and never on non-Buzz relays.
So an un-enrolled agent key gets virtual membership while its owner stays a member.
AgentAttestationScreen gains a 'Hold an attestation' section (paste the auth tag
JSON, verified against the current account, stored/removed) alongside the existing
owner-side issuance. Store is in-memory for now — persisting per-account is a
follow-up. 4 store tests added.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
Adds AgentAttestationScreen, reached from an "Attest" FAB on the console.
The owner enters an agent pubkey (npub/hex) and optional AttestationConditions
(kind, created_at before/after), and OwnerAttestation.sign() produces the
signed auth tag, displayed for copy to hand to the agent operator out-of-band.
Entirely offline (nothing is published) and gated on a raw private key: the
NIP-OA signature covers a hashed commitment rather than a Nostr event, so
NIP-46 bunker / NIP-55 external-signer accounts get an explanation instead of
the form. Inputs are validated (kind 0-65535, unix bounds 0-u32, npub/hex key,
no self-attestation) with human-readable errors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
Two-reviewer adversarial audit of the branch found no correctness/data-loss/
crash bugs. This closes one gap and applies three robustness fixes:
- Multi-filter search ordering: a REQ whose filters all carry a search term
(e.g. the client's search-across-kinds) was created_at-ordered via the union
path. Now relevance-ordered — unionSubqueriesIfNeeded(projectRank) projects
rank per branch, UNION ALL + GROUP BY row_id MIN(rank) dedups across branches
keeping the best score. Only when every branch is a search branch; mixed
search/non-search REQs and count/delete unions stay as before.
- prepareAuthorStreams/prepareTagStreams build cursors via buildStreams, which
closes already-prepared statements if a later prepare throws (was: stranded
checked-out, un-reset handles holding read locks in the pooled connection).
- Stream counts computed as Long so authors×kinds / values×kinds can't overflow
Int back into the eligible band and route a huge fan-out into the merge.
- Renamed CachedStatement.finalize() -> finalizeStatement(): a no-arg finalize()
is the JVM Object.finalize, risking a GC double-close of the native handle.
Tests: multi-filter search relevance + cross-branch dedup + count parity;
existing merge/cache/search suites green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
From the branch audit: verifies the search-relevance path holds under a
two-tag-key filter (requires both tags, ranks by bm25, no duplicate rows),
that count(filter) matches query size under a limit smaller than the match set
(NIP-45), and that deleting a non-searchable event (no FTS row) fires the
contentless delete trigger against an absent rowid harmlessly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
Three-reviewer pass over the branch. Fixes, most-severe first:
HIGH — channel-instance swap froze open screens. The subclass-upgrade design
replaced a group's RelayGroupChannel with a Buzz-typed instance on dialect
discovery, but screens/feeds/composers capture the instance for life, so all of
them kept rendering the orphan (frozen feed; composer stuck on kind 9). Removed
BuzzWorkspaceChannel entirely; RelayGroupChannel is a single stable type again.
Buzz-only overlay state (edit + canvas) now lives in BuzzWorkspaceStates, a
registry keyed by the channel UUID — dialect discovery no longer touches object
identity. The swap also silently dropped all relay-signed state (name, members,
admin status, pins, threads) and demoted the confirmed host; gone with the swap.
HIGH — wrong thread-root got messages relay-rejected AFTER the draft was
destroyed. A reply to a direct reply derived root=parent (buzzThreadRoot null on
a collapsed reply), which Buzz's ancestry validator rejects. Fallback is now
buzzThreadRoot() ?: buzzThreadReply() ?: parent.id. Also: minichat (kind 1111)
replies in Buzz channels are relay-rejected, so they're gated off for Buzz relays.
HIGH — pre-create defeated stray-redirect and marked the dialect off unverified
input. consumeBuzzTimelineEvent no longer pre-creates the channel; attachment
goes through the shared NIP-29 path (with its stray protection), and the dialect
is marked only off a VERIFIED event (markBuzzIfVerified) — a hostile relay can no
longer flip what the composer sends.
MEDIUM — engram tombstone divergence: a memory body missing `value` decoded as a
null tombstone (a deletion) where Buzz rejects it. `value` is now required (no
default); added NIP-AE slug-grammar validation. Pinned by test.
MEDIUM — unbounded edit overlay: pruneOldMessagesChannel now prunes overlay
entries for reaped messages. Overlay is keyed by channel id so own offline edits
(null relay) apply too.
Also: 40002 inbound reply linkage in computeReplyTo (we emit thread markers we
couldn't read); registered-but-unhandled 40901/40902/48001 now consumed;
unconditional Buzz-kind widening (kills the history-cursor-skip); stream mention
dedup; persona empty-list omission for content-hash parity; thread-marker
positional guard; edit-note-loading blank-row fallback.
Full amethyst unit suite + affected quartz suites green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
Any filter with a search term must be relevance-ranked (NIP-50), but only the
tag-free shape (makeSimpleSearch) was — search + a tag fell through to the
row-id-subquery path (prepareRowIDSubQueries/makeQueryIn), which ordered by
created_at.
prepareRowIDSubQueries gains projectRank: when a search filter joins event_fts,
it also projects the bm25 score as a `rank` column and cuts its LIMIT by rank
(most relevant, not newest); makeQueryIn(orderByRank) then presents the joined
result by that rank, created_at DESC as tie-break. Off by default, so
count/delete/union/negentropy (which must stay single-column and unranked) are
untouched. toSql wires it on whenever the filter carries a search term.
SearchRelevanceOrderTest adds a tag-scoped case (stronger-but-older outranks
weaker-but-newer, wrong-tag and non-matching excluded, limit cuts by score).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
NIP-50: results are returned "in descending order by quality of search result
... not by the usual .created_at", with the limit applied after the score. The
store sorted search by created_at DESC (pre-existing), so it returned the
newest matches rather than the best ones.
makeSimpleSearch (the search [+ kinds/authors/since/until] + limit shape) now
orders by FTS5 bm25 (ORDER BY event_fts.rank, created_at DESC as a tie-break).
Verified bm25 rank works on the contentless table through the join, and that a
stronger-but-older match outranks a weaker-but-newer one. The rarer
search+specific-tag shape and the negentropy snapshot still sort by created_at
(the row-id subquery can't carry rank; negentropy is a sync set) — documented.
This is a correctness fix, not a scaling one: bm25 scores every match, so
search latency still grows with the match set. Tests: SearchRelevanceOrderTest,
Fts5CapabilityProbe.bm25RankWorksOnContentlessTableInAJoin; QueryAssemblerTest
search plans updated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
Review feedback: the 40002 composer branch hand-rolled raw arrayOf tags. Now:
- New shared `buzz/threading` package: `buzzThread(root, parent)` builder verb
emitting Buzz's exact thread_tags wire form (["e",root,"","root"] +
["e",parent,"","reply"], collapsing when parent==root; the empty relay slot
is deliberate — MarkedETag.assemble's arrayOfNotNull would slide the marker
into the relay slot) and `buzzThreadRoot()/buzzThreadReply()` positional
readers. The forum verbs now delegate to it (streams and forum comments share
thread_tags in buzz-sdk), and the composer uses buzzThread + the typed
pTag(PTag(...)) verb instead of raw arrays.
- Dialect bootstrap fix: the single-group open-channel REQ now always includes
the Buzz timeline kinds. Without this, an undiscovered Buzz relay was a
chicken-and-egg: fleet subs only widen after BuzzRelayDialect marks the
relay, but the mark comes from consuming a Buzz kind no filter asked for.
Opening a channel is explicit one-group intent, so the wider ask is cheap and
matches nothing on vanilla relays; fleet-wide subs stay dialect-gated
(both behaviors pinned in BuzzTimelineKindsTest). The live relay's NIP-11
("Buzz Relay", supported_extensions=[nip-er,nip-pl]) is a future
connect-time marker.
- Live proof of the full workspace lifecycle against the running Buzz relay
(BuzzRelayLiveInteropTest, 3/3 green): discover the channel via its
relay-signed 39000 (queryable by #d), join with NIP-29 kind-9021 from a
second member, post a 40002 after joining, and leave with kind-9022 — the
exact Quartz events Amethyst's group UI sends.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
searchOrderByRowId ordered NIP-50 search by the FTS rowid (ingestion order) to
get O(limit) search, but NIP-01's limit requires the newest events by
created_at. Once ingestion diverges from created_at (any historical sync) that
returns the wrong events under a limit — a spec violation — so the flag, its
QueryBuilder branch, and its test are removed. Search stays created_at-ordered;
corpus-independent search is an external-engine job, not this index.
The contentless + rowid=row_id schema stays for the reasons that don't touch
ordering: the delete trigger now seeks by rowid (O(log n)) instead of scanning
by an FTS column (O(n)) — measured ~78× faster at 8k rows and widening, on a
path every deletion hits — and the index is smaller. Benchmark reframed around
the delete win and the honest (unchanged) search cost; plan doc updated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
The scale-curve report showed the SQLite store degrading with corpus size on
NIP-50 search (~18×) and the large-IN tag watcher, while point reads stayed
flat. Three read/size changes (write path and index set unchanged):
- FTS: rebuild event_fts as a contentless FTS5 table (content='',
contentless_delete=1) keyed by rowid = event_headers.row_id. Drops the
stored content copy (smaller index); external-content can't hold the
derived indexable text, so contentless is the correct primitive. Adds an
opt-in searchOrderByRowId strategy flag: ORDER BY event_fts.rowid DESC
early-terminates (O(limit), corpus-independent) at the cost of
ingestion-order results — flat ~0.22 ms vs created_at's 4.26 ms at 200k
(~19×). reindexAll ends with 'optimize'; the periodic optimize() folds in a
bounded segment 'merge'. DB version 4->5 with a drop-and-rebuild migration.
- MergeQueryExecutor: extend the k-way merge to the tag path (kinds + #e IN
[hundreds] + limit), one cursor per (value[,kind]) stream heap-merged to the
limit, deduping events that carry several queried values. O(limit + streams)
instead of collecting all matches and sorting.
- StatementCachingConnection: pool multiple handles per SQL so the merge's
many concurrent identical-SQL cursors all hit the cache (previously only the
first did) and repeated polls reuse their per-stream statements.
Tests: contentless migration (real v4 DB upgrade), rowid-order search, tag
merge correctness (incl. cross-stream dedup), statement pool, FTS5 capability
probe, and an FTS search-scaling benchmark. Plan in
quartz/plans/2026-07-21-sqlite-query-scaling.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
Surfaces block/buzz workspace channels inside the existing NIP-29 relay-group
experience — one group model, dialect-aware rendering, zero impact on vanilla
groups (kind-filtered feeds can never receive Buzz kinds by accident).
- BuzzRelayDialect (commons): per-relay capability registry. Event-shape
detection — the first Buzz-only kind consumed from a relay marks it; vanilla
relays never serve those kinds, so no false positives. NIP-11 marking can be
layered on later.
- BuzzWorkspaceChannel (commons): sibling of RelayGroupChannel (now `open`)
holding Buzz-only channel state: the kind-40003 edit overlay (never render
superseded text as current) and the newest kind-40100 canvas. Kind-9 chat and
kind-40002 stream messages share ONE timeline per group so mixed-dialect
conversations stay whole.
- LocalCache: consumes every registered Buzz kind (previously all fell into the
"Event Not Supported" branch). Timeline kinds attach to the group's channel,
materialized dialect-aware with an in-place upgrade (note migration) when the
dialect is discovered after the channel was first created as plain NIP-29.
Addressables store replaceably; the rest store as queryable regular events;
ephemeral signals (typing 20002, observer 24200, huddle reaction 24810,
pairing 24134) mark the dialect but are deliberately not persisted.
- RelayGroupFilterBuilders: group-chat REQs widen their timeline kind set with
40002/40003/40008/40099 only for marked relays; vanilla NIP-29 REQs unchanged.
Tests cover dialect detection + materialization, the plain-channel upgrade with
timeline migration, newest-edit-wins overlay ordering, and that filter builders
extend kinds only on marked relays. Full amethyst unit suite green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
Adds an env-gated jvmTest (BUZZ_RELAY_WS / BUZZ_MEMBER_SK / BUZZ_OWNER_SK; CI
skips it) that drives the real block/buzz relay booted from its own compose +
cargo build, with members enrolled via buzz-admin. Verified green against the
running relay:
- NIP-42 member auth, and NIP-OA/NIP-AA agent auth: a brand-new un-enrolled
agent key authenticates using only our owner-signed `auth` tag — the relay's
NIP-OA membership fallback accepts the Quartz-produced attestation.
- Channel lifecycle: kind:9007 create via the existing NIP-29 CreateGroupEvent
(Buzz channels are NIP-29 groups), kind:40002 publish, REQ round-trip with a
byte-identical echo (same id), EOSE.
Interop lessons encoded in the test + README: tenancy is host-bound (connect
with the bound Host or the WS upgrade 404s), `h` values must parse as UUIDs,
and the channel row must exist before channel-scoped kinds are accepted.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
Adds Quartz-native models for the entire block/buzz custom kind space, each as
an idiomatic per-NIP package (Event + KIND companion + tags/ + write-DSL +
read accessors, mirroring nip88Polls) and registered in EventFactory. Standard
NIP-29/34/51/42/43 kinds Buzz reuses are left to their existing Quartz classes.
Coverage:
- Agent identity: Persona 30175, Team 30176, Managed Agent 30177, Agent Profile 10100
- Agent telemetry (encrypted): Observer 24200, Engram 30174 (HMAC d-tag derivation
pinned to Buzz reference vectors), Turn Metric 44200 (prior commit)
- Workspace overlays: Reminders 30300, Push Lease 30350, DM Visibility 30622,
Workspace Profile 9033, Identity Archival 9035/9036/8002/8003/13535,
Channel Window 39005/39006, relay admin 9030-9032, moderation 9040-9044, 42000
- Messaging/collab: stream 40002-40100 (+sidecars), DMs 41001/41010-41012,
jobs 43001-43006, forum 45001-45003, workflow 30620/46001-46031,
notifications 44100/44101, presence 20001/20002, huddles 24810/48100-48106,
pairing 24134, audit 48001, media 49001, read-state (NIP-RS helpers on 30078)
All schemas confirmed against the authoritative Rust in a local block/buzz
checkout (buzz-core / buzz-sdk / buzz-relay), not the outdated prose NIPs.
Kinds only reserved-but-unbuilt in Buzz (jobs, workflow lifecycle, some
stream/sidecar/audit kinds) are modeled tolerantly and flagged in KDoc + README.
Kind conflicts with existing Amethyst classes are implemented but deliberately
NOT registered in EventFactory (incumbent keeps dispatch): 9041 (GoalEvent),
20001 (GeohashPresenceEvent), 39005 (GroupPinnedEvent), plus 49001 (Buzz marks
it non-wire) and 30078 read-state (reuses AppSpecificDataEvent).
156 Buzz tests pass; full quartz jvmTest green (no regressions).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
The remaining SmallReqFloorBenchmark waste, on the per-row replay, the
per-event live fanout, and the per-accepted-event index probe:
- LiveEventStore replay dedupe: a SeenIds holder with an inline lock
replaces the local-fn-plus-lambda that allocated one closure per
streamed row (and again per live delivery). Its HashSet is created
empty so the JVM defers the backing table to the first add — a 0-row
replay no longer allocates a 1024-slot table (was ~4 MB across the
benchmark's 1000 idle subs).
- Live fanout serializes the event body once and passes it through
onEachLive(event, body); RelaySession splices it into the per-sub
frame prefix. An event matching N live subscriptions paid N identical
Jackson passes before; now one. queryRaw's onEachLive signature gains
the body arg (EventSourceBackend default serializes inline, no
cross-sub memo, no regression). Measured: fanout 1->200 live subs
0.50 ms (2.5 us/sub).
- FilterIndex holds subscribers in one persistent map per dimension, so
candidatesFor (once per accepted ingest event) probes with the event's
own fields and allocates no IdKey/AuthorKey/KindKey/TagKey wrappers;
BucketKey now lives only in the rare register/unregister bookkeeping.
SmallReqFloorBenchmark grows a fanout stage (200 live subs, one submit)
to anchor the fanout number; it drives `live` directly and guards the
await with withTimeout so a future fanout regression fails fast.
Verified: quartz relay.server + FilterIndex suites (110 tests),
SmallReqFloorBenchmark, geode suite (126 tests).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
Three hot-path allocation cuts the SmallReqFloorBenchmark stages flagged:
- strippingSearchExtensions: index-loop guard returns the same list with
zero allocation when no filter carries a search term (every non-search
REQ/COUNT/snapshot, the overwhelming majority).
- EoseMessage/OkMessage: direct-buildString wire form on the escape-free
fast path (EOSE per REQ, OK per publish), skipping the generic
serializer's node tree; exotic subIds/reasons fall back. Shared
isEscapeFreeAscii helper in WireJson.kt, mirroring NegMsgMessage.
Verified: quartz relay.server + message-frame suites (110 tests).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
Addresses review feedback on the initial Buzz commit:
- Drop the central `BuzzKinds` registry (un-idiomatic). Each event now
declares its own `const val KIND` on its companion and is wired into
`EventFactory`, matching the nip88Polls-style per-NIP layout used across
Quartz (Event + tags/ + TagArrayBuilderExt write-DSL + TagArrayExt readers).
- Implement Agent Turn Metric (NIP-AM, kind:44200): an encrypted per-turn
token-usage/cost record published by an agent to its owner. content is a
NIP-44 v2 ciphertext of AgentTurnMetricPayload (camelCase), between the
agent (author + `agent` tag) and owner (`p` tag); either party decrypts.
- Verify against a vector generated by Buzz's OWN code rather than a
transcribed schema: a small generator on the real buzz-core emits a signed
44200 event with deterministic keys; AgentTurnMetricVectorTest dispatches it
through EventFactory, NIP-44-decrypts it, and asserts the payload — proving
end-to-end interop. Fixture + generator committed under jvmTest resources.
Schemas confirmed against buzz-core/src/agent_turn_metric.rs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
Introduces a top-level `buzz` package modelling the block/buzz protocol in
Quartz. Buzz is a NIP-29-family relay-group workspace (relay is the source of
truth, plaintext content, server-side membership) that layers an agent +
workspace vocabulary on top, so this package models only the Buzz-custom
extensions and reuses the existing NIP-29/34/51/42/43 classes for the standard
kinds.
- BuzzKinds: the full kind registry mirroring buzz-core `kind.rs`, annotated
with where each standard kind already lives in Quartz.
- Owner Attestation (NIP-OA): the owner-signed `auth` tag that lets agents act
as first-class members. Commitment = SHA-256("nostr:agent-auth:" + agent +
":" + conditions), BIP-340 Schnorr-signed by the owner; conditions grammar
(kind=/created_at</created_at>, canonical decimals, u16/u32 bounds) matches
buzz-sdk `nip_oa.rs` exactly.
- Tests pin Buzz's own published known-answer vector as a cross-implementation
compliance check: our preimage hash equals Buzz's SHA-256 and our verifier
accepts Buzz's reference signature.
Confirmed against the authoritative Rust (buzz-core/buzz-sdk), not the prose
NIP drafts, which lag the implementation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J8KBSw6smQRyXLiWHeDsZ8
Cold boot bumped the session revision ~292 times for 3 communities, driving 22
Messages rebuilds and re-deriving every plane subscription each time. Three
compounding causes, all measured on device:
1. Every refold republished state even when the fold was identical.
ConcordCommunityState and its components were plain classes, so StateFlow
conflation never applied and a prior-epoch wrap that didn't move the
anti-rollback floor still counted as a change. Make the fold result compare
by value (AuthorityResolver holds only immutable value fields; a data class
with a private constructor is fine).
2. A control wrap bumped twice — once from ingest() returning STRUCTURAL and
once from the per-session state watcher reacting to the same refold. Add
ConcordIngestOutcome.STRUCTURAL_FOLD for the two control-plane branches so
the manager leaves those to the watcher, which (given 1) now fires only on
genuine change. Guestbook and base-rekey keep STRUCTURAL: they mutate
members/the rekey buffer, not state, so no watcher covers them.
3. refold() and controlFloorsLocked() re-opened the WHOLE wrap buffer on every
control wrap, and opening a wrap is a NIP-44 decrypt + parse — making a
backfill quadratic in decryptions (~8.6k opens to ingest 93 wraps for one
community). Memoize editions by wrap id: one open per wrap, ingest() stays
synchronous and results are unchanged.
Measured over one cold boot: revision bumps 292 -> 87, Messages rebuilds
22 -> 7, and time from first fold to all 17 channels 43s -> 7.5s.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
FilterIndex registration runs on every REQ open/close but built each
new snapshot by copying both full maps — O(S) work and allocation per
REQ with S live subscriptions. Persistent (HAMT) maps keep the
wait-free single-load reads and CAS write loop while making a write
O(keys x log S) with structural sharing.
SmallReqFloorBenchmark grows a B@1k stage (1000 idle parked
subscriptions) to make the population cost visible, and its B stage
now enters queryRaw undispatched like production does: @1000 subs the
per-REQ cost drops 0.225 -> 0.151 ms and the measured population
penalty falls below run noise (was +0.011 ms per REQ).
With this and the undispatched replay, the in-process floor above the
raw store query is ~0.11 ms (was ~0.66 ms as first measured): A 0.120,
B 0.203, C 0.239 ms on a quiet machine.
Verified: FilterIndex tests, quartz relay.server suite.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
SmallReqFloorBenchmark showed the per-REQ floor on small results is
dominated by pipeline, not the store (raw query 0.125 ms vs 0.785 ms
session REQ->EOSE in-process). Half of the dispatch slice was the
scheduler hop between handleReq's launch and the query coroutine:
starting the job with CoroutineStart.UNDISPATCHED runs the stored
replay and EOSE inline on the receiving coroutine (the reader-pool
acquire doesn't suspend when a connection is free), parking only at
the live tail. Measured: dispatch+frames slice 0.397 -> 0.207 ms.
Commands on a connection are processed sequentially, so nothing can
target the subscription before the job lands in the registry at the
first suspension point.
Verified: quartz relay.server suite, SmallReqFloorBenchmark, geode
full test suite.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
TagAuthorIndexBenchmark at 1M events settles the flag: the DM-room
shape (kinds + authors + #p, 65 client assembler call sites) drops
14.2 ms -> 0.66 ms (~21x, growing with corpus size) while batch-insert
cost stays inside run noise (49.0 vs 47.4 us/event). Existing relay
DBs build the index on next open via ensureOptionalIndexes.
Also refreshes the docs the numbers made stale: IndexingStrategy KDoc
now records the 200k and 1M measurements instead of a TODO,
MergeQueryExecutor's tag-merge note points at the new relayBench
reactions-watch scenario, FsQueryPlanner/FsDriverSelectionBenchmark
reflect the landed cost-based pick (149 ms -> 4.0 ms at 30k events),
and RELAY.md documents that strategy flag flips materialize indexes on
the next open.
Verified: quartz jvmTest store suites, geode test (126), desktopApp
LocalRelayStore tests (5, incl. reopening a default-strategy DB with
the new pubkey-alone flag), relayBench compiles.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
Acts on the measured gaps from TagAuthorIndexBenchmark and
FsDriverSelectionBenchmark:
- FsQueryPlanner: replace the fixed tags -> kinds -> authors driver
order with a cost-based pick. Every legal driver (each tagsAll value,
each tags key's value union, the kind set, the author set) opens a
lazy directory iterator; all are drained in lockstep and the first to
exhaust (the smallest listing) drives, so a giant idx/kind tree is
never read past ~the smallest candidate's size. Fixes the 149 ms vs
3.4 ms (~44x at 30k events) authors+kinds+limit regression.
- EventIndexesModule.ensureOptionalIndexes + SQLiteEventStore: flag-
gated indexes are runtime config, not schema. An idempotent
CREATE INDEX IF NOT EXISTS pass now runs on every open, so flipping
an IndexingStrategy flag on an existing DB builds the index without
a user_version bump.
- Desktop LocalRelayStore: enable indexEventsByPubkeyAlone. Shared
ViewModels (Nip65RelayList, PrivateOutboxRelayList, VanishRequests)
replay authors-only filters that full-scanned without
(pubkey, created_at); existing DBs pick the index up on next open.
- relayBench Scenarios: add "conversation" (tag ∩ author ∩ kind, the
DM-room shape, 65 client assembler call sites) and "reactions-watch"
(kind 7 + #e IN 150 hottest notes) so the uncovered archetypes get
head-to-head numbers vs strfry.
- quartz build: forward tagBenchScale/fsBenchScale to the test JVM.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
Real numbers from the two new benchmarks, so the trade-offs are on the
decision points instead of in a chat log:
- IndexingStrategy.indexTagsWithKindAndPubkey: the KDoc called the
kinds+authors+tags shape "rarely used", but the client assembler
survey found 65 call sites. TagAuthorIndexBenchmark @ 200k events:
DM-room query 9.4 ms -> 0.6 ms (~15x) with the flag on, insert cost
+14% (41.5 -> 47.3 us/event). TODO: re-evaluate defaults (geode).
- MergeQueryExecutor: tag-path analogue of the follow-feed collect-all
sort (kinds + #e IN [hundreds] + limit never merges). Measured
12.8 ms cold / 6.0 ms since-bounded at 200k events; revisit if
relayBench shows it at relay scale.
- FsQueryPlanner: fixed driver order sends authors+kinds+limit (the
most common CLI shape) through the kind tree. FsDriverSelection-
Benchmark @ 30k events: 149 ms -> 3.4 ms (~44x) driving from the
author tree; TODO: cost-based pick by directory entry counts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
The 2026-07 client filter-assembler survey mapped 551 Filter
constructions to ~12 query archetypes. Two hot shapes had no benchmark
coverage in prodbench or relayBench, and the FS store had none at all:
- TagAuthorIndexBenchmark: the DM-room shape (kinds + authors + #p,
65 assembler call sites) with indexTagsWithKindAndPubkey off vs on,
including the insert-cost delta of the extra index; plus the
reactions watcher (kinds=[7], #e IN 300, limit) cold and
since-bounded, which has no tag-side k-way merge today.
- FsDriverSelectionBenchmark: FsQueryPlanner's fixed driver order
(tags → kinds → authors) on authors+kinds+limit — the most common
CLI shape — comparing the current kind-tree driver against an
author-tree driver with kind post-filter.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
the files are now pure-ASCII and visually unambiguous:
- BlossomPaymentSafetyTest: raw U+202E/U+202C test payload -> escapes
- BlossomPaymentRequired: BIDI_OVERRIDES char array -> escapes
- Sanitizer: RTL_OVERRIDES and ZERO_WIDTH regex classes -> escapes
Verified by mutation: with BIDI_OVERRIDES stripping disabled,
reasonBidiOverridesAreRemoved fails, proving the escaped payload still
carries a real U+202E. Emoji ZWJ sequences in RichTextParserTest are
intentionally untouched (functional joiners, not bidi controls).
Addresses review findings from a merge-time audit.
- **Complete the headline multi-value `clone` fix for PRs** (was applied only to
kind:30617). GitPullRequestEvent (1618) and GitPullRequestUpdateEvent (1619)
carry `clone` with the same spec shape but still emitted repeated single-value
tags and read only the first value — so the exact interop bug this branch set
out to kill was still live for PRs, both directions (ngit keeps only the last
repeated tag; we lost every URL after the first from ngit's multi-value tag).
Now both emit one multi-value `["clone", …]` tag and read both forms. Verified
on the wire + GitNip34InteropTest + CLI harness (40 checks).
- **Android git-status spoofing (GitStatusIndex)**: newest-status-wins with no
author check meant anyone could publish a kind-1632 and make someone else's
issue render closed. Now filter statuses to the repository owner (from the
status's own `a` tag), declared maintainers (from the cached announcement), or
the target item's author — matching NIP-34 and the CLI's derivation. Pre-existing
on main; this branch made the CLI/Android divergence visible.
- **CLI robustness**: `git comment`/`git patch` no longer block forever reading
stdin on an interactive TTY (amy is non-interactive — error instead). The local
`git` subprocesses in `git init`/`git apply` now drain stdout on a side thread
under a bounded `waitFor` + `destroyForcibly`, so a wedged git can't hang the
CLI.
Left as a follow-up (cosmetic): GitBrowseCommands.candidateUrls duplicates
GitRepositoryBrowserViewModel's — worth lifting to shared code, not worth the
cross-module coupling here.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UKMaNoK5M2PQKCAxhxWzPr
Findings from a review pass over the git-parity branch, with fixes:
- **`git init` announced a WRONG earliest-unique-commit on shallow clones**
(interop-critical). `git rev-list --max-parents=0 HEAD` returns the shallow
boundary commits, not the true root, so the repo would be announced under a
different cross-fork identity than ngit computes. Now: detect shallow clones
and omit the euc with a warning to pass `--earliest-commit`; on full clones
derive the deterministic `--first-parent` mainline root instead of an
arbitrary `tail -1`.
- **`git issues|patches|prs` silently truncated and mis-derived status** on
active repos: one single-page `drain` pulled items AND status events under a
shared cap, so status events (newer, more numerous) could crowd items out of
the window and the close-status that determines an item's state could fall
outside it → a closed item read as open. Now paginate the items
(`drainAllPages`) and fetch exactly the statuses that `e`-reference them.
Verified on the live amethyst repo: 51 PRs paginated, 19 correctly closed.
- **Pipe-buffer deadlocks** (latent): `GitInitCommand.git()` discards stderr to
the OS (a chatty command can no longer fill its stderr pipe and hang the
stdout read); `GitApplyCommand.runGit()` writes stdin on a background thread
while draining stdout, so a patch larger than the pipe buffer can't deadlock.
- Minor: `git cat` binary detection uses an index loop instead of boxing 8000
bytes; `GitRepositoryEvent.clones()/webs()` dedupe.
The harness `git init` test now runs against a fresh full checkout (this repo's
CI checkout is shallow) and adds a shallow-clone case asserting the euc is
omitted. 38/38.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UKMaNoK5M2PQKCAxhxWzPr
Verified Amethyst's NIP-34 events byte-for-byte against the ngit reference
implementation (DanConwayDev/ngit-cli) and the spec, and fixed three real
interop divergences in quartz — so ngit/gitworkshop and Amethyst read each
other's git repos, issues, patches, and PRs without losing data.
- Repository announcement `clone`/`web` were emitted as REPEATED single-value
tags (`["clone", a]`, `["clone", b]`). The spec and ngit use ONE multi-value
tag (`["clone", a, b]`), and ngit's parser keeps only the LAST of repeated
known tags — so multi-URL repos silently lost every URL but one in both
directions. Now emitted as a single multi-value tag; `clones()`/`webs()` read
BOTH the spec form and the legacy repeated form, so old events still parse.
(`relays`/`maintainers` were already correct multi-value tags.)
- Issues (kind 1621) were missing the `["p", <repo-owner>]` tag that patches and
PRs already include — a maintainer watching `#p` wouldn't see them. The
builder now adds it (fixes both the CLI and the Android issue-creation path,
which both passed an empty notify list).
- Patch / PR / PR-update `r` tags carried the `"euc"` marker
(`["r", commit, "euc"]`). Per the spec and ngit that marker belongs only on
the kind-30617 announcement; other `r` tags are plain `["r", commit]`. A `#r`
filter matches either shape, so this is a spec-compliance/byte-parity fix.
`alt` (NIP-31) tags are intentionally still omitted — quartz treats the generic
alt client-hint as deprecated, and ngit/gitworkshop parse the structured tags,
so it isn't required for interop.
Adds `GitNip34InteropTest` (5 cases: multi-value write, tolerant read of both
forms, issue p-tag, plain patch r-tag) and 4 wire-format assertions to the CLI
git harness (37 offline). No regressions in the nip34 or Search suites.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UKMaNoK5M2PQKCAxhxWzPr
Merges nostr proposal b07eb505 into main:
- feat(nip05): add Nip05Id.parseLenient for mention/text rendering
- feat(compose): wire NIP-05 popover mentions to nostr:nprofile1…
Also closes duplicate proposal 4b90b41f, which pointed at the same commits.
Beyond the feature, this replaces the unvalidated `Nip05Id("_", prefix)` raw
constructor in UserSuggestionState with `Nip05Id.parseLenient(prefix)`, closing
a hole where a typed mention such as `evil.com#x.bit` produced a GET to an
arbitrary host via `toUserUrl()`'s bare interpolation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes the privilege escalation: any BAN holder could ban the authorities above
them — including the owner — because the Banlist gate checked only the BAN bit.
Once banned, a member loses all authority (`hasPermission` is `!isBanned && ..`)
and honest clients drop their events, so a single edition from the most junior
moderator permanently silenced every admin above them.
CORD-04 §3 requires the rank half: "One hard rule binds every action: the actor
must hold the required bit and strictly outrank its target — equal cannot act on
equal (an admin cannot ban a peer admin)", restated as §5 step 3. Only §4, which
defines the Banlist, states the bit half alone — which is why both this client
and Armada shipped the same rank-blind gate.
§3 is stated per TARGET while the Banlist is one whole-list document, so it is
enforced as a DELTA rule: an edition may only add or remove npubs its signer
strictly outranks, judged against the roster settled behind it; the owner is
never a valid target (position 0 is "supreme and unremovable"); and entries the
signer may not act on are IGNORED rather than rejecting the edition, so one bad
entry cannot discard the bulk-ban §4 recommends as the collision remedy, and a
rogue cannot grief the list by forcing rejections.
ConcordModeration.currentBanned now reads the honored banlist through the
resolver instead of decoding the raw head. Besides picking up the fork healing
it was missing, this closes a laundering path: our own next ban/unban would
otherwise re-publish an entry our fold refuses, under our signature.
BREAKING (consensus): Armada has not shipped this rule, so banlists can differ
between clients until it does — we now ignore a ban Armada honors whenever the
signer did not outrank the target. Shipping the spec-conformant behaviour was
judged better than continuing to honor an escalation. Write-up to send upstream
is docs/concord-banlist-rank-conformance.md.
The three tests added in 0ae6bc6698 as @Ignore-d documentation now pass and are
un-ignored; two companions (a moderator still bans a plain member, the owner
still bans anyone) passed throughout and pin what the fix had to preserve.
Full :quartz:jvmTest and :commons:jvmTest suites green.
Still open and documented, not addressed here: a banned BAN holder can lift
their own ban (a fixpoint-ordering question that needs a spec ruling), and a
forked ban survives an unban that does not chain onto it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ban/Remove were offered to any BAN holder against any non-owner, ignoring rank
— unlike the role picker, which routes through `canActOn`. Both the Members
roster and the message-level path (`Account.concordBanTarget`, the chokepoint
for the quick-action menu, the note dropdown, the note action sections and the
chat action sheet) now require `canActOn(me, target, BAN)`.
This is NOT the no-op it first looked like. The premise that the fold would
drop such a ban is wrong, and a test proves it: BANLIST is a single whole-list
entity, so `authorizedHeads`/`banGate` gate on the author's BAN bit alone and
never rank-check the list's *contents*. A rank-5 moderator's ban of a rank-1
admin is therefore ACCEPTED by every client, and the admin then loses every
permission, since `hasPermission` is `!isBanned && ..`. It is privilege
escalation, not a silent no-op.
The fold is deliberately left alone. Armada has the identical gap — its
`banlistGate` calls the rank-blind `isAuthorized(.., Permissions.BAN)` while
its role path uses the rank-aware `canActOnPosition` — so rank-gating our fold
would make us ignore bans every other client honors, splitting the banlist
across clients. Closing it needs a spec change, like CORD-05. Refusing to
AUTHOR such a ban restricts only what we write, never what we accept, so it
cannot diverge consensus.
Three `@Ignore`-d tests in AuthorityResolverTest state the fold-level invariant
and currently fail by design; two companions assert the gate does not
over-correct (a moderator still bans a plain member; the owner still bans
anyone). Un-ignore the first three when the spec closes the gap.
The owner short-circuits the check rather than going through `canActOn`, which
begins at `hasPermission` and is false while banned — since a rogue BAN holder
*can* currently banlist the owner, routing them through it would let them be
locked out of moderating their own community.
Device-verified on Amethyst QA Concord as Dr. Edo (QA Lead, rank 2): Bob
(Admin, rank 1) now offers only the disabled "Roles… / You don't outrank this
member" where Ban and Remove used to be enabled, while the Helper (rank 5)
still offers Roles…, Ban and Remove.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An unauthorized control edition in the middle of an entity's chain
permanently froze that entity. Observed on device for a member's GRANT:
v0 owner (grant mods)
v1 owner (grant admins) <- fold stopped here, forever
v2 MIDTIER (escalation, correctly rejected)
v3,v4,v5 owner orphaned, unreachable
`AuthorityResolver` filtered unauthorized editions out BEFORE calling
`EditionFold.foldEntity`, and the walk only advances when the next
version cites the current head's hash. Removing v2 severed the chain, so
every honest edition above it was lost. Any member could permanently
freeze any member's role assignment — including the owner's ability to
change it — with a single event, recoverable only by a Refounding. It
predates the recent rank gates (verified with a zero-role identity); the
gates only widen which editions can poison.
Armada does not have this bug, and its approach settles the design.
Reading its control-plane fold (read for semantics only — Armada is
AGPLv3, Amethyst is MIT, no code taken): the chain walk runs over the
UNFILTERED set, producing an ordered candidate list — chain-verified head
first, then every remaining edition version-descending — and authority is
applied AFTERWARDS, per candidate, picking the first admissible one. A
rejected edition is skipped during the ascending admissibility walk
without truncating it. For the chain above, Armada picks v5.
So the fix is not to filter later but to gate later: `EditionFold` gains
candidate-based gated folding, and the resolver and community state now
gate per candidate instead of pre-filtering the pool. Authority checks
themselves are unchanged — only WHEN they run moved. Applied to ROLE,
GRANT, BANLIST, CHANNEL, METADATA and the authorized-head map.
The writer had to be fixed too, for a sharper reason than expected. With
an ungated `headOf`, a rogue banlist edition at the tip is read as
current state, so the owner's next ban REPUBLISHES THE ROGUE'S CONTENT
UNDER THE OWNER'S SIGNATURE — an unauthorized empty banlist laundered
into an owner-signed one the moment the owner bans anyone else. Tolerant
reading cannot heal that, because the resulting edition is genuinely
authorized. `ConcordModeration.headOf` now folds the authority-gated
heads, and `owner` is a REQUIRED parameter rather than defaulted, since a
silently-wrong default here is a consensus footgun.
Banlist healing is preserved with one necessary change: the ancestry walk
now runs over the full pool rather than the authorized subset. Ancestry is
structural — walking only authorized editions stops at the rejected one
and misreads genuine ancestors as concurrent forks, resurrecting bans an
unban had cleared.
Six regression tests, each verified to fail without the fix. Two process
notes worth recording: the first "without the fix" run reported BUILD
SUCCESSFUL because Gradle served a stale up-to-date `jvmTest` — trusting
it would have meant concluding the tests were worthless. And the
forged-edition test initially passed both ways because the forgery's
content coincided with the honest outcome; it was rewritten so the
mid-chain arm genuinely discriminates.
The rank-gate, rogue-higher-version, floor and rollback tests all pass
unchanged.
Known gap: `headOf` gates through the per-kind permission map, which is
coarser than the resolver's rank gates, so the writer can still pick a
head the reader rejects when an in-permission but out-of-rank edition
sits at the tip. Tolerant reading makes that benign, but it is not an
exact reader/writer match; tightening it needs the resolver to expose
per-entity heads.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two problems in the remote signer, both about a client getting something
without the user meaningfully agreeing to it.
**`get_public_key` and `get_relays` answered anyone.** Every other method
runs through `ifAuthorized`; these did not, and nothing required a prior
successful `connect`. The service decrypts and dispatches any well-formed
kind-24133 envelope, so anyone holding the `bunker://` URI — pasted into a
malicious app, posted for support, leaked in a screenshot — could ask it
which account it belongs to, without the secret and without connecting.
`get_relays` additionally handed over the inbox relay set. That defeated
the transport/identity split, which otherwise works: the relay-visible
traffic really is anonymous, since the p-tag and author are a transport
key and the payload is NIP-44.
Both now require the client to be paired. The authorizer interface gains
`isPaired` with NO default, so a future authorizer has to state its own
rule rather than silently inheriting "everyone is paired".
`ping` is deliberately left open. It reveals nothing the caller does not
already have — a signer is alive at a pubkey they hold — and first-party
behaviour could be confirmed but third-party clients that ping before
connecting could not be ruled out. Breaking a legitimate handshake to
close a minor oracle is a bad trade. The choice is pinned by a test that
also asserts the pairing check is never consulted, so it stays deliberate
rather than drifting back by accident.
**Decrypt consent showed nothing at all.** The bridge populated the
content preview and raw data only for signing requests, so a decrypt
request produced an empty preview block — no ciphertext, no counterparty,
not even the "Show event" toggle — leaving "AppName wants to read your
private messages" with *Allow always* as the primary button. Meanwhile
the coordinator documented the opposite: "Amethyst decrypts first, then
asks permission to expose." That was never implemented.
Now:
- The counterparty is resolved and shown, so the prompt reads "…read your
private messages **with Alice**". It never degrades to nothing —
cached name, else a shortened npub. Knowing *whose* messages is a
categorically different decision.
- The message is decrypted BEFORE prompting and the plaintext is the
preview, as documented. It is a local operation and nothing is exposed
until approval. Failure, blank and hang all collapse to an explanatory
string under a timeout, so the dialog is never empty and cannot stall.
- A narrower grant is offered ALONGSIDE the broad one, not instead of it:
`DecryptFrom(counterparty)` keyed `decrypt:<hex>` next to `Decrypt`.
The dialog's primary button becomes "Always allow for Alice" with the
broad option demoted. Because the ledger stores an opaque op key, no
persisted decision migrates and the storage format is untouched.
Scoping decrypt per counterparty *instead* would have been worse than
the bug: a DM client would prompt once per conversation, training users
to approve everything. A narrow option beside the broad one gives
granularity without the prompt explosion.
Also fixes a latent bug found on the way: `AllowForSession` recorded the
*requested* op rather than the *granted* one, which would have widened a
narrow session grant back to broad.
Verified by three sabotage passes; the tests that stayed green under them
are the intended negative guards. One existing test asserted the buggy
behaviour outright ("public reads are never gated") and was rewritten.
Not done: the batched consent sheet still records the broad op for
"remember" — offering the narrow choice per row there is a UX design
question, not a mechanical change.
Needs a device check before release: the decrypt preview runs the account
signer before consent. That is free for a local key, but an account backed
by an external NIP-55 signer (Amber) may show Amber's own prompt ahead of
Amethyst's.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Vitor PamplonaandGitHub
Claude
davotoula