mirror of
https://github.com/vitorpamplona/amethyst.git
synced 2026-10-05 19:28:25 +00:00
docs: add nip85-trusted-assertions and searchable-events skills
Completes the store-implementer skill set requested by the vespa-eventstore consumer: the NIP-85 trust-assertion model (kind map, tag vocabulary, value semantics, authorization conventions) and the NIP-50 indexing surface (the SearchableEvent contract plus an exhaustive kind -> indexableContent table external search engines can diff at version bumps). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ADBfjWhsXyHZb7ea2eeBhJ
This commit is contained in:
Claude
@@ -85,3 +85,30 @@ skills verified clean):
|
||||
`ParseReturn.entity` (the `Nip19Parser.Return.*` sealed class never existed);
|
||||
Event Store section corrected from "Android only" to commonMain/all platforms
|
||||
with the real `store.sqlite.EventStore` import and suspend generic `query<T>`.
|
||||
|
||||
## Phase 4 (2026-08): Store-implementer skills (external consumer request)
|
||||
|
||||
Three skills added at the request of an external Quartz consumer
|
||||
(vespa-eventstore — a server-side `IEventStore` on Vespa that asserts result
|
||||
parity against the SQLite store in CI). All three document the
|
||||
**store/relay-implementer's perspective**, which `quartz-integration` and
|
||||
`nostr-expert` (client-side) did not cover. Requirements doc: the skill-requests
|
||||
file reviewed 2026-08-04; the requester's items #4 (storage-lifecycle-nips) was
|
||||
folded into `event-store-semantics` per their own recommendation, and #5
|
||||
(relay-server/geode policies) was declined as not currently needed.
|
||||
|
||||
- **`event-store-semantics/`** — the `IEventStore`/SQLite-store behavioral
|
||||
contract as named rules (STORE-Fxx/Wxx/Dxx/Cxx/Sxx/Nxx) with a semantics
|
||||
changelog for pin-bump review. Written from `QueryBuilder`,
|
||||
`MergeQueryExecutor`, the seven `*Module.kt` files, and `IEventStore` KDoc.
|
||||
- **`nip85-trusted-assertions/`** — the NIP-85 model (10040/30382/30383/30384/
|
||||
30385), full tag vocabulary with value semantics, authorization conventions,
|
||||
worked JSON examples, stability notes.
|
||||
- **`searchable-events/`** — the `SearchableEvent` contract + maintenance
|
||||
mandate, with `references/searchable-kinds.md` holding the exhaustive
|
||||
kind → class → `indexableContent()` table (126 classes / 129 kinds) that
|
||||
external search engines diff at version bumps.
|
||||
|
||||
Follow-ups suggested but not implemented: a shared JSON test-vector corpus for
|
||||
filter semantics (testFixtures both the SQLite tests and external parity suites
|
||||
could run), and a snapshot test pinning the searchable-kind set.
|
||||
|
||||
@@ -0,0 +1,232 @@
|
||||
---
|
||||
name: nip85-trusted-assertions
|
||||
description: The NIP-85 trusted-assertions model in Quartz (`nip85TrustedAssertions/`) — kind 10040 trust-provider lists, kind 30382 contact cards / user assertions, 30383 event assertions, 30384 addressable assertions, 30385 external-id assertions. Use when building or parsing these events, working with the typed tags (RankTag, HopsTag, FollowerCountTag, ServiceProviderTag/ServiceType, …), wiring a consumer that resolves a 10040 provider entry to the 30382s it signs, ranking on assertion values, or touching the GrapeRank publisher, contact-card nicknames, or the trust projection of an external store.
|
||||
---
|
||||
|
||||
# NIP-85 Trusted Assertions — the Quartz model
|
||||
|
||||
Package: `quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip85TrustedAssertions/`.
|
||||
NIP-85 is still an evolving spec; **this package is the operative definition** of what
|
||||
Amethyst-family software writes and reads. This skill states the model (who signs what about
|
||||
whom), the exact kind/d-tag/tag vocabulary, and what consumers may — and may not — assume.
|
||||
|
||||
## The model in one paragraph
|
||||
|
||||
An **assertion is signed by the asserting party** (a trust provider service, or the user
|
||||
themself) **about a subject named in the d-tag**. All assertion kinds are addressable, so
|
||||
"latest card by provider P about subject S" is just the addressable coordinate
|
||||
`(kind, P, S)` and supersession is standard NIP-01 latest-wins. Discovery is the observer's
|
||||
**kind 10040 list**: each entry says *"for metric M on kind K, I trust provider P — fetch
|
||||
their assertions at relay R"*. Quartz enforces none of this cryptographically beyond normal
|
||||
event signatures; the 10040→assertion link is **consumer-side convention** (see
|
||||
"Authorization" below).
|
||||
|
||||
## Kind map
|
||||
|
||||
| Kind | Class | Kind class | d-tag = the subject | Content |
|
||||
|---|---|---|---|---|
|
||||
| 10040 | `list/TrustProviderListEvent` | replaceable | *(none — always `""`)* | NIP-44 private provider entries (optional) |
|
||||
| 30382 | `users/ContactCardEvent` | addressable | **target user's pubkey** (hex) | NIP-44 private tags (petname/summary/emoji) |
|
||||
| 30383 | `events/EventAssertionEvent` | addressable | **target event id** (hex) | `""` |
|
||||
| 30384 | `addressables/AddressableAssertionEvent` | addressable | **target coordinate** `kind:pubkey:dtag` | `""` |
|
||||
| 30385 | `externalIds/ExternalIdAssertionEvent` | addressable | **external identifier** (e.g. `isbn:978-0-13-468599-1`) | `""` |
|
||||
|
||||
Addresses: `ContactCardEvent.createAddress(owner, target)` → `Address(30382, owner, target)`
|
||||
(owner = signer, target = subject). `TrustProviderListEvent.createAddress(pubKey)` uses
|
||||
`FIXED_D_TAG = ""`. `AssertionEventTest.eventKindsAreCorrect` pins all five numbers.
|
||||
|
||||
`ContactCardEvent` is also a `SearchableEvent` — it indexes only the **public** petname/summary
|
||||
tags plus topics; the encrypted card content is intentionally never indexed.
|
||||
|
||||
## The 10040 provider entry (`ServiceProviderTag` / `ServiceType`)
|
||||
|
||||
There is **no fixed tag name**: `tag[0]` *is* the service string.
|
||||
|
||||
```json
|
||||
["30382:rank", "<provider pubkey, 64 hex>", "wss://nip85.brainstorm.world"]
|
||||
```
|
||||
|
||||
- `ServiceType(kind, type)` parses/renders `"<kind>:<type>"` — kind must be an int, the first
|
||||
`:` splits, colons in the remainder stay in `type`. `ServiceType.isOfKind` is the
|
||||
allocation-free prefix check.
|
||||
- `ServiceProviderTag.parse` requires ≥3 elements, non-empty service, 64-char pubkey
|
||||
(length-only check), and a **normalizable relay URL** (`RelayUrlNormalizer.normalizeOrNull`) —
|
||||
entries failing any check are silently dropped, which is what keeps foreign tags like
|
||||
`["client","nostria"]` out (regression-tested in `ServiceTypeParserTest`).
|
||||
- Entries may be **public** (tag array) or **private** (NIP-44 content); `create`/`add` take
|
||||
`isPrivate`. `remove` always needs decryption and strips from both sides by parsed-value
|
||||
equality.
|
||||
- `object ProviderTypes` (`list/tags/ServiceType.kt`) enumerates the *known* service types —
|
||||
`30382:rank`, `30382:followers`, `30382:first_created_at`, per-metric `30383:*`/`30384:*`/
|
||||
`30385:*`, etc. It is an **open vocabulary**: real 10040s in the wild (see the fiatjaf →
|
||||
brainstorm fixture in `commonTest/.../nip85TrustedAssertions/ServiceParser.kt`) carry types
|
||||
Quartz doesn't enumerate (`30382:personalizedGrapeRank_influence`, `30382:hops`,
|
||||
`30382:verifiedFollowersCount`, …). Parse any `kind:type`; special-case only what you rank on.
|
||||
|
||||
## Authorization — what a consumer may assume
|
||||
|
||||
- **A 30382 (or 30383/…) is meaningful to an observer only if its author is listed in the
|
||||
observer's 10040 for a matching service type.** Quartz does not enforce this; the consuming
|
||||
code does. The in-repo pattern is `commons/.../model/nip85TrustedAssertions/UserCardsCache.kt`:
|
||||
`rankFlow(trustProviderList)` picks the received card whose **author pubkey equals the
|
||||
provider entry's pubkey** and reads `rank()` from it. Assertions from unlisted signers are
|
||||
simply ignored for trust purposes (they may still be stored; dropping them — as an external
|
||||
store's orphan sweep does — is a legitimate storage policy, not a protocol rule).
|
||||
- What an entry authorizes is scoped by its `ServiceType`: `30382:rank` authorizes that
|
||||
provider's user-rank cards, nothing else. Amethyst models this as one provider slot per
|
||||
metric (`liveUserRankProvider`, `liveUserFollowerCount` in
|
||||
`amethyst/.../model/trustedAssertions/TrustProviderListState.kt`).
|
||||
- **Multi-provider combination is unprescribed.** When two listed providers assert different
|
||||
ranks, there is no spec'd merge; Amethyst avoids the question by selecting one provider per
|
||||
metric slot. Consumers choose their own policy — document it.
|
||||
- The relay URL in the entry is a **fetch hint, and it is honored**:
|
||||
`amethyst/.../UserCardsSubAssembler.kt` subscribes for cards at the provider's declared relay
|
||||
(`kinds=[30382], authors=[provider], #d=[targets]`).
|
||||
|
||||
### The dual use of kind 30382
|
||||
|
||||
The same kind serves two roles, distinguished **by author**:
|
||||
|
||||
1. **Provider WoT cards** — signed by a trust provider; public metric tags (`rank`,
|
||||
`followers`, `hops`, …); this is what 10040 discovery points at.
|
||||
2. **The account's own contact cards (nicknames, NIP-81-style)** — signed by the account,
|
||||
one per target user. The petname, summary, and their NIP-30 emoji mappings **always live in
|
||||
the NIP-44 encrypted content, never in public tags** (`ContactCardEvent.build`/
|
||||
`updatePetNameAndSummary` strip stray public copies; asserted by `ContactCardPetNameTest`).
|
||||
`commons/.../ContactCardsState.kt` keys everything on `author == account` and ignores
|
||||
provider cards.
|
||||
|
||||
## Tag vocabulary and value semantics
|
||||
|
||||
All tag classes share one shape: `TAG_NAME` + `parse(tag)` (null on wrong name/empty/non-numeric
|
||||
value — a bad tag is *dropped*, never an error) + `assemble(value)` → `[name, value.toString()]`.
|
||||
**A missing tag means "unknown" (`null` accessor), never zero.** There is deliberately no range
|
||||
validation (rank isn't clamped, hours aren't checked against 0–23, counts may be negative) —
|
||||
consumers must defend.
|
||||
|
||||
**On 30382** (`users/tags/`, accessors on `ContactCardEvent` and as `TagArray` extensions in
|
||||
`users/TagArrayExt.kt` so they also work on decrypted private arrays):
|
||||
|
||||
| Tag name | Accessor | Type | Semantics |
|
||||
|---|---|---|---|
|
||||
| `rank` | `rank()` | Int | Provider-relative score; higher is better. GrapeRank publishes `round(score × 100)` (so 0–100 in practice), but nothing enforces a scale — treat it as comparable only *within one provider*. |
|
||||
| `followers` | `followerCount()` | Int | Follower count as the provider computes it (cumulative, provider-defined). |
|
||||
| `hops` | `hops()` | Int | Shortest follow-path length **from the observer the provider computed for** to the subject (1 = directly followed). Mirrors Brainstorm GrapeRank's `hops`. The only tag with KDoc. |
|
||||
| `first_created_at` | `firstCreatedAt()` | Long | Unix seconds of subject's earliest known event. |
|
||||
| `post_cnt` / `reply_cnt` / `reactions_cnt` | `postCount()` etc. | Int | Activity counts. |
|
||||
| `zap_amt_recd` / `zap_amt_sent` | `zapAmountReceived()`/`…Sent()` | Long | Sats. |
|
||||
| `zap_cnt_recd` / `zap_cnt_sent` | `zapCountReceived()`/`…Sent()` | Int | Counts. |
|
||||
| `zap_avg_amt_day_recd` / `zap_avg_amt_day_sent` | `zapAvgAmountDay…()` | Long | Sats/day averages. |
|
||||
| `reports_cnt_recd` / `reports_cnt_sent` | `reportsCount…()` | Int | NIP-56 report counts. |
|
||||
| `t` (repeatable) | `topics()` | List\<String> | Subject's topics/interests. |
|
||||
| `active_hours_start` / `active_hours_end` | `activeHours…()` | Int | Hour-of-day; **no timezone is specified in code** — treat as provider-defined (UTC in practice) and unclamped. |
|
||||
| `petname` / `summary` | `petName()`/`summary()` | String | Nickname fields — conventionally private (see dual use above). |
|
||||
|
||||
**On 30383/30384** (`tags/`, shared): `rank`, `comment_cnt`, `quote_cnt`, `repost_cnt`,
|
||||
`reaction_cnt`, `zap_cnt` (Int) and `zap_amount` (Long, sats).
|
||||
**On 30385**: only `rank`, `comment_cnt`, `reaction_cnt`.
|
||||
|
||||
## Building and parsing (use the typed helpers, not raw `arrayOf`)
|
||||
|
||||
```kotlin
|
||||
// Provider list: declare a rank provider (this is what `amy graperank register` does)
|
||||
val tag = ServiceProviderTag(ProviderTypes.rank, providerPubkeyHex, relayUrl)
|
||||
val list = TrustProviderListEvent.create(tag, isPrivate = false, signer)
|
||||
// or append to an existing one:
|
||||
val updated = TrustProviderListEvent.add(existing, tag, isPrivate = false, signer)
|
||||
val providers: List<ServiceProviderTag> = updated.serviceProviders() // public
|
||||
val private = updated.privateTags(signer)?.serviceProviders() // private side
|
||||
|
||||
// Provider-style contact card (public metrics) — the GrapeRankPublisher pattern:
|
||||
val card = ContactCardEvent.create(
|
||||
targetUser = subjectPubkey,
|
||||
signer = providerSigner,
|
||||
publicInitializer = {
|
||||
rank(87)
|
||||
followers(1234)
|
||||
hops(2)
|
||||
},
|
||||
)
|
||||
card.aboutUser() // d-tag → subject pubkey
|
||||
card.rank() // 87
|
||||
|
||||
// Event assertion: unsigned template only (30383/84/85 have build(), no create())
|
||||
val template = EventAssertionEvent.build(targetEventId) {
|
||||
rank(12)
|
||||
reactionCount(40)
|
||||
zapAmount(2100)
|
||||
}
|
||||
val signed = signer.sign(template)
|
||||
```
|
||||
|
||||
## Worked end-to-end example
|
||||
|
||||
Observer `O` trusts provider `P` for user ranks (kind 10040, replaceable, by `O`):
|
||||
|
||||
```json
|
||||
{ "kind": 10040, "pubkey": "<O>",
|
||||
"tags": [
|
||||
["30382:rank", "<P>", "wss://nip85.brainstorm.world"],
|
||||
["30382:followers", "<P>", "wss://nip85.brainstorm.world"]
|
||||
],
|
||||
"content": "" }
|
||||
```
|
||||
|
||||
Provider `P` asserts about subject `S` (kind 30382, addressable at `30382:<P>:<S>`):
|
||||
|
||||
```json
|
||||
{ "kind": 30382, "pubkey": "<P>",
|
||||
"tags": [
|
||||
["d", "<S>"],
|
||||
["rank", "87"], ["followers", "1234"], ["hops", "2"]
|
||||
],
|
||||
"content": "" }
|
||||
```
|
||||
|
||||
`P` asserts about an event `E` (kind 30383, addressable at `30383:<P>:<E>`):
|
||||
|
||||
```json
|
||||
{ "kind": 30383, "pubkey": "<P>",
|
||||
"tags": [["d", "<E>"], ["rank", "12"], ["reaction_cnt", "40"], ["zap_amount", "2100"]],
|
||||
"content": "" }
|
||||
```
|
||||
|
||||
Consumption chain: read `O`'s 10040 → entry matching `ServiceType(30382, "rank")` → subscribe
|
||||
`{kinds:[30382], authors:["<P>"], "#d":["<S>", …]}` at the hinted relay → newest card per
|
||||
address wins → `rank()`.
|
||||
|
||||
Literal fixtures: `quartz/src/commonTest/.../nip85TrustedAssertions/ServiceParser.kt` (a real
|
||||
10040 — fiatjaf's, pointing at the Brainstorm provider) and `AssertionEventTest.kt` (all four
|
||||
assertion kinds with every tag populated).
|
||||
|
||||
## Freshness / supersession
|
||||
|
||||
Assertions are addressable: **latest per `(kind, author, d-tag)` wins**; there is no expiry tag
|
||||
convention and **no prescribed refresh cadence** — staleness policy is the consumer's.
|
||||
Writers should avoid churn: `GrapeRankPublisher` re-signs a card only when
|
||||
`(rank, followers, hops)` actually changed, and retracts with a NIP-09 kind-5 carrying the
|
||||
card's `a`-tag (`30382:<provider>:<target>`).
|
||||
|
||||
## Stability notes (as of 2026-08)
|
||||
|
||||
- **Settled** (shipped consumers on both ends): the kind map; `ServiceProviderTag` entry shape;
|
||||
`rank`/`followers`/`hops` on 30382; petname/summary-in-encrypted-content; 10040 relay-hint
|
||||
consumption.
|
||||
- **Written but lightly consumed** (parse, but gate ranking features carefully): the activity/
|
||||
zap/report count tags, `active_hours_*` (no timezone semantics), 30383/30384/30385 (builders +
|
||||
tests exist; no in-repo publisher yet).
|
||||
- **Known warts**: `ServiceProviderTag.assemble(id: ServiceProviderTag)` infers `Array<Any>` —
|
||||
dead code, don't use it; `SummaryTag.assemble(ip:)`/`ActiveHours*Tag.assemble(count:)` params
|
||||
are misnamed; the tests live under `commonTest/.../experimental/nip85TrustedAssertions/`
|
||||
(stale path); `TrustProviderListEvent` extends the addressable base, so a stray on-wire `d`
|
||||
tag is reflected by `dTag()` even though the convention is `""`.
|
||||
|
||||
## Where it's consumed (reading list)
|
||||
|
||||
- **Publisher**: `quartz/.../experimental/graperank/GrapeRankPublisher.kt` (canonical 30382
|
||||
writer), `cli/.../graperank/` (`amy graperank register|unregister|providers|publish`).
|
||||
- **Client model**: `commons/.../model/nip85TrustedAssertions/` (`ContactCardsState`,
|
||||
`UserCardsCache`, `ContactCardDecryptionCache`, `TrustProviderListDecryptionCache`),
|
||||
`amethyst/.../model/trustedAssertions/TrustProviderListState.kt`.
|
||||
- **Relay plumbing**: `commons/.../relayClient/assemblers/ContactCardFilters.kt`,
|
||||
`amethyst/.../reqCommand/user/watchers/UserCardsSubAssembler.kt`.
|
||||
@@ -0,0 +1,117 @@
|
||||
---
|
||||
name: searchable-events
|
||||
description: The NIP-50 indexing surface of Quartz — the `SearchableEvent` interface, which event kinds are searchable, exactly what text each kind's `indexableContent()` contributes, how the SQLite/filesystem stores consume it, and the NIP-50 `SearchQuery` extension grammar plus `SearchRelayListEvent` (kind 10007). Use when making a kind searchable, changing what a kind indexes, diffing the searchable set at a Quartz version bump (external search engines mirror this table), debugging why an event is or isn't found by search, or working with search extensions (`include:spam`, `domain:`, …).
|
||||
---
|
||||
|
||||
# Searchable Events — the NIP-50 indexing surface
|
||||
|
||||
## The contract
|
||||
|
||||
`quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip50Search/SearchableEvent.kt`:
|
||||
|
||||
```kotlin
|
||||
interface SearchableEvent {
|
||||
fun indexableContent(): String
|
||||
}
|
||||
```
|
||||
|
||||
One method; marker and extractor in one. An event kind is searchable **iff** its event class
|
||||
implements this interface **and** the class is wired into `EventFactory` (the stores probe
|
||||
searchability by kind through `EventFactory.create` — an unwired implementor is invisible).
|
||||
|
||||
Rules every implementation follows (keep them when adding one):
|
||||
|
||||
- **Plain text out.** Return the human-meaningful fields joined with `"\n"` (a handful of
|
||||
metadata-ish kinds use `" "`); no markup stripping is performed — markdown/asciidoc content
|
||||
goes in raw, JSON-content kinds (kind 0 metadata, marketplace stalls, channel info) **parse
|
||||
first and join the extracted fields**, never the raw JSON.
|
||||
- **Never throw, never null.** There is no defensive wrapper at any call site; a throw aborts
|
||||
the insert transaction. Parsed-JSON implementations use `?.let { … } ?: ""`.
|
||||
- **Only public data.** Encrypted content stays out (e.g. kind 30382 contact cards index only
|
||||
the public petname/summary/topics, never the NIP-44 payload).
|
||||
- Typical shapes: `content` alone (~33 kinds); `listOfNotNull(title(), content)`;
|
||||
`listOfNotNull(title(), summary(), content)`; lists index `title() + description()`.
|
||||
|
||||
## The full kind table
|
||||
|
||||
**`references/searchable-kinds.md`** in this skill holds the authoritative table — every
|
||||
implementor with its kind number, class, and the exact `indexableContent()` expression
|
||||
(126 concrete classes / 129 kind values as of 2026-08). Diff that file at a version bump to
|
||||
answer "did the searchable set or any kind's indexed text change?".
|
||||
|
||||
Notables that surprise people:
|
||||
|
||||
- **Kind 9735 (zap receipt) indexes the embedded zap request's content**
|
||||
(`zapRequest?.content.orEmpty()`) — receipts are searchable by the zapper's comment.
|
||||
- **Kind 0 / 31990** index many profile fields space-joined (name, about, nip05, lud16,
|
||||
website, picture URL, …).
|
||||
- **Kind 30063 is claimed twice** (`ReleaseArtifactSetEvent` in nip51Lists and the experimental
|
||||
`SoftwareReleaseEvent`); `EventFactory` resolves 30063 to `ReleaseArtifactSetEvent`, so
|
||||
`title()\ndescription()` is what actually gets indexed — `SoftwareReleaseEvent.indexableContent()`
|
||||
is dead on the store path.
|
||||
- Poll kinds (1068, 6969) append each option label on its own line.
|
||||
|
||||
## MANDATORY maintenance when you touch this surface
|
||||
|
||||
Adding `SearchableEvent` to a kind, removing it, or changing any `indexableContent()` body:
|
||||
|
||||
1. **Update `references/searchable-kinds.md`** in the same PR (external search engines — e.g.
|
||||
the Vespa-backed store's `SearchExtractors` — mirror this table at pin bumps; a silent
|
||||
change ships them stale search results).
|
||||
2. **Remember existing databases don't reindex themselves.** Old rows keep their old (or
|
||||
missing) FTS text until `IEventStore.reindexFullTextSearch()` runs — the KDoc on that method
|
||||
is the contract. App-side, schedule the resumable overload after shipping such a change.
|
||||
3. New implementors must be **registered in `EventFactory`** or the reindex scan and kind
|
||||
pre-filter (`FullTextSearchModule.isSearchableKind`) will never see them.
|
||||
|
||||
Eligibility policy: a kind becomes searchable when it carries human-authored, human-meaningful
|
||||
text (titles, bodies, names, descriptions). Pure-machine kinds (reactions, follow lists, zaps
|
||||
minus their comment, relay lists) stay out to keep the index small.
|
||||
|
||||
## How the stores consume it
|
||||
|
||||
**SQLite** (`nip01Core/store/sqlite/FullTextSearchModule.kt`):
|
||||
`CREATE VIRTUAL TABLE event_fts USING fts5(content, content='', contentless_delete=1)` —
|
||||
contentless, `rowid` = `event_headers.row_id`, an `AFTER DELETE` trigger keeps it in sync. On
|
||||
insert (when FTS is on and not deferred): `if (event is SearchableEvent)` → bind
|
||||
`event.indexableContent()` — the only method ever called. Tokenization is entirely SQLite's
|
||||
default FTS5 `unicode61`; queries are always a bound `event_fts MATCH ?` (never concatenated),
|
||||
ordered by bm25 `rank` then `created_at DESC`. Query-side semantics (relevance ordering,
|
||||
extension stripping, FTS-off behavior, deferred catch-up) are rules STORE-S01…S06 in the
|
||||
`event-store-semantics` skill.
|
||||
|
||||
**Filesystem store** (`jvmMain/.../store/fs/FsIndexer.kt` + `FsSearchTokenizer.kt`): tokenizes
|
||||
`indexableContent()` itself, approximating `unicode61` (split on non-letter/digit, lowercase);
|
||||
the same tokenizer runs on queries so drift cancels.
|
||||
|
||||
## NIP-50 client side
|
||||
|
||||
**`SearchQuery`** (`nip50Search/SearchQuery.kt`) — typed parse of the `search` filter string
|
||||
into `terms` + `extensions`. A whitespace token is an extension iff it looks like
|
||||
`lowercasekey:value` (the value not starting with `//`, so URLs stay free text); duplicate keys
|
||||
keep the last; unknown extensions are preserved (`extension(key)`). Typed accessors:
|
||||
`includeSpam`, `domain`, `language`, `sentiment`, `nsfw`. `stripExtensions()` /
|
||||
`Filter.strippingSearchExtensions()` is the bridge the built-in stores use — unsupported
|
||||
extensions are **ignored** (NIP-50), so an extensions-only search collapses to an unconstrained
|
||||
query, never match-nothing. A server-side store that implements its own extensions
|
||||
(`observer:`, `sort:rank`, …) receives the raw string (see the `IEventStore` KDoc) and should
|
||||
parse with `SearchQuery.parse` so its syntax stays compatible with what clients send.
|
||||
|
||||
**`SearchRelayListEvent`** — **kind 10007**, the user's search-relay list (NIP-51-style, public
|
||||
tags + NIP-44 private tags; *not* a `SearchableEvent` itself). Client consumption:
|
||||
`commons/.../actions/SearchActions.kt`, bootstrap defaults in
|
||||
`commons/.../account/AccountBootstrapEvents.kt`.
|
||||
|
||||
Don't confuse it with `commons/.../commons/search/SearchQuery.kt` — an app-level local-feed
|
||||
query model (authors/kinds/hashtags/or-terms), unrelated to the NIP-50 wire string.
|
||||
|
||||
## Tests (executable spec)
|
||||
|
||||
- `commonTest/.../nip50Search/SearchQueryTest.kt` — the extension grammar, token by token.
|
||||
- `commonTest/.../store/sqlite/SearchTest.kt` — per-kind indexing (kind 0 profile fields,
|
||||
40/41 channel JSON, 31924/30617), extension-token ignoring, reindex/resumable-reindex,
|
||||
FTS cleanup on replaceable rotation.
|
||||
- `commonTest/.../store/sqlite/SearchRelevanceOrderTest.kt` — bm25-before-recency ordering,
|
||||
limit-after-score, multi-filter rank union.
|
||||
- `commonTest/.../store/sqlite/NoFullTextSearchTest.kt` — FTS-off contract.
|
||||
- `jvmTest/.../store/fs/FsSearchTest.kt` — tokenizer parity for the filesystem store.
|
||||
@@ -0,0 +1,166 @@
|
||||
# Searchable kinds — the authoritative implementor table
|
||||
|
||||
Every concrete `SearchableEvent` implementor in Quartz, with the exact `indexableContent()`
|
||||
expression. **Update this file in the same PR as any change to the searchable set or to an
|
||||
`indexableContent()` body** (see SKILL.md). Verified against the code 2026-08-04.
|
||||
|
||||
Counts: 126 concrete classes covering 129 kind values (`GitStatusEvent` spans 4 kinds;
|
||||
kind 30063 has a collision — see the footnote). File paths are under
|
||||
`quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/`.
|
||||
|
||||
Separator legend: **NL** = `joinToString("\n")`, **SP** = `joinToString(" ")`.
|
||||
|
||||
| Kind | Class | Package | `indexableContent()` |
|
||||
|---|---|---|---|
|
||||
| 0 | MetadataEvent | nip01Core/metadata | `contactMetaData()?.let { listOfNotNull(it.name, it.displayName, it.about, it.nip05, it.lud06, it.lud16, it.website, it.picture, it.banner).joinToString(" ") } ?: ""` (SP) |
|
||||
| 1 | TextNoteEvent | nip10Notes | `listOfNotNull(subject(), content)` NL |
|
||||
| 9 | ChatEvent | nipC7Chats | `content` |
|
||||
| 11 | ThreadEvent | nip7DThreads | `listOfNotNull(title(), content)` NL |
|
||||
| 14 | ChatMessageEvent | nip17Dm/messages | `content` |
|
||||
| 20 | PictureEvent | nip68Picture | `listOfNotNull(title(), content)` NL |
|
||||
| 21 | VideoNormalEvent | nip71Video | inherited `RegularVideoEvent`: `listOfNotNull(title(), content)` NL |
|
||||
| 22 | VideoShortEvent | nip71Video | inherited `RegularVideoEvent`: `listOfNotNull(title(), content)` NL |
|
||||
| 24 | PublicMessageEvent | nipA4PublicMessages | `content` |
|
||||
| 40 | ChannelCreateEvent | nip28PublicChat/admin | `channelInfo().let { listOfNotNull(it.name, it.about, it.picture).joinToString(" ") }` (SP) |
|
||||
| 41 | ChannelMetadataEvent | nip28PublicChat/admin | same as kind 40 (SP) |
|
||||
| 42 | ChannelMessageEvent | nip28PublicChat/message | `content` |
|
||||
| 54 | PodcastEpisodeEvent | nipF4Podcasts/episode | `listOfNotNull(title(), description(), content)` NL |
|
||||
| 1010 | TextNoteModificationEvent | experimental/edits | `listOfNotNull(content, summary())` NL (content first) |
|
||||
| 1063 | FileHeaderEvent | nip94FileMetadata | `listOfNotNull(summary(), content)` NL |
|
||||
| 1065 | FileStorageHeaderEvent | experimental/nip95/header | `listOfNotNull(summary())` NL |
|
||||
| 1068 | PollEvent | nip88Polls/poll | `buildString { append(content); options().forEach { append('\n').append(it.label) } }` |
|
||||
| 1111 | CommentEvent | nip22Comments | `(listOf(content) + tags.hashtags())` NL |
|
||||
| 1163 | ProfileGalleryEntryEvent | experimental/profileGallery | `listOfNotNull(summary())` NL |
|
||||
| 1301 | WorkoutRecordEvent | experimental/fitness/workout | `listOfNotNull(title(), content)` NL |
|
||||
| 1311 | LiveActivitiesChatMessageEvent | nip53LiveActivities/chat | `(listOf(content) + tags.hashtags())` NL |
|
||||
| 1312 | LiveActivitiesRaidEvent | nip53LiveActivities/raid | `content` |
|
||||
| 1313 | LiveActivitiesClipEvent | nip53LiveActivities/clip | `listOfNotNull(title(), content)` NL |
|
||||
| 1315 | RoadEventReportEvent | experimental/roadstr/report | `content` |
|
||||
| 1337 | CodeSnippetEvent | nipC0CodeSnippets | `listOfNotNull(snippetName(), snippetDescription(), content)` NL |
|
||||
| 1617 | GitPatchEvent | nip34Git/patch | `content` |
|
||||
| 1618 | GitPullRequestEvent | nip34Git/pr | `listOfNotNull(subject(), content)` NL |
|
||||
| 1621 | GitIssueEvent | nip34Git/issue | `listOfNotNull(subject(), content)` NL |
|
||||
| 1622 | GitReplyEvent | nip34Git/reply | `content` |
|
||||
| 1630–1633 | GitStatusEvent | nip34Git/status | `content` (open/applied/closed/draft) |
|
||||
| 1808 | AudioHeaderEvent | experimental/audio/header | `content` |
|
||||
| 1985 | LabelEvent | nip32Labeling | `(listOf(content) + labels().map { it.label }).filter { it.isNotEmpty() }` NL |
|
||||
| 2003 | TorrentEvent | nip35Torrents | `listOfNotNull(title(), content)` NL |
|
||||
| 2004 | TorrentCommentEvent | nip35Torrents | `content` |
|
||||
| 2473 | BirdDetectionEvent | experimental/birdstar | `listOfNotNull(summary(), speciesName())` NL |
|
||||
| 3302 | ConcordChatEditEvent | concord/cord03Channels | `content` |
|
||||
| 5050 | NIP90TextGenerationRequestEvent | nip90Dvms/textGeneration | `inputs().filter { it.type == "prompt" \|\| it.type == "text" }.joinToString(" ") { it.value }` (SP) |
|
||||
| 5100 | NIP90ImageGenerationRequestEvent | nip90Dvms/imageGeneration | `listOfNotNull(prompt(), negativePrompt()).joinToString(" ")` (SP) |
|
||||
| 5129 | NappletSnapshotEvent | nip5dNapplets | `listOfNotNull(title(), description())` NL |
|
||||
| 5250 | NIP90TextToSpeechRequestEvent | nip90Dvms/textToSpeech | `text() ?: ""` |
|
||||
| 5302 | NIP90ContentSearchRequestEvent | nip90Dvms/contentSearch | `searchQuery() ?: ""` |
|
||||
| 5303 | NIP90PeopleSearchRequestEvent | nip90Dvms/peopleSearch | `searchQuery() ?: ""` |
|
||||
| 6969 | ZapPollEvent | experimental/zapPolls | `buildString { append(content); pollOptionsArray().forEach { append('\n').append(it.descriptor) } }` |
|
||||
| 8333 | OnchainZapEvent | nipBCOnchainZaps/zap | `content` |
|
||||
| 9002 | EditMetadataEvent | nip29RelayGroups/moderation | `(listOfNotNull(name(), about()) + hashtags())` NL |
|
||||
| 9041 | GoalEvent | nip75ZapGoals | `listOfNotNull(summary(), content)` NL |
|
||||
| 9321 | NutzapEvent | nip61Nutzaps/nutzap | `content` |
|
||||
| 9734 | LnZapRequestEvent | nip57Zaps | `content` |
|
||||
| 9735 | LnZapEvent | nip57Zaps | `zapRequest?.content.orEmpty()` — indexes the **embedded 9734's** content |
|
||||
| 9736 | Bolt12ZapEvent | nipB1Bolt12Zaps/zap | `content` |
|
||||
| 9737 | Bolt12ZapIntentEvent | nipB1Bolt12Zaps/intent | `content` |
|
||||
| 9802 | HighlightEvent | nip84Highlights | `listOfNotNull(comment(), context(), content)` NL |
|
||||
| 10003 | BookmarkListEvent | nip51Lists/bookmarkList | `listOfNotNull(title())` NL |
|
||||
| 10100 | AgentProfileEvent | buzz/agentProfiles | `profileOrNull()?.let { listOfNotNull(it.name, it.displayName).joinToString("\n") } ?: ""` |
|
||||
| 10154 | PodcastMetadataEvent | nipF4Podcasts/metadata | `listOfNotNull(title(), description())` NL |
|
||||
| 11871 | AttestorProficiencyEvent | experimental/attestations/proficiency | `listOfNotNull(description())` NL |
|
||||
| 12473 | BirdexEvent | experimental/birdstar | `(listOfNotNull(summary()) + speciesNames())` NL |
|
||||
| 15128 | RootSiteEvent | nip5aStaticWebsites | `listOfNotNull(title(), description())` NL |
|
||||
| 15129 | RootNappletEvent | nip5dNapplets | `listOfNotNull(title(), description())` NL |
|
||||
| 30000 | PeopleListEvent | nip51Lists/peopleList | `listOfNotNull(titleOrName(), description())` NL |
|
||||
| 30001 | OldBookmarkListEvent | nip51Lists/bookmarkList | `listOfNotNull(title())` NL |
|
||||
| 30002 | RelaySetEvent | nip51Lists/relaySets | `listOfNotNull(title(), description())` NL |
|
||||
| 30003 | LabeledBookmarkListEvent | nip51Lists/labeledBookmarkList | `listOfNotNull(titleOrName(), description())` NL |
|
||||
| 30004 | ArticleCurationSetEvent | nip51Lists/articleCurationSet | `listOfNotNull(title(), description())` NL |
|
||||
| 30005 | VideoCurationSetEvent | nip51Lists/videoCurationSet | `listOfNotNull(title(), description())` NL |
|
||||
| 30006 | PictureCurationSetEvent | nip51Lists/pictureCurationSet | `listOfNotNull(title(), description())` NL |
|
||||
| 30009 | BadgeDefinitionEvent | nip58Badges/definition | `listOfNotNull(name(), description(), content)` NL |
|
||||
| 30015 | InterestSetEvent | nip51Lists/interestSet | `(listOfNotNull(title(), description()) + publicHashtags())` NL |
|
||||
| 30017 | StallEvent | nip15Marketplace/stall | `stallData()?.let { listOfNotNull(it.name, it.description).joinToString("\n") } ?: ""` |
|
||||
| 30018 | ProductEvent | nip15Marketplace/product | `productData()?.let { (listOfNotNull(it.name, it.description) + categories()).joinToString("\n") } ?: ""` |
|
||||
| 30019 | MarketplaceEvent | nip15Marketplace/marketplace | `marketplaceData()?.let { listOfNotNull(it.name, it.about).joinToString("\n") } ?: ""` |
|
||||
| 30020 | AuctionEvent | nip15Marketplace/auction | `auctionData()?.let { (listOfNotNull(it.name, it.description) + tags.hashtags()).joinToString("\n") } ?: ""` |
|
||||
| 30023 | LongTextNoteEvent | nip23LongContent | `listOfNotNull(title(), summary(), content)` NL |
|
||||
| 30030 | EmojiPackEvent | nip30CustomEmoji/pack | `listOfNotNull(titleOrName(), description(), content)` NL |
|
||||
| 30054 | Podcasting20EpisodeEvent | nipXXPodcasting20/episode | `(listOfNotNull(title(), description(), content) + topics())` NL |
|
||||
| 30055 | Podcasting20TrailerEvent | nipXXPodcasting20/trailer | `listOfNotNull(title(), content)` NL |
|
||||
| 30063 | ReleaseArtifactSetEvent † | nip51Lists/releaseArtifactSet | `listOfNotNull(title(), description())` NL |
|
||||
| 30175 | PersonaEvent | buzz/apPersonas | `personaOrNull()?.let { listOfNotNull(it.displayName, it.systemPrompt).joinToString("\n") } ?: ""` |
|
||||
| 30176 | TeamEvent | buzz/teams | `teamOrNull()?.let { listOfNotNull(it.name, it.description, it.instructions).joinToString("\n") } ?: ""` |
|
||||
| 30177 | ManagedAgentEvent | buzz/managedAgents | `agentOrNull()?.let { listOfNotNull(it.name, it.systemPrompt).joinToString("\n") } ?: ""` |
|
||||
| 30267 | AppCurationSetEvent | nip51Lists/appCurationSet | `listOfNotNull(title(), description())` NL |
|
||||
| 30296 | InteractiveStoryPrologueEvent | experimental/interactiveStories | inherited base: `listOfNotNull(title(), summary(), content)` NL |
|
||||
| 30297 | InteractiveStorySceneEvent | experimental/interactiveStories | inherited base: `listOfNotNull(title(), summary(), content)` NL |
|
||||
| 30311 | LiveActivitiesEvent | nip53LiveActivities/streaming | `listOfNotNull(title(), summary(), content)` NL |
|
||||
| 30312 | MeetingSpaceEvent | nip53LiveActivities/meetingSpaces | `listOfNotNull(room(), summary(), content)` NL |
|
||||
| 30313 | MeetingRoomEvent | nip53LiveActivities/meetingSpaces | `listOfNotNull(title(), summary())` NL |
|
||||
| 30315 | StatusEvent | nip38UserStatus | `content` |
|
||||
| 30382 | ContactCardEvent | nip85TrustedAssertions/users | `(listOfNotNull(petName(), summary()) + topics())` NL — public tags only, never the NIP-44 content |
|
||||
| 30402 | ClassifiedsEvent | nip99Classifieds | `listOfNotNull(title(), summary(), content)` NL |
|
||||
| 30617 | GitRepositoryEvent | nip34Git/repository | `listOfNotNull(name(), description(), content)` NL |
|
||||
| 30620 | WorkflowDefEvent | buzz/workflow | `listOfNotNull(name(), content)` NL |
|
||||
| 30817 | NipTextEvent | experimental/nipsOnNostr | `listOfNotNull(title(), content)` NL |
|
||||
| 30818 | WikiNoteEvent | nip54Wiki | `listOfNotNull(title(), summary(), content)` NL |
|
||||
| 31337 | AudioTrackEvent | experimental/audio/track | `listOfNotNull(subject())` NL |
|
||||
| 31871 | AttestationEvent | experimental/attestations/attestation | `content` |
|
||||
| 31872 | AttestationRequestEvent | experimental/attestations/request | `content` |
|
||||
| 31873 | AttestorRecommendationEvent | experimental/attestations/recommendation | `listOfNotNull(description())` NL |
|
||||
| 31890 | FeedDefinitionEvent | feedDefinition | `title().orEmpty()` |
|
||||
| 31922 | CalendarDateSlotEvent | nip52Calendar/appt/day | `listOfNotNull(title(), summary(), content)` NL |
|
||||
| 31923 | CalendarTimeSlotEvent | nip52Calendar/appt/time | `listOfNotNull(title(), summary(), content)` NL |
|
||||
| 31924 | CalendarEvent | nip52Calendar/calendar | `listOfNotNull(title(), content)` NL |
|
||||
| 31925 | CalendarRSVPEvent | nip52Calendar/rsvp | `content` |
|
||||
| 31990 | AppDefinitionEvent | nip89AppHandlers/definition | `appMetaData()?.let { listOfNotNull(it.name, it.username, it.displayName, it.about, it.nip05, it.lud06, it.lud16, it.website, it.picture, it.banner, it.image).joinToString(" ") } ?: ""` (SP) |
|
||||
| 32267 | SoftwareApplicationEvent | experimental/nip82SoftwareApps/application | `listOfNotNull(name(), summary(), content)` NL |
|
||||
| 33401 | ExerciseTemplateEvent | experimental/fitness/workout | `listOfNotNull(title(), content)` NL |
|
||||
| 33863 | FundraiserEvent | experimental/agora | `listOfNotNull(title(), content)` NL |
|
||||
| 34139 | MusicPlaylistEvent | experimental/music/playlist | `listOfNotNull(title(), description(), content)` NL |
|
||||
| 34235 | VideoHorizontalEvent | nip71Video | inherited `AddressableVideoEvent`: `listOfNotNull(title(), content)` NL |
|
||||
| 34236 | VideoVerticalEvent | nip71Video | inherited `AddressableVideoEvent`: `listOfNotNull(title(), content)` NL |
|
||||
| 34550 | CommunityDefinitionEvent | nip72ModCommunities/definition | `listOfNotNull(name(), description(), rules(), content)` NL |
|
||||
| 35128 | NamedSiteEvent | nip5aStaticWebsites | `listOfNotNull(title(), description())` NL |
|
||||
| 35129 | NamedNappletEvent | nip5dNapplets | `listOfNotNull(title(), description())` NL |
|
||||
| 36787 | MusicTrackEvent | experimental/music/track | `listOfNotNull(title(), artist(), album(), content)` NL |
|
||||
| 38000 | MintRecommendationEvent | nip87Ecash/recommendation | `content` |
|
||||
| 38192 | Ps1SaveEvent | experimental/ps1saves | `listOfNotNull(summary(), saveTitle(), region(), filename())` NL |
|
||||
| 38383 | P2POrderEvent | nip69P2pOrderEvents | `(listOfNotNull(makerName(), currency()) + paymentMethods().orEmpty()).joinToString(" ")` (SP) |
|
||||
| 39000 | GroupMetadataEvent | nip29RelayGroups/metadata | `listOfNotNull(name(), about())` NL |
|
||||
| 39089 | FollowListEvent | nip51Lists/followList | `listOfNotNull(title(), description())` NL |
|
||||
| 39092 | MediaStarterPackEvent | nip51Lists/mediaStarterPack | `listOfNotNull(title(), description())` NL |
|
||||
| 39701 | WebBookmarkEvent | nipB0WebBookmarks | `listOfNotNull(title(), description())` NL |
|
||||
| 40002 | StreamMessageV2Event | buzz/stream | `content` |
|
||||
| 40100 | CanvasEvent | buzz/stream | `content` |
|
||||
| 45001 | ForumPostEvent | buzz/forum | `content` |
|
||||
| 45003 | ForumCommentEvent | buzz/forum | `content` |
|
||||
| 48106 | HuddleGuidelinesEvent | buzz/huddles | `content` |
|
||||
|
||||
† **Kind 30063 collision:** `experimental/nip82SoftwareApps/release/SoftwareReleaseEvent` also
|
||||
declares `KIND = 30063` and implements `SearchableEvent` (`content`), but `EventFactory` maps
|
||||
30063 to `ReleaseArtifactSetEvent`, so on every store path kind 30063 indexes
|
||||
`title()\ndescription()`. If the factory mapping ever changes, this table changes with it.
|
||||
|
||||
## Abstract bases (no kind of their own)
|
||||
|
||||
| Base class | Body | Concrete kinds |
|
||||
|---|---|---|
|
||||
| `InteractiveStoryBaseEvent` | `listOfNotNull(title(), summary(), content)` NL | 30296, 30297 |
|
||||
| `AddressableVideoEvent` | `listOfNotNull(title(), content)` NL | 34235, 34236 |
|
||||
| `RegularVideoEvent` | `listOfNotNull(title(), content)` NL | 21, 22 |
|
||||
|
||||
## How to regenerate / verify this table
|
||||
|
||||
```bash
|
||||
# All implementor files:
|
||||
grep -rln "override fun indexableContent" quartz/src/commonMain
|
||||
# For each, pair the KIND constant with the indexableContent() body.
|
||||
# Searchability on the store path additionally requires EventFactory registration:
|
||||
grep -n "<ClassName>" quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/utils/EventFactory.kt
|
||||
```
|
||||
|
||||
A CI-diffable snapshot test (assert the set of kinds whose `EventFactory` product implements
|
||||
`SearchableEvent` against a checked-in list) would make this table impossible to go stale —
|
||||
suggested follow-up, not yet implemented.
|
||||
Reference in New Issue
Block a user