From d76c3c3ef38bbacf9158b42dd22fdc25ab9a9bd6 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 10 Sep 2026 13:57:13 +0000 Subject: [PATCH] refactor: one kind window for search, checked against quartz MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit "Which kinds can a search return?" was answered in three places that disagreed. Android's relay subscription listed 34 kinds; desktop's filter factory listed its own copy, which had lost the calendar slots and code snippets; the local cache scan had no kind window at all, so it matched kinds no relay was ever asked for. Same query, three answers. Quartz already knew the protocol half of it — SearchableEvent, 142 reachable kinds, referenced zero times by the client — but only as an interface nothing could enumerate. So it gains SearchableKinds.ALL, and a test that sweeps the whole 16-bit kind space through EventFactory and asserts the recorded list is exactly what came back searchable. A kind added to quartz now lands as a failing test naming the number. Commons gains RenderableKinds: the 46 of those Amethyst has a card for, plus three (pin lists, poll responses, NNS records) that match on content alone and are named as such. Both front ends and the local scan read it. The audit that produced it found fifteen searchable, renderable kinds the search never asked for: pictures, all four video kinds, workouts, git repos, sites, napplets, meeting spaces and rooms, calendar events and software applications. Searching for your own pictures returned nothing. Two findings on the way, from the sweep rather than from reading: - quartz's indexable-content golden test pinned 126 kinds when 142 were searchable, so seventeen — every video kind among them — had their indexed text unpinned. It now reads SearchableKinds.ALL; the regenerated golden adds those seventeen and drops 31890, which is not buildable. - twelve event classes are absent from EventFactory, so their events parse as a plain Event and nothing ever calls indexableContent(). One of them, FeedDefinitionEvent, implements SearchableEvent. Left alone and written down in the plan: a factory bug is not search's to fix. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01DWTxEzzvD3mKkkgE4N7H66 --- .../loggedIn/search/SearchBarViewModel.kt | 14 +- .../2026-09-10-search-state-unification.md | 29 ++- .../relayClient/search/SearchPostsByText.kt | 92 +------- .../commons/search/RenderableKinds.kt | 197 +++++++++++++++++ .../commons/search/RenderableKindsTest.kt | 190 +++++++++++++++++ .../subscriptions/SearchFilterFactory.kt | 84 +------- .../quartz/nip50Search/SearchableKinds.kt | 199 ++++++++++++++++++ .../quartz/nip50Search/SearchableKindsTest.kt | 60 ++++++ .../nip50Search/IndexableContentGoldenTest.kt | 138 +----------- .../resources/indexable-content.golden | 18 +- 10 files changed, 716 insertions(+), 305 deletions(-) create mode 100644 commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/search/RenderableKinds.kt create mode 100644 commons/src/commonTest/kotlin/com/vitorpamplona/amethyst/commons/search/RenderableKindsTest.kt create mode 100644 quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip50Search/SearchableKinds.kt create mode 100644 quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/nip50Search/SearchableKindsTest.kt diff --git a/amethyst/src/main/java/com/vitorpamplona/amethyst/ui/screen/loggedIn/search/SearchBarViewModel.kt b/amethyst/src/main/java/com/vitorpamplona/amethyst/ui/screen/loggedIn/search/SearchBarViewModel.kt index 50f7bdf822..608c9ec07d 100644 --- a/amethyst/src/main/java/com/vitorpamplona/amethyst/ui/screen/loggedIn/search/SearchBarViewModel.kt +++ b/amethyst/src/main/java/com/vitorpamplona/amethyst/ui/screen/loggedIn/search/SearchBarViewModel.kt @@ -34,6 +34,7 @@ import com.vitorpamplona.amethyst.commons.actions.ConcordActions import com.vitorpamplona.amethyst.commons.model.User import com.vitorpamplona.amethyst.commons.relayClient.search.SearchQueryState import com.vitorpamplona.amethyst.commons.search.QueryParser +import com.vitorpamplona.amethyst.commons.search.RenderableKinds import com.vitorpamplona.amethyst.commons.search.SearchFilterBuilder import com.vitorpamplona.amethyst.commons.search.SearchPipeline import com.vitorpamplona.amethyst.commons.search.SearchScope @@ -406,9 +407,16 @@ class SearchBarViewModel( // could actually be one, so an ordinary query never pays for two scans. looksLikeAnEventId(term) -> LocalCache.search.findNotesStartingWith(term, account.hiddenUsers) else -> - // The same filters the REQ carries. Built by the pipeline rather than - // here, so the cache is asked exactly what the relays are asked. - LocalCache.search.findNotesMatching(SearchPipeline.filters(parsed, limit = 200), account.hiddenUsers) + // The same filters the REQ carries, over the same kind window. Built by + // the pipeline rather than here, so the cache is asked exactly what the + // relays are asked — the window was the last thing the two disagreed on, + // the local scan having had none at all and so matching kinds no relay was + // ever asked for. Asked flat rather than in RenderableKinds.GROUPS: the + // groups exist for a relay's per-filter cap, and a cache has none. + LocalCache.search.findNotesMatching( + SearchPipeline.filters(parsed, RenderableKinds.ALL, limit = 200), + account.hiddenUsers, + ) } val withDirect = (listOfNotNull(direct) + raw).distinctBy { it.idHex } val followed = if (follows != null) withDirect.filter { it.author?.pubkeyHex in follows } else withDirect diff --git a/commons/plans/2026-09-10-search-state-unification.md b/commons/plans/2026-09-10-search-state-unification.md index c4b4ea92a9..b29a663de7 100644 --- a/commons/plans/2026-09-10-search-state-unification.md +++ b/commons/plans/2026-09-10-search-state-unification.md @@ -35,8 +35,8 @@ Supporting measurements, taken 2026-09-10: inside result flows. - Four hand-maintained answers to "which kinds are searchable": the relay allowlist (34), the local denylist (13), `KindRegistry.aliases` (33), and - quartz's `SearchableEvent` — **139 implementors, referenced 0 times by the - client**. + quartz's `SearchableEvent` — 142 reachable kinds, **referenced 0 times by the + client**. *(The first two are now one list; see Phase 1 step 3.)* ## 2. The real blocker (it is not laziness) @@ -128,10 +128,24 @@ Do not attempt to unify these; they are real differences, not drift: one object, with the ordering documented and tested. 2. Repoint both `SearchBarViewModel` and `AdvancedSearchBarState` at it. No behaviour change intended — this is the regression-test bed for Phase 2. -3. **One kind-set source.** Derive the relay allowlist from quartz's - `SearchableEvent` rather than the hand-kept `SearchPostsByTextKinds1/2/3`, - and make the local denylist the *same* set expressed once. Fixes the - pictures/video/git/workout gap found on 2026-09-09 and stops it recurring. +3. **One kind-set source.** *(Done — `RenderableKinds` in `commons/search/`, + checked against `SearchableKinds` in quartz.)* The relay allowlist, desktop's + copy of it, and the local scan's absent window are now one list. Quartz gained + `SearchableKinds.ALL`, the 142 kinds `EventFactory` builds a `SearchableEvent` + for, verified by sweeping the whole 16-bit kind space in a test rather than + kept by hand; `RenderableKinds` is the 46 of those Amethyst has a card + for, plus three that match on `content` alone, and a test pins the rest so a new + searchable kind in quartz arrives as a decision. + + The audit that produced it found fifteen searchable, renderable kinds the + search never asked for — pictures, all four video kinds, workouts, git repos, + sites, napplets, meetings, calendar events, software apps. It also found that + quartz's own golden test was pinning 126 kinds when 142 were searchable, and + that twelve event classes (`FeedDefinitionEvent` among them) are never + registered in `EventFactory`, so their events parse as plain `Event` and + nothing ever indexes them. That last one is a quartz bug, filed here rather + than fixed: it is not search's to make. + 4. One debounce policy, stated once. *Size: ~400 lines moved/added, ~200 deleted. No module boundaries crossed.* @@ -180,7 +194,8 @@ refactoring, not after: - **A parity table**: one query × both front ends → same filters, same kept set, same order. This is the test that would have caught all four bugs. - **Scope × result-type**: the current 7 guards, pinned, before they collapse. -- **Kind-set parity**: relay allowlist == local allowlist == `SearchableEvent`. +- **Kind-set parity**: relay allowlist == local allowlist, and every kind in it + is one `SearchableEvent` covers. *(Done: `RenderableKindsTest`.)* ## 9. Open questions for the maintainer diff --git a/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relayClient/search/SearchPostsByText.kt b/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relayClient/search/SearchPostsByText.kt index 1ffb4796b1..1a2df153e4 100644 --- a/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relayClient/search/SearchPostsByText.kt +++ b/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relayClient/search/SearchPostsByText.kt @@ -23,92 +23,10 @@ package com.vitorpamplona.amethyst.commons.relayClient.search import com.vitorpamplona.amethyst.commons.relayClient.subscriptions.ExplainedFilter import com.vitorpamplona.amethyst.commons.relayClient.subscriptions.SubPurpose import com.vitorpamplona.amethyst.commons.search.QueryParser +import com.vitorpamplona.amethyst.commons.search.RenderableKinds import com.vitorpamplona.amethyst.commons.search.SearchPipeline -import com.vitorpamplona.quartz.experimental.audio.header.AudioHeaderEvent -import com.vitorpamplona.quartz.experimental.audio.track.AudioTrackEvent -import com.vitorpamplona.quartz.experimental.interactiveStories.InteractiveStoryPrologueEvent -import com.vitorpamplona.quartz.experimental.interactiveStories.InteractiveStorySceneEvent -import com.vitorpamplona.quartz.experimental.music.playlist.MusicPlaylistEvent -import com.vitorpamplona.quartz.experimental.music.track.MusicTrackEvent -import com.vitorpamplona.quartz.experimental.nipsOnNostr.NipTextEvent -import com.vitorpamplona.quartz.experimental.nns.NNSEvent -import com.vitorpamplona.quartz.experimental.zapPolls.ZapPollEvent import com.vitorpamplona.quartz.nip01Core.relay.client.pool.RelayBasedFilter import com.vitorpamplona.quartz.nip01Core.relay.normalizer.NormalizedRelayUrl -import com.vitorpamplona.quartz.nip10Notes.TextNoteEvent -import com.vitorpamplona.quartz.nip22Comments.CommentEvent -import com.vitorpamplona.quartz.nip23LongContent.LongTextNoteEvent -import com.vitorpamplona.quartz.nip28PublicChat.admin.ChannelCreateEvent -import com.vitorpamplona.quartz.nip28PublicChat.admin.ChannelMetadataEvent -import com.vitorpamplona.quartz.nip30CustomEmoji.pack.EmojiPackEvent -import com.vitorpamplona.quartz.nip51Lists.PinListEvent -import com.vitorpamplona.quartz.nip51Lists.bookmarkList.BookmarkListEvent -import com.vitorpamplona.quartz.nip51Lists.bookmarkList.OldBookmarkListEvent -import com.vitorpamplona.quartz.nip51Lists.followList.FollowListEvent -import com.vitorpamplona.quartz.nip51Lists.peopleList.PeopleListEvent -import com.vitorpamplona.quartz.nip52Calendar.appt.day.CalendarDateSlotEvent -import com.vitorpamplona.quartz.nip52Calendar.appt.time.CalendarTimeSlotEvent -import com.vitorpamplona.quartz.nip53LiveActivities.streaming.LiveActivitiesEvent -import com.vitorpamplona.quartz.nip54Wiki.WikiNoteEvent -import com.vitorpamplona.quartz.nip58Badges.definition.BadgeDefinitionEvent -import com.vitorpamplona.quartz.nip72ModCommunities.definition.CommunityDefinitionEvent -import com.vitorpamplona.quartz.nip84Highlights.HighlightEvent -import com.vitorpamplona.quartz.nip88Polls.poll.PollEvent -import com.vitorpamplona.quartz.nip88Polls.response.PollResponseEvent -import com.vitorpamplona.quartz.nip99Classifieds.ClassifiedsEvent -import com.vitorpamplona.quartz.nipA4PublicMessages.PublicMessageEvent -import com.vitorpamplona.quartz.nipC0CodeSnippets.CodeSnippetEvent -import com.vitorpamplona.quartz.nipF4Podcasts.episode.PodcastEpisodeEvent -import com.vitorpamplona.quartz.nipF4Podcasts.metadata.PodcastMetadataEvent - -val SearchPostsByTextKinds1 = - listOf( - TextNoteEvent.KIND, - LongTextNoteEvent.KIND, - BadgeDefinitionEvent.KIND, - PeopleListEvent.KIND, - BookmarkListEvent.KIND, - OldBookmarkListEvent.KIND, - AudioHeaderEvent.KIND, - AudioTrackEvent.KIND, - MusicTrackEvent.KIND, - MusicPlaylistEvent.KIND, - PodcastEpisodeEvent.KIND, - PodcastMetadataEvent.KIND, - PinListEvent.KIND, - ZapPollEvent.KIND, - ChannelCreateEvent.KIND, - ) - -val SearchPostsByTextKinds2 = - listOf( - ChannelMetadataEvent.KIND, - ClassifiedsEvent.KIND, - CommunityDefinitionEvent.KIND, - EmojiPackEvent.KIND, - HighlightEvent.KIND, - LiveActivitiesEvent.KIND, - PublicMessageEvent.KIND, - NNSEvent.KIND, - WikiNoteEvent.KIND, - CommentEvent.KIND, - ) - -val SearchPostsByTextKinds3 = - listOf( - InteractiveStoryPrologueEvent.KIND, - InteractiveStorySceneEvent.KIND, - FollowListEvent.KIND, - NipTextEvent.KIND, - PollEvent.KIND, - PollResponseEvent.KIND, - CalendarTimeSlotEvent.KIND, - CalendarDateSlotEvent.KIND, - CodeSnippetEvent.KIND, - ) - -private val SearchPostsByTextKindGroups = - listOf(SearchPostsByTextKinds1, SearchPostsByTextKinds2, SearchPostsByTextKinds3) /** * The REQs a search sends, from the text the reader typed. @@ -120,8 +38,10 @@ private val SearchPostsByTextKindGroups = * its own tokens and matched nothing. * * A `kind:` token names the window itself, and is asked as the one group it says. Only a query - * that names no kind falls back to the full set Amethyst can render — which is larger than most - * relays accept in one filter, so it is asked in three groups and the union merged client-side. + * that names no kind falls back to [RenderableKinds] — the kinds Amethyst can put on screen, + * which is more than most relays accept in one filter, so it is asked in groups and the union + * merged client-side. That list used to be spelled out here, and separately in desktop's filter + * factory, and the two had already drifted apart. */ fun searchPostsByText( searchString: String, @@ -133,7 +53,7 @@ fun searchPostsByText( // One group when the query names its own window — SearchPipeline.filters lets a `kind:` win // over the caller's fallback, so passing the groups here would be asking for a narrowing the // pipeline has already decided against. - val kindGroups = if (query.kinds.isNotEmpty()) listOf(null) else SearchPostsByTextKindGroups + val kindGroups = if (query.kinds.isNotEmpty()) listOf(null) else RenderableKinds.GROUPS return kindGroups.flatMap { kinds -> SearchPipeline.filters(query, kinds, limit = 100).map { filter -> diff --git a/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/search/RenderableKinds.kt b/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/search/RenderableKinds.kt new file mode 100644 index 0000000000..ce31b16abf --- /dev/null +++ b/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/search/RenderableKinds.kt @@ -0,0 +1,197 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.amethyst.commons.search + +import com.vitorpamplona.quartz.experimental.audio.header.AudioHeaderEvent +import com.vitorpamplona.quartz.experimental.audio.track.AudioTrackEvent +import com.vitorpamplona.quartz.experimental.fitness.workout.WorkoutRecordEvent +import com.vitorpamplona.quartz.experimental.interactiveStories.InteractiveStoryPrologueEvent +import com.vitorpamplona.quartz.experimental.interactiveStories.InteractiveStorySceneEvent +import com.vitorpamplona.quartz.experimental.music.playlist.MusicPlaylistEvent +import com.vitorpamplona.quartz.experimental.music.track.MusicTrackEvent +import com.vitorpamplona.quartz.experimental.nip82SoftwareApps.application.SoftwareApplicationEvent +import com.vitorpamplona.quartz.experimental.nipsOnNostr.NipTextEvent +import com.vitorpamplona.quartz.experimental.nns.NNSEvent +import com.vitorpamplona.quartz.experimental.zapPolls.ZapPollEvent +import com.vitorpamplona.quartz.nip10Notes.TextNoteEvent +import com.vitorpamplona.quartz.nip22Comments.CommentEvent +import com.vitorpamplona.quartz.nip23LongContent.LongTextNoteEvent +import com.vitorpamplona.quartz.nip28PublicChat.admin.ChannelCreateEvent +import com.vitorpamplona.quartz.nip28PublicChat.admin.ChannelMetadataEvent +import com.vitorpamplona.quartz.nip30CustomEmoji.pack.EmojiPackEvent +import com.vitorpamplona.quartz.nip34Git.repository.GitRepositoryEvent +import com.vitorpamplona.quartz.nip51Lists.PinListEvent +import com.vitorpamplona.quartz.nip51Lists.bookmarkList.BookmarkListEvent +import com.vitorpamplona.quartz.nip51Lists.bookmarkList.OldBookmarkListEvent +import com.vitorpamplona.quartz.nip51Lists.followList.FollowListEvent +import com.vitorpamplona.quartz.nip51Lists.peopleList.PeopleListEvent +import com.vitorpamplona.quartz.nip52Calendar.appt.day.CalendarDateSlotEvent +import com.vitorpamplona.quartz.nip52Calendar.appt.time.CalendarTimeSlotEvent +import com.vitorpamplona.quartz.nip52Calendar.calendar.CalendarEvent +import com.vitorpamplona.quartz.nip53LiveActivities.meetingSpaces.MeetingRoomEvent +import com.vitorpamplona.quartz.nip53LiveActivities.meetingSpaces.MeetingSpaceEvent +import com.vitorpamplona.quartz.nip53LiveActivities.streaming.LiveActivitiesEvent +import com.vitorpamplona.quartz.nip54Wiki.WikiNoteEvent +import com.vitorpamplona.quartz.nip58Badges.definition.BadgeDefinitionEvent +import com.vitorpamplona.quartz.nip5aStaticWebsites.NamedSiteEvent +import com.vitorpamplona.quartz.nip5aStaticWebsites.RootSiteEvent +import com.vitorpamplona.quartz.nip5dNapplets.NamedNappletEvent +import com.vitorpamplona.quartz.nip5dNapplets.RootNappletEvent +import com.vitorpamplona.quartz.nip68Picture.PictureEvent +import com.vitorpamplona.quartz.nip71Video.VideoHorizontalEvent +import com.vitorpamplona.quartz.nip71Video.VideoNormalEvent +import com.vitorpamplona.quartz.nip71Video.VideoShortEvent +import com.vitorpamplona.quartz.nip71Video.VideoVerticalEvent +import com.vitorpamplona.quartz.nip72ModCommunities.definition.CommunityDefinitionEvent +import com.vitorpamplona.quartz.nip84Highlights.HighlightEvent +import com.vitorpamplona.quartz.nip88Polls.poll.PollEvent +import com.vitorpamplona.quartz.nip88Polls.response.PollResponseEvent +import com.vitorpamplona.quartz.nip99Classifieds.ClassifiedsEvent +import com.vitorpamplona.quartz.nipA4PublicMessages.PublicMessageEvent +import com.vitorpamplona.quartz.nipC0CodeSnippets.CodeSnippetEvent +import com.vitorpamplona.quartz.nipF4Podcasts.episode.PodcastEpisodeEvent +import com.vitorpamplona.quartz.nipF4Podcasts.metadata.PodcastMetadataEvent + +/** + * The kinds a search can put on screen, in one place. + * + * A search that names no kind still cannot ask for "everything": a relay would answer with events + * Amethyst has no card for, and the reader would scroll past blanks. So the window is listed, and + * — because this list decides what a query reaches — it was previously listed **three times**: in + * the Android relay subscription, again in desktop's filter factory, and inverted as a denylist in + * the local cache scan. They drifted, as three copies do. Desktop's third group was missing the + * calendar slots and code snippets; the local scan had no kind window at all, so local and relay + * results for the same query came from different sets. + * + * ## What is in it + * + * Every kind here is one Amethyst renders *and* — with three named exceptions — one Quartz + * declares searchable, which `RenderableKindsTest` checks against + * [com.vitorpamplona.quartz.nip50Search.SearchableKinds]. That check is the point of the file: + * asking a relay for a kind whose text nothing indexes is a filter slot spent on a result that + * cannot come back, and *not* asking for a kind that is both searchable and renderable is a hole + * the reader experiences as "the search does not find my pictures". Both happened. The audit that + * produced this list found fifteen searchable, renderable kinds absent — pictures, every video + * kind, workouts, git repos, sites, napplets, meetings and calendar events among them. + */ +object RenderableKinds { + /** + * Kinds asked for despite Quartz not indexing their text. + * + * They are here because [ALL] answers "what can a result be", not "what can a text query + * match", and a chips-only query (`from:npub1… since:2024-01-01`) carries no text for anything + * to be indexed against. A relay that indexes raw `content` may still match them, and + * `EventSearchMatcher` does locally, so they are not dead weight — just not guaranteed. + */ + val MATCHES_ON_CONTENT_ONLY = + setOf( + PinListEvent.KIND, + PollResponseEvent.KIND, + NNSEvent.KIND, + ) + + /** + * Most relays reject a filter naming more kinds than this, which is the only reason [GROUPS] + * exists. It is a limit on the wire, not on what can be searched. + */ + const val MAX_KINDS_PER_FILTER = 16 + + /** Every kind, for a caller with no per-filter limit — the local cache scan. */ + val ALL = + listOf( + // notes and long-form + TextNoteEvent.KIND, + CommentEvent.KIND, + PublicMessageEvent.KIND, + LongTextNoteEvent.KIND, + WikiNoteEvent.KIND, + HighlightEvent.KIND, + CodeSnippetEvent.KIND, + NipTextEvent.KIND, + // pictures, video, audio + PictureEvent.KIND, + VideoNormalEvent.KIND, + VideoShortEvent.KIND, + VideoHorizontalEvent.KIND, + VideoVerticalEvent.KIND, + AudioHeaderEvent.KIND, + AudioTrackEvent.KIND, + MusicTrackEvent.KIND, + MusicPlaylistEvent.KIND, + PodcastEpisodeEvent.KIND, + PodcastMetadataEvent.KIND, + // interactive stories + InteractiveStoryPrologueEvent.KIND, + InteractiveStorySceneEvent.KIND, + // polls + PollEvent.KIND, + PollResponseEvent.KIND, + ZapPollEvent.KIND, + // channels, streams and rooms + ChannelCreateEvent.KIND, + ChannelMetadataEvent.KIND, + LiveActivitiesEvent.KIND, + MeetingSpaceEvent.KIND, + MeetingRoomEvent.KIND, + // communities and lists + CommunityDefinitionEvent.KIND, + PeopleListEvent.KIND, + FollowListEvent.KIND, + BookmarkListEvent.KIND, + OldBookmarkListEvent.KIND, + PinListEvent.KIND, + // calendar + CalendarDateSlotEvent.KIND, + CalendarTimeSlotEvent.KIND, + CalendarEvent.KIND, + // marketplace, apps, code and sites + ClassifiedsEvent.KIND, + SoftwareApplicationEvent.KIND, + GitRepositoryEvent.KIND, + RootSiteEvent.KIND, + NamedSiteEvent.KIND, + RootNappletEvent.KIND, + NamedNappletEvent.KIND, + // everything else with a card + BadgeDefinitionEvent.KIND, + EmojiPackEvent.KIND, + NNSEvent.KIND, + WorkoutRecordEvent.KIND, + ) + + /** + * [ALL] split into filters a relay will accept, balanced rather than greedy so growth adds a + * kind to each group instead of leaving a group of one behind. + */ + val GROUPS = ALL.balancedChunks(MAX_KINDS_PER_FILTER) + + private fun List.balancedChunks(max: Int): List> { + val groups = (size + max - 1) / max + if (groups <= 1) return listOf(this) + val base = size / groups + val remainder = size % groups + var from = 0 + return (0 until groups).map { index -> + val take = base + if (index < remainder) 1 else 0 + subList(from, from + take).toList().also { from += take } + } + } +} diff --git a/commons/src/commonTest/kotlin/com/vitorpamplona/amethyst/commons/search/RenderableKindsTest.kt b/commons/src/commonTest/kotlin/com/vitorpamplona/amethyst/commons/search/RenderableKindsTest.kt new file mode 100644 index 0000000000..10b3bd1489 --- /dev/null +++ b/commons/src/commonTest/kotlin/com/vitorpamplona/amethyst/commons/search/RenderableKindsTest.kt @@ -0,0 +1,190 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.amethyst.commons.search + +import com.vitorpamplona.quartz.nip50Search.SearchableKinds +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertTrue + +/** + * The kind window, against the two things that can make it wrong. + * + * It can ask for a kind whose text nothing indexes — a filter slot spent on a result that cannot + * come back — or it can leave out a kind that is both searchable and renderable, which the reader + * meets as "search does not find my pictures". Both had happened by the time this was written, in + * three copies of the list that disagreed with each other. + */ +class RenderableKindsTest { + @Test + fun everyKindAskedForIsOneQuartzIndexes() { + val notIndexed = RenderableKinds.ALL.filterNot { it in SearchableKinds.ALL } + assertEquals( + RenderableKinds.MATCHES_ON_CONTENT_ONLY, + notIndexed.toSet(), + "a kind is being asked for that Quartz does not index. Either it gained an " + + "indexableContent() and belongs in neither exception list, or it is a new " + + "exception that needs saying out loud in MATCHES_ON_CONTENT_ONLY.", + ) + } + + @Test + fun theKindsDeliberatelyLeftOutAreStillTheOnesLeftOut() { + // Pinned so that a kind Quartz *starts* indexing arrives here as a decision — show it in + // search or say why not — rather than as silence. Adding one to RenderableKinds.ALL is + // the fix when Amethyst has a card for it; extending this list is the fix when it does not. + val omitted = SearchableKinds.ALL.filterNot { it in RenderableKinds.ALL.toSet() } + assertEquals( + listOf( + // profiles and statuses — the People scope finds these, the Notes scope should not + 0, + 30315, + 30382, + // chat and DMs: found inside a channel or a conversation, not in a global feed + 9, + 14, + 42, + 1311, + 3302, + 9002, + 30177, + 39000, + 40002, + // raids and clips: side traffic belonging to one stream, not a global feed + 1312, + 1313, + // threads and forums Amethyst has no card for + 11, + 45001, + 45003, + 48106, + // citations + 31, + 32, + 33, + // edits, file headers and gallery entries: attachments to a note, not results + 1010, + 1063, + 1065, + 1163, + 32176, + 39092, + // git: the repo itself is searchable, its patch/issue/status traffic is not + 818, + 1617, + 1618, + 1621, + 1622, + 1630, + 1631, + 1632, + 1633, + // labels, ratings and trust assertions: metadata about other events + 1985, + 30392, + 30393, + 30394, + 30395, + 31871, + 31872, + 31873, + 11871, + 34259, + 31987, + // zaps and money + 8333, + 9321, + 9734, + 9735, + 9736, + 9737, + 9041, + 33863, + 38383, + 38000, + // DVM job requests + 5050, + 5100, + 5250, + 5302, + 5303, + // napplet snapshots: the app is searchable, a snapshot of it is not + 5129, + // torrents, roads, birds, canvases and other feeds Amethyst does not render + 1315, + 2003, + 2004, + 2473, + 12473, + 30142, + 30620, + 38192, + 40100, + // curation sets and relay/interest lists + 30002, + 30003, + 30004, + 30005, + 30006, + 30015, + 30267, + 30040, + 30041, + 30045, + 30063, + 39701, + // marketplace stalls and products: reached through their merchant, not searched + 30017, + 30018, + 30019, + 30020, + // podcasting 2.0 mirrors of kinds already in the window + 30054, + 30055, + // agents and personas + 10100, + 30175, + 30176, + // calendar RSVPs, exercise templates and app definitions + 31925, + 33401, + 31990, + ).sorted(), + omitted.sorted(), + "the set of searchable kinds not shown in search changed", + ) + } + + @Test + fun theGroupsAreTheWholeListAndNothingElse() { + assertEquals(RenderableKinds.ALL, RenderableKinds.GROUPS.flatten()) + assertEquals(RenderableKinds.ALL.distinct(), RenderableKinds.ALL) + } + + @Test + fun noGroupIsLargerThanARelayWillAcceptOrSmallEnoughToWaste() { + // Balanced, not greedy: a greedy split leaves a tail group of one, which costs a whole + // extra REQ to carry a single kind. + val sizes = RenderableKinds.GROUPS.map { it.size } + assertTrue(sizes.all { it <= RenderableKinds.MAX_KINDS_PER_FILTER }, "$sizes") + assertTrue(sizes.max() - sizes.min() <= 1, "groups are lopsided: $sizes") + } +} diff --git a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/subscriptions/SearchFilterFactory.kt b/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/subscriptions/SearchFilterFactory.kt index 4665601fea..50389efecd 100644 --- a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/subscriptions/SearchFilterFactory.kt +++ b/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/subscriptions/SearchFilterFactory.kt @@ -20,93 +20,21 @@ */ package com.vitorpamplona.amethyst.desktop.subscriptions +import com.vitorpamplona.amethyst.commons.search.RenderableKinds import com.vitorpamplona.amethyst.commons.search.SearchPipeline import com.vitorpamplona.amethyst.commons.search.SearchQuery -import com.vitorpamplona.quartz.experimental.audio.header.AudioHeaderEvent -import com.vitorpamplona.quartz.experimental.audio.track.AudioTrackEvent -import com.vitorpamplona.quartz.experimental.interactiveStories.InteractiveStoryPrologueEvent -import com.vitorpamplona.quartz.experimental.interactiveStories.InteractiveStorySceneEvent -import com.vitorpamplona.quartz.experimental.music.playlist.MusicPlaylistEvent -import com.vitorpamplona.quartz.experimental.music.track.MusicTrackEvent -import com.vitorpamplona.quartz.experimental.nipsOnNostr.NipTextEvent -import com.vitorpamplona.quartz.experimental.nns.NNSEvent -import com.vitorpamplona.quartz.experimental.zapPolls.ZapPollEvent import com.vitorpamplona.quartz.nip01Core.relay.filters.Filter -import com.vitorpamplona.quartz.nip10Notes.TextNoteEvent -import com.vitorpamplona.quartz.nip22Comments.CommentEvent -import com.vitorpamplona.quartz.nip23LongContent.LongTextNoteEvent -import com.vitorpamplona.quartz.nip28PublicChat.admin.ChannelCreateEvent -import com.vitorpamplona.quartz.nip28PublicChat.admin.ChannelMetadataEvent -import com.vitorpamplona.quartz.nip30CustomEmoji.pack.EmojiPackEvent -import com.vitorpamplona.quartz.nip51Lists.PinListEvent -import com.vitorpamplona.quartz.nip51Lists.bookmarkList.BookmarkListEvent -import com.vitorpamplona.quartz.nip51Lists.bookmarkList.OldBookmarkListEvent -import com.vitorpamplona.quartz.nip51Lists.followList.FollowListEvent -import com.vitorpamplona.quartz.nip51Lists.peopleList.PeopleListEvent -import com.vitorpamplona.quartz.nip53LiveActivities.streaming.LiveActivitiesEvent -import com.vitorpamplona.quartz.nip54Wiki.WikiNoteEvent -import com.vitorpamplona.quartz.nip58Badges.definition.BadgeDefinitionEvent -import com.vitorpamplona.quartz.nip72ModCommunities.definition.CommunityDefinitionEvent -import com.vitorpamplona.quartz.nip84Highlights.HighlightEvent -import com.vitorpamplona.quartz.nip88Polls.poll.PollEvent -import com.vitorpamplona.quartz.nip88Polls.response.PollResponseEvent -import com.vitorpamplona.quartz.nip99Classifieds.ClassifiedsEvent -import com.vitorpamplona.quartz.nipA4PublicMessages.PublicMessageEvent object SearchFilterFactory { - // Default kind groups (ported from Android SearchPostsByText) - private val defaultKindGroup1 = - listOf( - TextNoteEvent.KIND, - LongTextNoteEvent.KIND, - BadgeDefinitionEvent.KIND, - PeopleListEvent.KIND, - BookmarkListEvent.KIND, - OldBookmarkListEvent.KIND, - AudioHeaderEvent.KIND, - AudioTrackEvent.KIND, - MusicTrackEvent.KIND, - MusicPlaylistEvent.KIND, - PinListEvent.KIND, - ZapPollEvent.KIND, - ChannelCreateEvent.KIND, - ) - - private val defaultKindGroup2 = - listOf( - ChannelMetadataEvent.KIND, - ClassifiedsEvent.KIND, - CommunityDefinitionEvent.KIND, - EmojiPackEvent.KIND, - HighlightEvent.KIND, - LiveActivitiesEvent.KIND, - PublicMessageEvent.KIND, - NNSEvent.KIND, - WikiNoteEvent.KIND, - CommentEvent.KIND, - ) - - private val defaultKindGroup3 = - listOf( - InteractiveStoryPrologueEvent.KIND, - InteractiveStorySceneEvent.KIND, - FollowListEvent.KIND, - NipTextEvent.KIND, - PollEvent.KIND, - PollResponseEvent.KIND, - ) - - private val DEFAULT_KIND_GROUPS = listOf(defaultKindGroup1, defaultKindGroup2, defaultKindGroup3) - /** * The filters a query sends, per the shared search language — see * [com.vitorpamplona.amethyst.commons.search.SearchFilterBuilder], which owns how each token * becomes a NIP-01 filter field and how a hashtag or a scope fans out into a union. * - * The only thing this layer adds is the kind window. A query that names no kind still cannot - * ask a relay for "everything": the kinds Amethyst can render are listed explicitly, and - * there are more of them than most relays accept in one filter, so they are asked in three - * groups and the union is merged client-side. + * The only thing this layer adds is the kind window, and it no longer keeps its own copy of + * one: the list lived here and in Android's `searchPostsByText`, and the copy here had lost + * the calendar slots and code snippets, so the same query returned different kinds on the two + * platforms. Both now read [RenderableKinds]. */ fun createFilters( query: SearchQuery, @@ -116,6 +44,6 @@ object SearchFilterFactory { // SearchPipeline.filters lets the query's own `kind:` win over the window passed here, so // a named kind collapses the fan-out to one group on its own. if (query.kinds.isNotEmpty()) return SearchPipeline.filters(query, limit = limit) - return DEFAULT_KIND_GROUPS.flatMap { SearchPipeline.filters(query, it, limit) } + return RenderableKinds.GROUPS.flatMap { SearchPipeline.filters(query, it, limit) } } } diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip50Search/SearchableKinds.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip50Search/SearchableKinds.kt new file mode 100644 index 0000000000..e1862c2295 --- /dev/null +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip50Search/SearchableKinds.kt @@ -0,0 +1,199 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.quartz.nip50Search + +/** + * Every kind [com.vitorpamplona.quartz.utils.EventFactory] builds a [SearchableEvent] for. + * + * The list exists because the question "does this kind carry text worth matching?" was being + * answered by hand in four places at once — a relay allowlist, a local denylist, a search-token + * alias table, and this interface — and the four disagreed. [SearchableEvent] is the only one of + * them that is a fact about the protocol rather than a guess about it, so it is the one the + * others should be derived from, and this is that fact in a form the rest of the codebase can + * read. + * + * ## Why it is written out rather than computed + * + * There is no reflection in common code and no registry to walk: [com.vitorpamplona.quartz.utils.EventFactory] + * is a `when` over ~400 kinds. Computing this would mean building one event per kind across the + * whole 16-bit space on every startup. So the answer is recorded here and `SearchableKindsTest` + * does the sweep instead — it builds every kind from 0 to 65535 and asserts this list is exactly + * the set that came back searchable. A kind added to Quartz therefore lands as a failing test + * naming the number, not as a silent gap. + * + * ## "EventFactory builds", not "the class implements" + * + * These are not the same set, and the difference is a real defect rather than a subtlety: a class + * can implement [SearchableEvent] and never be registered in the factory, in which case its events + * parse as a plain [com.vitorpamplona.quartz.nip01Core.core.Event] and nothing ever calls its + * `indexableContent()`. `FeedDefinitionEvent` (31890) is in exactly that state, along with eleven + * other unregistered event classes. Since an unregistered kind cannot be searched no matter what + * it declares, the reachable set is the one worth listing — but the unreachable ones are a bug in + * the factory, not a decision. + */ +object SearchableKinds { + /** Sorted ascending. */ + val ALL = + listOf( + 0, // MetadataEvent + 1, // TextNoteEvent + 9, // ChatEvent + 11, // ThreadEvent + 14, // ChatMessageEvent + 20, // PictureEvent + 21, // VideoNormalEvent + 22, // VideoShortEvent + 24, // PublicMessageEvent + 31, // ExternalCitationEvent + 32, // HardcopyCitationEvent + 33, // PromptCitationEvent + 40, // ChannelCreateEvent + 41, // ChannelMetadataEvent + 42, // ChannelMessageEvent + 54, // PodcastEpisodeEvent + 818, // WikiMergeRequestEvent + 1010, // TextNoteModificationEvent + 1063, // FileHeaderEvent + 1065, // FileStorageHeaderEvent + 1068, // PollEvent + 1111, // CommentEvent + 1163, // ProfileGalleryEntryEvent + 1301, // WorkoutRecordEvent + 1311, // LiveActivitiesChatMessageEvent + 1312, // LiveActivitiesRaidEvent + 1313, // LiveActivitiesClipEvent + 1315, // RoadEventReportEvent + 1337, // CodeSnippetEvent + 1617, // GitPatchEvent + 1618, // GitPullRequestEvent + 1621, // GitIssueEvent + 1622, // GitReplyEvent + 1630, // GitStatusOpenEvent + 1631, // GitStatusAppliedEvent + 1632, // GitStatusClosedEvent + 1633, // GitStatusDraftEvent + 1808, // AudioHeaderEvent + 1985, // LabelEvent + 2003, // TorrentEvent + 2004, // TorrentCommentEvent + 2473, // BirdDetectionEvent + 3302, // ConcordChatEditEvent + 5050, // NIP90TextGenerationRequestEvent + 5100, // NIP90ImageGenerationRequestEvent + 5129, // NappletSnapshotEvent + 5250, // NIP90TextToSpeechRequestEvent + 5302, // NIP90ContentSearchRequestEvent + 5303, // NIP90PeopleSearchRequestEvent + 6969, // ZapPollEvent + 8333, // OnchainZapEvent + 9002, // EditMetadataEvent + 9041, // GoalEvent + 9321, // NutzapEvent + 9734, // LnZapRequestEvent + 9735, // LnZapEvent + 9736, // Bolt12ZapEvent + 9737, // Bolt12ZapIntentEvent + 9802, // HighlightEvent + 10003, // BookmarkListEvent + 10100, // AgentProfileEvent + 10154, // PodcastMetadataEvent + 11871, // AttestorProficiencyEvent + 12473, // BirdexEvent + 15128, // RootSiteEvent + 15129, // RootNappletEvent + 30000, // PeopleListEvent + 30001, // OldBookmarkListEvent + 30002, // RelaySetEvent + 30003, // LabeledBookmarkListEvent + 30004, // ArticleCurationSetEvent + 30005, // VideoCurationSetEvent + 30006, // PictureCurationSetEvent + 30009, // BadgeDefinitionEvent + 30015, // InterestSetEvent + 30017, // StallEvent + 30018, // ProductEvent + 30019, // MarketplaceEvent + 30020, // AuctionEvent + 30023, // LongTextNoteEvent + 30030, // EmojiPackEvent + 30040, // PublicationIndexEvent + 30041, // PublicationContentEvent + 30045, // BookshelfDirectoryEvent + 30054, // Podcasting20EpisodeEvent + 30055, // Podcasting20TrailerEvent + 30063, // ReleaseArtifactSetEvent + 30142, // LearningResourceEvent + 30175, // PersonaEvent + 30176, // TeamEvent + 30177, // ManagedAgentEvent + 30267, // AppCurationSetEvent + 30296, // InteractiveStoryPrologueEvent + 30297, // InteractiveStorySceneEvent + 30311, // LiveActivitiesEvent + 30312, // MeetingSpaceEvent + 30313, // MeetingRoomEvent + 30315, // StatusEvent + 30382, // ContactCardEvent + 30392, // UserTrustedListEvent + 30393, // EventTrustedListEvent + 30394, // AddressableTrustedListEvent + 30395, // ExternalIdTrustedListEvent + 30402, // ClassifiedsEvent + 30617, // GitRepositoryEvent + 30620, // WorkflowDefEvent + 30817, // NipTextEvent + 30818, // WikiNoteEvent + 31337, // AudioTrackEvent + 31871, // AttestationEvent + 31872, // AttestationRequestEvent + 31873, // AttestorRecommendationEvent + 31922, // CalendarDateSlotEvent + 31923, // CalendarTimeSlotEvent + 31924, // CalendarEvent + 31925, // CalendarRSVPEvent + 31987, // RelayReviewEvent + 31990, // AppDefinitionEvent + 32176, // BlossomPieceIndexEvent + 32267, // SoftwareApplicationEvent + 33401, // ExerciseTemplateEvent + 33863, // FundraiserEvent + 34139, // MusicPlaylistEvent + 34235, // VideoHorizontalEvent + 34236, // VideoVerticalEvent + 34259, // EntityRatingEvent + 34550, // CommunityDefinitionEvent + 35128, // NamedSiteEvent + 35129, // NamedNappletEvent + 36787, // MusicTrackEvent + 38000, // MintRecommendationEvent + 38192, // Ps1SaveEvent + 38383, // P2POrderEvent + 39000, // GroupMetadataEvent + 39089, // FollowListEvent + 39092, // MediaStarterPackEvent + 39701, // WebBookmarkEvent + 40002, // StreamMessageV2Event + 40100, // CanvasEvent + 45001, // ForumPostEvent + 45003, // ForumCommentEvent + 48106, // HuddleGuidelinesEvent + ) +} diff --git a/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/nip50Search/SearchableKindsTest.kt b/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/nip50Search/SearchableKindsTest.kt new file mode 100644 index 0000000000..c23e969128 --- /dev/null +++ b/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/nip50Search/SearchableKindsTest.kt @@ -0,0 +1,60 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.quartz.nip50Search + +import com.vitorpamplona.quartz.nip01Core.core.Event +import com.vitorpamplona.quartz.utils.EventFactory +import kotlin.test.Test +import kotlin.test.assertEquals + +/** + * [SearchableKinds.ALL] against the only authority there is: the factory. + * + * The list is written out because nothing in common code can walk a `when`, and a list written + * out is a list that goes stale. So it is checked the expensive way — every kind in the 16-bit + * space is built and asked whether it came back searchable — which costs one sweep in the test + * suite and buys the guarantee that the recorded set is never a guess. + * + * A new [SearchableEvent] in Quartz therefore fails here, naming its kind, rather than quietly + * being left out of everything downstream that reads this list. + */ +class SearchableKindsTest { + private fun build(kind: Int): Event = EventFactory.create("9".repeat(64), "a".repeat(64), 1L, kind, emptyArray(), "", "") + + @Test + fun theRecordedSetIsExactlyWhatTheFactoryBuilds() { + val actual = (0..65535).filter { build(it) is SearchableEvent } + assertEquals( + actual, + SearchableKinds.ALL, + "SearchableKinds.ALL is stale. Missing: ${actual - SearchableKinds.ALL.toSet()}; " + + "no longer searchable: ${SearchableKinds.ALL - actual.toSet()}.", + ) + } + + @Test + fun theListIsSortedAndHasNoDuplicates() { + // Read as a range in review and bisected by eye when a number is looked up; both need it + // ordered, and neither notices a repeat. + assertEquals(SearchableKinds.ALL.sorted(), SearchableKinds.ALL) + assertEquals(SearchableKinds.ALL.distinct(), SearchableKinds.ALL) + } +} diff --git a/quartz/src/jvmTest/kotlin/com/vitorpamplona/quartz/nip50Search/IndexableContentGoldenTest.kt b/quartz/src/jvmTest/kotlin/com/vitorpamplona/quartz/nip50Search/IndexableContentGoldenTest.kt index 0fc7a7697f..5a91ea36fc 100644 --- a/quartz/src/jvmTest/kotlin/com/vitorpamplona/quartz/nip50Search/IndexableContentGoldenTest.kt +++ b/quartz/src/jvmTest/kotlin/com/vitorpamplona/quartz/nip50Search/IndexableContentGoldenTest.kt @@ -154,135 +154,13 @@ class IndexableContentGoldenTest { } companion object { - /** Every kind whose event class implements [SearchableEvent], extracted from the sources. */ - val KINDS = - listOf( - 0, - 1, - 9, - 11, - 14, - 20, - 24, - 40, - 41, - 42, - 54, - 818, - 1010, - 1063, - 1065, - 1068, - 1111, - 1163, - 1301, - 1311, - 1312, - 1313, - 1315, - 1337, - 1617, - 1618, - 1621, - 1622, - 1808, - 1985, - 2003, - 2004, - 2473, - 3302, - 5050, - 5100, - 5129, - 5250, - 5302, - 5303, - 6969, - 8333, - 9002, - 9041, - 9321, - 9734, - 9735, - 9736, - 9737, - 9802, - 10003, - 10100, - 10154, - 11871, - 12473, - 15128, - 15129, - 30000, - 30001, - 30002, - 30003, - 30004, - 30005, - 30006, - 30009, - 30015, - 30017, - 30018, - 30019, - 30020, - 30023, - 30030, - 30040, - 30041, - 30045, - 30054, - 30055, - 30063, - 30142, - 30175, - 30176, - 30177, - 30267, - 30311, - 30312, - 30313, - 30315, - 30382, - 30402, - 30617, - 30620, - 30817, - 30818, - 31337, - 31871, - 31872, - 31873, - 31890, - 31922, - 31923, - 31924, - 31925, - 31987, - 31990, - 32176, - 32267, - 33401, - 33863, - 34139, - 34259, - 34550, - 35128, - 35129, - 36787, - 38000, - 38192, - 38383, - 39000, - 39089, - 39092, - 39701, - 40002, - 40100, - 45001, - 45003, - 48106, - ) + /** + * Every searchable kind, from the machine-checked list rather than a second hand-kept one. + * + * This was a list of its own until it drifted: seventeen searchable kinds — every video + * kind among them — were absent and so were never pinned by any of the tests below, and + * one entry (31890) was a kind the factory does not build at all. + */ + val KINDS = SearchableKinds.ALL } } diff --git a/quartz/src/jvmTest/resources/indexable-content.golden b/quartz/src/jvmTest/resources/indexable-content.golden index 161a2acdf6..379c24a8c3 100644 --- a/quartz/src/jvmTest/resources/indexable-content.golden +++ b/quartz/src/jvmTest/resources/indexable-content.golden @@ -4,7 +4,12 @@ 11 The Title\nThe content body. 14 The content body. 20 The Title\nThe content body. +21 The Title\nThe content body. +22 The Title\nThe content body. 24 The content body. +31 The Title\nThe Summary\nThe content body. +32 The Title\nThe Summary\nThe content body. +33 The Title\nThe Summary\nThe content body. 40 41 42 The content body. @@ -26,6 +31,10 @@ 1618 The Subject\nThe content body. 1621 The Subject\nThe content body. 1622 The content body. +1630 The content body. +1631 The content body. +1632 The content body. +1633 The content body. 1808 The content body. 1985 The content body. 2003 The Title\nThe content body. @@ -81,11 +90,17 @@ 30176 30177 30267 The Title\nThe Description +30296 The Title\nThe Summary\nThe content body. +30297 The Title\nThe Summary\nThe content body. 30311 The Title\nThe Summary\nThe content body. 30312 The Title\nThe Summary\nThe content body. 30313 The Title\nThe Summary 30315 The content body. 30382 The Summary\nhashtag1\nhashtag2 +30392 The Title +30393 The Title +30394 The Title +30395 The Title 30402 The Title\nThe Summary\nThe content body. 30617 The Name\nThe Description\nThe content body. 30620 The Name\nThe content body. @@ -95,7 +110,6 @@ 31871 The content body. 31872 The content body. 31873 The content body. -31890 31922 The Title\nThe Summary\nThe content body. 31923 The Title\nThe Summary\nThe content body. 31924 The Title\nThe content body. @@ -107,6 +121,8 @@ 33401 The Title\nThe content body. 33863 The Title\nThe content body. 34139 The Title\nThe Description\nThe content body. +34235 The Title\nThe content body. +34236 The Title\nThe content body. 34259 The content body. 34550 The Name\nThe Description\nThe Rules\nThe content body. 35128 The Title\nThe Description