Commit Graph
17528 Commits
Author SHA1 Message Date
Claude 853229a2c6 test(notifications): add Layer-3 end-to-end harness (amy + adb)
An agent-runnable script that drives the whole push-notification pipeline on a
real device/emulator: a second identity publishes each notification kind through
`amy` to the account on the phone, and the script reads back the notification
shade over `adb` to assert the right notification on the right channel — then
exercises cold-push metadata enrichment (title flips from raw pubkey to display
name in place) and dismiss-on-read (opening the note clears the tray entry).

Covers mention, reply, reaction, repost, picture, and DM; prints PASS/WARN/FAIL
and exits non-zero on any hard failure. tools/notification-e2e/{run.sh,README.md}.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0122uQ8BLHLeHDni81RBP26r
2026-07-23 14:26:41 +00:00
Claude f9af111daf fix(notifications): gate chess notifications behind the debug feature flag
NIP-64 chess is debug-only for now (see the isDebug gate on the drawer entry),
so its notifications and settings channel shouldn't appear in release.

- ChessNotification.notify early-returns when !isDebug.
- NotificationChannels omits the Chess channel from the settings list (and thus
  never creates it) in release builds.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0122uQ8BLHLeHDni81RBP26r
2026-07-23 13:29:29 +00:00
Claude 7a01fe9fe5 fix(notifications): audit fixes — wakelock coverage, no resurrection, bounded enrichment
Deep-audit follow-up on the per-kind notification rework. Fixes two correctness
bugs and cuts the enrichment path's CPU/battery/IPC cost.

Correctness:
- Cold-push loss: the immediate notification post ran on a detached
  applicationIOScope coroutine OUTSIDE any wakelock (the dispatcher's wakelock
  was already released), so in Doze the CPU could sleep before it reached the
  tray. The enrichment wakelock now wraps the initial render too.
- Resurrection: enrichment re-posts a notification as metadata arrives; if the
  user read/dismissed the post in-app mid-window it came back seconds later.
  dismissNotificationForEvent now records the event id, postStandard/
  postConversation skip a dismissed id, and the enrichment window stops early
  once dismissed.

Performance / battery:
- Channel creation is cached per-process again (was 2 Binder IPCs on every post
  AND every re-render; now once, matching the original field-caching).
- Re-renders are debounced (250ms), collapsing the StateFlow initial-value burst
  and rapid relay arrivals into one rebuild instead of ~3+ back-to-back — cutting
  redundant bitmap loads/crops, notify() calls, and group-summary re-posts.
- Concurrent enrichment windows are capped (Semaphore of 4) so a push burst can't
  hold N simultaneous 25s relay windows + wakelocks + subscriptions.

Behavioral audit found no regressions: routing is a strict superset of the
original and all gates/deep-links are preserved.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0122uQ8BLHLeHDni81RBP26r
2026-07-23 11:39:25 +00:00
Claude aba21c839f feat(notifications): per-kind, observable, beautifully-rendered push notifications
Rebuilds the Android tray-notification layer around a per-kind design system so
every Nostr notification reads at a glance and looks native to modern Android.

Framework:
- NotificationCategory: one table-driven entry per kind carrying its channel,
  accent color, monochrome status-bar icon, importance, group key, and settings
  icon. Reuses existing channel ids (no orphaned user settings) and adds
  Reposts/Media/Articles/Code/Badges channels, organized into system-settings
  channel groups (Messages/Social/Payments/Content/Developer/Games) so users can
  silence a single kind or a whole family.
- NotificationUtils: rewritten as a category-driven builder — postStandard
  (BigText, or BigPicture for media/badges) and postConversation (MessagingStyle
  for DMs/replies/chat/group). Adds setColor accents, circular avatars, colorized
  zaps, kept the NIP-30 emoji badge overlay, and setOnlyAlertOnce so enrichment
  updates replace silently.
- NotificationEnricher: makes notifications observable — renders immediately from
  cache, then subscribes to the involved npubs (author/zapper/chat members) and
  notes via userFinder/eventFinder, re-rendering in place as names, pictures,
  content, and post images arrive over a bounded relay window.

One file per notification (service/notifications/renderers/): DirectMessage,
GroupMessage (+welcome), Reply, Mention, Reaction, Zap (Lightning+nutzap+onchain),
Repost, Media, Article/Highlight, Code (git), Badge, Chess. EventNotificationConsumer
is now a slim policy dispatcher (account match + shared gates) delegating to them.

Closes push-vs-feed parity gaps: nutzaps (9321), onchain zaps (8333), reposts
(6/16), badge awards (8), and git PRs/updates (1618/1619) now render.

Dismiss-on-read is preserved (per-event id keying) so reading a post in-app
clears its tray notification. Adds monochrome status-bar drawables and the new
channel/title/group strings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0122uQ8BLHLeHDni81RBP26r
2026-07-23 05:16:08 +00:00
Claude 6d8a2f0950 docs(notifications): plan a per-kind beautiful push-notification redesign
Adds amethyst/plans/2026-07-23-push-notification-redesign.md: a rendering/UX
design for the Android tray notifications that gives each Nostr notification
kind (zap, reaction, reply, mention, DM, repost, nutzap, media, git, badge,
chess) its own accent color, status-bar icon, and notification style
(MessagingStyle / BigPictureStyle / colorized zap card), plus aggregation,
Conversations/Bubbles, and richer actions. Also documents the spec+builder
refactor, channel restructuring, icon/string work, parity gaps to close
(nutzap/onchain/repost/badge), a phased rollout, and testing.

Indexes the plan in amethyst/plans/README.md.
2026-07-23 04:18:18 +00:00
Vitor PamplonaandGitHub 56f8968172 Merge pull request #3676 from vitorpamplona/claude/privacy-options-modernize-9l51qs
Refactor privacy settings UI to use composable components
2026-07-22 21:06:51 -04:00
Claude 7c4090a800 feat(settings): modernize Privacy, Profile UI, Home Tabs & Calendar Reminder screens
Bring four settings screens onto the SettingsSection card kit used by the
UI Preferences / Security Filters screens, and drop the last Save/Cancel
flow in favor of auto-save + back-button-only navigation.

- Privacy Options: remove SavingTopBar + TorDialogViewModel staging; bind
  controls directly to TorSettingsFlow.tryEmit (the existing debounced
  propertyWatchFlow already persists changes), so edits save on change and
  the screen has only a back arrow. Rebuild the body from SettingsSection
  cards: a segmented Tor-engine tile, an Orbot-port block, a live preset
  picker row, and per-usage switch tiles.
- Profile UI & Home Tabs: reskin the ad-hoc rows into SettingsSection
  cards with SettingsSwitchTile / SegmentedChoiceTile (Home Tabs keeps the
  "can't disable the last tab" guard via the enabled flag).
- Calendar Reminders: swap the hand-rolled TopAppBar for
  TopBarWithBackButton and reskin into a SettingsSection card with a
  segmented lead-time selector, preserving the CalendarReminderPrefs writes
  and WorkManager schedule/cancel side-effects.
- Shared kit: make SegmentedChoiceTile internal for reuse and add a compact
  title-only SettingsSwitchTile overload with enabled support.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013JvrwLYbGVJQhabfZS7sBy
2026-07-23 00:57:09 +00:00
Vitor PamplonaandGitHub 7a9457e88b Merge pull request #3675 from vitorpamplona/claude/concordcommunitylistevent-relay-loading-s95nxm
Bootstrap Concord communities from pinned bottom-bar tabs
2026-07-22 18:52:09 -04:00
Claude 5166a5ac69 fix(concord): load pinned communities from their own relays
A Concord community pinned to the bottom bar wouldn't load at all when its
private kind-13302 joined-communities list wasn't already cached: the tab and
its server screen stayed blank. The list often lives only on the community's
own relays (Armada/Vector publish it there, never to the user's outbox), and
the only fetch that looked beyond the outbox — importConcordCommunities — was
triggered solely from the Concord hub and never queried the community's relays.

Carry each pinned community's bootstrap relays on its BottomBarEntry.Concord
tab (captured from the joined-list entry at pin time) and:

- importConcordCommunities now takes extra relays and folds in the relays saved
  on every pinned Concord tab, so the list is found where it actually lives;
- ConcordChannelPreload bootstraps app-wide: it fetches the list for any pinned
  community we don't yet know, so the tab and server screen fill in without the
  user ever opening the hub.

Once the list folds into the cache, ConcordChannelListState.liveCommunities
already surfaces it reactively (verified by a new late-arrival test) and the
plane preload picks the community up — so a late-arriving list with no local
backup now updates the tab and the Concord Channels screen.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RSiSXaHMEpuo3gcDuTZ24u
2026-07-22 22:40:46 +00:00
David KasparandGitHub 005ffa6b58 Merge pull request #3674 from vitorpamplona/l10n_crowdin_translations
New Crowdin Translations
2026-07-22 21:27:43 +01:00
vitorpamplonaandgithub-actions[bot] a37cc94481 chore: sync Crowdin translations and seed translator npub placeholders 2026-07-22 18:29:22 +00:00
Vitor PamplonaandGitHub 1e874239b8 Merge pull request #3673 from vitorpamplona/fix/no-client-tag-for-external-signers
fix(signer): never stamp our client tag on someone else's template
2026-07-22 14:26:27 -04:00
Vitor PamplonaandClaude Opus 4.8 636dd2afe7 fix(signer): never stamp our client tag on someone else's template
The NIP-89 client tag says "this app composed this event", so it belongs only
on templates Amethyst authored. NostrSignerWithClientTag was applied at the
account level, which meant it also fired on every event we sign on behalf of
an external client.

That is wrong twice over. It misattributes the event, and — because the tag is
appended before signing — it rewrites the exact bytes the caller is about to
have hashed into an id. NIP-07 callers routinely re-check the returned event
against the template they submitted, and block/buzz compares tags outright
(web/src/shared/lib/nostr-signer.ts):

    JSON.stringify(actual.tags) === JSON.stringify(expected.tags)

so joining a Buzz community from the in-app browser failed with "The NIP-07
extension returned an invalid signed event". Probed live over the WebView
devtools protocol: kind, created_at, content and pubkey all round-tripped
intact and only tags differed, by exactly the ["client","Amethyst"] we append.

Add NostrSigner.withoutClientTag() and use it at the two boundaries where the
template belongs to someone else:

- the napplet broker, covering napplets, nSites and web apps over NIP-07
- Nip46SignerState, where we act as another client's bunker — the same defect,
  and quieter, since that client never learns why its event changed underneath
  it

Amethyst's own events are untouched and still carry the tag. Unwrapping keeps
everything layered below (metering, NIP-13 mining) and leaves pubKey alone.
There is no NIP-55 provider surface to fix; we are only ever the client there.

Verified against Buzz's own four acceptance conditions after the change:
pubkey matches, sameUnsignedEvent true, id and sig present — and the invite
join then succeeded.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 13:57:39 -04:00
David KasparandGitHub 9380de7813 Merge pull request #3672 from vitorpamplona/l10n_crowdin_translations
New Crowdin Translations
2026-07-22 17:53:10 +01:00
vitorpamplonaandgithub-actions[bot] 8247d49afa chore: sync Crowdin translations and seed translator npub placeholders 2026-07-22 16:46:03 +00:00
Vitor PamplonaandGitHub 4eb1231270 Merge pull request #3669 from vitorpamplona/l10n_crowdin_translations
New Crowdin Translations
2026-07-22 12:43:23 -04:00
Vitor PamplonaandGitHub 70bcbd0bdb Merge pull request #3666 from davotoula/fix/addressable-author-loader-cme-alternative
Stop ConcurrentModificationException in AddressableAuthorRelayLoaderSubAssembler
2026-07-22 12:43:16 -04:00
Vitor PamplonaandGitHub 5cfb7c3454 Merge pull request #3670 from vitorpamplona/claude/ci-android-test-report-plugin-k9p0kb
Replace abandoned Android test report action with mikepenz/action-junit-report
2026-07-22 12:42:34 -04:00
davotoulaandgithub-actions[bot] adf9b68c7a chore: sync Crowdin translations and seed translator npub placeholders 2026-07-22 16:32:32 +00:00
Claude b86bc5b798 ci: replace abandoned android-test-report-action with mikepenz/action-junit-report
asadmansr/android-test-report-action@v1.2.0 is a Docker action whose
Dockerfile is `FROM ubuntu:18.04` + `apt-get install python` (Python 2).
Ubuntu 18.04 (bionic) is end-of-life, so its apt archives are now
unreliable and Python 2 has no installation candidate — the image rebuild
runs on every CI invocation, takes ~8 minutes, and has started hard-failing
the test-and-build-android job (`E: Package 'python' has no installation
candidate`). The action was last released in 2020 and is unmaintained, and
it was referenced by a movable tag rather than a pinned commit.

Swap it for mikepenz/action-junit-report (actively maintained, Apache-2.0,
JS action — no Docker rebuild), pinned to the v6.4.2 commit SHA. Use
annotate_only so it needs no `checks: write` permission and keeps working on
pull requests from forks; fail_on_failure keeps the job red when a unit test
fails, matching the previous step's behavior.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HMtpSuyr8FBh2ZQ22BV4ZP
2026-07-22 16:25:23 +00:00
davotoula e6f6edd29b update cs,sv,de,pt 2026-07-22 17:23:43 +01:00
davotoula 6ed955f0cf docs: language updates 2026-07-22 17:23:04 +01:00
Vitor PamplonaandGitHub 52ce590505 Merge pull request #3668 from vitorpamplona/claude/quartz-jvmtarget-17-v0g1u3
Downgrade JVM target from 21 to 17 in quartz module
2026-07-22 11:44:46 -04:00
Claude 5f954c0e04 build(quartz): target JVM 17 instead of JVM 21
Lower the quartz module's Kotlin jvmTarget for both the JVM and Android
compilations from JVM_21 to JVM_17, broadening the range of runtimes that
can consume the published library.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1Hn61gQJ1joUznESzUHU4
2026-07-22 15:42:02 +00:00
Vitor PamplonaandGitHub 6440445938 Merge pull request #3667 from vitorpamplona/fix/video-decoder-budget-and-aspect-ratio
fix(video): decoder-budget exhaustion and black bars on live streams
2026-07-22 11:35:53 -04:00
Vitor PamplonaandClaude Opus 4.8 edf30489d0 fix(video): size the player box so live streams stop rendering black bars
A live stream opened for the first time drew ~90px of black above and below
the picture. The video surface itself was correct 16:9; the box enclosing it
was not. Measured on a Pixel 9 emulator: container [0,274][1080,1062] (788px
= StreamingHeaderModifier's 300.dp cap) holding a TextureView of
[0,364][1080,972] (608px = 16:9 at 1080 wide), centred, so (788-608)/2 = 90px
per side.

Two independent causes, both needed fixing:

ContentWarningGate takes a `modifier` but drops it for anything not flagged
sensitive — the non-sensitive path emits `content()` bare. ZoomableContentView
was routing mediaSizingModifier() through exactly that parameter, so for
ordinary media the sizing never reached the layout at all. With no height
constraint the player stretched to whatever ceiling enclosed it and
letterboxed the frame inside. Apply the sizing to the inner Box, which is
always emitted.

Even applied, the ratio was unknown on a first play: a NIP-53 stream carries
no imeta `dim`, and MediaAspectRatioCache is only filled once the decoder
reports a size. The miss was frozen for the whole visit because the cache was
a plain LruCache read during composition, which triggers no recomposition when
it later fills — hence the bars vanishing only on a *second* visit to the same
stream. Back cache entries with snapshot state so a composition-time read
updates, and default an unknown video to 16:9 so the first layout already
lands in the right place.

VideoView keeps reading the cache inside remember() on purpose, with a comment
explaining why: making it observable there flips the ratio mid-playback, which
both adds an aspectRatio and emits an extra Spacer, and restructuring children
around a live AndroidView strands the player on a stale surface — the video
redraws at native size in the corner while layout bounds still look correct.

Verified on a cold cache: container and TextureView are both
[0,274][1080,882], against a header ending at 274 — zero gap. Feed image and
video layouts unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 11:24:30 -04:00
Vitor PamplonaandClaude Opus 4.8 041a4c1c71 fix(playback): enforce the decoder budget when acquiring players
MediaCodec instances are a per-process resource with a hard per-device
ceiling — the Android emulator's c2.goldfish.h264.decoder declares
`concurrent-instances max="4"`. Past that, MediaCodec.start() fails with
NO_MEMORY, MediaCodecRenderer reports "Failed to initialize decoder", and
the video surfaces to the user as "can't load". Opening a live stream after
scrolling a few feed videos reproduced this reliably; killing the process
made the same stream play, since that released every held codec.

The device ceiling was already computed by SimultaneousPlaybackCalculator,
but only reached ExoPlayerPool as `poolSize`, which governs how many idle
players are *retained*. The acquire path was uncapped
(`coldPool.poll() ?: builder.build(context)`), and MediaSessionPool held a
hardcoded LruCache(10) of sessions, each pinning a checked-out player. So a
4-decoder device would happily hold 10.

Enforce the budget where players are handed out:

- Track live decoders process-wide, counting checked-out and warm players
  (cold ones have been stop()'d and hold none). The counter and the pool
  registry are global because PlaybackService builds one pool for direct
  traffic and another for Tor-proxied traffic; a per-pool budget let the
  app hold twice the ceiling.
- Before a cold or fresh player is handed out, reclaim headroom by demoting
  warm players to cold — own pool first, then siblings. Warm entries are a
  scroll-back cache, so they are the right thing to give up under pressure.
- Size the session cache from the same device budget, keeping the previous
  10 as an upper bound so capable devices are unaffected.

Verified on the emulator: 9 codec allocations across a session with zero
NO_MEMORY and zero decoder-init failures, where allocation #5 previously
died.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 11:24:30 -04:00
davotoula 71620380fb update KotlinJpsPluginSettings version 2026-07-22 14:41:16 +01:00
davotoula 53a823e1b2 Code review:
- private monitor, and correct the commit() KDoc
- replace atomics handshake with a single monitor
- flag userFinder re-entrancy assumption in commit() KDoc
2026-07-22 14:41:16 +01:00
davotoula 2e4afbf75e fix: stop ConcurrentModificationException in AddressableAuthorRelayLoaderSubAssembler
`activeSubscriptions` was a plain LinkedHashSet iterated (`SetsKt.minus`) and
mutated (`clear`/`addAll`) with no synchronization, while `invalidateFilters()`
ran synchronously on whatever thread called subscribe/unsubscribe.

Those callers are genuinely concurrent: `ComposeSubscriptionManager` invokes
`invalidateKeys()` after releasing its own lock, and
`LifecycleAwareSubscription`'s 30s grace-period unsubscribe fires on a
`Dispatchers.Default` worker while composition subscribes from elsewhere.
Hence the reported `ConcurrentModificationException` on
`DefaultDispatcher-worker-70`.

This is the only member of `EventFinderFilterAssembler.group` that implements
`IEoseManager` directly; its two siblings extend `BaseEoseManager`, whose
`invalidateFilters` hands off to `BundledUpdate` and is therefore never run on
the caller's thread nor concurrently with itself.

Fix, matching that existing pattern and avoiding locks:

- Route through `BundledUpdate`. `BasicBundledUpdate` holds `isProcessing`
  under a Mutex, so only one body runs at a time — the concurrent
  iterate-vs-mutate window is gone by construction. It also moves the
  `allKeys()` scan and per-stub `getOrCreateUser` off the caller thread, which
  `ComposeSubscriptionManager` documents as "called by main. Keep it really
  fast."
- Hold the state in an `AtomicReference<Set<...>>` of immutable snapshots
  swapped with `exchange()`, plus an `AtomicBoolean` teardown flag, mirroring
  the `AtomicReference` + CAS idiom in `FilterIndex`/`BanStore`.
- `bundler.cancel()` cannot stop a body already executing (no suspension
  points), so `destroy()` flags first and an in-flight body compensates by
  releasing what it just acquired. Double-unsubscribe is a no-op.

No locks are introduced; the hot path is strictly cheaper than before.

Tested: the new concurrency test reproduces the exact production failure
against the pre-fix code (`ConcurrentModificationException` alongside the
overlap detector) and passes after. Full :amethyst suite green (941 tests).

Note the pre-existing `UserFinderQueryState` identity-equality churn is
deliberately NOT addressed here: the set-diff never converges because each run
allocates fresh wrappers. It widens this race window but is an independent
defect needing its own design decision.
2026-07-22 14:40:14 +01:00
Vitor PamplonaandGitHub c9e8cd4e0c Merge pull request #3665 from vitorpamplona/l10n_crowdin_translations
New Crowdin Translations
2026-07-22 09:25:40 -04:00
vitorpamplonaandgithub-actions[bot] 1c9f8f8d54 chore: sync Crowdin translations and seed translator npub placeholders 2026-07-22 13:24:31 +00:00
Vitor PamplonaandGitHub d72d7df0a0 Merge pull request #3664 from nrobi144/worktree-feat+desktop-polls
feat(desktop): NIP-88 polls (render, vote, create) + "Polls" search facet
2026-07-22 09:21:27 -04:00
nrobi144andClaude Opus 4.8 b2adb157ff feat(desktop): NIP-88 polls (render, vote, create) + "Polls" search facet
Adds full NIP-88 poll support to Amethyst Desktop and a search content-type
filter for polls.

Polls (DesktopPollCard):
- Render kind-1068 polls in feed + thread (and reposted/boosted polls) as an
  interactive card via NoteCard's bottomContent slot.
- Vote (single-choice radio / multi-choice checkbox), re-vote ("Change vote")
  seeded with the prior selection; hide-until-voted with a "View results" opt-in.
- Tallies reuse commons PollResponsesCache; responses are fetched from the
  poll's OWN declared relays (NIP-88 relay tags) unioned with connected relays,
  so the full tally loads regardless of the viewer's relay set. Votes are
  likewise published to the poll's relays (not just broadcastToAll).
- Result row marks the viewer's own choice (border + check), tap a row to see
  its voters, footer shows distinct-voter count + deadline/ended state, and the
  voter gallery draws the viewer front-most with a ring.
- Create polls from the composer (options, single/multi, optional deadline);
  the dialog content scrolls with a pinned Cancel/Publish row; a poll requires
  a question and >=2 options.

Wiring:
- DesktopLocalCache.consume for kind 1068/1018 (response links into pollState).
- DesktopFeedFilters + FilterBuilders surface polls; feed/thread interaction
  subscriptions fetch kind-1018 responses.
- Thread + profile pass myPubKeyHex so the viewer's vote-state renders.

Search "Polls" facet:
- KindRegistry preset + alias for kind 1068 (auto-renders the filter chip and a
  NIP-50 kind filter); SearchResultsList renders poll results interactively and
  SearchScreen fetches their responses.

Also:
- Read-only accounts see results instead of dead vote controls.
- Cold-start: the response subscription re-evaluates as relays connect.
- Pull the upstream fix for the pre-existing RelayLatencyTracker.sweep
  ConcurrentModificationException (synchronized(pending)) so relay-health
  reclassify no longer crashes the UI during search.

Ripple/shaping: clickable elements clip to their shape for bounded ripple.

Tests: commons PollResponsesCache (dedup/tally/WoT sort) + DesktopLocalCache
response-linking.

Deferred (noted in review): wall-clock re-check of a poll expiring mid-view;
mention-dropdown now inside the composer scroll.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 11:09:25 +03:00
Vitor PamplonaandGitHub 72b8df7f6d Merge pull request #3663 from vitorpamplona/claude/sqlite-query-performance-bh2j27
SQLite query scaling: FTS contentless, tag-merge, pooled statements
2026-07-21 19:38:22 -04:00
Claude d571bee394 fix(store): rank multi-filter search REQs + harden merge from branch audit
Two-reviewer adversarial audit of the branch found no correctness/data-loss/
crash bugs. This closes one gap and applies three robustness fixes:

- Multi-filter search ordering: a REQ whose filters all carry a search term
  (e.g. the client's search-across-kinds) was created_at-ordered via the union
  path. Now relevance-ordered — unionSubqueriesIfNeeded(projectRank) projects
  rank per branch, UNION ALL + GROUP BY row_id MIN(rank) dedups across branches
  keeping the best score. Only when every branch is a search branch; mixed
  search/non-search REQs and count/delete unions stay as before.
- prepareAuthorStreams/prepareTagStreams build cursors via buildStreams, which
  closes already-prepared statements if a later prepare throws (was: stranded
  checked-out, un-reset handles holding read locks in the pooled connection).
- Stream counts computed as Long so authors×kinds / values×kinds can't overflow
  Int back into the eligible band and route a huge fan-out into the merge.
- Renamed CachedStatement.finalize() -> finalizeStatement(): a no-arg finalize()
  is the JVM Object.finalize, risking a GC double-close of the native handle.

Tests: multi-filter search relevance + cross-branch dedup + count parity;
existing merge/cache/search suites green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
2026-07-21 23:33:33 +00:00
Claude 52ba19d324 test(store): adversarial search-rank cases (two-tag+search, count parity, non-searchable delete)
From the branch audit: verifies the search-relevance path holds under a
two-tag-key filter (requires both tags, ranks by bm25, no duplicate rows),
that count(filter) matches query size under a limit smaller than the match set
(NIP-45), and that deleting a non-searchable event (no FTS row) fires the
contentless delete trigger against an absent rowid harmlessly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
2026-07-21 23:12:59 +00:00
Claude de114d1621 fix(store): relevance-order search+tag queries too, not just tag-free search
Any filter with a search term must be relevance-ranked (NIP-50), but only the
tag-free shape (makeSimpleSearch) was — search + a tag fell through to the
row-id-subquery path (prepareRowIDSubQueries/makeQueryIn), which ordered by
created_at.

prepareRowIDSubQueries gains projectRank: when a search filter joins event_fts,
it also projects the bm25 score as a `rank` column and cuts its LIMIT by rank
(most relevant, not newest); makeQueryIn(orderByRank) then presents the joined
result by that rank, created_at DESC as tie-break. Off by default, so
count/delete/union/negentropy (which must stay single-column and unranked) are
untouched. toSql wires it on whenever the filter carries a search term.

SearchRelevanceOrderTest adds a tag-scoped case (stronger-but-older outranks
weaker-but-newer, wrong-tag and non-matching excluded, limit cuts by score).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
2026-07-21 22:54:47 +00:00
Claude 8e7b1b63be fix(store): order NIP-50 search by relevance (bm25), not created_at
NIP-50: results are returned "in descending order by quality of search result
... not by the usual .created_at", with the limit applied after the score. The
store sorted search by created_at DESC (pre-existing), so it returned the
newest matches rather than the best ones.

makeSimpleSearch (the search [+ kinds/authors/since/until] + limit shape) now
orders by FTS5 bm25 (ORDER BY event_fts.rank, created_at DESC as a tie-break).
Verified bm25 rank works on the contentless table through the join, and that a
stronger-but-older match outranks a weaker-but-newer one. The rarer
search+specific-tag shape and the negentropy snapshot still sort by created_at
(the row-id subquery can't carry rank; negentropy is a sync set) — documented.

This is a correctness fix, not a scaling one: bm25 scores every match, so
search latency still grows with the match set. Tests: SearchRelevanceOrderTest,
Fts5CapabilityProbe.bm25RankWorksOnContentlessTableInAJoin; QueryAssemblerTest
search plans updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
2026-07-21 22:21:54 +00:00
Claude e6a8f80826 fix(store): drop non-compliant rowid search ordering, keep FTS delete/size wins
searchOrderByRowId ordered NIP-50 search by the FTS rowid (ingestion order) to
get O(limit) search, but NIP-01's limit requires the newest events by
created_at. Once ingestion diverges from created_at (any historical sync) that
returns the wrong events under a limit — a spec violation — so the flag, its
QueryBuilder branch, and its test are removed. Search stays created_at-ordered;
corpus-independent search is an external-engine job, not this index.

The contentless + rowid=row_id schema stays for the reasons that don't touch
ordering: the delete trigger now seeks by rowid (O(log n)) instead of scanning
by an FTS column (O(n)) — measured ~78× faster at 8k rows and widening, on a
path every deletion hits — and the index is smaller. Benchmark reframed around
the delete win and the honest (unchanged) search cost; plan doc updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
2026-07-21 22:05:30 +00:00
Claude c7670a52d4 perf(store): scale NIP-50 search and tag-watcher queries with corpus size
The scale-curve report showed the SQLite store degrading with corpus size on
NIP-50 search (~18×) and the large-IN tag watcher, while point reads stayed
flat. Three read/size changes (write path and index set unchanged):

- FTS: rebuild event_fts as a contentless FTS5 table (content='',
  contentless_delete=1) keyed by rowid = event_headers.row_id. Drops the
  stored content copy (smaller index); external-content can't hold the
  derived indexable text, so contentless is the correct primitive. Adds an
  opt-in searchOrderByRowId strategy flag: ORDER BY event_fts.rowid DESC
  early-terminates (O(limit), corpus-independent) at the cost of
  ingestion-order results — flat ~0.22 ms vs created_at's 4.26 ms at 200k
  (~19×). reindexAll ends with 'optimize'; the periodic optimize() folds in a
  bounded segment 'merge'. DB version 4->5 with a drop-and-rebuild migration.

- MergeQueryExecutor: extend the k-way merge to the tag path (kinds + #e IN
  [hundreds] + limit), one cursor per (value[,kind]) stream heap-merged to the
  limit, deduping events that carry several queried values. O(limit + streams)
  instead of collecting all matches and sorting.

- StatementCachingConnection: pool multiple handles per SQL so the merge's
  many concurrent identical-SQL cursors all hit the cache (previously only the
  first did) and repeated polls reuse their per-stream statements.

Tests: contentless migration (real v4 DB upgrade), rowid-order search, tag
merge correctness (incl. cross-stream dedup), statement pool, FTS5 capability
probe, and an FTS search-scaling benchmark. Plan in
quartz/plans/2026-07-21-sqlite-query-scaling.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqGuBuSUsRudGerPqDuoKA
2026-07-21 21:45:42 +00:00
Vitor PamplonaandGitHub 35e61e1d98 Merge pull request #3662 from vitorpamplona/fix/docs-license-badge-and-skill-md
docs: fix license badge and refresh stale SKILL.md
2026-07-21 17:38:07 -04:00
Vitor PamplonaandClaude Opus 4.8 6730100853 docs: fix license badge and refresh stale SKILL.md
The README license badge label read "Apache-2.0" while LICENSE, PRIVACY.md
and every source header are MIT. Only the static label text was wrong (the
shields.io endpoint auto-detects), but it is the license on the front page.

SKILL.md had drifted from the codebase since the Kotlin DSL migration:

- All Gradle references pointed at Groovy `build.gradle` / `settings.gradle`;
  the repo is `.gradle.kts` throughout. Converted the snippets to Kotlin DSL
  and matched the repo's existing `getByName("release")` style.
- The plugins block listed `jetbrainsKotlinAndroid` (gone) and omitted
  `serialization` and `googleKsp`.
- compileSdk is 37, not 35. Added a pointer to libs.versions.toml so the
  number has a source of truth rather than drifting again.
- The client-tag section told readers to create
  `nip01Core/tags/clientTag/TagArrayBuilderExt.kt` and edit both `build()`
  functions in TextNoteEvent. That file already exists at
  `nip89AppHandlers/clientTag/`, and the tag is now applied centrally by the
  NostrSignerWithClientTag decorator — so rebranding is a one-constant edit
  to CLIENT_TAG_NAME.
- Default relays pointed at `quartz/src/main/java/...`, a path that does not
  exist in the KMP layout; they live in commons `defaults/`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 17:27:39 -04:00
Vitor PamplonaandGitHub 69671d0009 Merge pull request #3660 from vitorpamplona/claude/nostr-relay-filters-mapping-bu95ib
Optimize relay query performance: cost-based driver selection, tag-author indexing, and fanout memoization
2026-07-21 15:35:38 -04:00
Claude 078758888a perf(relay): remove live-path allocations in LiveEventStore + FilterIndex
The remaining SmallReqFloorBenchmark waste, on the per-row replay, the
per-event live fanout, and the per-accepted-event index probe:

- LiveEventStore replay dedupe: a SeenIds holder with an inline lock
  replaces the local-fn-plus-lambda that allocated one closure per
  streamed row (and again per live delivery). Its HashSet is created
  empty so the JVM defers the backing table to the first add — a 0-row
  replay no longer allocates a 1024-slot table (was ~4 MB across the
  benchmark's 1000 idle subs).
- Live fanout serializes the event body once and passes it through
  onEachLive(event, body); RelaySession splices it into the per-sub
  frame prefix. An event matching N live subscriptions paid N identical
  Jackson passes before; now one. queryRaw's onEachLive signature gains
  the body arg (EventSourceBackend default serializes inline, no
  cross-sub memo, no regression). Measured: fanout 1->200 live subs
  0.50 ms (2.5 us/sub).
- FilterIndex holds subscribers in one persistent map per dimension, so
  candidatesFor (once per accepted ingest event) probes with the event's
  own fields and allocates no IdKey/AuthorKey/KindKey/TagKey wrappers;
  BucketKey now lives only in the rare register/unregister bookkeeping.

SmallReqFloorBenchmark grows a fanout stage (200 live subs, one submit)
to anchor the fanout number; it drives `live` directly and guards the
await with withTimeout so a future fanout regression fails fast.

Verified: quartz relay.server + FilterIndex suites (110 tests),
SmallReqFloorBenchmark, geode suite (126 tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
2026-07-21 19:23:38 +00:00
Claude 387bfe99ee perf(relay): drop per-op allocations on the frame + search paths
Three hot-path allocation cuts the SmallReqFloorBenchmark stages flagged:

- strippingSearchExtensions: index-loop guard returns the same list with
  zero allocation when no filter carries a search term (every non-search
  REQ/COUNT/snapshot, the overwhelming majority).
- EoseMessage/OkMessage: direct-buildString wire form on the escape-free
  fast path (EOSE per REQ, OK per publish), skipping the generic
  serializer's node tree; exotic subIds/reasons fall back. Shared
  isEscapeFreeAscii helper in WireJson.kt, mirroring NegMsgMessage.

Verified: quartz relay.server + message-frame suites (110 tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RzkoN3SJHCZWRAiadXXG4w
2026-07-21 19:23:21 +00:00
Vitor PamplonaandGitHub 4df7354ffc Merge pull request #3658 from vitorpamplona/fix/concord-messages-cold-boot
fix: Concord groups missing or slow on the Messages tab after a cold boot
2026-07-21 14:38:08 -04:00
Vitor PamplonaandGitHub a11de43400 Merge pull request #3659 from vitorpamplona/fix/persist-keypackage-algo-lists
fix: persist the MIP-00 key-package and favorite-algo-feeds lists
2026-07-21 14:37:54 -04:00
Vitor PamplonaandClaude Opus 4.8 0a228a935b fix: persist the MIP-00 key-package and favorite-algo-feeds lists
AccountSettings declares 25 backup* fields and saves each on change, but two
were never wired into LocalPreferences — neither written to nor read back from
the encrypted prefs:

  - backupKeyPackageRelayList  (MIP-00, Marmot/MLS)
  - backupFavoriteAlgoFeedsList (kind 10090)

Both only ever existed for the lifetime of the process. Their consumers already
implement the restore-from-backup path — KeyPackageRelayListState's
normalizeKeyPackageRelayListWithBackup falls back to the field, and
FavoriteAlgoFeedsListState's init seeds the cache from it — so that code was
dead after every cold boot and the value read as empty until relays answered.

For the key-package list that matters beyond latency: it feeds
Account.publishRelaysFor(), which decides where this account's key packages are
published so others can add it to groups, and Account.updateKeyPackageRelays()
reads it as the *previous* list when computing an update.

Found by auditing all 25 backup* fields against their five LocalPreferences
wiring sites after the same gap turned up for the Concord community list; the
other 23, NIP-29's relay-group list included, are correctly wired.

Verified on device with a persist-then-cold-boot pair: the key-package list is
absent on the first boot and restores 1.6 s after the second. The favorite algo
feeds list could not be exercised on this account (it has no kind 10090, so
there is nothing to persist); it is wired identically and its type arguments
are compiler-checked, but it is not verified end to end.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 14:23:38 -04:00
Vitor PamplonaandClaude Opus 4.8 9ddf571b11 fix: persist the Concord community list so cold boot doesn't refetch it
AccountSettings has held backupConcordList and saved it on change since the
feature landed, but the field was never wired into LocalPreferences: it was
neither written to nor read back from the encrypted prefs. So it only ever
existed for the lifetime of the process, concordList() returned null on every
cold boot, and the kind-13302 joined-communities list had to be refetched from
relays before a single Concord plane could be subscribed.

Every sibling list — channel, community, hashtag, geohash, ephemeral chat,
relay group, trust provider — is persisted this way; Concord was the one that
was missed. That made it the only chat type whose rooms could not appear until
the network answered, which is the bulk of the cold-boot delay: the joined list
gates the control-plane REQ, the control plane gates the fold, and the fold
gates the channels.

Measured on device, boot -> first Concord plane wrap:
  - without the backup: liveCommunities sat empty for ~56 s waiting on the
    13302 fetch (first arrival from nostr.mom), first wrap at +45 s
  - with the backup restored: list decoded 1.6 s after boot (30 ms), first
    wrap at +6 s

Wired the same five sites the other lists use (pref key, save, read, parse,
restore). Prefs are encrypted and backupCashuWallet already sets the precedent
for persisting a secret-bearing event, so the community roots in the 13302
content are stored no differently than the wallet's.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 14:09:59 -04:00