Commit Graph
16200 Commits
Author SHA1 Message Date
davotoula 489e8ae50b Code review:
- enforce op equality by class and pin the tag-uniqueness invariant
- hoist crypto op equals/hashCode onto OpCrypto
2026-07-04 21:38:08 +02:00
davotoula 772b4ea8ed fix(quartz): repair equals/hashCode contracts in OTS ops and VerifyResult
Op instances key Timestamp.ops (MutableMap<Op, Timestamp>), so contract
violations corrupt hash-map behavior:

- OpKECCAK256 defined equals without hashCode, so equal instances hashed
  by identity — two equal keys could land in different buckets, producing
  duplicate branches or failed lookups in keccak256 timestamp trees. Add
  hashCode = TAG, mirroring OpSHA1/OpSHA256/OpRIPEMD160.
- OpBinary defined hashCode without equals — and its TAG referenced
  Op.TAG (0x00), a no-op XOR. Define the equals/hashCode pair once on
  OpBinary using tag() and drop the duplicated overrides from
  OpAppend/OpPrepend (behavior unchanged: same tag + same arg content).
- VerifyResult.equals cast without a type test (ClassCastException on
  foreign types instead of false) and hashCode force-cast the nullable
  timestamp (NPE for null-timestamp results). Convert to a data class;
  the custom toString and compareTo stay.
2026-07-04 21:37:24 +02:00
Vitor PamplonaandGitHub b7912244fb Merge pull request #3475 from vitorpamplona/claude/negentropy-kmp-update-s5m8cw
Upgrade negentropyKmp to v1.1.1
2026-07-04 12:49:20 -04:00
Claude bb6bf99586 chore: update negentropy-kmp to v1.1.1
Bumps the negentropy-kmp dependency from v1.0.2 to v1.1.1. The release is
backward compatible for consumers (StorageVector still provided; the codebase
only consumes IStorage and never implements it). Quartz compiles and all
NIP-77 negentropy tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M3659Rs6ayVm4XmxvbtpWE
2026-07-04 16:21:55 +00:00
Vitor PamplonaandGitHub df01354fa5 Merge pull request #3474 from davotoula/fix/unchecked-file-delete-results
Handle unchecked File.delete() return values across all modules
2026-07-04 11:59:21 -04:00
davotoula afe8c783c0 test(commons): cover deleteOrWarn and restrictToOwner helpers 2026-07-04 17:48:34 +02:00
davotoula 206c0979b1 Code review:
- align voice-file debug log with deleteOrWarn's is-gone contract
- Convert the delete-then-warn sites the sweep left hand-rolled in already
  touched files: ThumbnailDiskCache corrupt-file and temp-thumbnail cleanup,
  NappletBlobCache.put leftover temp, and SecureKeyStorage's bare delete of
  the fallback key file (the highest-stakes delete in that file).
- Drop the exists() guards left layered over deleteOrWarn — the helper
  already treats an absent file as silent success.
- Collapse AccountManager's legacy-file triple into a loop and drop the
  stale "silent" from its comment.
- Snapshot lastModified alongside length in NappletBlobCache.trimToSize so
  sortedBy compares in-memory values instead of stat-ing per comparison.
- Promote DesktopTorManager's private restrictToOwner into a shared
  File.restrictToOwner(tag) in commons (600 files / 700 dirs) — the repo's
  sixth private copy of this pattern was one too many; the remaining copies
  can migrate incrementally
2026-07-04 15:53:02 +02:00
davotoula 27650f2f77 fix: handle remaining unchecked File.delete() and Tor dir permission results 2026-07-04 15:52:33 +02:00
davotoula 2b9ffb4849 Code review:
- extract shared File.deleteOrWarn helper for cache eviction
2026-07-04 15:51:41 +02:00
davotoula c3d6c05482 fix: handle File.delete() return values in cache eviction and voice cleanup 2026-07-04 15:51:40 +02:00
Vitor PamplonaandGitHub 526ffde3f5 Merge pull request #3473 from davotoula/feat/birdstar-detection-kind-2473
support Birdstar bird detection events (kind 2473)
2026-07-04 07:30:25 -04:00
David KasparandGitHub a68572316b Merge pull request #3472 from vitorpamplona/l10n_crowdin_translations
New Crowdin Translations
2026-07-04 12:51:34 +02:00
davotoulaandgithub-actions[bot] 370a56ab69 chore: sync Crowdin translations and seed translator npub placeholders 2026-07-04 10:46:51 +00:00
David KasparandGitHub c535c64e67 Merge pull request #3471 from vitorpamplona/l10n_crowdin_translations
New Crowdin Translations
2026-07-04 12:44:47 +02:00
davotoula b8d3791ea3 Code review:
- only parse commonName when the alt tag has the Birdstar prefix
- summary() on both Birdstar events now uses the canonical NIP-31
  tags.alt() helper instead of a raw firstTagValue("alt") lookup
- speciesReference() only returns http(s) URLs since UIs render it as
  a clickable link (rejects e.g. javascript: schemes), with a test
- Detection card: parse tags once into a single remember slot, drop
  the near-dead '?: summary' title fallback, stop rebuilding the
  italic TextStyle every recomposition
- Hoist the duplicated bird-emoji literal into a shared BIRD_PREFIX
- Trim the redundant factory test to the assertIs idiom
2026-07-04 12:42:18 +02:00
davotoula a83b8e4064 feat: support Birdstar bird detection events (kind 2473)
- fetch Birdex life lists in home and profile relay REQs
- richer Birdstar cards — common name title, Wikidata link, bird emoji
- surface Birdstar bird detections in home and profile feeds
2026-07-04 10:45:20 +02:00
vitorpamplonaandgithub-actions[bot] 3d4c2f1963 chore: sync Crowdin translations and seed translator npub placeholders 2026-07-04 07:31:05 +00:00
Vitor PamplonaandGitHub 9f981117bd Merge pull request #3470 from vitorpamplona/claude/ci-subscribe-before-connect-test-2c5wow
fix(quartz): record REQ state before send in PoolRequests.syncState
2026-07-04 03:28:59 -04:00
Claude f64b2e6f1c fix(quartz): record REQ state before send in PoolRequests.syncState
On connect, syncFilters re-sends every desired REQ through
PoolRequests.syncState. It previously sent the frame and only recorded
the subscription as SENT afterward, in the post-send onSent callback. A
relay that answers faster than that callback runs — the in-process
transport used by the desktop launch-optimization tests, or any relay on
a fast path — can deliver the EOSE while the per-sub state still reads
"nothing in flight" (onConnecting cleared it, onSent hasn't recorded it).
The EOSE handler then sees empty filters, concludes it never sent a REQ,
and fires a duplicate, replaying the whole page a second time.

Pre-mark the sub as SENT under its lock before the frame leaves, mirroring
the decideCommandLocked pre-mark already used by sendToRelayIfChanged, so
a response can never race ahead of the record. The send stays
unconditional: this is a fresh-connection sync (onConnecting always
cleared the per-relay state first), so there is no in-flight REQ on the
new socket to dedupe against.

Fixes the flaky SubscribeBeforeConnectTest, which asserted a pre-connect
subscription delivers exactly its events once and intermittently saw them
doubled.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4kmtZoNUSXxwG23wD2JwP
2026-07-04 05:42:46 +00:00
Vitor PamplonaandGitHub ba9db9e2cb Merge pull request #3469 from vitorpamplona/claude/relay-performance-geode-quartz-v6lbys
feat(quartz): NostrServer.ingest — local write path with per-submission verify skip
2026-07-03 22:43:49 -04:00
Claude bae2031cf3 fix(geode): validate config knobs and mirror filter at boot
Fail-loud on config that would silently degrade the running relay:

- readers = 0 makes every query hang forever on an empty reader pool;
  readers < 0 crashes with an unrelated message. optimize_interval_seconds
  <= 0 busy-loops PRAGMA optimize under the writer mutex. StaticConfig
  .validate() (called at boot) rejects both.

- A typo in [[mirror]].filter (e.g. `kindss`) parsed to an empty
  match-everything filter through the tolerant deserializer — silently
  widening a trusted upstream's skip-verify scope to the whole firehose.
  MirrorFilterValidator strict-checks the filter JSON at boot: unknown
  keys and non-array list fields fail startup.

- The self-mirror guard now compares scheme-insensitively (ws:// vs
  wss:// for the same host is still us) via displayUrl().

- The maintenance loop rethrows CancellationException and no longer
  swallows Errors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-04 02:32:35 +00:00
Claude 401f36ee82 fix(geode): mirror trust bound to origin relay; reconnect advances since watermark
Two mirror hardening fixes from the audit:

- Trust leak: the skip-verify decision was keyed on the subscription id,
  but the client pool dispatches EVENTs by subscription id alone and
  every [[mirror]] upstream shares one client. A hostile untrusted
  upstream could answer with the trusted upstream's subscription id and
  ride its skip-verify into the store. startDown now drops any event
  whose delivering relay isn't the one that subscription dialed
  (relay != up.url). New MirrorWorkerTrustOriginTest injects a foreign
  subId frame from a hostile relay and asserts it never lands.

- Reconnect replay: `since` was frozen at boot, so a long-lived daemon
  re-streamed the whole backfill window on every upstream flap. The down
  path now tracks the newest ingested created_at and, on the
  connected->disconnected edge, advances the REQ's since to
  watermark - overlap before the reconnect re-sends it. Advancing only on
  disconnect means a stable link never re-queries. The reconnect test now
  asserts the watermark advanced.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-04 02:32:21 +00:00
Claude 9bb75d9bbd fix(quartz/desktop): close socket on connect failure; NODELAY for desktop pre-init client
TcpNoDelaySocketFactory's connecting overloads used
`socket().apply { connect(...) }`, which leaks the file descriptor if
bind/connect throws (the JDK's connecting Socket constructors close on
failure; ours didn't). Wrapped in a helper that closes on throw. OkHttp
only calls the no-arg overload, so this guards any other direct caller.

DesktopHttpClient's pre-init `simpleClient` (direct relay sockets opened
before setInstance) now gets the same TcpNoDelaySocketFactory as
directClient. failClosedClient is left as-is: it's a SOCKS client and
OkHttp bypasses the socket factory for SOCKS proxies.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-04 02:32:08 +00:00
Claude 2be0c9f880 fix(quartz): NostrServer.ingest verifies inline when the queue hook is off
ingest() bypasses the per-connection policy chain, which is where
VerifyPolicy lives. The IngestQueue verify hook only exists when
parallelVerify is true, so with parallelVerify = false and
skipVerify = false, ingest() previously verified nothing — an untrusted
mirror upstream on a relay running the legacy in-policy verify path could
inject forgeries. ingest() now verifies inline in that configuration
(same rejection reason as the queue), so the documented "default keeps
verify-everything semantics" holds regardless of parallelVerify. KDoc
also spells out that ingest() skips the entire policy chain (blacklists,
size limits), which callers must screen for themselves.

Test covers the parallelVerify = false server: forged rejected, trusted
skip and valid still land.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-04 02:31:54 +00:00
Claude 6ac2903a3a fix(quartz): live negentropy index — same-batch displacement, no-op kind-5, rebuild cap
Audit follow-ups on the live NIP-77 index, all with the store/scan
equivalence test extended to cover them:

- Same-batch replaceable displacement left a dead id in the index.
  applyAfterCommit applied all removes before all adds, so when a later
  row in one transaction displaced an earlier row of the same batch (two
  versions of one replaceable — the mirror-backfill hot path), the
  displaced row's remove no-op'd against an index that hadn't taken its
  add yet, then the add re-inserted it: the index advertised an id the
  trigger had already deleted. recordAccepted now cancels the pending add
  instead of queueing a remove (added is a LinkedHashSet for O(1)
  cancel).

- A kind-5 that deleted nothing (a delete broadcast for events this relay
  never stored — the common case) still invalidated the whole index,
  forcing a full-scan rebuild under the writer mutex on the next
  NEG-OPEN. DeletionRequestModule.insert now returns the rows it deleted;
  recordAccepted only invalidates when that count is > 0, else records the
  kind-5 as a plain row.

- The first NEG-OPEN over a corpus larger than the serve cap scanned the
  whole table uncapped, built a full index that could never produce a
  snapshot, and then maintained it forever for zero benefit.
  liveNegentropySnapshot now caps the rebuild scan at maxEntries + 1 and
  leaves the index unpopulated when the corpus is over-cap (the scan path
  answers NEG-ERR, as before).

- delete/deleteExpired/clearDB invalidate() moved inside the writer mutex
  so no NEG-OPEN can seal a snapshot of just-deleted rows, and no
  concurrent rebuild can be discarded by a late invalidate.

Also corrects the ~40 B/event heap figure to ~140 B (IdAndTime keeps the
id as a 64-char hex string, not 32 bytes) in the strategy/plan docs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-04 02:31:40 +00:00
Claude 3d23f563b1 feat(geode): mirror directions — dir = "down" | "up" | "both" (strfry-router parity)
Each [[mirror]] entry now takes strfry-router's dir:

 - down (default): pull — subscribe to the upstream and ingest, exactly
   as before.
 - up: push — an in-process session on the LOCAL relay subscribes with
   the same scoped filter (so backfill_seconds and filter behave
   identically in both directions, and the relay's own policy chain
   gates what leaves), and every matching event is handed to the
   client's outbox for the upstream, which owns delivery and re-sends
   across reconnects.
 - both: pull and push, with echo suppression: a per-upstream LRU of
   recently exchanged ids keeps an event pulled down from being pushed
   straight back (and vice versa when the upstream fans our own publish
   back). Eviction only costs a duplicate round trip — the stores'
   unique-id constraints stay the correctness backstop.

trusted (verify skip) remains a down-only concept; the up direction
never verifies since the upstream does its own gatekeeping.

Tests: up pushes both the backfill window and the live tail; both
converges two stores with disjoint content and holds exact counts after
the echo settles (no ping-pong). Live-published test events are
genuinely signed — the local publish path verifies, which is also what
the debugging showed: the mechanism was fine, the first version of the
tests was pushing forged events into a verifying relay.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-04 01:32:18 +00:00
Claude cc73639ae5 docs(geode): SQLite knobs A/B verdict — no winner, knobs stay off by default
Backlog items 4-5 close. Both-variants-in-one-run A/B at 50k, repeated
with relay order reversed: every delta flipped with the order (the
second-running relay won queries and ingest latency in BOTH runs), so
readers=8 / mmap_size=256MiB / temp_store=MEMORY / periodic PRAGMA
optimize are all noise-level on container-class hardware. The config
plumbing stays (hardware-dependent, operators should measure their own
box); the example config now says so explicitly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-04 00:50:37 +00:00
Claude 79373672df feat(geode/quartz): [database] tuning knobs + periodic PRAGMA optimize
Backlog items 4-5 plumbing, config-gated and off by default so quartz
library defaults stay untouched for the app-side stores:

 - quartz: SQLiteEventStore/EventStore accept extraPragmas (applied on
   every pooled connection AFTER the built-in configuration, so they
   can override it) and expose optimize() — an analysis_limit-bounded
   PRAGMA optimize for incremental planner-stats refresh.
 - geode: [database] readers / mmap_size / temp_store_memory map onto
   the store; optimize_interval_seconds drives a maintenance coroutine
   (cancelled first in the shutdown hook, before the store closes).

The A/B verdict on whether the example config should RECOMMEND any of
these on container-class hardware follows in the next commit — the
knobs themselves are operator tools worth having either way, since
mmap/temp-store value is hardware-dependent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-04 00:48:22 +00:00
Claude fd6662ca30 perf(quartz): TCP_NODELAY for every relay websocket client — kills CLOSE→REQ Nagle stalls
Found while attributing the small-REQ wire floor (backlog item 6,
latency half): geode's new WireReqFloorBenchmark measured a flat
43.7 ms per REQ round trip that survived every server-side change —
store configs, dispatchers, the pump — and then vanished when the
round's preceding CLOSE was dropped. Root cause is client-side: OkHttp
does not set TCP_NODELAY, relays never answer a CLOSE (NIP-01), so its
bytes sit unACKed for the peer's ~40 ms delayed-ACK window and Nagle
holds the next REQ behind them. CLOSE-then-REQ is a Nostr client's
hottest pattern — every feed/filter switch.

relayBench's harness client already shipped a no-delay socket factory
(which is why benchmark numbers never showed the stall) but the
production clients did not. New TcpNoDelaySocketFactory (quartz
jvmAndroid, next to BasicOkHttpWebSocket) is now used by the Android
relay pool factory, the Desktop relay client, amy's relay connections,
and geode's mirror worker. Direct connections only — SOCKS/Tor paths
are untouched.

With the factory, the benchmark puts geode's ~21-row REQ at ~1.25 ms
on the wire (matching relayBench): ~0.6 ms Ktor CIO+OkHttp loopback
floor, ~0.5 ms per-REQ server work (already investigated). Per-frame
burst cost measured negligible and the pump adds ~nothing, so the
send-path latency angle of backlog item 6 is closed as not-a-problem;
its ingest-CPU share remains a separate throughput question. Findings
recorded in quartz/plans/2026-07-04-small-req-floor.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-04 00:38:09 +00:00
Claude f22657c870 revert(quartz): inline small-REQ fast path — no wire-level win, floor is transport-side
Reverts the queryRawInline fast path (fb29d655, 1b786f31) per the
keep-only-winners rule. Three relayBench runs at 50k (baseline, cap-256
where the path never engaged, cap-512 where author-archive/by-ids/
500-limit feeds genuinely took it) showed no movement outside the
container drift band — strfry's own numbers drifted ±30% between runs
and inline-eligible scenarios moved the same as ineligible ones.

The in-process win was real but small (~17%, 0.60 -> 0.50 ms per
~21-row REQ); the wire-level p50 is 1.2-1.7 ms, so the missing ~1 ms
per REQ sits in the transport (Ktor frame send path + client round
trip) — backlog item 6 territory, not dispatch. Findings, numbers, and
the do-not-retry note live in quartz/plans/2026-07-04-small-req-floor.md;
SmallReqFloorBenchmark stays as the measurement tool.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-04 00:03:34 +00:00
Claude 1b786f31b4 perf(quartz): broaden inline REQ eligibility — ids-bounded filters, cap 512
The first cut's rule (every filter needs limit, sum <= 256) missed the
shapes relays actually receive: relayBench's author-archive carries
limit=500 and by-ids carries only an ids list, so the fast path never
engaged in the acceptance run. Bounds now come from limit OR the ids
count (ids are unique keys, the result cannot exceed the list), summed
against a 512 cap — a 500-row inline replay measures single-digit ms,
nothing a CLOSE could meaningfully preempt. Provably unbounded filters
(no limit, no ids — e.g. a bare tag filter) keep the launched path.

Pending: relayBench verdict decides whether the fast path stays at all,
per the keep-only-winners rule.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-03 23:58:48 +00:00
Claude fb29d655fb perf(quartz): inline fast path for bounded small REQs
Backlog item 2 (small-REQ dispatch floor), first cut. relayBench at 50k
shows geode ~2.5x slower than strfry when a REQ returns ~20 events
(author-archive 1.18 vs 0.48 ms EOSE p50) while WINNING the 500-event
scenarios — with tiny results the fixed per-REQ cost dominates. A new
stage benchmark (SmallReqFloorBenchmark) decomposes the floor: at ~21
rows, raw SQL is 0.18 ms and the remaining ~0.42 ms is live-subscription
machinery plus per-REQ coroutine dispatch.

A REQ whose filters all carry a limit summing to <= 256 now runs its
stored replay inline on the receive coroutine and only retains a
live-tail handle (SessionBackend.queryRawInline; LiveEventStore's
queryRaw is re-expressed on the same core) — no per-REQ launch, no Job,
no dispatcher handoffs, no awaitCancellation scaffolding. Unbounded
REQs keep the launched path so CLOSE can always interrupt a giant
replay. In-process time-to-EOSE for ~21-row REQs drops 0.60 -> 0.50 ms;
the bigger effect expected under concurrency (no per-REQ Job+dispatch
churn) is relayBench's to judge.

InlineReqFastPathTest pins wire-behavior parity: stored-then-EOSE
ordering, live tail delivery and CLOSE detachment, same-subId
replacement, launched fallback for unbounded and over-cap REQs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-03 23:54:19 +00:00
Claude 0d6a9b1e4d docs(quartz): record strfry head-to-head for the live negentropy index
Same container, 50k corpus, geode --no-search, strfry built from
source: identical-set reconcile 41.4 ms (geode) vs 30.1 ms (strfry) —
down to ~1.4x from the campaign-opening full-scan-per-open; cold
reconcile 177 vs 112 ms (geode's first open pays the lazy rebuild);
ingest at parity in this container; storage 72.8 vs 106 MiB. Notes the
zero-copy IStorage follow-up that would close the remaining ~11 ms.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-03 23:15:09 +00:00
Claude de388907c2 docs(quartz): record live-negentropy-index A/B results — shipped
Two relayBench runs (50k corpus, both geode variants side by side, then
order-reversed): identical-set reconcile 56/57 ms with the index vs
110/130 ms without in both orders (~2.2x, the post-write NEG-OPEN the
old cache always missed); ingest and first-ever reconcile deltas flip
with relay order, i.e. run-order noise — no regression. Also records
the run-order-bias protocol note for future A/Bs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-03 23:09:56 +00:00
Claude 73ee5cce99 perf(quartz/geode): serve full-set NEG-OPENs from the live negentropy index
Milestones 2-3 of quartz/plans/2026-07-03-incremental-negentropy-storage.md.

SQLiteEventStore now maintains the LiveNegentropyIndex when the strategy
opts in (geode's does by default; [negentropy].live_index = false turns
it off; app-side stores are untouched):

 - Write paths collect a LiveIndexDelta and apply it after COMMIT while
   still holding the writer mutex — rolled-back savepoint rows never
   reach the index and updates land in exact commit order (also vs the
   rebuild, which runs under the same mutex).
 - Replaceable/addressable overwrites report the row their BEFORE-INSERT
   trigger displaces via one indexed pre-SELECT that mirrors the trigger
   predicate (including the NIP-01 lowest-id tie-break and the
   d_tag-NULL case).
 - Paths that can't itemize (kind-5, vanish, delete-by-filter/id,
   expiration sweeps, clearDB) invalidate; the next NEG-OPEN rebuilds
   from one scan on the writer connection.
 - Until that first NEG-OPEN populates the index, ingest pays zero
   bookkeeping — the populated check happens under the writer mutex so
   it can't race the rebuild.

LiveEventStore serves a single unconstrained filter (the relay-relay
sync default; relayBench sends exactly this) from the index; everything
else keeps the scan+seal path and its single-slot cache.

Micro-benchmark at 50k events (LiveNegentropyBenchmark, in-container):
scan+seal cold path 80-100 ms; index post-write open 9-16 ms (~5-10x).
The relayBench A/B is the acceptance gate and comes next.

Correctness: LiveNegentropyIndexStoreTest asserts index content ==
snapshotIdsForNegentropy scan after every mutation pattern (overwrites,
losers, kind-5 rebuilds, filter deletes, mixed-outcome batches,
transactions); the full geode suite (NIP-77 + interop sync tests) runs
with the index on.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-03 23:05:23 +00:00
Claude 7ad7dee5fa feat(quartz): LiveNegentropyIndex — always-current (created_at, id) set for NIP-77
Milestone 1 of quartz/plans/2026-07-03-incremental-negentropy-storage.md.

Sorted array with binary-search insert (near-tail in the common case),
itemized remove for displaced rows, wholesale invalidate for delete
paths that can't itemize, and sealed snapshots memoized per mutation
generation — reconcile only reads, so one snapshot backs any number of
concurrent sessions and stays immutable under later writes. Over-cap
answers null so the caller keeps the strfry-parity NEG-ERR.

Not wired into any store yet; next milestones plumb displaced-row
deltas from the SQLite modules and serve index-total filters from it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-03 22:41:35 +00:00
Claude 175cac3e46 docs(quartz): plan — incremental live negentropy storage
Design for backlog item 3 of the relay performance campaign: an
always-current (created_at, id) index maintained from the store's write
path so cold NEG-OPENs stop paying the full scan + O(n log n) seal
(~340 ms at 50k events vs strfry's ~21 ms off its live tree). Covers
the snapshot/COW model, the removal-correctness split (RETURNING deltas
for replaceable overwrites, wholesale invalidation for rare delete
paths), IndexingStrategy gating so app-side stores are untouched, and
the micro + relayBench A/B measurement plan.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-03 22:36:06 +00:00
Claude c2cabf3c47 feat(geode): mirror survives upstream restarts — retry pump, ping keepalive, e2e proof
Measured over the real OkHttp transport (upstream killed mid-mirror and
restarted on the same port): NostrClient re-dials once on disconnect and
then falls back to its 60s keep-alive, which showed up as a 61s mirror
blackout when the immediate re-dial raced the port rebind.

 - Retry pump: the worker nudges the pool every 5s. Cheap and safe —
   reconnectIfNeedsTo skips connected relays and each relay's
   exponential backoff (1s doubling, 5min cap) still gates real dial
   attempts, so dead upstreams aren't hammered. Restart recovery drops
   from 61s to ~6s in the new reconnect test.
 - WebSocket pings (120s, same as the Android relay pool): without
   them a half-open connection (network drop, no FIN) never fires
   onDisconnected and the mirror would stall silently forever.
 - MirrorWorkerReconnectTest: end-to-end over a real Ktor port — ride
   through the drop, re-subscribe, drop the duplicate replay, pull the
   event that only exists on the new instance.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-03 22:26:15 +00:00
Claude a2b38cd1a3 feat(geode): per-upstream [[mirror]] filters, strfry-router parity
Each [[mirror]] entry takes an optional NIP-01 filter as a JSON object
string, like strfry-router's per-stream filter. It is applied twice,
matching strfry's design (cmd_router.cpp):

 - it shapes the REQ sent upstream (the mirror still owns since via
   backfill_seconds and strips limit — the subscription is unbounded);
 - every delivered event is re-checked against it before ingest, so an
   upstream answering outside its REQ — including a trusted one whose
   events skip signature verification — can only inject events inside
   the operator-declared scope. Out-of-scope deliveries surface on a
   'filtered' counter.

Malformed filter JSON fails the boot, not the first delivery. Several
disjoint scopes for one upstream = repeat [[mirror]] with the same url.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-03 21:54:49 +00:00
Claude 50e259b495 feat(geode): [[mirror]] upstream streaming with relay-to-relay trust
Backlog item 1 of the relay performance campaign: skip signature
verification for events ingested from explicitly configured trusted
upstream relays, strfry-router style.

geode now dials each [[mirror]] url from the config, subscribes to
everything newer than now - backfill_seconds, and feeds the stream
through NostrServer.ingest (same group-commit writer + live fanout as
client publishes). The NostrClient underneath owns reconnects, backoff
and REQ re-sync; duplicate replays after a reconnect are rejected by
the store's unique-id constraint and only surface as counters.

trusted = true is the per-upstream trust switch: events from that
connection skip Schnorr verify. The trusted identity is the URL this
relay dialed (TLS-authenticated for wss://), never anything an inbound
peer claims. Default is false — mirror-but-verify — and a relay with no
[[mirror]] entries behaves exactly as before.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-03 21:22:46 +00:00
Claude 32b15d55d0 feat(quartz): NostrServer.ingest — local write path with per-submission verify skip
Adds a public local-ingest entry to NostrServer for events that don't
arrive over a client connection (mirror/sync workers, import jobs). It
routes through the same group-commit IngestQueue and live fanout as a
client publish, and each Submission can opt out of the parallel
Schnorr-verify hook — the relay-to-relay trust model, for events
streamed from an upstream that already verified them (verify profiles
at ~8% of busy ingest CPU).

Library defaults are unchanged: skipVerify defaults to false everywhere
and nothing in the client-publish path can set it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-03 21:22:31 +00:00
Vitor PamplonaandGitHub 4e895c20dd Merge pull request #3468 from vitorpamplona/claude/dedup-decode-benchmark-ci-rl7v8u
Make dedup decode benchmark resilient to CI load variance
2026-07-03 17:21:52 -04:00
Claude 130135250b fix: deflake DedupDecodeBenchmark timing assertion on loaded CI runners
A single-sample wall-clock speedup assertion (>1.5x) flakes when the
shared CI machine is loaded. Apply the same pattern already used by
ParallelVerifyBenchmark: retry up to 3 measurement attempts taking the
best, keep a hard 1.05x floor that a real regression (~1.0x when dedup
saves no work) can never pass, and warn instead of fail in the noise
band. The deterministic parsed/reused-count correctness gate still
fails hard on every attempt.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SzMix4ZivxoYrcn88TMqtF
2026-07-03 21:18:32 +00:00
David KasparandGitHub ae5e0ff8ed Merge pull request #3467 from vitorpamplona/l10n_crowdin_translations
New Crowdin Translations
2026-07-03 22:27:50 +02:00
vitorpamplonaandgithub-actions[bot] e2ab7ffd6a chore: sync Crowdin translations and seed translator npub placeholders 2026-07-03 19:59:42 +00:00
Vitor PamplonaandGitHub 55698947e4 Merge pull request #3466 from vitorpamplona/claude/relay-benchmark-strfry-geode-4v2wxq
Add relayBench: comprehensive Nostr relay benchmark suite
2026-07-03 15:57:40 -04:00
Claude 6760ba450c fix: FTS catch-up worker must not leak an uncaught exception at shutdown
The worker's writer-mutex fast path can slip past its own scope
cancellation and run one batch against a store that close() already
freed. The resulting closed-connection exception escaped scope.launch
under a SupervisorJob with no handler — harmless in production, but the
kotlinx-coroutines-test global handler attributes it to whatever runTest
starts next (seen on CI as EventSourceServerTest.countUsesSource failing
with UncaughtExceptionsBeforeTest). Catch, log, and stop the pass:
nothing depends on it — the pre-search drain covers search correctness.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NeoCvXnTxsKzqurkmjdC46
2026-07-03 19:22:43 +00:00
David KasparandGitHub c81fc8cb02 Merge pull request #3465 from vitorpamplona/l10n_crowdin_translations
New Crowdin Translations
2026-07-03 21:00:22 +02:00
Claude 35449c7437 fix: reclaim pool connections before close; strip NIP-50 extensions in queryRaw
Two issues surfaced by rebasing onto main:

- SQLiteConnectionPool.close() freed the writer's native handle without
  taking the writer mutex, so a block still running on another thread
  (the deferred-FTS catch-up worker's current batch outlives its scope
  cancellation) could call sqlite3_prepare on a freed sqlite3* — native
  heap corruption, SIGSEGV in sqlite3DbMallocRawNN (reproduced twice in
  :geode:test). close() now reclaims every reader from the channel and
  acquires the writer mutex before closing handles, then releases so
  stragglers get the managed closed-connection exception. Idempotent
  via a closed flag.

- main's NIP-50 fix (7deda28d) strips search-extension tokens before
  every store query, but the zero-decode queryRaw replay path added on
  this branch predates it and passed raw filters through — an
  extensions-only search would hit SQLite FTS as column syntax and
  error. queryRaw now strips like query/count/snapshot do.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NeoCvXnTxsKzqurkmjdC46
2026-07-03 18:51:38 +00:00
vitorpamplonaandgithub-actions[bot] 58a72c02fa chore: sync Crowdin translations and seed translator npub placeholders 2026-07-03 18:39:52 +00:00