Restructure `amy relay` from verb-first `relay add URL --type T` to noun-first
`relay <noun> <verb>`, matching amy's `marmot group …` / `cashu mint …`
convention. The relay-list type is now a required path segment (no implicit
default), and a bare noun lists that bucket.
NIP-65 (kind:10002) is fronted by two facet-nouns, `outbox` (write) and `inbox`
(read), replacing the `--marker` flag. They edit the single 10002 event and
apply the spec's merge rules:
- outbox add R on a read-only R → both
- inbox add R on a write-only R → both
- outbox remove R on a both-R → read (stays in inbox)
- inbox remove R on a both-R → write (stays in outbox)
- dropping the last facet removes R entirely
`relay nip65` shows the combined view; `nip65 remove`/`clear` edit the whole
event.
Other buckets are noun+verb: `relay dm|key-package|search|private|blocked|
trusted|proxy|indexer|broadcast|feeds <add|remove|set|clear|list>`. `set` needs
≥1 URL; `clear` empties. `relay add|remove URL` (no noun) stays as the
transport fan-out (nip65 both + dm + key-package).
BREAKING (cli --json/args): removes `relay add/remove/set --type T` and
`--marker`; `relay list` overview now keys nip65 as `outbox`/`inbox`/`nip65`
and the DM bucket as `dm` (was `inbox`). In-repo harnesses updated
(cache/dm/marmot setup drop `--type all`; cache T5 asserts `.dm`).
Verified end-to-end: merge semantics, encrypted NIP-51 round-trips, facet
set/clear, fan-out, aliases, and error paths.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EjHzNewJ2sfBGCcSwe35Mc
`relay set --type T` with no URLs is now rejected (bad_args, exit 2) instead
of silently wiping the list — a bare empty `set` is almost always a shell
variable that expanded to nothing. Emptying a bucket is explicit: pass
`--clear` (mutually exclusive with URLs).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EjHzNewJ2sfBGCcSwe35Mc
Expand `amy relay` from the 3 transport lists (nip65/inbox/key_package) to
every relay-list bucket Amethyst's relay-settings screen manages, and add
remove/set verbs alongside add/list.
Buckets (kind): nip65 (10002, read/write markers), inbox/dm (10050),
key_package (10051), search (10007), private (10013), blocked (10006),
trusted (10089), proxy (10087), indexer (10086), broadcast (10088),
feeds/favorites (10012). The private NIP-51 lists are signed NIP-44-encrypted
via the quartz event factories, exactly like the app. Local relays (device
pref, no event) and named relay sets (30002) are intentionally out of scope.
New/changed commands:
- `relay add URL --type T [--marker read|write|both]` — `--marker` sets the
nip65 role; `all` still means nip65+inbox+key_package.
- `relay remove URL --type T` — new.
- `relay set --type T [URL…] [--marker …]` — new; replace a whole bucket
(no URLs clears it).
- `relay list [--type T]` — lists every bucket, or one.
- `relay publish-lists` — now broadcasts every configured list.
Thin-assembly only: buckets are a small registry over the existing quartz
`create`/`relays` factories; adds one generic `Context.latestReplaceable`
helper. `--json` is additive — legacy keys (`nip65`/`inbox`/`key_package`,
`nip65_event_id`/…) are unchanged, so the existing test harnesses keep passing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EjHzNewJ2sfBGCcSwe35Mc
`putPending` stores each relay's pending map as
`Collections.synchronizedMap(LinkedHashMap(...))` and wraps every writer
path (`putPending`, `recordSent`, `recordIncoming`, `recordDisconnect`)
in `synchronized(perRelay)`. `sweep` iterated `pending.entries.iterator()`
without taking the same lock, violating the wrapper's Javadoc contract.
Any concurrent websocket-thread write during `RelayHealthStore.reclassify`'s
sweep threw `ConcurrentModificationException` on the underlying
`LinkedHashMap$LinkedHashIterator`. Because `reclassify` schedules sweep
on the AWT dispatcher, the CME killed Amethyst Desktop's Compose render
thread and froze the UI.
Wrap both inner iterator loops in `synchronized(pending) { ... }` — the
exact synchronization the wrapper's Javadoc prescribes for manual
iteration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
On Desktop, tapping a sidebar nav item while a detail screen (profile,
thread, article, editor) was open only mutated the sidebar destination.
The opaque `AnimatedContent` overlay driven by `ColumnNavigationState`
kept covering the (already-swapped) root content until the user hit
Back, creating the impression that the click did nothing.
Fix: emit a `clearOverlaySignal` from `SinglePaneState.navigate` and
`DeckState.focusExistingColumn`. Each layout collects the signal in a
`LaunchedEffect` and calls `navState.clear()`, draining any pending
detail stack so the tapped destination is what the user actually sees.
- SINGLE_PANE: one signal (Unit), one layout-local `navState`.
- DECK: signal payload is the column id; each `DeckColumnContainer`
filters on `column.id`, so only the focused column's detail clears —
other columns' navigation stacks are preserved.
- Same-item taps also clear (signal fires unconditionally, unlike a
StateFlow value comparison).
- `onOpenSettings` uses the same navigate / focusExistingColumn paths
and inherits the fix automatically.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Use the app's cached stringRes helper instead of raw stringResource,
matching the dominant convention in ui/note/types
- Drive the icon animation from the Compose frame clock (withFrameMillis)
instead of a delay loop, so the ticker suspends whenever the composition
stops drawing rather than waking the main dispatcher 4x/sec from the
back stack
- Drop the unconsumed memoryCardId/blockState/blockHash accessors; the
tag schema stays documented in the class KDoc
- Document the frames arrays as frozen: mutating them in place would
silently break the @Immutable skip contract; build a new instance
to change pixels
- Replace the API-29-deprecated Bitmap.createBitmap(IntArray, ...)
overload with createBitmap(w, h, config) + setPixels
Extend the showOnchainWallet preference to the zap flows, which also surface an
on-chain rail:
- the on-chain rail on each zap-amount chip in ZapAmountChoicePopup (folded into
the shared railCapability so every caller — reaction row, ReusableZapButton —
is covered)
- the "Send on-chain instead" hand-off button in the custom zap dialog
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UM57Rq5iUPL1SGhzhzTusa
Some users don't want the on-chain wallet surfaced. Add a `showOnchainWallet`
UI preference (default true, so behavior is unchanged) that hides it across the
app when turned off:
- the "Bitcoin" card on the Wallet screen (OnchainSection)
- the on-chain chip on profiles (DisplayPaymentRailChips)
- the on-chain rail in the Send Payment screen
The flag lives in the existing UiSettings/UiSettingsFlow display-preferences
system (persisted to the shared-settings DataStore, alongside the other
show/hide profile toggles) and is exposed as a switch on the Profile UI
settings screen.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UM57Rq5iUPL1SGhzhzTusa
A relay that refuses negentropy with a NEG-ERR whose reason merely starts
with "blocked" (e.g. "blocked: Negentropy sync is disabled" from a relay
that has NIP-77 turned off, or an auth/ban refusal) was misclassified as a
strfry `max_sync_events` overflow by `isOverflow`. Overflow triggers
created_at window-splitting, so every split re-opened, was refused again,
and the splitter fanned out breadth-first across the whole created_at range
(~2^31 windows). The call therefore never threw NegentropySyncException (so
`negentropySyncOrFetch` never took its paging fallback) and never tripped
the idle watchdog (the relay answered every NEG-OPEN promptly), so it hung
indefinitely. A second relay whose refusal string did not start with
"blocked" fell through to `Failed` -> paging and completed, which is why the
two behaved differently despite advertising the same NIPs.
Narrow `isOverflow` to genuine "result set too large" signals only; a bare
`blocked:` refusal now maps to a hard failure and fails over to paging.
Adds a regression test driving an in-process relay that refuses every
NEG-OPEN with "blocked: Negentropy sync is disabled" while still serving
plain REQ: negentropySyncOrFetch now pages and delivers every event instead
of hanging.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015UF3eh76rRiwAuPg32rwiz
The `import`/`export` engine is pure protocol/store logic — it operates only on
the `IEventStore` interface and Quartz event types (Event, OptimizedJsonMapper,
verify, Filter), with zero geode dependency — so per the sharing philosophy
("quartz = Nostr business logic, protocol, data") it belongs in Quartz, not in
the geode app. Any Quartz consumer (a relay, the `amy` CLI, a desktop
backup/restore) can now reuse it.
- move `com.vitorpamplona.geode.ImportExport` →
`com.vitorpamplona.quartz.nip01Core.store.NdjsonImportExport` (commonMain,
next to IEventStore); rename for a clear library-level name.
- geode keeps only the CLI glue (verb dispatch, arg parsing, file/stdin/stdout,
the stderr summary) in Main.kt, delegating to the Quartz engine.
- move the test into quartz jvmTest, rebuilt on Quartz's own EventFactory +
NostrSignerSync (real Schnorr signing) instead of geode fixtures.
No behavior change — `geode import`/`export` work exactly as before (verified
end-to-end previously); this is purely where the code lives.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Bulk NDJSON import/export as first-class geode subcommands, mirroring
`strfry import` / `strfry export` (one JSON event per line — the interchange
format for seeding a relay, migrating between relays, or taking a backup):
geode import [--db …] [--no-verify] [FILE…] # files, or stdin when none
geode export [--db …] # NDJSON to stdout
Both stream — memory is bounded to one batch (import) / one event (export), so
a multi-million-event corpus round-trips in roughly constant memory. `import`
verifies signatures by default (same `Event.verify()` the relay's VerifyPolicy
uses), upholding the relay's verify-by-default stance rather than trusting the
file; `--no-verify` is the trusted-input escape hatch. Verb dispatch is
backward-compatible: a bare `geode --port …` (no verb) still serves.
This makes the benchmark-only `CorpusServerMain` redundant — a corpus source is
now just `geode import` into a DB, then a normal `geode` serve — so it's
deleted, removing benchmark-only code from the production geode artifact (the
question that started this). The 1M sync-throughput plan is updated to describe
sources via `geode import` + serve.
Also fixes a native-target CI break: MergeQueryCorrectnessTest used the
deprecated `String(CharArray)` (error-level on Kotlin/Native) — switched to
`CharArray.concatToString()`.
Verified end-to-end through the packaged `geode` binary: import (file + stdin,
--no-verify), export round-trip, and verify-on rejecting bad signatures.
ImportExportTest covers the counts, duplicate handling, malformed-line
skipping, and verify accepting a freshly-signed event while rejecting bad sigs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Follow-up to the audit fixes so nothing describes the pre-fix behavior:
- CorpusServerMain: drop the leftover "reuses an already loaded DB … skips
the reload" comment above `val dbFile` — the sentinel-gated reuse it
described is now spelled out in the block just below it.
- sync-throughput-1m plan: the up-catch-up now streams `negentropyReconcile`
(publishing each onHaveIds batch) instead of materializing the full diff
via negentropyReconcileIds; note the O(batch) memory win at 1M.
- follow-feed plan: the k-way merge dedups repeated authors/kinds, and its
id-ASC tie-break is byte-exact vs the single-SQL path only when the store
indexes id (useAndIndexIdOnOrderBy) — otherwise ties fall in rowid order.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Two benchmark-integrity bugs that could invalidate sync numbers while a run
reports success:
- CorpusServerMain keyed its serve-existing decision only on port + row
count, so re-running a port with a different corpus/maxCount, or after a
load crashed mid-way, silently served a stale/partial DB. Gate reuse on a
completion sentinel keyed on corpus identity (path + byte length) and
maxCount, written only after a full load; on any mismatch the prior DB is
dropped and reloaded.
- CorpusDownloader treated CLOSED identically to EOSE, so a relay ending a
sub early (rate-limit/policy) after sending a partial page advanced the
cursor past the unsent tail — silent corpus loss. Treat CLOSED as a soft
failure (null → reconnect and retry the same cursor; the id dedup set
absorbs the re-fetch), distinct from EOSE which means the page is
complete.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
runCatchUpUp used negentropyReconcileIds, which builds the FULL need+have
id lists in memory even though the up direction only needs haveIds — on a
large window (e.g. 1M local events against an empty upstream) that is a
~100 MB+ heap spike per convergence round, plus a needIds list built and
immediately discarded.
Switch to the streaming negentropyReconcile: publish each haveIds batch as
it arrives (bounded to one batch of ids) and drop the need direction via a
no-op onNeedIds. The publish still suspends the reconcile round, so the
back-pressure and the reconcile-as-delivery-check convergence loop are
unchanged — only the peak memory drops from O(window) to O(batch).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
MergeQueryExecutor's winner-picker tie-breaks equal created_at by id ASC,
but each per-stream cursor sorted by created_at DESC only, and repeated
authors/kinds opened duplicate cursors:
- id tie-break: thread the IndexingStrategy through run()/prepareStreams
and append ", id ASC" to the per-stream ORDER BY when
useAndIndexIdOnOrderBy is set — matching every sibling query in
QueryBuilder. The id-indexed order comes straight off the index (no
extra sort, lazy cursor preserved), so the merge now matches the
single-SQL path byte-for-byte on same-second same-author events. Without
the id index the tie stays in rowid order (a valid NIP-01 newest-N);
documented on the class.
- dedup: streamCount/prepareStreams now operate on distinct authors and
kinds, so a filter with a repeated pubkey can no longer open two
identical cursors and emit each matching event twice (the single-SQL
IN(…) path already dedups).
Adds two MergeQueryCorrectnessTest cases the suite was missing: a
within-stream same-second tie sliced by the limit, and duplicate authors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
geode paged-REQ sink (verify on, FTS off):
geode→geode 4,963 ev/s (completes) strfry→geode 5,801 ev/s (completes)
Findings: strfry has no REQ bulk sync (stream is live-only limit:0), so the
strfry-sink REQ pairs are n/a. strfry→geode over REQ now completes (earlier
FTS-on stall was strfry killing the slow client at its 32MB pending cap; FTS-off
keeps pace). REQ is ~15-30% slower than negentropy for the geode sink, and
geode-as-a-REQ-source is slower than strfry (SQLite range scans vs LMDB).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
All four source→sink pairings via NIP-77, geode sink configured like strfry
(verify on, FTS off), run sequentially:
geode→geode 6,971 ev/s strfry→geode 6,690 ev/s
strfry→strfry 2,682 ev/s geode→strfry 2,127 ev/s
Key findings: the sink sets the rate (geode ~6.7-7.0k, strfry ~2.1-2.7k from
either source — geode ingests ~2.6-3.3x faster); full geode↔strfry negentropy
interop both directions; strfry-source count is lower because its negentropy
snapshot drops NIP-40-expired events (geode, no cutoff, offers its full set).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Reuse the file-backed source DB when it already holds events instead of
deleting + reloading the corpus every boot, so re-running a single sync pair
skips the multi-minute load.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
- MirrorSyncThroughputTest: add -DsyncVerify (default true — `strfry sync` always
verifies received events, so the negentropy sink verifies too for an
apples-to-apples comparison) and -DsyncFts (default true; pass false to match
strfry, which has no NIP-50). Both forwarded through the geode test task.
- CorpusServerMain: a benchmark-only tool that boots a real geode relay
(geode's default indexing) preloaded with an NDJSON corpus over a file-backed
store and serves forever, so `strfry sync` / another geode / the negentropy
sink can reconcile against a geode source holding the same 1M corpus a strfry
source does.
Used to run the 4-pair 1M negentropy sync comparison (geode↔geode, strfry→geode,
strfry↔strfry, geode→strfry).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
strfry's `sync --dir` is bidirectional (its source: doUp = both||up,
doDown = both||down), but geode's catch-up was down-only. Add the up half so
`dir=up`/`dir=both` reconcile-and-push matches `strfry sync --dir both`.
runCatchUpUp reconciles the local set against the upstream (negentropyReconcileIds)
and publishes the events we hold that the upstream lacks (the reconcile's `have`
ids). Symmetric to the down catch-up: same one `dir`, live up-session starts at
`now` when the up catch-up covers history.
Reliability: client.publish's outbox is best-effort under a bulk burst (each
publish also churns a reconnect — measured ~1-2% dropped per pass), so the push
runs as a reconcile→push convergence loop. Each round re-reconciles — the
reconcile IS the delivery check against the upstream — and re-pushes only the
stragglers until the have-diff is empty. Test observed 3000 → 69 → 2 → 0 across
3 rounds, lossless.
Test: MirrorNegentropyCatchUpTest.negentropyCatchUpPushesUp pushes 3000 local
events to an empty (no-verify) sink and asserts all 3000 land. The four existing
mirror tests still pass (up catch-up needs a store + negentropyBackfill, both off
by default).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
geode's MirrorWorker mirrored `strfry router` (live REQ streaming) but had no
`strfry sync` equivalent, so backfilling a large foreign relay from empty could
not complete: a plain REQ dump of the history overruns the sink and strfry kills
the slow client at its maxPendingOutboundBytes cap (see
relayBench/plans/2026-07-04-sync-throughput-1m.md).
MirrorWorker now runs a one-shot NIP-77 "sync" catch-up per down/both upstream
before the live tail, using strfry's own vocabulary — one `[[mirror]]` entry,
one `dir` driving both phases:
- Catch-up reconciles the local set against the upstream over the
[now - backfill_seconds, now] window and downloads only the diff via the
existing INostrClient.negentropySyncOrFetch — client-paced (strfry can't
overrun us) and it completes the pull. Reconcile-against-local means a warm
restart re-fetches nothing it already holds, like `strfry sync`.
- Either mode, transparently: negentropySyncOrFetch auto-falls back to paged
REQ for an upstream without NIP-77 — no config toggle.
- Live REQ tail unchanged; it starts at `now` when catch-up is on (history is
the sync's job). The windows overlap at `now`; the store's unique-id
constraint dedups the seam.
Changes:
- quartz: add a backward-compatible `localEntries` param to the public
negentropySync / negentropySyncOrFetch (default empty = prior behavior) so the
reconcile diffs against a caller-supplied local set.
- geode MirrorWorker: `runCatchUp()` (bounded, backpressured ingest; same
trusted-scope re-check as the live path; failure is non-fatal). New `store` +
`negentropyBackfill` ctor params; default off so existing live-REQ tests are
unchanged. Main opts production in.
- Test: MirrorNegentropyCatchUpTest isolates catch-up from the live tail by
preloading historical events a live-only sub cannot deliver, then proves the
post-boot event still arrives (3000 catch-up + 1 live = 3001).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Adds a NIP-77 negentropy client path to MirrorSyncThroughputTest (default for
external sources; `-DsyncMode=req` keeps the paged-REQ drain). geode reconciles
its empty set against strfry, then client-paced fetch-by-ids + ingest.
Result on the 1M damus.io corpus (strfry source):
- reconcile (empty local → 995,024 need-ids): 11.4 s, 64 rounds.
- fetch + ingest: ~7,000 ev/s steady-state.
- overall: 994,936 / 997,980 in 171.0 s => 5,818 ev/s — COMPLETES.
This is the apples-to-apples counterpart to strfry→strfry's `strfry sync`
(both negentropy, same corpus, empty→full): strfry→geode 5,818 ev/s vs
strfry→strfry ~2,550 ev/s — geode ingests real content ~2.3x faster and
finishes the pull. The paged-REQ path, by contrast, stalls at ~310k every time
(strfry kills a slow REQ client at its 32 MB maxPendingOutboundBytes cap),
confirming that cross-relay bulk sync from strfry requires negentropy, not REQ.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Adds a strfry→geode drain mode to MirrorSyncThroughputTest and documents the
three-way 1M-corpus sync throughput comparison.
Findings (relayBench/plans/2026-07-04-sync-throughput-1m.md):
- strfry→strfry (strfry sync / negentropy): ~2,550 ev/s, completes.
- geode→geode (MirrorWorker REQ): 13,161 ev/s, completes — but inflated by
synthetic 4-byte content; real-content ingest is ~7,000 ev/s.
- strfry→geode (paged REQ drain, real corpus): sustains ~7,000 ev/s but does
NOT finish — strfry hard-kills a slower REQ client at its 32 MB
maxPendingOutboundBytes cap (~310k events in; confirmed by the source log's
"Pending: 32.01M" disconnects). Independent of heap (2G/10G) and the live
negentropy index (on/off).
Conclusion: strfry's REQ serving is structurally hostile to any client slower
than its scan (buffers outbound, then kills or OOMs). Only NIP-77 negentropy —
pull/reconcile-based and client-paced — completes a cross-relay bulk pull, which
is why strfry's own sync uses it. A production geode backfill from a foreign
relay should use negentropy, not the live-tail REQ path.
Also documents MirrorWorker's unbounded intake channel (a deliberate live-tail
trade) OOMing under a 1M bulk backfill, and forwards the sync* system
properties through the geode test task.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
LiveNegentropyIndex kept a sorted ArrayList and paid an O(n) element shift per
incremental insert. That's cheap for near-tail live traffic (created_at ≈ now),
but a mirror/import backfill delivers historical, out-of-order events, so every
insert memmoves ~n/2 entries and the whole sync goes O(n^2) — a geode→geode 1M
mirror crawled to <300 ev/s once the index passed ~130k, versus a sustained
~20k ev/s with the index off.
When an insert lands more than REBUILD_THRESHOLD (4096) from the tail, drop the
index instead of shifting: it rebuilds in one O(n log n) scan on the next
NEG-OPEN (liveNegentropySnapshot already does this when unpopulated), and while
unpopulated newDeltaOrNull skips delta tracking, so backfill costs O(1) per
event. Near-tail live inserts keep the cheap incremental path. NIP-77
convergence and byte-exact tests still pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Adds -DsyncLiveIndex and a fast (no-live-index) source preload. Rate curves show
geode's mirror sustains ~20k ev/s with the live index off, but collapses to
O(n^2) with it on: the LiveNegentropyIndex is a sorted ArrayList whose per-event
insert is O(n), and a backfill delivers historical (non-near-tail) events, so
every insert memmoves ~n/2 entries.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Default EventStore(null) tokenizes FTS synchronously on every insert, which
dominates ingest and misrepresents mirror sync throughput. Use geode's actual
RelayIndexingStrategy (deferred FTS, live negentropy index) for both source and
sink, and log instantaneous events/s every 3s so the rate is visible during the
run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Two pieces measuring how fast each sink pulls a large corpus from a source using
its native sync client:
- relayBench/sync-throughput-strfry.sh: `strfry import` N events into a source
strfry, boot it, `strfry sync --dir down` an empty sink, report events/s.
- MirrorSyncThroughputTest: geode downstream pulls via the real MirrorWorker
(WebSocket). Default in-process geode source (geode→geode); with
-DsyncSourceUrl it mirrors an external relay (e.g. strfry) for strfry→geode.
Sized by -DsyncN.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
The ~19/40k event shortfall in the geode↔geode sync is not a geode event-loss
bug: geode is proven lossless across the store, the concurrent IngestQueue
pipeline, RelaySession (OK-true only post-commit), and the real MirrorWorker
WebSocket path (50k/50k). The shortfall is in the benchmark's hand-rolled
fetchByIds+publish delta transfer. Points the fix at the harness (or at driving
convergence through geode's real mirror) rather than geode.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Streams 50k events from an upstream KtorRelay to a downstream via the production
MirrorWorker (real OkHttp WebSocket + trusted skipVerify ingest) and asserts the
downstream receives every one — 50000/50000, 0 missing.
This closes the last untested layer: the in-process guards (BatchInsertLossTest,
ConcurrentIngestLossTest) call IngestQueue.submit directly, bypassing the wire.
With this, geode is proven lossless end-to-end — store, concurrent pipeline,
background pool contention, RelaySession, and the real WS mirror path. The
~0.05% shortfall seen in the geode↔geode relayBench sync is therefore an
artifact of the harness's hand-rolled fetchByIds+publish delta transfer, not a
geode event-loss bug.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Drives the full IngestQueue pipeline — parallel verify, greedy-drain group
commit, a concurrent deferred-FTS catch-up worker taking the pool writer, and
windowed concurrent submits via the trusted (skipVerify) mirror path — over the
clean 200k corpus, asserting every Accepted regular event is queryable after.
Passes (199,612 in/accepted/stored, 0 lost), together with BatchInsertLossTest
proving geode's ingest is lossless at every in-process layer. The geode↔geode
sync event-loss therefore lives above the store+queue — in the real Ktor
WebSocket path or the benchmark harness's hand-rolled delta transfer, which the
in-process paths bypass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Drives the exact store path — batchInsertEvents with geode's indexing strategy
in 64-event batches — over the clean 200k corpus and asserts every Accepted
kind-1 (regular, never replaced/deleted here) is queryable afterward. Passes
(199,612 in, 199,612 stored), which is the point: it proves the sequential
store path is lossless and narrows the geode↔geode sync event-loss to the
concurrent IngestQueue pipeline (async verify + greedy-drain batching +
concurrent WS submits), not the store itself.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Report, per sync pair: fetch coverage (did the peer REQ return every needed id,
without duplicates?), publish acks (accepted/rejected/unacked per target), and —
for any event a relay ends up missing — whether it was in the delta batch
delivered to that relay. In-batch-but-absent isolates ingest loss
(ack-without-persist) from a fetch/read gap, turning a vague "missing N" into a
pinned layer.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
The harness's reconcile reference only collapses replaceable kinds; it ignores
NIP-62 Request to Vanish, NIP-09 deletions, and NIP-40 expiration. So a
compliant relay (geode, which honors NIP-62) is falsely flagged "did not
converge" and the phantom events inflate its reconcile round count, masking
negentropy speedups at the wire. Documents the symptom (traced to one
ALL_RELAYS vanish pubkey in the damus corpus), the census (7 vanish pubkeys,
381 expiry events, 0 deletions in the first 200k), and the fix design for
whoever implements it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
When the identical-set reconcile shows a side missing events the reference set
has, log those events grouped by kind (plus a few samples with their tag keys)
instead of leaving a bare "did not converge". The harness already holds every
event by id, so it can resolve exactly what a relay dropped — turning a vague
verdict into an actionable ingest-semantics diagnosis (NIP-09 deletion / NIP-40
expiration / validation).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Op instances key Timestamp.ops (MutableMap<Op, Timestamp>), so contract
violations corrupt hash-map behavior:
- OpKECCAK256 defined equals without hashCode, so equal instances hashed
by identity — two equal keys could land in different buckets, producing
duplicate branches or failed lookups in keccak256 timestamp trees. Add
hashCode = TAG, mirroring OpSHA1/OpSHA256/OpRIPEMD160.
- OpBinary defined hashCode without equals — and its TAG referenced
Op.TAG (0x00), a no-op XOR. Define the equals/hashCode pair once on
OpBinary using tag() and drop the duplicated overrides from
OpAppend/OpPrepend (behavior unchanged: same tag + same arg content).
- VerifyResult.equals cast without a type test (ClassCastException on
foreign types instead of false) and hashCode force-cast the nullable
timestamp (NPE for null-timestamp results). Convert to a data class;
the custom toString and compareTo stay.
v1.2.0 (on Maven Central) speeds up the library's own reconcile and fingerprint
walk on top of the v1.1.1 PrefixSumStorageVector wiring. At the 1M relayBench
slice shape (NegentropyReconcileBenchmark, converges exactly, need/have=200k):
client reconcile 264 → 178 ms, seal 424 → 320 ms, and the library's O(range)
fingerprint walk 447 → 252 ms (~1.8×). Our prefix-sum path still answers each
range fingerprint in 0.7 ms (356× the now-faster walk). All NIP-77 tests pass.
Update the reconcile-profiling plan: the fix shipped via the upstream
IStorage.fingerprint seam (v1.1.1) rather than a quartz-side fast server; record
the v1.0.2 → v1.1.1 → v1.2.0 progression.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Vitor PamplonaandGitHub
Claude
nrobi144andClaude Opus 4.7
David KasparandGitHub
vitorpamplonaandgithub-actions[bot]