The first cut appended :geode:test to the build-desktop matrix command.
That was wrong twice over: geode is JVM-only, so it ran 3× across the
ubuntu/macos/windows matrix, and — because org.gradle.parallel=true —
its default suite's CPU-heavy throughput benchmarks (a 1M-event mirror
sync, WireReqFloor, NegentropyServerReconcile) ran concurrently with the
timing-sensitive quartz relay-client tests, flaking
NostrClientReqBypassingRelayLimitsTest.denseSecondBeyondCapIsSteppedPastWithoutStalling.
Move :geode:test into its own test-geode job (needs: lint, ubuntu, JVM
21) so it runs once and its benchmark load can't starve another module's
timing assertions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KCdJwdhGtmLZ12ViS56S3k
geode was runnable only via ./gradlew :geode:run and was absent from CI.
Give it the same release process as the amy CLI (it's the same kind of
application-plugin JVM module), plus the pieces a long-running server
daemon needs that a one-shot CLI does not.
- Main.kt: add terminal --version/-V and --help/-h flags so a packaged
binary has a fast, exit-0 command (Homebrew test block, package smoke
checks, Docker healthcheck).
- build.gradle.kts: jlinkRuntime + geodeImage (portable flat app-image
with a bundled JRE, plus config.example.toml + geode.service under
share/) + jpackageDeb/jpackageRpm, mirroring cli/. No Compose to
exclude — geode depends only on :quartz.
- Dockerfile + .dockerignore: multi-stage image (gradle installDist ->
temurin JRE), the primary channel for relay operators.
- packaging/: systemd unit, macOS hardened-runtime entitlements, and a
reference Homebrew formula.
- scripts/asset-name.sh: geode_asset_name/collect_geode_assets under the
canonical geode-<version>-<family>-<arch>.<ext> scheme.
- create-release.yml: build-geode matrix (tarball + deb/rpm + no-JRE jvm
bundle, with a serve+NIP-11 smoke test of the jlink image) and a
docker-geode job pushing ghcr.io/<owner>/geode:<version> (+ :latest).
- bump-homebrew-geode-formula.yml: auto-sync the reference formula on
stable releases.
- build.yml: run :geode:test in CI (it ran in no workflow before).
- README.md + plans/2026-07-24-geode-release.md: operator docs + design.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KCdJwdhGtmLZ12ViS56S3k
Backlog items 4-5 close. Both-variants-in-one-run A/B at 50k, repeated
with relay order reversed: every delta flipped with the order (the
second-running relay won queries and ingest latency in BOTH runs), so
readers=8 / mmap_size=256MiB / temp_store=MEMORY / periodic PRAGMA
optimize are all noise-level on container-class hardware. The config
plumbing stays (hardware-dependent, operators should measure their own
box); the example config now says so explicitly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Audited all 143 plan files across the 10 plans/ folders. Each plan now
carries a Status header (shipped | in-progress | queued | abandoned)
backed by codebase evidence, and every folder has a README.md index
grouping plans by status.
Shipped plans were moved into a per-folder plans/archive/ (via git mv,
history preserved) so each plans/ folder surfaces only live work:
shipped (archived): 122 in-progress: 8 queued: 7 abandoned: 4
docs/plans/ is the frozen legacy folder; its plans were stamped and
indexed in place (48 of 52 archived) but it remains closed to new plans.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016hpUivtmq4pgzqRbY6MYrA
Ktor's WebSocket layer is unbounded by default, so [limits].max_ws_frame_bytes
was a strfry-parity tuning knob nobody actually used — drop the whole
LimitsSection from StaticConfig and the matching constructor param from
KtorRelay. The negentropy-large-corpus plan note is updated accordingly.
maxAdminBodyBytes is a real defense-in-depth cap (NIP-98 forces us to
read the body for the payload sha256 before we can authenticate, so an
unbounded read is a pre-auth DoS vector). Keep the cap, but stop
exposing it as an operator knob — it was never plumbed through to
StaticConfig and 1 MiB is ~1000× any plausible NIP-86 RPC payload.
Hardcoded at the Nip86HttpRoute construction site with the rationale
inline. Nip86HttpRoute.maxBodyBytes stays so tests can drive the 413
boundary.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rename RelayConfig → StaticConfig and the persistence/RelayStateStore →
config/RuntimeConfig so the two configuration tiers — boot-time TOML
versus operator-mutable runtime state — are obvious from the class
names. The `persistence/` package is gone; everything related to config
lives in `config/`.
For overlapping fields (NIP-11 info doc, pubkey / kind allow-deny
lists), StaticConfig now seeds RuntimeConfig on first boot via the new
RuntimeConfig(seed = …) constructor and an AuthorizationSeed type. The
runtime file wins from then on: later edits to [authorization] in the
TOML are ignored once an admin RPC has written the snapshot. As a
consequence, KindAllowDenyPolicy and PubkeyAllowDenyPolicy are no
longer stacked in geode's policy chain — BanListPolicy already reads
the same lists from the BanStore, and double-stacking would silently
diverge after the first admin mutation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Renames in the geode module to make naming honest:
- Relay -> RelayEngine (the transport-agnostic core)
- LocalRelayServer -> KtorRelay (Ktor/CIO transport; defaults to 127.0.0.1
but supports public deployment, so "Local" was misleading)
- RelayHub -> InProcessRelays (URL-keyed registry that doubles as a
WebsocketBuilder for in-JVM tests; "Hub"/"Pool" implied substitutable
resources, but each URL maps to a distinct relay)
Also moves the test-only helpers `preload` and `publish` out of
RelayEngine and into a testFixtures extension file. As part of that,
`publish` now waits for the relay's OK reply before returning so that
publish-then-subscribe is deterministic — previously the fire-and-forget
submit let a subscription register after publish returned but before the
ingest queue fanned out, causing ephemeral kinds (not persisted) to leak
to late subscribers. Fixes the flaky
`Nip01ComplianceTest.ephemeralEventIsNotStoredAndDoesNotShowOnFollowupReq`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Conflicts:
- quartz/.../LiveEventStore.kt: main added IngestQueue (group-commit)
and a fire-and-forget submit() path; this branch replaced the
SharedFlow live fanout with FilterIndex-driven dispatch. Resolved
by keeping main's submit/insert pipeline shape (IngestQueue ingest
ctor param, submit() callback, insert() wrapping submit via
CompletableDeferred) but routing the on-Accepted fanout through
FilterIndex.candidatesFor instead of newEventStream.tryEmit.
Added private fanout(event) helper. Kept main's
snapshotIdsForNegentropy method.
- geode/.../LoadBenchmark.kt: both sides added a new @Test method.
Kept fanoutScaling (this branch) and publishPipelinedSingleClient
(main).
Align the NIP-77 large-corpus plan with strfry's actual defaults:
500_000-byte frame cap (not 64 KB), no default since-window, 200
shared sub cap, max_sync_events overflow protection, exact NEG-ERR
wording, and 32-byte id storage. Add a strfry round-trip benchmark
and call out the BTreeLMDB pre-built tree as a follow-up.
Tier 3 used to say operators "must omit VerifyPolicy from their
policy chain" when parallelVerify is on — that turned out to be
the AUTH-verify regression caught in the audit. Updated the plan
to describe the real wiring: VerifyPolicy was split into a
parameterised base with two singletons, and composePolicy swaps in
VerifyAuthOnlyPolicy so AUTH commands keep signature verification
even when the IngestQueue takes EVENT verify.
Verifies the FilterIndex path in LiveEventStore: each event matches
exactly ONE of N subscribers, so without an index the relay walks
all N subs per event (O(N)); with the author-keyed index the lookup
is O(1) and per-event latency stays flat as N grows.
Different from fanoutLatency (broadcast: 1 event reaches every
sub). Here per-event work is lookup-bound rather than
delivery-bound. Configurable via -DfanoutScalingSubs (comma list)
and -DfanoutScalingEvents.
Measured (laptop, JDK 21, full sweep):
100 subs, 2k events: p50=1.35ms p99=5.71ms
1000 subs, 2k events: p50=1.05ms p99=3.05ms
5000 subs, 1k events: p50=1.01ms p99=2.96ms
p50 ~1ms across N — the predicted O(1) scaling. Default events count
adapts downward at high N to stay below WebSocketSessionPump's
8192-frame outbound cap (test-client read side is the bottleneck
above ~5k subs, not the relay).
Plan doc updated with the measured numbers in the "How to verify"
section.
Closes the last item on the event-ingestion-batching plan: signature
verification no longer serialises on each connection's WebSocket
pump. Instead, IngestQueue takes a `verify: ((Event) -> Boolean)?`
hook and fan-outs the per-batch verify across Dispatchers.Default
(`coroutineScope { events.map { async { verify(it) } }.awaitAll() }`)
before opening the SQLite transaction. Failed verifies pre-mark
Rejected and skip the insert.
Wiring:
- NostrServer takes `parallelVerify: Boolean = false` (opt-in to
preserve existing behaviour for direct library users).
- geode.Relay forwards a matching flag.
- Main.kt enables it whenever signature checking is on (config
`[options].parallel_verify = true`, default true), and when so,
composePolicy is told to skip VerifyPolicy from the chain to
avoid double-verifying every event.
- New CLI escape hatch `--no-parallel-verify` for the legacy path.
Bench: adds publishGroupCommitSingleClient (sequential publish-and-
confirm; 500 EPS regression floor for the synchronous path) — the
companion to the existing pipelined bench that exercises the
group-commit + parallel-verify wins.
Plan doc updated to describe what shipped (batchInsert + SAVEPOINTs
in Tier 1, IngestQueue mechanics in Tier 2, the verify hook in
Tier 3) and to drop the obsolete `synchronous=NORMAL` confirmation
note — the project ships `synchronous=OFF` and intentionally keeps
that.
Replace the SharedFlow-based broadcast in LiveEventStore with an
inverted index over filter-bearing subscribers. Each REQ registers
its filters into a per-store FilterIndex<LiveSubscription>; insert()
calls index.candidatesFor(event) and only delivers to candidates
whose Filter.match still passes. Cuts the per-event walk from
O(N_subs * N_filters) to a few hash lookups plus match() over a
small candidate set.
FilterIndex itself lives next to Filter.kt (commonMain, KMP-friendly,
AtomicReference + COW) so other call sites with the same shape
(LocalCache.observables, ObservableEventStore.changes) can reuse it.
Each filter contributes entries on its single most-selective dimension
(ids > authors > tags > tagsAll > kinds > unindexed) to keep buckets
narrow and avoid Set-dedupe work in candidatesFor.
The historical-replay race the previous SharedFlow + onSubscription
handoff closed is preserved by registering BEFORE replay starts and
deduping seen ids until EOSE.
OK frames carry the event id, so clients pair replies by id rather
than by arrival order. NIP-01 also treats OK true as "accepted," not
"fsynced." That removes two constraints the plan was carrying:
- no per-connection FIFO requirement on OKs
- no need to delay OKs until after batch fsync
Tier 1 can fan OKs out as soon as the per-row INSERT returns inside
the open transaction, hiding the group-commit fsync entirely from
publisher latency. Tier 2 drops the order-preserving commit log and
just sends OKs straight to outQueue. The pipelined benchmark now
checks one-OK-per-event-id rather than ordering.
Marks Sketches A and B done, with a note that A took the simpler
Channel.UNLIMITED + AtomicInteger cap path the original Risks section
called out, sidestepping the channel-swap that the plan first
sketched.
Records that the streaming-filter slice of Sketch C landed in Quartz,
and that the larger envelope-streaming work the plan called out is
unnecessary because MessageDeserializer / CommandDeserializer /
EventDeserializer were already streaming — only the filter sub-object
went through readTree.
Adds the verification benchmarks (connectionsHeldOpen10k,
connectionsHeldOpenWithFanout) to the verification section, with a
correction that what's measured is JVM heap not OS RSS.
Carries forward the not-done items (fan-out de-duplication,
filter-matching index, Netty engine) into the open-work section,
pointing at live-broadcast-fanout-index.md for the highest-leverage
remaining work.
Four sketches, queued by impact, each grounded in current code paths
and observed benchmark numbers:
- event-ingestion-batching: SQLite group commit + EVENT pipelining +
off-thread Schnorr verify. Targets 5–10× EPS on a fast SSD.
- live-broadcast-fanout-index: indexed filter matching to replace the
O(N_subs × N_filters) per-event walk in LiveEventStore. Targets
flat fanout p99 up to high subscriber counts.
- connection-scaling: shrink the per-session outQueue footprint
(currently the dominant per-conn cost), tune Ktor CIO group sizes,
reduce JSON parse allocations. Targets 10 000+ concurrent conns.
- negentropy-large-corpus: id-and-time-only snapshot path so NEG-OPEN
on a 5M-event store doesn't materialise full Event objects, plus
bounded-window defaults and concurrent-session caps.
Each plan names the verification benchmark to add. Plans are queued,
not committed work — README orders them by expected impact.
Claude
Vitor PamplonaandClaude Opus 4.7