Commit Graph
13 Commits
Author SHA1 Message Date
Claude bae2031cf3 fix(geode): validate config knobs and mirror filter at boot
Fail-loud on config that would silently degrade the running relay:

- readers = 0 makes every query hang forever on an empty reader pool;
  readers < 0 crashes with an unrelated message. optimize_interval_seconds
  <= 0 busy-loops PRAGMA optimize under the writer mutex. StaticConfig
  .validate() (called at boot) rejects both.

- A typo in [[mirror]].filter (e.g. `kindss`) parsed to an empty
  match-everything filter through the tolerant deserializer — silently
  widening a trusted upstream's skip-verify scope to the whole firehose.
  MirrorFilterValidator strict-checks the filter JSON at boot: unknown
  keys and non-array list fields fail startup.

- The self-mirror guard now compares scheme-insensitively (ws:// vs
  wss:// for the same host is still us) via displayUrl().

- The maintenance loop rethrows CancellationException and no longer
  swallows Errors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-04 02:32:35 +00:00
Claude 3d23f563b1 feat(geode): mirror directions — dir = "down" | "up" | "both" (strfry-router parity)
Each [[mirror]] entry now takes strfry-router's dir:

 - down (default): pull — subscribe to the upstream and ingest, exactly
   as before.
 - up: push — an in-process session on the LOCAL relay subscribes with
   the same scoped filter (so backfill_seconds and filter behave
   identically in both directions, and the relay's own policy chain
   gates what leaves), and every matching event is handed to the
   client's outbox for the upstream, which owns delivery and re-sends
   across reconnects.
 - both: pull and push, with echo suppression: a per-upstream LRU of
   recently exchanged ids keeps an event pulled down from being pushed
   straight back (and vice versa when the upstream fans our own publish
   back). Eviction only costs a duplicate round trip — the stores'
   unique-id constraints stay the correctness backstop.

trusted (verify skip) remains a down-only concept; the up direction
never verifies since the upstream does its own gatekeeping.

Tests: up pushes both the backfill window and the live tail; both
converges two stores with disjoint content and holds exact counts after
the echo settles (no ping-pong). Live-published test events are
genuinely signed — the local publish path verifies, which is also what
the debugging showed: the mechanism was fine, the first version of the
tests was pushing forged events into a verifying relay.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-04 01:32:18 +00:00
Claude cc73639ae5 docs(geode): SQLite knobs A/B verdict — no winner, knobs stay off by default
Backlog items 4-5 close. Both-variants-in-one-run A/B at 50k, repeated
with relay order reversed: every delta flipped with the order (the
second-running relay won queries and ingest latency in BOTH runs), so
readers=8 / mmap_size=256MiB / temp_store=MEMORY / periodic PRAGMA
optimize are all noise-level on container-class hardware. The config
plumbing stays (hardware-dependent, operators should measure their own
box); the example config now says so explicitly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-04 00:50:37 +00:00
Claude 79373672df feat(geode/quartz): [database] tuning knobs + periodic PRAGMA optimize
Backlog items 4-5 plumbing, config-gated and off by default so quartz
library defaults stay untouched for the app-side stores:

 - quartz: SQLiteEventStore/EventStore accept extraPragmas (applied on
   every pooled connection AFTER the built-in configuration, so they
   can override it) and expose optimize() — an analysis_limit-bounded
   PRAGMA optimize for incremental planner-stats refresh.
 - geode: [database] readers / mmap_size / temp_store_memory map onto
   the store; optimize_interval_seconds drives a maintenance coroutine
   (cancelled first in the shutdown hook, before the store closes).

The A/B verdict on whether the example config should RECOMMEND any of
these on container-class hardware follows in the next commit — the
knobs themselves are operator tools worth having either way, since
mmap/temp-store value is hardware-dependent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-04 00:48:22 +00:00
Claude a2b38cd1a3 feat(geode): per-upstream [[mirror]] filters, strfry-router parity
Each [[mirror]] entry takes an optional NIP-01 filter as a JSON object
string, like strfry-router's per-stream filter. It is applied twice,
matching strfry's design (cmd_router.cpp):

 - it shapes the REQ sent upstream (the mirror still owns since via
   backfill_seconds and strips limit — the subscription is unbounded);
 - every delivered event is re-checked against it before ingest, so an
   upstream answering outside its REQ — including a trusted one whose
   events skip signature verification — can only inject events inside
   the operator-declared scope. Out-of-scope deliveries surface on a
   'filtered' counter.

Malformed filter JSON fails the boot, not the first delivery. Several
disjoint scopes for one upstream = repeat [[mirror]] with the same url.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-03 21:54:49 +00:00
Claude 50e259b495 feat(geode): [[mirror]] upstream streaming with relay-to-relay trust
Backlog item 1 of the relay performance campaign: skip signature
verification for events ingested from explicitly configured trusted
upstream relays, strfry-router style.

geode now dials each [[mirror]] url from the config, subscribes to
everything newer than now - backfill_seconds, and feeds the stream
through NostrServer.ingest (same group-commit writer + live fanout as
client publishes). The NostrClient underneath owns reconnects, backoff
and REQ re-sync; duplicate replays after a reconnect are rejected by
the store's unique-id constraint and only surface as counters.

trusted = true is the per-upstream trust switch: events from that
connection skip Schnorr verify. The trusted identity is the URL this
relay dialed (TLS-authenticated for wss://), never anything an inbound
peer claims. Default is false — mirror-but-verify — and a relay with no
[[mirror]] entries behaves exactly as before.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
2026-07-03 21:22:46 +00:00
Claude 58c580cd34 feat(geode): --no-search / [options].full_text_search — run without NIP-50
strfry implements no NIP-50 at all, so geode's default setup pays FTS
tokenization on every searchable event for a feature the other side of a
benchmark isn't providing. The new switch removes it cleanly:

- geode: --no-search CLI flag and [options].full_text_search TOML key
  (default true). When off, the store skips the FTS index entirely,
  NIP-11 stops advertising 50 (an explicit [info] nips list stays
  operator-authoritative), and search filters match nothing via the
  existing QueryBuilder guard. RelayIndexingStrategy becomes
  relayIndexingStrategy(fullTextSearch) with the stock val kept.

- relayBench: --geode-no-search runs the geode entry with the flag (relay
  named geode-nosearch in reports); README documents the apples-to-apples
  rationale and a recipe for benchmarking both geode flavors side by side.

Alternating A/B on the 10k corpus: 4,767/4,792 ev/s without search vs
4,111/3,977 with — NIP-50 costs ~18% of ingest throughput at current
write-path speed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NeoCvXnTxsKzqurkmjdC46
2026-07-03 18:37:39 +00:00
Claude f7afc56752 feat(quartz): add optional NIP-42 AUTH relay policy
Introduce OptionalAuthPolicy, a relay-server policy that runs the full
NIP-42 challenge/verify handshake — emitting the AUTH challenge on connect
and recording verified pubkeys into the connection scope — but never
requires it: EVENT, REQ, and COUNT are always accepted, so clients that
ignore the challenge keep working.

It subclasses FullAuthPolicy and only relaxes the EVENT/REQ/COUNT gates, so
the authorize() hook and per-connection authenticatedUsers set behave
identically; downstream policies can still gate or rewrite on caller
identity. Wire it into geode via an optional_auth config option and a
--optional-auth CLI flag (ignored when require_auth/--auth is set, since
mandatory AUTH already sends the challenge).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BJWy4dBBYLbqthhLvbcZh7
2026-07-01 19:30:19 +00:00
Vitor PamplonaandClaude Opus 4.7 445fbf8cda refactor(quartz, geode): derive NIP-86 publicUrl from relay.url; drop isEnabled branch
Two simplifications:

1. Derive admin URL from RelayEngine.url. NIP-86 spec mandates that
   the admin endpoint is "the same URI as ws(s)://, called via
   http(s)://" — so KtorRelay derives the NIP-98 binding URL by
   calling relay.url.toHttp() instead of accepting publicUrl as a
   separate config. Single source of truth (info.relay_url),
   accidental misconfiguration impossible, no Host-header fallback.
   StaticConfig.AdminSection.public_url is gone.

2. Uniform code path for admin-enabled vs disabled. Empty allow-list
   isn't a special case anywhere: Nip86Server.isAuthorized returns
   false for everyone, dispatch rejects, Nip86HttpHandler returns
   NotAdmin → 403. KtorRelay always assembles the Nip86HttpRoute
   and registers the POST endpoint; Nip86Server.isEnabled() and the
   "is admin on?" branches in the handler and route are deleted.

Nip86EndToEndTest's admin-disabled case is now a behavioral test:
sign a valid token, expect 403 NotAdmin (was 405 with the no-route
variant, was 403 with the fake-disabled-route variant).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 18:19:24 -04:00
Vitor PamplonaandClaude Opus 4.7 f4bcf38caf refactor(geode): drop unused KtorRelay knobs (maxFrameBytes, maxAdminBodyBytes)
Ktor's WebSocket layer is unbounded by default, so [limits].max_ws_frame_bytes
was a strfry-parity tuning knob nobody actually used — drop the whole
LimitsSection from StaticConfig and the matching constructor param from
KtorRelay. The negentropy-large-corpus plan note is updated accordingly.

maxAdminBodyBytes is a real defense-in-depth cap (NIP-98 forces us to
read the body for the payload sha256 before we can authenticate, so an
unbounded read is a pre-auth DoS vector). Keep the cap, but stop
exposing it as an operator knob — it was never plumbed through to
StaticConfig and 1 MiB is ~1000× any plausible NIP-86 RPC payload.
Hardcoded at the Nip86HttpRoute construction site with the rationale
inline. Nip86HttpRoute.maxBodyBytes stays so tests can drive the 413
boundary.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:14:15 -04:00
Claude 289bc4bd5c perf(quartz): Tier 3 — parallel Schnorr verify in IngestQueue
Closes the last item on the event-ingestion-batching plan: signature
verification no longer serialises on each connection's WebSocket
pump. Instead, IngestQueue takes a `verify: ((Event) -> Boolean)?`
hook and fan-outs the per-batch verify across Dispatchers.Default
(`coroutineScope { events.map { async { verify(it) } }.awaitAll() }`)
before opening the SQLite transaction. Failed verifies pre-mark
Rejected and skip the insert.

Wiring:
- NostrServer takes `parallelVerify: Boolean = false` (opt-in to
  preserve existing behaviour for direct library users).
- geode.Relay forwards a matching flag.
- Main.kt enables it whenever signature checking is on (config
  `[options].parallel_verify = true`, default true), and when so,
  composePolicy is told to skip VerifyPolicy from the chain to
  avoid double-verifying every event.
- New CLI escape hatch `--no-parallel-verify` for the legacy path.

Bench: adds publishGroupCommitSingleClient (sequential publish-and-
confirm; 500 EPS regression floor for the synchronous path) — the
companion to the existing pipelined bench that exercises the
group-commit + parallel-verify wins.

Plan doc updated to describe what shipped (batchInsert + SAVEPOINTs
in Tier 1, IngestQueue mechanics in Tier 2, the verify hook in
Tier 3) and to drop the obsolete `synchronous=NORMAL` confirmation
note — the project ships `synchronous=OFF` and intentionally keeps
that.
2026-05-07 23:00:30 +00:00
Claude 0f7d8696de perf(geode): adaptive outQueue + CIO pool sizing for 10k+ connections
Implements geode/plans/2026-05-07-connection-scaling.md to push the
single-relay connection ceiling past the ~2k floor measured by
LoadBenchmark.connectionsHeldOpen.

- WebSocketSessionPump: switch outQueue to Channel.UNLIMITED bounded
  by an AtomicInteger backlog cap. Idle connections no longer reserve
  an 8 192-slot fixed buffer; lazy-allocated head segments cost only
  a few hundred bytes per idle session. Slow-client policy preserved:
  once outstanding > MAX_OUTGOING_BUFFER, the queue is closed and
  the connection drops, so NIP-01 ordering is never silently
  violated.
- LocalRelayServer / RelayConfig.NetworkSection: expose CIO
  connectionGroupSize / workerGroupSize / callGroupSize so big-VM
  operators can tune Ktor's event-loop pools without forking. Switch
  to the serverConfig {} + configure {} embeddedServer overload so
  CIO tunables can be set.
- LoadBenchmark: add connectionsHeldOpen10k (asserts 10 000 idle
  WebSockets settle under a 1 GiB heap ceiling) and
  connectionsHeldOpenWithFanout (5 000 subscribers, 10 EPS fan-out
  for 10 s, reports p50/p99 last-fanout latency).
- config.example.toml: document the three CIO tunables and the
  ulimit -n requirement for operators targeting >1k connections.
2026-05-07 20:12:50 +00:00
Claude c2f24a5213 refactor: rename quartz-relay module to geode
The relay implementation now stands as its own module. Fitting brand
for a Nostr relay shipped alongside the Quartz library — a geode is a
rock that holds quartz inside.

- Rename module directory quartz-relay/ -> geode/.
- Rename Gradle module path :quartz-relay -> :geode (settings.gradle).
- Rename Kotlin package com.vitorpamplona.quartz.relay -> com.vitorpamplona.geode.
- Rename application name + main class binding to match.
- Update the relay's NIP-11 advertised name to "geode" and the
  software URL to /tree/main/geode. Test asserting the doc updated.
- Refresh comments / Main.kt usage line / config.example.toml header.
- Update consumers: quartz/build.gradle.kts dependency path, and
  quartz NostrClient tests that import the in-process RelayHub.
2026-05-07 13:12:34 +00:00