From cc73639ae5fc2daf4d1c53299737f2fd50a26a41 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 4 Jul 2026 00:50:37 +0000 Subject: [PATCH] =?UTF-8?q?docs(geode):=20SQLite=20knobs=20A/B=20verdict?= =?UTF-8?q?=20=E2=80=94=20no=20winner,=20knobs=20stay=20off=20by=20default?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Backlog items 4-5 close. Both-variants-in-one-run A/B at 50k, repeated with relay order reversed: every delta flipped with the order (the second-running relay won queries and ingest latency in BOTH runs), so readers=8 / mmap_size=256MiB / temp_store=MEMORY / periodic PRAGMA optimize are all noise-level on container-class hardware. The config plumbing stays (hardware-dependent, operators should measure their own box); the example config now says so explicitly. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A --- PLANS.md | 2 +- geode/config.example.toml | 5 +++ geode/plans/2026-07-04-sqlite-knobs-ab.md | 50 +++++++++++++++++++++++ geode/plans/README.md | 8 +++- 4 files changed, 63 insertions(+), 2 deletions(-) create mode 100644 geode/plans/2026-07-04-sqlite-knobs-ab.md diff --git a/PLANS.md b/PLANS.md index e3474caec3..1b8267abd5 100644 --- a/PLANS.md +++ b/PLANS.md @@ -38,7 +38,7 @@ listing (including archived plans), open that folder's `README.md`. | cli | 6 | 5 | 1 | 0 | 0 | [cli/plans](cli/plans/README.md) | | quic | 4 | 3 | 0 | 0 | 1 | [quic/plans](quic/plans/README.md) | | quic/interop | 1 | 1 | 0 | 0 | 0 | [quic/interop/plans](quic/interop/plans/README.md) | -| geode | 4 | 4 | 0 | 0 | 0 | [geode/plans](geode/plans/README.md) | +| geode | 5 | 4 | 0 | 0 | 0 | [geode/plans](geode/plans/README.md) | | docs (frozen) | 52 | 48 | 2 | 0 | 2 | [docs/plans](docs/plans/README.md) | ## Live work (not shipped) diff --git a/geode/config.example.toml b/geode/config.example.toml index 6c50f51848..17f15c8d31 100644 --- a/geode/config.example.toml +++ b/geode/config.example.toml @@ -46,6 +46,11 @@ path = "/" in_memory = false file = "/var/lib/geode/events.db" +# NOTE: the four knobs below measured as pure noise in relayBench A/B +# runs on 4-core container hardware at 50k events (see +# geode/plans/2026-07-04-sqlite-knobs-ab.md) — their value is +# hardware-dependent, so benchmark on YOUR box before enabling. +# # Reader-connection pool size (file-backed stores only). Default 4. # readers = 4 diff --git a/geode/plans/2026-07-04-sqlite-knobs-ab.md b/geode/plans/2026-07-04-sqlite-knobs-ab.md new file mode 100644 index 0000000000..b1d5146fae --- /dev/null +++ b/geode/plans/2026-07-04-sqlite-knobs-ab.md @@ -0,0 +1,50 @@ +# Server-side SQLite knobs — A/B verdict: no winner on container hardware + +**Status: closed (negative result recorded).** Backlog items 4–5 of the +relay performance campaign. + +## What was tested + +The knobs the campaign had previously reverted as noise-inconclusive, +re-tested under the A/B protocol — both variants in ONE relayBench run +(identical container conditions), then the run repeated with the relay +order reversed to expose order bias. 50k synthetic corpus, all four +knobs on the candidate at once: + +```toml +[database] +readers = 8 # reader pool (default 4) +mmap_size = 268435456 # 256 MiB +temp_store_memory = true +optimize_interval_seconds = 15 # PRAGMA optimize between ingest and queries +``` + +## Result + +Every observed delta flipped with run order. In BOTH runs the +second-running relay won most query scenarios and ingest latency — +regardless of which variant it was: + +- run 1 (plain → knobs): knobs won 7/10 query scenarios, better ingest + p99 (8.2 vs 16.0 ms). +- run 2 (knobs → plain): plain won 6/10 query scenarios, better ingest + p50/p99/throughput (8,552 vs 8,015 ev/s). + +Storage size identical (±1 MiB). The periodic `PRAGMA optimize` also +did not move the author-archive/planner-sensitive scenarios. + +## Verdict + +**All knobs stay off by default.** The `[database]` config plumbing +ships anyway (readers / mmap_size / temp_store_memory / +optimize_interval_seconds) because their value is hardware-dependent — +an operator on NVMe with a large page cache or a memory-constrained VPS +should measure on their own box — and `optimize_interval_seconds` +remains sensible operationally for long-running relays even without a +measurable win at 50k-fresh-database scale. + +**Do not re-benchmark these on container-class 4-core hardware** without +first fixing the run-order bias: the protocol note in +`quartz/plans/2026-07-03-incremental-negentropy-storage.md` applies — +same-run A/B plus order reversal is the minimum, and anything that +doesn't survive the order flip is noise. diff --git a/geode/plans/README.md b/geode/plans/README.md index 14807fb2a5..43c34bc5a1 100644 --- a/geode/plans/README.md +++ b/geode/plans/README.md @@ -1,6 +1,6 @@ # geode plans -_Audited 2026-06-30. 4 plans: 4 shipped (archived), 0 in-progress, 0 queued, 0 abandoned._ +_Audited 2026-06-30. 5 plans: 4 shipped (archived), 0 in-progress, 0 queued, 1 closed (negative result)._ Performance-focused design docs for future work. Each file is a self-contained sketch — problem statement, observed numbers, proposed @@ -26,3 +26,9 @@ regressions show up in the regular CI matrix once they're enabled. - [archive/2026-05-07-live-broadcast-fanout-index.md](archive/2026-05-07-live-broadcast-fanout-index.md) - [archive/2026-05-07-connection-scaling.md](archive/2026-05-07-connection-scaling.md) - [archive/2026-05-07-negentropy-large-corpus.md](archive/2026-05-07-negentropy-large-corpus.md) + +## Closed (negative result) + +| Plan | Verdict | +| ---- | ------- | +| [2026-07-04-sqlite-knobs-ab.md](2026-07-04-sqlite-knobs-ab.md) | readers/mmap/temp_store/PRAGMA-optimize knobs: no winner on container hardware; every delta flipped with run order. Knobs ship config-gated, off by default. |