BasicRelayClient collapsed a connection failure into a message string built
from the throwable's text alone. Message text is localized and inconsistent
across platforms, so a listener can't reliably tell a busy relay (a connect
timeout) from a dead one (bad domain / TLS misconfig) from it. Always append
the exception class name (SocketTimeoutException / UnknownHostException /
SSLHandshakeException / ConnectException …), which is stable, so listeners
can classify the failure by type. Message text is preserved; the type is
added in parentheses. Updated the one test that pinned the old format.
The GrapeRank engine, TrustGraph (compact int-CSR) and TrustGraphBuilder
are pure Nostr-social-graph computation over HexKeys — no UI, no Compose,
and no commons-only dependency. They're a utility for implementing the
NIP-85 rank assertions quartz already models, so they belong in quartz
rather than commons. Move commons/wot -> quartz experimental/graperank
(package com.vitorpamplona.quartz.experimental.graperank), including both
commonTest suites, and repoint the CLI import. TrustGraphBuilder was already
protocol-agnostic (takes HexKey lists; the caller does the event->edge
extraction), so nothing had to change but the package. Makes the algorithm
reusable by the Android app for spam/trust filtering without pulling in
commons.
Audit follow-ups before merge:
- amy fetch default limit is now the same on both paths: absent --limit → 100
for plain AND --paginate (previously --paginate silently meant "unbounded").
`--limit 0` is the explicit opt-in to drain everything (unbounded); negative
is rejected. The effective limit is carried on the filter so both paths agree.
- drainAllPages sizes its SeenIds for CLI-scale fetches (initialSlotsPow2 = 12,
~64 KB) instead of the large-walk default (~16 MB eagerly allocated per fetch);
it grows if an unbounded drain needs it.
- fetchAllPages clamps the inclusive advance to `min(pageMinTs, boundary)` so a
misbehaving relay that answers with an event past the requested `until` can't
push the cursor upward — the boundary dedup and termination rely on `until`
never increasing. No-op for honest relays (they only return events ≤ until).
Verified live: default and --paginate both cap at 100; --limit 50 → 50; --limit 0
--paginate drains the full window (>100); paging tests + SeenIds tests still pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015YEbdqCRPkszkGCoi89RMt
Two changes to the paginated fetch path:
- Cross-relay dedup before verify. drainAllPages' single consumer now runs a
SeenIds filter: the same widely-mirrored event arrives once per relay, and the
repeats are dropped BEFORE the expensive Schnorr verify + store instead of
after (they were only trimmed by FetchCommand's distinctBy). An id is marked
seen only once it verifies, so a forged copy (valid id, bad sig) delivered
first can't suppress the genuine one from another relay. Adds SeenIds.contains
(peek without recording) for that check-then-add.
- `amy fetch --paginate` no longer forces a --limit. With --limit N it still
pages up to N per relay; WITHOUT --limit it drains the whole filter unbounded
(the filter's null limit flows straight through). Plain (non-paginate) fetch
still trims to the default 100.
Verified live: unbounded --paginate over a ~20-min nos.lol firehose window
returns 406 (all unique, 3s) vs the old 100 cap; --limit 50 caps at 50; default
caps at 100; cross-relay fetch stays count==uniq.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015YEbdqCRPkszkGCoi89RMt
Drop @Synchronized from add/reset/size: SeenIds is now documented as
single-writer (not thread-safe). Callers dedup across concurrent relay
producers by funneling events into one consumer that owns the instance — the
one-consumer ingest pattern used elsewhere — which keeps a single global set,
stays lock-free, and lets resize run without coordination.
With the JVM-only @Synchronized gone the class is pure common Kotlin
(LongArray + Hex.readLong), so it moves from the jvmAndroid source set to
commonMain and is now available on every target.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015YEbdqCRPkszkGCoi89RMt
A run-scoped "already seen this id" filter for large, mostly-duplicate id
streams (a broad relay walk re-receiving the same event from many relays).
Keys on the first 128 bits of the id, sliced straight out of the hex with
Hex.readLong (table lookups, no parse, no allocation), in one open-addressed
LongArray — ~16 bytes/entry and the 64-char String is never retained, so tens
of millions of ids cost ~1 GB instead of a HashSet<String>'s ~6 GB. add() is
O(1) and synchronized.
Lives in the jvmAndroid source set (uses @Synchronized; a 40M-id walk is a
server-side concern). Ports the caller's implementation with the
parseUnsignedLong hot path swapped for Hex.readLong (~45-70 ns/op cheaper).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015YEbdqCRPkszkGCoi89RMt
fetchAllPages advanced with `until = oldest - 1` (exclusive) and no dedup. That
skips any event sharing the boundary second that didn't fit in the page — which
happens at *every* page boundary landing inside a second, not just pathological
dense ones — silently dropping events. An in-process probe with no second denser
than the relay's page cap still lost one event straddling the boundary.
Page inclusively now: `until = oldest created_at of the previous page`, and drop
the re-fetched boundary events by id. The dedup set is bounded to just the current
boundary second (`until` only decreases, so duplicates can only recur there), so
memory stays O(one second), never O(total).
A single second denser than the relay's page cap can't be drained (its tail is
unreachable — no client-side fix; raising the request limit is futile since we
already send one above the relay's cap). Once a page yields nothing new we step
strictly past that second so paging keeps progressing to older events instead of
stalling forever.
Tests: boundary-straddle retrieves all 6 (was 5); dense-second-beyond-cap steps
past without stalling and still delivers the neighbours. Verified on live relays
(strfry / nostr-rs-relay / khatru): ground-truthing each dense internal second
against the paginated set shows no gaps, incl. a 36-event second fully retrieved.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015YEbdqCRPkszkGCoi89RMt
The search-single-page logic left two lists with different roles: the listener
counted matches over the full `pagedFilters` (including a search filter already
dropped from paging) while the subscription only sent `remainingFilters`. That
worked — the dropped filter's count was unused and `advancesCursor` kept its
hits off the cursor — but it read as if a non-subscribed filter still mattered.
Collapse to one `activeFilters` list (index + filter) that is both what we
subscribe and what the listener iterates, so counting can't drift from what was
asked. Behavior is identical; the multi-filter and search tests still pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015YEbdqCRPkszkGCoi89RMt
Amy's one-shot queries all go through Context.drain, a single REQ drained to
EOSE — so a relay that caps its REQ response (strfry's per-REQ limit, ~500)
silently truncates the result with no way to page past it.
Extract the per-relay fetchAllPages fan-out that already lived privately in
EventSync into a reusable quartz accessory, fetchAllPagesFromPool: a
sliding-window pool (maxConcurrentRelays) that paginates each relay on its own
`until` cursor, tags every event with its source relay, and does not dedup
across relays. EventSync now delegates to it (its private downloadPool/
downloadFromRelay are deleted — no behavior change: perRelayFilters is already
ordered by and complete over the relay list).
Add Context.drainAllPages, the paged sibling of drain: same verify+store and
per-relay tagging, but fully draining sets larger than one REQ. Wire it into
`amy fetch` behind --paginate/--all (filter mode only), pushing the limit into
the filter so paging stays bounded. sync (NIP-77) and fetch stay separate
interfaces.
Tests: fetchAllPagesFromPool fan-out/tagging/no-cross-relay-dedup.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015YEbdqCRPkszkGCoi89RMt
NIP-50 search results are ranked by relevance, not created_at, so paging a
search filter by an `until` cursor silently degrades a top-N search into a
full time-walk of the corpus — and never terminates against a relay that
runs FTS over its whole corpus regardless of `until`.
fetchAllPages now queries a `search` filter on its first page only: it is
dropped from every later page and its hits neither advance nor drag back the
`until` cursor that co-resident non-search filters page with. onNewPage also
moves below the empty-page break so it never announces a page that isn't
fetched. Adds a test proving a search filter returns a single relay page
while a plain filter over the same capped relay still pages through the set.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015YEbdqCRPkszkGCoi89RMt
Add Hex.toLong64/toLong128/toLong256 (plus the shared readLong helper) to
pack the first 64, 128 or 256 bits of a hex string into a single Long, two
Longs or four Longs. Big-endian, allocation-light, branch-free — 16 table
lookups and shifts per word. Useful as cheap map/set keys or bucket hashes
for 32-byte event ids and pubkeys without decoding to a ByteArray.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019CU1wR6NvQmdmNNsPe9GuN
- Use the app's cached stringRes helper instead of raw stringResource,
matching the dominant convention in ui/note/types
- Drive the icon animation from the Compose frame clock (withFrameMillis)
instead of a delay loop, so the ticker suspends whenever the composition
stops drawing rather than waking the main dispatcher 4x/sec from the
back stack
- Drop the unconsumed memoryCardId/blockState/blockHash accessors; the
tag schema stays documented in the class KDoc
- Document the frames arrays as frozen: mutating them in place would
silently break the @Immutable skip contract; build a new instance
to change pixels
- Replace the API-29-deprecated Bitmap.createBitmap(IntArray, ...)
overload with createBitmap(w, h, config) + setPixels
A relay that refuses negentropy with a NEG-ERR whose reason merely starts
with "blocked" (e.g. "blocked: Negentropy sync is disabled" from a relay
that has NIP-77 turned off, or an auth/ban refusal) was misclassified as a
strfry `max_sync_events` overflow by `isOverflow`. Overflow triggers
created_at window-splitting, so every split re-opened, was refused again,
and the splitter fanned out breadth-first across the whole created_at range
(~2^31 windows). The call therefore never threw NegentropySyncException (so
`negentropySyncOrFetch` never took its paging fallback) and never tripped
the idle watchdog (the relay answered every NEG-OPEN promptly), so it hung
indefinitely. A second relay whose refusal string did not start with
"blocked" fell through to `Failed` -> paging and completed, which is why the
two behaved differently despite advertising the same NIPs.
Narrow `isOverflow` to genuine "result set too large" signals only; a bare
`blocked:` refusal now maps to a hard failure and fails over to paging.
Adds a regression test driving an in-process relay that refuses every
NEG-OPEN with "blocked: Negentropy sync is disabled" while still serving
plain REQ: negentropySyncOrFetch now pages and delivers every event instead
of hanging.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015UF3eh76rRiwAuPg32rwiz
The `import`/`export` engine is pure protocol/store logic — it operates only on
the `IEventStore` interface and Quartz event types (Event, OptimizedJsonMapper,
verify, Filter), with zero geode dependency — so per the sharing philosophy
("quartz = Nostr business logic, protocol, data") it belongs in Quartz, not in
the geode app. Any Quartz consumer (a relay, the `amy` CLI, a desktop
backup/restore) can now reuse it.
- move `com.vitorpamplona.geode.ImportExport` →
`com.vitorpamplona.quartz.nip01Core.store.NdjsonImportExport` (commonMain,
next to IEventStore); rename for a clear library-level name.
- geode keeps only the CLI glue (verb dispatch, arg parsing, file/stdin/stdout,
the stderr summary) in Main.kt, delegating to the Quartz engine.
- move the test into quartz jvmTest, rebuilt on Quartz's own EventFactory +
NostrSignerSync (real Schnorr signing) instead of geode fixtures.
No behavior change — `geode import`/`export` work exactly as before (verified
end-to-end previously); this is purely where the code lives.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Bulk NDJSON import/export as first-class geode subcommands, mirroring
`strfry import` / `strfry export` (one JSON event per line — the interchange
format for seeding a relay, migrating between relays, or taking a backup):
geode import [--db …] [--no-verify] [FILE…] # files, or stdin when none
geode export [--db …] # NDJSON to stdout
Both stream — memory is bounded to one batch (import) / one event (export), so
a multi-million-event corpus round-trips in roughly constant memory. `import`
verifies signatures by default (same `Event.verify()` the relay's VerifyPolicy
uses), upholding the relay's verify-by-default stance rather than trusting the
file; `--no-verify` is the trusted-input escape hatch. Verb dispatch is
backward-compatible: a bare `geode --port …` (no verb) still serves.
This makes the benchmark-only `CorpusServerMain` redundant — a corpus source is
now just `geode import` into a DB, then a normal `geode` serve — so it's
deleted, removing benchmark-only code from the production geode artifact (the
question that started this). The 1M sync-throughput plan is updated to describe
sources via `geode import` + serve.
Also fixes a native-target CI break: MergeQueryCorrectnessTest used the
deprecated `String(CharArray)` (error-level on Kotlin/Native) — switched to
`CharArray.concatToString()`.
Verified end-to-end through the packaged `geode` binary: import (file + stdin,
--no-verify), export round-trip, and verify-on rejecting bad signatures.
ImportExportTest covers the counts, duplicate handling, malformed-line
skipping, and verify accepting a freshly-signed event while rejecting bad sigs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
MergeQueryExecutor's winner-picker tie-breaks equal created_at by id ASC,
but each per-stream cursor sorted by created_at DESC only, and repeated
authors/kinds opened duplicate cursors:
- id tie-break: thread the IndexingStrategy through run()/prepareStreams
and append ", id ASC" to the per-stream ORDER BY when
useAndIndexIdOnOrderBy is set — matching every sibling query in
QueryBuilder. The id-indexed order comes straight off the index (no
extra sort, lazy cursor preserved), so the merge now matches the
single-SQL path byte-for-byte on same-second same-author events. Without
the id index the tie stays in rowid order (a valid NIP-01 newest-N);
documented on the class.
- dedup: streamCount/prepareStreams now operate on distinct authors and
kinds, so a filter with a repeated pubkey can no longer open two
identical cursors and emit each matching event twice (the single-SQL
IN(…) path already dedups).
Adds two MergeQueryCorrectnessTest cases the suite was missing: a
within-stream same-second tie sliced by the limit, and duplicate authors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
geode's MirrorWorker mirrored `strfry router` (live REQ streaming) but had no
`strfry sync` equivalent, so backfilling a large foreign relay from empty could
not complete: a plain REQ dump of the history overruns the sink and strfry kills
the slow client at its maxPendingOutboundBytes cap (see
relayBench/plans/2026-07-04-sync-throughput-1m.md).
MirrorWorker now runs a one-shot NIP-77 "sync" catch-up per down/both upstream
before the live tail, using strfry's own vocabulary — one `[[mirror]]` entry,
one `dir` driving both phases:
- Catch-up reconciles the local set against the upstream over the
[now - backfill_seconds, now] window and downloads only the diff via the
existing INostrClient.negentropySyncOrFetch — client-paced (strfry can't
overrun us) and it completes the pull. Reconcile-against-local means a warm
restart re-fetches nothing it already holds, like `strfry sync`.
- Either mode, transparently: negentropySyncOrFetch auto-falls back to paged
REQ for an upstream without NIP-77 — no config toggle.
- Live REQ tail unchanged; it starts at `now` when catch-up is on (history is
the sync's job). The windows overlap at `now`; the store's unique-id
constraint dedups the seam.
Changes:
- quartz: add a backward-compatible `localEntries` param to the public
negentropySync / negentropySyncOrFetch (default empty = prior behavior) so the
reconcile diffs against a caller-supplied local set.
- geode MirrorWorker: `runCatchUp()` (bounded, backpressured ingest; same
trusted-scope re-check as the live path; failure is non-fatal). New `store` +
`negentropyBackfill` ctor params; default off so existing live-REQ tests are
unchanged. Main opts production in.
- Test: MirrorNegentropyCatchUpTest isolates catch-up from the live tail by
preloading historical events a live-only sub cannot deliver, then proves the
post-boot event still arrives (3000 catch-up + 1 live = 3001).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
LiveNegentropyIndex kept a sorted ArrayList and paid an O(n) element shift per
incremental insert. That's cheap for near-tail live traffic (created_at ≈ now),
but a mirror/import backfill delivers historical, out-of-order events, so every
insert memmoves ~n/2 entries and the whole sync goes O(n^2) — a geode→geode 1M
mirror crawled to <300 ev/s once the index passed ~130k, versus a sustained
~20k ev/s with the index off.
When an insert lands more than REBUILD_THRESHOLD (4096) from the tail, drop the
index instead of shifting: it rebuilds in one O(n log n) scan on the next
NEG-OPEN (liveNegentropySnapshot already does this when unpopulated), and while
unpopulated newDeltaOrNull skips delta tracking, so backfill costs O(1) per
event. Near-tail live inserts keep the cheap incremental path. NIP-77
convergence and byte-exact tests still pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Drives the full IngestQueue pipeline — parallel verify, greedy-drain group
commit, a concurrent deferred-FTS catch-up worker taking the pool writer, and
windowed concurrent submits via the trusted (skipVerify) mirror path — over the
clean 200k corpus, asserting every Accepted regular event is queryable after.
Passes (199,612 in/accepted/stored, 0 lost), together with BatchInsertLossTest
proving geode's ingest is lossless at every in-process layer. The geode↔geode
sync event-loss therefore lives above the store+queue — in the real Ktor
WebSocket path or the benchmark harness's hand-rolled delta transfer, which the
in-process paths bypass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Drives the exact store path — batchInsertEvents with geode's indexing strategy
in 64-event batches — over the clean 200k corpus and asserts every Accepted
kind-1 (regular, never replaced/deleted here) is queryable afterward. Passes
(199,612 in, 199,612 stored), which is the point: it proves the sequential
store path is lossless and narrows the geode↔geode sync event-loss to the
concurrent IngestQueue pipeline (async verify + greedy-drain batching +
concurrent WS submits), not the store itself.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Op instances key Timestamp.ops (MutableMap<Op, Timestamp>), so contract
violations corrupt hash-map behavior:
- OpKECCAK256 defined equals without hashCode, so equal instances hashed
by identity — two equal keys could land in different buckets, producing
duplicate branches or failed lookups in keccak256 timestamp trees. Add
hashCode = TAG, mirroring OpSHA1/OpSHA256/OpRIPEMD160.
- OpBinary defined hashCode without equals — and its TAG referenced
Op.TAG (0x00), a no-op XOR. Define the equals/hashCode pair once on
OpBinary using tag() and drop the duplicated overrides from
OpAppend/OpPrepend (behavior unchanged: same tag + same arg content).
- VerifyResult.equals cast without a type test (ClassCastException on
foreign types instead of false) and hashCode force-cast the nullable
timestamp (NPE for null-timestamp results). Convert to a data class;
the custom toString and compareTo stay.
kmp-negentropy v1.1.1 adds the `IStorage.fingerprint(begin, end)` seam we needed
and ships `PrefixSumStorageVector` — a drop-in IStorage that builds an additive
prefix-sum table on seal() and answers any range fingerprint in O(1) instead of
re-walking the range. Range fingerprints are the CPU-bound core of a NEG-MSG on
a large snapshot; profiling pinned them as the steady-state reconcile cost geode
lost multiples on (not serialization).
Seal a `PrefixSumStorageVector` in both `NegentropyServerSession.sealVector`
(server / relay-relay responder, also backs the `LiveNegentropyIndex` snapshot
cache) and `NegentropySession` (initiator). Byte-identical to the plain vector —
only the fingerprint path is accelerated. `NegentropyPrefixFingerprintTest` now
also asserts the library's `PrefixSumStorageVector.fingerprint` matches the plain
walk over 2000 random ranges + boundaries at 50k; the reconcile-shaped mix
measures 601× (447 ms → 0.7 ms).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
The `rawQueryPathMatchesDecodedQuery` case called `store.rawQuery(filter)`, but
`EventStore` only exposes the streaming `rawQuery(filters, onEach)` — the
list-returning overload lives on the inner `SQLiteEventStore`. Point the check
at `store.store.rawQuery(filter)` so the zero-decode path is actually exercised.
Record the shipped k-way merge result in the plan doc: a fresh 1M relayBench run
has geode `follow-feed` at 18.8 ms vs strfry 17.7 ms (down from 97.7 ms, now at
parity) and 46,258 ev/s vs strfry 15,365 @8conn, both returning the same 500.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
The home-feed REQ (`authors=[…] (+ kinds=[…]) [+ since/until] limit=N`,
newest-first) is one of the most common relay queries. SQLite serves it by
seeking every `(kind, pubkey)` combo and feeding *all* matching rows through a
LIMIT-bounded sorter, so it reads O(the followed set's whole matching history)
— on a cold on-disk 1M corpus that was the `follow-feed` regression (relayBench:
97.7 ms vs strfry 17.6 ms).
Add `MergeQueryExecutor`, an app-level k-way merge that opens one lazy
newest-first cursor per stream off the existing composite indexes
(`query_by_kind_pubkey_created`, or `query_by_pubkey_created` for authors-only),
merges their heads `(created_at DESC, id ASC)` and stops at the limit — reading
only O(limit + streams) rows regardless of how much history the authors have.
It reuses indexes that already exist, so write throughput and on-disk size are
untouched. Eligibility is narrow (2..2048 streams, simple filter, explicit
limit, no ids/d-tags); everything else falls through to the single-SQL plan.
Wired into both `query` and the zero-decode `rawQuery` paths (the relay REQ hot
path) and the single-element filter-list variants, so `LiveEventStore` REQs go
through it.
`MergeQueryCorrectnessTest` proves the merge returns exactly the single-SQL
top-N — vs an independent Kotlin reference and vs the SQL path — across
distinct/tied created_at, since/until windows, authors-only, fewer-than-limit,
streaming onEach, and the raw path. `FollowFeedReadBenchmark` gains a `merge`
variant: at 1.05M events it's flat ~10-12 ms across both prolific-recent and
sparse-old, where `scan` is catastrophic on sparse follows (1995 ms) and
`current` is disk-bound on prolific ones.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
follow-feed (kinds=[1,6] × 150 authors, ORDER BY created_at DESC LIMIT 500)
was geode's 5.5× loss (97.7ms vs strfry 17.6ms). Investigated whether any
change is worth it, across read + write + size.
Read (FollowFeedReadBenchmark, in-memory, scale 5 ≈ 1.05M events):
prolific-recent sparse-old
current 5.7 ms 1.9 ms
scan (strfry) 1.0 ms 1601.9 ms
union 316.9 ms 20.0 ms
- scan (created_at index + early LIMIT) wins for active follows but is
catastrophic for sparse/inactive follows AND grows with corpus size
(234ms→1601ms from scale 1→5) — following rarely-posting accounts is
common, so it'd be a severe regression.
- union (300 per-branch subqueries) is dominated by branch overhead.
- current is the only robust option — flat across scale, bounded by the
followed set, never catastrophic. The 97.7ms is a worst case (the 150
MOST prolific authors, disk-bound reading all their matching rows).
No safe SQL-level swap exists; each alternative trades geode's worst case
for a worse one on a common workload. The only universal improvement is
strfry's app-level k-way merge (O(LIMIT+streams)) — a real new executor,
not a SQL tweak.
Write & size: neutral for every candidate — all reuse existing indexes
(query_by_kind_pubkey_created / query_by_created_at_id), none adds a
CREATE INDEX, so ingest throughput and storage are untouched regardless of
choice. A new index was considered and rejected (taxes every write, helps
one shape, reverts under ANALYZE).
Decision: keep the current composite plan. Full write-up in
quartz/plans/2026-07-04-follow-feed-read-tradeoff.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Answers 'could a new index beat the INDEXED BY pin without the pin?' for
the profiles shape (kind=0 AND pubkey IN(...) ORDER BY created_at DESC).
Measured:
- stock unhinted: scans query_by_kind_created (1.40ms) — the bug.
- pinned composite: seek + tiny sort (0.23ms) — the shipped fix.
- new (pubkey,kind,created_at) index, unhinted: STILL scans — no help.
- new (kind,pubkey,created_at ASC) index, unhinted: picks the seek (0.23ms)
BUT that's a no-stats cost-model artifact — ANALYZE reverts it to the
scan (1.47ms). Fragile, and a full duplicate of the DESC composite.
Root reason: ORDER BY created_at over a multi-value pubkey IN(...) needs a
sort no matter the index (no B-tree gives global created_at order across
pubkeys), and SQLite prefers the one sort-free plan — the full-kind scan.
Only the explicit pin reliably overrides that. A new index would add
write+storage cost on every event for the whole relay, help only this one
shape, and break under ANALYZE — so the free, deterministic, scoped pin is
strictly better. Diagnostic evidence for the design choice.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
The server's per-round reconcile spends a large slice turning the ~1MB hex
reconcile frame into wire JSON: the generic (Jackson) serializer wraps the
hex string in a value node and scans every char for JSON escapes a
[0-9a-f] payload can never contain, then re-copies.
NegMsgMessage.toJson() now builds ["NEG-MSG","<sub>","<hex>"] directly —
no node tree, no escape scan of the hex. Fast path fires only for
escape-free printable-ASCII subIds (what the JSON encoder emits verbatim);
exotic subIds fall back to the generic serializer, so output is
byte-identical. RelaySession.send routes through message.toJson() (default
unchanged for every other message type).
Measured (toJson + UTF-8, per frame): 64KiB 2.5×, 250KiB 2.6×, 500KiB
(strfry cap) 2.8× — ~2.5ms saved per NEG-MSG, ~35ms over a 14-round
reconcile. Correctness: a subId battery asserts byte-identity with the
generic path, and GeodeVsStrfryNegentropySyncTest (real strfry) reconciles
against the fast-built frames.
Also records the ingest-latency candidate as measured-not-worth-it: the
IngestQueue pipeline overhead is only ~0.17ms p50, <10% of the ~2.4ms
receipt→queryable gap — that gap lives in the REQ-visibility path, not the
writer. Full write-up in
quartz/plans/2026-07-04-sync-serialization-and-ingest-latency.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
Measurement harnesses for the two remaining relayBench gaps, isolating each
cost so a fix can be judged on the delta:
- IngestLatencyBenchmark: times single-event submit→onComplete through the
group-commit IngestQueue vs a direct batchInsert. The delta is the
pipeline's coroutine-handoff overhead — the receipt→queryable latency the
1M run measured geode losing (4.68ms vs strfry 2.32ms).
- NegMsgSerializationBenchmark: times the NEG-MSG wire path (Hex.encode →
MessageKSerializer JsonElement tree → UTF-8) against a direct StringBuilder
build, asserting byte-identical output. Isolates the ~40% serialization
slice of the server reconcile the JFR flagged.
Both compile and are self-contained; results + fixes to follow.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
After the profiles fix, sweep every relayBench query shape with EXPLAIN
QUERY PLAN against a realistic multi-kind, tagged store to check for other
scan-that-should-seek mis-costs. Result: clean — every scenario seeks an
index (or does an index scan + LIMIT, like firehose's created_at walk).
The remaining TEMP B-TREE sorts are all over bounded result sets
(author/id count or LIMIT), not the profiles-scale pathology. The tag
queries (thread/notifications/hashtag) do an inherent subquery + sort;
hashtag is the slowest scenario but strfry is equally slow there, so it's
absolute cost, not a competitive gap.
Kept as a regression guard: asserts no scenario full-table-scans a base
table (bare SCAN without USING INDEX).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
The profiles scenario — Filter(kinds=[0], authors=[50]), no limit — was
geode's ~100x loss to strfry on the 1M corpus (99.5ms vs 0.94ms).
Root cause: the REQ path always appends ORDER BY created_at DESC. For a
multi-author pubkey IN(...) filter, query_by_kind_created (kind,
created_at) satisfies that order for free by scanning an ENTIRE kind, so
SQLite prefers it over the selective query_by_kind_pubkey_created (which
would need a sort). The scan is O(all kind-0 profiles) — cheap at 2k, the
99.5ms at 1M. ANALYZE does not fix it (verified: even a reopened store
reading fresh sqlite_stat1 keeps the scan, since the ORDER BY genuinely
lets the scan skip a sort). A single author is costed right and already
seeks; only the IN-list of >1 is mis-costed.
Fix: pin INDEXED BY query_by_kind_pubkey_created for exactly that shape —
multi-author + kinds, no ids, no d-tags, no limit — keeping the ORDER BY.
SQLite seeks the authors and sorts the small result: identical rows,
identical newest-first order (zero behavior change), ~8x at 2k profiles,
growing to ~100x at 1M. Limited feeds (home/global) keep the created_at
scan + early LIMIT; single-author and d-tag queries are untouched. The
index is created unconditionally so the hint never dangles.
(Considered dropping the ORDER BY for no-limit author queries — faster and
hint-free, but it changes on-the-wire result ordering, which broke
FsParityTest's ordered-parity assertions, so it's client-visible. Rejected
in favor of the order-preserving pin.)
Adds ProfilesQueryPlanBenchmark (regression guard: asserts the live REQ
plan seeks the composite index, not the kind scan) and
plans/2026-07-04-profiles-query-plan.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
- only parse commonName when the alt tag has the Birdstar prefix
- summary() on both Birdstar events now uses the canonical NIP-31
tags.alt() helper instead of a raw firstTagValue("alt") lookup
- speciesReference() only returns http(s) URLs since UIs render it as
a clickable link (rejects e.g. javascript: schemes), with a test
- Detection card: parse tags once into a single remember slot, drop
the near-dead '?: summary' title fallback, stop rebuilding the
italic TextStyle every recomposition
- Hoist the duplicated bird-emoji literal into a shared BIRD_PREFIX
- Trim the redundant factory test to the assertIs idiom
- fetch Birdex life lists in home and profile relay REQs
- richer Birdstar cards — common name title, Wikidata link, bird emoji
- surface Birdstar bird detections in home and profile feeds
profiles (Filter(kinds=[0], authors=[50]), no limit) was the one query
geode lost to strfry on the 1M corpus — 99.5ms vs 0.94ms (~100x).
Root cause: the REQ path appends ORDER BY created_at DESC even with no
limit. query_by_kind_created (kind, created_at) satisfies that order for
free while scanning EVERY kind-0 profile, so SQLite prefers it over the
ideal query_by_kind_pubkey_created (which would need a sort). The scan is
O(all profiles) — cheap at 2k, the 99.5ms at 1M.
ANALYZE does not help: verified that even a reopened store reading fresh
sqlite_stat1 keeps the scan, because the ORDER BY genuinely lets the scan
avoid a sort.
Two fixes measured (both return identical rows), scoped to no-limit
kinds+authors filters:
- Fix A: drop ORDER BY when limit==null -> planner picks the composite
index itself (~7x at 2k profiles, ~100x at 1M). Changes result order
across authors (a NIP-01 SHOULD; clients re-sort).
- Fix B: force INDEXED BY query_by_kind_pubkey_created + keep ORDER BY
(~same speed, newest-first preserved, at the cost of a scoped hint).
Adds ProfilesQueryPlanBenchmark (prints plans+timings, asserts row-count
equivalence) and plans/2026-07-04-profiles-query-plan.md. No production
change yet — the fix is a core QueryBuilder behavior/ordering decision.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
On connect, syncFilters re-sends every desired REQ through
PoolRequests.syncState. It previously sent the frame and only recorded
the subscription as SENT afterward, in the post-send onSent callback. A
relay that answers faster than that callback runs — the in-process
transport used by the desktop launch-optimization tests, or any relay on
a fast path — can deliver the EOSE while the per-sub state still reads
"nothing in flight" (onConnecting cleared it, onSent hasn't recorded it).
The EOSE handler then sees empty filters, concludes it never sent a REQ,
and fires a duplicate, replaying the whole page a second time.
Pre-mark the sub as SENT under its lock before the frame leaves, mirroring
the decideCommandLocked pre-mark already used by sendToRelayIfChanged, so
a response can never race ahead of the record. The send stays
unconditional: this is a fresh-connection sync (onConnecting always
cleared the per-relay state first), so there is no in-flight REQ on the
new socket to dedupe against.
Fixes the flaky SubscribeBeforeConnectTest, which asserted a pre-connect
subscription delivers exactly its events once and intermittently saw them
doubled.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4kmtZoNUSXxwG23wD2JwP
The 1M relayBench run had geode losing the negentropy phase to strfry
(initial reconcile 6066ms/27r vs 1270ms/14r; identical-set 1947ms vs
557ms). Three layered benchmarks pin where the time actually goes:
- NegentropyReconcileBenchmark (quartz): the kmp-negentropy server loop
in isolation is ~200ms for the full 14-round exchange — the reconcile
ALGORITHM is not the bottleneck. (An early version showed 22s/139r;
that was a benchmark bug — index slices over randomly-sorted ids
scatter the diff. Real relayBench slices are contiguous time ranges;
monotonic created_at fixes it and matches strfry's round count.)
- NegentropyServerReconcileBenchmark (geode): the real in-process geode
server over loopback is 3214ms — 15x the library loop. JFR of the
server call-trees: ~40% hex/UTF-8/JSON serialization of the payloads,
~26% actual reconcile, rest allocation. The gap is the JVM
constant-factor tax on hex-in-JSON, which strfry pays in C++, not a
single hotspot.
- NegentropyPrefixFingerprintTest (quartz): the one algorithmic lever.
Negentropy's fingerprint is an additive sum mod 2^256, so a prefix-sum
table answers any range in O(1). Proven bit-for-bit identical to the
library over 2000 random ranges, and 460x faster per call — the fix
for the ~26% reconcile slice (dominant in the identical-set case).
Not yet wired: the library instantiates FingerprintCalculator
internally, so shipping prefix-sum needs a kmp-negentropy change (or a
quartz-side fast server). Full write-up + artifacts in
quartz/plans/2026-07-04-negentropy-reconcile-profiling.md.
Benchmarks are CI-safe (small defaults / opt-in gates); JFR via
-PnegProfile, scale via -DnegBenchN, geode server bench via
-DnegServerBench=1.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EZeWww5TJnzBZKPoc6mvU
TcpNoDelaySocketFactory's connecting overloads used
`socket().apply { connect(...) }`, which leaks the file descriptor if
bind/connect throws (the JDK's connecting Socket constructors close on
failure; ours didn't). Wrapped in a helper that closes on throw. OkHttp
only calls the no-arg overload, so this guards any other direct caller.
DesktopHttpClient's pre-init `simpleClient` (direct relay sockets opened
before setInstance) now gets the same TcpNoDelaySocketFactory as
directClient. failClosedClient is left as-is: it's a SOCKS client and
OkHttp bypasses the socket factory for SOCKS proxies.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
ingest() bypasses the per-connection policy chain, which is where
VerifyPolicy lives. The IngestQueue verify hook only exists when
parallelVerify is true, so with parallelVerify = false and
skipVerify = false, ingest() previously verified nothing — an untrusted
mirror upstream on a relay running the legacy in-policy verify path could
inject forgeries. ingest() now verifies inline in that configuration
(same rejection reason as the queue), so the documented "default keeps
verify-everything semantics" holds regardless of parallelVerify. KDoc
also spells out that ingest() skips the entire policy chain (blacklists,
size limits), which callers must screen for themselves.
Test covers the parallelVerify = false server: forged rejected, trusted
skip and valid still land.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Audit follow-ups on the live NIP-77 index, all with the store/scan
equivalence test extended to cover them:
- Same-batch replaceable displacement left a dead id in the index.
applyAfterCommit applied all removes before all adds, so when a later
row in one transaction displaced an earlier row of the same batch (two
versions of one replaceable — the mirror-backfill hot path), the
displaced row's remove no-op'd against an index that hadn't taken its
add yet, then the add re-inserted it: the index advertised an id the
trigger had already deleted. recordAccepted now cancels the pending add
instead of queueing a remove (added is a LinkedHashSet for O(1)
cancel).
- A kind-5 that deleted nothing (a delete broadcast for events this relay
never stored — the common case) still invalidated the whole index,
forcing a full-scan rebuild under the writer mutex on the next
NEG-OPEN. DeletionRequestModule.insert now returns the rows it deleted;
recordAccepted only invalidates when that count is > 0, else records the
kind-5 as a plain row.
- The first NEG-OPEN over a corpus larger than the serve cap scanned the
whole table uncapped, built a full index that could never produce a
snapshot, and then maintained it forever for zero benefit.
liveNegentropySnapshot now caps the rebuild scan at maxEntries + 1 and
leaves the index unpopulated when the corpus is over-cap (the scan path
answers NEG-ERR, as before).
- delete/deleteExpired/clearDB invalidate() moved inside the writer mutex
so no NEG-OPEN can seal a snapshot of just-deleted rows, and no
concurrent rebuild can be discarded by a late invalidate.
Also corrects the ~40 B/event heap figure to ~140 B (IdAndTime keeps the
id as a 64-char hex string, not 32 bytes) in the strategy/plan docs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Backlog items 4-5 plumbing, config-gated and off by default so quartz
library defaults stay untouched for the app-side stores:
- quartz: SQLiteEventStore/EventStore accept extraPragmas (applied on
every pooled connection AFTER the built-in configuration, so they
can override it) and expose optimize() — an analysis_limit-bounded
PRAGMA optimize for incremental planner-stats refresh.
- geode: [database] readers / mmap_size / temp_store_memory map onto
the store; optimize_interval_seconds drives a maintenance coroutine
(cancelled first in the shutdown hook, before the store closes).
The A/B verdict on whether the example config should RECOMMEND any of
these on container-class hardware follows in the next commit — the
knobs themselves are operator tools worth having either way, since
mmap/temp-store value is hardware-dependent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Found while attributing the small-REQ wire floor (backlog item 6,
latency half): geode's new WireReqFloorBenchmark measured a flat
43.7 ms per REQ round trip that survived every server-side change —
store configs, dispatchers, the pump — and then vanished when the
round's preceding CLOSE was dropped. Root cause is client-side: OkHttp
does not set TCP_NODELAY, relays never answer a CLOSE (NIP-01), so its
bytes sit unACKed for the peer's ~40 ms delayed-ACK window and Nagle
holds the next REQ behind them. CLOSE-then-REQ is a Nostr client's
hottest pattern — every feed/filter switch.
relayBench's harness client already shipped a no-delay socket factory
(which is why benchmark numbers never showed the stall) but the
production clients did not. New TcpNoDelaySocketFactory (quartz
jvmAndroid, next to BasicOkHttpWebSocket) is now used by the Android
relay pool factory, the Desktop relay client, amy's relay connections,
and geode's mirror worker. Direct connections only — SOCKS/Tor paths
are untouched.
With the factory, the benchmark puts geode's ~21-row REQ at ~1.25 ms
on the wire (matching relayBench): ~0.6 ms Ktor CIO+OkHttp loopback
floor, ~0.5 ms per-REQ server work (already investigated). Per-frame
burst cost measured negligible and the pump adds ~nothing, so the
send-path latency angle of backlog item 6 is closed as not-a-problem;
its ingest-CPU share remains a separate throughput question. Findings
recorded in quartz/plans/2026-07-04-small-req-floor.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Reverts the queryRawInline fast path (fb29d655, 1b786f31) per the
keep-only-winners rule. Three relayBench runs at 50k (baseline, cap-256
where the path never engaged, cap-512 where author-archive/by-ids/
500-limit feeds genuinely took it) showed no movement outside the
container drift band — strfry's own numbers drifted ±30% between runs
and inline-eligible scenarios moved the same as ineligible ones.
The in-process win was real but small (~17%, 0.60 -> 0.50 ms per
~21-row REQ); the wire-level p50 is 1.2-1.7 ms, so the missing ~1 ms
per REQ sits in the transport (Ktor frame send path + client round
trip) — backlog item 6 territory, not dispatch. Findings, numbers, and
the do-not-retry note live in quartz/plans/2026-07-04-small-req-floor.md;
SmallReqFloorBenchmark stays as the measurement tool.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
The first cut's rule (every filter needs limit, sum <= 256) missed the
shapes relays actually receive: relayBench's author-archive carries
limit=500 and by-ids carries only an ids list, so the fast path never
engaged in the acceptance run. Bounds now come from limit OR the ids
count (ids are unique keys, the result cannot exceed the list), summed
against a 512 cap — a 500-row inline replay measures single-digit ms,
nothing a CLOSE could meaningfully preempt. Provably unbounded filters
(no limit, no ids — e.g. a bare tag filter) keep the launched path.
Pending: relayBench verdict decides whether the fast path stays at all,
per the keep-only-winners rule.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Backlog item 2 (small-REQ dispatch floor), first cut. relayBench at 50k
shows geode ~2.5x slower than strfry when a REQ returns ~20 events
(author-archive 1.18 vs 0.48 ms EOSE p50) while WINNING the 500-event
scenarios — with tiny results the fixed per-REQ cost dominates. A new
stage benchmark (SmallReqFloorBenchmark) decomposes the floor: at ~21
rows, raw SQL is 0.18 ms and the remaining ~0.42 ms is live-subscription
machinery plus per-REQ coroutine dispatch.
A REQ whose filters all carry a limit summing to <= 256 now runs its
stored replay inline on the receive coroutine and only retains a
live-tail handle (SessionBackend.queryRawInline; LiveEventStore's
queryRaw is re-expressed on the same core) — no per-REQ launch, no Job,
no dispatcher handoffs, no awaitCancellation scaffolding. Unbounded
REQs keep the launched path so CLOSE can always interrupt a giant
replay. In-process time-to-EOSE for ~21-row REQs drops 0.60 -> 0.50 ms;
the bigger effect expected under concurrency (no per-REQ Job+dispatch
churn) is relayBench's to judge.
InlineReqFastPathTest pins wire-behavior parity: stored-then-EOSE
ordering, live tail delivery and CLOSE detachment, same-subId
replacement, launched fallback for unbounded and over-cap REQs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Milestones 2-3 of quartz/plans/2026-07-03-incremental-negentropy-storage.md.
SQLiteEventStore now maintains the LiveNegentropyIndex when the strategy
opts in (geode's does by default; [negentropy].live_index = false turns
it off; app-side stores are untouched):
- Write paths collect a LiveIndexDelta and apply it after COMMIT while
still holding the writer mutex — rolled-back savepoint rows never
reach the index and updates land in exact commit order (also vs the
rebuild, which runs under the same mutex).
- Replaceable/addressable overwrites report the row their BEFORE-INSERT
trigger displaces via one indexed pre-SELECT that mirrors the trigger
predicate (including the NIP-01 lowest-id tie-break and the
d_tag-NULL case).
- Paths that can't itemize (kind-5, vanish, delete-by-filter/id,
expiration sweeps, clearDB) invalidate; the next NEG-OPEN rebuilds
from one scan on the writer connection.
- Until that first NEG-OPEN populates the index, ingest pays zero
bookkeeping — the populated check happens under the writer mutex so
it can't race the rebuild.
LiveEventStore serves a single unconstrained filter (the relay-relay
sync default; relayBench sends exactly this) from the index; everything
else keeps the scan+seal path and its single-slot cache.
Micro-benchmark at 50k events (LiveNegentropyBenchmark, in-container):
scan+seal cold path 80-100 ms; index post-write open 9-16 ms (~5-10x).
The relayBench A/B is the acceptance gate and comes next.
Correctness: LiveNegentropyIndexStoreTest asserts index content ==
snapshotIdsForNegentropy scan after every mutation pattern (overwrites,
losers, kind-5 rebuilds, filter deletes, mixed-outcome batches,
transactions); the full geode suite (NIP-77 + interop sync tests) runs
with the index on.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Milestone 1 of quartz/plans/2026-07-03-incremental-negentropy-storage.md.
Sorted array with binary-search insert (near-tail in the common case),
itemized remove for displaced rows, wholesale invalidate for delete
paths that can't itemize, and sealed snapshots memoized per mutation
generation — reconcile only reads, so one snapshot backs any number of
concurrent sessions and stays immutable under later writes. Over-cap
answers null so the caller keeps the strfry-parity NEG-ERR.
Not wired into any store yet; next milestones plumb displaced-row
deltas from the SQLite modules and serve index-total filters from it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
Claude
davotoula
Vitor PamplonaandGitHub