Files

T

Laan Tungir aedd22436e v2.1.16 - Implemented api-worker timer + LISTEN/NOTIFY monitoring (Phases 1,2,5) and fixed thread-connection binding bug that stopped dashboard updates

2026-07-25 10:07:08 -04:00

22 KiB

Raw Permalink Blame History

Caching Relay Daemon - Architecture Plan

Status: Draft v3 - adds NIP-65 outbox model: bootstrap relays for discovery, kind-10002 per-pubkey relay lists, minimum covering set relay selection, local-relay-first on subsequent startups.

Goal

A standalone C99 daemon that acts as a "caching relay feeder". It:

Reads a .jsonc config file listing one or more root npubs (e.g. your own npub), a set of upstream relays, a local relay URL, and configurable event kinds.
For each root npub, fetches its kind-3 contact list to discover followed pubkeys.
Subscribes live to events of the configured kinds from the union of followed pubkeys (root npubs + their follows).
Performs a throttled backfill of historical events per followed pubkey, spread out over time so upstream relays are not hammered.
Re-publishes every fetched event to the local relay via a plain WebSocket EVENT client connection (relay-agnostic - works with c-relay, c-relay-pg, or any Nostr relay).
Builds as a statically linked C99 binary following the c-relay model, linking against nostr_core_lib and c_utils_lib.

The user's Nostr client then points only at the local relay and gets a fast, pre-populated feed without doing any fan-out itself.

Design Principles

Relay-agnostic on the sink side. The daemon is just a Nostr client publishing EVENT messages over WebSocket. It does not touch the local relay's database directly. This keeps it decoupled and safe.
Reuse nostr_core_lib. All Nostr protocol concerns (event validation, kind-3 parsing, relay pool, WebSocket client, NIP-19 npub decoding) come from nostr_core_lib. The daemon is orchestration logic only.
Single statically linked binary. Built with the same Makefile pattern as c-relay: link libnostr_core_x64.a (or arm64) + libc_utils.a + system libs (-lwebsockets -lsqlite3 -lssl -lcrypto -lsecp256k1 -lcurl -lz -ldl -lpthread -lm).
C99, -Wall -Wextra -std=c99 -g -O2 matching c-relay's CFLAGS.
No config-file framework. Hand-rolled .jsonc parser using cJSON (strip // and /* */ comments before parsing, since cJSON does not natively support JSONC).

Architecture Overview

flowchart TD
    subgraph Config
        CFG[caching_relay.jsonc]
    end

    subgraph Daemon
        MAIN[main.c - lifecycle / signals]
        CFGP[config.c - parse jsonc]
        FOLLOW[follow_graph.c - root npubs + kind-3 resolution]
        BACKFILL[backfill.c - throttled historical pull]
        LIVE[live_subscriber.c - open subscriptions]
        SINK[relay_sink.c - publish EVENT to local relay]
        STATE[state.c - in-memory follow set + seen cache]
    end

    subgraph nostr_core_lib
        UPOOL[core_relay_pool.c - upstream pool - query/subscribe]
        SPOOL[core_relay_pool.c - sink pool - publish only]
        NIP01[nip001.c - validate / parse]
        NIP19[nip019.c - npub decode]
        WS[nostr_websocket - client WS]
    end

    subgraph Upstream
        R1[wss://relay.damus.io]
        R2[wss://nos.lol]
        R3[wss://...]
    end

    subgraph Local
        LOCAL[wss://127.0.0.1:8888 - c-relay / c-relay-pg / any]
    end

    CFG --> CFGP
    CFGP --> MAIN
    MAIN --> FOLLOW
    FOLLOW -->|query kind 3| UPOOL
    UPOOL --> R1
    UPOOL --> R2
    UPOOL --> R3
    FOLLOW --> STATE
    MAIN --> LIVE
    LIVE -->|subscribe authors + kinds| UPOOL
    LIVE -->|on_event| SINK
    MAIN --> BACKFILL
    BACKFILL -->|query per pubkey since T| UPOOL
    BACKFILL -->|on_event| SINK
    SINK -->|publish_async| SPOOL
    SPOOL -->|EVENT json| LOCAL
    STATE --> LIVE
    STATE --> BACKFILL
    CFGP -->|read/write state| CFG

Data Flow

sequenceDiagram
    participant D as Daemon
    participant UP as upstream_pool
    participant SP as sink_pool
    participant U as Upstream Relays
    participant L as Local Relay

    D->>D: parse caching_relay.jsonc + state
    D->>UP: add_relay x N upstreams
    D->>SP: add_relay local only
    D->>UP: query_sync kind=3 authors=root_npubs
    U-->>UP: kind-3 events
    UP-->>D: followed pubkeys set
    D->>D: merge root + follows into author set
    D->>UP: subscribe live authors=author_set kinds=configured since=now
    loop live loop - pump upstream_pool_run
        U-->>UP: EVENT
        UP-->>D: on_event callback
        D->>SP: publish_async EVENT
        SP->>L: EVENT json
        D->>SP: pump sink_pool_run to flush callbacks
    end
    loop backfill - progressive window expansion
        D->>UP: query_sync authors=pubkey_i kinds since=now-window limit=K
        U-->>UP: historical events
        UP-->>D: events
        D->>SP: publish_async each EVENT
        D->>D: sleep tick_interval_seconds
        D->>D: on full round-robin pass - advance window, write state to jsonc
    end

Config File Format

caching_relay.jsonc (JSONC = JSON with comments). The config file is the single source of truth and is also the daemon's persistent state store: the daemon rewrites it to disk whenever the backfill window advances, so a restart resumes exactly where it left off.

{
  // Root npubs whose follows list we crawl
  "root_npubs": [
    "npub1...",
    "npub1..."
  ],
  // Upstream relays to pull events from
  "upstream_relays": [
    "wss://relay.damus.io",
    "wss://nos.lol",
    "wss://relay.nostr.band"
  ],
  // Local relay to feed events into (publish-only, never queried)
  "local_relay": "ws://127.0.0.1:8888",
  // Event kinds to cache
  "kinds": [1, 3, 6, 10000, 30023],

  // ---- Backfill: progressive window expansion ----
  "backfill": {
    "enabled": true,
    // Window schedule in seconds-from-now, applied in order.
    // The daemon first pulls everything newer than (now - 24h).
    // Once complete, it expands to (now - 7d), then (now - 30d), etc.
    "window_schedule_seconds": [86400, 604800, 2592000, 7776000, 31536000],
    // Per-pubkey query limit per tick (keeps individual queries light)
    "events_per_tick": 50,
    // Delay between pubkey backfill ticks (throttle to be polite)
    "tick_interval_seconds": 5,
    // Delay between completing one window and starting the next
    "window_cooldown_seconds": 60
  },

  // ---- Live subscription ----
  "live": {
    "enabled": true,
    "resubscribe_interval_seconds": 300
  },

  // Refresh the follow graph periodically
  "follow_graph_refresh_seconds": 600,

  // ---- Persistent state (managed by the daemon; do not hand-edit) ----
  // The daemon writes these back to this file as backfill progresses.
  // On restart it reads them to avoid re-pulling already-cached history.
  "state": {
    // How far back we have fully backfilled, as a unix timestamp.
    // Starts at 0 / absent on first run. Advances as each window completes.
    "backfilled_until": 0,
    // Index into window_schedule_seconds we are currently working on.
    "current_window_index": 0,
    // Round-robin cursor: which followed pubkey we backfill next.
    "backfill_cursor": 0
  }
}

State write-back rules:

The daemon rewrites the .jsonc file (preserving comments is not required on rewrite - it may emit plain JSON once it has been modified at runtime; the comments are only for the user's initial authoring convenience).
Write-back happens: (a) when a window completes and backfilled_until advances, (b) on graceful shutdown, (c) periodically (e.g. every 60s) so a crash loses at most one minute of cursor progress.
A write-ahead temp file + rename is used so the config is never left half-written.

Progressive Window-Expansion Backfill

Instead of a fixed per-pubkey cursor, the daemon uses a global window that expands backwards in time in discrete steps. This is simpler, gives a natural "recent first" experience, and is trivially resumable from the config file.

Window schedule

backfill.window_schedule_seconds is an ordered list, e.g. [86400, 604800, 2592000, 7776000, 31536000] = 1 day, 1 week, 1 month, 3 months, 1 year.

Algorithm

On startup, read state.backfilled_until (unix ts) and state.current_window_index from the config.
If backfilled_until == 0 (first run), set the current target window to window_schedule[0] (e.g. 24h). The backfill since cutoff is now - window_schedule[0].
Round-robin through every followed pubkey. For each pubkey, issue nostr_relay_pool_query_sync with:
- authors = [pubkey]
- kinds = configured kinds
- since = now - current_window_seconds (i.e. the full current window, not an incremental slice - the local relay dedups, so re-overlap is free and simpler than tracking per-pubkey cursors)
- limit = events_per_tick
Sleep tick_interval_seconds between pubkeys (throttle).
When one full round-robin pass over all followed pubkeys completes for the current window:
- Set state.backfilled_until = now - current_window_seconds.
- Advance state.current_window_index += 1.
- Persist the config file (temp + rename).
- Sleep window_cooldown_seconds before starting the next (wider) window.
Repeat with the next (wider) window. The since cutoff moves further back in time, so each window pulls the additional older slice. Because the local relay dedups inserts, events that fall in the overlap with the previous window are simply ignored on insert - no correctness issue.
After the last (widest) window completes, the daemon switches to steady-state: it only keeps the live subscription running and periodically re-runs the widest window to catch any events that migrated into scope (e.g. a followed user backfilling their own old notes to a different relay). backfilled_until stays pinned at the oldest window.

Restart behavior

On restart, the daemon reads backfilled_until and current_window_index.
It resumes at the current window (the one that was in progress when it stopped). Because each window re-pulls since = now - window_seconds, a partial window just re-runs from the start of the round-robin - cheap and correct.
state.backfill_cursor records which pubkey in the round-robin was next; this is a minor optimization and may be reset to 0 on restart without harm.

Throttling summary

tick_interval_seconds paces individual pubkey queries (e.g. 5s).
window_cooldown_seconds paces window-to-window transitions (e.g. 60s).
events_per_tick caps each query's result size (e.g. 50).
Per-relay query latency from nostr_relay_pool_get_relay_query_latency can be used to prefer fast relays for backfill; slow relays are still used for the live subscription (where completeness matters more than speed).

File Layout

caching_relay/
  plans/plan.md            (this file)
  Makefile                 (c-relay-style, static link)
  build_static.sh          (Alpine MUSL static builder, adapted from c-relay)
  Dockerfile.alpine-musl   (adapted from c-relay)
  src/
    main.c                 (lifecycle, signal handling, main loop)
    main.h
    config.c / config.h    (jsonc parse + validation)
    follow_graph.c / .h    (resolve root npubs -> kind-3 -> followed set)
    live_subscriber.c / .h (open-ended live subscription on the pool)
    backfill.c / .h        (throttled historical pull worker)
    relay_sink.c / .h      (publish EVENT to local relay via dedicated sink pool)
    state.c / .h           (in-memory author set + seen-event ring buffer; config state read/write)
    jsonc_strip.c / .h     (strip // and /* */ comments before cJSON_Parse)
    log.c / log.h          (colored stderr logging, c-relay style)
  examples/
    caching_relay.jsonc    (sample config)
  README.md

Build Model (mirrors c-relay)

Makefile with:
- CC = gcc, CFLAGS = -Wall -Wextra -std=c99 -g -O2
- INCLUDES = -I. -Isrc -I../nostr_core_lib -I../nostr_core_lib/nostr_core -I../nostr_core_lib/cjson -I../nostr_core_lib/nostr_websocket -I../c_utils_lib/src
- LIBS = -lwebsockets -lssl -lcrypto -lsecp256k1 -lcurl -lz -ldl -lpthread -lm -L../c_utils_lib -lc_utils
- Note: no -lsqlite3 - the daemon itself does not use SQLite. (c-relay uses SQLite for its event store, but that is the local relay's concern, not the daemon's.)
- NOSTR_CORE_LIB = ../nostr_core_lib/libnostr_core_x64.a (or arm64)
- Build nostr_core_lib with --nips=1,6,19 (1 basic, 6 keys, 19 npub bech32). NIP-42 is deferred to a later phase (public relays only for now). Kind-3 parsing is plain cJSON tag walking in NIP-01, no special NIP-02 module needed.
- Single static link line: $(CC) $(CFLAGS) $(INCLUDES) $(MAIN_SRC) -o $(TARGET) $(NOSTR_CORE_LIB) $(C_UTILS_LIB) $(LIBS)
build_static.sh adapted from c-relay's Alpine MUSL Docker builder to produce a truly static binary.

Key nostr_core_lib APIs Used

nostr_relay_pool_create() / nostr_relay_pool_add_relay() / nostr_relay_pool_destroy()
nostr_relay_pool_query_sync() - one-shot historical/backfill queries (kind-3 fetch, per-pubkey backfill)
nostr_relay_pool_subscribe() - long-lived live subscription with on_event / on_eose callbacks
nostr_relay_pool_run() / nostr_relay_pool_poll() - event loop driving both live sub and backfill
nostr_relay_pool_publish_async() - publish fetched events to the local relay via a dedicated single-relay sink pool (separate from the upstream query pool, so the local relay is never included in query_sync fan-out)
nostr_validate_event() - validate before republishing (defensive; upstream events should already be valid)
NIP-19: nostr_nip19_decode (or equivalent) to convert npub1... -> hex pubkey for filters
cJSON for filter construction and kind-3 tag parsing (["p", "<hex>", "<relay>", "<petname>"])

Concurrency Model

Keep it simple and single-threaded with a cooperative event loop, matching c-relay's spirit:

Two nostr_relay_pool_t instances, both driven from the main thread:
1. upstream_pool - holds all upstream_relays. Used for query_sync (kind-3 fetch, backfill) and the long-lived live subscribe.
2. sink_pool - holds only local_relay. Used exclusively for publish_async of fetched events. Never queried.
The main loop alternates between:
- nostr_relay_pool_run(upstream_pool, timeout_ms) to pump live sub events,
- a time-sliced backfill step (one pubkey query_sync per tick_interval_seconds),
- nostr_relay_pool_run(sink_pool, 0) to flush pending publish callbacks.
Live subscription's on_event callback calls relay_sink_publish() which enqueues an publish_async on the sink pool.
Backfill query_sync is synchronous and blocks the upstream loop briefly - acceptable since events_per_tick is small and tick_interval_seconds provides pacing.
If profiling later shows backfill blocking the live sub too much, backfill can be moved to a second thread with its own upstream pool. Start without that.

State Persistence

No SQLite in the daemon. The daemon's only persistent state is the backfill window cursor, and that lives in the .jsonc config file itself under the state object (see Config File Format above).
The followed-pubkey set is re-derived from kind-3 on every follow_graph_refresh_seconds tick and on startup - it is not persisted.
Event dedup is delegated entirely to the local relay (c-relay's INSERT OR IGNORE on event id). The daemon keeps a small in-memory ring buffer of recently-published event ids only to avoid redundant publish attempts within a single run; this is not persisted.
Config write-back uses a temp file + atomic rename so the config is never left half-written, even on crash.

Signals & Lifecycle

SIGINT / SIGTERM -> graceful shutdown: close subscriptions, destroy pool, close sink WS.
SIGHUP -> reload config (re-read jsonc, refresh follow graph, adjust subscriptions).
Logs to stderr with c-relay-style color prefixes.

Decisions Resolved

Sink pool. Use a dedicated single-relay nostr_relay_pool_t for the local sink, separate from the upstream pool. The local relay is never queried, only published to. Confirmed.
- Update (v3): On subsequent startups, the local relay IS queried for kind-3 and kind-10002 events (fast local cache lookup). It is still never queried for general event backfill -- only for discovery metadata.
NIP-42 auth. Deferred to a later phase. Phase 1 targets public relays only. Confirmed.
Kind-3 freshness. Use the most recent kind-3 per root npub (nostr_relay_pool_get_event with authors=[npub], kinds=[3]). Re-resolve on follow_graph_refresh_seconds interval. Confirmed.
State persistence. No SQLite in the daemon. The .jsonc config file is the state store; the daemon writes back state.backfilled_until, state.current_window_index, and state.backfill_cursor as backfill progresses. Confirmed.
Backfill strategy. Progressive window expansion (24h -> 7d -> 30d -> 90d -> 365d), round-robin per pubkey within each window, full-window re-pull (local relay dedups). Confirmed.
NIP-65 outbox model. upstream_relays in config are bootstrap relays only. The daemon discovers each followed pubkey's outbox relays via kind 10002, then computes the minimum covering set of relays (greedy set cover) that covers all followed pubkeys. Bootstrap relays are always included in the final set. Pubkeys with no kind 10002 fall back to bootstrap relays. Confirmed.
Local-relay-first on subsequent startups. On startup, if state.backfilled_until > 0, the daemon queries the local relay first for kind-3 and kind-10002 events (fast, no network). Falls back to bootstrap relays for any pubkey not found locally. Confirmed.
Admin kinds. Root (admin) npubs get a separate kind list (admin_kinds) with ["*"] support for all kinds. Two live subscriptions: follows_sub (non-admin, regular kinds) and admin_sub (admin, admin_kinds). Confirmed.

NIP-65 Outbox Model Design (Phase 2)

New module: `relay_discovery.c`

Responsibilities:

For each followed pubkey, fetch their most recent kind 10002 (relay list).
- First-time: query from bootstrap relays.
- Subsequent: query from local relay first, bootstrap fallback.
Parse r tags from kind 10002 events: ["r", "<url>"] or ["r", "<url>", "read"] / ["r", "<url>", "write"].
- We care about "read" relays (we are reading events FROM them).
- If no read/write marker, assume both.
Build a map: pubkey -> list of outbox relay URLs.
Publish all kind-10002 events to the local relay (cache them for next startup).

Greedy set cover algorithm

Input:  pubkey_to_relays map {pubkey -> [relay1, relay2, ...]}
Output: minimal set of relay URLs covering all pubkeys

1. uncovered = set of all pubkeys
2. selected = empty set
3. Add all bootstrap relays to selected (always included)
4. For each bootstrap relay, remove its known pubkeys from uncovered
   (bootstrap relays cover pubkeys with no 10002)
5. While uncovered is not empty:
   a. Find relay R that covers the most pubkeys in uncovered
   b. If no relay covers any uncovered pubkey, break (orphaned pubkeys)
   c. Add R to selected
   d. Remove all pubkeys covered by R from uncovered
6. Return selected

Data structures

/* Per-pubkey outbox relay list */
typedef struct {
    char pubkey[CR_HEX_LEN];
    char relays[CR_MAX_RELAYS_PER_PUBKEY][CR_URL_LEN];
    int  relay_count;
} cr_outbox_entry_t;

/* Relay-to-pubkeys coverage map (for set cover) */
typedef struct {
    char url[CR_URL_LEN];
    char pubkeys[CR_MAX_PUBKEYS_PER_RELAY][CR_HEX_LEN];
    int  pubkey_count;
} cr_relay_coverage_t;

/* Result of relay discovery */
typedef struct {
    cr_outbox_entry_t  *outboxes;      /* per-pubkey relay lists */
    int                 outbox_count;
    char                selected_relays[CR_MAX_UPSTREAM][CR_URL_LEN];
    int                 selected_count;
} cr_relay_map_t;

Startup sequence change

OLD:
  load config -> create pools -> resolve follow graph -> open live sub -> backfill

NEW:
  load config -> create pools ->
    resolve follow graph (local-first on subsequent) ->
    discover outbox relays (kind 10002, local-first on subsequent) ->
    compute minimum covering set ->
    add selected relays to upstream_pool ->
    log selected relays + coverage ->
    open live sub -> backfill (per-pubkey from their outbox relays)

Backfill change

Instead of fanning out each backfill query to ALL upstream relays, backfill queries each pubkey from that pubkey's specific outbox relays (or bootstrap relays as fallback). This is more efficient and more polite to relays that don't have that pubkey's events.

New config fields

None required. upstream_relays is reinterpreted as bootstrap relays. Optionally a max_outbox_relays cap could be added later if the covering set grows too large.

Implementation Todo List

Phase 0 - Local relay sanity check (DONE)

Start a local c-relay and verify read/write with nak. DONE.

Phase 1 - Daemon implementation (DONE)

1-12. All implemented, built, and tested. See git history. DONE.

Phase 2 - NIP-65 outbox model

Implement relay_discovery.c/.h: fetch kind-10002 per pubkey, parse r tags, build pubkey -> relays map, publish 10002 events to local relay.
Implement greedy set cover: compute minimum covering relay set from the outbox map, always include bootstrap relays.
Update follow_graph.c: query local relay first for kind-3 on subsequent startups, fall back to bootstrap relays.
Update backfill.c: query each pubkey from their specific outbox relays instead of all upstream relays.
Update main.c: insert relay discovery phase between follow graph resolution and live subscription. Add discovered relays to upstream_pool. Log selected relays and coverage.
Update caching_relay_config.jsonc: update comments to say "bootstrap relays" instead of "upstream relays".
Build and test NIP-65 outbox model end-to-end with a real npub.
Update README.md if any flowchart details change during implementation.

22 KiB Raw Permalink Blame History