c-relay/plans/database_architecture_analysis.md

# Database Architecture Analysis: Thread Pool vs Client-Server DB vs LMDB

## Current Architecture Problem

The c-relay runs a **single-threaded event loop** via `lws_service()` in [`start_websocket_relay()`](src/websockets.c:2637). Every WebSocket callback — including [`handle_req_message()`](src/main.c:1101) and [`store_event()`](src/main.c:787) — executes **synchronously** inside this loop. When a REQ query takes 672ms (as documented in your [query analysis report](query_analysis_report.md:39)), **every other connected client is frozen** for that duration.

### The Blocking Chain

```mermaid
sequenceDiagram
    participant C1 as Client 1
    participant LWS as lws_service loop
    participant DB as SQLite g_db
    participant C2 as Client 2

    C1->>LWS: REQ with tag filter
    LWS->>DB: sqlite3_prepare + step loop
    Note over LWS,DB: BLOCKED 50-672ms
    C2->>LWS: EVENT submission
    Note over C2: Waiting... event loop frozen
    DB-->>LWS: Results returned
    LWS-->>C1: EVENT responses + EOSE
    LWS->>DB: store_event for C2
    Note over LWS,DB: BLOCKED again for write
    DB-->>LWS: Write complete
    LWS-->>C2: OK response
```

### What the Numbers Tell Us

From your production data:
- **2.7 GB database** but only **186 MB actual data** — 98% is indexes and overhead
- **4 million `event_tags` rows** with 3 indexes = ~2 GB of index space
- **Average query: 10.4ms**, worst case: **672ms**
- **98% of queries are REQ reads**, only 2% are writes
- **~120 new subscriptions/minute** = 2 queries/second minimum hitting the DB

The single `sqlite3* g_db` connection is shared across all operations with **no mutex protection on the connection itself** — it works only because everything runs on one thread. This is the fundamental bottleneck.

---

## Option 2: SQLite Thread Pool

### How It Works

Create a pool of N worker threads, each with its own `sqlite3*` connection to the same database file. SQLite WAL mode (already enabled) supports **concurrent readers**. The architecture becomes:

```mermaid
flowchart TD
    LWS[lws_service event loop] -->|REQ arrives| Q[Thread-safe job queue]
    LWS -->|EVENT arrives| WQ[Write queue - single writer]
    Q --> T1[Reader Thread 1 - own sqlite3*]
    Q --> T2[Reader Thread 2 - own sqlite3*]
    Q --> T3[Reader Thread 3 - own sqlite3*]
    Q --> T4[Reader Thread 4 - own sqlite3*]
    WQ --> TW[Writer Thread - own sqlite3*]
    T1 -->|results| CB[Callback to lws event loop]
    T2 -->|results| CB
    T3 -->|results| CB
    T4 -->|results| CB
    TW -->|OK/error| CB
    CB -->|queue_message| LWS
```

### Key Design Points

1. **Read path**: REQ queries dispatched to thread pool. Each worker opens its own `sqlite3*` connection. SQLite WAL allows unlimited concurrent readers.

2. **Write path**: EVENT inserts go through a single dedicated writer thread. SQLite only allows one writer at a time anyway — this serializes writes cleanly.

3. **Result delivery**: Worker threads cannot call `lws_write()` directly (libwebsockets is not thread-safe). Instead, they push results into a per-session message queue and call `lws_cancel_service()` to wake the event loop, which then drains the queue.

4. **Connection lifecycle**: Each thread opens its own connection with `PRAGMA journal_mode=WAL` and `PRAGMA busy_timeout=5000`.

### What Changes in the Codebase

| Component | Current | Thread Pool |
|-----------|---------|-------------|
| [`g_db`](src/main.c:49) | Single global connection | One per thread + writer connection |
| [`handle_req_message()`](src/main.c:1101) | Synchronous SQL in callback | Package filter into job, dispatch to pool |
| [`store_event()`](src/main.c:787) | Synchronous INSERT in callback | Dispatch to writer thread |
| [`handle_count_message()`](src/websockets.c:2863) | Synchronous COUNT in callback | Dispatch to pool |
| Result delivery | Direct `queue_message()` | Worker pushes to queue + `lws_cancel_service()` |
| Config reads | Direct `sqlite3_prepare` on `g_db` | Can stay synchronous with own connection or cache |

### Performance Characteristics

| Metric | Value |
|--------|-------|
| **Concurrent reads** | N readers in parallel (N = thread count, typically 4-8) |
| **Write throughput** | Same as current — SQLite serializes writes regardless |
| **Event loop latency** | Near-zero — REQ no longer blocks the loop |
| **Max theoretical read throughput** | ~4-8x current (limited by disk I/O, not CPU) |
| **Memory overhead** | ~50-100 MB per connection (page cache) |
| **Latency per query** | Same as current per-query, but no head-of-line blocking |

### Limitations

- **Write contention**: SQLite still allows only ONE writer at a time. With WAL, readers don't block writers and writers don't block readers, but two simultaneous writes will serialize. At your write rate (~56 events/hour), this is a non-issue.
- **Database size**: The 2.7 GB index bloat problem remains. Thread pool doesn't fix the schema — it fixes the concurrency.
- **Scaling ceiling**: Beyond ~8 reader threads, you hit diminishing returns due to disk I/O contention on a single SQLite file.
- **Complexity**: Need a proper job queue, thread lifecycle management, and careful handling of the lws ↔ worker thread boundary.

---

## Option 3: Client-Server Database (PostgreSQL)

### How It Works

Replace SQLite with PostgreSQL. The relay connects via `libpq` (PostgreSQL C client library). PostgreSQL runs as a separate process with its own connection pooling, query planner, and MVCC concurrency.

```mermaid
flowchart TD
    LWS[lws_service event loop] -->|async query| PG[libpq async connection pool]
    PG -->|TCP/Unix socket| PGS[PostgreSQL Server]
    PGS --> D1[Disk - WAL]
    PGS --> D2[Shared Buffers - RAM cache]
    PGS --> W1[Worker Process 1]
    PGS --> W2[Worker Process 2]
    PGS --> WN[Worker Process N]
    W1 -->|results| PG
    PG -->|callback| LWS
```

### Performance Characteristics

| Metric | Value |
|--------|-------|
| **Concurrent reads** | Unlimited — each query gets its own backend process |
| **Concurrent writes** | True concurrent writes with row-level locking |
| **Event loop latency** | Near-zero with async `libpq` |
| **Max theoretical throughput** | 10-100x SQLite depending on hardware |
| **Memory overhead** | ~10 MB per PostgreSQL backend + shared_buffers |
| **Latency per query** | Slightly higher for simple queries due to TCP/IPC overhead, but much better for complex queries due to superior query planner |
| **Index efficiency** | Far superior — PostgreSQL B-tree indexes are more space-efficient, supports partial indexes, GIN indexes for JSON |

### What Changes in the Codebase

**Everything touching `sqlite3*` must be rewritten.** This is ~258 call sites across 8 files:

| File | sqlite3 calls | Scope of change |
|------|--------------|-----------------|
| [`src/main.c`](src/main.c) | ~80 | Complete rewrite of store_event, handle_req_message, retrieve_event, init_database |
| [`src/config.c`](src/config.c) | ~60 | All config table operations |
| [`src/websockets.c`](src/websockets.c) | ~15 | COUNT queries, connection tracking |
| [`src/api.c`](src/api.c) | ~40 | All monitoring/stats queries |
| [`src/dm_admin.c`](src/dm_admin.c) | ~20 | Auth rules, WoT sync |
| [`src/subscriptions.c`](src/subscriptions.c) | ~15 | Subscription logging |
| [`src/nip009.c`](src/nip009.c) | ~10 | Event deletion |
| [`src/ip_ban.c`](src/ip_ban.c) | ~15 | IP ban persistence |
| [`src/request_validator.c`](src/request_validator.c) | ~8 | Auth rule checks |

### Advantages Over Thread Pool

1. **True write concurrency**: PostgreSQL handles concurrent writes with row-level locking. If your relay grows to thousands of events/hour, this matters.
2. **Superior query planner**: PostgreSQL's cost-based optimizer is far more sophisticated than SQLite's. Complex tag queries with JOINs will be faster.
3. **Index efficiency**: PostgreSQL's indexes are more compact. Your 2 GB of SQLite indexes would likely be ~500 MB in PostgreSQL. GIN indexes on JSONB would eliminate the `event_tags` table entirely.
4. **JSONB native**: PostgreSQL has native JSONB with indexing. You could store tags as JSONB and query them directly with `@>` operator — no denormalized `event_tags` table needed.
5. **Connection pooling**: PgBouncer or built-in pooling handles thousands of concurrent connections efficiently.
6. **Operational tooling**: `pg_stat_statements`, `EXPLAIN ANALYZE`, `pg_dump`, replication, etc.
7. **Horizontal scaling**: Read replicas possible for future growth.

### Disadvantages vs Thread Pool

1. **Massive rewrite**: ~258 call sites across 8 files. This is essentially rewriting the entire data layer.
2. **External dependency**: PostgreSQL must be installed, configured, and maintained. SQLite is zero-config embedded.
3. **Deployment complexity**: Your current deployment is a single binary + SQLite file. PostgreSQL adds a service dependency.
4. **Latency for simple queries**: A simple `SELECT 1 FROM events WHERE id=?` is ~0.1ms in SQLite vs ~0.5ms in PostgreSQL due to IPC overhead. For your 98% read workload, this adds up.
5. **Memory footprint**: PostgreSQL server uses 100-500 MB baseline. SQLite uses ~50 MB.
6. **Testing complexity**: Tests need a running PostgreSQL instance.

---

## Option 4: LMDB (Lightning Memory-Mapped Database)

### What Is LMDB?

LMDB is an embedded key-value store created by Howard Chu for OpenLDAP. It's what **strfry** — the fastest known Nostr relay — uses as its storage engine. It's fundamentally different from both SQLite and PostgreSQL:

- **Memory-mapped**: The entire database is `mmap()`'d into the process address space. Reads are literally pointer dereferences — no syscalls, no copies, no serialization.
- **B+ tree on disk**: Data is stored in a B+ tree that maps directly to memory pages. The OS page cache IS the database cache.
- **MVCC with copy-on-write**: Readers never block writers, writers never block readers. No locks needed for reads at all.
- **Zero-copy reads**: When you read a value, you get a pointer directly into the mmap'd region. No `malloc`, no `memcpy`.
- **Single-writer, multiple-reader**: Like SQLite WAL, but implemented at a much lower level with near-zero overhead.

### How It Would Work for c-relay

LMDB is a **key-value store**, not a relational database. There's no SQL. You design your own indexes as separate "databases" (sub-B-trees within the same file):

```mermaid
flowchart TD
    subgraph LMDB Environment
        DB1[events_by_id: event_id -> event_json]
        DB2[events_by_pubkey: pubkey+created_at -> event_id]
        DB3[events_by_kind: kind+created_at -> event_id]
        DB4[events_by_tag: tag_name+tag_value -> event_id]
        DB5[events_by_time: created_at -> event_id]
        DB6[config: key -> value]
        DB7[auth_rules: rule_id -> rule_data]
    end
    
    REQ[REQ query] -->|lookup| DB3
    REQ -->|lookup| DB4
    DB3 -->|get event_id| DB1
    DB4 -->|get event_id| DB1
    DB1 -->|zero-copy pointer| Response[Send to client]
```

### The Key Insight: Why strfry Is Fast

strfry doesn't use SQL at all. When a REQ comes in with `kinds: [1], #p: [pubkey123]`, strfry:

1. Opens a read-only LMDB transaction (no locks, no copies)
2. Seeks to `tag_p:pubkey123` in the tag index B-tree
3. Iterates matching event IDs
4. For each ID, does a direct pointer lookup in the events B-tree
5. Returns the raw bytes — zero-copy, no JSON parsing, no serialization

Compare this to your current c-relay flow:
1. Build SQL string with parameter binding
2. `sqlite3_prepare_v2()` — parse SQL, build query plan
3. `sqlite3_step()` — traverse B-tree, copy data to SQLite's page cache, then copy to your buffer
4. `sqlite3_column_text()` — copy string out of SQLite's internal format
5. `cJSON_Parse()` — parse the JSON string back into objects (for expiration check)
6. Build EVENT message string
7. Queue for sending

Steps 2-5 are **completely eliminated** with LMDB. The event JSON bytes go straight from the mmap'd file to the WebSocket send buffer.

### Performance Characteristics

| Metric | SQLite (current) | SQLite Thread Pool | PostgreSQL | LMDB |
|--------|-----------------|-------------------|------------|------|
| **Read latency** | 0.1-672ms | Same per-query | 0.5-50ms | **0.001-0.1ms** |
| **Concurrent reads** | 1 (single thread) | 4-8 | Unlimited | **Unlimited** (lock-free) |
| **Write latency** | 1-10ms | Same | 0.5-5ms | **0.01-1ms** |
| **Concurrent writes** | 1 | 1 | Many | **1** (single writer) |
| **Memory copies per read** | 3-4 | 3-4 | 2-3 | **0** (zero-copy) |
| **Index overhead** | ~2 GB for 186 MB data | Same | ~500 MB | **~200-400 MB** |
| **CPU per query** | High (SQL parse + JSON parse) | Same | Medium (SQL parse) | **Minimal** (pointer math) |

### What Changes in the Codebase

This is the **largest rewrite** of all options because you're replacing SQL with manual index management:

| Component | Current (SQLite) | LMDB Equivalent |
|-----------|-----------------|------------------|
| Schema definition | SQL DDL in [`sql_schema.h`](src/sql_schema.h) | C code defining named databases + key formats |
| [`store_event()`](src/main.c:787) | SQL INSERT with 9 bound params | `mdb_put()` into events db + `mdb_put()` into each index db |
| [`handle_req_message()`](src/main.c:1101) | Dynamic SQL builder (80 lines) | Manual cursor iteration across index databases with set intersection |
| Tag queries | `SELECT ... FROM event_tags WHERE tag_name=? AND tag_value IN (...)` | `mdb_cursor_get()` on tag index with `MDB_SET_RANGE` |
| Config reads | `SELECT value FROM config WHERE key=?` | `mdb_get()` on config database |
| Views/triggers | SQL views for analytics, triggers for replaceable events | Manual C code for all of it |
| Expiration check | `cJSON_Parse()` on every row | Can store expiration as separate indexed field — skip expired during cursor iteration |

### Advantages

1. **Raw speed**: 100-1000x faster reads than SQLite for your workload. strfry handles 10,000+ concurrent connections on modest hardware.
2. **Zero-copy**: Event JSON goes from disk → mmap → WebSocket buffer with no intermediate copies.
3. **No SQL overhead**: No query parsing, no query planning, no result materialization.
4. **Embedded**: Like SQLite, it's a library linked into your binary. No external server process.
5. **Proven for Nostr**: strfry demonstrates this works at scale for exactly this use case.
6. **Compact storage**: B+ tree is more space-efficient than SQLite's B-tree + WAL + journal overhead.
7. **Crash-safe**: MVCC with copy-on-write means the database is always consistent, even after power loss.
8. **No event loop blocking**: Read transactions are so fast (microseconds) they can run inline in the lws callback without needing a thread pool.

### Disadvantages

1. **No SQL**: You lose the ability to write ad-hoc queries. All query patterns must be pre-designed as index databases. Your admin SQL query API ([`api.c`](src/api.c) `execute_admin_sql_query()`) would need to be completely rethought.
2. **Manual index management**: Every query pattern needs its own index database. Adding a new filter type means adding a new index and backfilling it.
3. **Largest rewrite**: More code changes than even PostgreSQL, because you're replacing a query language with manual data structure operations.
4. **Single writer**: Like SQLite, only one write transaction at a time. Fine for your 56 events/hour, but a hard ceiling.
5. **No complex queries**: Queries like "top 10 pubkeys by event count" or "events per kind distribution" require manual aggregation in C code. Your monitoring views ([`event_kinds_view`](src/sql_schema.h:276), [`time_stats_view`](src/sql_schema.h:297)) would need C implementations.
6. **Memory mapping limits**: The database size is limited by virtual address space. On 64-bit systems this is effectively unlimited, but you must set `mapsize` at open time.
7. **Learning curve**: LMDB's API is low-level. Cursor management, transaction scoping, and key design require careful thought.
8. **Loss of admin SQL API**: Your current admin API supports arbitrary SQL queries via DM commands. This would be impossible with LMDB — you'd need to build specific query endpoints for each operation.

### The strfry Precedent

strfry's architecture is worth studying:
- Uses LMDB with custom indexes for each Nostr filter type
- Handles tag queries by maintaining a `tag_name:tag_value → event_id` index
- Supports NIP-01 filters by intersecting results from multiple index cursors
- Achieves sub-millisecond query times even with millions of events
- Single-threaded event loop (like your current design) but doesn't need a thread pool because reads are microseconds

However, strfry is **purpose-built** around LMDB from day one. Retrofitting LMDB into an existing SQLite-based codebase is significantly harder than starting fresh.

---

## Option 5: Document/JSON Databases

Since Nostr events are JSON documents, document databases seem like a natural fit. Let's evaluate the relevant options.

### CouchDB

CouchDB stores JSON documents natively, uses HTTP as its protocol, and builds indexes via JavaScript map-reduce views. You have experience with it.

**How it would work**: Each Nostr event becomes a CouchDB document. You'd create views for each query pattern — by kind, by pubkey, by tag, etc. CouchDB's MVCC model means readers never block writers.

**Why it's NOT a good fit for a Nostr relay**:

| Factor | Assessment |
|--------|-----------|
| **Latency** | HTTP API adds 1-5ms per request. Your current SQLite averages 10ms, so CouchDB wouldn't be faster for simple queries. |
| **C integration** | CouchDB is an Erlang server with an HTTP API. From C, every query is an HTTP request via libcurl. This is far heavier than a direct function call to SQLite or LMDB. |
| **View building** | Map-reduce views are written in JavaScript and built lazily. First query after data changes triggers a full view rebuild — this would cause massive latency spikes. |
| **Deployment** | Requires Erlang runtime + CouchDB server. Much heavier than PostgreSQL. |
| **Concurrency** | Good MVCC, but the HTTP overhead negates the benefit for an embedded relay. |
| **Real-time** | CouchDB's `_changes` feed could be useful for subscription broadcasting, but the HTTP polling model adds latency vs in-process callbacks. |

**Verdict**: CouchDB is excellent for web applications where you're already using HTTP, but for a C relay that needs microsecond-level response times, the HTTP layer is a dealbreaker.

### MongoDB

MongoDB is the most popular document database. It stores BSON (binary JSON), has rich query operators, and supports secondary indexes on any field including nested JSON paths.

**How it would work**: Events stored as BSON documents. Create indexes on `kind`, `pubkey`, `created_at`, and `tags` (using multikey indexes on arrays). Queries use MongoDB's query language which maps well to Nostr filters.

**Why it's a mixed fit**:

| Factor | Assessment |
|--------|-----------|
| **Query model** | Excellent — MongoDB's query operators map almost 1:1 to Nostr filters. `{kind: {$in: [1,7]}, tags: {$elemMatch: {0: "p", 1: "pubkey123"}}}` |
| **C driver** | `libmongoc` is mature and well-maintained. Supports async operations. |
| **Performance** | 0.5-5ms per query — faster than your current SQLite for complex queries, slower for simple ones. |
| **Index efficiency** | Multikey indexes on the tags array would eliminate the `event_tags` table problem. |
| **Deployment** | Requires MongoDB server. Heavy — 500 MB+ RAM baseline. |
| **Operational** | Needs replica set for durability. Single-node MongoDB is not crash-safe by default. |
| **Memory** | WiredTiger engine uses ~50% of RAM for cache by default. On a small VPS this is aggressive. |

**Verdict**: MongoDB's query model is the best match for Nostr filters of any database, but the operational overhead is massive for a single relay. It's designed for clusters, not embedded use.

### RethinkDB

RethinkDB is a document database with built-in real-time push via "changefeeds" — when a document changes, subscribed queries automatically receive the update.

**Why it's interesting for Nostr**: The changefeed model maps directly to Nostr subscriptions. When a new event is stored, RethinkDB could automatically push it to all matching subscriptions without your relay needing to do the [`broadcast_event_to_subscriptions()`](src/subscriptions.c:806) logic.

**Why it's NOT practical**:
- RethinkDB development has slowed significantly (community-maintained since 2017)
- C driver is not officially supported
- Deployment complexity similar to MongoDB
- Query performance is slower than MongoDB for simple lookups

**Verdict**: Interesting concept but not production-viable for a C relay.

### UnQLite — The Embedded Document Store

UnQLite is worth a closer look. It's an **embedded** key-value and document store — like SQLite but for JSON. Single C file, no external dependencies, public domain license.

**How it would work**:

```c
// Store event
unqlite_kv_store(pDb, event_id, 64, event_json, event_json_len);

// Retrieve event by ID
unqlite_kv_fetch(pDb, event_id, 64, buffer, &buf_len);

// For queries: use Jx9 scripting engine (embedded JavaScript-like language)
// Or: maintain manual indexes like LMDB
```

| Factor | Assessment |
|--------|-----------|
| **Integration** | Single C file, embeds like SQLite. No external dependencies. |
| **Performance** | Faster than SQLite for key-value operations, but Jx9 scripting is slow for complex queries. |
| **Query model** | Jx9 scripting language for document queries — but it's interpreted and slow. Manual KV indexes are fast but require same work as LMDB. |
| **Maturity** | Less battle-tested than SQLite or LMDB. Smaller community. |
| **Concurrency** | Single-writer, multiple-reader (like SQLite). |

**Verdict**: UnQLite is essentially a less mature, less performant LMDB with an optional (slow) document query layer. If you're going embedded KV, LMDB is the better choice.

### Summary: Document DBs for Nostr Relays

| Database | Query Fit | C Integration | Performance | Deployment | Verdict |
|----------|-----------|---------------|-------------|------------|---------|
| **CouchDB** | Good | Poor (HTTP) | Slow (HTTP overhead) | Heavy (Erlang) | ❌ Wrong paradigm for embedded relay |
| **MongoDB** | Excellent | Good (libmongoc) | Good | Heavy (server + replica set) | ⚠️ Great queries, bad deployment model |
| **RethinkDB** | Interesting | Poor (no C driver) | Medium | Heavy | ❌ Not production-viable |
| **UnQLite** | Medium | Excellent (single .c) | Medium | None (embedded) | ⚠️ Less mature LMDB alternative |
| **PostgreSQL JSONB** | Excellent | Good (libpq) | Good | Medium (server) | ✅ Best SQL + JSON hybrid |
| **LMDB** | Manual | Excellent (C library) | Best | None (embedded) | ✅ Best raw performance |

**The key insight**: For a Nostr relay, the "document database" that actually wins is either:
- **PostgreSQL with JSONB** — if you want SQL + JSON indexing + server model
- **LMDB with manual indexes** — if you want maximum performance + embedded model

The dedicated document databases (CouchDB, MongoDB) are designed for web application backends where HTTP latency is acceptable and operational complexity is managed by a team. For a single-binary C relay, they add overhead without proportional benefit.

---

## The Dashboard Problem: Separate Read Path

A real-time web dashboard that queries event statistics, subscription analytics, and connection metrics is **exactly the kind of workload that kills a single-process SQLite relay**. Here's why, and how each architecture handles it.

### Current Architecture: Dashboard Kills the Relay

Right now, dashboard queries run through the same [`g_db`](src/main.c:49) connection inside the same `lws_service()` event loop. When the dashboard runs analytics from [`event_kinds_view`](src/sql_schema.h:276) or [`time_stats_view`](src/sql_schema.h:297), these are **full table scans** on 65K+ events. Each one takes 100-500ms. While they run, every WebSocket client is frozen. Running these every few seconds for a "real-time" dashboard would make the relay unusable.

### How Each Architecture Solves This

```mermaid
flowchart LR
    subgraph Current - Everything Shares One Thread
        WS1[WebSocket Clients] --> EL[Event Loop]
        DASH1[Dashboard Queries] --> EL
        EL --> DB1[SQLite g_db]
    end
```

```mermaid
flowchart LR
    subgraph PostgreSQL - True Separation
        WS2[WebSocket Clients] --> RELAY[Relay Process]
        RELAY -->|connection pool| PG[PostgreSQL]
        DASHWEB[Dashboard Web Server] -->|own connection pool| PG
    end
```

```mermaid
flowchart LR
    subgraph LMDB - Multi-Process Reads
        WS3[WebSocket Clients] --> RELAY2[Relay Process]
        RELAY2 -->|mmap read txn| LMDB1[LMDB File]
        DASHWEB2[Dashboard Process] -->|own mmap read txn| LMDB1
    end
```

```mermaid
flowchart LR
    subgraph SQLite Thread Pool - Partial Separation
        WS4[WebSocket Clients] --> EL2[Event Loop]
        EL2 -->|dispatch| POOL[Thread Pool]
        POOL -->|own connections| DB2[SQLite WAL]
        DASH2[Dashboard] -->|also in pool| DB2
    end
```

### Detailed Comparison for Dashboard Use Case

| Factor | SQLite Thread Pool | PostgreSQL | LMDB |
|--------|-------------------|------------|------|
| **Separate dashboard process** | No — SQLite multi-process access is fragile | Yes — any number of processes can connect | Yes — multiple processes can mmap the same file for reads |
| **Dashboard query language** | SQL | SQL | No SQL — must build custom analytics endpoints in C |
| **Real-time refresh cost** | Medium — analytics queries compete with relay queries in the pool | Low — PostgreSQL handles concurrent analytics + relay queries independently | Low for simple metrics, High for complex aggregations |
| **Separate web server** | Difficult — SQLite file locking issues | Easy — any web framework connects to PostgreSQL | Possible — separate process opens LMDB read-only, but must build custom API |
| **Dashboard tech stack** | Must be embedded in relay | **Any**: Grafana, React, Python Flask, Go — all connect to PostgreSQL | Must build custom: LMDB has no standard query interface |

### This Changes the Recommendation

The dashboard requirement is a **strong argument for PostgreSQL**:

1. **True process isolation**: The relay and dashboard are completely separate processes. A heavy dashboard query never affects WebSocket latency.

2. **Use existing tools**: You could point **Grafana** directly at PostgreSQL and get a beautiful real-time dashboard with zero custom code. Or use any web framework to build a custom one.

3. **SQL for analytics**: Dashboard queries are inherently analytical — aggregations, time-series, top-N. SQL is the right tool. With LMDB, you'd hand-code every aggregation in C.

4. **Materialized views**: PostgreSQL can pre-compute expensive analytics:

```sql
-- Pre-computed, refreshed every 30 seconds in background
CREATE MATERIALIZED VIEW event_stats_mv AS
SELECT kind, COUNT(*) as count,
       ROUND(COUNT(*) * 100.0 / (SELECT COUNT(*) FROM events), 2) as pct
FROM events GROUP BY kind;

-- Dashboard reads this instantly, no table scan
SELECT * FROM event_stats_mv ORDER BY count DESC;
```

### LMDB + PostgreSQL Hybrid (Best of Both Worlds)

If you want LMDB's raw performance for the relay hot path AND a rich dashboard:

```mermaid
flowchart TD
    subgraph Relay Process
        WS[WebSocket Handler] -->|store and query| LMDB2[LMDB - Events]
        WS -->|async write| PG2[PostgreSQL - Analytics]
    end
    
    subgraph Dashboard Process
        DASH[Web Dashboard] -->|read only| PG2
    end
```

- **LMDB**: Handles all real-time event storage and REQ queries at microsecond latency
- **PostgreSQL**: Receives async copies of events for analytics/dashboard queries
- **Dashboard**: Queries PostgreSQL exclusively, never touches LMDB

This gives the best of both worlds but adds complexity — two databases to maintain plus async replication logic.

---

## Horizontal Scaling: Multiple Relay Instances in Practice

With a client-server database, you unlock something fundamentally impossible with SQLite or LMDB: **multiple relay instances sharing the same database**. This section goes deep on how this works in practice.

### Architecture Overview

```mermaid
flowchart TD
    INTERNET[Internet - Nostr Clients] -->|wss://relay.example.com| NGINX[nginx reverse proxy - TLS termination + load balancing]
    
    NGINX -->|ws://localhost:8888| R1[c-relay Instance 1 - port 8888]
    NGINX -->|ws://localhost:8889| R2[c-relay Instance 2 - port 8889]
    NGINX -->|ws://localhost:8890| R3[c-relay Instance 3 - port 8890]
    NGINX -->|http://localhost:3000| DASHWEB[Dashboard Web Server]
    
    R1 -->|libpq connection pool| PGPOOL[PgBouncer - connection pooler]
    R2 -->|libpq connection pool| PGPOOL
    R3 -->|libpq connection pool| PGPOOL
    
    PGPOOL -->|pooled connections| PG[PostgreSQL Primary]
    DASHWEB -->|read queries| PG
    
    PG -->|streaming replication| REPLICA[Read Replica - optional]
    DASHWEB -.->|heavy analytics| REPLICA
```

### How Load Balancing Works in Practice

nginx already handles TLS termination for most Nostr relays. Adding WebSocket load balancing is a small configuration change:

```nginx
# nginx.conf - WebSocket load balancing for c-relay
upstream relay_backends {
    # ip_hash ensures a client always hits the same instance
    # (important for WebSocket session stickiness)
    ip_hash;
    
    server 127.0.0.1:8888;
    server 127.0.0.1:8889;
    server 127.0.0.1:8890;
}

server {
    listen 443 ssl;
    server_name relay.example.com;
    
    # TLS config...
    
    location / {
        proxy_pass http://relay_backends;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header X-Real-IP $remote_addr;
        proxy_read_timeout 86400;  # Keep WebSocket alive for 24h
    }
    
    # Dashboard on separate path
    location /dashboard {
        proxy_pass http://127.0.0.1:3000;
    }
}
```

**Session stickiness** (`ip_hash`) ensures that once a client connects to Instance 2, all their subsequent WebSocket frames go to Instance 2. This is important because subscriptions are held in-memory per instance.

### Starting Multiple Instances

Each instance is the same binary, just on a different port, all pointing to the same PostgreSQL:

```bash
# Instance 1
./build/c_relay_x86 --port 8888 --db-host localhost --db-name crelay &

# Instance 2
./build/c_relay_x86 --port 8889 --db-host localhost --db-name crelay &

# Instance 3
./build/c_relay_x86 --port 8890 --db-host localhost --db-name crelay &
```

Or with systemd, you'd use a template unit:

```ini
# /etc/systemd/system/c-relay@.service
[Unit]
Description=C-Relay Nostr Instance %i
After=postgresql.service

[Service]
ExecStart=/opt/c-relay/c_relay_x86 --port %i --db-host localhost --db-name crelay
Restart=always
User=c-relay

[Install]
WantedBy=multi-user.target
```

```bash
systemctl enable c-relay@8888 c-relay@8889 c-relay@8890
systemctl start c-relay@8888 c-relay@8889 c-relay@8890
```

### The Subscription Broadcasting Challenge (Detailed)

This is the most important technical challenge with multiple instances. When Instance 1 receives a new EVENT and stores it in PostgreSQL, Instance 2 and Instance 3 need to know about it so they can broadcast to their connected subscribers.

```mermaid
sequenceDiagram
    participant Client_A as Client A - connected to Instance 1
    participant I1 as Instance 1
    participant PG as PostgreSQL
    participant I2 as Instance 2
    participant I3 as Instance 3
    participant Client_B as Client B - connected to Instance 2
    participant Client_C as Client C - connected to Instance 3

    Client_A->>I1: EVENT - new kind:1 note
    I1->>PG: INSERT INTO events...
    PG-->>I1: OK
    I1->>I1: broadcast to local subscribers
    I1->>PG: NOTIFY new_event with event_id
    
    Note over PG: PostgreSQL delivers notification to all listeners
    
    PG-->>I2: NOTIFY: new_event event_id
    PG-->>I3: NOTIFY: new_event event_id
    
    I2->>PG: SELECT event_json FROM events WHERE id = event_id
    PG-->>I2: event JSON
    I2->>I2: match against local subscriptions
    I2->>Client_B: EVENT message - if subscription matches
    
    I3->>PG: SELECT event_json FROM events WHERE id = event_id
    PG-->>I3: event JSON
    I3->>I3: match against local subscriptions
    I3->>Client_C: EVENT message - if subscription matches
```

#### Solution: PostgreSQL LISTEN/NOTIFY

This is built into PostgreSQL — no additional infrastructure needed:

```c
// === In the relay's event loop (modified lws_service loop) ===

// Setup: create a dedicated connection for LISTEN
PGconn* notify_conn = PQconnectdb("host=localhost dbname=crelay");
PQexec(notify_conn, "LISTEN new_event");
int notify_fd = PQsocket(notify_conn);  // Get the socket fd for poll()

// After storing an event:
void on_event_stored(PGconn* write_conn, const char* event_id) {
    char notify_cmd[128];
    snprintf(notify_cmd, sizeof(notify_cmd),
             "NOTIFY new_event, '%s'", event_id);
    PQexec(write_conn, notify_cmd);
}

// In the main event loop (runs every lws_service iteration):
void check_cross_instance_events(PGconn* notify_conn) {
    // Non-blocking check for notifications
    PQconsumeInput(notify_conn);
    
    PGnotify* notify;
    while ((notify = PQnotifies(notify_conn)) != NULL) {
        // Another instance stored a new event
        const char* event_id = notify->extra;
        
        // Fetch the event and check against local subscriptions
        cJSON* event = db_get_event_by_id(event_id);
        if (event) {
            broadcast_event_to_subscriptions(event);
            cJSON_Delete(event);
        }
        
        PQfreemem(notify);
    }
}
```

**Performance**: LISTEN/NOTIFY adds ~1-5ms latency for cross-instance delivery. For a Nostr relay, this is imperceptible — clients already expect network latency.

**Payload limit**: NOTIFY payloads are limited to 8000 bytes. For event IDs (64 hex chars), this is fine. For larger payloads, you'd store the event first and send just the ID.

#### Alternative: Redis Pub/Sub

If you later need even lower latency or more sophisticated routing:

```c
// Using hiredis (Redis C client)
redisContext* redis = redisConnect("127.0.0.1", 6379);

// After storing event:
redisCommand(redis, "PUBLISH new_event %s", event_json);

// Subscriber (in each instance):
redisCommand(redis, "SUBSCRIBE new_event");
// Then poll for messages in the event loop
```

Redis adds sub-millisecond pub/sub but requires running a Redis server. For most relays, PostgreSQL LISTEN/NOTIFY is sufficient.

### Zero-Downtime Deployment in Practice

```mermaid
sequenceDiagram
    participant LB as nginx
    participant I1 as Instance 1 - v1.0
    participant I2 as Instance 2 - v1.0
    participant I3 as Instance 3 - v1.0
    participant I1_NEW as Instance 1 - v1.1

    Note over LB,I3: Normal operation: 3 instances serving traffic
    
    LB->>I1: Mark upstream as down
    Note over I1: Drain: wait for existing connections to close or timeout
    I1->>I1: Graceful shutdown
    
    Note over LB: Traffic now goes to I2 and I3 only
    
    I1_NEW->>I1_NEW: Start with new binary
    I1_NEW->>LB: Health check passes
    LB->>I1_NEW: Mark upstream as up
    
    Note over LB,I1_NEW: Instance 1 now running v1.1, repeat for I2 and I3
```

In practice with nginx:

```bash
#!/bin/bash
# rolling_deploy.sh - Zero-downtime deployment

for port in 8888 8889 8890; do
    echo "Deploying instance on port $port..."
    
    # 1. Tell nginx to stop sending new connections
    # (mark server as "down" in upstream config, reload nginx)
    sed -i "s/server 127.0.0.1:$port;/server 127.0.0.1:$port down;/" /etc/nginx/nginx.conf
    nginx -s reload
    
    # 2. Wait for existing connections to drain (30 seconds)
    sleep 30
    
    # 3. Stop old instance
    systemctl stop c-relay@$port
    
    # 4. Deploy new binary
    cp ./build/c_relay_x86 /opt/c-relay/c_relay_x86
    
    # 5. Start new instance
    systemctl start c-relay@$port
    
    # 6. Wait for health check
    sleep 5
    
    # 7. Re-enable in nginx
    sed -i "s/server 127.0.0.1:$port down;/server 127.0.0.1:$port;/" /etc/nginx/nginx.conf
    nginx -s reload
    
    echo "Instance on port $port deployed successfully"
done
```

### Connection Pooling with PgBouncer

Each relay instance needs multiple database connections (for concurrent queries). Without pooling, 3 instances × 10 connections = 30 PostgreSQL backend processes. With PgBouncer:

```ini
# /etc/pgbouncer/pgbouncer.ini
[databases]
crelay = host=127.0.0.1 port=5432 dbname=crelay

[pgbouncer]
listen_port = 6432
listen_addr = 127.0.0.1
auth_type = md5
pool_mode = transaction    # Return connection to pool after each transaction
max_client_conn = 200      # Total connections from all relay instances
default_pool_size = 20     # Actual PostgreSQL connections
```

Relay instances connect to PgBouncer (port 6432) instead of PostgreSQL directly (port 5432). PgBouncer multiplexes 200 client connections onto 20 actual PostgreSQL connections.

### Scaling Scenarios

| Scenario | Instances | Connections | Events/hour | Setup |
|----------|-----------|-------------|-------------|-------|
| **Current** | 1 | ~1,200 | 56 | Single process + SQLite |
| **Small upgrade** | 2 | ~2,500 | 500 | 2 instances + PostgreSQL |
| **Medium relay** | 4 | ~5,000 | 5,000 | 4 instances + PostgreSQL + PgBouncer |
| **Large relay** | 8 | ~10,000 | 50,000 | 8 instances + PostgreSQL + read replica |
| **Multi-region** | 2-4 per region | ~50,000 | 500,000 | Multiple servers + PostgreSQL replication |

### Why This Is Impossible with SQLite/LMDB

| Feature | SQLite | LMDB | PostgreSQL | MySQL |
|---------|--------|------|------------|-------|
| **Multiple writer processes** | No — file lock | No — single writer | Yes — row-level locking | Yes — row-level locking |
| **Cross-process notifications** | No | No | Yes — LISTEN/NOTIFY | No built-in — need polling or external |
| **Connection pooling** | N/A | N/A | Yes — PgBouncer | Yes — ProxySQL |
| **Read replicas** | No | No | Yes — streaming | Yes — replication |
| **Load balancing** | Impossible | Impossible | Natural fit | Natural fit |
| **Max instances** | 1 | 1 | Unlimited | Unlimited |

---

## SQL Database Comparison: PostgreSQL vs MySQL vs MariaDB

Since we're considering a client-server SQL database, let's compare the main contenders.

### PostgreSQL

**The gold standard for data integrity and advanced features.**

| Aspect | Details |
|--------|---------|
| **JSONB support** | Native binary JSON with GIN indexing. `SELECT * FROM events WHERE tags @> '[["p","pubkey123"]]'` — indexed, fast. Eliminates the `event_tags` table entirely. |
| **LISTEN/NOTIFY** | Built-in pub/sub for cross-instance event broadcasting. No external dependencies. |
| **Partial indexes** | `CREATE INDEX idx_active ON events(kind) WHERE kind < 20000 OR kind >= 30000` — index only non-ephemeral events, saving space. |
| **MVCC** | True multi-version concurrency. Readers never block writers. No "locked" errors. |
| **Query planner** | Cost-based optimizer with statistics. Automatically chooses the best index for each query. |
| **C client library** | `libpq` — mature, well-documented, supports async queries. |
| **Materialized views** | Pre-compute expensive analytics, refresh on schedule. Dashboard reads are instant. |
| **Full-text search** | Built-in `tsvector` for NIP-50 search support. |
| **Memory** | ~200-500 MB baseline. `shared_buffers` should be ~25% of RAM. |
| **Nostr relay precedent** | Used by **nostream** (TypeScript relay) — proven at scale for Nostr. |

### MySQL / MariaDB

**The most widely deployed database. Simpler than PostgreSQL but less feature-rich.**

| Aspect | Details |
|--------|---------|
| **JSON support** | MySQL 5.7+ has JSON type with `JSON_CONTAINS()` and `JSON_EXTRACT()`. But **no GIN-equivalent index** — JSON queries do full scans or use generated columns with regular indexes. |
| **Cross-instance notifications** | No built-in equivalent to LISTEN/NOTIFY. Would need Redis, polling, or MySQL's binlog streaming. |
| **Partial indexes** | Not supported in MySQL. MariaDB has limited support. |
| **Concurrency** | InnoDB has row-level locking and MVCC, similar to PostgreSQL. |
| **Query planner** | Simpler than PostgreSQL's. Historically weaker for complex queries, but MySQL 8.0 improved significantly. |
| **C client library** | `libmysqlclient` or `libmariadb` — mature, well-documented. |
| **Materialized views** | Not supported natively. Must use tables + triggers or scheduled jobs. |
| **Full-text search** | InnoDB full-text indexes available but less flexible than PostgreSQL's. |
| **Memory** | ~100-300 MB baseline. Generally lighter than PostgreSQL. |
| **Nostr relay precedent** | No major Nostr relay uses MySQL. |

### MariaDB

MariaDB is a MySQL fork with some additional features:
- Better JSON support than MySQL (but still no GIN indexes)
- `CONNECT` engine for external data sources
- `ColumnStore` engine for analytics (interesting for dashboard)
- Generally compatible with MySQL client libraries

### Head-to-Head for Nostr Relay Use Case

| Feature | PostgreSQL | MySQL 8.0 | MariaDB |
|---------|-----------|-----------|---------|
| **JSON tag indexing** | **GIN index on JSONB — O(log n) lookups** | Generated column + B-tree — works but manual | Similar to MySQL |
| **Eliminates event_tags table** | **Yes — JSONB @> operator with GIN** | No — still need denormalized table or generated columns | No |
| **Cross-instance pub/sub** | **LISTEN/NOTIFY — built-in** | Need external solution (Redis, polling) | Need external solution |
| **Partial indexes** | **Yes — index only what matters** | No | Limited |
| **Materialized views** | **Yes — instant dashboard reads** | No — manual implementation | No |
| **NIP-50 full-text search** | **tsvector — powerful and indexed** | InnoDB FTS — adequate | InnoDB FTS — adequate |
| **Async C client** | **libpq async mode** | libmysqlclient async (MySQL 8.0+) | libmariadb async |
| **Connection pooling** | **PgBouncer — battle-tested** | ProxySQL — good | ProxySQL — good |
| **Ease of setup** | Medium — more config options | Easy — simpler defaults | Easy — simpler defaults |
| **Community/docs** | Excellent | Excellent | Good |
| **Hosting availability** | Every cloud provider | Every cloud provider | Most cloud providers |

### The Killer Feature: JSONB + GIN Indexes

This is why PostgreSQL wins for Nostr specifically. Your current schema has the `event_tags` table (4 million rows, ~424 MB data, ~2 GB indexes) solely because SQLite can't efficiently query JSON arrays.

With PostgreSQL JSONB + GIN:

```sql
-- PostgreSQL schema (no event_tags table needed!)
CREATE TABLE events (
    id TEXT PRIMARY KEY,
    pubkey TEXT NOT NULL,
    created_at BIGINT NOT NULL,
    kind INTEGER NOT NULL,
    content TEXT NOT NULL,
    sig TEXT NOT NULL,
    tags JSONB NOT NULL DEFAULT '[]',
    event_json TEXT NOT NULL
);

-- GIN index on tags — handles ALL tag queries
CREATE INDEX idx_events_tags ON events USING GIN (tags);

-- Query: find events with #p tag matching a pubkey
-- This uses the GIN index — O(log n), not a table scan
SELECT event_json FROM events
WHERE tags @> '[["p", "pubkey_hex_here"]]'
AND kind IN (1, 6, 7)
ORDER BY created_at DESC
LIMIT 100;

-- Query: find events with #e tag
SELECT event_json FROM events
WHERE tags @> '[["e", "event_id_here"]]'
ORDER BY created_at DESC
LIMIT 100;
```

**No `event_tags` table. No 4 million denormalized rows. No 2 GB of indexes.** The GIN index on the JSONB `tags` column handles everything in ~100-200 MB.

Compare to your current SQLite approach in [`handle_req_message()`](src/main.c:1459):
```sql
-- Current: subquery into event_tags table
AND id IN (SELECT event_id FROM event_tags
           WHERE tag_name = ? AND tag_value IN (?))
```

The PostgreSQL version is both simpler to write AND faster to execute.

### MySQL's JSON Workaround

MySQL can't do GIN indexes on JSON. The workaround is generated columns:

```sql
-- MySQL: must create generated columns for each tag type you want to index
ALTER TABLE events ADD COLUMN tag_p JSON
    GENERATED ALWAYS AS (
        JSON_EXTRACT(tags, '$[*]' ) -- complex extraction needed
    ) VIRTUAL;
CREATE INDEX idx_tag_p ON events(tag_p);
```

This is fragile, requires a generated column per tag type, and doesn't handle arbitrary tag names like `#g` (geohash), `#t` (hashtag), `#type`, etc. You'd need to know all tag types in advance.

### Verdict: PostgreSQL Is the Clear Winner for Nostr

PostgreSQL's JSONB + GIN indexes are **purpose-built** for the exact problem Nostr relays face: querying JSON arrays efficiently. No other SQL database matches this capability. Combined with LISTEN/NOTIFY for multi-instance broadcasting and materialized views for dashboards, PostgreSQL is the optimal SQL database for a Nostr relay.

MySQL/MariaDB would work, but you'd still need the `event_tags` denormalization table, you'd need Redis or polling for cross-instance events, and you'd need manual materialized view implementations. It's more work for less capability.

### Cost Perspective

Running multiple relay instances with PostgreSQL on a single VPS:
- **PostgreSQL**: ~200-500 MB RAM baseline
- **PgBouncer**: ~10 MB RAM
- **Each relay instance**: ~50-100 MB RAM
- **Dashboard web server**: ~50-100 MB RAM
- **4 instances + PostgreSQL + PgBouncer + dashboard**: ~1-2 GB total RAM
- A **$20/month VPS** with 4 cores and 4 GB RAM handles this easily
- A **$40/month VPS** with 8 cores and 8 GB RAM handles 8 instances comfortably

---

## Head-to-Head Comparison for YOUR Workload

Based on your actual production data (2.7 GB DB, 65K events, 120 REQ/min, 56 writes/hour):

| Factor | SQLite Thread Pool | PostgreSQL | LMDB |
|--------|-------------------|------------|------|
| **Fixes event loop blocking** | Yes | Yes | Yes — reads so fast they don't block |
| **Fixes 672ms worst-case queries** | Partially — queries still take 672ms but don't block others | Yes — better query planner + GIN indexes | **Yes — sub-millisecond reads** |
| **Fixes 2 GB index bloat** | No — same schema | Yes — more efficient indexes, JSONB eliminates event_tags | **Yes — B+ tree is more compact** |
| **Write throughput** | Adequate for 56 events/hour | Overkill for 56 events/hour | Adequate — single writer like SQLite |
| **Code changes** | ~300-500 lines new code, ~200 lines modified | ~2000-3000 lines rewritten | **~3000-4000 lines rewritten** |
| **Risk** | Low — additive change, SQLite stays | High — complete data layer replacement | **Highest — no SQL, manual indexes** |
| **Deployment change** | None — still single binary | Major — need PostgreSQL server | **None — still embedded library** |
| **Time to implement** | Moderate | Large | **Largest** |
| **Future scaling ceiling** | ~1000 concurrent connections | ~10,000+ concurrent connections | **~10,000+ like strfry** |
| **Rollback difficulty** | Easy — remove thread pool, back to synchronous | Very hard — different database entirely | **Very hard — different paradigm** |
| **Keeps admin SQL API** | Yes | Yes (different SQL dialect) | **No — must rebuild as specific endpoints** |
| **Raw read performance** | Same as current | 5-10x faster | **100-1000x faster** |
| **Operational simplicity** | Same as current | Needs DBA knowledge | **Same as current — single file** |

---

## Recommendation: Phased Approach

### Phase 1: Database Abstraction Layer (prerequisite for any path)

Create a `db_ops.h` / `db_ops.c` that wraps all database operations behind a clean interface:

```c
// db_ops.h - Database operations interface
typedef struct db_result db_result_t;
typedef struct db_event_filter db_event_filter_t;

// Core event operations
int db_store_event(const char* event_json, const char* id, const char* pubkey,
                   int kind, long created_at, const char* tags_json);
int db_event_exists(const char* event_id);
int db_delete_event(const char* event_id);
const char* db_get_event_json(const char* event_id);  // returns pointer, caller must not free for LMDB

// Query operations - backend-agnostic filter
db_result_t* db_query_events(const db_event_filter_t* filter);
int db_count_events(const db_event_filter_t* filter);
const char* db_result_next(db_result_t* result);  // returns event_json pointer
void db_result_free(db_result_t* result);

// Config operations
const char* db_get_config(const char* key);
int db_set_config(const char* key, const char* value);

// Lifecycle
int db_init(const char* path);
void db_close(void);
```

This consolidates the 258 scattered `sqlite3_*` calls into one file. The interface is designed to work with **any** backend:
- SQLite: `db_result_next()` copies from `sqlite3_column_text()`
- LMDB: `db_result_next()` returns a zero-copy pointer into mmap'd memory
- PostgreSQL: `db_result_next()` returns from `PQgetvalue()`

### Phase 2: Choose Your Backend

#### Path A: Thread Pool (fastest to implement, lowest risk)
- Keep SQLite, add worker threads for reads
- **Best if**: You want immediate relief and minimal disruption
- **Gets you**: Event loop unblocked, 4-8x effective throughput

#### Path B: LMDB (maximum performance, proven for Nostr)
- Replace SQLite with LMDB behind the abstraction layer
- **Best if**: You want strfry-level performance and are willing to invest in the rewrite
- **Gets you**: Sub-millisecond reads, zero-copy, 100-1000x faster queries, compact storage
- **Loses**: Admin SQL query API, ad-hoc analytics queries

#### Path C: PostgreSQL (maximum flexibility, operational overhead)
- Replace SQLite with PostgreSQL behind the abstraction layer
- **Best if**: You want SQL + concurrency + operational tooling
- **Gets you**: True concurrent writes, superior query planner, GIN indexes, replication
- **Loses**: Embedded simplicity, single-binary deployment

### Phase 3 (if LMDB chosen): Rebuild Admin Analytics

Replace the SQL-based admin API with purpose-built LMDB query endpoints:
- Event kind distribution → iterate events db, aggregate in C
- Top pubkeys → maintain a separate counter database
- Time-based stats → range scan on time index
- Or: keep a small SQLite database alongside LMDB just for analytics/config (hybrid approach)

---

## Bottom Line

Here's how I'd rank the options for **your specific situation** (65K events, 2.7 GB DB, 120 REQ/min, medium traffic relay):

### If you want the fastest fix with lowest risk:
**→ Thread Pool (Path A)**. Unblocks the event loop immediately. Your 672ms queries still take 672ms, but they no longer freeze every other client. This buys you time to plan a bigger change.

### If you want the best long-term architecture:
**→ LMDB (Path B)**. This is what the fastest Nostr relay in existence uses, and for good reason. Your 672ms queries become sub-millisecond. Your 2.7 GB database becomes ~400 MB. You don't need a thread pool because reads are so fast they can run inline. The downside is it's the biggest rewrite and you lose SQL flexibility.

### If you want SQL + performance:
**→ PostgreSQL (Path C)**. You keep SQL, get concurrent writes, get a better query planner, and get operational tooling. But you add an external dependency and deployment complexity.

### The pragmatic path:
**Phase 1 (abstraction layer) → Phase 2A (thread pool for immediate relief) → Phase 2B (LMDB migration behind the abstraction layer)**. This gives you immediate improvement while building toward the optimal architecture. The abstraction layer from Phase 1 makes the LMDB migration a contained change rather than a scattered rewrite across 8 files.

**Regardless of which backend you choose, Phase 1 (abstraction layer) is the right first step.** It cleans up the codebase, makes testing easier, and enables any future backend swap.