# API Upgrade Plan: Monitoring Thread + PostgreSQL Push

## Current API Architecture

### How Monitoring Works Today

The relay publishes monitoring data as **kind 24567 ephemeral Nostr events**. The admin dashboard is a standard Nostr client that subscribes to these events.

```
Dashboard connects → subscribes to kind 24567 with d-tags
Relay detects subscription → starts generating monitoring events
Relay queries DB → builds JSON → signs event → broadcasts to subscribers
Dashboard receives events → renders stats/charts
```

### Current Monitoring Event Types

| d-tag | Query Function | What It Contains | Triggered By |
|-------|---------------|-----------------|-------------|
| `event_kinds` | `query_event_kind_distribution()` | Event count per kind, total events | Event storage |
| `time_stats` | `query_time_based_statistics()` | Events in last 24h/7d/30d | Event storage |
| `top_pubkeys` | `query_top_pubkeys()` | Top 10 pubkeys by event count | Event storage |
| `subscription_details` | `query_subscription_details()` | Active subscriptions list | Subscription changes |
| `cpu_metrics` | `query_cpu_metrics()` | PID, memory, CPU, connections | Both triggers |

### Current Admin Command System

Two parallel admin interfaces exist:

**1. Kind 23456 Admin Events (WebSocket)**
- Admin sends NIP-44 encrypted commands as kind 23456 events
- Relay decrypts, processes, responds with kind 23457 events
- Commands: config get/set, auth rules, system commands, SQL queries

**2. NIP-17 DM Admin (Direct Messages)**
- Admin sends gift-wrapped DMs (kind 1059) to relay
- Relay unwraps, processes commands, responds via DM
- Commands: stats, config, help, SQL queries

### Current Problems

1. **All monitoring runs on lws-main thread** — DB queries block the event loop
2. **Monitoring is triggered by event storage** — `generate_event_driven_monitoring()` runs synchronously after each event insert
3. **No dedicated monitoring thread** — monitoring piggybacks on the relay's main processing
4. **Throttle is time-based only** — `kind_24567_reporting_throttle_sec` (default 5s) but queries still run on main thread
5. **Dashboard polls via subscription** — no way to get data without the relay actively generating events

---

## Upgrade Plan

### Phase 1: Dedicated API Thread (`api-worker`)

**What:** Create a new thread that owns all monitoring event generation. It runs on a timer, queries PostgreSQL with its own connection, builds and signs monitoring events, and pushes them into the broadcast system.

**Thread behavior:**
```
api-worker thread:
  1. Open own PG connection
  2. Loop:
     a. Sleep for throttle_sec interval
     b. Check if anyone is subscribed to kind 24567
     c. If yes: run all monitoring queries
     d. Build kind 24567 events with results
     e. Sign events with relay key
     f. Push completed events to lws-main for broadcast
     g. Repeat
```

**Key design decisions:**
- The api-worker thread does ALL the heavy work (queries + signing)
- It only hands off pre-built, signed events to lws-main for broadcast
- lws-main never blocks on monitoring queries
- The thread sleeps when no one is subscribed (zero overhead)

**Files to change:**
- `src/api.c`: Refactor `generate_monitoring_event_for_type()` to be callable from any thread
- `src/websockets.c`: Add monitor thread lifecycle (start/stop), completion queue for broadcast
- `src/thread_pool.h`: Add api-worker thread config options

**New thread in the process:**
```
c_relay_pg process
├── lws-main           — WebSocket event loop
├── db-read-1..N       — Async REQ/COUNT queries
├── event-worker       — Async EVENT ingestion
├── db-write-1..N      — Async sub logging, misc writes
└── api-worker         — NEW: periodic monitoring event generation
```

### Phase 2: Remove Monitoring from Main Thread Path

**What:** Remove `generate_event_driven_monitoring()` and `generate_subscription_driven_monitoring()` calls from the synchronous event processing path.

**Currently these are called from:**
- After event storage (event-driven monitoring)
- After subscription create/close (subscription-driven monitoring)

**After change:**
- The api-worker thread handles all monitoring on its own timer
- Event storage and subscription changes no longer trigger monitoring directly
- The throttle interval controls how fresh the data is (default 5 seconds)

**Benefit:** Event processing becomes faster — no monitoring overhead per event.

### Phase 3: Dashboard Uses Normal Subscriptions for Real-Time Events

**What:** Instead of the relay generating special monitoring events for "recent events," the dashboard simply subscribes to all events using standard Nostr REQ filters.

**Current approach:** Dashboard subscribes to kind 24567 monitoring events that contain aggregated stats.

**New approach for real-time event feed:**
```javascript
// Dashboard subscribes to ALL events on the relay
relayPool.subscribeMany([url], [
  { kinds: [1, 3, 5, 7, ...], limit: 50 }  // Recent events
], {
  onevent(event) {
    // Render in real-time event feed
    addEventToFeed(event);
  }
});
```

**What this replaces:** The "Relay Events" page in the dashboard currently shows events from monitoring. Instead, it would show live events as they arrive via normal Nostr subscription.

**What stays as monitoring events:** Aggregated stats (event_kinds, time_stats, top_pubkeys, cpu_metrics, subscription_details) still need to be generated by the relay because they require SQL aggregation queries that a client can't do.

### Phase 4: Evaluate Which Monitoring Events Can Be Replaced

For each current monitoring event type, here's whether it should stay as a relay-generated event or be replaced by dashboard-side logic:

| d-tag | Current: Relay generates | Could dashboard do it? | Recommendation |
|-------|------------------------|----------------------|----------------|
| `event_kinds` | SQL: `SELECT kind, COUNT(*) FROM events GROUP BY kind` | No — requires DB aggregation | **Keep as monitoring event** |
| `time_stats` | SQL: `SELECT COUNT(*) FROM events WHERE created_at > ?` for 24h/7d/30d | No — requires DB aggregation | **Keep as monitoring event** |
| `top_pubkeys` | SQL: `SELECT pubkey, COUNT(*) FROM events GROUP BY pubkey ORDER BY count DESC LIMIT 10` | No — requires DB aggregation | **Keep as monitoring event** |
| `subscription_details` | SQL: `SELECT * FROM active_subscriptions_log` | No — requires DB access | **Keep as monitoring event** |
| `cpu_metrics` | Reads `/proc/self/stat`, memory info | No — requires server-side access | **Keep as monitoring event** |
| Real-time event feed | Currently via monitoring events | **Yes — normal Nostr subscription** | **Replace with REQ subscription** |
| Auth rules list | Currently via admin command | No — requires DB access | **Keep as admin command** |
| Config values | Currently via admin command | No — requires DB access | **Keep as admin command** |

**Conclusion:** All 5 monitoring event types should stay as relay-generated events (they require SQL aggregation or server-side data). The only thing that changes is the real-time event feed, which becomes a normal subscription.

### Phase 5: PostgreSQL LISTEN/NOTIFY for Monitoring Triggers (Future)

**What:** Instead of the monitor thread polling on a timer, PostgreSQL can push notifications when data changes.

**Example triggers:**
```sql
-- Notify when a new event is stored
CREATE OR REPLACE FUNCTION notify_event_stored() RETURNS trigger AS $$
BEGIN
  PERFORM pg_notify('event_stored', json_build_object(
    'kind', NEW.kind,
    'pubkey', substring(NEW.pubkey, 1, 8)
  )::text);
  RETURN NEW;
END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER trg_notify_event_stored
  AFTER INSERT ON events
  FOR EACH ROW EXECUTE FUNCTION notify_event_stored();
```

**How the monitor thread would use it:**
```
api-worker thread:
  1. LISTEN event_stored
  2. Wait for notification (blocks efficiently)
  3. On notification: batch up for throttle_sec, then generate monitoring events
  4. Repeat
```

**Benefit:** Zero-polling. The monitor thread only wakes up when data actually changes.

**This is optional and can be added later.** The timer-based approach in Phase 1 works well and is simpler.

---

## Updated Thread Layout (After All Phases)

```
c_relay_pg process
├── lws-main           — WebSocket event loop (never blocks on DB for monitoring)
├── db-read-1..4       — Async REQ/COUNT queries
├── event-worker       — Async EVENT ingestion (validate + store)
├── db-write-1..2      — Async sub logging, IP bans, misc writes
└── api-worker         — Periodic monitoring event generation (own PG conn)
```

**Total: 9 threads, 9 PG connections** (with 4 readers + 2 writers)

---

## Implementation Priority

| Phase | Description | Complexity | Impact |
|-------|-------------|-----------|--------|
| 1 | Dedicated api-worker thread | Medium | High — unblocks main thread |
| 2 | Remove monitoring from event processing path | Low | Medium — faster event processing |
| 3 | Dashboard uses normal subscriptions for event feed | Low (frontend only) | Medium — simpler, more reliable |
| 4 | Evaluate monitoring event types | Analysis only | Confirms architecture |
| 5 | PostgreSQL LISTEN/NOTIFY triggers | Medium | Low (optimization) — add later |

**Recommended start:** Phase 1 + 2 together (they're coupled), then Phase 3 (frontend change), then Phase 5 when needed.

---

## Relationship to Thread Pool Plan

This plan complements `thread_pool_pg_optimization_plan.md`:

- Thread pool plan handles **reader/writer scaling** for client-facing operations
- This plan handles **monitoring/API operations** on a dedicated thread
- Both plans share the same PG connection model (one connection per thread)
- The api-worker thread is independent from the thread pool — it has its own lifecycle

The combined target is:
```
lws-main (1) + readers (4) + event-worker (1) + writers (2) + api-worker (1) = 9 threads
```
