client/docs/cache-first-page-patterns.md

# Cache-First Page Implementation Patterns

## Lessons Learned & Best Practices for client Pages

---

## The Core Problem

Every page in this app communicates with Nostr relays through a shared Web Worker via `init-ndk.mjs`. Relay responses are inherently slow and unreliable — they may take seconds, time out after 15s, or never arrive at all. If page initialization `await`s relay fetches, the user stares at a blank screen.

Meanwhile, NDK maintains an IndexedDB cache (via Dexie) that contains previously-seen events. This cache is local and fast (sub-100ms). **Pages must render from cache first, then hydrate from relays in the background.**

---

## The Three Data APIs

### 1. `queryCache(filters)` — Local IndexedDB only (FAST)
- **Timeout**: 5 seconds
- **Source**: Dexie/IndexedDB only, no network
- **Use for**: Initial page render, instant UI population
- **Import**: `import { queryCache } from './js/init-ndk.mjs'`

### 2. `ndkFetchEvents(filters)` — Relay fetch (SLOW)
- **Timeout**: 15 seconds
- **Source**: Nostr relays via NDK in the worker
- **Use for**: Background hydration after cache render
- **Import**: `import { ndkFetchEvents } from './js/init-ndk.mjs'`
- **WARNING**: Never `await` this in the critical render path if cache data exists

### 3. `subscribe(filters, opts)` — Live streaming (ONGOING)
- **No timeout**: Stays open, delivers events via `window 'ndkEvent'` events
- **Source**: Cache first (if `cacheUsage: 'CACHE_FIRST'`), then relays
- **Use for**: Live updates after initial render
- **Import**: `import { subscribe } from './js/init-ndk.mjs'`

---

## The Golden Rule

```
NEVER await a relay call (ndkFetchEvents) in the critical render path
if you can get data from cache first.
```

---

## Standard Page Loading Pattern

```mermaid
sequenceDiagram
    participant Page
    participant Cache as queryCache - IndexedDB
    participant Relay as ndkFetchEvents - Relays
    participant Sub as subscribe - Live

    Page->>Cache: queryCache filters
    Cache-->>Page: cached events - instant
    Page->>Page: render UI from cache

    Page->>Relay: void ndkFetchEvents filters
    Note over Relay: fire-and-forget, no await
    Relay-->>Page: relay events - eventually
    Page->>Page: merge and re-render if newer

    Page->>Sub: subscribe filters, CACHE_FIRST
    Note over Sub: stays open for live updates
    Sub-->>Page: ndkEvent window events
    Page->>Page: upsert and re-render
```

---

## Implementation Template

### Step 1: Import both `queryCache` and `ndkFetchEvents`

```javascript
import {
  initNDKPage,
  getPubkey,
  queryCache,
  ndkFetchEvents,
  subscribe,
  // ... other imports
} from './js/init-ndk.mjs';
```

### Step 2: Cache-first data loading function

```javascript
async function loadPageData() {
  const pubkey = await getPubkey();

  // Phase 1: Cache (blocking — but fast)
  let items = [];
  try {
    const cached = await queryCache({
      kinds: [MY_KIND],
      authors: [pubkey],
      limit: 50
    });
    items = Array.isArray(cached) ? cached : [];
    renderItems(items);
    console.log('[my-page] Rendered from cache:', items.length);
  } catch (err) {
    console.warn('[my-page] Cache query failed:', err?.message);
  }

  // Phase 2: Relay hydration (non-blocking — fire and forget)
  void ndkFetchEvents({
    kinds: [MY_KIND],
    authors: [pubkey],
    limit: 50
  }).then((relayEvents) => {
    if (Array.isArray(relayEvents) && relayEvents.length > 0) {
      // Merge with existing items, dedupe by id
      const merged = mergeAndDedupe(items, relayEvents);
      renderItems(merged);
      console.log('[my-page] Hydrated from relays:', relayEvents.length);
    }
  }).catch((err) => {
    console.warn('[my-page] Relay hydration failed:', err?.message);
  });

  // Phase 3: Live subscription for ongoing updates
  subscribe(
    { kinds: [MY_KIND], authors: [pubkey] },
    { closeOnEose: false, cacheUsage: 'CACHE_FIRST' }
  );
}
```

### Step 3: Handle live events via ndkEvent listener

```javascript
window.addEventListener('ndkEvent', async (event) => {
  const evt = event.detail;
  if (!evt || evt.kind !== MY_KIND) return;
  // Upsert into your data structure and re-render
  upsertItem(evt);
  renderItems(getAllItems());
});
```

---

## Common Anti-Patterns to Avoid

### Anti-Pattern 1: Awaiting relay fetch in main()

```javascript
// BAD — blocks page for up to 15 seconds
async function main() {
  const events = await ndkFetchEvents({ kinds: [1], authors: [pubkey] });
  renderFeed(events);
}
```

```javascript
// GOOD — render from cache, hydrate in background
async function main() {
  const cached = await queryCache({ kinds: [1], authors: [pubkey] });
  renderFeed(cached);
  void ndkFetchEvents({ kinds: [1], authors: [pubkey] }).then(renderFeed);
}
```

### Anti-Pattern 2: Sequential relay fetches in a loop

```javascript
// BAD — each iteration blocks for up to 15 seconds
for (const item of items) {
  const refs = await ndkFetchEvents({ ids: [item.refId] });
  item.image = extractImage(refs[0]);
}
```

```javascript
// GOOD — try cache first, relay in background
for (const item of items) {
  let refs = [];
  try { refs = await queryCache({ ids: [item.refId] }); } catch (_) {}

  const ref = refs.find(r => r.id === item.refId);
  item.image = ref ? extractImage(ref) : '';

  if (!ref) {
    // Fire-and-forget relay lookup
    void ndkFetchEvents({ ids: [item.refId] }).then((relayRefs) => {
      const relayRef = relayRefs?.find(r => r.id === item.refId);
      if (relayRef) {
        item.image = extractImage(relayRef);
        // Re-render if needed
      }
    }).catch(() => {});
  }
}
```

### Anti-Pattern 3: Awaiting relay fetch for a single event by ID

```javascript
// BAD — blocks detail page for up to 15 seconds
if (isEventMode) {
  const events = await ndkFetchEvents({ ids: [targetEventId], limit: 1 });
  renderSingleEvent(events[0]);
}
```

```javascript
// GOOD — render from cache instantly, relay hydrates in background
if (isEventMode) {
  let matched = null;
  try {
    const cached = await queryCache({ ids: [targetEventId], limit: 1 });
    matched = Array.isArray(cached) ? cached.find(e => e?.id === targetEventId) : null;
  } catch (_) {}

  if (matched) renderSingleEvent(matched);

  void ndkFetchEvents({ ids: [targetEventId], limit: 1 }).then(relayEvents => {
    const relayMatch = Array.isArray(relayEvents)
      ? relayEvents.find(e => e?.id === targetEventId) : null;
    if (relayMatch) renderSingleEvent(relayMatch);
  }).catch(() => {});
}
```

### Anti-Pattern 4: Embedded content hydration using only relays

When a rendered post contains `nostr:nevent1...` references, the hydration code
fetches the referenced event to display an inline preview. This is a secondary
fetch triggered *after* the main content renders — easy to miss.

```javascript
// BAD — embedded note hydration blocks on relay, shows "Failed to fetch" on timeout
const events = await ndkFetchEventsFn(filter);
```

```javascript
// GOOD — try cache first, fall back to relay
let events = [];
if (queryCacheFn) {
  try { events = await queryCacheFn(filter); } catch (_) {}
}
if (!events || events.length === 0) {
  events = await ndkFetchEventsFn(filter);
}
```

### Anti-Pattern 5: Interaction data fetched only from relays

Social interaction queries — likes, reposts, zaps, comments — use `#e` tag
filters. These should also be cache-first so the interaction bar populates
instantly on page refresh.

```javascript
// BAD — interaction counts blank for 15 seconds
const allEvents = await ndkFetchEvents(filters);
applyInteractions(allEvents);
```

```javascript
// GOOD — show cached counts immediately, relay updates in background
let hadCache = false;
if (queryCacheFn) {
  try {
    const cached = await queryCacheFn(filters);
    hadCache = applyInteractions(cached, 'cache');
  } catch (_) {}
}

if (hadCache) {
  void ndkFetchEvents(filters).then(relay => applyInteractions(relay, 'relays')).catch(() => {});
} else {
  const relay = await ndkFetchEvents(filters);
  applyInteractions(relay, 'relays');
}
```

### Anti-Pattern 6: Module loading that blocks on relays

```javascript
// BAD — configureMuteList + loadMuteList blocks if relay is slow
configureMuteList({ ndkFetchEvents, getPubkey });
await loadMuteList(); // blocks for 15s if no cache

// GOOD — pass queryCache so cache loads instantly, relay hydrates in background
configureMuteList({ ndkFetchEvents, queryCache, getPubkey });
await loadMuteList(); // returns instantly from cache, relays fire-and-forget
```

---

## Module Configuration Checklist

When configuring shared modules like `mute-list.mjs`, always pass **both** `queryCache` and `ndkFetchEvents`:

```javascript
configureMuteList({
  ndkFetchEvents,
  queryCache,        // <-- REQUIRED for cache-first loading
  publishEvent,
  getPubkey,
  nip44Encrypt: ...,
  nip44Decrypt: ...
});
```

---

## Profile Resolution

The `profile-cache.mjs` module already implements cache-first patterns internally. When creating a profile cache instance, pass `queryCache`:

```javascript
const profileCache = createProfileCache({
  fetchCachedProfile,
  ndkFetchEvents,
  storeProfile,
  queryCache,           // <-- enables cache-first profile lookups
  refreshIntervalMs: 20 * 60 * 1000
});
```

---

## Timeout Reference

| API | Timeout | Blocking? |
|-----|---------|-----------|
| `queryCache()` | 5s | Yes (but fast — local IndexedDB) |
| `ndkFetchEvents()` | 15s | **Only if you await it** |
| `subscribe()` | None | No (event-driven) |
| `publishEvent()` | 15s | Yes (must await for confirmation) |

---

## Worker Request Latency Pattern (Critical for Wallet Actions)

Some UI actions call worker RPCs via `sendWorkerRequest()` and have strict front-end timeouts (for example, 45s for `walletSendNutzap`, 90s for `walletPayInvoice`).

If a worker handler does both:
1. **critical state updates** (required for user-visible success), and
2. **slow relay-side writes** (proof republish, history events, cleanup),

then awaiting both in sequence can cause front-end timeout errors even when the payment actually succeeded.

### Anti-Pattern 7: Responding after non-critical relay writes

```javascript
// BAD — response blocked behind slow relay publishing
await publishDirectProofs();
await publishSpendingHistoryEvent(tx);
port.postMessage({ type: 'response', requestId, data: successPayload });
```

```javascript
// GOOD — respond after critical path, background the rest
const payload = getWalletBalancePayload();
port.postMessage({ type: 'response', requestId, data: successPayload });
broadcast({ type: 'walletBalanceUpdated', data: payload });

void (async () => {
  try { await publishDirectProofs(); } catch (e) { console.warn(e); }
  try { await publishSpendingHistoryEvent(tx); } catch (e) { console.warn(e); }
})();
```

### Rule of thumb for worker handlers

- **Await only critical path steps** needed to determine success/failure for the request.
- **Respond immediately** once local state is coherent.
- **Background non-critical relay writes** with `void` async blocks and warning logs.
- **Do not hide failures silently**: warn in logs so reconciliation/debug remains possible.

This is now the expected pattern for latency-sensitive wallet operations like `walletPayInvoice` and `walletSendNutzap`.

---

## Dependency Threading Checklist

When a page passes dependencies to a module, and that module passes them to a
sub-module, `queryCache` must be threaded through **every layer**. Missing it at
any hop means the inner module falls back to relay-only fetches.

```
page.html  →  post-interactions2.mjs  →  post-interactions.mjs
  queryCache ✅     queryCache ✅            queryCacheFn ✅
```

**Rule**: If you add `queryCache` to a page's dependency object, grep for every
module in the chain that destructures and forwards those deps. Each one must
explicitly destructure and pass `queryCache` through.

```javascript
// post-interactions2.mjs — MUST destructure and forward queryCache
export function initPostCards(deps = {}) {
  const { ndkFetchEvents, queryCache, /* ... */ } = deps;

  const interactions = initInteractions({
    ndkFetchEvents,
    queryCache,    // <-- MUST be forwarded here
    // ...
  });
}
```

```javascript
// post-interactions.mjs — MUST accept and store queryCache
let queryCacheFn = null;

export function initInteractions(ndkFunctions = {}) {
  const { ndkFetchEvents, queryCache, /* ... */ } = ndkFunctions;
  queryCacheFn = typeof queryCache === 'function' ? queryCache : null;
}
```

---

## Render & Decrypt Performance Lessons (from msg.html, v0.4.20–v0.4.23)

These lessons were learned during the messaging page optimization cycle. They
apply to any page that does expensive async work (decryption, signing, heavy
DOM rendering) triggered by event-driven callbacks.

### Lesson 1: Measure first, then optimize

Add lightweight `performance.now()` instrumentation to every phase of your
page's critical path. Without timing data, you will guess wrong about where
the bottleneck is. In `msg.html`, we added a rolling perf report system
(`startMsgPerfReport` / `markMsgPerf` / `finalizeMsgPerfReport`) that logs
step-by-step timing to the console and exposes `window.__msgPerfReports` for
programmatic inspection. This immediately revealed that 63 seconds of wall
time was spent in eager decrypt — not in cache queries or DOM rendering.

### Lesson 2: Expensive async render functions need single-flight guards

If `renderThread()` (or any expensive async render function) can be called
from multiple event-driven paths — live subscription events, profile fetches,
gift-wrap processing, relay hydration callbacks — concurrent invocations will
pile up. Each one does the same expensive work (decrypt, DOM build), and all
but the last one get thrown away.

**Pattern**: Wrap the render function in a single-flight coalescing guard:

```javascript
let renderInFlight = null;
let renderQueued = false;

async function renderThread() {
  if (renderInFlight) {
    renderQueued = true;
    return renderInFlight;
  }
  renderInFlight = (async () => {
    do {
      renderQueued = false;
      await renderThreadInner();
    } while (renderQueued);
  })().finally(() => { renderInFlight = null; });
  return renderInFlight;
}
```

This collapses N concurrent calls into at most 2 executions (current + one
queued rerun).

### Lesson 3: Deduplicate in-flight async work, not just completed results

Caching the *result* of an async operation (e.g., `event._decodedKind4`) is
not enough if multiple callers start the operation before the first one
finishes. You must also deduplicate the *in-flight promise* itself.

**Pattern**: Use a `Map` keyed by operation identity to store the active
promise:

```javascript
const inFlightDecrypts = new Map();

async function decryptEvent(event, counterparty) {
  if (event._decoded) return event._decoded;

  const key = event.id + ':' + counterparty;
  if (inFlightDecrypts.has(key)) return inFlightDecrypts.get(key);

  const promise = doActualDecrypt(event, counterparty)
    .finally(() => inFlightDecrypts.delete(key));
  inFlightDecrypts.set(key, promise);
  return promise;
}
```

In `msg.html`, this reduced 290 decrypt calls (29 renders × 10 messages)
down to 10.

### Lesson 4: Status text should reflect progress, not just phase entry

Showing "Decrypting recent messages… 10" and then going silent for 60 seconds
is worse than showing nothing — it looks frozen. Update the status on every
iteration: "Decrypting recent messages… 3/10". This gives the user confidence
that work is happening.

### Lesson 5: Audit every call site that triggers re-render

Non-awaited render calls from places like `ensureProfileName()` and
`processKind4Event()` can fire dozens of times during background hydration.
Each one triggers a full render cycle. Treat render triggers like network
calls: audit them, guard them, and suppress them during batch operations
(e.g., using a `suppressUiRefresh` flag).

### Lesson 6: "Small" per-item latency explodes in serial loops

A 6-second decrypt call seems tolerable for one message. But 10 messages in a
serial loop = 60 seconds. And if that loop runs 29 times concurrently due to
a render storm, the user waits over a minute. Always consider the
multiplicative effect of per-item costs × loop iterations × concurrent
invocations.

### Lesson 7: Cache-first is necessary but not sufficient

Even after making `hydrateConversationHistory()` cache-first (rendering from
cache before relay hydration), the page still stalled because the *render
path itself* contained expensive blocking work (decrypt). Cache-first solves
the data-fetch bottleneck; you still need to address compute bottlenecks in
the render pipeline.

### Lesson 8: Keep perf tooling in production-safe form

Lightweight timing helpers that log to `console.groupCollapsed` and store
results in a bounded array are cheap enough to ship. They cost nothing when
nobody opens DevTools, and they save hours when debugging the next regression.

---

## Summary Rules

1. **Always import `queryCache`** alongside `ndkFetchEvents` in every page
2. **Render from cache first** — `await queryCache()` is safe to block on (fast, local)
3. **Never await `ndkFetchEvents` in the render path** — use `void` for fire-and-forget
4. **Use `subscribe` with `cacheUsage: 'CACHE_FIRST'`** for live updates
5. **Pass `queryCache` to all module configurations** (mute-list, profile-cache, etc.)
6. **Thread `queryCache` through every dependency hop** — page → wrapper → inner module
7. **Apply cache-first to secondary fetches** — embedded notes, interaction data, not just main content
8. **Avoid sequential relay fetches in loops** — batch or use cache-first per item
9. **Always handle relay timeouts gracefully** — the cache result is good enough for initial render
10. **Guard expensive async render functions with single-flight coalescing** — prevent render storms
11. **Deduplicate in-flight async work** — cache the promise, not just the result
12. **Audit all re-render trigger sites** — suppress during batch operations
13. **Show incremental progress in status text** — update per-item, not per-phase
14. **For worker RPC handlers, respond before non-critical relay writes** — background republish/history/cleanup work with warning logs