laantungir/client
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implement feed outbox model pre-warming (Phases 1-3) and relays.html connection history toggle with performance fixes (Phase 4)

Browse Source 
Phase 1 (ndk-worker.js): Add warmOutbox, setRelayEventLogging, and getDiscoveredRelays RPC handlers. Gate broadcastRelayEvent and logRelayEvent IndexedDB writes behind relayEventLoggingEnabled flag (default off). trackRelayRead/trackRelayWrite left ungated to preserve footer activity carrots.

Phase 2 (init-ndk.mjs): Add warmOutbox, setRelayEventLogging, and getDiscoveredRelays page API exports with proper request/response wiring and timeouts.

Phase 3 (feed.html): Pre-warm outbox tracker via warmOutbox() before bootstrapFeedPosts and fetchFeedWindow calls so NDK routes author-based subscriptions to follows' write relays before short-lived fetchEvents EOSE.

Phase 4 (relays.html): Add 'Show connection history' sidenav toggle (sidenavRowToggle pattern from feed.html), gate ndkRelayEvent listener and DB loading behind connectionHistoryEnabled flag, switch live event stream to incremental DOM appending, sync state on init/beforeunload.
This commit is contained in:
Laan Tungir
2026-06-26 09:38:31 -04:00
parent 3ed0c35611
commit c758333269
6 changed files with 1409 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
plans/feed-outbox-assessment.md
+932
View File
@@ -0,0 +1,932 @@
# Feed Outbox Model — Assessment & Plan
## Your suspicion is correct, and NDK already has the machinery built in.
You are missing posts from some follows because the feed only queries the
**relays you subscribe to** (your read/both relays from your kind 10002 list).
If a follow posts exclusively to relays you are *not* connected to, those posts
never arrive. This is exactly the problem the Nostr **outbox model** (NIP-65 +
NIP-17 relay lists) solves: instead of only listening on your relays, you look
up each follow's kind 10002 relay list and fetch their posts from *their* write
relays.
The good news: **NDK's outbox model is already enabled** in your bundle and is
already wired into the subscription path. The bad news: it is being defeated by
a race condition in how [`www/feed.html`](www/feed.html) bootstraps the feed.
---
## How NDK's outbox model works (already in your bundle)
Relevant code in [`www/ndk-core.bundle.js`](www/ndk-core.bundle.js):
1. **Outbox is on by default** — [`NDK`](www/ndk-core.bundle.js:43195) constructor
at line 43295: `if (!(opts.enableOutboxModel === false))` creates an
`outboxPool` (default relays `wss://purplepag.es/`, `wss://nos.lol/`) and an
`outboxTracker`. Your worker never sets `enableOutboxModel: false`, so it is
active. Confirmed: [`www/ndk-worker.js`](www/ndk-worker.js:1506) `ndkOptions`
does not disable it.
2. **Author-based subscriptions trigger outbox tracking** —
[`NDK.subscribe`](www/ndk-core.bundle.js:43665): when a filter has `authors`,
it calls `this.outboxTracker?.trackUsers(authors)`. This fetches each
author's kind 10002 (and kind 3 fallback) relay list from the outbox pool.
3. **Relay sets are computed per-author** —
[`calculateRelaySetsFromFilter`](www/ndk-core.bundle.js:33864) calls
[`getRelaysForFilterWithAuthors`](www/ndk-core.bundle.js:31650) →
[`chooseRelayCombinationForPubkeys`](www/ndk-core.bundle.js:31594) which
picks ~2 write relays per author and splits the `authors` array across the
relays each author actually writes to.
4. **Late outbox data refreshes subscriptions** — when
[`OutboxTracker`](www/ndk-core.bundle.js:42790) resolves a user's relay list,
it emits `user:relay-list-updated`, and the NDK constructor (line 43301)
calls `subscription.refreshRelayConnections()` to add newly-discovered
relays to the live subscription.
5. **Fallback for unknown authors** —
[`chooseRelayCombinationForPubkeys`](www/ndk-core.bundle.js:31639): authors
with no known relay list are sent to `pool.permanentAndConnectedRelays()`
(your read relays). So outbox never *loses* events vs. the old behavior; it
only *adds* coverage.
So in principle, a plain `ndk.subscribe({ kinds:[1], authors:[...] })` already
does outbox routing. The feature you want exists.
---
## Why your feed still misses people — the root cause
The problem is a **race / staleness condition** in
[`www/feed.html`](www/feed.html), not a missing NDK feature.
### The bootstrap path (lines 481–545)
[`fetchFeedWindow`](www/feed.html:481) does this for each time window:
1. `queryCache(filters)` — reads the Dexie cache synchronously.
2. `ndkFetchEvents(filters)` — fires a relay subscription with
`closeOnEose: true` (via [`handleNdkFetchEvents`](www/ndk-worker.js:6316)).
The filters are `{ kinds:[1], authors, since, limit }`. This *does* go through
NDK's outbox routing. But there are two issues:
#### Issue A — Outbox data is not pre-warmed
`outboxTracker.trackUsers` is called inside `NDK.subscribe`, but it is
**fire-and-forget** relative to `EOSE`. The subscription starts immediately
against whatever relays are currently known (initially: your read relays only,
since the tracker is empty). When the outbox data arrives moments later,
`refreshRelayConnections` adds the new relays — but only for **long-lived**
subscriptions. For a `closeOnEose: true` `fetchEvents` call, the subscription
may `EOSE` and close on your read relays *before* the outbox tracker finishes
resolving the follow's kind 10002 lists. Result: you get posts only from
follows who happen to post to your read relays.
This is the core bug. The bootstrap windows in
[`FEED_BOOTSTRAP_WINDOWS_SECONDS`](www/feed.html:179) fire a burst of
short-lived `fetchEvents` calls that race the outbox tracker.
#### Issue B — The live subscription is fine, but late
[`ensureLiveFeedSubscription`](www/feed.html:547) creates a long-lived
`subscribe({ kinds:[1], authors, since })` with `closeOnEose:false`. This one
*does* benefit from `refreshRelayConnections` — once the outbox tracker
resolves, new relays get added and late posts stream in. But:
- It only covers events **after** `since` (newest known − 30s, or now − 1h).
Historical posts from follows on foreign relays that arrived *before* the
outbox data resolved are never backfilled.
- The bootstrap is what fills the initial feed view, and that's where Issue A
bites.
#### Issue C — Outbox tracker entries expire in 2 minutes
[`OutboxTracker`](www/ndk-core.bundle.js:42800): `entryExpirationTimeInMS: 2 * 60 * 1000`.
The tracker is an LRU with a 2-minute TTL. If the live subscription was created
and the tracker entries expired, a later `refreshRelayConnections` won't fire
because there's no `user:relay-list-updated` event to re-trigger it. The
subscription keeps running on whatever relays it had. This is mostly fine for
the live sub, but means any *new* `fetchEvents` call (e.g. "See More" or a new
follow added via [`ingestContactList`](www/feed.html:567)) starts cold again.
---
## What does NOT need to change
- **Do not** disable the outbox model or build a custom relay-fetching layer.
NDK already does this; you'd be reimplementing
[`getRelayListForUsers`](www/ndk-core.bundle.js:42662) and
[`chooseRelayCombinationForPubkeys`](www/ndk-core.bundle.js:31594).
- **Do not** manually connect to every follow's write relays in the main pool.
NDK's `pool.getRelay(url, true, true)` (temporary relay) already handles
per-subscription temporary connections via
[`calculateRelaySetsFromFilter`](www/ndk-core.bundle.js:33864). Adding them
permanently would bloat the main pool.
## What DOES need to change
The fix is to **pre-warm the outbox tracker before issuing the short-lived
bootstrap fetches**, so that by the time `fetchEvents` runs, NDK already knows
each follow's write relays and routes the subscription correctly.
NDK exposes this via `ndk.outboxTracker.trackUsers(pubkeys)`. Your worker does
not currently call it directly. The cleanest fix is a new worker RPC,
`warmOutbox(pubkeys)`, that calls `ndk.outboxTracker.trackUsers(pubkeys)` and
awaits it, plus a feed.html change to call it before
[`bootstrapFeedPosts`](www/feed.html:507).
---
## Unified Implementation Plan
The work is organized into phases that build on each other. Each phase is
self-contained and deliverable independently. The phases merge the feed
outbox fix, the relays.html performance fix, the outbox visualization, and
the Amethyst-parity relay management features into one coherent sequence.
### Phase 1 — Worker: expose outbox pre-warming + relay-event logging toggle
**Files:** [`www/ndk-worker.js`](www/ndk-worker.js)
- [ ] **1.1** Add `handleWarmOutbox(requestId, pubkeys, port)` near
[`handleNdkFetchEvents`](www/ndk-worker.js:6316). It should:
- Guard `if (!ndk?.outboxTracker) { respond empty ok }`.
- `await ndk.outboxTracker.trackUsers(pubkeys)`.
- Respond `{ type: 'warmOutboxResult', requestId, ok: true, count: pubkeys.length }`.
- Wrap in try/catch; never reject (pre-warming is best-effort).
- [ ] **1.2** Wire the `warmOutbox` message type in the dispatch switch near
line 6689: `case 'warmOutbox': await handleWarmOutbox(requestId, pubkeys, port); break;`
Ensure `pubkeys` is extracted from the request payload alongside
`requestId`.
- [ ] **1.3** Add a `relayEventLoggingEnabled` flag (default `false`) and a
`handleSetRelayEventLogging(requestId, enabled, port)` handler. When
`false`, [`broadcastRelayEvent`](www/ndk-worker.js:917) skips the
`broadcast()` call AND [`logRelayEvent`](www/ndk-worker.js:899) skips
`writeRelayEventToDb`. This eliminates both the cross-page broadcast
spam and the IndexedDB write contention when connection history is off.
- [ ] **1.4** Wire the `setRelayEventLogging` message type in the dispatch
switch.
- [ ] **1.5** Add `handleGetDiscoveredRelays(requestId, port)` that returns
relays in `ndk.pool.relays` that are *not* in `relayTypes` (your kind
10002 list). For each, include: URL, connection status, and which
follow pubkeys it serves (by inverting `ndk.outboxTracker.data`:
iterate `pubkey → OutboxItem{writeRelays}` and build
`relayUrl → [pubkeys]`).
- [ ] **1.6** Wire the `getDiscoveredRelays` message type in the dispatch
switch.
### Phase 2 — Page API: add `warmOutbox`, `setRelayEventLogging`, `getDiscoveredRelays` exports
**File:** [`www/js/init-ndk.mjs`](www/js/init-ndk.mjs)
- [ ] **2.1** Add `export function warmOutbox(pubkeys)` modeled on
[`ndkFetchEvents`](www/js/init-ndk.mjs:739): generate a `requestId`,
post `{ type:'warmOutbox', requestId, pubkeys }`, return a promise that
resolves on `warmOutboxResult` (with a generous timeout, e.g. 15s, that
resolves rather than rejects — pre-warm is best-effort).
- [ ] **2.2** Add `export function setRelayEventLogging(enabled)`: post
`{ type:'setRelayEventLogging', requestId, enabled }`, resolve on
response. Fire-and-forget is fine (no need to await).
- [ ] **2.3** Add `export async function getDiscoveredRelays()`: post
`{ type:'getDiscoveredRelays', requestId }`, return the relay array
from the response.
### Phase 3 — Feed: pre-warm outbox before bootstrap
**File:** [`www/feed.html`](www/feed.html)
- [ ] **3.1** Import `warmOutbox` in the existing import block (line 143).
- [ ] **3.2** In [`ingestContactList`](www/feed.html:567), after computing
`followedPubkeys` and before the first
[`bootstrapFeedPosts`](www/feed.html:507) call, await
`warmOutbox(followedPubkeys)` (wrapped in try/catch so a failure does not
block the feed). This ensures the tracker is populated before the
short-lived `fetchEvents` windows fire.
- [ ] **3.3** In [`ingestContactList`](www/feed.html:567), when
`newAuthors.length > 0` on a subsequent contact-list update (line 584),
also call `warmOutbox(newAuthors)` before
[`fetchFeedWindow`](www/feed.html:481) so newly-added follows are
routed correctly.
- [ ] **3.4** Optional but recommended: in
[`ensureLiveFeedSubscription`](www/feed.html:547), call
`warmOutbox(authors)` fire-and-forget before `subscribe(...)` so the live
sub's `refreshRelayConnections` has data ready. This is belt-and-suspenders
since `NDK.subscribe` already calls `trackUsers`, but doing it explicitly
avoids relying on the internal call's timing.
### Phase 4 — relays.html: connection history toggle + performance fixes
**File:** [`www/relays.html`](www/relays.html)
- [ ] **4.1** Add [`sidenav-sections.css`](www/css/sidenav-sections.css) to
the `<head>` (the page currently only includes `client.css`).
- [ ] **4.2** Replace the "Relay management options coming soon..." placeholder
in [`openNav`](www/relays.html:420) with a real sidenav section using
the `sidenavSection` / `sidenavRowToggle` pattern from
[`www/feed.html`](www/feed.html:99):
```html
<div id="divRelaySettings" class="sidenavSection">
<div class="sidenavSectionTitle">Relays</div>
<label class="sidenavRowToggle" for="chkShowConnectionHistory">
<span>Show connection history</span>
<input id="chkShowConnectionHistory" type="checkbox" />
</label>
</div>
```
- [ ] **4.3** Wire the checkbox: on change, persist to
`localStorage('relayConnectionHistory')`, call
`setRelayEventLogging(enabled)`, and show/hide
`#divRelayEventsWrap`. Default: **off**.
- [ ] **4.4** When the toggle is off, skip
[`loadRelayEventsFromDb`](www/relays.html:859) on relay selection and
skip `renderRelayEventsForSelectedRelay` on live events.
- [ ] **4.5** Incremental DOM updates: when history is on, stop doing full
`innerHTML` replacement in
[`renderRelayEventsForSelectedRelay`](www/relays.html:915). Append a
single `.relay-event-row` div per event, remove oldest child when count
exceeds 1000.
- [ ] **4.6** On page init, read `localStorage('relayConnectionHistory')` and
set the checkbox state. Call `setRelayEventLogging(enabled)` to sync
the worker. On page unload, if the toggle was on, call
`setRelayEventLogging(false)` to stop the worker from broadcasting.
### Phase 5 — relays.html: discovered relays table (visualize outbox)
**File:** [`www/relays.html`](www/relays.html)
- [ ] **5.1** Add a collapsible "Discovered Relays (Outbox)" section below the
main relay table. Columns: Relay URL | Connected | Serving (count of
follows) | Sample Follows (first 3 npubs). Read-only — no toggles, no
remove.
- [ ] **5.2** Call `getDiscoveredRelays()` on page init and on each
`refreshRelayData` cycle (every 5s). Render the results in the new
section.
- [ ] **5.3** Add a small "Outbox discovery" status line showing the outbox
pool relays (`purplepag.es`, `nos.lol`) and their connection state.
This is informational — "Outbox discovery: connected to purplepag.es,
nos.lol".
### Phase 6 — Verification (feed outbox + discovered relays)
- [ ] **6.1** Open `feed.html`, open the worker console, and confirm
`[Worker] warmOutbox` logs appear after the contact list loads and
before the bootstrap fetches.
- [ ] **6.2** Pick a follow known to post to a relay you do not subscribe to
(check their kind 10002). Confirm their posts now appear in the feed.
- [ ] **6.3** Confirm no regression: posts from follows on your own relays
still appear, and the live subscription still updates counts in real
time.
- [ ] **6.4** Open `relays.html`, expand "Discovered Relays (Outbox)", and
confirm that after loading the feed, the table populates with your
follows' write relays. Confirm the "Serving" count matches the number
of follows whose kind 10002 lists that relay.
- [ ] **6.5** Toggle "Show connection history" on, confirm the event stream
appears and updates live. Toggle off, confirm the stream hides and the
page feels faster. Check the worker console — no relay event
broadcasts when off.
---
## Why this is the minimal, correct fix
- It uses NDK's existing outbox infrastructure — no custom relay logic.
- It only changes the *timing* of when the tracker is populated, not the
routing algorithm.
- It is best-effort and non-blocking: if pre-warming fails or times out, the
feed falls back to the current behavior (your read relays only), which is
strictly a subset of what outbox routing would return.
- It directly addresses the race between short-lived `fetchEvents` EOSE and
the async `trackUsers` resolution.
## Risks / notes
- **Outbox pool relays:** NDK's default outbox pool is `purplepag.es` and
`nos.lol`. These are the relays it queries for kind 10002 lists. If they are
slow or rate-limiting, `trackUsers` can take up to its 1s timeout per batch
(see [`getRelayListForUsers`](www/ndk-core.bundle.js:42662) `timeout = 1e3`).
The 15s page-side timeout in Phase 2.1 accommodates this for ~400-pubkey
batches.
- **Large follow lists:** `trackUsers` batches in slices of 400
([`OutboxTracker.trackUsers`](www/ndk-core.bundle.js:42810)). A user with
1000+ follows will take a few batches. This is fine; the live sub still
works during pre-warm.
- **2-minute TTL:** If a user leaves the feed tab open for a long time, the
tracker entries expire. The live subscription keeps its already-added relays,
so this is not a problem for ongoing streaming. It only matters for *new*
`fetchEvents` calls, which is why Phase 3.3 re-warms for new follows.
---
## Footer & sidenav relay visualization — keeping it unchanged
You want the footer and sidenav relay indicators (managed by
[`www/js/relay-ui.mjs`](www/js/relay-ui.mjs)) to continue showing only your
subscribed relays, unchanged. I traced the data flow to confirm our changes
won't affect them, and identified one minor edge case to guard against.
### How the footer/sidenav gets relay data today
1. [`prepareRelayVisualizationData`](www/js/relay-ui.mjs:235) calls
`getRelayData()` → worker's
[`handleGetRelayData`](www/ndk-worker.js:6401).
2. `handleGetRelayData` builds the relay list from `relayTypes` (your kind
10002 map) — **not** from `ndk.pool.relays`. This is the key line:
[`relayTypes.size > 0 ? Array.from(relayTypes.entries()) : ...`](www/ndk-worker.js:6415).
So once your kind 10002 is loaded, the footer only ever shows your chosen
relays, regardless of what temporary relays are in the pool.
3. Activity carrots (read/write animations) are driven by
`ndkRelayActivity` window events, which come from
[`trackRelayRead`](www/ndk-worker.js:812) /
[`trackRelayWrite`](www/ndk-worker.js:834). These broadcast
`{ type: 'relayActivity', ... }` — a **separate** broadcast from
`broadcastRelayEvent` (the debug event stream).
### Impact of our changes on the footer/sidenav
| Change | Affects footer/sidenav? | Why |
|---|---|---|
| Phase 1: `warmOutbox` | **No** | Only calls `ndk.outboxTracker.trackUsers`. Does not touch `relayTypes` or `handleGetRelayData`. |
| Phase 1: `setRelayEventLogging` | **No** | Only gates `broadcastRelayEvent` / `logRelayEvent`. Does NOT gate `trackRelayRead` / `trackRelayWrite` (the activity broadcasts). Activity carrots continue working. |
| Phase 1: `getDiscoveredRelays` | **No** | New RPC, only called by relays.html. Does not touch `handleGetRelayData`. |
| Phase 3: feed pre-warm | **No** | Adds temporary relays to `ndk.pool`, but `handleGetRelayData` uses `relayTypes`, not the pool. |
| Phase 9: `relayConnectionFilter` | **No** | Filters which relays NDK connects to, but does not change `relayTypes` or `handleGetRelayData`. |
### The one edge case to guard against
[`handleGetRelayData`](www/ndk-worker.js:6415) has a fallback: when
`relayTypes.size === 0` (before your kind 10002 is fetched), it uses
`ndk.pool.relays.keys()`. After our outbox fix, temporary outbox relays will
be in `ndk.pool.relays`, so during this brief startup window the footer could
show discovered relays that aren't yours.
This is **transient** — `relayTypes` populates within seconds of login when
the worker fetches your kind 10002. But to be safe, we should guard the
fallback:
- [ ] **Guard 1 — Filter the fallback (optional).** In
[`handleGetRelayData`](www/ndk-worker.js:6417), when using the
`ndk.pool.relays.keys()` fallback, filter out temporary relays. NDK
relays added via `useTemporaryRelay` may have a way to detect their
temporary status. If not practical to detect, leave as-is — the
fallback is only used for a few seconds at startup and the visual
impact is minimal (a few extra icons that disappear once `relayTypes`
loads).
This guard is **optional** — the fallback is already transient and
self-correcting. It's listed here for completeness, not as a required step.
### What we will NOT change
- [`www/js/relay-ui.mjs`](www/js/relay-ui.mjs) — no changes. The footer and
sidenav rendering logic stays exactly as-is.
- [`handleGetRelayData`](www/ndk-worker.js:6401) — no changes to the primary
path (the `relayTypes` branch). It will continue to show only your kind
10002 relays.
- `trackRelayRead` / `trackRelayWrite` — no changes. Activity carrots
continue to fire on your subscribed relays.
---
## Relay UI implications — how outbox relays should appear
This is an important design question. Once outbox routing is active, NDK will
be connecting to **three categories** of relays, but your current UI only
surfaces one of them. Here is how they map to NDK's internal pools and what
your UI should do about each.
### The three categories of relays NDK uses
1. **Main pool — your read/both relays** (from your kind 10002). These are the
relays you chose in `relay.html`. Managed by
[`updateRelays`](www/ndk-worker.js:2088), which only adds read/both relays
to `ndk.pool` (write-only relays are deliberately excluded — line 2131).
These are what [`handleGetRelayData`](www/ndk-worker.js:6401) reports today.
2. **Outbox pool — relay-list discovery relays** (`purplepag.es`, `nos.lol`).
This is `ndk.outboxPool`, a separate `NDKPool` instance created in the NDK
constructor ([`www/ndk-core.bundle.js:43296`](www/ndk-core.bundle.js:43296)).
It is used *only* to fetch kind 10002 / kind 3 relay lists for authors. It
does **not** carry your feed posts. Your worker never touches it directly.
3. **Temporary relays — your follows' write relays.** When outbox routing
resolves a follow's kind 10002, NDK calls
[`pool.useTemporaryRelay`](www/ndk-core.bundle.js:35529) to connect to that
follow's write relays on demand, scoped to the subscription that needs them.
These relays are added to `ndk.pool.relays` but marked temporary — they are
removed after 30s of inactivity (`removeIfUnusedAfter = 3e4`). **These are
the relays that actually carry the missing posts.**
### What your UI currently shows
[`handleGetRelayData`](www/ndk-worker.js:6401) builds the relay list from
`relayTypes` (your kind 10002 map) or, as a fallback, `ndk.pool.relays.keys()`.
This means:
- **Category 1** is always shown. ✅
- **Category 2** (outbox pool) is never shown — it's a separate pool object
the worker doesn't read. This is arguably correct: these relays don't carry
your content, they're just a directory service. Showing them would clutter
the UI with relays the user didn't pick and can't edit.
- **Category 3** (temporary relays) is *partially* visible: they exist in
`ndk.pool.relays`, so the fallback path at line 6417 would include them.
But once `relayTypes` is populated (after the first kind 10002 fetch), the
fallback is not used and temporary relays are **hidden**. They also come and
go every 30s, which would make the footer flicker if shown.
### Recommended UI treatment
The cleanest mental model for users is: **"My relays" vs. "Discovered relays."**
- **Footer + sidenav (relay-ui.mjs):** Keep showing only **Category 1** — the
user's own read/both relays. This is what the user controls and what
[`relay.html`](www/relay.html) edits. Showing temporary outbox relays here
would be noisy (they appear/disappear every 30s) and misleading (the user
can't disconnect or edit them). The footer should answer "are *my* relays
healthy?", not "what is NDK connected to right now?"
→ **No change required to [`www/js/relay-ui.mjs`](www/js/relay-ui.mjs).**
The current `handleGetRelayData` already filters to `relayTypes`, which is
exactly right.
- **relay.html (the relay management page):** Consider adding a **read-only
"Discovered relays" section** below the editable relay table. This would
surface Category 3 (temporary relays currently in the pool) so a curious
user can see which of their follows' relays NDK is pulling from. This is
informational only — no add/remove/edit. Implementation would be a new
worker RPC `getDiscoveredRelays` that returns
`Array.from(ndk.pool.relays.values()).filter(r => r is temporary && not in
relayTypes)` with their status and which follow(s) they serve.
This is **optional for the initial outbox fix** and can be deferred. The
core feed fix (Phases 1–3) does not depend on it.
- **Outbox pool (Category 2):** Do not surface in the UI. These are
infrastructure relays (a directory service), not content relays. If a user
wants to know why relay-list discovery is slow, that's a debug/diagnostic
concern, not a relay-management concern.
### Why not show temporary relays in the footer
Concrete reasons, in case this comes up in review:
1. **Flicker.** Temporary relays are removed after 30s of inactivity
([`useTemporaryRelay`](www/ndk-core.bundle.js:35529)
`removeIfUnusedAfter = 3e4`). A feed that loads, then goes idle, would show
relays appearing and disappearing in the footer every 30 seconds.
2. **No user control.** The footer relays are clickable and the sidenav relays
are editable. Temporary relays can't be edited — they're driven by follows'
kind 10002 lists. Showing them as if they were user relays violates the
edit affordance.
3. **Scale.** A user following 500 people across 200 distinct relays would
see a footer full of icons they didn't choose. The footer is a health
indicator for *your* setup, not a live connection map.
4. **Redundancy.** [`setRelayActivityState`](www/js/relay-ui.mjs:145) already
shows read/write activity carrots on *your* relays when they carry traffic.
If a follow's post comes through a temporary relay and then gets re-fetched
on your relay (or vice versa), the activity indicator on your relay still
fires. The user sees activity; they don't need to see the temporary relay
itself.
### Summary
| Relay category | Pool | Carries feed posts? | Show in footer/sidenav? | Show in relay.html? |
|---|---|---|---|---|
| 1. Your read/both relays | `ndk.pool` (permanent) | Yes | Yes (current behavior) | Yes, editable (current) |
| 2. Outbox discovery relays | `ndk.outboxPool` | No (only kind 10002/3) | No | No |
| 3. Follows' write relays | `ndk.pool` (temporary, 30s TTL) | Yes | No | Optional: read-only section |
The feed outbox fix (Phases 1–3) requires **no changes** to the relay UI.
The optional "Discovered relays" view in relay.html can be a follow-up task.
---
## Amethyst comparison — what the most comprehensive Nostr client does
I examined [`~/lt/amethyst`](../amethyst) (both the Android `amethyst/` module and
the `desktopApp/` module, plus the shared `commons/` and `quartz/` libraries).
Amethyst is indeed the most thorough relay implementation in the Nostr
ecosystem. Here is what it does that your client does not, and what it would
take to bring your client to parity.
### Amethyst's relay set architecture
Amethyst does not have one relay list. It has **eleven distinct relay lists**,
each backed by a different Nostr event kind, each serving a different purpose.
From [`Account.kt`](../amethyst/amethyst/src/main/java/com/vitorpamplona/amethyst/model/Account.kt:330)
and the quartz event definitions:
| Relay list | NIP / Kind | Purpose | Your client has it? |
|---|---|---|---|
| NIP-65 Outbox/Inbox | 10002 | Where you publish / where others find you | ✅ Yes ([`relay.html`](www/relay.html)) |
| DM Inbox | 10050 (NIP-17) | Where others send you encrypted DMs | ❌ No |
| Search | 10007 (NIP-50) | Relays for full-text search queries | ❌ No |
| Blocked | 10006 (NIP-51) | Relays you refuse to connect to | ❌ No |
| Trusted | NIP-51 private | Relays you trust for Tor routing | ❌ No |
| Proxy | NIP-51 private | Relays to use as proxies | ❌ No |
| Broadcast | NIP-51 private | Extra relays to broadcast every event to | ❌ No |
| Indexer | NIP-51 private | Relays for kind/author indexing lookups | ❌ No |
| Relay Feeds | NIP-51 private | Relays that serve as feed sources | ❌ No |
| Private Storage | 10037 (NIP-37) | Private outbox relays for drafts | ❌ No |
| KeyPackage | 10051 (MIP-00) | MLS key package discovery relays | ❌ No |
On top of these **eleven published lists**, Amethyst computes **five derived
relay sets** by merging sources ([`Account.kt:415-462`](../amethyst/amethyst/src/main/java/com/vitorpamplona/amethyst/model/Account.kt:415)):
| Derived set | Sources | Used for |
|---|---|---|
| `homeRelays` | NIP-65 + private + local | Home feed subscriptions |
| `outboxRelays` | NIP-65 + private + local + broadcast | Where you publish your events |
| `dmRelays` | DM list + NIP-65 + private + local | DM send/receive |
| `notificationRelays` | NIP-65 read + local | Notifications/zap receipts |
| `followPlusAllMineWithIndex` | Follows' outboxes + your NIP-65 + indexer | Global feed with indexing |
| `followPlusAllMineWithSearch` | Follows' outboxes + your NIP-65 + search | Global feed with search |
| `defaultGlobalRelays` | Follows' outboxes + your NIP-65 | Global feed fallback |
The key insight: **Amethyst's "outbox model" is not just NDK's
`outboxTracker`**. It is a multi-layer system where:
1. **Your own relay lists** (11 kinds) define where *you* publish and what
*you* refuse.
2. **Derived sets** merge your lists with your follows' NIP-65 lists to
compute the actual relay set for each subscription type.
3. **Per-event routing** ([`Account.kt:1130-1290`](../amethyst/amethyst/src/main/java/com/vitorpamplona/amethyst/model/Account.kt:1130))
chooses relays dynamically: reactions go to the original author's inbox
relays + your outbox; replies go to the parent author's inbox + tagged
users' inboxes + your outbox; deletions go to your outbox + the original
event's relays.
### What Amethyst's UI shows
The desktop relay settings screen
([`RelayConfigTab.kt`](../amethyst/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/ui/relay/RelayConfigTab.kt:56))
has **five collapsible sections**, each independently editable:
1. **Connected Relays** — the live pool, with add/remove. Collapsed by default.
2. **NIP-65 Inbox/Outbox** (kind 10002) — editable, publishes a kind 10002.
3. **DM Relays** (kind 10050) — editable, publishes a kind 10050.
4. **Search Relays** (kind 10007) — editable, publishes a kind 10007.
5. **Blocked Relays** (kind 10006) — editable, publishes a kind 10006.
Each section has its own editor component
([`Nip65RelayEditor`](../amethyst/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/ui/relay/Nip65RelayEditor.kt:70),
[`DmRelayEditor`](../amethyst/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/ui/relay/DmRelayEditor.kt:59),
[`SearchRelayEditor`](../amethyst/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/ui/relay/SearchRelayEditor.kt:58),
[`BlockedRelayEditor`](../amethyst/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/ui/relay/BlockedRelayEditor.kt:57))
that signs and publishes the appropriate event kind.
There is also a **relay health system**
([`RelayHealthStore`](../amethyst/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relays/health/RelayHealthStore.kt:102),
[`ClassifyRelayHealth`](../amethyst/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relays/health/ClassifyRelayHealth.kt:53))
that monitors which relays in your lists are actually responding, classifies
them as healthy/dead/snoozed, and shows an "Unhealthy Relays" popup with a
one-click "remove from all lists" action via
[`RelayListMutator.removeFromAllUserLists`](../amethyst/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relays/health/RelayListMutator.kt:50).
### What it would take to add all of this to your client
This is a large undertaking. Here is a phased breakdown, ordered by value.
#### Tier 1 — High value, moderate effort (recommended next)
These directly improve the feed and core UX:
- [ ] **DM Relay List (kind 10050)** — Your client already does NIP-17 DMs
([`www/msg.html`](www/msg.html)). Without a kind 10050 list, other
clients don't know where to send you DMs. Add: worker fetch of kind
10050, a `dmRelayList` state, an editor section in `relay.html`, and
use it in the DM publish path. NDK does not manage this for you — it's
application-level.
- [ ] **Search Relay List (kind 10007)** — If you have or want NIP-50 search,
this lets the user configure which search relays to query. Add: worker
fetch, state, editor, and use in search queries.
- [ ] **Blocked Relay List (kind 10006)** — A relay-level blocklist (distinct
from your NIP-51 mute list which is pubkey-based). Add: worker fetch,
state, editor, and a `relayConnectionFilter` on the NDK instance so
NDK refuses to connect to blocked relays. This is the one that
integrates with NDK: set `ndk.relayConnectionFilter = (url) =>
!blockedRelays.has(url)` and the outbox tracker will skip blocked
relays ([`www/ndk-core.bundle.js:42835`](www/ndk-core.bundle.js:42835)).
#### Tier 2 — Medium value, moderate effort
- [ ] **Relay health monitoring** — Periodic ping/connection checks on your
configured relays, with a UI indicator for dead relays and a "remove"
action. Your worker already has
[`startRelayHealthCheck`](www/ndk-worker.js:1595) but it only
reconnects; it doesn't classify or surface health to the UI.
- [ ] **Per-event publish routing** — Instead of publishing to all your
relays, route events to the relevant relays: reactions to the original
author's inbox relays, replies to parent author + tagged users, etc.
This requires fetching the recipient's NIP-65 list (NDK's
`getWriteRelaysFor` can help) and is what makes Amethyst's events
reliably reach the right people.
- [ ] **Broadcast Relay List (NIP-51 private)** — Extra relays to always
include when publishing. Some users want their events on more relays
than their NIP-65 list. Add: NIP-44-encrypted kind 10051 list, state,
editor, merge into `outboxRelays` at publish time.
#### Tier 3 — Lower value, high effort (only if needed)
- [ ] **Trusted / Proxy Relay Lists (NIP-51 private)** — Only relevant if you
add Tor support. Amethyst uses these to decide which relays route
through Tor ([`TorRelayEvaluation`](../amethyst/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/tor/TorRelayEvaluation.kt:27)).
Without Tor, these are unnecessary.
- [ ] **Indexer / Relay Feeds Lists (NIP-51 private)** — Specialized relay
lists for kind-discovery and feed-source relays. Only useful if you
build features that need them (e.g., "find all kind 30023 authors").
- [ ] **Private Storage Relay List (kind 10037)** — For NIP-37 drafts. Only
relevant if you implement draft events.
- [ ] **KeyPackage Relay List (kind 10051)** — For MLS group messaging
(MIP-00). Only relevant if you implement the marmot/MLS protocol.
- [ ] **Custom Relay Sets (kind 30002)** — Amethyst's quartz library defines
[`RelaySetEvent`](../amethyst/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip51Lists/relaySets/RelaySetEvent.kt:44)
(kind 30002, NIP-51 parameterized) for user-defined named relay sets.
These are NIP-44-encrypted private lists with a title/description. This
is the "many different relay sets" feature you mentioned — users can
create arbitrary named sets (e.g., "My backup relays", "High-latency
relays") and reference them. This is a power-user feature; it requires
a full CRUD UI, NIP-44 encryption, and application-level logic to
consume the sets.
### Architectural recommendation
**Do not try to replicate Amethyst's architecture wholesale.** Amethyst is a
Kotlin/Compose app with a reactive state-flow system; your client is a
web-worker architecture built on NDK. The key differences:
1. **NDK already handles outbox routing** (the `outboxTracker` + temporary
relays). Amethyst reimplements this in
[`FollowListOutboxOrProxyRelays`](../amethyst/amethyst/src/main/java/com/vitorpamplona/amethyst/model/Account.kt:455)
because its quartz relay pool does not have NDK's automatic outbox model.
You do **not** need to reimplement follow-outbox computation — you need to
pre-warm it (Phases 1–3 above).
2. **NDK does not manage application-level relay lists** (DM, search, blocked,
etc.). These are your responsibility. Amethyst has 11 state classes for
these; you would need equivalent worker-side state and relay.html editor
sections.
3. **The highest-ROI additions** are the Blocked list (because it integrates
with NDK's `relayConnectionFilter` and improves outbox routing safety) and
the DM list (because you already do NIP-17 DMs but don't advertise where
to send them). These two alone would bring your client to functional
parity with Amethyst for most users.
4. **The "many relay sets" feature** (kind 30002) is the most complex and
lowest-ROI. Defer it unless you have a specific use case. Amethyst itself
does not heavily surface custom relay sets in its main UI — they exist in
the quartz library but the desktop `RelayConfigTab` only shows the 5
fixed sections.
### Summary: what to do now vs. later
| Priority | Task | Effort | Depends on feed fix? |
|---|---|---|---|
| **Now** | Feed outbox pre-warm (Phases 1–3) | Small | — |
| **Now** | Discovered relays read-only view in relay.html | Small | No |
| **Next** | Blocked relay list (kind 10006) + `relayConnectionFilter` | Medium | No |
| **Next** | DM relay list (kind 10050) | Medium | No |
| **Later** | Search relay list (kind 10007) | Medium | No |
| **Later** | Relay health UI | Medium | No |
| **Later** | Per-event publish routing | Large | No |
| **Later** | Broadcast relay list (NIP-51) | Medium | No |
| **Maybe** | Trusted/Proxy/Indexer/RelayFeeds/PrivateStorage/KeyPackage lists | Large | No |
| **Maybe** | Custom relay sets (kind 30002) | Large | No |
---
## relays.html page — current state and redesign plan
I examined [`www/relays.html`](www/relays.html) (1464 lines). Here is what it
does today, the problems you identified, and a concrete plan for the page.
### Current structure
The page has two main areas:
1. **Relay table** ([`createRelayTable`](www/relays.html:553), lines 553–748) —
A table with columns: Remove | Relay URL | Connected | Read | Write |
DM Inbox | Reads | Writes | Connection Time. Each row is a relay from your
kind 10002 list. The Read/Write checkboxes toggle the NIP-65 marker and
republish kind 10002. The DM Inbox checkbox toggles membership in your
kind 10050 list ([`publishDmInboxRelayList`](www/relays.html:1010)). There
is an add-relay row at the bottom with the same toggles. The table
refreshes every 5 seconds
([`setInterval(refreshRelayData, 5000)`](www/relays.html:1446)).
2. **Connection history stream** ([`divRelayEvents`](www/relays.html:260),
lines 859–939, 1200–1272) — A chat-like log of every relay frame
(REQ/EVENT/OK/EOSE/NOTICE/AUTH/etc.) for the selected relay. Events arrive
via `ndkRelayEvent` window events
([line 1405](www/relays.html:1405)), which the worker broadcasts on every
relay message via [`broadcastRelayEvent`](www/ndk-worker.js:917). The
worker persists each event to IndexedDB
([`writeRelayEventToDb`](www/ndk-worker.js:709)) and prunes to 200 per
relay every 60s ([`RELAY_EVENT_LOG_LIMIT = 200`](www/ndk-worker.js:307),
[`RELAY_EVENTS_PRUNE_INTERVAL_MS`](www/ndk-worker.js:311)). The page loads
history from IndexedDB on relay selection
([`loadRelayEventsFromDb`](www/relays.html:859)) and appends live events.
### Problem 1 — Connection history is heavy and slows the app
The performance issue has three layers:
1. **Worker-side: every relay frame is persisted to IndexedDB.**
[`logRelayEvent`](www/ndk-worker.js:899) calls
`void writeRelayEventToDb(entry)` on every single relay message. On an
active feed with 5+ relays, this is dozens of IndexedDB writes per second.
Each write opens a transaction, adds a row, and closes. This is fire-and-
forget but the I/O contention on the `relay-events` IndexedDB database
competes with the NDK cache adapter's own IndexedDB traffic.
2. **Worker-side: every relay frame is broadcast to all pages.**
[`broadcastRelayEvent`](www/ndk-worker.js:917) sends every event to every
connected port. Pages that don't care about relay events (feed.html,
msg.html, etc.) still receive and dispatch them. On `relays.html` itself,
the `ndkRelayEvent` listener
([line 1405](www/relays.html:1405)) calls `logRelayEvent` which appends to
the in-memory array and re-renders the entire event stream DOM
([`renderRelayEventsForSelectedRelay`](www/relays.html:915)) on every
frame.
3. **Page-side: full DOM re-render on every event.**
`renderRelayEventsForSelectedRelay` does `divRelayEvents.innerHTML = ...`
with up to 1000 entries ([line 1261](www/relays.html:1261) caps at 1000).
On a busy relay, this is a full innerHTML replacement every few
milliseconds. This is the direct cause of the slowdown you see.
### Solution: a toggle for connection history
The fix is to make the connection history opt-in, both on the page and in the
worker. The toggle goes in the relays.html sidenav, replacing the placeholder
text "Relay management options coming soon..."
([`openNav`](www/relays.html:420) sets `divSideNavBody.innerHTML` to that
string). It uses the same `sidenavSection` / `sidenavRowToggle` pattern as
[`www/feed.html`](www/feed.html:99) uses for "Autoplay videos":
```html
<div id="divRelaySettings" class="sidenavSection">
<div class="sidenavSectionTitle">Relays</div>
<label class="sidenavRowToggle" for="chkShowConnectionHistory">
<span>Show connection history</span>
<input id="chkShowConnectionHistory" type="checkbox" />
</label>
</div>
```
This requires adding the
[`sidenav-sections.css`](www/css/sidenav-sections.css) stylesheet to
`relays.html` (it is not currently included — the page uses only
`client.css`).
**Implementation:** This is covered by **Phase 1** (worker:
`setRelayEventLogging` RPC + flag that stops both broadcasting and IndexedDB
writes) and **Phase 4** (relays.html: sidenav toggle, show/hide the event
stream, skip DB loads, incremental DOM updates) in the unified plan above.
Default: **off** — the history is a debug tool.
### Problem 2 — Where to show outbox / discovered relays
You want to visualize the outbox model working. Today the table only shows
your kind 10002 relays. After the feed outbox fix (Phases 1–3), NDK will be
connecting to your follows' write relays as temporary relays — but they are
invisible on this page.
The right design is a **second table below the main one**, clearly labeled as
discovered/outbox relays, read-only.
These are implemented in **Phase 1** (worker RPC), **Phase 2** (page API),
and **Phase 5** (relays.html table) of the unified plan above. The worker
exposes `getDiscoveredRelays` (Phase 1.5–1.6), the page API wraps it (Phase
2.3), and relays.html renders it in a collapsible "Discovered Relays
(Outbox)" section with columns: Relay URL | Connected | Serving (count of
follows) | Sample Follows (first 3 npubs). Read-only — no toggles, no
remove. This is where you see the outbox model working: when you load the
feed, this table populates with your follows' write relays. An optional
"Outbox discovery" status line shows the outbox pool relays
(`purplepag.es`, `nos.lol`) and their connection state.
### Problem 3 — Setting different relay types (Amethyst-style sections)
Today the table has Read / Write / DM Inbox columns. To reach Amethyst-level
relay management, the page should grow **collapsible sections** for each
relay list kind, rather than cramming everything into one table's columns.
Recommended layout (top to bottom):
1. **Your Relays (NIP-65, kind 10002)** — the current table, with Read/Write
toggles. This is what you have now. Keep as-is.
2. **DM Inbox Relays (kind 10050)** — Currently a column in the main table.
Move to its own section so it's not conflated with NIP-65 read/write.
Each relay has a remove button. Add-relay input at the bottom. This is a
refactor of the existing DM Inbox column into a dedicated section.
3. **Search Relays (kind 10007)** — New section. Editable list. Publishes
kind 10007. Requires worker support to fetch/publish kind 10007 (new RPC).
4. **Blocked Relays (kind 10006)** — New section. Editable list. Publishes
kind 10006. Requires worker support, and setting
`ndk.relayConnectionFilter` so NDK refuses to connect to these relays.
5. **Discovered Relays (Outbox)** — Read-only, from Phase 5 above.
6. **Connection History** — The debug stream, behind the toggle from Phase 4.
Each section is collapsible (like Amethyst's
[`CollapsibleSection`](../amethyst/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/ui/relay/RelayConfigTab.kt:168))
so the page doesn't become overwhelming. Default state: NIP-65 expanded, DM
Inbox expanded, others collapsed.
### Phase 7 — relays.html: refactor DM Inbox into its own section
**File:** [`www/relays.html`](www/relays.html)
- [ ] **7.1** Remove the "DM Inbox" column from the main relay table
([`createRelayTable`](www/relays.html:553), line 591 header and line 642
body cell).
- [ ] **7.2** Add a new collapsible "DM Inbox Relays (kind 10050)" section
below the main table. Each relay in `dmInboxRelays` gets a row with a
remove button. Add-relay input at the bottom. Reuses the existing
[`publishDmInboxRelayList`](www/relays.html:1010) logic.
- [ ] **7.3** The add-relay row in the main table loses its DM Inbox toggle
(line 658). DM inbox relays are added from the DM Inbox section instead.
### Phase 8 — relays.html: Search relays section (kind 10007)
**Files:** [`www/ndk-worker.js`](www/ndk-worker.js), [`www/js/init-ndk.mjs`](www/js/init-ndk.mjs), [`www/relays.html`](www/relays.html)
- [ ] **8.1** Worker: add fetch/publish for kind 10007. Add a
`handleGetSearchRelayList` handler that fetches kind 10007 for the
current pubkey, and a `handlePublishSearchRelayList` handler that
publishes a kind 10007 event with the relay list.
- [ ] **8.2** Page API: add `getSearchRelayList()` and
`publishSearchRelayList(relays)` exports in init-ndk.mjs.
- [ ] **8.3** relays.html: Add a collapsible "Search Relays (kind 10007)"
section. Editable list with add/remove. Loads from
`getSearchRelayList()` on init, publishes via
`publishSearchRelayList()` on change.
### Phase 9 — relays.html: Blocked relays section (kind 10006) + relayConnectionFilter
**Files:** [`www/ndk-worker.js`](www/ndk-worker.js), [`www/js/init-ndk.mjs`](www/js/init-ndk.mjs), [`www/relays.html`](www/relays.html)
- [ ] **9.1** Worker: add fetch/publish for kind 10006. Add a
`handleGetBlockedRelayList` handler and a
`handlePublishBlockedRelayList` handler.
- [ ] **9.2** Worker: set `ndk.relayConnectionFilter` to skip blocked relays.
When the blocked relay list is loaded/updated, set
`ndk.relayConnectionFilter = (url) => !blockedRelays.has(normalizeRelayUrl(url))`.
This makes NDK's outbox tracker skip blocked relays
([`www/ndk-core.bundle.js:42835`](www/ndk-core.bundle.js:42835)) and
prevents temporary relay connections to blocked relays.
- [ ] **9.3** Page API: add `getBlockedRelayList()` and
`publishBlockedRelayList(relays)` exports.
- [ ] **9.4** relays.html: Add a collapsible "Blocked Relays (kind 10006)"
section. Editable list with add/remove.
### Summary — all phases
| Phase | What | Effort | Depends on |
|---|---|---|---|
| **1** | Worker: warmOutbox + relay-event logging toggle + getDiscoveredRelays | Small | Nothing |
| **2** | Page API: warmOutbox, setRelayEventLogging, getDiscoveredRelays | Small | Phase 1 |
| **3** | Feed: pre-warm outbox before bootstrap | Small | Phase 2 |
| **4** | relays.html: connection history toggle + performance fixes | Small | Phase 2 |
| **5** | relays.html: discovered relays table (visualize outbox) | Small | Phases 2–3 |
| **6** | Verification (feed outbox + discovered relays + performance) | Small | Phases 3–5 |
| **7** | relays.html: refactor DM Inbox into its own section | Medium | Nothing |
| **8** | relays.html: Search relays section (kind 10007) | Medium | Nothing |
| **9** | relays.html: Blocked relays section (kind 10006) + relayConnectionFilter | Medium | Nothing |
**Phases 1–6 are the core deliverable** — they fix the feed, fix the
performance, and let you see the outbox model working. Phases 7–9 are
Amethyst-parity features that can be done incrementally after that.
www/feed.html
+18 -1
View File
@@ -163,7 +163,8 @@
onUserSettings,
ensureMuteListLoaded,
isEventMuted,
addMute
addMute,
warmOutbox
} from './js/init-ndk.mjs';
import { HamburgerMorphing } from './hamburger_morphing/hamburger.mjs';
import { initFooterRelayStatus, updateFooterRelayStatus, initSidenavRelaySection, updateSidenavRelaySection, setRelayActivityState } from './js/relay-ui.mjs';
@@ -555,6 +556,10 @@ import { initPostCards } from './js/post-interactions2.mjs';
const now = Math.floor(Date.now() / 1000);
const since = newestKnown > 0 ? Math.max(0, newestKnown - 30) : (now - 3600);
// Fire-and-forget: NDK.subscribe already calls trackUsers internally,
// but doing it explicitly avoids relying on internal timing.
void warmOutbox(authors).catch(() => {});
subscribe(
{ kinds: [1], authors, since },
{ closeOnEose: false, cacheUsage: 'CACHE_FIRST' }
@@ -577,11 +582,23 @@ import { initPostCards } from './js/post-interactions2.mjs';
const allAuthors = Array.from(feedPubkeys);
if (!hasBootstrappedFeed) {
// Pre-warm the outbox tracker so NDK knows each follow's write relays
// before the short-lived bootstrap fetchEvents calls fire.
try {
await warmOutbox(followedPubkeys);
} catch (e) {
console.warn('[feed.html] warmOutbox failed (non-blocking):', e?.message || e);
}
await bootstrapFeedPosts(allAuthors);
return;
}
if (newAuthors.length > 0) {
try {
await warmOutbox(newAuthors);
} catch (e) {
console.warn('[feed.html] warmOutbox for new authors failed (non-blocking):', e?.message || e);
}
try {
await fetchFeedWindow(newAuthors, { limit: INITIAL_POSTS_LOAD });
renderFeed();
www/js/init-ndk.mjs
+153
View File
@@ -433,6 +433,37 @@ function handleWorkerMessage(event) {
pending.resolve(message.events || []);
}
}
} else if (message.type === 'warmOutboxResult') {
// Handle warmOutbox response — best-effort pre-warm; always resolves.
const pending = pendingRequests.get(message.requestId);
if (pending) {
pendingRequests.delete(message.requestId);
pending.resolve({
ok: !!message.ok,
count: message.count || 0,
skipped: !!message.skipped,
error: message.error || null
});
}
} else if (message.type === 'setRelayEventLoggingResult') {
// Handle setRelayEventLogging response — fire-and-forget, but resolve
// the promise if the caller chose to await it.
const pending = pendingRequests.get(message.requestId);
if (pending) {
pendingRequests.delete(message.requestId);
pending.resolve({
ok: !!message.ok,
enabled: !!message.enabled,
error: message.error || null
});
}
} else if (message.type === 'getDiscoveredRelaysResult') {
// Handle getDiscoveredRelays response
const pending = pendingRequests.get(message.requestId);
if (pending) {
pendingRequests.delete(message.requestId);
pending.resolve(message.relays || []);
}
} else if (message.type === 'fetchCachedProfileResult') {
// Handle cached profile response
const pending = pendingRequests.get(message.requestId);
@@ -763,6 +794,128 @@ export async function ndkFetchEvents(filters) {
});
}
/**
* Pre-warm the NDK outbox tracker for a set of follow pubkeys.
*
* This proactively resolves each follow's kind 10002 relay list via
* ndk.outboxTracker.trackUsers so that subsequent short-lived fetchEvents
* subscriptions route to the correct write relays instead of racing the
* tracker. Pre-warming is best-effort: the returned promise always resolves
* (never rejects). On timeout it resolves with { ok: false, timedOut: true }.
*
* @param {string[]} pubkeys - Array of hex pubkeys to pre-warm.
* @returns {Promise<{ok: boolean, count?: number, skipped?: boolean, timedOut?: boolean, error?: string}>}
*/
export async function warmOutbox(pubkeys) {
if (!ndkWorker) {
throw new Error('NDK worker not initialized. Call initNDKPage() first.');
}
const list = Array.isArray(pubkeys) ? pubkeys.filter(Boolean) : [];
if (list.length === 0) {
return { ok: true, count: 0, skipped: true };
}
return new Promise((resolve) => {
requestCounter++;
const requestId = `warmOutbox_${Date.now()}_${requestCounter}`;
pendingRequests.set(requestId, { resolve, reject: resolve });
// Best-effort: resolve (not reject) on timeout.
setTimeout(() => {
if (pendingRequests.has(requestId)) {
pendingRequests.delete(requestId);
resolve({ ok: false, timedOut: true });
}
}, 15000);
ndkWorker.port.postMessage({
type: 'warmOutbox',
requestId: requestId,
pubkeys: list
});
});
}
/**
* Toggle relay event logging in the worker.
*
* When enabled, the worker broadcasts relayEvent messages to all ports and
* persists relay events to IndexedDB. When disabled (default), both are
* skipped to avoid cross-page broadcast spam and IndexedDB write contention.
* trackRelayRead/trackRelayWrite (relayActivity events) are not affected.
*
* Fire-and-forget: the returned promise resolves when the worker acknowledges,
* but callers may ignore it.
*
* @param {boolean} enabled - Whether relay event logging should be on.
* @returns {Promise<{ok: boolean, enabled: boolean, error?: string}>}
*/
export function setRelayEventLogging(enabled) {
if (!ndkWorker) {
throw new Error('NDK worker not initialized. Call initNDKPage() first.');
}
return new Promise((resolve) => {
requestCounter++;
const requestId = `setRelayEventLogging_${Date.now()}_${requestCounter}`;
pendingRequests.set(requestId, { resolve, reject: resolve });
// Fire-and-forget timeout — resolve gracefully if the worker never
// acknowledges (it always should, but be safe).
setTimeout(() => {
if (pendingRequests.has(requestId)) {
pendingRequests.delete(requestId);
resolve({ ok: false, enabled: !!enabled, error: 'setRelayEventLogging timeout' });
}
}, 5000);
ndkWorker.port.postMessage({
type: 'setRelayEventLogging',
requestId: requestId,
enabled: !!enabled
});
});
}
/**
* Get the set of "discovered" relays currently in the NDK pool that are NOT
* part of the user's own kind 10002 relay list. These are temporary outbox
* relays NDK connected to in order to fetch events from followed authors who
* write to relays the user does not subscribe to.
*
* Each entry is { url, status, connected, servingPubkeys: string[] }.
*
* @returns {Promise<Array<{url: string, status: number, connected: boolean, servingPubkeys: string[]}>>}
*/
export async function getDiscoveredRelays() {
if (!ndkWorker) {
throw new Error('NDK worker not initialized. Call initNDKPage() first.');
}
return new Promise((resolve) => {
requestCounter++;
const requestId = `getDiscoveredRelays_${Date.now()}_${requestCounter}`;
pendingRequests.set(requestId, { resolve, reject: resolve });
// Resolve with [] on timeout — this is a read-only diagnostic call.
setTimeout(() => {
if (pendingRequests.has(requestId)) {
pendingRequests.delete(requestId);
resolve([]);
}
}, 5000);
ndkWorker.port.postMessage({
type: 'getDiscoveredRelays',
requestId: requestId
});
});
}
/**
* Fetch events from all relays in parallel
* Returns events grouped by relay URL
www/js/version.json
+3 -3
View File
@@ -1,5 +1,5 @@
{
"VERSION": "v0.7.39",
"VERSION_NUMBER": "0.7.39",
"BUILD_DATE": "2026-06-26T11:26:09.788Z"
"VERSION": "v0.7.40",
"VERSION_NUMBER": "0.7.40",
"BUILD_DATE": "2026-06-26T13:38:31.838Z"
}
www/ndk-worker.js
+224 -3
View File
@@ -304,6 +304,12 @@ const NUTZAP_MINTLIST_PREFETCH_BATCH = 50; // max authors per fetchEvents call
const NUTZAP_MINTLIST_PREFETCH_TTL_MS = 6 * 60 * 60 * 1000; // 6h freshness guard
// Relay activity tracking
// When false (default), broadcastRelayEvent skips the cross-page broadcast()
// and logRelayEvent skips the IndexedDB write. trackRelayRead/trackRelayWrite
// (which emit relayActivity events for footer/sidenav carrots) are NOT gated
// by this flag and always run. Toggled on by the relays page via the
// setRelayEventLogging RPC when the user opens connection history.
let relayEventLoggingEnabled = false;
const RELAY_EVENT_LOG_LIMIT = 200;
const RELAY_EVENTS_DB_NAME = 'relay-events';
const RELAY_EVENTS_DB_VERSION = 2;
@@ -908,13 +914,24 @@ function logRelayEvent(relayUrl, eventType, data = undefined, side = 'relay', ti
data: serializeRelayEventData(data)
};
// Fire-and-forget persistence so relay handling is never blocked by IndexedDB I/O.
void writeRelayEventToDb(entry);
// Only persist to IndexedDB when relay event logging is enabled.
// When the relays page (connection history) is closed, this avoids
// IndexedDB write contention from the high-frequency relay event stream.
if (relayEventLoggingEnabled) {
// Fire-and-forget persistence so relay handling is never blocked by IndexedDB I/O.
void writeRelayEventToDb(entry);
}
return entry;
}
function broadcastRelayEvent(relayUrl, eventType, data = undefined, side = 'relay') {
// Skip the cross-page broadcast entirely when relay event logging is off.
// This eliminates relayEvent message spam to every open tab/page when the
// connection-history UI is not visible. trackRelayRead/trackRelayWrite
// (relayActivity events) are intentionally NOT gated and still broadcast.
if (!relayEventLoggingEnabled) return;
const entry = logRelayEvent(relayUrl, eventType, data, side, Date.now());
if (!entry) return;
@@ -6345,6 +6362,196 @@ async function handleNdkFetchEvents(requestId, filters, port) {
}
}
// Handle outbox pre-warm request — proactively resolves each follow's kind
// 10002 relay list via ndk.outboxTracker.trackUsers so that subsequent
// short-lived fetchEvents subscriptions route to the correct write relays
// instead of racing the tracker. Pre-warming is best-effort: this handler
// never throws; it always responds with an ok/skipped/error payload.
async function handleWarmOutbox(requestId, pubkeys, port) {
try {
if (!ndk?.outboxTracker) {
port.postMessage({
type: 'warmOutboxResult',
requestId,
ok: true,
count: 0,
skipped: true
});
return;
}
const list = Array.isArray(pubkeys) ? pubkeys.filter(Boolean) : [];
if (list.length === 0) {
port.postMessage({
type: 'warmOutboxResult',
requestId,
ok: true,
count: 0,
skipped: true
});
return;
}
await ndk.outboxTracker.trackUsers(list);
port.postMessage({
type: 'warmOutboxResult',
requestId,
ok: true,
count: list.length
});
} catch (err) {
console.warn('[Worker] warmOutbox failed:', err?.message || err);
port.postMessage({
type: 'warmOutboxResult',
requestId,
ok: false,
error: err?.message || String(err)
});
}
}
// Handle relay event logging toggle. When enabled, broadcastRelayEvent will
// emit relayEvent messages to all ports and logRelayEvent will persist entries
// to IndexedDB. When disabled (default), both are skipped to avoid cross-page
// broadcast spam and IndexedDB write contention. trackRelayRead/trackRelayWrite
// are NOT affected by this flag.
async function handleSetRelayEventLogging(requestId, enabled, port) {
try {
relayEventLoggingEnabled = !!enabled;
port.postMessage({
type: 'setRelayEventLoggingResult',
requestId,
ok: true,
enabled: relayEventLoggingEnabled
});
} catch (err) {
console.warn('[Worker] setRelayEventLogging failed:', err?.message || err);
port.postMessage({
type: 'setRelayEventLoggingResult',
requestId,
ok: false,
error: err?.message || String(err),
enabled: relayEventLoggingEnabled
});
}
}
// Handle getDiscoveredRelays request — returns relays currently in the NDK
// pool that are NOT part of the user's own kind 10002 relay list (relayTypes).
// These are temporary outbox relays NDK connected to in order to fetch events
// from followed authors who write to relays the user does not subscribe to.
// For each discovered relay we also report which followed pubkeys it serves,
// by inverting ndk.outboxTracker.data (pubkey -> OutboxItem{writeRelays}).
async function handleGetDiscoveredRelays(requestId, port) {
try {
if (!ndk?.pool?.relays) {
port.postMessage({
type: 'getDiscoveredRelaysResult',
requestId,
relays: []
});
return;
}
// Build the set of the user's own relay URLs (normalized) from
// relayTypes (kind 10002). These are the "configured" relays; anything
// else in the pool is a discovered/temporary outbox relay.
const ownRelays = new Set();
for (const url of relayTypes.keys()) {
const normalized = normalizeRelayUrl(url);
if (normalized) ownRelays.add(normalized);
}
// Build relayUrl -> [pubkeys] map by inverting the outbox tracker.
// ndk.outboxTracker.data is an LRUCache (typescript-lru-cache).
// We try a few iteration strategies to be resilient to API differences.
const relayServesPubkeys = new Map(); // normalizedUrl -> Set<pubkey>
try {
const trackerData = ndk?.outboxTracker?.data;
if (trackerData) {
const collectForPubkey = (pubkey) => {
try {
const outboxItem = trackerData.get(pubkey);
const writeRelays = outboxItem?.writeRelays;
if (!writeRelays) return;
for (const r of writeRelays) {
const rUrl = r?.url ? normalizeRelayUrl(r.url) : (typeof r === 'string' ? normalizeRelayUrl(r) : null);
if (!rUrl) continue;
if (!relayServesPubkeys.has(rUrl)) {
relayServesPubkeys.set(rUrl, new Set());
}
relayServesPubkeys.get(rUrl).add(pubkey);
}
} catch (_) {
// ignore per-pubkey errors
}
};
// Strategy 1: standard Map-like entries()
let iterated = false;
try {
if (typeof trackerData.entries === 'function') {
for (const [pubkey] of trackerData.entries()) {
collectForPubkey(pubkey);
}
iterated = true;
}
} catch (_) {}
// Strategy 2: keys() + get()
if (!iterated) {
try {
if (typeof trackerData.keys === 'function') {
for (const pubkey of trackerData.keys()) {
collectForPubkey(pubkey);
}
iterated = true;
}
} catch (_) {}
}
// Strategy 3: iterate as Map directly
if (!iterated && trackerData instanceof Map) {
for (const pubkey of trackerData.keys()) {
collectForPubkey(pubkey);
}
}
}
} catch (err) {
console.warn('[Worker] getDiscoveredRelays: outbox tracker inversion failed:', err?.message || err);
}
const relays = [];
for (const relay of ndk.pool.relays.values()) {
const normalized = normalizeRelayUrl(relay?.url);
if (!normalized) continue;
// Skip relays that are part of the user's own kind 10002 list.
if (ownRelays.has(normalized)) continue;
const servingSet = relayServesPubkeys.get(normalized);
relays.push({
url: relay.url,
status: relay.status,
connected: relay.status >= 5,
servingPubkeys: servingSet ? Array.from(servingSet) : []
});
}
port.postMessage({
type: 'getDiscoveredRelaysResult',
requestId,
relays
});
} catch (err) {
console.warn('[Worker] getDiscoveredRelays failed:', err?.message || err);
port.postMessage({
type: 'getDiscoveredRelaysResult',
requestId,
relays: []
});
}
}
function handleSignResponse(requestId, result, error) {
const pending = pendingSignRequests.get(requestId);
@@ -6650,7 +6857,9 @@ self.onconnect = (event) => {
playlistIdentifier,
ownerPubkey,
playlistKind,
eventIds
eventIds,
pubkeys,
enabled
} = e.data;
switch(type) {
@@ -6690,6 +6899,18 @@ self.onconnect = (event) => {
await handleNdkFetchEvents(requestId, filters, port);
break;
case 'warmOutbox':
await handleWarmOutbox(requestId, pubkeys, port);
break;
case 'setRelayEventLogging':
await handleSetRelayEventLogging(requestId, enabled, port);
break;
case 'getDiscoveredRelays':
await handleGetDiscoveredRelays(requestId, port);
break;
case 'setActiveSigningPort':
activeSigningPort = port;
console.log('[Worker] Active signing port set explicitly by page');
www/relays.html
+79 -5
View File
@@ -7,6 +7,7 @@
<title>RELAYS</title>
<link rel="stylesheet" href="./css/client.css" />
<link rel="stylesheet" href="./css/sidenav-sections.css" />
<!-- Initialize theme BEFORE any components load -->
<script>
@@ -332,7 +333,7 @@
/* ================================================================
IMPORTS
================================================================ */
import { initNDKPage, getPubkey, injectHeaderAvatar, disconnect, getRelayData, getRelayStats, reconnectRelay, publishEvent, getVersion, updateVersionDisplay, ndkFetchEvents } from './js/init-ndk.mjs';
import { initNDKPage, getPubkey, injectHeaderAvatar, disconnect, getRelayData, getRelayStats, reconnectRelay, publishEvent, getVersion, updateVersionDisplay, ndkFetchEvents, setRelayEventLogging } from './js/init-ndk.mjs';
import { HamburgerMorphing } from "./hamburger_morphing/hamburger.mjs";
import { initFooterRelayStatus, updateFooterRelayStatus, initSidenavRelaySection, updateSidenavRelaySection, setRelayActivityState } from './js/relay-ui.mjs';
@@ -370,6 +371,7 @@ const versionInfo = await getVersion();
let selectedRelayUrl = null; // Relay row currently selected for debug event history
const relayEventHistory = new Map(); // relay.url -> [{ timestamp, rawTimestamp, eventType, side, summary }]
const relayUiStats = new Map(); // relay.url -> { readCount, writeCount }
let connectionHistoryEnabled = false; // Toggled via sidenav; gates live relay event logging
const RELAY_EVENTS_DB_NAME = 'relay-events';
const RELAY_EVENTS_DB_VERSION = 1;
@@ -417,7 +419,41 @@ const versionInfo = await getVersion();
if (hamburgerInstance) {
hamburgerInstance.animateTo('arrow_left');
}
divSideNavBody.innerHTML = "Relay management options coming soon...";
divSideNavBody.innerHTML = `
<div id="divRelaySettings" class="sidenavSection">
<div class="sidenavSectionTitle">Relays</div>
<label class="sidenavRowToggle" for="chkShowConnectionHistory">
<span>Show connection history</span>
<input id="chkShowConnectionHistory" type="checkbox" />
</label>
</div>
`;
// Wire up the connection history toggle
const chkShowConnectionHistory = document.getElementById('chkShowConnectionHistory');
if (chkShowConnectionHistory) {
chkShowConnectionHistory.checked = localStorage.getItem('relayConnectionHistory') === 'true';
chkShowConnectionHistory.addEventListener('change', () => {
const checked = chkShowConnectionHistory.checked;
connectionHistoryEnabled = checked;
localStorage.setItem('relayConnectionHistory', checked);
setRelayEventLogging(checked);
const wrap = document.getElementById('divRelayEventsWrap');
if (wrap) {
wrap.style.display = checked ? '' : 'none';
}
if (checked) {
if (selectedRelayUrl) {
loadRelayEventsFromDb(selectedRelayUrl);
}
renderRelayEventsForSelectedRelay();
} else {
if (divRelayEvents) {
divRelayEvents.innerHTML = '';
}
}
});
}
// Initialize version bar buttons when sidenav opens (lazy load)
if (!logoutHamburger) {
@@ -942,7 +978,9 @@ const versionInfo = await getVersion();
if (!relayUrl) return;
selectedRelayUrl = normalizeRelayUrlForDb(relayUrl);
await createRelayTable(currentRelayList);
await loadRelayEventsFromDb(selectedRelayUrl);
if (connectionHistoryEnabled) {
await loadRelayEventsFromDb(selectedRelayUrl);
}
renderRelayEventsForSelectedRelay();
};
@@ -1267,7 +1305,25 @@ const versionInfo = await getVersion();
}
if (selectedRelayUrl === normalizedRelayUrl) {
renderRelayEventsForSelectedRelay();
// Incremental append instead of full re-render
const sideClass = side === 'client' ? 'client' : 'relay';
const label = side === 'client' ? 'CLIENT ➜ RELAY' : 'RELAY ➜ CLIENT';
const dataPart = summary ? `\n${summary}` : '';
const text = `[${timestamp}] ${label} ${eventType.toUpperCase()}${dataPart}`;
const row = document.createElement('div');
row.className = `relay-event-row ${sideClass}`;
const bubble = document.createElement('div');
bubble.className = 'relay-event-bubble';
bubble.textContent = text;
row.appendChild(bubble);
divRelayEvents.appendChild(row);
// Remove oldest if over 1000
while (divRelayEvents.children.length > 1000) {
divRelayEvents.removeChild(divRelayEvents.firstChild);
}
divRelayEvents.scrollTop = divRelayEvents.scrollHeight;
}
};
@@ -1403,6 +1459,7 @@ const versionInfo = await getVersion();
// Note: We need to listen on the worker port, not window
// This will be set up via a custom event from init-ndk.mjs
window.addEventListener('ndkRelayEvent', (event) => {
if (!connectionHistoryEnabled) return;
const { relayUrl, event: eventType, data, timestamp, side } = event.detail;
logRelayEvent(relayUrl, eventType, data, timestamp, side);
});
@@ -1444,7 +1501,24 @@ const versionInfo = await getVersion();
// Refresh relay table every 5 seconds
setInterval(refreshRelayData, 5000);
// Sync connection history toggle state from localStorage
connectionHistoryEnabled = localStorage.getItem('relayConnectionHistory') === 'true';
setRelayEventLogging(connectionHistoryEnabled);
if (!connectionHistoryEnabled) {
const wrap = document.getElementById('divRelayEventsWrap');
if (wrap) {
wrap.style.display = 'none';
}
}
// Stop the worker from broadcasting relay events when the page is closing
window.addEventListener('beforeunload', () => {
if (connectionHistoryEnabled) {
setRelayEventLogging(false);
}
});
console.log('[relay2.html] Initialization complete');
} catch (error) {
console.error('[relay2.html] Initialization failed:', error);