Files
c-relay-pg/plans/profile_fetch_latency_test_results.md
T
Laan Tungir 50653dc86a v2.1.38 - Custom backfill feature: ad-hoc backfill jobs with arbitrary NIP-01 filters + admin UI isolation
- New caching_custom_backfill_jobs and caching_custom_backfill_batches tables
- Admin API (POST create/cancel, GET list) at admin/api/custom_backfill.php
- Admin UI form with preset buttons and job status table on Backfill page
- Caching daemon module (custom_backfill.c) processes jobs independently
- pg_inbox functions for job/batch claim, progress update, and cancel
- Three batch modes: author-batched, id-batched, time-window scan
- Fixed done_batches overcount on job completion
- Admin config now environment-variable driven (C_RELAY_DB_*)
- admin/serve.sh supports separate instances for different databases
- make_and_restart_relay.sh only kills relay on target port, not all relays
2026-08-07 14:49:13 -04:00

3.9 KiB

Profile Fetch Latency Test Results — Option A vs Profiles Table

Test Environment

  • Relay: C-Relay-PG on ws://localhost:7777 (PostgreSQL backend)
  • Database: 1,070,741 kind-1 events, 8,734 kind-0 events, 8,734 profiles
  • Client: Python 3.13, websockets 15.0.1, localhost (no network latency)
  • Test script: tests/profile_fetch_latency_test.py

Option A — Two Standard REQs (measured)

Metric Time
Phase 1: kind-1 REQ
REQ send → first EVENT 20.85 ms
REQ send → EOSE 23.81 ms
Inter-event gap (min/avg/max) 0.01 / 0.01 / 0.54 ms
Events received 200
Unique authors 115
Phase 2: kind-0 REQ
REQ send → first EVENT 83.49 ms
REQ send → EOSE 83.63 ms
Profiles received 10 of 115 (8.7%)
End-to-end (kind-1 REQ → kind-0 EOSE) 107.67 ms

Key observations

  1. Phase 1 is fast (24ms EOSE for 200 events) — the local relay streams events at ~0.01ms inter-arrival.
  2. Phase 2 is slower (83ms to first kind-0 event) — the relay must scan the events table, parse JSON content, verify signatures, etc. Even though it's a local relay, the full event pipeline has overhead.
  3. Only 8.7% of authors had kind-0 events — most kind-1 authors never published a kind-0 to this relay. This is the real bottleneck, not latency.

Profiles Table Query Speed (measured)

EXPLAIN ANALYZE SELECT pubkey, name, display_name, about, picture, banner,
                    nip05, website, lud16, lud06
             FROM profiles
            WHERE pubkey = ANY($1::text[]);
Metric Time
Planning Time 12.70 ms (one-time, cached)
Execution Time 2.21 ms
Index type Primary key (Bitmap Index Scan)
Rows returned 7 (of 7 requested)

The profiles table query is ~40x faster than the kind-0 REQ path (2.2ms vs 83ms).


Comparison: Option A vs Profiles Table

Aspect Option A (kind-0 REQ) Profiles table query
Latency 83ms to first event 2.2ms
Payload size Full kind-0 event (sig, tags, raw content) Only parsed fields (name, picture, etc.)
Client work Must parse kind-0 content JSON, handle replaceable dedup Ready-to-render fields
Missing profiles Returns nothing for missing pubkeys Returns nothing for missing pubkeys
Protocol Standard Nostr WebSocket Custom (HTTP or WebSocket)
Portability Works with any relay Relay-specific

The Real Problem

The test revealed a more fundamental issue than latency: most kind-1 authors don't have kind-0 events on this relay. Only 8.7% of authors had profiles. This means:

  1. Option A would show blank names/pictures for 91% of posts.
  2. Profiles table has the same data (it's synced from kind-0 events), so it has the same gap.
  3. The caching daemon doesn't cache kind-0 by default — it caches whatever caching_kinds is configured to.

To make either option viable, the relay needs more kind-0 data

The caching daemon should include kind-0 in its cached kinds, or a separate profile-caching pass should backfill kind-0 events for all known pubkeys.


Recommendation

Option B (batched HTTP profiles from the profiles table) is the clear winner for a no-storage local client:

  • 2.2ms query time vs 83ms for kind-0 REQ
  • Ready-to-render fields (no client-side kind-0 JSON parsing)
  • Smaller payload (no sig, tags, raw content)
  • Single HTTP call for all authors at once

But it requires:

  1. A new HTTP endpoint (e.g., POST /api/profiles accepting {"pubkeys":[...]})
  2. The caching daemon to include kind-0 in its cache kinds (or a separate profile backfill pass)

Option C (relay auto-hydrates profiles before EOSE) would be even better for the client (one subscription, everything arrives before EOSE), but requires more relay-side work.