Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a766f08738 | ||
|
|
23c37abe3a | ||
|
|
5b3ccfd1f4 | ||
|
|
38a027e626 | ||
|
|
a042aca524 | ||
|
|
20902e1c5d | ||
|
|
7233818da9 | ||
|
|
86cbf7ff16 | ||
|
|
3035e71ca5 | ||
|
|
179b160cf3 | ||
|
|
aedd22436e | ||
|
|
8fdc362a8b | ||
|
|
67cf3973c5 | ||
|
|
c02ae2be5b | ||
|
|
fd3efdd01f | ||
|
|
232f93c16f | ||
|
|
9a9365dcfa | ||
|
|
5577b57149 | ||
|
|
22c943d495 | ||
|
|
7e3d9b6825 | ||
|
|
547d22d09f | ||
|
|
ce9ae5a488 | ||
|
|
9b248b740f | ||
|
|
0f77aeb72b | ||
|
|
bb64651a0c | ||
|
|
17be9c2b03 | ||
|
|
f396c622b6 | ||
|
|
cbd260c1ae | ||
|
|
bc9ac290ab | ||
|
|
192eeb248d | ||
|
|
da6c505420 | ||
|
|
c077c209e5 | ||
|
|
9bf02702c2 | ||
|
|
0920cc092d | ||
|
|
a89460be5d | ||
|
|
ff2a3aa335 | ||
|
|
94b61e8a7c | ||
|
|
b0c0754e83 | ||
|
|
3265e3d114 | ||
|
|
927659ece1 | ||
|
|
b6ff4150b4 | ||
|
|
63bc526163 | ||
|
|
a1f712236a | ||
|
|
06e6c17b7b | ||
|
|
0751a7c55c | ||
|
|
f1728932a9 | ||
|
|
ef8bdef2a8 | ||
|
|
c11a8ba292 | ||
|
|
0f124fe575 | ||
|
|
6b20452fab | ||
|
|
10c19bc243 | ||
|
|
3d7aa2196f | ||
|
|
a416c3f275 | ||
|
|
bba9baabc3 | ||
|
|
31187c4c4f | ||
|
|
55f862b879 | ||
|
|
76c9b3fcf0 | ||
|
|
929bd09164 | ||
|
|
d17f1dd8d5 | ||
|
|
207a949835 | ||
|
|
d08f4e4221 | ||
|
|
c96736fa6a | ||
|
|
f8ec4ae924 | ||
|
|
fe7304ac7f | ||
|
|
e5d39c984b | ||
|
|
5e45f21e35 | ||
|
|
5321a238b8 | ||
|
|
083bc14972 | ||
|
|
11aaccba9b | ||
|
|
bd1bbd763d | ||
|
|
2bd7aa5a10 | ||
|
|
361912ec85 | ||
|
|
0de491382e | ||
|
|
3148bbbee7 | ||
|
|
3965ba04d8 | ||
|
|
b96af938bd | ||
|
|
89c8248013 | ||
|
|
d8f477c6cf | ||
|
|
040eeadb13 | ||
|
|
83f8b0ab88 | ||
|
|
b82f7eaaf3 | ||
|
|
0dc5b75d7c | ||
|
|
e94b1a81e3 | ||
|
|
1adabdbc4e |
@@ -11,3 +11,4 @@ copy_executable_local.sh
|
||||
nostr_login_lite/
|
||||
style_guide/
|
||||
nostr-tools
|
||||
.test_keys
|
||||
@@ -1,6 +1,3 @@
|
||||
[submodule "nostr_core_lib"]
|
||||
path = nostr_core_lib
|
||||
url = https://git.laantungir.net/laantungir/nostr_core_lib.git
|
||||
[submodule "c_utils_lib"]
|
||||
path = c_utils_lib
|
||||
url = ssh://git@git.laantungir.net:2222/laantungir/c_utils_lib.git
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
ADMIN_PUBKEY='6a04ab98d9e4774ad806e302dddeb63bea16b5cb5f223ee77478e861bb583eb3'
|
||||
SERVER_PRIVKEY='1111111111111111111111111111111111111111111111111111111111111111'
|
||||
+11
-11
@@ -1,12 +1,12 @@
|
||||
|
||||
# AGENTS.md - AI Agent Integration Guide for Architect Mode
|
||||
|
||||
**Project-Specific Information for AI Agents Working with C-Relay in Architect Mode**
|
||||
**Project-Specific Information for AI Agents Working with C-Relay-PG in Architect Mode**
|
||||
|
||||
## Critical Architecture Understanding
|
||||
|
||||
### System Architecture Overview
|
||||
C-Relay implements a **unique event-based configuration architecture** that fundamentally differs from traditional Nostr relays:
|
||||
C-Relay-PG implements a **unique event-based configuration architecture** that fundamentally differs from traditional Nostr relays:
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
|
||||
@@ -51,9 +51,9 @@ C-Relay implements a **unique event-based configuration architecture** that fund
|
||||
## Architectural Decision Analysis
|
||||
|
||||
### Configuration System Design
|
||||
**Traditional Approach vs C-Relay:**
|
||||
**Traditional Approach vs C-Relay-PG:**
|
||||
```
|
||||
Traditional: C-Relay:
|
||||
Traditional: C-Relay-PG:
|
||||
config.json → kind 33334 events
|
||||
ENV variables → cryptographically signed tags
|
||||
File watching → database polling/restart
|
||||
@@ -119,18 +119,18 @@ File watching → database polling/restart
|
||||
```
|
||||
Developer Machine:
|
||||
├── ./make_and_restart_relay.sh
|
||||
├── build/c_relay_x86
|
||||
├── build/c_relay_pg_x86
|
||||
├── build/<relay_pubkey>.db
|
||||
└── relay.log
|
||||
```
|
||||
|
||||
### Production SystemD Deployment
|
||||
```
|
||||
/opt/c-relay/:
|
||||
├── c_relay_x86
|
||||
/opt/c-relay-pg/:
|
||||
├── c_relay_pg_x86
|
||||
├── <relay_pubkey>.db
|
||||
├── systemd service (c-relay.service)
|
||||
└── c-relay user isolation
|
||||
├── systemd service (c-relay-pg.service)
|
||||
└── c-relay-pg user isolation
|
||||
```
|
||||
|
||||
### Container Deployment Architecture
|
||||
@@ -144,7 +144,7 @@ Container:
|
||||
|
||||
### Reverse Proxy Architecture
|
||||
```
|
||||
Internet → Nginx/HAProxy → C-Relay
|
||||
Internet → Nginx/HAProxy → C-Relay-PG
|
||||
├── WebSocket upgrade handling
|
||||
├── SSL termination
|
||||
└── Rate limiting
|
||||
@@ -292,7 +292,7 @@ Admin Signs Event → WebSocket Submit → Validate → Store → Restart Requir
|
||||
**Decision**: Same port serves both protocols
|
||||
**Consequences**: Simplified deployment, protocol detection overhead, libwebsockets dependency
|
||||
|
||||
These architectural decisions form the foundation of C-Relay's unique approach to Nostr relay implementation and should be carefully considered when planning extensions or modifications.
|
||||
These architectural decisions form the foundation of C-Relay-PG's unique approach to Nostr relay implementation and should be carefully considered when planning extensions or modifications.
|
||||
**
|
||||
|
||||
[Response interrupted by a tool use result. Only one tool may be used at a time and should be placed at the end of the message.]
|
||||
Executable
BIN
Binary file not shown.
@@ -1,6 +1,6 @@
|
||||
# AGENTS.md - AI Agent Integration Guide
|
||||
|
||||
**Project-Specific Information for AI Agents Working with C-Relay**
|
||||
**Project-Specific Information for AI Agents Working with C-Relay-PG**
|
||||
|
||||
## Critical Build Commands
|
||||
|
||||
@@ -15,9 +15,9 @@
|
||||
- Starts relay in background with proper logging
|
||||
|
||||
### Architecture-Specific Binary Outputs
|
||||
- **x86_64**: `./build/c_relay_x86`
|
||||
- **ARM64**: `./build/c_relay_arm64`
|
||||
- **Other**: `./build/c_relay_$(ARCH)`
|
||||
- **x86_64**: `./build/c_relay_pg_x86`
|
||||
- **ARM64**: `./build/c_relay_pg_arm64`
|
||||
- **Other**: `./build/c_relay_pg_$(ARCH)`
|
||||
|
||||
### Database File Naming Convention
|
||||
- **Format**: `<relay_pubkey>.db` (NOT `.nrdb` as shown in docs)
|
||||
@@ -75,10 +75,10 @@
|
||||
### Process Management
|
||||
```bash
|
||||
# Kill existing relay processes
|
||||
pkill -f "c_relay_"
|
||||
pkill -f "c_relay_pg_"
|
||||
|
||||
# Check running processes
|
||||
ps aux | grep c_relay_
|
||||
ps aux | grep c_relay_pg_
|
||||
|
||||
# Force kill port binding
|
||||
fuser -k 8888/tcp
|
||||
@@ -95,7 +95,7 @@ fuser -k 8888/tcp
|
||||
- Event configuration tests: `tests/event_config_tests.sh`
|
||||
|
||||
### SystemD Integration Considerations
|
||||
- Service runs as `c-relay` user in `/opt/c-relay`
|
||||
- Service runs as `c-relay-pg` user in `/opt/c-relay-pg`
|
||||
- Database files created in WorkingDirectory automatically
|
||||
- No environment variables needed (event-based config)
|
||||
- Resource limits: 65536 file descriptors, 4096 processes
|
||||
@@ -137,7 +137,7 @@ fuser -k 8888/tcp
|
||||
## Quick Debugging Commands
|
||||
```bash
|
||||
# Check relay status
|
||||
ps aux | grep c_relay_ && netstat -tln | grep 8888
|
||||
ps aux | grep c_relay_pg_ && netstat -tln | grep 8888
|
||||
|
||||
# View logs
|
||||
tail -f relay.log
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# C-Relay API Documentation
|
||||
# C-Relay-PG API Documentation
|
||||
|
||||
Complete API reference for the C-Relay event-based administration system and advanced features.
|
||||
Complete API reference for the C-Relay-PG event-based administration system and advanced features.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
@@ -21,7 +21,7 @@ Complete API reference for the C-Relay event-based administration system and adv
|
||||
|
||||
## Overview
|
||||
|
||||
C-Relay uses an innovative **event-based administration system** where all configuration and management commands are sent as cryptographically signed Nostr events. This provides:
|
||||
C-Relay-PG uses an innovative **event-based administration system** where all configuration and management commands are sent as cryptographically signed Nostr events. This provides:
|
||||
|
||||
- **Cryptographic security**: All commands must be signed with the admin private key
|
||||
- **Audit trail**: Complete history of all administrative actions
|
||||
@@ -63,8 +63,8 @@ Store the admin private key securely:
|
||||
export C_RELAY_ADMIN_KEY="nsec1abc123..."
|
||||
|
||||
# Secure file
|
||||
echo "nsec1abc123..." > ~/.c-relay-admin
|
||||
chmod 600 ~/.c-relay-admin
|
||||
echo "nsec1abc123..." > ~/.c-relay-pg-admin
|
||||
chmod 600 ~/.c-relay-pg-admin
|
||||
|
||||
# Password manager (recommended)
|
||||
# Store in 1Password, Bitwarden, etc.
|
||||
@@ -167,7 +167,7 @@ Examples:
|
||||
"data": [
|
||||
{
|
||||
"key": "relay_name",
|
||||
"value": "C-Relay",
|
||||
"value": "C-Relay-PG",
|
||||
"data_type": "string",
|
||||
"category": "relay",
|
||||
"description": "Relay name displayed in NIP-11"
|
||||
@@ -506,10 +506,10 @@ ORDER BY count DESC
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `relay_name` | string | "C-Relay" | Relay name (NIP-11) |
|
||||
| `relay_name` | string | "C-Relay-PG" | Relay name (NIP-11) |
|
||||
| `relay_description` | string | "C Nostr Relay" | Relay description |
|
||||
| `relay_contact` | string | "" | Admin contact info |
|
||||
| `relay_software` | string | "c-relay" | Software identifier |
|
||||
| `relay_software` | string | "c-relay-pg" | Software identifier |
|
||||
| `relay_version` | string | auto | Software version |
|
||||
| `supported_nips` | string | "1,9,11,13,15,20,33,40,42,45,50,70" | Supported NIPs |
|
||||
| `language_tags` | string | "*" | Supported languages |
|
||||
@@ -576,7 +576,7 @@ ORDER BY count DESC
|
||||
|
||||
## Real-time Monitoring
|
||||
|
||||
C-Relay provides subscription-based real-time monitoring using ephemeral events (kind 24567).
|
||||
C-Relay-PG provides subscription-based real-time monitoring using ephemeral events (kind 24567).
|
||||
|
||||
### Activation
|
||||
|
||||
|
||||
+42
-31
@@ -1,12 +1,14 @@
|
||||
# Alpine-based MUSL static binary builder for C-Relay
|
||||
# Alpine-based MUSL static binary builder for C-Relay-PG
|
||||
# Produces truly portable binaries with zero runtime dependencies
|
||||
|
||||
ARG DEBUG_BUILD=false
|
||||
ARG DB_BACKEND=sqlite
|
||||
|
||||
FROM alpine:3.19 AS builder
|
||||
|
||||
# Re-declare build argument in this stage
|
||||
# Re-declare build arguments in this stage
|
||||
ARG DEBUG_BUILD=false
|
||||
ARG DB_BACKEND=sqlite
|
||||
|
||||
# Install build dependencies
|
||||
RUN apk add --no-cache \
|
||||
@@ -26,9 +28,11 @@ RUN apk add --no-cache \
|
||||
curl-static \
|
||||
sqlite-dev \
|
||||
sqlite-static \
|
||||
postgresql-dev \
|
||||
linux-headers \
|
||||
wget \
|
||||
bash
|
||||
bash \
|
||||
openssh-client
|
||||
|
||||
# Set working directory
|
||||
WORKDIR /build
|
||||
@@ -68,15 +72,8 @@ RUN cd /tmp && \
|
||||
make install && \
|
||||
rm -rf /tmp/libwebsockets
|
||||
|
||||
# Copy only submodule configuration and git directory
|
||||
COPY .gitmodules /build/.gitmodules
|
||||
COPY .git /build/.git
|
||||
|
||||
# Clean up any stale submodule references (nips directory is not a submodule)
|
||||
RUN git rm --cached nips 2>/dev/null || true
|
||||
|
||||
# Initialize submodules (cached unless .gitmodules changes)
|
||||
RUN git submodule update --init --recursive
|
||||
# Submodules are provided in the local build context and copied explicitly below.
|
||||
# Avoid network/SSH dependency inside Docker image build.
|
||||
|
||||
# Copy nostr_core_lib source files (cached unless nostr_core_lib changes)
|
||||
COPY nostr_core_lib /build/nostr_core_lib/
|
||||
@@ -84,59 +81,73 @@ COPY nostr_core_lib /build/nostr_core_lib/
|
||||
# Copy c_utils_lib source files (cached unless c_utils_lib changes)
|
||||
COPY c_utils_lib /build/c_utils_lib/
|
||||
|
||||
# Build c_utils_lib with MUSL-compatible flags (cached unless c_utils_lib changes)
|
||||
# Build c_utils_lib static library for relay logging utilities
|
||||
# (build only debug.c to avoid broken/missing version header surface in submodule revision)
|
||||
RUN cd c_utils_lib && \
|
||||
sed -i 's/CFLAGS = -Wall -Wextra -std=c99 -O2 -g/CFLAGS = -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -Wall -Wextra -std=c99 -O2 -g/' Makefile && \
|
||||
make clean && \
|
||||
make
|
||||
mkdir -p build && \
|
||||
gcc -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -Wall -Wextra -std=c99 -O2 -g \
|
||||
-Isrc -Iinclude -c src/debug.c -o build/debug.o && \
|
||||
ar rcs libc_utils.a build/debug.o
|
||||
|
||||
# Build nostr_core_lib with required NIPs (cached unless nostr_core_lib changes)
|
||||
# Disable fortification in build.sh to prevent __*_chk symbol issues
|
||||
# NIPs: 001(Basic), 006(Keys), 013(PoW), 017(DMs), 019(Bech32), 044(Encryption), 059(Gift Wrap - required by NIP-17)
|
||||
# NIPs: 001(Basic), 004(Encryption), 006(Keys), 013(PoW), 017(DMs), 019(Bech32), 044(Encryption), 059(Gift Wrap - required by NIP-17)
|
||||
# NIP-04 is required by nostr_signer.c (NIP-46 bunker signer uses NIP-04 encryption)
|
||||
RUN cd nostr_core_lib && \
|
||||
chmod +x build.sh && \
|
||||
sed -i 's/CFLAGS="-Wall -Wextra -std=c99 -fPIC -O2"/CFLAGS="-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -Wall -Wextra -std=c99 -fPIC -O2"/' build.sh && \
|
||||
rm -f *.o *.a 2>/dev/null || true && \
|
||||
./build.sh --nips=1,6,13,17,19,44,59
|
||||
./build.sh --nips=1,4,6,13,17,19,44,59
|
||||
|
||||
# Copy c-relay source files LAST (only this layer rebuilds on source changes)
|
||||
# Copy c-relay-pg source files LAST (only this layer rebuilds on source changes)
|
||||
COPY src/ /build/src/
|
||||
COPY Makefile /build/Makefile
|
||||
|
||||
# Build c-relay with full static linking (only rebuilds when src/ changes)
|
||||
# Build c-relay-pg with full static linking (only rebuilds when src/ changes)
|
||||
# Disable fortification to avoid __*_chk symbols that don't exist in MUSL
|
||||
# Use conditional compilation flags based on DEBUG_BUILD argument
|
||||
# Use conditional compilation flags based on DEBUG_BUILD and DB_BACKEND build args
|
||||
RUN if [ "$DEBUG_BUILD" = "true" ]; then \
|
||||
CFLAGS="-g -O2 -DDEBUG"; \
|
||||
STRIP_CMD="echo 'Keeping debug symbols'"; \
|
||||
echo "Building with DEBUG symbols enabled (optimized with -O2)"; \
|
||||
else \
|
||||
CFLAGS="-O2"; \
|
||||
STRIP_CMD="strip /build/c_relay_static"; \
|
||||
STRIP_CMD="strip /build/c_relay_pg_static"; \
|
||||
echo "Building optimized production binary (symbols stripped)"; \
|
||||
fi && \
|
||||
gcc -static $CFLAGS -Wall -Wextra -std=c99 \
|
||||
if [ "$DB_BACKEND" = "postgres" ]; then \
|
||||
DB_FLAGS="-DDB_BACKEND_POSTGRES -DHAVE_LIBPQ"; \
|
||||
DB_LIBS="-lpq -lpgcommon -lpgport"; \
|
||||
echo "Compiling with PostgreSQL backend"; \
|
||||
else \
|
||||
DB_FLAGS=""; \
|
||||
DB_LIBS=""; \
|
||||
echo "Compiling with SQLite backend"; \
|
||||
fi && \
|
||||
gcc -static $CFLAGS $DB_FLAGS -Wall -Wextra -std=c99 \
|
||||
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 \
|
||||
-I. -Ic_utils_lib/src -Inostr_core_lib -Inostr_core_lib/nostr_core \
|
||||
-Inostr_core_lib/cjson -Inostr_core_lib/nostr_websocket \
|
||||
-Inostr_core_lib/cjson -Inostr_core_lib/nostr_websocket -I/usr/include/postgresql \
|
||||
src/main.c src/config.c src/dm_admin.c src/request_validator.c \
|
||||
src/nip009.c src/nip011.c src/nip013.c src/nip040.c src/nip042.c \
|
||||
src/websockets.c src/subscriptions.c src/api.c src/embedded_web_content.c \
|
||||
-o /build/c_relay_static \
|
||||
src/websockets.c src/subscriptions.c src/api.c src/embedded_web_content.c src/ip_ban.c \
|
||||
src/caching_inbox_poller.c src/caching_service_launcher.c \
|
||||
src/db_ops.c src/db_ops_sqlite.c src/db_ops_postgres.c src/thread_pool.c \
|
||||
-o /build/c_relay_pg_static \
|
||||
c_utils_lib/libc_utils.a \
|
||||
nostr_core_lib/libnostr_core_x64.a \
|
||||
-lwebsockets -lssl -lcrypto -lsqlite3 -lsecp256k1 \
|
||||
-lcurl -lz -lpthread -lm -ldl && \
|
||||
-lcurl -lz -lpthread -lm -ldl $DB_LIBS && \
|
||||
eval "$STRIP_CMD"
|
||||
|
||||
# Verify it's truly static
|
||||
RUN echo "=== Binary Information ===" && \
|
||||
file /build/c_relay_static && \
|
||||
ls -lh /build/c_relay_static && \
|
||||
file /build/c_relay_pg_static && \
|
||||
ls -lh /build/c_relay_pg_static && \
|
||||
echo "=== Checking for dynamic dependencies ===" && \
|
||||
(ldd /build/c_relay_static 2>&1 || echo "Binary is static") && \
|
||||
(ldd /build/c_relay_pg_static 2>&1 || echo "Binary is static") && \
|
||||
echo "=== Build complete ==="
|
||||
|
||||
# Output stage - just the binary
|
||||
FROM scratch AS output
|
||||
COPY --from=builder /build/c_relay_static /c_relay_static
|
||||
COPY --from=builder /build/c_relay_pg_static /c_relay_pg_static
|
||||
@@ -1,28 +1,40 @@
|
||||
# C-Relay Makefile
|
||||
# C-Relay-PG Makefile
|
||||
|
||||
CC = gcc
|
||||
CFLAGS = -Wall -Wextra -std=c99 -g -O2
|
||||
INCLUDES = -I. -Ic_utils_lib/src -Inostr_core_lib -Inostr_core_lib/nostr_core -Inostr_core_lib/cjson -Inostr_core_lib/nostr_websocket
|
||||
INCLUDES = -I. -Ic_utils_lib/src -Inostr_core_lib -Inostr_core_lib/nostr_core -Inostr_core_lib/cjson -Inostr_core_lib/nostr_websocket -I/usr/include/postgresql
|
||||
LIBS = -lsqlite3 -lwebsockets -lz -ldl -lpthread -lm -L/usr/local/lib -lsecp256k1 -lssl -lcrypto -L/usr/local/lib -lcurl -Lc_utils_lib -lc_utils
|
||||
|
||||
DB_BACKEND ?= sqlite
|
||||
|
||||
# Build directory
|
||||
BUILD_DIR = build
|
||||
|
||||
# Source files
|
||||
MAIN_SRC = src/main.c src/config.c src/dm_admin.c src/request_validator.c src/nip009.c src/nip011.c src/nip013.c src/nip040.c src/nip042.c src/websockets.c src/subscriptions.c src/api.c src/embedded_web_content.c
|
||||
MAIN_SRC = src/main.c src/config.c src/dm_admin.c src/request_validator.c src/nip009.c src/nip011.c src/nip013.c src/nip040.c src/nip042.c src/websockets.c src/subscriptions.c src/api.c src/embedded_web_content.c src/ip_ban.c src/thread_pool.c src/caching_inbox_poller.c src/caching_service_launcher.c
|
||||
DB_OPS_SRC = src/db_ops.c
|
||||
|
||||
ifeq ($(DB_BACKEND),postgres)
|
||||
CFLAGS += -DDB_BACKEND_POSTGRES -DHAVE_LIBPQ
|
||||
DB_OPS_SRC += src/db_ops_postgres.c
|
||||
LIBS += -lpq
|
||||
else
|
||||
DB_OPS_SRC += src/db_ops_sqlite.c
|
||||
endif
|
||||
|
||||
NOSTR_CORE_LIB = nostr_core_lib/libnostr_core_x64.a
|
||||
C_UTILS_LIB = c_utils_lib/libc_utils.a
|
||||
|
||||
# Architecture detection
|
||||
ARCH = $(shell uname -m)
|
||||
ifeq ($(ARCH),x86_64)
|
||||
TARGET = $(BUILD_DIR)/c_relay_x86
|
||||
TARGET = $(BUILD_DIR)/c_relay_pg_x86
|
||||
else ifeq ($(ARCH),aarch64)
|
||||
TARGET = $(BUILD_DIR)/c_relay_arm64
|
||||
TARGET = $(BUILD_DIR)/c_relay_pg_arm64
|
||||
else ifeq ($(ARCH),arm64)
|
||||
TARGET = $(BUILD_DIR)/c_relay_arm64
|
||||
TARGET = $(BUILD_DIR)/c_relay_pg_arm64
|
||||
else
|
||||
TARGET = $(BUILD_DIR)/c_relay_$(ARCH)
|
||||
TARGET = $(BUILD_DIR)/c_relay_pg_$(ARCH)
|
||||
endif
|
||||
|
||||
# Default target
|
||||
@@ -81,19 +93,19 @@ force-version:
|
||||
@$(MAKE) src/main.h
|
||||
|
||||
# Build the relay
|
||||
$(TARGET): $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
|
||||
@echo "Compiling C-Relay for architecture: $(ARCH)"
|
||||
$(CC) $(CFLAGS) $(INCLUDES) $(MAIN_SRC) -o $(TARGET) $(NOSTR_CORE_LIB) $(C_UTILS_LIB) $(LIBS)
|
||||
$(TARGET): $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(DB_OPS_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
|
||||
@echo "Compiling C-Relay-PG for architecture: $(ARCH) (backend: $(DB_BACKEND))"
|
||||
$(CC) $(CFLAGS) $(INCLUDES) $(MAIN_SRC) $(DB_OPS_SRC) -o $(TARGET) $(NOSTR_CORE_LIB) $(C_UTILS_LIB) $(LIBS)
|
||||
@echo "Build complete: $(TARGET)"
|
||||
|
||||
# Build for specific architectures
|
||||
x86: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
|
||||
@echo "Building C-Relay for x86_64..."
|
||||
$(CC) $(CFLAGS) $(INCLUDES) $(MAIN_SRC) -o $(BUILD_DIR)/c_relay_x86 $(NOSTR_CORE_LIB) $(C_UTILS_LIB) $(LIBS)
|
||||
@echo "Build complete: $(BUILD_DIR)/c_relay_x86"
|
||||
x86: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(DB_OPS_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
|
||||
@echo "Building C-Relay-PG for x86_64 (backend: $(DB_BACKEND))..."
|
||||
$(CC) $(CFLAGS) $(INCLUDES) $(MAIN_SRC) $(DB_OPS_SRC) -o $(BUILD_DIR)/c_relay_pg_x86 $(NOSTR_CORE_LIB) $(C_UTILS_LIB) $(LIBS)
|
||||
@echo "Build complete: $(BUILD_DIR)/c_relay_pg_x86"
|
||||
|
||||
arm64: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
|
||||
@echo "Cross-compiling C-Relay for ARM64..."
|
||||
arm64: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(DB_OPS_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
|
||||
@echo "Cross-compiling C-Relay-PG for ARM64 (backend: $(DB_BACKEND))..."
|
||||
@if ! command -v aarch64-linux-gnu-gcc >/dev/null 2>&1; then \
|
||||
echo "ERROR: ARM64 cross-compiler not found."; \
|
||||
echo "Install with: make install-cross-tools"; \
|
||||
@@ -116,9 +128,9 @@ arm64: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(NOSTR_CORE_LIB) $(
|
||||
fi
|
||||
@echo "Using aarch64-linux-gnu-gcc with ARM64 libraries..."
|
||||
PKG_CONFIG_PATH=/usr/lib/aarch64-linux-gnu/pkgconfig:/usr/share/pkgconfig \
|
||||
aarch64-linux-gnu-gcc $(CFLAGS) $(INCLUDES) $(MAIN_SRC) -o $(BUILD_DIR)/c_relay_arm64 $(NOSTR_CORE_LIB) $(C_UTILS_LIB) \
|
||||
aarch64-linux-gnu-gcc $(CFLAGS) $(INCLUDES) $(MAIN_SRC) $(DB_OPS_SRC) -o $(BUILD_DIR)/c_relay_pg_arm64 $(NOSTR_CORE_LIB) $(C_UTILS_LIB) \
|
||||
-L/usr/lib/aarch64-linux-gnu $(LIBS)
|
||||
@echo "Build complete: $(BUILD_DIR)/c_relay_arm64"
|
||||
@echo "Build complete: $(BUILD_DIR)/c_relay_pg_arm64"
|
||||
|
||||
# Install ARM64 cross-compilation dependencies
|
||||
install-arm64-deps:
|
||||
@@ -160,7 +172,7 @@ test: $(TARGET)
|
||||
init-db:
|
||||
@echo "Database initialization is now handled automatically when the server starts."
|
||||
@echo "The schema is embedded in the binary - no external files needed."
|
||||
@echo "To manually recreate database: rm -f db/c_nostr_relay.db && ./build/c_relay_x86"
|
||||
@echo "To manually recreate database: rm -f db/c_nostr_relay.db && ./build/c_relay_pg_x86"
|
||||
|
||||
# Clean build artifacts
|
||||
clean:
|
||||
@@ -180,7 +192,7 @@ install-deps:
|
||||
|
||||
# Help
|
||||
help:
|
||||
@echo "C-Relay Build System"
|
||||
@echo "C-Relay-PG Build System"
|
||||
@echo ""
|
||||
@echo "Targets:"
|
||||
@echo " all Build the relay for current architecture (default)"
|
||||
@@ -209,15 +221,15 @@ help:
|
||||
# Build fully static MUSL binaries using Docker
|
||||
static-musl-x86_64:
|
||||
@echo "Building fully static MUSL binary for x86_64..."
|
||||
docker buildx build --platform linux/amd64 -f examples/deployment/static-builder.Dockerfile -t c-relay-static-builder-x86_64 --load .
|
||||
docker run --rm -v $(PWD)/build:/output c-relay-static-builder-x86_64 sh -c "cp /c_relay_static_musl_x86_64 /output/"
|
||||
@echo "Static binary created: build/c_relay_static_musl_x86_64"
|
||||
docker buildx build --platform linux/amd64 -f examples/deployment/static-builder.Dockerfile -t c-relay-pg-static-builder-x86_64 --load .
|
||||
docker run --rm -v $(PWD)/build:/output c-relay-pg-static-builder-x86_64 sh -c "cp /c_relay_pg_static_musl_x86_64 /output/"
|
||||
@echo "Static binary created: build/c_relay_pg_static_musl_x86_64"
|
||||
|
||||
static-musl-arm64:
|
||||
@echo "Building fully static MUSL binary for ARM64..."
|
||||
docker buildx build --platform linux/arm64 -f examples/deployment/static-builder.Dockerfile -t c-relay-static-builder-arm64 --load .
|
||||
docker run --rm -v $(PWD)/build:/output c-relay-static-builder-arm64 sh -c "cp /c_relay_static_musl_x86_64 /output/c_relay_static_musl_arm64"
|
||||
@echo "Static binary created: build/c_relay_static_musl_arm64"
|
||||
docker buildx build --platform linux/arm64 -f examples/deployment/static-builder.Dockerfile -t c-relay-pg-static-builder-arm64 --load .
|
||||
docker run --rm -v $(PWD)/build:/output c-relay-pg-static-builder-arm64 sh -c "cp /c_relay_pg_static_musl_x86_64 /output/c_relay_pg_static_musl_arm64"
|
||||
@echo "Static binary created: build/c_relay_pg_static_musl_arm64"
|
||||
|
||||
static-musl: static-musl-x86_64 static-musl-arm64
|
||||
@echo "Built static MUSL binaries for both architectures"
|
||||
|
||||
+6
-6
@@ -45,21 +45,21 @@ Also included is a more standard administrative web front end. This front end co
|
||||
|
||||
## Screenshots
|
||||
|
||||

|
||||

|
||||
Main page with real time updates.
|
||||
|
||||

|
||||

|
||||
Set your configuration preferences.
|
||||
|
||||

|
||||

|
||||
View current subscriptions
|
||||
|
||||

|
||||

|
||||
Add npubs to white or black lists.
|
||||
|
||||

|
||||

|
||||
Run sql queries on the database.
|
||||
|
||||

|
||||

|
||||
Light mode.
|
||||
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
# C-Relay: High-Performance Nostr Relay
|
||||
# C-Relay-PG: High-Performance Nostr Relay
|
||||
|
||||
A blazingly fast, production-ready Nostr relay implemented in C with an innovative event-based configuration system. Built for performance, security, and ease of deployment.
|
||||
|
||||
## 🚀 Why C-Relay?
|
||||
## 🚀 Why C-Relay-PG?
|
||||
|
||||
### Event-Based Configuration
|
||||
Unlike traditional relays that require config files, C-Relay uses **cryptographically signed Nostr events** for all configuration. This means:
|
||||
Unlike traditional relays that require config files, C-Relay-PG uses **cryptographically signed Nostr events** for all configuration. This means:
|
||||
- **Zero config files** - Everything stored in the database
|
||||
- **Real-time updates** - Changes applied instantly without restart
|
||||
- **Cryptographic security** - All changes must be signed by admin
|
||||
@@ -36,7 +36,7 @@ Control your relay by sending direct messages from any Nostr client:
|
||||
|
||||
## 📋 Supported NIPs
|
||||
|
||||
C-Relay implements a comprehensive set of Nostr Improvement Proposals:
|
||||
C-Relay-PG implements a comprehensive set of Nostr Improvement Proposals:
|
||||
|
||||
- ✅ **NIP-01**: Basic protocol flow implementation
|
||||
- ✅ **NIP-09**: Event deletion
|
||||
@@ -89,12 +89,12 @@ Download and run - no dependencies required:
|
||||
|
||||
```bash
|
||||
# Download the latest static release
|
||||
wget https://git.laantungir.net/laantungir/c-relay/releases/download/v0.6.0/c-relay-v0.6.0-linux-x86_64-static
|
||||
chmod +x c-relay-v0.6.0-linux-x86_64-static
|
||||
mv c-relay-v0.6.0-linux-x86_64-static c-relay
|
||||
wget https://git.laantungir.net/laantungir/c-relay-pg/releases/download/v0.6.0/c-relay-pg-v0.6.0-linux-x86_64-static
|
||||
chmod +x c-relay-pg-v0.6.0-linux-x86_64-static
|
||||
mv c-relay-pg-v0.6.0-linux-x86_64-static c-relay-pg
|
||||
|
||||
# Run the relay
|
||||
./c-relay
|
||||
./c-relay-pg
|
||||
```
|
||||
|
||||
**Important**: On first startup, save the **Admin Private Key** displayed in the console. You'll need it for all administrative operations.
|
||||
@@ -107,8 +107,8 @@ sudo apt install -y build-essential git sqlite3 libsqlite3-dev \
|
||||
libwebsockets-dev libssl-dev libsecp256k1-dev libcurl4-openssl-dev zlib1g-dev
|
||||
|
||||
# Clone and build
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
git submodule update --init --recursive
|
||||
./make_and_restart_relay.sh
|
||||
```
|
||||
@@ -136,8 +136,8 @@ The web interface provides:
|
||||
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
git submodule update --init --recursive
|
||||
|
||||
# Build
|
||||
@@ -147,25 +147,25 @@ make clean && make
|
||||
sudo systemd/install-service.sh
|
||||
|
||||
# Start and enable
|
||||
sudo systemctl start c-relay
|
||||
sudo systemctl enable c-relay
|
||||
sudo systemctl start c-relay-pg
|
||||
sudo systemctl enable c-relay-pg
|
||||
|
||||
# Capture admin keys from logs
|
||||
sudo journalctl -u c-relay | grep "Admin Private Key"
|
||||
sudo journalctl -u c-relay-pg | grep "Admin Private Key"
|
||||
```
|
||||
|
||||
### Docker Deployment
|
||||
|
||||
```bash
|
||||
# Build Docker image
|
||||
docker build -f Dockerfile.alpine-musl -t c-relay .
|
||||
docker build -f Dockerfile.alpine-musl -t c-relay-pg .
|
||||
|
||||
# Run container
|
||||
docker run -d \
|
||||
--name c-relay \
|
||||
--name c-relay-pg \
|
||||
-p 8888:8888 \
|
||||
-v /path/to/data:/data \
|
||||
c-relay
|
||||
c-relay-pg
|
||||
```
|
||||
|
||||
### Cloud Deployment
|
||||
@@ -181,7 +181,7 @@ See [`docs/deployment_guide.md`](docs/deployment_guide.md) for detailed deployme
|
||||
|
||||
## 🔧 Configuration
|
||||
|
||||
C-Relay uses an innovative event-based configuration system. All settings are managed through signed Nostr events.
|
||||
C-Relay-PG uses an innovative event-based configuration system. All settings are managed through signed Nostr events.
|
||||
|
||||
### Basic Configuration
|
||||
|
||||
@@ -239,8 +239,8 @@ The admin private key is displayed **only once** during first startup. Store it
|
||||
|
||||
```bash
|
||||
# Save to secure location
|
||||
echo "ADMIN_PRIVKEY=your_admin_private_key" > ~/.c-relay-admin
|
||||
chmod 600 ~/.c-relay-admin
|
||||
echo "ADMIN_PRIVKEY=your_admin_private_key" > ~/.c-relay-pg-admin
|
||||
chmod 600 ~/.c-relay-pg-admin
|
||||
```
|
||||
|
||||
### Production Security
|
||||
@@ -269,9 +269,9 @@ Contributions are welcome! Please:
|
||||
|
||||
## 🔗 Links
|
||||
|
||||
- **Repository**: [https://github.com/your-org/c-relay](https://github.com/your-org/c-relay)
|
||||
- **Releases**: [https://git.laantungir.net/laantungir/c-relay/releases](https://git.laantungir.net/laantungir/c-relay/releases)
|
||||
- **Issues**: [https://github.com/your-org/c-relay/issues](https://github.com/your-org/c-relay/issues)
|
||||
- **Repository**: [https://github.com/your-org/c-relay-pg](https://github.com/your-org/c-relay-pg)
|
||||
- **Releases**: [https://git.laantungir.net/laantungir/c-relay-pg/releases](https://git.laantungir.net/laantungir/c-relay-pg/releases)
|
||||
- **Issues**: [https://github.com/your-org/c-relay-pg/issues](https://github.com/your-org/c-relay-pg/issues)
|
||||
- **Nostr Protocol**: [https://github.com/nostr-protocol/nostr](https://github.com/nostr-protocol/nostr)
|
||||
|
||||
## 💬 Support
|
||||
|
||||
@@ -41,7 +41,7 @@ This guide is specifically tailored for C programs that use:
|
||||
|
||||
```bash
|
||||
# 1. Copy the Dockerfile template (see below)
|
||||
cp /path/to/c-relay/Dockerfile.alpine-musl ./Dockerfile.static
|
||||
cp /path/to/c-relay-pg/Dockerfile.alpine-musl ./Dockerfile.static
|
||||
|
||||
# 2. Customize for your project (see Customization section)
|
||||
vim Dockerfile.static
|
||||
@@ -466,13 +466,13 @@ docker build -t my-app:latest .
|
||||
docker run --rm my-app:latest --help
|
||||
```
|
||||
|
||||
## Reusing c-relay Files
|
||||
## Reusing c-relay-pg Files
|
||||
|
||||
You can directly copy these files from c-relay:
|
||||
You can directly copy these files from c-relay-pg:
|
||||
|
||||
### 1. Dockerfile.alpine-musl
|
||||
```bash
|
||||
cp /path/to/c-relay/Dockerfile.alpine-musl ./Dockerfile.static
|
||||
cp /path/to/c-relay-pg/Dockerfile.alpine-musl ./Dockerfile.static
|
||||
```
|
||||
|
||||
Then customize:
|
||||
@@ -482,7 +482,7 @@ Then customize:
|
||||
|
||||
### 2. build_static.sh
|
||||
```bash
|
||||
cp /path/to/c-relay/build_static.sh ./
|
||||
cp /path/to/c-relay-pg/build_static.sh ./
|
||||
```
|
||||
|
||||
Then customize:
|
||||
@@ -492,7 +492,7 @@ Then customize:
|
||||
|
||||
### 3. .dockerignore (Optional)
|
||||
```bash
|
||||
cp /path/to/c-relay/.dockerignore ./
|
||||
cp /path/to/c-relay-pg/.dockerignore ./
|
||||
```
|
||||
|
||||
Helps speed up Docker builds by excluding unnecessary files.
|
||||
@@ -522,7 +522,7 @@ Helps speed up Docker builds by excluding unnecessary files.
|
||||
- [Alpine Linux](https://alpinelinux.org/)
|
||||
- [nostr_core_lib](https://github.com/chebizarro/nostr_core_lib)
|
||||
- [Static Linking Best Practices](https://www.musl-libc.org/faq.html)
|
||||
- [c-relay Implementation](./docs/musl_static_build.md)
|
||||
- [c-relay-pg Implementation](./docs/musl_static_build.md)
|
||||
|
||||
## Example: Minimal Nostr Client
|
||||
|
||||
|
||||
+133
@@ -0,0 +1,133 @@
|
||||
# C-Relay-PG PHP Admin Page
|
||||
|
||||
A traditional PHP admin interface for the caching service that connects
|
||||
directly to PostgreSQL, bypassing the NIP-44 64KB encryption limit of the
|
||||
Nostr admin API. Handles thousands of followed pubkeys via server-side
|
||||
pagination.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Install PHP + PHP-FPM + PostgreSQL extension
|
||||
|
||||
```bash
|
||||
# Debian/Ubuntu
|
||||
sudo apt install php-fpm php-pgsql
|
||||
|
||||
# RHEL/Fedora/Alma
|
||||
sudo dnf install php-fpm php-pgsql
|
||||
```
|
||||
|
||||
### 2. Copy admin files to the server
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /opt/c-relay-pg/admin
|
||||
sudo cp -r admin/* /opt/c-relay-pg/admin/
|
||||
```
|
||||
|
||||
### 3. Configure database credentials
|
||||
|
||||
Edit `/opt/c-relay-pg/admin/lib/config.php` and set the PostgreSQL
|
||||
connection parameters to match your relay's `--db-*` flags:
|
||||
|
||||
```php
|
||||
return [
|
||||
'db_host' => 'localhost',
|
||||
'db_port' => '5432',
|
||||
'db_name' => 'crelay',
|
||||
'db_user' => 'crelay',
|
||||
'db_password' => 'crelay',
|
||||
// ...
|
||||
];
|
||||
```
|
||||
|
||||
### 4. Set up HTTP Basic Auth
|
||||
|
||||
```bash
|
||||
sudo htpasswd -c /opt/c-relay-pg/admin/.htpasswd admin
|
||||
# Enter a password when prompted
|
||||
```
|
||||
|
||||
### 5. Add nginx location block
|
||||
|
||||
Add this to your existing nginx server block (the one that proxies to
|
||||
the relay on port 8888):
|
||||
|
||||
```nginx
|
||||
# PHP admin page for caching service
|
||||
location /admin/ {
|
||||
alias /opt/c-relay-pg/admin/;
|
||||
index index.php;
|
||||
|
||||
auth_basic "Relay Admin";
|
||||
auth_basic_user_file /opt/c-relay-pg/admin/.htpasswd;
|
||||
|
||||
location ~ \.php$ {
|
||||
fastcgi_pass unix:/run/php/php-fpm.sock;
|
||||
fastcgi_index index.php;
|
||||
include fastcgi_params;
|
||||
fastcgi_param SCRIPT_FILENAME $request_filename;
|
||||
}
|
||||
|
||||
# Deny access to the lib/ directory (contains DB credentials)
|
||||
location ^~ /admin/lib/ {
|
||||
deny all;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
> **Note:** The `fastcgi_pass` socket path varies by distro:
|
||||
> - Debian/Ubuntu: `unix:/run/php/php8.2-fpm.sock`
|
||||
> - RHEL/Fedora: `unix:/run/php-fpm/www.sock`
|
||||
>
|
||||
> Check with: `ls /run/php/` or `ls /run/php-fpm/`
|
||||
|
||||
### 6. Reload nginx
|
||||
|
||||
```bash
|
||||
sudo nginx -t && sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
### 7. Visit the admin page
|
||||
|
||||
```
|
||||
https://relay.yourdomain.com/admin/
|
||||
```
|
||||
|
||||
Enter the HTTP Basic Auth credentials you set in step 4.
|
||||
|
||||
## Pages
|
||||
|
||||
| Page | URL | Description |
|
||||
|------|-----|-------------|
|
||||
| Dashboard | `/admin/` | Service state, backfill progress, error summary, active target |
|
||||
| Follows | `/admin/follows.php` | Paginated followed-pubkey table with names, event counts, relay status |
|
||||
| Relays | `/admin/relays.php` | Per (author, relay) backfill progress with descriptive error statuses |
|
||||
| Config | `/admin/config-edit.php` | Read/edit caching config values; bumps config generation on save |
|
||||
| Inbox | `/admin/inbox.php` | Queue depth monitor for `caching_event_inbox` |
|
||||
|
||||
## Security
|
||||
|
||||
- **HTTP Basic Auth** protects all pages (nginx `auth_basic`)
|
||||
- **`lib/` directory denied** via nginx (contains DB credentials)
|
||||
- **PDO prepared statements** everywhere (no SQL injection)
|
||||
- **HTTPS** via existing nginx SSL config
|
||||
- **No CORS needed** — same origin as the relay
|
||||
|
||||
## How It Works
|
||||
|
||||
The PHP pages query PostgreSQL directly using PDO with the same
|
||||
`crelay` database user the relay uses. The dashboard auto-refreshes
|
||||
every 10 seconds via AJAX polling of `api/status.php`. Paginated tables
|
||||
(follows, relays) use server-side pagination with `LIMIT`/`OFFSET`.
|
||||
|
||||
The config editor writes to the `config` table and bumps
|
||||
`caching_config_generation`, which the caching service detects and
|
||||
hot-reloads — no relay restart needed.
|
||||
|
||||
## Upgrading to Real-Time (Future)
|
||||
|
||||
If you later need true push updates (sub-second), you can add a small
|
||||
Node.js WebSocket server alongside the PHP pages that uses PostgreSQL
|
||||
`LISTEN/NOTIFY` to push updates to connected browsers. The PHP pages
|
||||
and JSON endpoints are reusable — the WebSocket server would just
|
||||
replace the polling `fetch()` calls in the JavaScript.
|
||||
@@ -0,0 +1,69 @@
|
||||
<?php
|
||||
/**
|
||||
* C-Relay-PG Admin — JSON status endpoint for dashboard polling.
|
||||
*
|
||||
* Returns all dashboard stats in a single JSON object.
|
||||
* Called by index.php's JavaScript every 10 seconds.
|
||||
*/
|
||||
require_once __DIR__ . '/../lib/helpers.php';
|
||||
|
||||
$pdo = db();
|
||||
|
||||
// Service state
|
||||
$state = $pdo->query("SELECT * FROM caching_service_state WHERE id = 1")->fetch();
|
||||
|
||||
// Active backfill target (table may not exist yet on fresh start)
|
||||
$active = ['active_pubkey' => '', 'active_relay' => ''];
|
||||
try {
|
||||
$active = $pdo->query("SELECT active_pubkey, active_relay FROM caching_backfill_active WHERE id = 1")->fetch() ?: $active;
|
||||
} catch (PDOException $e) { /* table doesn't exist yet */ }
|
||||
|
||||
// Error relay summary (consecutive_errors column may not exist yet on fresh start)
|
||||
$error_stats = ['auto_completed' => 0, 'error_count' => 0, 'timeout_count' => 0];
|
||||
try {
|
||||
$error_stats = $pdo->query("
|
||||
SELECT
|
||||
COUNT(*) FILTER (WHERE consecutive_errors >= 3) AS auto_completed,
|
||||
COUNT(*) FILTER (WHERE last_status LIKE 'error%') AS error_count,
|
||||
COUNT(*) FILTER (WHERE last_status = 'timeout') AS timeout_count
|
||||
FROM caching_backfill_relay_progress
|
||||
")->fetch() ?: $error_stats;
|
||||
} catch (PDOException $e) { /* column/table doesn't exist yet */ }
|
||||
|
||||
// Inbox stats
|
||||
$inbox = $pdo->query("
|
||||
SELECT
|
||||
COUNT(*) AS pending,
|
||||
COUNT(*) FILTER (WHERE source_class = 'live') AS live,
|
||||
COUNT(*) FILTER (WHERE source_class = 'backfill') AS backfill
|
||||
FROM caching_event_inbox
|
||||
")->fetch();
|
||||
|
||||
// Event count
|
||||
$events = $pdo->query("SELECT COUNT(*) AS total FROM events")->fetch();
|
||||
|
||||
// Heartbeat age
|
||||
$hb_age = intval($state['heartbeat_at'] ?? 0);
|
||||
$hb_ago = $hb_age > 0 ? time() - $hb_age : 0;
|
||||
|
||||
json_response([
|
||||
'service_state' => $state['service_state'] ?? 'unknown',
|
||||
'heartbeat_ago' => $hb_ago > 0 ? $hb_ago . 's ago' : 'never',
|
||||
'config_generation' => intval($state['config_generation'] ?? 0),
|
||||
'followed_count' => intval($state['followed_author_count'] ?? 0),
|
||||
'selected_relays' => intval($state['selected_relay_count'] ?? 0),
|
||||
'connected_relays' => intval($state['connected_relay_count'] ?? 0),
|
||||
'bf_complete' => intval($state['backfill_authors_complete'] ?? 0),
|
||||
'bf_total' => intval($state['backfill_authors_total'] ?? 0),
|
||||
'events_fetched' => intval($state['events_fetched'] ?? 0),
|
||||
'inbox_inserts' => intval($state['inbox_inserts'] ?? 0),
|
||||
'db_events' => intval($events['total'] ?? 0),
|
||||
'inbox_pending' => intval($inbox['pending'] ?? 0),
|
||||
'inbox_live' => intval($inbox['live'] ?? 0),
|
||||
'inbox_backfill' => intval($inbox['backfill'] ?? 0),
|
||||
'error_count' => intval($error_stats['error_count'] ?? 0),
|
||||
'timeout_count' => intval($error_stats['timeout_count'] ?? 0),
|
||||
'auto_completed' => intval($error_stats['auto_completed'] ?? 0),
|
||||
'active_pubkey' => $active['active_pubkey'] ?? '',
|
||||
'active_relay' => $active['active_relay'] ?? '',
|
||||
]);
|
||||
@@ -0,0 +1,419 @@
|
||||
/*
|
||||
* C-Relay-PG Admin — PHP caching service admin pages.
|
||||
* Styled to match the original api/index.css Nostr admin interface.
|
||||
* Courier New monospace, black/white/red color scheme, dark mode by default.
|
||||
*/
|
||||
|
||||
:root {
|
||||
/* Core Variables — matches api/index.css light mode (default) */
|
||||
--primary-color: #000000;
|
||||
--secondary-color: #ffffff;
|
||||
--accent-color: #ff0000;
|
||||
--muted-color: #dddddd;
|
||||
--border-color: var(--muted-color);
|
||||
--font-family: "Courier New", Courier, monospace;
|
||||
--border-radius: 5px;
|
||||
--border-width: 1px;
|
||||
}
|
||||
|
||||
* {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
body {
|
||||
font-family: var(--font-family);
|
||||
background-color: var(--secondary-color);
|
||||
color: var(--primary-color);
|
||||
padding: 0;
|
||||
max-width: 1200px;
|
||||
margin: 0 auto;
|
||||
}
|
||||
|
||||
a {
|
||||
color: var(--primary-color);
|
||||
text-decoration: none;
|
||||
}
|
||||
a:hover { color: var(--accent-color); }
|
||||
|
||||
/* ================================
|
||||
SIDE NAVIGATION (matches original)
|
||||
================================ */
|
||||
.side-nav {
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: -300px;
|
||||
width: 280px;
|
||||
height: 100vh;
|
||||
background: var(--secondary-color);
|
||||
border-right: var(--border-width) solid var(--border-color);
|
||||
z-index: 1000;
|
||||
transition: left 0.3s ease;
|
||||
overflow-y: auto;
|
||||
padding-top: 80px;
|
||||
}
|
||||
.side-nav.open { left: 0; }
|
||||
|
||||
.side-nav-overlay {
|
||||
position: fixed;
|
||||
top: 0; left: 0; width: 100%; height: 100%;
|
||||
background: rgba(0, 0, 0, 0.5);
|
||||
z-index: 999;
|
||||
display: none;
|
||||
}
|
||||
.side-nav-overlay.show { display: block; }
|
||||
|
||||
.nav-menu { list-style: none; padding: 0; margin: 0; }
|
||||
.nav-menu li { border-bottom: var(--border-width) solid var(--muted-color); }
|
||||
.nav-menu li:last-child { border-bottom: none; }
|
||||
|
||||
.nav-item {
|
||||
display: block;
|
||||
padding: 15px 20px;
|
||||
color: var(--primary-color);
|
||||
text-decoration: none;
|
||||
font-family: var(--font-family);
|
||||
font-size: 16px;
|
||||
font-weight: bold;
|
||||
transition: all 0.2s ease;
|
||||
cursor: pointer;
|
||||
border: 2px solid var(--secondary-color);
|
||||
background: none;
|
||||
width: 100%;
|
||||
text-align: left;
|
||||
}
|
||||
.nav-item:hover {
|
||||
background: var(--muted-color);
|
||||
color: var(--accent-color);
|
||||
}
|
||||
.nav-item.active {
|
||||
text-decoration: underline;
|
||||
padding-left: 16px;
|
||||
}
|
||||
|
||||
.nav-footer {
|
||||
position: absolute;
|
||||
bottom: 20px; left: 0; right: 0;
|
||||
padding: 0 20px;
|
||||
}
|
||||
.nav-footer-btn {
|
||||
display: block;
|
||||
width: 100%;
|
||||
padding: 12px 20px;
|
||||
margin-bottom: 8px;
|
||||
color: var(--primary-color);
|
||||
border: 1px solid var(--border-color);
|
||||
border-radius: 4px;
|
||||
font-family: var(--font-family);
|
||||
font-size: 14px;
|
||||
font-weight: bold;
|
||||
cursor: pointer;
|
||||
transition: all 0.2s ease;
|
||||
background: none;
|
||||
}
|
||||
.nav-footer-btn:hover {
|
||||
background: var(--muted-color);
|
||||
border-color: var(--accent-color);
|
||||
}
|
||||
|
||||
/* ================================
|
||||
HEADER (matches original)
|
||||
================================ */
|
||||
.main-header {
|
||||
background-color: var(--secondary-color);
|
||||
padding: 15px 20px;
|
||||
z-index: 100;
|
||||
max-width: 1200px;
|
||||
margin: 0 auto;
|
||||
}
|
||||
.header-content {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
position: relative;
|
||||
}
|
||||
.header-title {
|
||||
margin: 0;
|
||||
font-size: 24px;
|
||||
font-weight: bolder;
|
||||
color: var(--primary-color);
|
||||
cursor: pointer;
|
||||
transition: all 0.2s ease;
|
||||
}
|
||||
.header-title:hover { opacity: 0.8; }
|
||||
.header-title .relay-letter { display: inline-block; }
|
||||
|
||||
.menu-btn {
|
||||
background: none;
|
||||
border: var(--border-width) solid var(--border-color);
|
||||
border-radius: var(--border-radius);
|
||||
color: var(--primary-color);
|
||||
font-family: var(--font-family);
|
||||
font-size: 20px;
|
||||
padding: 5px 12px;
|
||||
cursor: pointer;
|
||||
width: auto;
|
||||
margin: 0;
|
||||
}
|
||||
.menu-btn:hover { border-color: var(--accent-color); }
|
||||
|
||||
/* ================================
|
||||
SECTIONS (matches original)
|
||||
================================ */
|
||||
.section {
|
||||
background: var(--secondary-color);
|
||||
border: var(--border-width) solid var(--border-color);
|
||||
border-radius: var(--border-radius);
|
||||
padding: 20px;
|
||||
margin-bottom: 20px;
|
||||
margin-left: 5px;
|
||||
margin-right: 5px;
|
||||
}
|
||||
.section-header {
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
padding-bottom: 15px;
|
||||
font-size: 16px;
|
||||
font-weight: normal;
|
||||
font-family: var(--font-family);
|
||||
color: var(--primary-color);
|
||||
}
|
||||
|
||||
/* ================================
|
||||
TABLES (matches .config-table)
|
||||
================================ */
|
||||
.config-table {
|
||||
border: 1px solid var(--border-color);
|
||||
border-radius: var(--border-radius);
|
||||
width: 100%;
|
||||
border-collapse: separate;
|
||||
border-spacing: 0;
|
||||
margin: 10px 0;
|
||||
overflow: hidden;
|
||||
}
|
||||
.config-table th,
|
||||
.config-table td {
|
||||
border: 0.1px solid var(--muted-color);
|
||||
padding: 4px 8px;
|
||||
text-align: left;
|
||||
font-family: var(--font-family);
|
||||
font-size: 10px;
|
||||
}
|
||||
.config-table th {
|
||||
font-weight: bold;
|
||||
height: 24px;
|
||||
line-height: 24px;
|
||||
}
|
||||
.config-table tbody tr:hover {
|
||||
background-color: rgba(0, 0, 0, 0.05);
|
||||
}
|
||||
.config-table-container {
|
||||
overflow-x: auto;
|
||||
max-width: 100%;
|
||||
}
|
||||
|
||||
/* ================================
|
||||
INPUTS / BUTTONS (matches original)
|
||||
================================ */
|
||||
input, textarea, select {
|
||||
width: 100%;
|
||||
padding: 8px;
|
||||
background: var(--secondary-color);
|
||||
color: var(--primary-color);
|
||||
border: var(--border-width) solid var(--border-color);
|
||||
border-radius: var(--border-radius);
|
||||
font-family: var(--font-family);
|
||||
font-size: 14px;
|
||||
box-sizing: border-box;
|
||||
transition: all 0.2s ease;
|
||||
}
|
||||
input:focus, textarea:focus, select:focus {
|
||||
border-color: var(--accent-color);
|
||||
outline: none;
|
||||
}
|
||||
|
||||
button {
|
||||
width: 100%;
|
||||
padding: 8px;
|
||||
background: var(--secondary-color);
|
||||
color: var(--primary-color);
|
||||
border: var(--border-width) solid var(--border-color);
|
||||
border-radius: var(--border-radius);
|
||||
font-family: var(--font-family);
|
||||
font-size: 14px;
|
||||
cursor: pointer;
|
||||
margin: 5px 0;
|
||||
font-weight: bold;
|
||||
transition: all 0.2s ease;
|
||||
}
|
||||
button:hover { border-color: var(--accent-color); }
|
||||
button:active {
|
||||
background: var(--accent-color);
|
||||
color: var(--secondary-color);
|
||||
}
|
||||
|
||||
/* ================================
|
||||
STAT CARDS (dashboard)
|
||||
================================ */
|
||||
.cards {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fill, minmax(200px, 1fr));
|
||||
gap: 10px;
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
.card {
|
||||
background: var(--secondary-color);
|
||||
border: var(--border-width) solid var(--border-color);
|
||||
border-radius: var(--border-radius);
|
||||
padding: 12px;
|
||||
}
|
||||
.card .label {
|
||||
color: var(--muted-color);
|
||||
font-size: 10px;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
}
|
||||
.card .value {
|
||||
font-size: 20px;
|
||||
font-weight: bold;
|
||||
margin-top: 4px;
|
||||
}
|
||||
.card .sub {
|
||||
color: var(--muted-color);
|
||||
font-size: 10px;
|
||||
margin-top: 4px;
|
||||
}
|
||||
|
||||
/* ================================
|
||||
STATUS BADGES
|
||||
================================ */
|
||||
.badge {
|
||||
display: inline-block;
|
||||
padding: 2px 6px;
|
||||
border-radius: 3px;
|
||||
font-size: 10px;
|
||||
font-weight: bold;
|
||||
border: var(--border-width) solid var(--border-color);
|
||||
}
|
||||
.badge-success { border-color: #4caf50; color: #4caf50; }
|
||||
.badge-error { border-color: var(--accent-color); color: var(--accent-color); }
|
||||
.badge-warning { border-color: #ff9800; color: #ff9800; }
|
||||
.badge-muted { border-color: var(--muted-color); color: var(--muted-color); }
|
||||
|
||||
/* ================================
|
||||
STATUS INDICATORS
|
||||
================================ */
|
||||
.status-working { color: var(--accent-color); }
|
||||
.status-complete { color: #4caf50; }
|
||||
.status-error { color: var(--accent-color); }
|
||||
|
||||
/* ================================
|
||||
PAGINATION
|
||||
================================ */
|
||||
.pagination {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: 12px 0;
|
||||
gap: 16px;
|
||||
}
|
||||
.pagination a {
|
||||
padding: 6px 14px;
|
||||
background: var(--secondary-color);
|
||||
border: var(--border-width) solid var(--border-color);
|
||||
border-radius: var(--border-radius);
|
||||
color: var(--primary-color);
|
||||
font-size: 14px;
|
||||
width: auto;
|
||||
}
|
||||
.pagination a:hover {
|
||||
border-color: var(--accent-color);
|
||||
color: var(--accent-color);
|
||||
text-decoration: none;
|
||||
}
|
||||
.pagination span { color: var(--muted-color); font-size: 12px; }
|
||||
|
||||
/* ================================
|
||||
FILTERS BAR
|
||||
================================ */
|
||||
.filters {
|
||||
display: flex;
|
||||
gap: 10px;
|
||||
margin-bottom: 16px;
|
||||
flex-wrap: wrap;
|
||||
align-items: center;
|
||||
}
|
||||
.filters select, .filters input {
|
||||
width: auto;
|
||||
min-width: 120px;
|
||||
}
|
||||
.filters input[type="text"] { min-width: 250px; }
|
||||
.filters button {
|
||||
width: auto;
|
||||
margin: 0;
|
||||
min-width: 80px;
|
||||
}
|
||||
|
||||
/* ================================
|
||||
CONFIG EDITOR FORM
|
||||
================================ */
|
||||
.config-form .row {
|
||||
display: grid;
|
||||
grid-template-columns: 250px 1fr;
|
||||
gap: 12px;
|
||||
padding: 12px 0;
|
||||
border-bottom: 0.1px solid var(--muted-color);
|
||||
align-items: start;
|
||||
}
|
||||
.config-form .row label {
|
||||
font-weight: bold;
|
||||
color: var(--primary-color);
|
||||
font-size: 12px;
|
||||
}
|
||||
.config-form .row .hint {
|
||||
color: var(--muted-color);
|
||||
font-size: 10px;
|
||||
margin-top: 4px;
|
||||
}
|
||||
.config-form button {
|
||||
width: auto;
|
||||
margin-top: 16px;
|
||||
padding: 10px 24px;
|
||||
}
|
||||
|
||||
/* ================================
|
||||
MISC
|
||||
================================ */
|
||||
.pubkey {
|
||||
font-family: var(--font-family);
|
||||
font-size: 10px;
|
||||
color: var(--muted-color);
|
||||
}
|
||||
.npub-link {
|
||||
font-family: var(--font-family);
|
||||
font-size: 10px;
|
||||
color: var(--primary-color);
|
||||
text-decoration: none;
|
||||
}
|
||||
.npub-link:hover { color: var(--accent-color); }
|
||||
|
||||
h2 {
|
||||
font-weight: normal;
|
||||
text-align: center;
|
||||
font-size: 16px;
|
||||
font-family: var(--font-family);
|
||||
color: var(--primary-color);
|
||||
margin-bottom: 16px;
|
||||
}
|
||||
|
||||
/* ================================
|
||||
RESPONSIVE
|
||||
================================ */
|
||||
@media (max-width: 768px) {
|
||||
.cards { grid-template-columns: 1fr 1fr; }
|
||||
.config-form .row { grid-template-columns: 1fr; }
|
||||
.filters { flex-direction: column; align-items: stretch; }
|
||||
.filters select, .filters input, .filters button { width: 100%; }
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
<?php
|
||||
/**
|
||||
* C-Relay-PG Admin — Caching Config Editor.
|
||||
* On save, bumps caching_config_generation so the caching service hot-reloads.
|
||||
*/
|
||||
require_once __DIR__ . '/lib/helpers.php';
|
||||
|
||||
$pdo = db();
|
||||
$message = '';
|
||||
$error = '';
|
||||
|
||||
if ($_SERVER['REQUEST_METHOD'] === 'POST') {
|
||||
$fields = [
|
||||
'caching_root_npubs', 'caching_bootstrap_relays', 'caching_kinds',
|
||||
'caching_admin_kinds', 'caching_live_enabled', 'caching_backfill_enabled',
|
||||
'caching_backfill_page_size', 'caching_backfill_tick_interval_ms',
|
||||
'caching_follow_graph_refresh_seconds', 'caching_relay_discovery_refresh_seconds',
|
||||
'caching_max_followed_pubkeys', 'caching_max_upstream_relays',
|
||||
'caching_max_relays_per_pubkey', 'caching_query_timeout_ms',
|
||||
'caching_live_resubscribe_seconds',
|
||||
];
|
||||
try {
|
||||
$pdo->beginTransaction();
|
||||
foreach ($fields as $key) {
|
||||
$val = $_POST[$key] ?? null;
|
||||
if ($val === null) continue;
|
||||
$pdo->prepare("UPDATE config SET value = ? WHERE key = ?")->execute([$val, $key]);
|
||||
}
|
||||
$gen = $pdo->query("SELECT value FROM config WHERE key = 'caching_config_generation'")->fetchColumn();
|
||||
$new_gen = intval($gen) + 1;
|
||||
$pdo->prepare("UPDATE config SET value = ? WHERE key = 'caching_config_generation'")->execute([$new_gen]);
|
||||
$pdo->commit();
|
||||
$message = "Configuration saved. Config generation bumped to $new_gen. The caching service will reload automatically.";
|
||||
} catch (Exception $e) {
|
||||
$pdo->rollBack();
|
||||
$error = 'Failed to save: ' . $e->getMessage();
|
||||
}
|
||||
}
|
||||
|
||||
$config_keys = [
|
||||
'caching_root_npubs' => 'Root npubs (comma-separated npub... values)',
|
||||
'caching_bootstrap_relays' => 'Bootstrap relays (comma-separated wss://... URLs)',
|
||||
'caching_kinds' => 'Kinds to cache (comma-separated integers)',
|
||||
'caching_admin_kinds' => 'Admin kinds (* for all, or comma-separated)',
|
||||
'caching_live_enabled' => 'Live subscription enabled (true/false)',
|
||||
'caching_backfill_enabled' => 'Backfill enabled (true/false)',
|
||||
'caching_backfill_page_size' => 'Backfill page size (events per tick)',
|
||||
'caching_backfill_tick_interval_ms' => 'Backfill tick interval (ms)',
|
||||
'caching_follow_graph_refresh_seconds' => 'Follow graph refresh interval (seconds)',
|
||||
'caching_relay_discovery_refresh_seconds' => 'Relay discovery refresh (seconds)',
|
||||
'caching_max_followed_pubkeys' => 'Max followed pubkeys',
|
||||
'caching_max_upstream_relays' => 'Max upstream relays',
|
||||
'caching_max_relays_per_pubkey' => 'Max relays per pubkey',
|
||||
'caching_query_timeout_ms' => 'Query timeout (ms)',
|
||||
'caching_live_resubscribe_seconds' => 'Live resubscribe interval (seconds)',
|
||||
];
|
||||
|
||||
$values = [];
|
||||
foreach (array_keys($config_keys) as $key) {
|
||||
$stmt = $pdo->prepare("SELECT value FROM config WHERE key = ?");
|
||||
$stmt->execute([$key]);
|
||||
$values[$key] = $stmt->fetchColumn() ?: '';
|
||||
}
|
||||
|
||||
admin_header('config', 'C-Relay-PG Admin — Config');
|
||||
?>
|
||||
|
||||
<div class="section">
|
||||
<div class="section-header">CACHING CONFIGURATION</div>
|
||||
|
||||
<?php if ($message): ?>
|
||||
<p class="status-complete" style="padding:10px;border:1px solid #4caf50;border-radius:5px;margin-bottom:16px">✓ <?= e($message) ?></p>
|
||||
<?php endif; ?>
|
||||
<?php if ($error): ?>
|
||||
<p class="status-error" style="padding:10px;border:1px solid var(--accent-color);border-radius:5px;margin-bottom:16px">✗ <?= e($error) ?></p>
|
||||
<?php endif; ?>
|
||||
|
||||
<form class="config-form" method="post">
|
||||
<?php foreach ($config_keys as $key => $label): ?>
|
||||
<div class="row">
|
||||
<div>
|
||||
<label for="<?= e($key) ?>"><?= e($label) ?></label>
|
||||
<div class="hint"><?= e($key) ?></div>
|
||||
</div>
|
||||
<div>
|
||||
<input type="text" id="<?= e($key) ?>" name="<?= e($key) ?>" value="<?= e($values[$key]) ?>">
|
||||
</div>
|
||||
</div>
|
||||
<?php endforeach; ?>
|
||||
<button type="submit">Save & Apply (bumps config generation)</button>
|
||||
</form>
|
||||
|
||||
<p style="margin-top:16px;color:var(--muted-color);font-size:10px">
|
||||
Saving bumps <code>caching_config_generation</code> so the caching service
|
||||
detects the change and hot-reloads. No relay restart needed.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<?php admin_footer(); ?>
|
||||
@@ -0,0 +1,153 @@
|
||||
<?php
|
||||
/**
|
||||
* C-Relay-PG Admin — Followed Pubkeys (paginated).
|
||||
* Replaces the NIP-44-encrypted caching_follows_status admin command.
|
||||
*/
|
||||
require_once __DIR__ . '/lib/helpers.php';
|
||||
|
||||
$pdo = db();
|
||||
$page = current_page();
|
||||
$per = cfg('follows_per_page', 50);
|
||||
$offset = ($page - 1) * $per;
|
||||
|
||||
$filter = query_param('filter', 'all');
|
||||
$search = query_param('search', '');
|
||||
$sort = query_param('sort', 'events');
|
||||
|
||||
$where = [];
|
||||
$params = [];
|
||||
if ($filter === 'incomplete') {
|
||||
$where[] = 'fp.backfill_complete = false';
|
||||
} elseif ($filter === 'root') {
|
||||
$where[] = 'fp.is_root = true';
|
||||
} elseif ($filter === 'error') {
|
||||
$where[] = "EXISTS (SELECT 1 FROM caching_backfill_relay_progress rp WHERE rp.author_pubkey = fp.pubkey AND rp.last_status LIKE 'error%')";
|
||||
}
|
||||
if ($search) {
|
||||
$where[] = '(fp.pubkey ILIKE ? OR e.content::json->>\'name\' ILIKE ? OR e.content::json->>\'display_name\' ILIKE ?)';
|
||||
$params[] = "%$search%"; $params[] = "%$search%"; $params[] = "%$search%";
|
||||
}
|
||||
$where_sql = $where ? 'WHERE ' . implode(' AND ', $where) : '';
|
||||
|
||||
$sort_map = [
|
||||
'events' => 'fp.events_fetched DESC',
|
||||
'name' => "COALESCE(e.content::json->>'display_name', e.content::json->>'name') ASC NULLS LAST",
|
||||
'recent' => 'fp.last_seen DESC',
|
||||
'complete' => 'fp.backfill_complete ASC, fp.events_fetched DESC',
|
||||
];
|
||||
$sort_sql = $sort_map[$sort] ?? $sort_map['events'];
|
||||
|
||||
$stmt = $pdo->prepare("SELECT COUNT(*) FROM caching_followed_pubkeys fp LEFT JOIN LATERAL (SELECT content FROM events WHERE pubkey = fp.pubkey AND kind = 0 ORDER BY created_at DESC LIMIT 1) e ON true $where_sql");
|
||||
$stmt->execute($params);
|
||||
$total = intval($stmt->fetchColumn());
|
||||
|
||||
$sql = "
|
||||
SELECT fp.pubkey, fp.is_root, fp.backfill_complete, fp.events_fetched,
|
||||
fp.first_seen, fp.last_seen,
|
||||
e.content::json->>'name' AS name,
|
||||
e.content::json->>'display_name' AS display_name,
|
||||
e.content::json->>'picture' AS picture,
|
||||
e.content::json->>'nip05' AS nip05,
|
||||
(SELECT COUNT(*) FROM events WHERE pubkey = fp.pubkey) AS total_events
|
||||
FROM caching_followed_pubkeys fp
|
||||
LEFT JOIN LATERAL (SELECT content FROM events WHERE pubkey = fp.pubkey AND kind = 0 ORDER BY created_at DESC LIMIT 1) e ON true
|
||||
$where_sql
|
||||
ORDER BY $sort_sql
|
||||
LIMIT $per OFFSET $offset
|
||||
";
|
||||
$stmt = $pdo->prepare($sql);
|
||||
$stmt->execute($params);
|
||||
$follows = $stmt->fetchAll();
|
||||
|
||||
admin_header('follows', 'C-Relay-PG Admin — Follows');
|
||||
?>
|
||||
|
||||
<div class="section">
|
||||
<div class="section-header">FOLLOWED PUBKEYS (<?= number_format($total) ?> total)</div>
|
||||
|
||||
<form class="filters" method="get">
|
||||
<select name="filter" onchange="this.form.submit()">
|
||||
<option value="all" <?= $filter==='all'?'selected':'' ?>>All</option>
|
||||
<option value="incomplete" <?= $filter==='incomplete'?'selected':'' ?>>Incomplete only</option>
|
||||
<option value="root" <?= $filter==='root'?'selected':'' ?>>Root only</option>
|
||||
<option value="error" <?= $filter==='error'?'selected':'' ?>>Has error relays</option>
|
||||
</select>
|
||||
<select name="sort" onchange="this.form.submit()">
|
||||
<option value="events" <?= $sort==='events'?'selected':'' ?>>Sort: Events fetched</option>
|
||||
<option value="name" <?= $sort==='name'?'selected':'' ?>>Sort: Name</option>
|
||||
<option value="recent" <?= $sort==='recent'?'selected':'' ?>>Sort: Recently seen</option>
|
||||
<option value="complete" <?= $sort==='complete'?'selected':'' ?>>Sort: Backfill status</option>
|
||||
</select>
|
||||
<input type="text" name="search" value="<?= e($search) ?>" placeholder="Search name or pubkey…">
|
||||
<button type="submit">Search</button>
|
||||
<?php if ($filter !== 'all' || $search): ?>
|
||||
<a href="follows.php" style="font-size:12px">Clear</a>
|
||||
<?php endif; ?>
|
||||
</form>
|
||||
|
||||
<div class="config-table-container">
|
||||
<table class="config-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>npub</th>
|
||||
<th>Root?</th>
|
||||
<th>Events in DB</th>
|
||||
<th>Backfill</th>
|
||||
<th>Last Seen</th>
|
||||
<th>Relays</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<?php foreach ($follows as $f):
|
||||
$npub = hex_to_npub($f['pubkey']);
|
||||
$name = $f['display_name'] ?: $f['name'];
|
||||
$rc = ['total' => 0, 'incomplete' => 0, 'errors' => 0];
|
||||
try {
|
||||
$relay_count = $pdo->prepare("SELECT COUNT(*) AS total, COUNT(*) FILTER (WHERE complete = false) AS incomplete, COUNT(*) FILTER (WHERE last_status LIKE 'error%') AS errors FROM caching_backfill_relay_progress WHERE author_pubkey = ?");
|
||||
$relay_count->execute([$f['pubkey']]);
|
||||
$rc = $relay_count->fetch() ?: $rc;
|
||||
} catch (PDOException $ex) {}
|
||||
?>
|
||||
<tr>
|
||||
<td>
|
||||
<?php if ($name): ?>
|
||||
<strong><?= e($name) ?></strong>
|
||||
<?php if ($f['nip05']): ?>
|
||||
<br><span style="color:var(--muted-color)">✓ <?= e($f['nip05']) ?></span>
|
||||
<?php endif; ?>
|
||||
<?php else: ?>
|
||||
<span style="color:var(--muted-color);font-style:italic">unknown</span>
|
||||
<?php endif; ?>
|
||||
</td>
|
||||
<td><a href="https://njump.me/<?= e($npub) ?>" target="_blank" class="npub-link"><?= e(trunc($npub, 22)) ?></a></td>
|
||||
<td><?= $f['is_root'] ? '<span class="badge badge-warning">root</span>' : '' ?></td>
|
||||
<td><?= number_format(intval($f['total_events'])) ?></td>
|
||||
<td><?= $f['backfill_complete']
|
||||
? '<span class="badge badge-success">✓ complete</span>'
|
||||
: '<span class="badge badge-warning">in progress</span>' ?></td>
|
||||
<td><?= time_ago(intval($f['last_seen'])) ?></td>
|
||||
<td>
|
||||
<?php if (intval($rc['total']) > 0): ?>
|
||||
<?= intval($rc['total']) ?> relays (<?= intval($rc['incomplete']) ?> incomplete
|
||||
<?php if (intval($rc['errors']) > 0): ?>
|
||||
, <span class="status-error"><?= intval($rc['errors']) ?> errors</span>
|
||||
<?php endif; ?>)
|
||||
<br><a href="relays.php?pubkey=<?= e($f['pubkey']) ?>" style="font-size:10px">View relay details →</a>
|
||||
<?php else: ?>
|
||||
<span style="color:var(--muted-color)">—</span>
|
||||
<?php endif; ?>
|
||||
</td>
|
||||
</tr>
|
||||
<?php endforeach; ?>
|
||||
<?php if (empty($follows)): ?>
|
||||
<tr><td colspan="7" style="text-align:center;color:var(--muted-color);padding:20px">No followed pubkeys found.</td></tr>
|
||||
<?php endif; ?>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<?= pagination($page, $per, $total, 'follows.php' . ($filter !== 'all' ? '?filter=' . e($filter) : '') . ($search ? '&search=' . e($search) : '') . ($sort !== 'events' ? '&sort=' . e($sort) : '')) ?>
|
||||
</div>
|
||||
|
||||
<?php admin_footer(); ?>
|
||||
+111
@@ -0,0 +1,111 @@
|
||||
<?php
|
||||
/**
|
||||
* C-Relay-PG Admin — Inbox Queue Monitor.
|
||||
*/
|
||||
require_once __DIR__ . '/lib/helpers.php';
|
||||
|
||||
$pdo = db();
|
||||
|
||||
$stats = $pdo->query("
|
||||
SELECT
|
||||
COUNT(*) AS pending,
|
||||
COUNT(*) FILTER (WHERE source_class = 'live') AS live,
|
||||
COUNT(*) FILTER (WHERE source_class = 'backfill') AS backfill,
|
||||
COUNT(*) FILTER (WHERE source_class = 'discovery') AS discovery,
|
||||
COUNT(*) FILTER (WHERE priority = 0) AS high_priority,
|
||||
COALESCE(EXTRACT(EPOCH FROM NOW())::BIGINT - MIN(received_at), 0) AS oldest_age
|
||||
FROM caching_event_inbox
|
||||
")->fetch() ?: ['pending' => 0, 'live' => 0, 'backfill' => 0, 'discovery' => 0, 'high_priority' => 0, 'oldest_age' => 0];
|
||||
|
||||
$recent = $pdo->query("
|
||||
SELECT event_id, event_json->>'pubkey' AS pubkey, event_json->>'kind' AS kind,
|
||||
source_relay, source_class, priority, received_at
|
||||
FROM caching_event_inbox
|
||||
ORDER BY received_at DESC
|
||||
LIMIT 20
|
||||
")->fetchAll();
|
||||
|
||||
admin_header('inbox', 'C-Relay-PG Admin — Inbox');
|
||||
?>
|
||||
|
||||
<div class="section">
|
||||
<div class="section-header">INBOX QUEUE MONITOR</div>
|
||||
|
||||
<div class="cards">
|
||||
<div class="card">
|
||||
<div class="label">Pending Events</div>
|
||||
<div class="value" id="inbox-pending"><?= number_format(intval($stats['pending'])) ?></div>
|
||||
<div class="sub" id="inbox-detail"><?= intval($stats['live']) ?> live, <?= intval($stats['backfill']) ?> backfill, <?= intval($stats['discovery']) ?> discovery</div>
|
||||
</div>
|
||||
<div class="card">
|
||||
<div class="label">High Priority</div>
|
||||
<div class="value"><?= number_format(intval($stats['high_priority'])) ?></div>
|
||||
<div class="sub">priority = 0 (live events)</div>
|
||||
</div>
|
||||
<div class="card">
|
||||
<div class="label">Oldest Pending</div>
|
||||
<div class="value" id="inbox-oldest"><?= intval($stats['oldest_age']) ?>s</div>
|
||||
<div class="sub">age of oldest event in queue</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<?php if (intval($stats['oldest_age']) > 300): ?>
|
||||
<p class="status-error" style="padding:10px;border:1px solid var(--accent-color);border-radius:5px;margin-bottom:16px">
|
||||
⚠ Oldest pending event is <?= intval($stats['oldest_age']) ?>s old — the inbox poller may not be keeping up.
|
||||
Check that <code>caching_inbox_enabled = true</code> in the config table and the relay is running.
|
||||
</p>
|
||||
<?php endif; ?>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<div class="section-header">RECENT EVENTS IN QUEUE (last 20)</div>
|
||||
<div class="config-table-container">
|
||||
<table class="config-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Event ID</th>
|
||||
<th>Pubkey</th>
|
||||
<th>Kind</th>
|
||||
<th>Source Relay</th>
|
||||
<th>Class</th>
|
||||
<th>Priority</th>
|
||||
<th>Received</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<?php foreach ($recent as $r): ?>
|
||||
<tr>
|
||||
<td class="pubkey"><?= e(trunc($r['event_id'], 16)) ?></td>
|
||||
<td class="pubkey"><?= e(trunc($r['pubkey'], 16)) ?></td>
|
||||
<td><?= e($r['kind']) ?></td>
|
||||
<td><?= e($r['source_relay'] ?? '—') ?></td>
|
||||
<td><span class="badge badge-muted"><?= e($r['source_class']) ?></span></td>
|
||||
<td><?= intval($r['priority']) ?></td>
|
||||
<td><?= time_ago(intval($r['received_at'])) ?></td>
|
||||
</tr>
|
||||
<?php endforeach; ?>
|
||||
<?php if (empty($recent)): ?>
|
||||
<tr><td colspan="7" style="text-align:center;color:var(--muted-color);padding:20px">Inbox is empty — all events have been dequeued.</td></tr>
|
||||
<?php endif; ?>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
async function refreshInbox() {
|
||||
try {
|
||||
const res = await fetch('api/status.php');
|
||||
if (!res.ok) return;
|
||||
const d = await res.json();
|
||||
console.log('[inbox] refreshed at', new Date().toLocaleTimeString(), '— pending:', d.inbox_pending);
|
||||
const p = document.getElementById('inbox-pending');
|
||||
const det = document.getElementById('inbox-detail');
|
||||
if (p) p.textContent = (d.inbox_pending ?? 0).toLocaleString();
|
||||
if (det) det.textContent = (d.inbox_live ?? 0) + ' live, ' + (d.inbox_backfill ?? 0) + ' backfill';
|
||||
} catch (e) {}
|
||||
}
|
||||
setInterval(refreshInbox, <?= cfg('refresh_ms', 10000) ?>);
|
||||
</script>
|
||||
|
||||
<?php admin_footer(); ?>
|
||||
+158
@@ -0,0 +1,158 @@
|
||||
<?php
|
||||
/**
|
||||
* C-Relay-PG Admin — Dashboard.
|
||||
* Auto-refreshes every 10 seconds via AJAX polling of api/status.php.
|
||||
*/
|
||||
require_once __DIR__ . '/lib/helpers.php';
|
||||
|
||||
$pdo = db();
|
||||
|
||||
// Service state
|
||||
$state = $pdo->query("SELECT * FROM caching_service_state WHERE id = 1")->fetch() ?: [];
|
||||
|
||||
// Active backfill target (table may not exist yet on fresh start)
|
||||
$active = ['active_pubkey' => '', 'active_relay' => ''];
|
||||
try {
|
||||
$active = $pdo->query("SELECT active_pubkey, active_relay FROM caching_backfill_active WHERE id = 1")->fetch() ?: $active;
|
||||
} catch (PDOException $e) {}
|
||||
|
||||
// Error relay summary
|
||||
$error_stats = ['auto_completed' => 0, 'error_count' => 0, 'timeout_count' => 0, 'incomplete' => 0, 'total' => 0];
|
||||
try {
|
||||
$error_stats = $pdo->query("
|
||||
SELECT
|
||||
COUNT(*) FILTER (WHERE consecutive_errors >= 3) AS auto_completed,
|
||||
COUNT(*) FILTER (WHERE last_status LIKE 'error%') AS error_count,
|
||||
COUNT(*) FILTER (WHERE last_status = 'timeout') AS timeout_count,
|
||||
COUNT(*) FILTER (WHERE complete = false) AS incomplete,
|
||||
COUNT(*) AS total
|
||||
FROM caching_backfill_relay_progress
|
||||
")->fetch() ?: $error_stats;
|
||||
} catch (PDOException $e) {}
|
||||
|
||||
// Inbox stats
|
||||
$inbox = $pdo->query("
|
||||
SELECT
|
||||
COUNT(*) AS pending,
|
||||
COUNT(*) FILTER (WHERE source_class = 'live') AS live,
|
||||
COUNT(*) FILTER (WHERE source_class = 'backfill') AS backfill,
|
||||
COALESCE(EXTRACT(EPOCH FROM NOW())::BIGINT - MIN(received_at), 0) AS oldest_age
|
||||
FROM caching_event_inbox
|
||||
")->fetch() ?: ['pending' => 0, 'live' => 0, 'backfill' => 0, 'oldest_age' => 0];
|
||||
|
||||
// Event counts
|
||||
$event_stats = $pdo->query("SELECT COUNT(*) AS total, COUNT(DISTINCT pubkey) AS authors FROM events")->fetch() ?: ['total' => 0, 'authors' => 0];
|
||||
$kind_stats = $pdo->query("SELECT kind, COUNT(*) AS cnt FROM events GROUP BY kind ORDER BY cnt DESC LIMIT 10")->fetchAll();
|
||||
|
||||
admin_header('dashboard', 'C-Relay-PG Admin — Dashboard');
|
||||
?>
|
||||
|
||||
<div class="section">
|
||||
<div class="section-header">CACHING SERVICE STATUS</div>
|
||||
<div class="cards" id="dashboard-cards">
|
||||
<div class="card">
|
||||
<div class="label">Service State</div>
|
||||
<div class="value" id="card-state"><?= e($state['service_state'] ?? 'unknown') ?></div>
|
||||
<div class="sub" id="card-heartbeat">Heartbeat: <?= time_ago(intval($state['heartbeat_at'] ?? 0)) ?></div>
|
||||
</div>
|
||||
<div class="card">
|
||||
<div class="label">Followed Authors</div>
|
||||
<div class="value" id="card-follows"><?= intval($state['followed_author_count'] ?? 0) ?></div>
|
||||
<div class="sub" id="card-relays"><?= intval($state['selected_relay_count'] ?? 0) ?> relays, <?= intval($state['connected_relay_count'] ?? 0) ?> connected</div>
|
||||
</div>
|
||||
<div class="card">
|
||||
<div class="label">Backfill Progress</div>
|
||||
<div class="value" id="card-backfill"><?= intval($state['backfill_authors_complete'] ?? 0) ?> / <?= intval($state['backfill_authors_total'] ?? 0) ?></div>
|
||||
<div class="sub" id="card-backfill-pct"><?= ($state['backfill_authors_total'] ?? 0) > 0 ? round(intval($state['backfill_authors_complete'] ?? 0) / intval($state['backfill_authors_total']) * 100) : 0 ?>% complete</div>
|
||||
</div>
|
||||
<div class="card">
|
||||
<div class="label">Events Fetched</div>
|
||||
<div class="value" id="card-events"><?= number_format(intval($state['events_fetched'] ?? 0)) ?></div>
|
||||
<div class="sub" id="card-inserts"><?= number_format(intval($state['inbox_inserts'] ?? 0)) ?> inbox inserts</div>
|
||||
</div>
|
||||
<div class="card">
|
||||
<div class="label">Events in DB</div>
|
||||
<div class="value" id="card-db-events"><?= number_format(intval($event_stats['total'] ?? 0)) ?></div>
|
||||
<div class="sub"><?= number_format(intval($event_stats['authors'] ?? 0)) ?> authors</div>
|
||||
</div>
|
||||
<div class="card">
|
||||
<div class="label">Inbox Pending</div>
|
||||
<div class="value" id="card-inbox"><?= number_format(intval($inbox['pending'] ?? 0)) ?></div>
|
||||
<div class="sub" id="card-inbox-detail"><?= intval($inbox['live'] ?? 0) ?> live, <?= intval($inbox['backfill'] ?? 0) ?> backfill</div>
|
||||
</div>
|
||||
<div class="card">
|
||||
<div class="label">Error Relays</div>
|
||||
<div class="value status-error" id="card-errors"><?= intval($error_stats['error_count'] ?? 0) ?></div>
|
||||
<div class="sub" id="card-errors-detail"><?= intval($error_stats['auto_completed'] ?? 0) ?> auto-completed</div>
|
||||
</div>
|
||||
<div class="card">
|
||||
<div class="label">Config Generation</div>
|
||||
<div class="value" id="card-gen"><?= intval($state['config_generation'] ?? 0) ?></div>
|
||||
<div class="sub">Updated <?= time_ago(intval($state['updated_at'] ?? 0)) ?></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<div class="section-header">ACTIVE BACKFILL TARGET</div>
|
||||
<div id="active-target" style="text-align:center;padding:10px">
|
||||
<?php if (!empty($active['active_pubkey'])): ?>
|
||||
<p class="status-working">⚡ Working on: <span class="pubkey"><?= e(trunc($active['active_pubkey'], 16)) ?></span> @ <?= e($active['active_relay'] ?? '') ?></p>
|
||||
<?php else: ?>
|
||||
<p class="status-complete">✓ Caching complete — no active backfill target</p>
|
||||
<?php endif; ?>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<div class="section-header">TOP EVENT KINDS</div>
|
||||
<div class="config-table-container">
|
||||
<table class="config-table">
|
||||
<thead><tr><th>Kind</th><th>Count</th></tr></thead>
|
||||
<tbody id="kind-table">
|
||||
<?php foreach ($kind_stats as $ks): ?>
|
||||
<tr><td><?= intval($ks['kind']) ?></td><td><?= number_format(intval($ks['cnt'])) ?></td></tr>
|
||||
<?php endforeach; ?>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
const REFRESH_MS = <?= cfg('refresh_ms', 10000) ?>;
|
||||
async function refreshDashboard() {
|
||||
console.log('[dashboard] polling api/status.php at', new Date().toLocaleTimeString());
|
||||
try {
|
||||
const res = await fetch('api/status.php');
|
||||
if (!res.ok) { console.warn('[dashboard] status.php returned', res.status); return; }
|
||||
const d = await res.json();
|
||||
console.log('[dashboard] refreshed at', new Date().toLocaleTimeString(), '— state:', d.service_state, 'follows:', d.followed_count, 'events:', d.db_events);
|
||||
const set = (id, val) => { const el = document.getElementById(id); if (el) el.textContent = val; };
|
||||
set('card-state', d.service_state || 'unknown');
|
||||
set('card-heartbeat', 'Heartbeat: ' + (d.heartbeat_ago || 'never'));
|
||||
set('card-follows', d.followed_count ?? 0);
|
||||
set('card-relays', (d.selected_relays ?? 0) + ' relays, ' + (d.connected_relays ?? 0) + ' connected');
|
||||
set('card-backfill', (d.bf_complete ?? 0) + ' / ' + (d.bf_total ?? 0));
|
||||
set('card-backfill-pct', (d.bf_total > 0 ? Math.round(d.bf_complete / d.bf_total * 100) : 0) + '% complete');
|
||||
set('card-events', (d.events_fetched ?? 0).toLocaleString());
|
||||
set('card-inserts', (d.inbox_inserts ?? 0).toLocaleString() + ' inbox inserts');
|
||||
set('card-db-events', (d.db_events ?? 0).toLocaleString());
|
||||
set('card-inbox', (d.inbox_pending ?? 0).toLocaleString());
|
||||
set('card-inbox-detail', (d.inbox_live ?? 0) + ' live, ' + (d.inbox_backfill ?? 0) + ' backfill');
|
||||
set('card-errors', d.error_count ?? 0);
|
||||
set('card-errors-detail', (d.auto_completed ?? 0) + ' auto-completed');
|
||||
set('card-gen', d.config_generation ?? 0);
|
||||
const atEl = document.getElementById('active-target');
|
||||
if (d.active_pubkey && d.active_relay) {
|
||||
atEl.innerHTML = '<p class="status-working">⚡ Working on: <span class="pubkey">' + d.active_pubkey.substring(0,16) + '…</span> @ ' + d.active_relay + '</p>';
|
||||
} else if (d.active_pubkey) {
|
||||
atEl.innerHTML = '<p class="status-working">⚡ Working on: <span class="pubkey">' + d.active_pubkey.substring(0,16) + '…</span></p>';
|
||||
} else {
|
||||
atEl.innerHTML = '<p class="status-complete">✓ Caching complete — no active backfill target</p>';
|
||||
}
|
||||
} catch (e) { console.error('[dashboard] refresh error:', e); }
|
||||
}
|
||||
setInterval(refreshDashboard, REFRESH_MS);
|
||||
console.log('[dashboard] auto-refresh started, interval:', REFRESH_MS, 'ms');
|
||||
</script>
|
||||
<?php admin_footer(); ?>
|
||||
@@ -0,0 +1,27 @@
|
||||
<?php
|
||||
/**
|
||||
* C-Relay-PG Admin — Database connection configuration.
|
||||
*
|
||||
* This file is NOT web-accessible (denied by nginx location block).
|
||||
* It contains the PostgreSQL connection string used by all admin pages.
|
||||
*
|
||||
* Edit these values to match your relay's PostgreSQL credentials.
|
||||
*/
|
||||
|
||||
// PostgreSQL connection parameters.
|
||||
// These should match the --db-* flags passed to the c-relay-pg binary
|
||||
// or the connection string passed to caching_relay -p "...".
|
||||
return [
|
||||
'db_host' => 'localhost',
|
||||
'db_port' => '5432',
|
||||
'db_name' => 'crelay',
|
||||
'db_user' => 'crelay',
|
||||
'db_password' => 'crelay',
|
||||
|
||||
// Page sizes for paginated tables.
|
||||
'follows_per_page' => 50,
|
||||
'relays_per_page' => 50,
|
||||
|
||||
// Auto-refresh interval for dashboard polling (milliseconds).
|
||||
'refresh_ms' => 10000,
|
||||
];
|
||||
@@ -0,0 +1,44 @@
|
||||
<?php
|
||||
/**
|
||||
* C-Relay-PG Admin — PDO database connection helper.
|
||||
*
|
||||
* Provides a singleton PDO connection to the relay's PostgreSQL database.
|
||||
* All admin pages use db() to get the connection.
|
||||
*/
|
||||
|
||||
require_once __DIR__ . '/config.php';
|
||||
|
||||
function db(): PDO {
|
||||
static $pdo = null;
|
||||
if ($pdo === null) {
|
||||
$cfg = require __DIR__ . '/config.php';
|
||||
$dsn = sprintf(
|
||||
'pgsql:host=%s;port=%d;dbname=%s;user=%s;password=%s',
|
||||
$cfg['db_host'],
|
||||
$cfg['db_port'],
|
||||
$cfg['db_name'],
|
||||
$cfg['db_user'],
|
||||
$cfg['db_password']
|
||||
);
|
||||
try {
|
||||
$pdo = new PDO($dsn, null, null, [
|
||||
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
|
||||
PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC,
|
||||
PDO::ATTR_EMULATE_PREPARES => false,
|
||||
]);
|
||||
} catch (PDOException $e) {
|
||||
http_response_code(500);
|
||||
die('Database connection failed: ' . htmlspecialchars($e->getMessage()));
|
||||
}
|
||||
}
|
||||
return $pdo;
|
||||
}
|
||||
|
||||
/** Get a config value from the admin config file. */
|
||||
function cfg(string $key, $default = null) {
|
||||
static $cfg = null;
|
||||
if ($cfg === null) {
|
||||
$cfg = require __DIR__ . '/config.php';
|
||||
}
|
||||
return $cfg[$key] ?? $default;
|
||||
}
|
||||
@@ -0,0 +1,191 @@
|
||||
<?php
|
||||
/**
|
||||
* C-Relay-PG Admin — Shared helper functions.
|
||||
*/
|
||||
|
||||
require_once __DIR__ . '/db.php';
|
||||
|
||||
/** HTML-escape a string. */
|
||||
function e(?string $s): string {
|
||||
return htmlspecialchars($s ?? '', ENT_QUOTES, 'UTF-8');
|
||||
}
|
||||
|
||||
/** Format a Unix timestamp as a human-readable relative time. */
|
||||
function time_ago(int $ts): string {
|
||||
if ($ts <= 0) return 'never';
|
||||
$diff = time() - $ts;
|
||||
if ($diff < 60) return $diff . 's ago';
|
||||
if ($diff < 3600) return floor($diff / 60) . 'm ago';
|
||||
if ($diff < 86400) return floor($diff / 3600) . 'h ago';
|
||||
return floor($diff / 86400) . 'd ago';
|
||||
}
|
||||
|
||||
/** Format a Unix timestamp as a date/time string. */
|
||||
function fmt_date(int $ts): string {
|
||||
if ($ts <= 0) return '-';
|
||||
return date('Y-m-d H:i:s', $ts);
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert a 64-char hex pubkey to npub (bech32).
|
||||
* Uses the standard bech32 encoding (NIP-19).
|
||||
*/
|
||||
function hex_to_npub(string $hex): string {
|
||||
if (!preg_match('/^[0-9a-fA-F]{64}$/', $hex)) return $hex;
|
||||
$hex = strtolower($hex);
|
||||
|
||||
// Bech32 constants
|
||||
$charset = 'qpzry9x8gf2tvdw0s3jn54khce6mua7l';
|
||||
$data = [];
|
||||
for ($i = 0; $i < strlen($hex); $i += 2) {
|
||||
$data[] = intval(substr($hex, $i, 2), 16);
|
||||
}
|
||||
// Convert 8-bit groups to 5-bit groups
|
||||
$conv = [];
|
||||
$buffer = 0;
|
||||
$bits = 0;
|
||||
foreach ($data as $byte) {
|
||||
$buffer = ($buffer << 8) | $byte;
|
||||
$bits += 8;
|
||||
while ($bits >= 5) {
|
||||
$conv[] = ($buffer >> ($bits - 5)) & 31;
|
||||
$bits -= 5;
|
||||
}
|
||||
}
|
||||
if ($bits > 0) {
|
||||
$conv[] = ($buffer << (5 - $bits)) & 31;
|
||||
}
|
||||
|
||||
// Compute checksum — HRP values are positions of each char in the charset.
|
||||
$values = array_merge(
|
||||
array_map(function($c) {
|
||||
return strpos('qpzry9x8gf2tvdw0s3jn54khce6mua7l', $c);
|
||||
}, str_split('npub')),
|
||||
$conv
|
||||
);
|
||||
$gen = [0x3b6a57b2, 0x26508e6d, 0x1ea119fa, 0x3d4233dd, 0x2a1462b3];
|
||||
$chk = 1;
|
||||
foreach ($values as $v) {
|
||||
$top = $chk >> 25;
|
||||
$chk = (($chk & 0x1ffffff) << 5) ^ $v;
|
||||
for ($i = 0; $i < 5; $i++) {
|
||||
if (($top >> $i) & 1) $chk ^= $gen[$i];
|
||||
}
|
||||
}
|
||||
$chk ^= 1;
|
||||
$checksum = [];
|
||||
for ($i = 0; $i < 6; $i++) {
|
||||
$checksum[] = ($chk >> (5 * (5 - $i))) & 31;
|
||||
}
|
||||
|
||||
$result = 'npub1';
|
||||
foreach (array_merge($conv, $checksum) as $v) {
|
||||
$result .= $charset[$v];
|
||||
}
|
||||
return $result;
|
||||
}
|
||||
|
||||
/** Truncate a string to $len chars with ellipsis. */
|
||||
function trunc(string $s, int $len = 16): string {
|
||||
if (strlen($s) <= $len) return $s;
|
||||
return substr($s, 0, $len) . '…';
|
||||
}
|
||||
|
||||
/** Build pagination links HTML. */
|
||||
function pagination(int $page, int $per_page, int $total, string $base_url): string {
|
||||
$total_pages = max(1, ceil($total / $per_page));
|
||||
if ($total_pages <= 1) return '';
|
||||
|
||||
$html = '<div class="pagination">';
|
||||
if ($page > 1) {
|
||||
$html .= '<a href="' . e($base_url) . '?page=' . ($page - 1) . '">← Prev</a>';
|
||||
}
|
||||
$html .= '<span>Page ' . $page . ' of ' . $total_pages . ' (' . $total . ' total)</span>';
|
||||
if ($page < $total_pages) {
|
||||
$html .= '<a href="' . e($base_url) . '?page=' . ($page + 1) . '">Next →</a>';
|
||||
}
|
||||
$html .= '</div>';
|
||||
return $html;
|
||||
}
|
||||
|
||||
/** Get the current page number from the query string. */
|
||||
function current_page(): int {
|
||||
$p = intval($_GET['page'] ?? 1);
|
||||
return max(1, $p);
|
||||
}
|
||||
|
||||
/** JSON response helper for AJAX endpoints. */
|
||||
function json_response($data): void {
|
||||
header('Content-Type: application/json');
|
||||
echo json_encode($data, JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE);
|
||||
exit;
|
||||
}
|
||||
|
||||
/** Get a query parameter with a default. */
|
||||
function query_param(string $key, $default = null) {
|
||||
return $_GET[$key] ?? $default;
|
||||
}
|
||||
|
||||
/**
|
||||
* Render the shared HTML header with side navigation.
|
||||
* Pass the active page name to highlight the current nav item.
|
||||
*/
|
||||
function admin_header(string $active = '', string $title = 'C-Relay-PG Admin'): void {
|
||||
$pages = [
|
||||
'dashboard' => ['index.php', 'Dashboard'],
|
||||
'follows' => ['follows.php', 'Follows'],
|
||||
'relays' => ['relays.php', 'Relays'],
|
||||
'config' => ['config-edit.php', 'Config'],
|
||||
'inbox' => ['inbox.php', 'Inbox'],
|
||||
];
|
||||
?>
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title><?= e($title) ?></title>
|
||||
<link rel="stylesheet" href="assets/style.css">
|
||||
</head>
|
||||
<body>
|
||||
<!-- Side Navigation -->
|
||||
<nav class="side-nav" id="side-nav">
|
||||
<ul class="nav-menu">
|
||||
<?php foreach ($pages as $key => [$url, $label]): ?>
|
||||
<li><a class="nav-item<?= $active === $key ? ' active' : '' ?>" href="<?= e($url) ?>"><?= e($label) ?></a></li>
|
||||
<?php endforeach; ?>
|
||||
</ul>
|
||||
</nav>
|
||||
<div class="side-nav-overlay" id="side-nav-overlay" onclick="closeNav()"></div>
|
||||
|
||||
<!-- Header -->
|
||||
<div class="main-header">
|
||||
<div class="header-content">
|
||||
<button class="menu-btn" onclick="openNav()">☰</button>
|
||||
<div class="header-title" onclick="location.href='index.php'">
|
||||
<span class="relay-letter">R</span><span class="relay-letter">E</span><span class="relay-letter">L</span><span class="relay-letter">A</span><span class="relay-letter">Y</span>
|
||||
</div>
|
||||
<div style="width:40px"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
function openNav() {
|
||||
document.getElementById('side-nav').classList.add('open');
|
||||
document.getElementById('side-nav-overlay').classList.add('show');
|
||||
}
|
||||
function closeNav() {
|
||||
document.getElementById('side-nav').classList.remove('open');
|
||||
document.getElementById('side-nav-overlay').classList.remove('show');
|
||||
}
|
||||
</script>
|
||||
<?php
|
||||
}
|
||||
|
||||
/** Render the shared HTML footer. */
|
||||
function admin_footer(): void {
|
||||
?>
|
||||
</body>
|
||||
</html>
|
||||
<?php
|
||||
}
|
||||
@@ -0,0 +1,132 @@
|
||||
<?php
|
||||
/**
|
||||
* C-Relay-PG Admin — Per-Relay Backfill Progress.
|
||||
*/
|
||||
require_once __DIR__ . '/lib/helpers.php';
|
||||
|
||||
$pdo = db();
|
||||
$page = current_page();
|
||||
$per = cfg('relays_per_page', 50);
|
||||
$offset = ($page - 1) * $per;
|
||||
|
||||
$filter = query_param('filter', 'all');
|
||||
$pubkey = query_param('pubkey', '');
|
||||
|
||||
$where = [];
|
||||
$params = [];
|
||||
if ($filter === 'incomplete') {
|
||||
$where[] = 'rp.complete = false';
|
||||
} elseif ($filter === 'error') {
|
||||
$where[] = "rp.last_status LIKE 'error%'";
|
||||
} elseif ($filter === 'complete') {
|
||||
$where[] = 'rp.complete = true';
|
||||
}
|
||||
if ($pubkey) {
|
||||
$where[] = 'rp.author_pubkey = ?';
|
||||
$params[] = $pubkey;
|
||||
}
|
||||
$where_sql = $where ? 'WHERE ' . implode(' AND ', $where) : '';
|
||||
|
||||
$total = 0;
|
||||
$relays = [];
|
||||
try {
|
||||
$stmt = $pdo->prepare("SELECT COUNT(*) FROM caching_backfill_relay_progress rp $where_sql");
|
||||
$stmt->execute($params);
|
||||
$total = intval($stmt->fetchColumn());
|
||||
|
||||
$sql = "
|
||||
SELECT rp.author_pubkey, rp.relay_url, rp.until_cursor, rp.complete,
|
||||
rp.events_fetched,
|
||||
COALESCE(rp.consecutive_errors, 0) AS consecutive_errors,
|
||||
rp.last_status, rp.updated_at,
|
||||
e.content::json->>'name' AS name,
|
||||
e.content::json->>'display_name' AS display_name
|
||||
FROM caching_backfill_relay_progress rp
|
||||
LEFT JOIN LATERAL (SELECT content FROM events WHERE pubkey = rp.author_pubkey AND kind = 0 ORDER BY created_at DESC LIMIT 1) e ON true
|
||||
$where_sql
|
||||
ORDER BY rp.complete ASC, COALESCE(rp.consecutive_errors, 0) DESC, rp.updated_at DESC
|
||||
LIMIT $per OFFSET $offset
|
||||
";
|
||||
$stmt = $pdo->prepare($sql);
|
||||
$stmt->execute($params);
|
||||
$relays = $stmt->fetchAll();
|
||||
} catch (PDOException $ex) {}
|
||||
|
||||
function status_badge(string $status): string {
|
||||
if ($status === 'eose') return '<span class="badge badge-success">eose</span>';
|
||||
if ($status === 'timeout') return '<span class="badge badge-warning">timeout</span>';
|
||||
if (str_starts_with($status, 'error')) return '<span class="badge badge-error">' . e($status) . '</span>';
|
||||
if (str_starts_with($status, 'NOTICE')) return '<span class="badge badge-warning">' . e($status) . '</span>';
|
||||
if (str_starts_with($status, 'CLOSED')) return '<span class="badge badge-warning">' . e($status) . '</span>';
|
||||
if ($status === '') return '<span class="badge badge-muted">—</span>';
|
||||
return '<span class="badge badge-muted">' . e($status) . '</span>';
|
||||
}
|
||||
|
||||
admin_header('relays', 'C-Relay-PG Admin — Relay Progress');
|
||||
?>
|
||||
|
||||
<div class="section">
|
||||
<div class="section-header">PER-RELAY BACKFILL PROGRESS (<?= number_format($total) ?> total)</div>
|
||||
|
||||
<form class="filters" method="get">
|
||||
<select name="filter" onchange="this.form.submit()">
|
||||
<option value="all" <?= $filter==='all'?'selected':'' ?>>All</option>
|
||||
<option value="incomplete" <?= $filter==='incomplete'?'selected':'' ?>>Incomplete only</option>
|
||||
<option value="error" <?= $filter==='error'?'selected':'' ?>>Errors only</option>
|
||||
<option value="complete" <?= $filter==='complete'?'selected':'' ?>>Complete only</option>
|
||||
</select>
|
||||
<?php if ($pubkey): ?>
|
||||
<input type="hidden" name="pubkey" value="<?= e($pubkey) ?>">
|
||||
<span style="font-size:10px;color:var(--muted-color)">Filtered by: <span class="pubkey"><?= e(trunc($pubkey, 16)) ?></span></span>
|
||||
<a href="relays.php?filter=<?= e($filter) ?>" style="font-size:10px">Clear</a>
|
||||
<?php endif; ?>
|
||||
</form>
|
||||
|
||||
<div class="config-table-container">
|
||||
<table class="config-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Author</th>
|
||||
<th>Relay URL</th>
|
||||
<th>Complete</th>
|
||||
<th>Events</th>
|
||||
<th>Errors</th>
|
||||
<th>Last Status</th>
|
||||
<th>Updated</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<?php foreach ($relays as $r):
|
||||
$name = $r['display_name'] ?: $r['name'];
|
||||
$npub = hex_to_npub($r['author_pubkey']);
|
||||
?>
|
||||
<tr <?= intval($r['consecutive_errors']) >= 3 ? 'style="background:rgba(255,0,0,0.05)"' : '' ?>>
|
||||
<td>
|
||||
<?php if ($name): ?>
|
||||
<strong><?= e($name) ?></strong>
|
||||
<?php else: ?>
|
||||
<span style="color:var(--muted-color);font-style:italic">unknown</span>
|
||||
<?php endif; ?>
|
||||
<br><a href="follows.php?search=<?= e($r['author_pubkey']) ?>" class="npub-link"><?= e(trunc($npub, 20)) ?></a>
|
||||
</td>
|
||||
<td><?= e($r['relay_url']) ?></td>
|
||||
<td><?= $r['complete'] ? '<span class="badge badge-success">✓</span>' : '<span class="badge badge-warning">…</span>' ?></td>
|
||||
<td><?= number_format(intval($r['events_fetched'])) ?></td>
|
||||
<td><?= intval($r['consecutive_errors']) > 0
|
||||
? '<span class="status-error">' . intval($r['consecutive_errors']) . '</span>'
|
||||
: '0' ?></td>
|
||||
<td><?= status_badge($r['last_status']) ?></td>
|
||||
<td><?= time_ago(intval($r['updated_at'])) ?></td>
|
||||
</tr>
|
||||
<?php endforeach; ?>
|
||||
<?php if (empty($relays)): ?>
|
||||
<tr><td colspan="7" style="text-align:center;color:var(--muted-color);padding:20px">No relay progress rows found.</td></tr>
|
||||
<?php endif; ?>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<?= pagination($page, $per, $total, 'relays.php' . ($filter !== 'all' ? '?filter=' . e($filter) : '') . ($pubkey ? '&pubkey=' . e($pubkey) : '')) ?>
|
||||
</div>
|
||||
|
||||
<?php admin_footer(); ?>
|
||||
+191
@@ -1291,6 +1291,184 @@ body.dark-mode .sql-results-table tbody tr:nth-child(even) {
|
||||
/* background-color: var(--secondary-color); */
|
||||
}
|
||||
|
||||
/* ================================
|
||||
WEB OF TRUST (WoT) STYLES
|
||||
================================ */
|
||||
|
||||
.wot-status-row, .wot-stats-row {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: 8px 0;
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.wot-indicator {
|
||||
padding: 2px 10px;
|
||||
border-radius: 4px;
|
||||
font-weight: bold;
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
.wot-indicator.wot-found { background: #28a745; color: white; }
|
||||
.wot-indicator.wot-missing { background: #dc3545; color: white; }
|
||||
.wot-indicator.wot-unknown { background: #6c757d; color: white; }
|
||||
|
||||
.wot-level-selector { padding: 10px 0; }
|
||||
.wot-level-selector label { display: block; margin-bottom: 5px; font-weight: bold; }
|
||||
|
||||
.wot-level-btn { min-width: 100px; }
|
||||
.wot-level-btn.active {
|
||||
background: var(--accent-color, #ff0000);
|
||||
color: white;
|
||||
border-color: var(--accent-color, #ff0000);
|
||||
}
|
||||
|
||||
.wot-level-description {
|
||||
font-size: 12px;
|
||||
color: var(--primary-color);
|
||||
margin-top: 5px;
|
||||
font-style: italic;
|
||||
opacity: 0.8;
|
||||
}
|
||||
|
||||
/* Dark mode adjustments for WoT */
|
||||
body.dark-mode .wot-indicator.wot-found { background: #28a745; }
|
||||
body.dark-mode .wot-indicator.wot-missing { background: #dc3545; }
|
||||
body.dark-mode .wot-indicator.wot-unknown { background: #6c757d; }
|
||||
|
||||
/* ================================
|
||||
ADMIN ACCESS GATE STYLES
|
||||
================================ */
|
||||
|
||||
/* Access Denied Overlay */
|
||||
.access-denied-overlay {
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: 0;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
background: rgba(0, 0, 0, 0.85);
|
||||
z-index: 9999;
|
||||
display: none;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.access-denied-content {
|
||||
background: var(--card-bg);
|
||||
border: 2px solid #dc3545;
|
||||
border-radius: 12px;
|
||||
padding: 40px 60px;
|
||||
text-align: center;
|
||||
max-width: 500px;
|
||||
box-shadow: 0 10px 40px rgba(220, 53, 69, 0.3);
|
||||
}
|
||||
|
||||
.access-denied-icon {
|
||||
font-size: 64px;
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
|
||||
.access-denied-content h2 {
|
||||
color: #dc3545;
|
||||
font-size: 32px;
|
||||
margin-bottom: 20px;
|
||||
letter-spacing: 2px;
|
||||
}
|
||||
|
||||
.access-denied-message {
|
||||
font-size: 16px;
|
||||
color: var(--primary-color);
|
||||
margin-bottom: 10px;
|
||||
line-height: 1.5;
|
||||
}
|
||||
|
||||
.access-denied-submessage {
|
||||
font-size: 14px;
|
||||
color: var(--muted-color);
|
||||
margin-bottom: 30px;
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
.access-denied-logout-btn {
|
||||
background: #dc3545;
|
||||
color: white;
|
||||
border: none;
|
||||
padding: 12px 40px;
|
||||
font-size: 16px;
|
||||
font-weight: bold;
|
||||
border-radius: 6px;
|
||||
cursor: pointer;
|
||||
transition: all 0.3s ease;
|
||||
}
|
||||
|
||||
.access-denied-logout-btn:hover {
|
||||
background: #c82333;
|
||||
transform: translateY(-2px);
|
||||
box-shadow: 0 4px 12px rgba(220, 53, 69, 0.4);
|
||||
}
|
||||
|
||||
/* Admin Verification Loading Overlay */
|
||||
.admin-verification-overlay {
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: 0;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
background: rgba(0, 0, 0, 0.8);
|
||||
z-index: 9998;
|
||||
display: none;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.admin-verification-content {
|
||||
background: var(--card-bg);
|
||||
border: 1px solid var(--border-color);
|
||||
border-radius: 12px;
|
||||
padding: 40px 60px;
|
||||
text-align: center;
|
||||
max-width: 450px;
|
||||
}
|
||||
|
||||
.admin-verification-content h3 {
|
||||
color: var(--accent-color);
|
||||
font-size: 20px;
|
||||
margin-bottom: 15px;
|
||||
}
|
||||
|
||||
.admin-verification-content p {
|
||||
color: var(--muted-color);
|
||||
font-size: 14px;
|
||||
margin-top: 15px;
|
||||
}
|
||||
|
||||
/* Spinner Animation */
|
||||
.spinner {
|
||||
width: 50px;
|
||||
height: 50px;
|
||||
border: 4px solid var(--border-color);
|
||||
border-top: 4px solid var(--accent-color);
|
||||
border-radius: 50%;
|
||||
animation: spin 1s linear infinite;
|
||||
margin: 0 auto 20px;
|
||||
}
|
||||
|
||||
@keyframes spin {
|
||||
0% { transform: rotate(0deg); }
|
||||
100% { transform: rotate(360deg); }
|
||||
}
|
||||
|
||||
/* Dark mode adjustments */
|
||||
body.dark-mode .access-denied-content {
|
||||
background: var(--card-bg);
|
||||
}
|
||||
|
||||
body.dark-mode .admin-verification-content {
|
||||
background: var(--card-bg);
|
||||
}
|
||||
|
||||
.subscription-detail-row:hover {
|
||||
background-color: var(--muted-color);
|
||||
}
|
||||
@@ -1308,3 +1486,16 @@ body.dark-mode .sql-results-table tbody tr:nth-child(even) {
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
|
||||
/* ================================
|
||||
IP BANS TABLE - compact rows
|
||||
================================ */
|
||||
#ip-bans-table td, #ip-bans-table th {
|
||||
padding: 4px 8px;
|
||||
line-height: 1.3;
|
||||
font-size: 13px;
|
||||
}
|
||||
#ip-bans-table button {
|
||||
padding: 2px 8px;
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
+359
-16
@@ -4,7 +4,7 @@
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>C-Relay Admin</title>
|
||||
<title>C-Relay-PG Admin</title>
|
||||
<link rel="stylesheet" href="/api/index.css">
|
||||
</head>
|
||||
|
||||
@@ -16,7 +16,9 @@
|
||||
<li><button class="nav-item" data-page="subscriptions">Subscriptions</button></li>
|
||||
<li><button class="nav-item" data-page="configuration">Configuration</button></li>
|
||||
<li><button class="nav-item" data-page="authorization">Authorization</button></li>
|
||||
<li><button class="nav-item" data-page="ip-bans">IP BANS</button></li>
|
||||
<li><button class="nav-item" data-page="relay-events">Relay Events</button></li>
|
||||
<li><button class="nav-item" data-page="caching">Caching</button></li>
|
||||
<li><button class="nav-item" data-page="dm">DM</button></li>
|
||||
<li><button class="nav-item" data-page="database">Database Query</button></li>
|
||||
</ul>
|
||||
@@ -41,7 +43,7 @@
|
||||
<span class="relay-letter" data-letter="Y">Y</span>
|
||||
</div>
|
||||
<div class="relay-info">
|
||||
<div id="relay-name" class="relay-name">C-Relay</div>
|
||||
<div id="relay-name" class="relay-name">C-Relay-PG</div>
|
||||
<div id="relay-description" class="relay-description">Loading...</div>
|
||||
<div id="relay-pubkey-container" class="relay-pubkey-container">
|
||||
<div id="relay-pubkey" class="relay-pubkey">Loading...</div>
|
||||
@@ -67,6 +69,17 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Access Denied Overlay (shown when non-admin user logs in) -->
|
||||
<div id="access-denied-overlay" class="access-denied-overlay" style="display: none;">
|
||||
<div class="access-denied-content">
|
||||
<div class="access-denied-icon">⛔</div>
|
||||
<h2>ACCESS DENIED</h2>
|
||||
<p class="access-denied-message">This interface is restricted to the relay administrator.</p>
|
||||
<p class="access-denied-submessage">The logged-in account does not have admin privileges.</p>
|
||||
<button type="button" class="access-denied-logout-btn" onclick="logout()">LOGOUT</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- DATABASE STATISTICS Section -->
|
||||
<!-- Subscribe to kind 24567 events to receive real-time monitoring data -->
|
||||
<div class="section flex-section" id="databaseStatisticsSection" style="display: none;">
|
||||
@@ -100,6 +113,10 @@
|
||||
<td>Process ID</td>
|
||||
<td id="process-id">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>WebSocket Connections</td>
|
||||
<td id="websocket-connections">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Active Subscriptions</td>
|
||||
<td id="active-subscriptions">-</td>
|
||||
@@ -108,14 +125,14 @@
|
||||
<td>Memory Usage</td>
|
||||
<td id="memory-usage">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>CPU Core</td>
|
||||
<td id="cpu-core">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>CPU Usage</td>
|
||||
<td id="cpu-usage">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>CPU Core</td>
|
||||
<td id="cpu-core">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Oldest Event</td>
|
||||
<td id="oldest-event">-</td>
|
||||
@@ -187,6 +204,7 @@
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Rank</th>
|
||||
<th>Name</th>
|
||||
<th>Pubkey</th>
|
||||
<th>Event Count</th>
|
||||
<th>Percentage</th>
|
||||
@@ -194,7 +212,7 @@
|
||||
</thead>
|
||||
<tbody id="stats-pubkeys-table-body">
|
||||
<tr>
|
||||
<td colspan="4" style="text-align: center; font-style: italic;">No data loaded</td>
|
||||
<td colspan="5" style="text-align: center; font-style: italic;">No data loaded</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
@@ -251,6 +269,7 @@
|
||||
</div>
|
||||
|
||||
<!-- Auth Rules Management - Moved after configuration -->
|
||||
<!-- AUTH RULES MANAGEMENT SECTION -->
|
||||
<div class="section flex-section" id="authRulesSection" style="display: none;">
|
||||
<div class="section-header">
|
||||
AUTH RULES MANAGEMENT
|
||||
@@ -273,16 +292,11 @@ AUTH RULES MANAGEMENT
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<!-- Simplified Auth Rule Input Section -->
|
||||
<!-- Auth Rule Input Section -->
|
||||
<div id="authRuleInputSections" style="display: block;">
|
||||
|
||||
<!-- Combined Pubkey Auth Rule Section -->
|
||||
|
||||
|
||||
<div class="input-group">
|
||||
<label for="authRulePubkey">Pubkey (nsec or hex):</label>
|
||||
<input type="text" id="authRulePubkey" placeholder="nsec1... or 64-character hex pubkey">
|
||||
|
||||
<label for="authRulePubkey">Public Key (npub or hex):</label>
|
||||
<input type="text" id="authRulePubkey" placeholder="npub1... or 64-character hex pubkey">
|
||||
</div>
|
||||
<div id="whitelistWarning" class="warning-box" style="display: none;">
|
||||
<strong>⚠️ WARNING:</strong> Adding whitelist rules changes relay behavior to whitelist-only
|
||||
@@ -296,10 +310,51 @@ AUTH RULES MANAGEMENT
|
||||
BLACKLIST</button>
|
||||
<button type="button" id="refreshAuthRulesBtn">REFRESH</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<!-- WEB OF TRUST SECTION -->
|
||||
<div class="section flex-section" id="wotSection" style="display: none;">
|
||||
<div class="section-header">
|
||||
WEB OF TRUST
|
||||
</div>
|
||||
|
||||
<!-- Kind 3 Status Indicator -->
|
||||
<div id="wotKind3Status" class="wot-status-row">
|
||||
<span>Admin Contact List (kind 3):</span>
|
||||
<span id="wotKind3Indicator" class="wot-indicator wot-unknown">Checking...</span>
|
||||
</div>
|
||||
|
||||
<!-- WoT Level Selector -->
|
||||
<div class="wot-level-selector">
|
||||
<label>WoT Level:</label>
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="wotLevel0Btn" class="wot-level-btn" onclick="setWotLevel(0)">
|
||||
OFF
|
||||
</button>
|
||||
<button type="button" id="wotLevel1Btn" class="wot-level-btn" onclick="setWotLevel(1)">
|
||||
WRITE ONLY
|
||||
</button>
|
||||
<button type="button" id="wotLevel2Btn" class="wot-level-btn" onclick="setWotLevel(2)">
|
||||
FULL
|
||||
</button>
|
||||
</div>
|
||||
<div class="wot-level-description" id="wotLevelDescription">
|
||||
Level 0: Open relay — anyone can read and write
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- WoT Stats -->
|
||||
<div class="wot-stats-row">
|
||||
<span>Whitelisted Pubkeys:</span>
|
||||
<span id="wotWhitelistCount">—</span>
|
||||
</div>
|
||||
|
||||
<!-- Sync Button -->
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="wotSyncBtn" onclick="syncWot()">SYNC FROM KIND 3</button>
|
||||
<button type="button" id="wotRefreshBtn" onclick="loadWotStatus()">REFRESH STATUS</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@@ -334,12 +389,138 @@ AUTH RULES MANAGEMENT
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- IP BANS Section -->
|
||||
<div class="section" id="ipBansSection" style="display: none;">
|
||||
<div class="section-header">
|
||||
IP BAN MANAGEMENT
|
||||
</div>
|
||||
|
||||
<!-- Statistics Cards -->
|
||||
<div class="input-group">
|
||||
<div class="config-table-container">
|
||||
<table class="config-table" id="ip-bans-stats-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Total IPs Tracked</th>
|
||||
<th>Currently Banned</th>
|
||||
<th>Total Bans Issued</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td id="ip-bans-total">-</td>
|
||||
<td id="ip-bans-active">-</td>
|
||||
<td id="ip-bans-issued">-</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Add Ban Form -->
|
||||
<div class="input-group">
|
||||
<h3>Manually Ban IP Address</h3>
|
||||
<div class="form-group">
|
||||
<label for="ban-ip-input">IP Address:</label>
|
||||
<input type="text" id="ban-ip-input" placeholder="192.168.1.100">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="ban-duration-select">Ban Duration:</label>
|
||||
<select id="ban-duration-select">
|
||||
<option value="3600">1 Hour</option>
|
||||
<option value="86400" selected>24 Hours</option>
|
||||
<option value="604800">7 Days</option>
|
||||
<option value="2592000">30 Days</option>
|
||||
<option value="31536000">1 Year</option>
|
||||
<option value="999999999">Permanent</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="add-ban-btn">BAN IP</button>
|
||||
</div>
|
||||
<div id="add-ban-status" class="status-message"></div>
|
||||
</div>
|
||||
|
||||
<!-- Whitelist Management -->
|
||||
<div class="input-group">
|
||||
<h3>IP Whitelist (Never Banned)</h3>
|
||||
<p style="font-size:13px;opacity:0.8;">IPs in this list are never idle-banned. Comma-separated.</p>
|
||||
<div class="form-group">
|
||||
<label for="whitelist-ip-input">Add IP to Whitelist:</label>
|
||||
<input type="text" id="whitelist-ip-input" placeholder="103.81.231.220">
|
||||
</div>
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="add-whitelist-btn">ADD TO WHITELIST</button>
|
||||
</div>
|
||||
<div id="whitelist-status" class="status-message"></div>
|
||||
<div id="whitelist-current" style="margin-top:8px;font-size:13px;"></div>
|
||||
</div>
|
||||
|
||||
<!-- Filter Controls -->
|
||||
<div class="input-group">
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="ip-ban-filter-all" class="active">All IPs</button>
|
||||
<button type="button" id="ip-ban-filter-banned">Currently Banned</button>
|
||||
<button type="button" id="ip-ban-filter-expired">Expired</button>
|
||||
<button type="button" id="refresh-ip-bans-btn">REFRESH</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- IP Bans List -->
|
||||
<div class="input-group">
|
||||
<label>Banned IP Addresses:</label>
|
||||
<div class="config-table-container">
|
||||
<table class="config-table" id="ip-bans-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>IP Address</th>
|
||||
<th>Status</th>
|
||||
<th>Banned Until</th>
|
||||
<th>Failures</th>
|
||||
<th>Authed Successfully</th>
|
||||
<th>Connection Attempts</th>
|
||||
<th>Actions</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody id="ip-bans-tbody">
|
||||
<tr>
|
||||
<td colspan="7" style="text-align: center;">Click REFRESH to load IP bans</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- RELAY EVENTS Section -->
|
||||
<div class="section" id="relayEventsSection" style="display: none;">
|
||||
<div class="section-header">
|
||||
RELAY EVENTS MANAGEMENT
|
||||
</div>
|
||||
|
||||
<!-- Live Relay Event Feed -->
|
||||
<div class="input-group">
|
||||
<h3>Live Relay Event Feed (Normal Nostr Subscription)</h3>
|
||||
<div class="config-table-container">
|
||||
<table class="config-table" id="live-relay-events-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Time</th>
|
||||
<th>Kind</th>
|
||||
<th>Pubkey</th>
|
||||
<th>ID</th>
|
||||
<th>Content Preview</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody id="live-relay-events-table-body">
|
||||
<tr>
|
||||
<td colspan="5" style="text-align: center;">Waiting for live events...</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Kind 0: User Metadata -->
|
||||
<div class="input-group">
|
||||
<h3>Kind 0: User Metadata</h3>
|
||||
@@ -400,6 +581,164 @@ AUTH RULES MANAGEMENT
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- CACHING Section -->
|
||||
<div class="section" id="cachingSection" style="display: none;">
|
||||
<div class="section-header">CACHING</div>
|
||||
|
||||
<!-- Caching Service Status (read-only) -->
|
||||
<div class="input-group">
|
||||
<h3>Caching Service Status</h3>
|
||||
<div id="caching-service-status" class="status-display">
|
||||
<p>Service status will appear here after connecting.</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Relay Inbox Status (read-only) -->
|
||||
<div class="input-group">
|
||||
<h3>Relay Inbox Status</h3>
|
||||
<div id="caching-inbox-status" class="status-display">
|
||||
<p>Inbox status will appear here after connecting.</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Caching Configuration (editable) -->
|
||||
<div class="input-group">
|
||||
<h3>Caching Configuration</h3>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="caching-enabled">Enable Caching:</label>
|
||||
<select id="caching-enabled">
|
||||
<option value="false">Disabled</option>
|
||||
<option value="true">Enabled</option>
|
||||
</select>
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="caching-inbox-enabled">Enable Inbox Consumer:</label>
|
||||
<select id="caching-inbox-enabled">
|
||||
<option value="false">Disabled</option>
|
||||
<option value="true">Enabled</option>
|
||||
</select>
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="caching-root-npubs">Root Npubs (one per line):</label>
|
||||
<textarea id="caching-root-npubs" rows="4" placeholder="npub1...">npub13lm5wf8dvsdnc2894pkhch9uf8phvw9varrv8zf4sc885hhdmc8q6lx7ks</textarea>
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="caching-bootstrap-relays">Bootstrap Relays (one per line):</label>
|
||||
<textarea id="caching-bootstrap-relays" rows="4" placeholder="wss://relay.damus.io">wss://relay.damus.io
|
||||
wss://nos.lol
|
||||
wss://relay.primal.net
|
||||
wss://laantungir.net/relay</textarea>
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="caching-kinds">Kinds (comma-separated):</label>
|
||||
<input type="text" id="caching-kinds" placeholder="1,3,6,10000,30023" value="0,1,3,6,10000,10002,30023">
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="caching-admin-kinds">Admin Kinds (comma-separated, * for all):</label>
|
||||
<input type="text" id="caching-admin-kinds" placeholder="*">
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="caching-live-enabled">Live Subscriptions:</label>
|
||||
<select id="caching-live-enabled">
|
||||
<option value="true">Enabled</option>
|
||||
<option value="false">Disabled</option>
|
||||
</select>
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="caching-backfill-enabled">Backfill:</label>
|
||||
<select id="caching-backfill-enabled">
|
||||
<option value="true">Enabled</option>
|
||||
<option value="false">Disabled</option>
|
||||
</select>
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="caching-backfill-windows">Backfill Windows (seconds, comma-separated):</label>
|
||||
<input type="text" id="caching-backfill-windows" placeholder="86400,604800,2592000,7776000,31536000">
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="caching-backfill-page-size">Backfill Page Size:</label>
|
||||
<input type="number" id="caching-backfill-page-size" placeholder="50">
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="caching-backfill-tick-interval">Backfill Tick Interval (ms):</label>
|
||||
<input type="number" id="caching-backfill-tick-interval" placeholder="5000">
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="caching-inbox-batch-size">Inbox Batch Size:</label>
|
||||
<input type="number" id="caching-inbox-batch-size" placeholder="150">
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="caching-inbox-active-poll">Inbox Active Poll (ms):</label>
|
||||
<input type="number" id="caching-inbox-active-poll" placeholder="200">
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="caching-inbox-idle-poll">Inbox Idle Poll (ms):</label>
|
||||
<input type="number" id="caching-inbox-idle-poll" placeholder="5000">
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="caching-service-binary-path">Caching Service Binary Path:</label>
|
||||
<input type="text" id="caching-service-binary-path" placeholder="./caching_relay" value="./caching_relay">
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="caching-service-pg-conn">Caching Service PG Connection:</label>
|
||||
<input type="text" id="caching-service-pg-conn" placeholder="host=localhost port=5432 dbname=crelay user=crelay password=crelay" value="host=localhost port=5432 dbname=crelay user=crelay password=crelay">
|
||||
</div>
|
||||
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="caching-apply-btn">APPLY CONFIGURATION</button>
|
||||
<button type="button" id="caching-reset-progress-btn">RESET BACKFILL PROGRESS</button>
|
||||
</div>
|
||||
<div id="caching-config-status" class="status-message"></div>
|
||||
</div>
|
||||
|
||||
<div class="input-group">
|
||||
<h3>Caching Service Control</h3>
|
||||
<p>Start or stop the external caching service process. Set <code>caching_service_binary_path</code> and <code>caching_service_pg_conn</code> above and click APPLY CONFIGURATION before starting.</p>
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="caching-start-service-btn">START CACHING SERVICE</button>
|
||||
<button type="button" id="caching-stop-service-btn">STOP CACHING SERVICE</button>
|
||||
</div>
|
||||
<div id="caching-service-control-status" class="status-message"></div>
|
||||
</div>
|
||||
|
||||
<div class="input-group">
|
||||
<h3>Followed Pubkeys</h3>
|
||||
<p>Pubkeys being cached with their profile names, event counts, outbox relays, and backfill status.</p>
|
||||
<div id="caching-follows-status" class="status-message"></div>
|
||||
<table id="caching-follows-table" style="width: 100%; border-collapse: collapse; font-size: 13px;">
|
||||
<thead>
|
||||
<tr style="border-bottom: 2px solid var(--border-color, #444); text-align: left;">
|
||||
<th style="padding: 6px 8px;">Name</th>
|
||||
<th style="padding: 6px 8px;">npub</th>
|
||||
<th style="padding: 6px 8px;">Root?</th>
|
||||
<th style="padding: 6px 8px;">Events in DB</th>
|
||||
<th style="padding: 6px 8px;">Backfill</th>
|
||||
<th style="padding: 6px 8px; width: 40%;">Relays (status, events fetched)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody id="caching-follows-table-body">
|
||||
<tr><td colspan="6" style="text-align: center; font-style: italic; padding: 12px;">Loading follows status...</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- SQL QUERY Section -->
|
||||
<div class="section" id="sqlQuerySection" style="display: none;">
|
||||
<div class="section-header">
|
||||
@@ -419,6 +758,10 @@ AUTH RULES MANAGEMENT
|
||||
<option value="event_kinds">Event Kinds Distribution</option>
|
||||
<option value="time_stats">Time-based Statistics</option>
|
||||
</optgroup>
|
||||
<optgroup label="IP Ban Queries">
|
||||
<option value="banned_ips_summary">Banned IPs Summary</option>
|
||||
<option value="banned_ips_list">All Banned IP Addresses</option>
|
||||
</optgroup>
|
||||
<optgroup label="Query History" id="history-group">
|
||||
<!-- Dynamically populated from localStorage -->
|
||||
</optgroup>
|
||||
|
||||
+1791
-103
File diff suppressed because it is too large
Load Diff
+95
-1
@@ -3297,8 +3297,18 @@ var NostrTools = (() => {
|
||||
this.enablePing = opts.enablePing;
|
||||
}
|
||||
async ensureRelay(url, params) {
|
||||
const rawUrl = url;
|
||||
const debugEnabled = typeof window !== "undefined" && !!window.__SIMPLE_POOL_DEBUG__;
|
||||
url = normalizeURL(url);
|
||||
let relay = this.relays.get(url);
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL ensureRelay START", {
|
||||
rawUrl,
|
||||
normalizedUrl: url,
|
||||
hadRelay: !!relay,
|
||||
relayMapKeysBefore: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
if (!relay) {
|
||||
relay = new AbstractRelay(url, {
|
||||
verifyEvent: this.trustedRelayURLs.has(url) ? alwaysTrue : this.verifyEvent,
|
||||
@@ -3306,13 +3316,38 @@ var NostrTools = (() => {
|
||||
enablePing: this.enablePing
|
||||
});
|
||||
relay.onclose = () => {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL ensureRelay onclose", {
|
||||
normalizedUrl: url,
|
||||
relayMapKeysBeforeDelete: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
this.relays.delete(url);
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL ensureRelay onclose complete", {
|
||||
normalizedUrl: url,
|
||||
relayMapKeysAfterDelete: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
};
|
||||
if (params?.connectionTimeout)
|
||||
relay.connectionTimeout = params.connectionTimeout;
|
||||
this.relays.set(url, relay);
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL ensureRelay created relay", {
|
||||
normalizedUrl: url,
|
||||
relayMapKeysAfterCreate: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
}
|
||||
await relay.connect();
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL ensureRelay connected", {
|
||||
normalizedUrl: url,
|
||||
relayConnected: relay.connected,
|
||||
relayMapKeysAfterConnect: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
return relay;
|
||||
}
|
||||
close(relays) {
|
||||
@@ -3334,16 +3369,26 @@ var NostrTools = (() => {
|
||||
}
|
||||
subscribeMany(relays, filters, params) {
|
||||
params.onauth = params.onauth || params.doauth;
|
||||
const debugEnabled = typeof window !== "undefined" && !!window.__SIMPLE_POOL_DEBUG__;
|
||||
const request = [];
|
||||
const uniqUrls = [];
|
||||
for (let i2 = 0; i2 < relays.length; i2++) {
|
||||
const url = normalizeURL(relays[i2]);
|
||||
if (uniqUrls.indexOf(url) === -1) {
|
||||
uniqUrls.push(url);
|
||||
for (let f2 = 0; f2 < filters.length; f2++) {
|
||||
request.push({ url, filter: filters[f2] });
|
||||
}
|
||||
}
|
||||
}
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL subscribeMany", {
|
||||
relaysInput: relays,
|
||||
uniqUrls,
|
||||
filtersCount: filters.length,
|
||||
requestsCount: request.length
|
||||
});
|
||||
}
|
||||
return this.subscribeMap(request, params);
|
||||
}
|
||||
subscribeMap(requests, params) {
|
||||
@@ -3391,14 +3436,31 @@ var NostrTools = (() => {
|
||||
_knownIds.add(id);
|
||||
return have;
|
||||
};
|
||||
const debugEnabled = typeof window !== "undefined" && !!window.__SIMPLE_POOL_DEBUG__;
|
||||
const allOpened = Promise.all(
|
||||
requests.map(async ({ url, filter }, i2) => {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL subscribeMap request", {
|
||||
index: i2,
|
||||
url,
|
||||
filterKinds: filter?.kinds,
|
||||
hasAuthorFilter: !!filter?.authors,
|
||||
hasPTagFilter: !!filter?.["#p"]
|
||||
});
|
||||
}
|
||||
let relay;
|
||||
try {
|
||||
relay = await this.ensureRelay(url, {
|
||||
connectionTimeout: params.maxWait ? Math.max(params.maxWait * 0.8, params.maxWait - 1e3) : void 0
|
||||
});
|
||||
} catch (err) {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL subscribeMap ensureRelay FAILED", {
|
||||
index: i2,
|
||||
url,
|
||||
error: err?.message || String(err)
|
||||
});
|
||||
}
|
||||
handleClose(i2, err?.message || String(err));
|
||||
return;
|
||||
}
|
||||
@@ -3480,18 +3542,50 @@ var NostrTools = (() => {
|
||||
return events[0] || null;
|
||||
}
|
||||
publish(relays, event, options) {
|
||||
return relays.map(normalizeURL).map(async (url, i2, arr) => {
|
||||
const debugEnabled = typeof window !== "undefined" && !!window.__SIMPLE_POOL_DEBUG__;
|
||||
const normalizedRelays = relays.map(normalizeURL);
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL publish START", {
|
||||
relaysInput: relays,
|
||||
normalizedRelays,
|
||||
eventKind: event?.kind,
|
||||
eventId: event?.id
|
||||
});
|
||||
}
|
||||
return normalizedRelays.map(async (url, i2, arr) => {
|
||||
if (arr.indexOf(url) !== i2) {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL publish duplicate URL", { url, index: i2 });
|
||||
}
|
||||
return Promise.reject("duplicate url");
|
||||
}
|
||||
let r = await this.ensureRelay(url);
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL publish ensured relay", {
|
||||
url,
|
||||
relayConnected: r?.connected,
|
||||
relayMapKeys: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
return r.publish(event).catch(async (err) => {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL publish ERROR", {
|
||||
url,
|
||||
error: err?.message || String(err)
|
||||
});
|
||||
}
|
||||
if (err instanceof Error && err.message.startsWith("auth-required: ") && options?.onauth) {
|
||||
await r.auth(options.onauth);
|
||||
return r.publish(event);
|
||||
}
|
||||
throw err;
|
||||
}).then((reason) => {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL publish SUCCESS", {
|
||||
url,
|
||||
reason
|
||||
});
|
||||
}
|
||||
if (this.trackRelays) {
|
||||
let set = this.seenOn.get(event.id);
|
||||
if (!set) {
|
||||
|
||||
@@ -0,0 +1,813 @@
|
||||
<!DOCTYPE html>
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<html lang="en" dir="ltr">
|
||||
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<title>TEMPLATE</title>
|
||||
|
||||
<link rel="stylesheet" href="./css/client.css" />
|
||||
|
||||
<!-- Initialize theme BEFORE any components load -->
|
||||
<script>
|
||||
(function () {
|
||||
const savedTheme = localStorage.getItem('theme');
|
||||
if (savedTheme === 'dark') {
|
||||
document.documentElement.classList.add('dark-mode');
|
||||
if (document.body) {
|
||||
document.body.classList.add('dark-mode');
|
||||
}
|
||||
}
|
||||
})();
|
||||
</script>
|
||||
<link rel="shortcut icon" type="image/x-icon" href="./favicon/favicon-dots2.ico" />
|
||||
|
||||
<!-- SVG.js library (required by HamburgerMorphing) -->
|
||||
<script src="https://cdn.jsdelivr.net/npm/@svgdotjs/svg.js@3.0/dist/svg.min.js"></script>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<!-- ================================================================
|
||||
HAMBURGER BUTTON (Fixed, separate from header)
|
||||
================================================================
|
||||
The hamburger button is a fixed element outside the header
|
||||
to ensure it stays visible above the sidenav (z-index: 10 > 3).
|
||||
================================================================ -->
|
||||
<div id="divSvgHam" class="divHeaderButtons">
|
||||
<!-- HamburgerMorphing will be injected here -->
|
||||
</div>
|
||||
|
||||
<!-- ================================================================
|
||||
HEADER
|
||||
================================================================
|
||||
Standard header with title (center).
|
||||
================================================================ -->
|
||||
<div id="divHeader">
|
||||
<div id="divHeaderFlexLeft">
|
||||
<!-- Hamburger is now separate fixed element -->
|
||||
</div>
|
||||
|
||||
<div id="divHeaderFlexCenter">
|
||||
<div class="divHeaderText"></div>
|
||||
</div>
|
||||
|
||||
<div id="divHeaderFlexRight">
|
||||
<!-- No button in header right - logout is in sidenav footer -->
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- ================================================================
|
||||
BODY
|
||||
================================================================
|
||||
Main content area. Add your page-specific content here.
|
||||
================================================================ -->
|
||||
<div id="divBody">
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
<!-- ================================================================
|
||||
FOOTER
|
||||
================================================================
|
||||
Three-section footer layout:
|
||||
- Left: Relay status animations (HamburgerMorphing instances)
|
||||
- Center: General status information
|
||||
- Right: Additional information
|
||||
================================================================ -->
|
||||
<div id="divFooter">
|
||||
<div id="divFooterLeft" class="divFooterBox"></div>
|
||||
<div id="divFooterCenter" class="divFooterBox"></div>
|
||||
<div id="divFooterRight" class="divFooterBox"></div>
|
||||
<div id="divFooterBalance" class="divFooterBox">0 sats</div>
|
||||
</div>
|
||||
|
||||
<!-- ================================================================
|
||||
SIDENAV
|
||||
================================================================
|
||||
Slide-out navigation panel. Opens from left when hamburger clicked.
|
||||
Uses flexbox layout to pin version bar to bottom.
|
||||
Includes a version bar footer with theme toggle and logout buttons.
|
||||
================================================================ -->
|
||||
<div id="divSideNav">
|
||||
<div id="divSideNavHeader">
|
||||
<!-- No close button - use main hamburger to close -->
|
||||
</div>
|
||||
|
||||
<div id="divSideNavBody">
|
||||
<div id="divFiles"></div>
|
||||
</div>
|
||||
|
||||
<div id="divAiSection" class="sidenavSection">
|
||||
<div id="divAiSectionTitle" class="sidenavSectionTitle">AI</div>
|
||||
<div id="divAiList" class="sidenavSectionList">
|
||||
<div id="divAiProvidersList">No saved providers yet.</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<div id="divRelaySection">
|
||||
<div id="divRelaySectionTitle">
|
||||
リレー
|
||||
</div>
|
||||
<div id="divRelayList">
|
||||
Loading relays...
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="divBlossomSection">
|
||||
|
||||
<div id="divBlossomSectionTitle">ブロッサム</div>
|
||||
|
||||
<div id="divBlossomList">Loading blossom servers...</div>
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
<div id="divVersionBar">
|
||||
<span id="versionDisplay">v0.0.1</span>
|
||||
<div id="divVersionBarButtons">
|
||||
<button id="themeToggleButton" title="Toggle Dark/Light Mode">
|
||||
<div id="themeToggleHamburgerContainer"></div>
|
||||
</button>
|
||||
<button id="logoutButton" title="Logout">
|
||||
<div id="logoutHamburgerContainer"></div>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- ================================================================
|
||||
REQUIRED SCRIPTS
|
||||
================================================================
|
||||
These scripts must be loaded in this order:
|
||||
1. nostr.bundle.js - Nostr tools library
|
||||
2. nostr-lite.js - Authentication modal (nostr-login-lite)
|
||||
================================================================ -->
|
||||
<script src="./nostr.bundle.js"></script>
|
||||
<script src="./nostr-lite.js"></script>
|
||||
|
||||
<script type="module">
|
||||
/* ================================================================
|
||||
IMPORTS
|
||||
================================================================
|
||||
Import shared NDK functionality from init-ndk.mjs:
|
||||
- initNDKPage() - Initialize authentication and worker
|
||||
- getPubkey() - Get current user's pubkey
|
||||
- subscribe() - Create NDK subscriptions
|
||||
- publishEvent() - Publish events via NDK
|
||||
- disconnect() - Disconnect from worker
|
||||
- getRelayData() - Get relay connection data
|
||||
- getRelayStats() - Get relay activity statistics
|
||||
|
||||
Import HamburgerMorphing for animated icons
|
||||
================================================================ */
|
||||
import {
|
||||
initNDKPage,
|
||||
getPubkey, injectHeaderAvatar,
|
||||
subscribe,
|
||||
publishEvent,
|
||||
disconnect,
|
||||
getVersion,
|
||||
updateVersionDisplay,
|
||||
getUserSettings,
|
||||
patchUserSettings,
|
||||
onUserSettings
|
||||
} from './js/init-ndk.mjs';
|
||||
import { HamburgerMorphing } from "./hamburger_morphing/hamburger.mjs";
|
||||
import { initFooterRelayStatus, updateFooterRelayStatus, initSidenavRelaySection, updateSidenavRelaySection, setRelayActivityState } from './js/relay-ui.mjs';
|
||||
|
||||
import { initBlossomSection, updateBlossomSection } from './js/blossom-ui.mjs';
|
||||
|
||||
import { initAiSectionWithLocalConfig } from './js/ai-ui.mjs';
|
||||
// Version will be loaded asynchronously
|
||||
const versionInfo = await getVersion();
|
||||
const VERSION = versionInfo.VERSION;
|
||||
console.log(`[template.html ${VERSION}] Loading...`);
|
||||
|
||||
/* ================================================================
|
||||
GLOBAL VARIABLES
|
||||
================================================================
|
||||
Track state for hamburger menu, relay status, and theme.
|
||||
================================================================ */
|
||||
let updateIntervalId = null;
|
||||
let currentPubkey = null;
|
||||
|
||||
/*
|
||||
AUTH STATE MODEL (Template reference)
|
||||
------------------------------------------------------------------
|
||||
This template now demonstrates three auth modes for standalone pages:
|
||||
|
||||
- required (default):
|
||||
Behaves like existing pages: login is required immediately.
|
||||
|
||||
- optional:
|
||||
Page can render public/read-only data without login, but can still
|
||||
prompt login later for user actions (publish, settings, etc).
|
||||
|
||||
- none:
|
||||
Never auto-login on load (pure public page).
|
||||
|
||||
URL behavior in this template:
|
||||
- If ?auth=required|optional|none is present, it wins.
|
||||
- Otherwise, if URL includes ?npub=... or ?pubkey=..., mode defaults
|
||||
to optional because pages with explicit profile targets are commonly
|
||||
public-readable.
|
||||
- Otherwise, mode defaults to required.
|
||||
*/
|
||||
let isAuthenticated = false;
|
||||
let authMode = 'required';
|
||||
let authedPageInitialized = false;
|
||||
let relayActivityListenersBound = false;
|
||||
|
||||
// Hamburger menu
|
||||
let hamburgerInstance = null;
|
||||
let isNavOpen = false;
|
||||
|
||||
// Version bar buttons
|
||||
let logoutHamburger = null;
|
||||
let themeToggleHamburger = null;
|
||||
let isDarkMode = false;
|
||||
|
||||
// App-wide user settings (NIP-78 kind 30078, d:user-settings)
|
||||
let pageSettings = {};
|
||||
let unsubscribeUserSettings = null;
|
||||
|
||||
/* ================================================================
|
||||
DOM VARIABLES
|
||||
================================================================
|
||||
Cache DOM element references for better performance.
|
||||
================================================================ */
|
||||
const divBody = document.getElementById("divBody");
|
||||
const divSideNav = document.getElementById("divSideNav");
|
||||
const divSideNavBody = document.getElementById("divSideNavBody");
|
||||
const divFooterCenter = document.getElementById("divFooterCenter");
|
||||
const divFooterRight = document.getElementById("divFooterRight");
|
||||
|
||||
/* ================================================================
|
||||
HAMBURGER MENU
|
||||
================================================================
|
||||
Initialize and control the animated hamburger menu.
|
||||
================================================================ */
|
||||
function initHamburgerMenu() {
|
||||
hamburgerInstance = new HamburgerMorphing('#divSvgHam', {
|
||||
foreground: 'var(--primary-color)',
|
||||
background: 'var(--secondary-color)',
|
||||
hover: 'var(--accent-color)'
|
||||
});
|
||||
hamburgerInstance.animateTo('burger');
|
||||
}
|
||||
|
||||
/* ================================================================
|
||||
SIDENAV FUNCTIONS
|
||||
================================================================
|
||||
Open/close sidenav with hamburger morphing animation.
|
||||
================================================================ */
|
||||
function openNav() {
|
||||
divSideNav.style.zIndex = 3;
|
||||
divSideNav.style.width = "clamp(400px, 50vw, 600px)";
|
||||
isNavOpen = true;
|
||||
if (hamburgerInstance) {
|
||||
hamburgerInstance.animateTo('arrow_left');
|
||||
}
|
||||
|
||||
|
||||
// Initialize version bar buttons when sidenav opens (lazy load)
|
||||
if (!logoutHamburger) {
|
||||
logoutHamburger = new HamburgerMorphing('#logoutHamburgerContainer', {
|
||||
size: 24,
|
||||
foreground: 'var(--primary-color)',
|
||||
background: 'var(--secondary-color)',
|
||||
hover: 'var(--accent-color)'
|
||||
});
|
||||
logoutHamburger.animateTo('x');
|
||||
}
|
||||
|
||||
if (!themeToggleHamburger) {
|
||||
themeToggleHamburger = new HamburgerMorphing('#themeToggleHamburgerContainer', {
|
||||
size: 24,
|
||||
foreground: 'var(--primary-color)',
|
||||
background: 'var(--secondary-color)',
|
||||
hover: 'var(--accent-color)'
|
||||
});
|
||||
|
||||
// Determine current theme
|
||||
const savedTheme = localStorage.getItem('theme');
|
||||
isDarkMode = savedTheme === 'dark' || document.body.classList.contains('dark-mode');
|
||||
const initialShape = isDarkMode ? 'moon' : 'circle';
|
||||
themeToggleHamburger.animateTo(initialShape);
|
||||
}
|
||||
}
|
||||
|
||||
function closeNav() {
|
||||
divSideNav.style.width = "0vw";
|
||||
divSideNav.style.zIndex = -1;
|
||||
isNavOpen = false;
|
||||
if (hamburgerInstance) {
|
||||
hamburgerInstance.animateTo('burger');
|
||||
}
|
||||
}
|
||||
|
||||
function toggleNav() {
|
||||
if (isNavOpen) {
|
||||
closeNav();
|
||||
} else {
|
||||
openNav();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/* ================================================================
|
||||
AUTH MODE HELPERS
|
||||
================================================================
|
||||
These functions are meant as reusable guidance for future pages.
|
||||
================================================================ */
|
||||
function hasTargetPubkeyInUrl() {
|
||||
const params = new URLSearchParams(window.location.search || '');
|
||||
const npub = String(params.get('npub') || '').trim();
|
||||
const pubkey = String(params.get('pubkey') || '').trim();
|
||||
return Boolean(npub || pubkey);
|
||||
}
|
||||
|
||||
function resolveAuthModeFromUrl() {
|
||||
const params = new URLSearchParams(window.location.search || '');
|
||||
const explicitAuth = String(params.get('auth') || '').trim().toLowerCase();
|
||||
if (explicitAuth === 'required' || explicitAuth === 'optional' || explicitAuth === 'none') {
|
||||
return explicitAuth;
|
||||
}
|
||||
|
||||
// Convention: explicit target profiles are public-readable by default.
|
||||
if (hasTargetPubkeyInUrl()) {
|
||||
return 'optional';
|
||||
}
|
||||
|
||||
return 'required';
|
||||
}
|
||||
|
||||
function isAuthRequiredError(error) {
|
||||
const message = String(error?.message || error || '').toLowerCase();
|
||||
return message.includes('authentication required');
|
||||
}
|
||||
|
||||
async function initializeAuthentication(mode) {
|
||||
// required: existing behavior, throw if auth fails.
|
||||
if (mode === 'required') {
|
||||
await initNDKPage();
|
||||
currentPubkey = await getPubkey();
|
||||
isAuthenticated = true;
|
||||
return;
|
||||
}
|
||||
|
||||
// none: public page, no login attempt on load.
|
||||
if (mode === 'none') {
|
||||
isAuthenticated = false;
|
||||
currentPubkey = null;
|
||||
return;
|
||||
}
|
||||
|
||||
// optional: try silent/normal init; if auth required, continue public.
|
||||
try {
|
||||
await initNDKPage();
|
||||
currentPubkey = await getPubkey();
|
||||
isAuthenticated = true;
|
||||
} catch (error) {
|
||||
if (isAuthRequiredError(error)) {
|
||||
console.log('[template.html] Optional auth mode: continuing unauthenticated');
|
||||
isAuthenticated = false;
|
||||
currentPubkey = null;
|
||||
return;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async function initializeAuthenticatedPageFeatures() {
|
||||
if (!isAuthenticated || authedPageInitialized) return;
|
||||
|
||||
await injectHeaderAvatar(currentPubkey);
|
||||
console.log('[template.html] Authenticated as:', currentPubkey);
|
||||
|
||||
// Hydrate app-wide user settings for this page
|
||||
try {
|
||||
pageSettings = await getUserSettings();
|
||||
} catch (error) {
|
||||
console.warn('[template.html] getUserSettings failed:', error);
|
||||
pageSettings = {};
|
||||
}
|
||||
|
||||
// Subscribe to live user settings updates (cross-tab + publish echoes)
|
||||
if (!unsubscribeUserSettings) {
|
||||
unsubscribeUserSettings = onUserSettings((settings) => {
|
||||
pageSettings = settings || {};
|
||||
// TODO: Re-render page-specific UI from pageSettings here.
|
||||
});
|
||||
}
|
||||
|
||||
// Initialize relay-dependent UI only once authenticated.
|
||||
initFooterRelayStatus();
|
||||
initSidenavRelaySection();
|
||||
await initBlossomSection();
|
||||
initAiSectionWithLocalConfig();
|
||||
await UpdateFooter();
|
||||
|
||||
if (!updateIntervalId) {
|
||||
updateIntervalId = setInterval(UpdateFooter, 1000);
|
||||
}
|
||||
|
||||
// Relay activity listeners only matter after worker init/auth.
|
||||
if (!relayActivityListenersBound) {
|
||||
window.addEventListener('ndkRelayActivity', (event) => {
|
||||
const { relayUrl, activity, stats } = event.detail;
|
||||
console.log(`[template.html] Relay activity: ${relayUrl} - ${activity}`, stats);
|
||||
setRelayActivityState(relayUrl, activity);
|
||||
});
|
||||
|
||||
window.addEventListener('message', (event) => {
|
||||
if (event.data && event.data.type === 'relayActivity') {
|
||||
const { relayUrl, activity } = event.data;
|
||||
console.log(`[template.html] Relay activity: ${relayUrl} - ${activity}`);
|
||||
setRelayActivityState(relayUrl, activity);
|
||||
}
|
||||
});
|
||||
|
||||
relayActivityListenersBound = true;
|
||||
}
|
||||
|
||||
authedPageInitialized = true;
|
||||
}
|
||||
|
||||
async function promptLoginIfNeeded() {
|
||||
if (isAuthenticated) return true;
|
||||
|
||||
await initNDKPage();
|
||||
currentPubkey = await getPubkey();
|
||||
isAuthenticated = true;
|
||||
await initializeAuthenticatedPageFeatures();
|
||||
return true;
|
||||
}
|
||||
|
||||
/* ================================================================
|
||||
UPDATE FOOTER
|
||||
================================================================
|
||||
Update footer sections with relay status, pubkey, and other info.
|
||||
Called periodically by update loop.
|
||||
================================================================ */
|
||||
const UpdateFooter = async () => {
|
||||
|
||||
try {
|
||||
// Update relay status visuals in footer and sidenav
|
||||
await updateFooterRelayStatus();
|
||||
await updateSidenavRelaySection();
|
||||
|
||||
await updateBlossomSection();
|
||||
// Clear center and right sections
|
||||
divFooterCenter.innerHTML = '';
|
||||
divFooterRight.innerHTML = '';
|
||||
} catch (error) {
|
||||
console.error('[template.html] Error updating footer:', error);
|
||||
}
|
||||
};
|
||||
|
||||
/* ================================================================
|
||||
LOGOUT
|
||||
================================================================
|
||||
Complete logout process:
|
||||
1. Stop update loop
|
||||
2. Disconnect from NDK worker
|
||||
3. Logout from nostr-login-lite
|
||||
4. Clear all storage (localStorage, sessionStorage, IndexedDB)
|
||||
5. Reload page
|
||||
================================================================ */
|
||||
const Logout = async () => {
|
||||
console.log("[template.html] Starting logout process...");
|
||||
|
||||
// Stop the update loop
|
||||
if (updateIntervalId) {
|
||||
clearInterval(updateIntervalId);
|
||||
updateIntervalId = null;
|
||||
}
|
||||
|
||||
// Disconnect from worker
|
||||
disconnect();
|
||||
|
||||
// Logout from nostr-login-lite
|
||||
if (window.NOSTR_LOGIN_LITE && window.NOSTR_LOGIN_LITE.logout) {
|
||||
await window.NOSTR_LOGIN_LITE.logout();
|
||||
}
|
||||
|
||||
// Clear all storage
|
||||
localStorage.clear();
|
||||
sessionStorage.clear();
|
||||
|
||||
// Clear IndexedDB
|
||||
if (window.indexedDB) {
|
||||
const databases = await window.indexedDB.databases();
|
||||
for (const db of databases) {
|
||||
if (db.name) {
|
||||
window.indexedDB.deleteDatabase(db.name);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
console.log("[template.html] Logged out, reloading page");
|
||||
location.reload(true);
|
||||
};
|
||||
|
||||
/* ================================================================
|
||||
EVENT LISTENERS
|
||||
================================================================
|
||||
Wire up UI interactions.
|
||||
Main hamburger button click handler is set up in main() after initialization.
|
||||
================================================================ */
|
||||
|
||||
/* ================================================================
|
||||
SUBSCRIPTION EXAMPLE
|
||||
================================================================
|
||||
Example of how to subscribe to Nostr events:
|
||||
|
||||
const sub = subscribe(
|
||||
{ kinds: [1], authors: [pubkey], limit: 10 },
|
||||
{ closeOnEose: false, cacheUsage: 'CACHE_FIRST' }
|
||||
);
|
||||
|
||||
Cache usage options:
|
||||
- 'CACHE_FIRST' - Check cache first, then relays
|
||||
- 'ONLY_RELAY' - Only query relays
|
||||
- 'ONLY_CACHE' - Only query cache
|
||||
- 'PARALLEL' - Query cache and relays simultaneously
|
||||
|
||||
Listen for events via window events:
|
||||
window.addEventListener('ndkEvent', (event) => {
|
||||
const evt = event.detail;
|
||||
console.log('Received event:', evt);
|
||||
});
|
||||
================================================================ */
|
||||
|
||||
/* ================================================================
|
||||
PUBLISH EXAMPLE
|
||||
================================================================
|
||||
Example of how to publish a Nostr event:
|
||||
|
||||
const event = {
|
||||
created_at: Math.floor(Date.now() / 1000),
|
||||
kind: 1,
|
||||
tags: [],
|
||||
content: "Hello, Nostr!"
|
||||
};
|
||||
|
||||
try {
|
||||
const result = await publishEvent(event);
|
||||
console.log("✅ Published to:", result.relayResults.successful);
|
||||
console.log("❌ Failed:", result.relayResults.failed);
|
||||
console.log("Total relays:", result.totalRelays);
|
||||
} catch (error) {
|
||||
console.error("Publish error:", error);
|
||||
}
|
||||
|
||||
Note: Events are automatically signed by the NDK worker using
|
||||
the message-based signer (which calls window.nostr.signEvent).
|
||||
================================================================ */
|
||||
|
||||
/* ================================================================
|
||||
USER SETTINGS EXAMPLE (NIP-78)
|
||||
================================================================
|
||||
Read/subscribe/write helper pattern for all pages:
|
||||
|
||||
// Read latest merged settings (cache + relay hydrated by worker)
|
||||
const settings = await getUserSettings();
|
||||
|
||||
// Subscribe to cross-tab updates
|
||||
const unsubscribe = onUserSettings((nextSettings) => {
|
||||
// Re-render page from nextSettings
|
||||
});
|
||||
|
||||
// Patch only your feature namespace
|
||||
await patchUserSettings({
|
||||
myFeature: {
|
||||
someFlag: true
|
||||
}
|
||||
});
|
||||
|
||||
// On page teardown (if applicable)
|
||||
// unsubscribe();
|
||||
================================================================ */
|
||||
|
||||
/* ================================================================
|
||||
INITIALIZATION
|
||||
================================================================
|
||||
Main initialization sequence:
|
||||
1. Initialize hamburger menu
|
||||
2. Set up hamburger click handler
|
||||
3. Resolve auth mode from URL/query policy
|
||||
4. Initialize authentication based on mode
|
||||
5. Initialize authenticated-only features (if signed in)
|
||||
6. Set up version bar button listeners
|
||||
7. Restore sidenav state
|
||||
8. Update version display
|
||||
|
||||
Notes:
|
||||
- required mode = existing behavior (prompt login on load)
|
||||
- optional mode = allow public load, login later on demand
|
||||
- none mode = no auto-login on load
|
||||
================================================================ */
|
||||
(async function main() {
|
||||
console.log("[template.html] Starting initialization...");
|
||||
|
||||
try {
|
||||
// Initialize hamburger menu first
|
||||
initHamburgerMenu();
|
||||
|
||||
// Add click handler to hamburger
|
||||
const divSvgHam = document.getElementById('divSvgHam');
|
||||
if (divSvgHam) {
|
||||
divSvgHam.addEventListener('click', toggleNav);
|
||||
}
|
||||
|
||||
// Initialize version bar buttons
|
||||
const themeToggleButton = document.getElementById('themeToggleButton');
|
||||
const logoutButton = document.getElementById('logoutButton');
|
||||
|
||||
if (themeToggleButton) {
|
||||
themeToggleButton.addEventListener('click', () => {
|
||||
isDarkMode = !isDarkMode;
|
||||
if (isDarkMode) {
|
||||
localStorage.setItem('theme', 'dark');
|
||||
} else {
|
||||
localStorage.setItem('theme', 'light');
|
||||
}
|
||||
// Save sidenav state before reload
|
||||
localStorage.setItem('sidenavWasOpen', isNavOpen ? 'true' : 'false');
|
||||
window.location.reload();
|
||||
});
|
||||
}
|
||||
|
||||
if (logoutButton) {
|
||||
logoutButton.addEventListener('click', async () => {
|
||||
try {
|
||||
// In optional/none modes this doubles as a "Sign in" entry point.
|
||||
if (!isAuthenticated) {
|
||||
await promptLoginIfNeeded();
|
||||
return;
|
||||
}
|
||||
await Logout();
|
||||
} catch (error) {
|
||||
console.error('Logout/login action failed:', error);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Resolve and initialize page auth policy.
|
||||
authMode = resolveAuthModeFromUrl();
|
||||
console.log('[template.html] Resolved auth mode:', authMode);
|
||||
await initializeAuthentication(authMode);
|
||||
|
||||
// Initialize authenticated features only when signed in.
|
||||
await initializeAuthenticatedPageFeatures();
|
||||
|
||||
/* ============================================================
|
||||
EXAMPLE: Subscribe to events
|
||||
============================================================
|
||||
Uncomment to subscribe to user's notes:
|
||||
|
||||
const notesSub = subscribe(
|
||||
{ kinds: [1], authors: [currentPubkey], limit: 10 },
|
||||
{ closeOnEose: false, cacheUsage: 'CACHE_FIRST' }
|
||||
);
|
||||
console.log("[template.html] Subscribed to kind 1");
|
||||
============================================================ */
|
||||
|
||||
/* ============================================================
|
||||
EXAMPLE: Listen for events from worker
|
||||
============================================================
|
||||
Uncomment to handle incoming events:
|
||||
|
||||
window.addEventListener('ndkEvent', (event) => {
|
||||
const evt = event.detail;
|
||||
console.log("[template.html] Received event:", evt.kind, evt.pubkey);
|
||||
|
||||
if (evt.pubkey === currentPubkey) {
|
||||
if (evt.kind === 1) {
|
||||
// Handle note event
|
||||
console.log("Note:", evt.content);
|
||||
}
|
||||
}
|
||||
});
|
||||
============================================================ */
|
||||
|
||||
/* ============================================================
|
||||
EXAMPLE: Listen for cached profile
|
||||
============================================================
|
||||
Uncomment to handle cached profile data:
|
||||
|
||||
window.addEventListener('ndkProfile', (event) => {
|
||||
console.log("[template.html] Cached profile:", event.detail);
|
||||
// event.detail contains profile object (name, about, etc.)
|
||||
});
|
||||
============================================================ */
|
||||
|
||||
// Restore sidenav state if it was open before theme toggle
|
||||
const sidenavWasOpen = localStorage.getItem('sidenavWasOpen');
|
||||
if (sidenavWasOpen === 'true') {
|
||||
localStorage.removeItem('sidenavWasOpen');
|
||||
openNav();
|
||||
}
|
||||
|
||||
// Optional UX note for public mode pages.
|
||||
if (!isAuthenticated && (authMode === 'optional' || authMode === 'none')) {
|
||||
divFooterCenter.textContent = 'Public mode';
|
||||
divFooterRight.textContent = 'Sign in from side menu for private features';
|
||||
}
|
||||
|
||||
// Update version display
|
||||
await updateVersionDisplay();
|
||||
|
||||
console.log('[template.html] Initialization complete');
|
||||
} catch (error) {
|
||||
console.error('[template.html] Initialization failed:', error);
|
||||
divBody.innerHTML = `<div style="text-align: center; padding: 50px;">
|
||||
<div style="font-size: 24px; margin-bottom: 20px; color: red;">❌ Authentication Error</div>
|
||||
<div style="font-size: 16px; color: #666;">${error.message}</div>
|
||||
<div style="margin-top: 20px;">
|
||||
<button onclick="location.reload()" style="padding: 10px 20px; font-size: 16px;">Retry</button>
|
||||
</div>
|
||||
</div>`;
|
||||
}
|
||||
})();
|
||||
|
||||
/* ================================================================
|
||||
WORKER MESSAGE TYPES
|
||||
================================================================
|
||||
The NDK worker can send these message types:
|
||||
|
||||
1. 'response' - Response to init/subscribe/publish requests
|
||||
- data.profile - User profile (from init)
|
||||
- data.relays - User relays (from init)
|
||||
- data.success - Publish success status
|
||||
- data.relayResults - Relay publish results
|
||||
|
||||
2. 'event' - Nostr event from subscription
|
||||
- Dispatched as 'ndkEvent' window event
|
||||
- event.detail contains the Nostr event
|
||||
|
||||
3. 'eose' - End of stored events for subscription
|
||||
- Dispatched as 'ndkEose' window event
|
||||
- event.detail.subId contains subscription ID
|
||||
|
||||
4. 'signRequest' - Request to sign event/encrypt/decrypt
|
||||
- Handled automatically by init-ndk.mjs
|
||||
- Calls window.nostr methods and sends response
|
||||
|
||||
5. 'error' - Error from worker
|
||||
- Logged to console automatically
|
||||
|
||||
6. 'relayActivity' - Relay read/write activity notification
|
||||
- Dispatched as 'ndkRelayActivity' window event
|
||||
- Used to animate relay status icons in footer
|
||||
================================================================ */
|
||||
|
||||
/* ================================================================
|
||||
DISTRIBUTED ARCHITECTURE NOTES
|
||||
================================================================
|
||||
Each page is independently accessible and self-contained:
|
||||
|
||||
1. Authentication persists via nostr-login-lite localStorage
|
||||
- Login once on any page
|
||||
- All other pages automatically authenticated
|
||||
|
||||
2. NDK SharedWorker is shared across all tabs/pages
|
||||
- Single NDK instance manages all connections
|
||||
- Subscriptions from all pages handled by one worker
|
||||
- Events broadcast to all connected pages
|
||||
|
||||
3. Dexie cache is shared across all pages
|
||||
- IndexedDB persists across sessions
|
||||
- Cache-first queries are fast
|
||||
- Reduces relay load
|
||||
|
||||
4. User settings are centralized and shared
|
||||
- Worker hydrates kind 30078 (`d:user-settings`) on init
|
||||
- Pages read via getUserSettings()
|
||||
- Pages patch via patchUserSettings({ featureNamespace: ... })
|
||||
- Pages subscribe via onUserSettings() for live updates
|
||||
|
||||
5. Each page can be distributed independently
|
||||
- Copy template.html and customize
|
||||
- No dependencies on other pages
|
||||
- Works standalone or as part of suite
|
||||
|
||||
6. Message-based signer bridges worker and page
|
||||
- Worker's NDK uses MessageBasedSigner
|
||||
- Signer sends sign requests to page
|
||||
- Page calls window.nostr.signEvent()
|
||||
- Response sent back to worker
|
||||
- NDK completes signing and publishing
|
||||
|
||||
7. Relay status visualization
|
||||
- Footer left section shows connected relays
|
||||
- Each relay has animated icon (HamburgerMorphing)
|
||||
- Icons morph based on activity (read/write)
|
||||
- Temporary animations show real-time activity
|
||||
================================================================ */
|
||||
</script>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
+54
-13
@@ -1,6 +1,6 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Build fully static MUSL binaries for C-Relay using Alpine Docker
|
||||
# Build fully static MUSL binaries for C-Relay-PG using Alpine Docker
|
||||
# Produces truly portable binaries with zero runtime dependencies
|
||||
|
||||
set -e
|
||||
@@ -11,19 +11,52 @@ DOCKERFILE="$SCRIPT_DIR/Dockerfile.alpine-musl"
|
||||
|
||||
# Parse command line arguments
|
||||
DEBUG_BUILD=false
|
||||
if [[ "$1" == "--debug" ]]; then
|
||||
DEBUG_BUILD=true
|
||||
DB_BACKEND="${DB_BACKEND:-postgres}"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--debug)
|
||||
DEBUG_BUILD=true
|
||||
shift
|
||||
;;
|
||||
--db-backend)
|
||||
if [[ -z "$2" ]]; then
|
||||
echo "ERROR: --db-backend requires a value (sqlite|postgres)"
|
||||
exit 1
|
||||
fi
|
||||
DB_BACKEND="$2"
|
||||
shift 2
|
||||
;;
|
||||
--db-backend=*)
|
||||
DB_BACKEND="${1#*=}"
|
||||
shift
|
||||
;;
|
||||
*)
|
||||
echo "ERROR: Unknown argument: $1"
|
||||
echo "Usage: $0 [--debug] [--db-backend postgres|sqlite]"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ "$DB_BACKEND" != "sqlite" && "$DB_BACKEND" != "postgres" ]]; then
|
||||
echo "ERROR: Invalid DB backend '$DB_BACKEND'. Use sqlite or postgres."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ "$DEBUG_BUILD" == "true" ]]; then
|
||||
echo "=========================================="
|
||||
echo "C-Relay MUSL Static Binary Builder (DEBUG MODE)"
|
||||
echo "C-Relay-PG MUSL Static Binary Builder (DEBUG MODE)"
|
||||
echo "=========================================="
|
||||
else
|
||||
echo "=========================================="
|
||||
echo "C-Relay MUSL Static Binary Builder (PRODUCTION MODE)"
|
||||
echo "C-Relay-PG MUSL Static Binary Builder (PRODUCTION MODE)"
|
||||
echo "=========================================="
|
||||
fi
|
||||
echo "Project directory: $SCRIPT_DIR"
|
||||
echo "Build directory: $BUILD_DIR"
|
||||
echo "Debug build: $DEBUG_BUILD"
|
||||
echo "DB backend: $DB_BACKEND"
|
||||
echo ""
|
||||
|
||||
# Create build directory
|
||||
@@ -63,20 +96,25 @@ ARCH=$(uname -m)
|
||||
case "$ARCH" in
|
||||
x86_64)
|
||||
PLATFORM="linux/amd64"
|
||||
OUTPUT_NAME="c_relay_static_x86_64"
|
||||
OUTPUT_NAME="c_relay_pg_static_x86_64"
|
||||
;;
|
||||
aarch64|arm64)
|
||||
PLATFORM="linux/arm64"
|
||||
OUTPUT_NAME="c_relay_static_arm64"
|
||||
OUTPUT_NAME="c_relay_pg_static_arm64"
|
||||
;;
|
||||
*)
|
||||
echo "WARNING: Unknown architecture: $ARCH"
|
||||
echo "Defaulting to linux/amd64"
|
||||
PLATFORM="linux/amd64"
|
||||
OUTPUT_NAME="c_relay_static_${ARCH}"
|
||||
OUTPUT_NAME="c_relay_pg_static_${ARCH}"
|
||||
;;
|
||||
esac
|
||||
|
||||
# Append _debug suffix to output name for debug builds so production binary is never overwritten
|
||||
if [ "$DEBUG_BUILD" = true ]; then
|
||||
OUTPUT_NAME="${OUTPUT_NAME}_debug"
|
||||
fi
|
||||
|
||||
echo "Building for platform: $PLATFORM"
|
||||
echo "Output binary: $OUTPUT_NAME"
|
||||
echo ""
|
||||
@@ -111,14 +149,15 @@ echo "=========================================="
|
||||
echo "This will:"
|
||||
echo " - Use Alpine Linux (native MUSL)"
|
||||
echo " - Build all dependencies statically"
|
||||
echo " - Compile c-relay with full static linking"
|
||||
echo " - Compile c-relay-pg with full static linking"
|
||||
echo ""
|
||||
|
||||
$DOCKER_CMD build \
|
||||
--platform "$PLATFORM" \
|
||||
--build-arg DEBUG_BUILD=$DEBUG_BUILD \
|
||||
--build-arg DB_BACKEND=$DB_BACKEND \
|
||||
-f "$DOCKERFILE" \
|
||||
-t c-relay-musl-builder:latest \
|
||||
-t c-relay-pg-musl-builder:latest \
|
||||
--progress=plain \
|
||||
. || {
|
||||
echo ""
|
||||
@@ -140,16 +179,17 @@ echo "=========================================="
|
||||
$DOCKER_CMD build \
|
||||
--platform "$PLATFORM" \
|
||||
--build-arg DEBUG_BUILD=$DEBUG_BUILD \
|
||||
--build-arg DB_BACKEND=$DB_BACKEND \
|
||||
--target builder \
|
||||
-f "$DOCKERFILE" \
|
||||
-t c-relay-static-builder-stage:latest \
|
||||
-t c-relay-pg-static-builder-stage:latest \
|
||||
. > /dev/null 2>&1
|
||||
|
||||
# Create a temporary container to copy the binary
|
||||
CONTAINER_ID=$($DOCKER_CMD create c-relay-static-builder-stage:latest)
|
||||
CONTAINER_ID=$($DOCKER_CMD create c-relay-pg-static-builder-stage:latest)
|
||||
|
||||
# Copy binary from container
|
||||
$DOCKER_CMD cp "$CONTAINER_ID:/build/c_relay_static" "$BUILD_DIR/$OUTPUT_NAME" || {
|
||||
$DOCKER_CMD cp "$CONTAINER_ID:/build/c_relay_pg_static" "$BUILD_DIR/$OUTPUT_NAME" || {
|
||||
echo "ERROR: Failed to extract binary from container"
|
||||
$DOCKER_CMD rm "$CONTAINER_ID" 2>/dev/null
|
||||
exit 1
|
||||
@@ -219,6 +259,7 @@ if [ "$DEBUG_BUILD" = true ]; then
|
||||
else
|
||||
echo "Build Type: PRODUCTION (optimized, stripped)"
|
||||
fi
|
||||
echo "DB Backend: $DB_BACKEND"
|
||||
if [ "$TRULY_STATIC" = true ]; then
|
||||
echo "Linkage: Fully static binary (Alpine MUSL-based)"
|
||||
echo "Portability: Works on ANY Linux distribution"
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
# Database files (from c-relay running in this directory)
|
||||
*.db
|
||||
*.db-shm
|
||||
*.db-wal
|
||||
|
||||
# Build artifacts
|
||||
*.o
|
||||
*.a
|
||||
|
||||
# Binary (built into root, not version-controlled)
|
||||
caching_relay
|
||||
caching_relay_static_*
|
||||
|
||||
# Logs
|
||||
*.log
|
||||
crelay.log
|
||||
|
||||
# Test configs (not for version control)
|
||||
test_config*.jsonc
|
||||
|
||||
# Note: caching_relay_config.jsonc IS version-controlled (it's the default
|
||||
# config + state store). Don't ignore it.
|
||||
|
||||
# Docker build context temp (copied nostr_core_lib during static build)
|
||||
nostr_core_lib/
|
||||
|
||||
# Editor
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
.vscode/
|
||||
@@ -0,0 +1,125 @@
|
||||
# Alpine-based MUSL static binary builder for caching_relay
|
||||
# Produces a truly portable binary with zero runtime dependencies.
|
||||
# Adapted from c-relay's Dockerfile.alpine-musl.
|
||||
|
||||
ARG DEBUG_BUILD=false
|
||||
|
||||
FROM alpine:3.19 AS builder
|
||||
|
||||
ARG DEBUG_BUILD=false
|
||||
|
||||
# Install build dependencies
|
||||
RUN apk add --no-cache \
|
||||
build-base \
|
||||
musl-dev \
|
||||
git \
|
||||
cmake \
|
||||
pkgconfig \
|
||||
autoconf \
|
||||
automake \
|
||||
libtool \
|
||||
openssl-dev \
|
||||
openssl-libs-static \
|
||||
zlib-dev \
|
||||
zlib-static \
|
||||
curl-dev \
|
||||
curl-static \
|
||||
sqlite-dev \
|
||||
linux-headers \
|
||||
wget \
|
||||
bash
|
||||
|
||||
WORKDIR /build
|
||||
|
||||
# Build libsecp256k1 static
|
||||
RUN cd /tmp && \
|
||||
git clone https://github.com/bitcoin-core/secp256k1.git && \
|
||||
cd secp256k1 && \
|
||||
./autogen.sh && \
|
||||
./configure --enable-static --disable-shared --prefix=/usr \
|
||||
CFLAGS="-fPIC" && \
|
||||
make -j$(nproc) && \
|
||||
make install && \
|
||||
rm -rf /tmp/secp256k1
|
||||
|
||||
# Build libwebsockets static with minimal features
|
||||
RUN cd /tmp && \
|
||||
git clone --depth 1 --branch v4.3.3 https://github.com/warmcat/libwebsockets.git && \
|
||||
cd libwebsockets && \
|
||||
mkdir build && cd build && \
|
||||
cmake .. \
|
||||
-DLWS_WITH_STATIC=ON \
|
||||
-DLWS_WITH_SHARED=OFF \
|
||||
-DLWS_WITH_SSL=ON \
|
||||
-DLWS_WITHOUT_TESTAPPS=ON \
|
||||
-DLWS_WITHOUT_TEST_SERVER=ON \
|
||||
-DLWS_WITHOUT_TEST_CLIENT=ON \
|
||||
-DLWS_WITHOUT_TEST_PING=ON \
|
||||
-DLWS_WITH_HTTP2=OFF \
|
||||
-DLWS_WITH_LIBUV=OFF \
|
||||
-DLWS_WITH_LIBEVENT=OFF \
|
||||
-DLWS_IPV6=ON \
|
||||
-DCMAKE_BUILD_TYPE=Release \
|
||||
-DCMAKE_INSTALL_PREFIX=/usr \
|
||||
-DCMAKE_C_FLAGS="-fPIC" && \
|
||||
make -j$(nproc) && \
|
||||
make install && \
|
||||
rm -rf /tmp/libwebsockets
|
||||
|
||||
# Copy nostr_core_lib source (sibling project)
|
||||
COPY nostr_core_lib /build/nostr_core_lib/
|
||||
|
||||
# Build nostr_core_lib with the NIPs needed by the relay pool + signer:
|
||||
# 1 (basic), 6 (keys), 19 (npub bech32), 4 (legacy encryption, signer dep),
|
||||
# 42 (auth, relay pool dep), 44 (modern encryption, signer dep)
|
||||
RUN cd nostr_core_lib && \
|
||||
chmod +x build.sh && \
|
||||
sed -i 's/CFLAGS="-Wall -Wextra -std=c99 -fPIC -O2"/CFLAGS="-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -Wall -Wextra -std=c99 -fPIC -O2"/' build.sh && \
|
||||
rm -f *.o *.a 2>/dev/null || true && \
|
||||
./build.sh --nips=1,4,6,19,42,44 && \
|
||||
if [ -f libnostr_core_arm64.a ]; then \
|
||||
cp libnostr_core_arm64.a libnostr_core.a; \
|
||||
elif [ -f libnostr_core_x64.a ]; then \
|
||||
cp libnostr_core_x64.a libnostr_core.a; \
|
||||
else \
|
||||
echo "ERROR: No supported nostr_core static library produced"; \
|
||||
ls -la *.a 2>/dev/null || true; \
|
||||
exit 1; \
|
||||
fi
|
||||
|
||||
# Copy caching_relay source LAST (only this layer rebuilds on source changes)
|
||||
COPY src/ /build/src/
|
||||
|
||||
# Build caching_relay with full static linking.
|
||||
# No sqlite (the daemon doesn't use it), no c_utils (not needed).
|
||||
RUN if [ "$DEBUG_BUILD" = "true" ]; then \
|
||||
CFLAGS="-g -O2 -DDEBUG -fno-omit-frame-pointer"; \
|
||||
STRIP_CMD="echo 'Keeping debug symbols'"; \
|
||||
else \
|
||||
CFLAGS="-O2"; \
|
||||
STRIP_CMD="strip /build/caching_relay_static"; \
|
||||
fi && \
|
||||
gcc -static $CFLAGS -Wall -Wextra -std=c99 \
|
||||
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 \
|
||||
-I. -Isrc -Inostr_core_lib -Inostr_core_lib/nostr_core \
|
||||
-Inostr_core_lib/cjson -Inostr_core_lib/nostr_websocket \
|
||||
src/main.c src/debug.c src/jsonc_strip.c src/config.c src/state.c \
|
||||
src/follow_graph.c src/relay_sink.c src/live_subscriber.c \
|
||||
src/backfill.c src/relay_discovery.c \
|
||||
-o /build/caching_relay_static \
|
||||
nostr_core_lib/libnostr_core.a \
|
||||
-lwebsockets -lssl -lcrypto -lsecp256k1 \
|
||||
-lcurl -lz -lpthread -lm -ldl && \
|
||||
eval "$STRIP_CMD"
|
||||
|
||||
# Verify it's truly static
|
||||
RUN echo "=== Binary Information ===" && \
|
||||
file /build/caching_relay_static && \
|
||||
ls -lh /build/caching_relay_static && \
|
||||
echo "=== Checking for dynamic dependencies ===" && \
|
||||
(ldd /build/caching_relay_static 2>&1 || echo "Binary is static") && \
|
||||
echo "=== Build complete ==="
|
||||
|
||||
# Output stage - just the binary
|
||||
FROM scratch AS output
|
||||
COPY --from=builder /build/caching_relay_static /caching_relay_static
|
||||
@@ -0,0 +1,67 @@
|
||||
# caching_relay Makefile - statically linked C99 binary, c-relay style
|
||||
|
||||
# Use bash for reliable glob expansion in recipes (dash handles globs differently).
|
||||
SHELL := /bin/bash
|
||||
|
||||
CC = gcc
|
||||
CFLAGS = -Wall -Wextra -std=c99 -g -O2
|
||||
|
||||
# nostr_core_lib is inside the c-relay-pg project root (one dir up from caching/).
|
||||
NOSTR_CORE_DIR = ../nostr_core_lib
|
||||
|
||||
INCLUDES = -I. -Isrc -I$(NOSTR_CORE_DIR) -I$(NOSTR_CORE_DIR)/nostr_core \
|
||||
-I$(NOSTR_CORE_DIR)/cjson -I$(NOSTR_CORE_DIR)/nostr_websocket \
|
||||
-I/usr/include/postgresql
|
||||
|
||||
# -lsqlite3: nostr_core_lib's request_validator.x64.o references SQLite symbols;
|
||||
# we use --whole-archive so the linker pulls in all objects (needed for NIP-42
|
||||
# cross-references within the archive), which means SQLite must be satisfied.
|
||||
# No c_utils_lib: nostr_core_lib does not depend on it for the NIPs we use.
|
||||
# -lpq: PostgreSQL client library for the caching_event_inbox integration.
|
||||
LIBS = -lwebsockets -lssl -lcrypto -lsecp256k1 -lcurl -lz -ldl -lpthread -lm -lpq -lsqlite3
|
||||
|
||||
MAIN_SRC = src/main.c src/debug.c src/jsonc_strip.c src/config.c src/state.c \
|
||||
src/follow_graph.c src/relay_sink.c src/live_subscriber.c \
|
||||
src/backfill.c src/relay_discovery.c \
|
||||
src/pg_inbox.c src/pg_config.c
|
||||
|
||||
# Architecture detection
|
||||
ARCH = $(shell uname -m)
|
||||
ifeq ($(ARCH),x86_64)
|
||||
NOSTR_CORE_LIB = $(NOSTR_CORE_DIR)/libnostr_core_x64.a
|
||||
else ifeq ($(ARCH),aarch64)
|
||||
NOSTR_CORE_LIB = $(NOSTR_CORE_DIR)/libnostr_core_arm64.a
|
||||
else ifeq ($(ARCH),arm64)
|
||||
NOSTR_CORE_LIB = $(NOSTR_CORE_DIR)/libnostr_core_arm64.a
|
||||
else
|
||||
NOSTR_CORE_LIB = $(NOSTR_CORE_DIR)/libnostr_core_x64.a
|
||||
endif
|
||||
|
||||
# Binary goes in the c-relay-pg build/ directory so the launcher can find it.
|
||||
TARGET = ../build/caching_relay
|
||||
|
||||
all: $(TARGET)
|
||||
|
||||
# Build nostr_core_lib with the NIPs needed by the relay pool + signer:
|
||||
# 1 (basic), 6 (keys), 19 (npub bech32), 4 (legacy encryption, signer dep),
|
||||
# 42 (auth, relay pool dep), 44 (modern encryption, signer dep)
|
||||
$(NOSTR_CORE_LIB):
|
||||
@echo "Building nostr_core_lib with required NIPs..."
|
||||
cd $(NOSTR_CORE_DIR) && ./build.sh --nips=1,4,6,19,42,44
|
||||
|
||||
$(TARGET): $(MAIN_SRC) $(NOSTR_CORE_LIB)
|
||||
@echo "Compiling caching_relay for architecture: $(ARCH)"
|
||||
@# Extract all objects from the static library and link them directly.
|
||||
@# This avoids archive symbol resolution ordering issues (NIP-42 cross-refs).
|
||||
@# Use a fixed temp dir and chain all commands with && so failures stop the build.
|
||||
@rm -rf /tmp/cr_lib && mkdir -p /tmp/cr_lib && \
|
||||
ar x $(NOSTR_CORE_LIB) --output=/tmp/cr_lib && \
|
||||
echo "Extracted $$(ls /tmp/cr_lib/*.o | wc -l) objects" && \
|
||||
$(CC) $(CFLAGS) $(INCLUDES) $(MAIN_SRC) /tmp/cr_lib/*.o -o $(TARGET) $(LIBS) && \
|
||||
rm -rf /tmp/cr_lib && \
|
||||
echo "Build complete: $(TARGET)"
|
||||
|
||||
clean:
|
||||
rm -f $(TARGET)
|
||||
|
||||
.PHONY: all clean
|
||||
@@ -0,0 +1,297 @@
|
||||
# caching_relay
|
||||
|
||||
A C99 daemon that caches Nostr events from people you follow into a local relay,
|
||||
so your Nostr client can point at a single fast local relay instead of fanning
|
||||
out to dozens of upstream relays.
|
||||
|
||||
## What it does
|
||||
|
||||
1. Reads a `.jsonc` config file listing your **root npub(s)**, upstream relays,
|
||||
a local relay URL, and the event kinds to cache.
|
||||
2. For each root npub, fetches its kind-3 contact list to discover **followed
|
||||
pubkeys** (your follows + their follows).
|
||||
3. **Live-subscribes** to new events of the configured kinds from the union of
|
||||
followed pubkeys.
|
||||
4. **Backfills** historical events using a progressive window-expansion strategy
|
||||
(24h → 7d → 30d → 90d → 365d), round-robin per pubkey, throttled to be polite
|
||||
to upstream relays.
|
||||
5. **Re-publishes** every fetched event to the local relay via a plain WebSocket
|
||||
`EVENT` client connection (relay-agnostic - works with c-relay, c-relay-pg,
|
||||
or any Nostr relay).
|
||||
6. Persists its backfill progress **in the config file itself** so a restart
|
||||
resumes where it left off.
|
||||
|
||||
## Build
|
||||
|
||||
### Static binary (recommended, c-relay style)
|
||||
|
||||
Produces a truly portable statically-linked MUSL binary with zero runtime
|
||||
dependencies:
|
||||
|
||||
```bash
|
||||
./build_static.sh
|
||||
# or with debug symbols:
|
||||
./build_static.sh --debug
|
||||
# or cross-compile for arm64:
|
||||
./build_static.sh --arch arm64
|
||||
```
|
||||
|
||||
Output: `caching_relay` (in the project root).
|
||||
|
||||
Requires Docker. The build runs in an Alpine container that compiles
|
||||
libsecp256k1, libwebsockets, and nostr_core_lib from source, then statically
|
||||
links everything.
|
||||
|
||||
### Local build (if you have the shared libs installed)
|
||||
|
||||
```bash
|
||||
make
|
||||
```
|
||||
|
||||
Output: `caching_relay` (in the project root).
|
||||
|
||||
Requires: `libwebsockets`, `openssl`, `libsecp256k1`, `libcurl`, `zlib` shared
|
||||
libraries, and a pre-built `../nostr_core_lib/libnostr_core_x64.a`.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
./caching_relay -d 3
|
||||
```
|
||||
|
||||
The daemon looks for `caching_relay_config.jsonc` in the current directory by default.
|
||||
You can override with `-c`:
|
||||
|
||||
```bash
|
||||
./caching_relay -c /path/to/my_config.jsonc -d 3
|
||||
```
|
||||
|
||||
Options:
|
||||
- `-c, --config <file>` - Path to `.jsonc` config file (default: `./caching_relay_config.jsonc`)
|
||||
- `-d, --debug <level>` - Log level 1-4 (1=error, 2=warn, 3=info, 4=debug). Default 3.
|
||||
- `-h, --help` - Show help
|
||||
|
||||
Signals:
|
||||
- `SIGINT` / `SIGTERM` - Graceful shutdown (saves state, closes connections)
|
||||
- `SIGHUP` - Reload config (preserves backfill state)
|
||||
|
||||
## Config file
|
||||
|
||||
The config file [`caching_relay_config.jsonc`](caching_relay_config.jsonc) lives in the
|
||||
project root. It is JSONC (JSON with `//` and `/* */` comments). The daemon
|
||||
rewrites it as plain JSON when saving state (comments are not preserved on
|
||||
rewrite).
|
||||
|
||||
Key fields:
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `root_npubs` | npubs whose kind-3 follows list we crawl |
|
||||
| `upstream_relays` | bootstrap relays for initial discovery (kind-3, kind-10002). Outbox relays are discovered dynamically via NIP-65. |
|
||||
| `local_relay` | relay to publish cached events into (never queried) |
|
||||
| `kinds` | event kinds to cache for followed people (e.g. `[1, 3, 6, 10000, 30023]`) |
|
||||
| `admin_kinds` | kinds to follow specifically for root (admin) npubs. Use `["*"]` for all kinds. If omitted, admin uses same `kinds` as everyone else. |
|
||||
| `backfill.window_schedule_seconds` | progressive window sizes in seconds |
|
||||
| `backfill.events_per_tick` | max events per pubkey per backfill tick |
|
||||
| `backfill.tick_interval_seconds` | delay between pubkey backfills (throttle) |
|
||||
| `live.enabled` | enable live subscription |
|
||||
| `follow_graph_refresh_seconds` | how often to re-resolve the follow graph |
|
||||
| `state.*` | managed by the daemon - do not hand-edit |
|
||||
|
||||
## Architecture
|
||||
|
||||
See [`plans/plan.md`](plans/plan.md) for the full architecture document with
|
||||
Mermaid diagrams.
|
||||
|
||||
The daemon uses a **NIP-65 outbox model**: instead of querying a fixed set of
|
||||
bootstrap relays for everything, it discovers which relays each followed pubkey
|
||||
actually posts to (via kind 10002 relay lists) and connects to those relays
|
||||
dynamically. The `upstream_relays` in the config serve only as **bootstrap
|
||||
relays** -- the initial set used to discover kind-3 and kind-10002 events before
|
||||
the outbox relay map is built.
|
||||
|
||||
### Relay pool selection: minimum covering set
|
||||
|
||||
After discovering each followed pubkey's outbox relays (from their kind 10002),
|
||||
the daemon computes the **minimum set of relays that covers all followed
|
||||
pubkeys**. This is the classic set cover problem, solved with a greedy
|
||||
approximation:
|
||||
|
||||
1. Build a map: `{relay_url -> set of pubkeys that list it in their 10002}`
|
||||
2. Greedily pick the relay that covers the most uncovered pubkeys
|
||||
3. Repeat until all pubkeys are covered (or no more relays to pick)
|
||||
4. Always include the bootstrap relays in the final set (they may have events
|
||||
from pubkeys that don't publish a kind 10002)
|
||||
|
||||
This minimizes the number of WebSocket connections while ensuring every
|
||||
followed pubkey is reachable. The daemon logs the selected relay set and which
|
||||
pubkeys each relay covers.
|
||||
|
||||
Pubkeys that have no kind 10002 (or whose 10002 lists no relays) are covered by
|
||||
the bootstrap relays as a fallback.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ caching_relay │
|
||||
│ │
|
||||
upstream relays │ upstream_pool sink_pool │ local relay
|
||||
(damus, nos.lol) │ (query + subscribe) (publish only) │ (c-relay)
|
||||
│ │ │ │ │ │
|
||||
│ │ follow_graph relay_sink │ │
|
||||
└──────────►│ live_subscriber ──────►│ ├─────►│
|
||||
│ backfill ──────────────►│ │ │
|
||||
│ │ │ │ │
|
||||
│ state + seen ring │ │ │
|
||||
└──────────────────────────────────────────┘ │
|
||||
```
|
||||
|
||||
Two `nostr_relay_pool_t` instances:
|
||||
- **upstream_pool** - holds bootstrap relays initially, then dynamically adds
|
||||
outbox relays discovered from kind 10002. Used for `query_sync` (kind-3 fetch,
|
||||
kind-10002 fetch, backfill) and the long-lived live subscription.
|
||||
- **sink_pool** - holds only the local relay; used exclusively for
|
||||
`publish_async`. Never queried.
|
||||
|
||||
## Startup Flow
|
||||
|
||||
### First-time startup (empty local relay)
|
||||
|
||||
The daemon has never run before. The local relay has no cached events. The
|
||||
daemon must bootstrap from the config's `upstream_relays` to discover the
|
||||
outbox relays for each followed pubkey.
|
||||
|
||||
```
|
||||
START
|
||||
|
|
||||
v
|
||||
Load config (caching_relay_config.jsonc)
|
||||
| - root_npubs, upstream_relays (bootstrap), kinds, admin_kinds
|
||||
| - state.backfilled_until == 0 => first-time startup
|
||||
|
|
||||
v
|
||||
Create upstream_pool with bootstrap relays only
|
||||
Create sink_pool with local_relay
|
||||
|
|
||||
v
|
||||
Phase 1: Resolve follow graph (from bootstrap relays)
|
||||
| - For each root npub: query_sync kind=3 from bootstrap relays
|
||||
| - Parse "p" tags => followed pubkey set
|
||||
| - Log: "follow: resolved N followed pubkeys"
|
||||
|
|
||||
v
|
||||
Phase 2: Discover outbox relays (NIP-65, kind 10002) from bootstrap relays
|
||||
| - For each followed pubkey: query_sync kind=10002 from bootstrap relays
|
||||
| - Parse "r" tags => per-pubkey relay list
|
||||
| - Publish all kind-10002 events to local relay (cache them)
|
||||
| - Dynamically add discovered relays to upstream_pool
|
||||
| - Log: "relay_discovery: found N relays from M pubkeys"
|
||||
| - Log: "upstream_pool: now connected to N relays" + list them
|
||||
|
|
||||
v
|
||||
Phase 3: Open live subscriptions on upstream_pool
|
||||
| - follows_sub: non-admin pubkeys + regular kinds
|
||||
| - admin_sub: admin npubs + admin_kinds (or all kinds)
|
||||
|
|
||||
v
|
||||
Phase 4: Begin progressive backfill
|
||||
| - Window 0: 24h -> query each pubkey from their outbox relays
|
||||
| - Window 1: 7d -> ...
|
||||
| - Window 2: 30d -> ...
|
||||
| - Publish all fetched events to local relay
|
||||
| - Save state to config file as each window completes
|
||||
|
|
||||
v
|
||||
Steady-state: live sub + periodic follow-graph refresh + periodic
|
||||
kind-10002 refresh + backfill re-cycle
|
||||
|
|
||||
v
|
||||
SHUTDOWN (SIGINT/SIGTERM) -> save state -> clean exit
|
||||
```
|
||||
|
||||
### Subsequent startup (local relay already has cached events)
|
||||
|
||||
The daemon has run before. The local relay already has kind-3 and kind-10002
|
||||
events cached from the previous run. The daemon can read these from the local
|
||||
relay directly (fast, no network round-trip to bootstrap relays) and only falls
|
||||
back to bootstrap relays for pubkeys it cannot find locally.
|
||||
|
||||
```
|
||||
START
|
||||
|
|
||||
v
|
||||
Load config (caching_relay_config.jsonc)
|
||||
| - state.backfilled_until > 0 => subsequent startup
|
||||
| - state.current_window_index, backfill_cursor preserved
|
||||
|
|
||||
v
|
||||
Create upstream_pool with bootstrap relays
|
||||
Create sink_pool with local_relay
|
||||
|
|
||||
v
|
||||
Phase 1: Resolve follow graph (from LOCAL relay first, bootstrap fallback)
|
||||
| - Query local relay for kind=3 per root npub
|
||||
| - If found locally: use it (fast, no upstream query)
|
||||
| - If not found: fall back to bootstrap relays
|
||||
| - Parse "p" tags => followed pubkey set
|
||||
| - Log: "follow: resolved N followed pubkeys (L local, B bootstrap)"
|
||||
|
|
||||
v
|
||||
Phase 2: Discover outbox relays (from LOCAL relay first, bootstrap fallback)
|
||||
| - Query local relay for kind=10002 per followed pubkey
|
||||
| - If found locally: use it (fast)
|
||||
| - If not found: fall back to bootstrap relays, cache result to local
|
||||
| - Parse "r" tags => per-pubkey relay list
|
||||
| - Dynamically add discovered relays to upstream_pool
|
||||
| - Log: "relay_discovery: found N relays (L local, B bootstrap)"
|
||||
| - Log: "upstream_pool: now connected to N relays" + list them
|
||||
|
|
||||
v
|
||||
Phase 3: Open live subscriptions on upstream_pool
|
||||
| - (same as first-time)
|
||||
|
|
||||
v
|
||||
Phase 4: Resume backfill from saved state
|
||||
| - Resume at state.current_window_index, state.backfill_cursor
|
||||
| - No re-pull of already-backfilled windows
|
||||
| - Continue progressive window expansion from where it left off
|
||||
|
|
||||
v
|
||||
Steady-state (same as first-time)
|
||||
|
|
||||
v
|
||||
SHUTDOWN -> save state -> clean exit
|
||||
```
|
||||
|
||||
### Relay logging
|
||||
|
||||
The daemon logs all relay activity so you can see exactly which relays are
|
||||
being used:
|
||||
|
||||
- `upstream: added wss://relay.damus.io` -- each relay added to the pool
|
||||
- `relay_discovery: found 47 relays from 195 pubkeys` -- outbox discovery summary
|
||||
- `upstream_pool: 50 relays connected:` -- full relay list at startup
|
||||
- `relay_discovery: pubkey X -> wss://relay.example.com (local)` -- per-pubkey
|
||||
relay source (local cache vs bootstrap)
|
||||
- `backfill: pubkey[3/195] abc123 -> wss://relay.example.com (12 events)` --
|
||||
which relay served each backfill query
|
||||
|
||||
## Dependencies
|
||||
|
||||
- [nostr_core_lib](../nostr_core_lib) - built with NIPs 1, 4, 6, 19, 42, 44
|
||||
- libwebsockets, openssl, libsecp256k1, libcurl, zlib (all statically linked
|
||||
in the Docker build)
|
||||
|
||||
## Testing
|
||||
|
||||
Start a local c-relay, then run the daemon:
|
||||
|
||||
```bash
|
||||
# Start local relay (from c-relay project)
|
||||
cd ../c-relay/build && ./c_relay_static_x86_64 -p 8888 &
|
||||
|
||||
# Run the caching daemon (uses ./caching_relay_config.jsonc by default)
|
||||
./caching_relay -d 3
|
||||
|
||||
# Verify events are cached (from another terminal)
|
||||
nak req -k 1 -l 10 ws://127.0.0.1:8888
|
||||
```
|
||||
@@ -0,0 +1 @@
|
||||
0.0.2
|
||||
Executable
+147
@@ -0,0 +1,147 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Build fully static MUSL binary for caching_relay using Alpine Docker.
|
||||
# Produces a truly portable binary with zero runtime dependencies.
|
||||
# Adapted from c-relay's build_static.sh.
|
||||
|
||||
set -e
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
DOCKERFILE="$SCRIPT_DIR/Dockerfile.alpine-musl"
|
||||
|
||||
# Parse command line arguments
|
||||
DEBUG_BUILD=false
|
||||
TARGET_ARCH=""
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--debug)
|
||||
DEBUG_BUILD=true
|
||||
shift
|
||||
;;
|
||||
--arch)
|
||||
if [[ -z "$2" ]]; then
|
||||
echo "ERROR: --arch requires a value"
|
||||
echo "Usage: $0 [--debug] [--arch <arm64|x86_64>]"
|
||||
exit 1
|
||||
fi
|
||||
case "$2" in
|
||||
arm64|x86_64)
|
||||
TARGET_ARCH="$2"
|
||||
;;
|
||||
*)
|
||||
echo "ERROR: Unsupported architecture '$2'"
|
||||
echo "Supported values: arm64, x86_64"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
shift 2
|
||||
;;
|
||||
*)
|
||||
echo "ERROR: Unknown argument '$1'"
|
||||
echo "Usage: $0 [--debug] [--arch <arm64|x86_64>]"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ "$DEBUG_BUILD" = true ]; then
|
||||
echo "=========================================="
|
||||
echo "caching_relay MUSL Static Binary Builder (DEBUG MODE)"
|
||||
echo "=========================================="
|
||||
else
|
||||
echo "=========================================="
|
||||
echo "caching_relay MUSL Static Binary Builder (PRODUCTION MODE)"
|
||||
echo "=========================================="
|
||||
fi
|
||||
echo "Project directory: $SCRIPT_DIR"
|
||||
echo ""
|
||||
|
||||
if ! command -v docker &> /dev/null; then
|
||||
echo "ERROR: Docker is not installed or not in PATH"
|
||||
exit 1
|
||||
fi
|
||||
if ! docker info &> /dev/null; then
|
||||
echo "ERROR: Docker daemon is not running or user not in docker group"
|
||||
exit 1
|
||||
fi
|
||||
echo "Docker is available and running"
|
||||
echo ""
|
||||
|
||||
# Detect host architecture
|
||||
ARCH=$(uname -m)
|
||||
case "$ARCH" in
|
||||
x86_64) ARCH="x86_64";;
|
||||
aarch64) ARCH="arm64";;
|
||||
arm64) ARCH="arm64";;
|
||||
esac
|
||||
if [[ -n "$TARGET_ARCH" ]]; then
|
||||
ARCH="$TARGET_ARCH"
|
||||
echo "Using target architecture: $ARCH"
|
||||
else
|
||||
echo "Detected host architecture: $ARCH"
|
||||
fi
|
||||
echo ""
|
||||
|
||||
# nostr_core_lib is inside the c-relay-pg project root (one dir up from caching/).
|
||||
# Copy it into the Docker build context (excluding .git and build artifacts).
|
||||
NOSTR_CORE_DIR="$SCRIPT_DIR/../nostr_core_lib"
|
||||
if [ ! -d "$NOSTR_CORE_DIR" ]; then
|
||||
echo "ERROR: nostr_core_lib not found at $NOSTR_CORE_DIR"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
NEEDS_COPY=1
|
||||
if [ -d "$SCRIPT_DIR/nostr_core_lib" ] && [ ! -L "$SCRIPT_DIR/nostr_core_lib" ]; then
|
||||
# Already a real directory (e.g. from a previous run); assume it's good.
|
||||
NEEDS_COPY=0
|
||||
fi
|
||||
if [ "$NEEDS_COPY" = "1" ]; then
|
||||
echo "Copying nostr_core_lib into build context..."
|
||||
rm -rf "$SCRIPT_DIR/nostr_core_lib"
|
||||
mkdir -p "$SCRIPT_DIR/nostr_core_lib"
|
||||
rsync -a --exclude='.git' --exclude='*.a' --exclude='*.o' \
|
||||
--exclude='.venv*' --exclude='backups' --exclude='verify_*' \
|
||||
--exclude='rewrite_mirror' --exclude='websocket_debug' \
|
||||
"$NOSTR_CORE_DIR/" "$SCRIPT_DIR/nostr_core_lib/"
|
||||
COPIED_NOSTR_CORE=1
|
||||
fi
|
||||
|
||||
# Build args
|
||||
BUILD_ARGS="--build-arg DEBUG_BUILD=$DEBUG_BUILD"
|
||||
if [ "$ARCH" = "arm64" ] && [ "$(uname -m)" != "aarch64" ] && [ "$(uname -m)" != "arm64" ]; then
|
||||
BUILD_ARGS="$BUILD_ARGS --platform linux/arm64"
|
||||
fi
|
||||
|
||||
IMAGE_TAG="caching_relay_builder:latest"
|
||||
|
||||
echo "Building Docker image..."
|
||||
docker build $BUILD_ARGS \
|
||||
-f "$DOCKERFILE" \
|
||||
-t "$IMAGE_TAG" \
|
||||
"$SCRIPT_DIR" 2>&1
|
||||
|
||||
BUILD_RC=$?
|
||||
|
||||
# Clean up the copied nostr_core_lib to keep the project dir tidy.
|
||||
if [ "${COPIED_NOSTR_CORE:-0}" = "1" ]; then
|
||||
echo "Cleaning up copied nostr_core_lib from build context..."
|
||||
rm -rf "$SCRIPT_DIR/nostr_core_lib"
|
||||
fi
|
||||
|
||||
if [ $BUILD_RC -ne 0 ]; then
|
||||
exit $BUILD_RC
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Extracting binary from image..."
|
||||
# The output stage is FROM scratch with no CMD, so pass an empty command.
|
||||
CONTAINER_ID=$(docker create "$IMAGE_TAG" "")
|
||||
docker cp "$CONTAINER_ID:/caching_relay_static" "$SCRIPT_DIR/caching_relay"
|
||||
docker rm "$CONTAINER_ID" >/dev/null
|
||||
chmod +x "$SCRIPT_DIR/caching_relay"
|
||||
|
||||
echo ""
|
||||
echo "=== Build complete ==="
|
||||
ls -lh "$SCRIPT_DIR/caching_relay"
|
||||
file "$SCRIPT_DIR/caching_relay"
|
||||
@@ -0,0 +1,431 @@
|
||||
/*
|
||||
* caching_relay - relay-by-relay per-author until-cursor drain backfill.
|
||||
*
|
||||
* Each (author, relay) pair has its own persistent until_cursor stored in
|
||||
* the caching_backfill_relay_progress table. Each backfill tick picks the
|
||||
* next author that still has at least one incomplete relay (round-robin),
|
||||
* then queries each incomplete relay one at a time with
|
||||
* since=0 & until=cursor & limit=page_size
|
||||
* publishes the page, and advances that relay's cursor to
|
||||
* oldest_event_created_at - 1. A relay is marked complete when a page
|
||||
* returns fewer than page_size events (the relay is drained for this
|
||||
* author). An author is marked complete in caching_followed_pubkeys when
|
||||
* all their relay progress rows are complete.
|
||||
*/
|
||||
#define _GNU_SOURCE
|
||||
#include "backfill.h"
|
||||
#include "follow_graph.h"
|
||||
#include "pg_inbox.h"
|
||||
#include "debug.h"
|
||||
|
||||
#include "../nostr_core_lib/cjson/cJSON.h"
|
||||
|
||||
#include <string.h>
|
||||
#include <stdlib.h>
|
||||
#include <time.h>
|
||||
|
||||
/* ------------------------------------------------------------------ */
|
||||
/* Global access for status reporting */
|
||||
/* ------------------------------------------------------------------ */
|
||||
|
||||
/* Singleton pointer set in cr_backfill_init, used by the accessor
|
||||
* functions below so the main loop / status reporter can see what
|
||||
* the backfill is currently working on. */
|
||||
static cr_backfill_t *g_bf = NULL;
|
||||
|
||||
const char *cr_backfill_active_pubkey(void) {
|
||||
return g_bf ? g_bf->active_pubkey : "";
|
||||
}
|
||||
const char *cr_backfill_active_relay(void) {
|
||||
return g_bf ? g_bf->active_relay : "";
|
||||
}
|
||||
int cr_backfill_active_got_eose(void) {
|
||||
return g_bf ? g_bf->active_relay_got_eose : 0;
|
||||
}
|
||||
|
||||
/* ------------------------------------------------------------------ */
|
||||
/* EOSE callback for synchronous_query_relays_with_progress */
|
||||
/* ------------------------------------------------------------------ */
|
||||
|
||||
/* Context passed to the query callback. Records whether the relay sent
|
||||
* EOSE (we got everything) or timed out (partial result). */
|
||||
typedef struct {
|
||||
int got_eose; /* 1 if EOSE received, 0 if timed out/errored */
|
||||
int events_count; /* events reported by callback */
|
||||
char last_status[256]; /* last status string from callback (may include message) */
|
||||
} backfill_query_ctx_t;
|
||||
|
||||
static void backfill_query_callback(const char *relay_url, const char *status,
|
||||
const char *event_id, int events_received,
|
||||
int total_relays, int completed_relays,
|
||||
void *user_data) {
|
||||
(void)relay_url; (void)total_relays; (void)completed_relays;
|
||||
backfill_query_ctx_t *ctx = (backfill_query_ctx_t *)user_data;
|
||||
if (!ctx || !status) return;
|
||||
|
||||
/* Ignore the synthetic "all_complete" status from core_relays.c — it
|
||||
* is sent AFTER the real EOSE/timeout and would overwrite the real
|
||||
* status with a misleading "all_complete" string, and clobber
|
||||
* events_count with total_unique_events. The real per-relay status
|
||||
* (eose/timeout/error) is what we need for completion decisions. */
|
||||
if (strcmp(status, "all_complete") == 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
/* For NOTICE/CLOSED/error, event_id carries the descriptive message
|
||||
* content (relay's own text for NOTICE/CLOSED, or our transport-level
|
||||
* diagnosis for error). Include it in the status for display.
|
||||
* Priority: relay's own response (NOTICE/CLOSED) > our error > bare
|
||||
* status string. */
|
||||
if ((strcmp(status, "NOTICE") == 0 || strcmp(status, "CLOSED") == 0) &&
|
||||
event_id && event_id[0] != '\0') {
|
||||
snprintf(ctx->last_status, sizeof(ctx->last_status), "%s: %s", status, event_id);
|
||||
} else if (strcmp(status, "error") == 0) {
|
||||
if (event_id && event_id[0] != '\0') {
|
||||
snprintf(ctx->last_status, sizeof(ctx->last_status), "error: %s", event_id);
|
||||
} else {
|
||||
snprintf(ctx->last_status, sizeof(ctx->last_status), "error");
|
||||
}
|
||||
} else {
|
||||
snprintf(ctx->last_status, sizeof(ctx->last_status), "%s", status);
|
||||
}
|
||||
|
||||
if (strcmp(status, "eose") == 0) {
|
||||
ctx->got_eose = 1;
|
||||
} else if (strcmp(status, "timeout") == 0) {
|
||||
ctx->got_eose = 0;
|
||||
} else if (strcmp(status, "error") == 0) {
|
||||
ctx->got_eose = 0;
|
||||
}
|
||||
/* NOTICE/CLOSED don't change got_eose — the relay might still send
|
||||
* events or EOSE after a NOTICE. */
|
||||
ctx->events_count = events_received;
|
||||
}
|
||||
|
||||
/* ------------------------------------------------------------------ */
|
||||
/* Helpers */
|
||||
/* ------------------------------------------------------------------ */
|
||||
|
||||
/* Returns 1 if the PostgreSQL inbox is initialized (PG mode active).
|
||||
* Probes the config table; a non-NULL result means the connection is up. */
|
||||
static int pg_mode_active(void) {
|
||||
char *v = pg_inbox_get_config_value("caching_root_npubs");
|
||||
if (v) { free(v); return 1; }
|
||||
char *v2 = pg_inbox_get_config_value("caching_kinds");
|
||||
if (v2) { free(v2); return 1; }
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* Find the oldest (minimum) created_at among an array of event JSON objects.
|
||||
* Returns 1 and sets *out_oldest, or 0 if no events / no valid created_at. */
|
||||
static int find_oldest_created_at(cJSON **events, int count, long *out_oldest) {
|
||||
long oldest = 0;
|
||||
int found = 0;
|
||||
for (int i = 0; i < count; i++) {
|
||||
cJSON *ca = cJSON_GetObjectItem(events[i], "created_at");
|
||||
if (!ca || !cJSON_IsNumber(ca)) continue;
|
||||
long ts = (long)ca->valuedouble;
|
||||
if (!found || ts < oldest) {
|
||||
oldest = ts;
|
||||
found = 1;
|
||||
}
|
||||
}
|
||||
if (found && out_oldest) *out_oldest = oldest;
|
||||
return found;
|
||||
}
|
||||
|
||||
/* Build the kinds array for a given pubkey (admin vs regular).
|
||||
* Returns NULL if no kinds filter should be applied (admin_all_kinds). */
|
||||
static cJSON *build_kinds(cr_config_t *cfg, const char *pk) {
|
||||
int is_admin = cr_follow_is_root(cfg, pk);
|
||||
if (is_admin && cfg->admin_all_kinds) {
|
||||
return NULL;
|
||||
}
|
||||
cJSON *kinds = cJSON_CreateArray();
|
||||
if (is_admin && cfg->admin_kind_count > 0) {
|
||||
for (int i = 0; i < cfg->admin_kind_count; i++)
|
||||
cJSON_AddItemToArray(kinds, cJSON_CreateNumber(cfg->admin_kinds[i]));
|
||||
} else {
|
||||
for (int i = 0; i < cfg->kind_count; i++)
|
||||
cJSON_AddItemToArray(kinds, cJSON_CreateNumber(cfg->kinds[i]));
|
||||
}
|
||||
return kinds;
|
||||
}
|
||||
|
||||
/* Resolve the relay URL list to query for a given pubkey. Sets *out_urls and
|
||||
* *out_n. Caller frees *out_urls (but not the strings, which alias internal
|
||||
* storage). Returns 0 on success, -1 if no relays available. */
|
||||
static int resolve_relays(nostr_relay_pool_t *upstream,
|
||||
const cr_relay_map_t *relay_map,
|
||||
const char *pk,
|
||||
const char ***out_urls,
|
||||
int *out_n) {
|
||||
*out_urls = NULL;
|
||||
*out_n = 0;
|
||||
|
||||
/* Collect outbox relays for this author (from kind-10002). */
|
||||
int outbox_count = 0;
|
||||
const cr_outbox_entry_t *oe = NULL;
|
||||
if (relay_map) {
|
||||
oe = cr_relay_map_get_outbox(relay_map, pk);
|
||||
if (oe) outbox_count = oe->relay_count;
|
||||
}
|
||||
|
||||
/* Also collect all upstream (bootstrap) relays. We query BOTH the
|
||||
* author's outbox relays AND the bootstrap relays to maximize coverage.
|
||||
* Different relays may have different subsets of the author's events. */
|
||||
char **listed = NULL;
|
||||
nostr_pool_relay_status_t *statuses = NULL;
|
||||
int upstream_count = nostr_relay_pool_list_relays(upstream, &listed, &statuses);
|
||||
|
||||
/* Merge outbox + upstream, deduplicating by URL. */
|
||||
int max_total = outbox_count + upstream_count;
|
||||
if (max_total <= 0) {
|
||||
free(listed);
|
||||
free(statuses);
|
||||
return 0;
|
||||
}
|
||||
|
||||
const char **urls = malloc(max_total * sizeof(char *));
|
||||
if (!urls) { free(listed); free(statuses); return -1; }
|
||||
|
||||
int n = 0;
|
||||
|
||||
/* Add outbox relays first. */
|
||||
if (oe) {
|
||||
for (int j = 0; j < oe->relay_count && n < max_total; j++) {
|
||||
urls[n++] = oe->relays[j];
|
||||
}
|
||||
}
|
||||
|
||||
/* Add upstream relays, skipping duplicates. */
|
||||
for (int j = 0; j < upstream_count && n < max_total; j++) {
|
||||
int dup = 0;
|
||||
for (int k = 0; k < n; k++) {
|
||||
if (strcmp(urls[k], listed[j]) == 0) { dup = 1; break; }
|
||||
}
|
||||
if (!dup) {
|
||||
urls[n++] = listed[j];
|
||||
}
|
||||
}
|
||||
|
||||
free(listed);
|
||||
free(statuses);
|
||||
|
||||
*out_urls = urls;
|
||||
*out_n = n;
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* ------------------------------------------------------------------ */
|
||||
/* Init */
|
||||
/* ------------------------------------------------------------------ */
|
||||
|
||||
void cr_backfill_init(cr_backfill_t *bf, cr_config_t *cfg) {
|
||||
(void)cfg;
|
||||
memset(bf, 0, sizeof(*bf));
|
||||
g_bf = bf; /* Set singleton for status accessors */
|
||||
|
||||
/* In PG mode, check whether any incomplete authors remain in the DB.
|
||||
* If so, backfill is in progress; otherwise we're in steady state.
|
||||
* Outside PG mode there is no per-author table, so we just start. */
|
||||
if (pg_mode_active()) {
|
||||
int complete = 0, total = 0;
|
||||
if (pg_inbox_count_backfill_progress(&complete, &total) == 0) {
|
||||
bf->in_progress = (total > 0 && complete < total) ? 1 : 0;
|
||||
DEBUG_INFO("backfill: init complete=%d/%d -> in_progress=%d",
|
||||
complete, total, bf->in_progress);
|
||||
return;
|
||||
}
|
||||
/* Count query failed - assume in progress so we keep trying. */
|
||||
bf->in_progress = 1;
|
||||
DEBUG_WARN("backfill: init could not read progress counts, "
|
||||
"defaulting to in_progress=1");
|
||||
return;
|
||||
}
|
||||
|
||||
/* Legacy (non-PG) mode: no per-author table; just start. */
|
||||
bf->in_progress = 1;
|
||||
DEBUG_INFO("backfill: init (legacy mode) in_progress=1");
|
||||
}
|
||||
|
||||
/* ------------------------------------------------------------------ */
|
||||
/* Tick */
|
||||
/* ------------------------------------------------------------------ */
|
||||
|
||||
int cr_backfill_tick(cr_backfill_t *bf, cr_config_t *cfg,
|
||||
nostr_relay_pool_t *upstream,
|
||||
cr_pubkey_set_t *followed,
|
||||
cr_sink_t *sink, const cr_relay_map_t *relay_map) {
|
||||
(void)followed; /* followed set is tracked in the DB now */
|
||||
(void)relay_map; /* relays are tracked per-author in the DB */
|
||||
|
||||
if (!cfg->backfill.enabled) return -2;
|
||||
if (!bf->in_progress) return -2;
|
||||
|
||||
/* Throttle: one query per tick_interval. */
|
||||
time_t now = time(NULL);
|
||||
if (bf->last_tick && (now - bf->last_tick) < cfg->backfill.tick_interval_seconds)
|
||||
return 0;
|
||||
|
||||
/* Pick the next author that has at least one incomplete relay progress
|
||||
* row (round-robin). */
|
||||
char pk[CR_HEX_LEN];
|
||||
if (pg_inbox_pick_next_author_with_incomplete_relays(pk, sizeof(pk),
|
||||
&bf->author_round_cursor) != 0) {
|
||||
/* No incomplete authors - steady state. */
|
||||
bf->in_progress = 0;
|
||||
DEBUG_INFO("backfill: no authors with incomplete relays, steady-state");
|
||||
return -2;
|
||||
}
|
||||
|
||||
int page_size = cfg->backfill.events_per_tick;
|
||||
if (page_size < 1) page_size = 500;
|
||||
|
||||
bf->last_tick = now;
|
||||
|
||||
/* Get the incomplete relays for this author. */
|
||||
cJSON *relays = pg_inbox_get_incomplete_relays(pk);
|
||||
if (!relays) {
|
||||
DEBUG_WARN("backfill: no incomplete relays returned for %s", pk);
|
||||
/* Author has no relay rows - mark complete so we don't loop. */
|
||||
pg_inbox_check_and_mark_author_complete(pk);
|
||||
return 1;
|
||||
}
|
||||
|
||||
int relay_count = cJSON_GetArraySize(relays);
|
||||
DEBUG_TRACE("backfill: author %s has %d incomplete relay(s)", pk, relay_count);
|
||||
|
||||
cr_sink_set_source_class(sink, CR_SINK_CLASS_BACKFILL);
|
||||
|
||||
/* Query each incomplete relay one at a time. */
|
||||
for (int r = 0; r < relay_count; r++) {
|
||||
cJSON *entry = cJSON_GetArrayItem(relays, r);
|
||||
if (!entry) continue;
|
||||
cJSON *url_node = cJSON_GetObjectItem(entry, "relay_url");
|
||||
cJSON *cur_node = cJSON_GetObjectItem(entry, "until_cursor");
|
||||
if (!url_node || !cJSON_IsString(url_node)) continue;
|
||||
const char *relay_url = cJSON_GetStringValue(url_node);
|
||||
long until_cursor = (cur_node && cJSON_IsNumber(cur_node))
|
||||
? (long)cur_node->valuedouble : 0;
|
||||
if (until_cursor == 0) until_cursor = (long)now;
|
||||
|
||||
/* Build the filter for this single relay. */
|
||||
cJSON *filter = cJSON_CreateObject();
|
||||
cJSON *authors = cJSON_CreateArray();
|
||||
cJSON_AddItemToArray(authors, cJSON_CreateString(pk));
|
||||
cJSON_AddItemToObject(filter, "authors", authors);
|
||||
cJSON *kinds = build_kinds(cfg, pk);
|
||||
if (kinds) cJSON_AddItemToObject(filter, "kinds", kinds);
|
||||
cJSON_AddItemToObject(filter, "since", cJSON_CreateNumber(0.0));
|
||||
cJSON_AddItemToObject(filter, "until", cJSON_CreateNumber((double)until_cursor));
|
||||
cJSON_AddItemToObject(filter, "limit", cJSON_CreateNumber((double)page_size));
|
||||
|
||||
DEBUG_TRACE("backfill: %s @ %s (until=%ld, limit=%d)",
|
||||
pk, relay_url, until_cursor, page_size);
|
||||
|
||||
/* Record the active target for UI status display (both
|
||||
* in-memory for local accessors and in PG for the relay). */
|
||||
strncpy(bf->active_pubkey, pk, sizeof(bf->active_pubkey) - 1);
|
||||
bf->active_pubkey[sizeof(bf->active_pubkey) - 1] = '\0';
|
||||
strncpy(bf->active_relay, relay_url, sizeof(bf->active_relay) - 1);
|
||||
bf->active_relay[sizeof(bf->active_relay) - 1] = '\0';
|
||||
bf->active_relay_got_eose = 0;
|
||||
pg_inbox_set_active_target(pk, relay_url);
|
||||
|
||||
/* Query this single relay using synchronous_query_relays_with_progress
|
||||
* which gives us an EOSE/timeout callback — the relay's own signal
|
||||
* of whether it sent everything or we timed out. */
|
||||
backfill_query_ctx_t qctx = {0, 0};
|
||||
const char *one_url = relay_url;
|
||||
int ev_count = 0;
|
||||
cJSON **events = synchronous_query_relays_with_progress(
|
||||
&one_url, 1, filter, RELAY_QUERY_ALL_RESULTS,
|
||||
&ev_count, 30, /* timeout in seconds */
|
||||
backfill_query_callback, &qctx,
|
||||
0, NULL /* nip42 disabled, no private key */);
|
||||
cJSON_Delete(filter);
|
||||
|
||||
bf->active_relay_got_eose = qctx.got_eose;
|
||||
|
||||
if (!events || ev_count == 0) {
|
||||
/* No events from this relay - it is drained for this author.
|
||||
* (ev_count == 0 with EOSE means truly empty; without EOSE it
|
||||
* means a connection failure — keep incomplete to retry.
|
||||
* The pg_inbox_update_relay_progress SQL will auto-mark
|
||||
* complete once consecutive_errors reaches 3.) */
|
||||
int mark_complete = qctx.got_eose ? 1 : 0;
|
||||
pg_inbox_update_relay_progress(pk, relay_url, until_cursor,
|
||||
mark_complete, 0, qctx.last_status);
|
||||
free(events);
|
||||
if (!qctx.got_eose) {
|
||||
/* Error or timeout with no events — surface the
|
||||
* descriptive status at WARN so operators can see why. */
|
||||
DEBUG_WARN("backfill: %s @ %s failed: %s (will auto-complete after 3 consecutive errors)",
|
||||
pk, relay_url, qctx.last_status);
|
||||
} else {
|
||||
DEBUG_LOG("backfill: %s @ %s complete (no events, eose)",
|
||||
pk, relay_url);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
/* Find oldest timestamp, then publish every event. */
|
||||
long oldest_ts = 0;
|
||||
find_oldest_created_at(events, ev_count, &oldest_ts);
|
||||
|
||||
for (int k = 0; k < ev_count; k++) {
|
||||
cr_sink_publish(sink, events[k]);
|
||||
cJSON_Delete(events[k]);
|
||||
}
|
||||
free(events);
|
||||
bf->events_total += ev_count;
|
||||
|
||||
DEBUG_LOG("backfill: %s @ %s -> %d events (until=%ld, oldest=%ld, eose=%d)",
|
||||
pk, relay_url, ev_count, until_cursor, oldest_ts, qctx.got_eose);
|
||||
|
||||
/* Completion decision based on EOSE status:
|
||||
* - ev_count >= page_size: saturated, advance cursor, keep incomplete
|
||||
* - ev_count < page_size AND got EOSE: relay is truly drained
|
||||
* - ev_count < page_size AND no EOSE (timeout): partial result,
|
||||
* advance cursor but keep incomplete to retry next tick
|
||||
* - Guard: if cursor didn't advance (oldest_ts >= until_cursor),
|
||||
* mark complete to avoid infinite loop on identical timestamps. */
|
||||
long next_cursor = (oldest_ts > 0) ? oldest_ts - 1 : 0;
|
||||
int cursor_advanced = (oldest_ts > 0 && oldest_ts < until_cursor);
|
||||
|
||||
if (ev_count >= page_size) {
|
||||
pg_inbox_update_relay_progress(pk, relay_url, next_cursor, 0, ev_count, qctx.last_status);
|
||||
} else if (qctx.got_eose) {
|
||||
/* Relay sent EOSE with < page_size events — truly drained. */
|
||||
pg_inbox_update_relay_progress(pk, relay_url, next_cursor, 1, ev_count, qctx.last_status);
|
||||
} else {
|
||||
/* Timed out before EOSE — partial result. Keep incomplete
|
||||
* to retry, unless the cursor didn't advance (stall guard).
|
||||
* Timeouts DO count toward the consecutive_errors limit
|
||||
* (a relay that consistently times out is effectively dead
|
||||
* for backfill purposes), so the SQL will auto-mark
|
||||
* complete after 3 consecutive timeouts/errors. */
|
||||
int mark_complete = cursor_advanced ? 0 : 1;
|
||||
pg_inbox_update_relay_progress(pk, relay_url, next_cursor,
|
||||
mark_complete, ev_count, qctx.last_status);
|
||||
DEBUG_WARN("backfill: %s @ %s partial: %s (%d events, will auto-complete after 3 consecutive timeouts/errors)",
|
||||
pk, relay_url, qctx.last_status, ev_count);
|
||||
}
|
||||
}
|
||||
|
||||
/* Clear active target — done with this author. */
|
||||
bf->active_relay[0] = '\0';
|
||||
pg_inbox_set_active_target(pk, "");
|
||||
|
||||
cJSON_Delete(relays);
|
||||
|
||||
/* After querying all incomplete relays for this author, check whether
|
||||
* the author is now fully complete. */
|
||||
int crc = pg_inbox_check_and_mark_author_complete(pk);
|
||||
if (crc == 1) {
|
||||
DEBUG_LOG("backfill: author %s fully complete (all relays drained)", pk);
|
||||
}
|
||||
|
||||
return 1;
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
/*
|
||||
* caching_relay - per-author until-cursor drain backfill.
|
||||
*
|
||||
* Each followed author has a single persistent cursor (starting at `now`)
|
||||
* stored in the caching_followed_pubkeys table. Each backfill tick picks
|
||||
* the next incomplete author (round-robin), queries
|
||||
* since=0 & until=cursor & limit=page_size
|
||||
* publishes the page, and sets cursor = oldest_event_created_at - 1.
|
||||
* Repeat until a page returns fewer than page_size events (beginning of
|
||||
* the author's history reached), then mark the author complete.
|
||||
*/
|
||||
#ifndef CACHING_RELAY_BACKFILL_H
|
||||
#define CACHING_RELAY_BACKFILL_H
|
||||
|
||||
#include "config.h"
|
||||
#include "state.h"
|
||||
#include "relay_sink.h"
|
||||
#include "relay_discovery.h"
|
||||
#include "../nostr_core_lib/nostr_core/nostr_core.h"
|
||||
|
||||
typedef struct {
|
||||
int in_progress; /* 1 if any incomplete authors remain */
|
||||
long events_total; /* events pulled across all ticks (diagnostic) */
|
||||
time_t last_tick; /* last time an author was queried (throttle) */
|
||||
int author_round_cursor;/* round-robin cursor for author selection */
|
||||
/* Current backfill target (for UI status display). */
|
||||
char active_pubkey[65]; /* pubkey currently being backfilled (or "") */
|
||||
char active_relay[256]; /* relay URL currently being queried (or "") */
|
||||
int active_relay_got_eose; /* 1 if last query got EOSE, 0 if timed out */
|
||||
} cr_backfill_t;
|
||||
|
||||
/* Global access to the current backfill target for status reporting.
|
||||
* Returns pointers to static storage in the singleton cr_backfill_t.
|
||||
* Safe to call from the main loop; not thread-safe but backfill is
|
||||
* single-threaded. */
|
||||
const char *cr_backfill_active_pubkey(void);
|
||||
const char *cr_backfill_active_relay(void);
|
||||
int cr_backfill_active_got_eose(void);
|
||||
|
||||
/* Initialize backfill state. Checks the caching_followed_pubkeys table for
|
||||
* any incomplete authors; sets in_progress accordingly. */
|
||||
void cr_backfill_init(cr_backfill_t *bf, cr_config_t *cfg);
|
||||
|
||||
/* Perform one backfill tick. Picks the next incomplete author from the DB
|
||||
* (round-robin), queries one page of history, publishes it to the sink, and
|
||||
* updates the per-author cursor in the DB.
|
||||
*
|
||||
* Returns:
|
||||
* 1 if a tick was performed (an author was queried)
|
||||
* 0 if throttled (tick_interval not elapsed) - caller should pump pools
|
||||
* -2 if backfill is complete (no incomplete authors) - caller goes steady-state
|
||||
*/
|
||||
int cr_backfill_tick(cr_backfill_t *bf, cr_config_t *cfg,
|
||||
nostr_relay_pool_t *upstream,
|
||||
cr_pubkey_set_t *followed,
|
||||
cr_sink_t *sink, const cr_relay_map_t *relay_map);
|
||||
|
||||
#endif /* CACHING_RELAY_BACKFILL_H */
|
||||
@@ -0,0 +1,290 @@
|
||||
/*
|
||||
* caching_relay - config parsing + persistent state (in the .jsonc file itself)
|
||||
*/
|
||||
#define _GNU_SOURCE
|
||||
#include "config.h"
|
||||
#include "jsonc_strip.h"
|
||||
#include "debug.h"
|
||||
|
||||
#include "../nostr_core_lib/cjson/cJSON.h"
|
||||
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <errno.h>
|
||||
#include <unistd.h>
|
||||
#include <fcntl.h>
|
||||
|
||||
static int read_file(const char *path, char **out, size_t *out_len) {
|
||||
FILE *f = fopen(path, "rb");
|
||||
if (!f) {
|
||||
DEBUG_ERROR("cannot open config '%s': %s", path, strerror(errno));
|
||||
return -1;
|
||||
}
|
||||
if (fseek(f, 0, SEEK_END) != 0) { fclose(f); return -1; }
|
||||
long sz = ftell(f);
|
||||
if (sz < 0) { fclose(f); return -1; }
|
||||
rewind(f);
|
||||
char *buf = malloc(sz + 1);
|
||||
if (!buf) { fclose(f); return -1; }
|
||||
size_t n = fread(buf, 1, sz, f);
|
||||
fclose(f);
|
||||
buf[n] = '\0';
|
||||
*out = buf;
|
||||
if (out_len) *out_len = n;
|
||||
return 0;
|
||||
}
|
||||
|
||||
static void copy_string_array(cJSON *arr, char *dst, int max, int *count_out,
|
||||
int elem_len) {
|
||||
int count = 0;
|
||||
if (arr && cJSON_IsArray(arr)) {
|
||||
cJSON *item;
|
||||
cJSON_ArrayForEach(item, arr) {
|
||||
if (count >= max) break;
|
||||
if (!cJSON_IsString(item)) continue;
|
||||
const char *s = cJSON_GetStringValue(item);
|
||||
if (!s) continue;
|
||||
strncpy(dst + count * elem_len, s, elem_len - 1);
|
||||
dst[count * elem_len + (elem_len - 1)] = '\0';
|
||||
count++;
|
||||
}
|
||||
}
|
||||
*count_out = count;
|
||||
}
|
||||
|
||||
static void copy_int_array(cJSON *arr, int *dst, int max, int *count_out) {
|
||||
int count = 0;
|
||||
if (arr && cJSON_IsArray(arr)) {
|
||||
cJSON *item;
|
||||
cJSON_ArrayForEach(item, arr) {
|
||||
if (count >= max) break;
|
||||
if (cJSON_IsNumber(item)) {
|
||||
dst[count++] = (int)cJSON_GetNumberValue(item);
|
||||
}
|
||||
}
|
||||
}
|
||||
*count_out = count;
|
||||
}
|
||||
|
||||
static void copy_long_array(cJSON *arr, long *dst, int max, int *count_out) {
|
||||
int count = 0;
|
||||
if (arr && cJSON_IsArray(arr)) {
|
||||
cJSON *item;
|
||||
cJSON_ArrayForEach(item, arr) {
|
||||
if (count >= max) break;
|
||||
if (cJSON_IsNumber(item)) {
|
||||
dst[count++] = (long)cJSON_GetNumberValue(item);
|
||||
}
|
||||
}
|
||||
}
|
||||
*count_out = count;
|
||||
}
|
||||
|
||||
int cr_config_load(cr_config_t *cfg, const char *path) {
|
||||
memset(cfg, 0, sizeof(*cfg));
|
||||
strncpy(cfg->path, path, sizeof(cfg->path) - 1);
|
||||
|
||||
char *raw = NULL;
|
||||
size_t raw_len = 0;
|
||||
if (read_file(path, &raw, &raw_len) != 0) return -1;
|
||||
|
||||
char *stripped = jsonc_strip_comments(raw, raw_len);
|
||||
free(raw);
|
||||
if (!stripped) {
|
||||
DEBUG_ERROR("failed to strip comments from config");
|
||||
return -1;
|
||||
}
|
||||
|
||||
cJSON *root = cJSON_Parse(stripped);
|
||||
free(stripped);
|
||||
if (!root) {
|
||||
DEBUG_ERROR("failed to parse config JSON");
|
||||
return -1;
|
||||
}
|
||||
|
||||
/* root_npubs */
|
||||
copy_string_array(cJSON_GetObjectItem(root, "root_npubs"),
|
||||
(char *)cfg->root_npubs, CR_MAX_ROOT_NPUBS,
|
||||
&cfg->root_npub_count, CR_NPUB_LEN);
|
||||
|
||||
/* upstream_relays */
|
||||
copy_string_array(cJSON_GetObjectItem(root, "upstream_relays"),
|
||||
(char *)cfg->upstream_relays, CR_MAX_UPSTREAM,
|
||||
&cfg->upstream_count, CR_URL_LEN);
|
||||
|
||||
/* local_relay */
|
||||
cJSON *local = cJSON_GetObjectItem(root, "local_relay");
|
||||
if (local && cJSON_IsString(local)) {
|
||||
strncpy(cfg->local_relay, cJSON_GetStringValue(local), CR_URL_LEN - 1);
|
||||
}
|
||||
|
||||
/* kinds */
|
||||
copy_int_array(cJSON_GetObjectItem(root, "kinds"),
|
||||
cfg->kinds, CR_MAX_KINDS, &cfg->kind_count);
|
||||
|
||||
/* admin_kinds - kinds to follow specifically for root (admin) npubs.
|
||||
* Supports [*] to mean "all kinds" (no kind filter for admin). */
|
||||
cJSON *ak = cJSON_GetObjectItem(root, "admin_kinds");
|
||||
if (ak && cJSON_IsArray(ak)) {
|
||||
cJSON *item;
|
||||
cJSON_ArrayForEach(item, ak) {
|
||||
if (cJSON_IsString(item)) {
|
||||
const char *s = cJSON_GetStringValue(item);
|
||||
if (s && strcmp(s, "*") == 0) {
|
||||
cfg->admin_all_kinds = 1;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!cfg->admin_all_kinds) {
|
||||
copy_int_array(ak, cfg->admin_kinds, CR_MAX_KINDS, &cfg->admin_kind_count);
|
||||
}
|
||||
}
|
||||
|
||||
/* backfill */
|
||||
cJSON *bf = cJSON_GetObjectItem(root, "backfill");
|
||||
if (bf) {
|
||||
cfg->backfill.enabled = cJSON_IsTrue(cJSON_GetObjectItem(bf, "enabled"));
|
||||
cJSON *ept = cJSON_GetObjectItem(bf, "events_per_tick");
|
||||
if (ept) cfg->backfill.events_per_tick = (int)cJSON_GetNumberValue(ept);
|
||||
cJSON *tis = cJSON_GetObjectItem(bf, "tick_interval_seconds");
|
||||
if (tis) cfg->backfill.tick_interval_seconds = (int)cJSON_GetNumberValue(tis);
|
||||
}
|
||||
|
||||
/* live */
|
||||
cJSON *lv = cJSON_GetObjectItem(root, "live");
|
||||
if (lv) {
|
||||
cfg->live.enabled = cJSON_IsTrue(cJSON_GetObjectItem(lv, "enabled"));
|
||||
cJSON *rsi = cJSON_GetObjectItem(lv, "resubscribe_interval_seconds");
|
||||
if (rsi) cfg->live.resubscribe_interval_seconds = (int)cJSON_GetNumberValue(rsi);
|
||||
}
|
||||
|
||||
/* follow_graph_refresh_seconds */
|
||||
cJSON *fgr = cJSON_GetObjectItem(root, "follow_graph_refresh_seconds");
|
||||
if (fgr) cfg->follow_graph_refresh_seconds = (int)cJSON_GetNumberValue(fgr);
|
||||
|
||||
/* state */
|
||||
cJSON *st = cJSON_GetObjectItem(root, "state");
|
||||
if (st) {
|
||||
cJSON *bu = cJSON_GetObjectItem(st, "backfilled_until");
|
||||
if (bu) cfg->state.backfilled_until = (long)cJSON_GetNumberValue(bu);
|
||||
}
|
||||
|
||||
cJSON_Delete(root);
|
||||
|
||||
/* Defaults if missing. */
|
||||
if (cfg->backfill.events_per_tick == 0) cfg->backfill.events_per_tick = 500;
|
||||
if (cfg->backfill.tick_interval_seconds == 0) cfg->backfill.tick_interval_seconds = 5;
|
||||
if (cfg->live.resubscribe_interval_seconds == 0) cfg->live.resubscribe_interval_seconds = 300;
|
||||
if (cfg->follow_graph_refresh_seconds == 0) cfg->follow_graph_refresh_seconds = 600;
|
||||
|
||||
/* Validation. */
|
||||
if (cfg->root_npub_count == 0) {
|
||||
DEBUG_ERROR("config: at least one root_npub required");
|
||||
return -1;
|
||||
}
|
||||
if (cfg->upstream_count == 0) {
|
||||
DEBUG_ERROR("config: at least one upstream_relay required");
|
||||
return -1;
|
||||
}
|
||||
if (cfg->local_relay[0] == '\0') {
|
||||
DEBUG_ERROR("config: local_relay required");
|
||||
return -1;
|
||||
}
|
||||
if (cfg->kind_count == 0) {
|
||||
DEBUG_ERROR("config: at least one kind required");
|
||||
return -1;
|
||||
}
|
||||
|
||||
DEBUG_INFO("config loaded: %d root npubs, %d upstream relays, %d kinds",
|
||||
cfg->root_npub_count, cfg->upstream_count, cfg->kind_count);
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* Build a fresh cJSON tree from the in-memory config and write it out.
|
||||
* We re-emit plain JSON (comments are not preserved on rewrite). */
|
||||
int cr_config_save_state(cr_config_t *cfg) {
|
||||
cJSON *root = cJSON_CreateObject();
|
||||
|
||||
cJSON *npubs = cJSON_CreateArray();
|
||||
for (int i = 0; i < cfg->root_npub_count; i++)
|
||||
cJSON_AddItemToArray(npubs, cJSON_CreateString(cfg->root_npubs[i]));
|
||||
cJSON_AddItemToObject(root, "root_npubs", npubs);
|
||||
|
||||
cJSON *upstream = cJSON_CreateArray();
|
||||
for (int i = 0; i < cfg->upstream_count; i++)
|
||||
cJSON_AddItemToArray(upstream, cJSON_CreateString(cfg->upstream_relays[i]));
|
||||
cJSON_AddItemToObject(root, "upstream_relays", upstream);
|
||||
|
||||
cJSON_AddItemToObject(root, "local_relay", cJSON_CreateString(cfg->local_relay));
|
||||
|
||||
cJSON *kinds = cJSON_CreateArray();
|
||||
for (int i = 0; i < cfg->kind_count; i++)
|
||||
cJSON_AddItemToArray(kinds, cJSON_CreateNumber(cfg->kinds[i]));
|
||||
cJSON_AddItemToObject(root, "kinds", kinds);
|
||||
|
||||
/* admin_kinds */
|
||||
cJSON *akinds = cJSON_CreateArray();
|
||||
if (cfg->admin_all_kinds) {
|
||||
cJSON_AddItemToArray(akinds, cJSON_CreateString("*"));
|
||||
} else {
|
||||
for (int i = 0; i < cfg->admin_kind_count; i++)
|
||||
cJSON_AddItemToArray(akinds, cJSON_CreateNumber(cfg->admin_kinds[i]));
|
||||
}
|
||||
cJSON_AddItemToObject(root, "admin_kinds", akinds);
|
||||
|
||||
cJSON *bf = cJSON_CreateObject();
|
||||
cJSON_AddItemToObject(bf, "enabled", cJSON_CreateBool(cfg->backfill.enabled));
|
||||
cJSON_AddItemToObject(bf, "events_per_tick", cJSON_CreateNumber(cfg->backfill.events_per_tick));
|
||||
cJSON_AddItemToObject(bf, "tick_interval_seconds", cJSON_CreateNumber(cfg->backfill.tick_interval_seconds));
|
||||
cJSON_AddItemToObject(root, "backfill", bf);
|
||||
|
||||
cJSON *lv = cJSON_CreateObject();
|
||||
cJSON_AddItemToObject(lv, "enabled", cJSON_CreateBool(cfg->live.enabled));
|
||||
cJSON_AddItemToObject(lv, "resubscribe_interval_seconds", cJSON_CreateNumber(cfg->live.resubscribe_interval_seconds));
|
||||
cJSON_AddItemToObject(root, "live", lv);
|
||||
|
||||
cJSON_AddItemToObject(root, "follow_graph_refresh_seconds",
|
||||
cJSON_CreateNumber(cfg->follow_graph_refresh_seconds));
|
||||
|
||||
cJSON *st = cJSON_CreateObject();
|
||||
cJSON_AddItemToObject(st, "backfilled_until", cJSON_CreateNumber(cfg->state.backfilled_until));
|
||||
cJSON_AddItemToObject(root, "state", st);
|
||||
|
||||
char *json = cJSON_Print(root);
|
||||
cJSON_Delete(root);
|
||||
if (!json) {
|
||||
DEBUG_ERROR("config save: failed to serialize");
|
||||
return -1;
|
||||
}
|
||||
|
||||
/* Atomic write: temp file + rename. */
|
||||
char tmp[1100];
|
||||
snprintf(tmp, sizeof(tmp), "%s.tmp", cfg->path);
|
||||
int fd = open(tmp, O_WRONLY | O_CREAT | O_TRUNC, 0644);
|
||||
if (fd < 0) {
|
||||
DEBUG_ERROR("config save: cannot open tmp '%s': %s", tmp, strerror(errno));
|
||||
free(json);
|
||||
return -1;
|
||||
}
|
||||
size_t jlen = strlen(json);
|
||||
ssize_t w = write(fd, json, jlen);
|
||||
close(fd);
|
||||
free(json);
|
||||
if (w < 0 || (size_t)w != jlen) {
|
||||
DEBUG_ERROR("config save: short write");
|
||||
return -1;
|
||||
}
|
||||
if (rename(tmp, cfg->path) != 0) {
|
||||
DEBUG_ERROR("config save: rename failed: %s", strerror(errno));
|
||||
return -1;
|
||||
}
|
||||
DEBUG_LOG("config state saved: backfilled_until=%ld",
|
||||
cfg->state.backfilled_until);
|
||||
return 0;
|
||||
}
|
||||
|
||||
void cr_config_free(cr_config_t *cfg) {
|
||||
(void)cfg;
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
/*
|
||||
* caching_relay - config parsing + persistent state (in the .jsonc file itself)
|
||||
*/
|
||||
#ifndef CACHING_RELAY_CONFIG_H
|
||||
#define CACHING_RELAY_CONFIG_H
|
||||
|
||||
#include <stddef.h>
|
||||
#include <time.h>
|
||||
|
||||
/* Maximums to keep things statically sized and simple. */
|
||||
#define CR_MAX_ROOT_NPUBS 16
|
||||
#define CR_MAX_UPSTREAM 32
|
||||
#define CR_MAX_KINDS 32
|
||||
#define CR_NPUB_LEN 64 /* npub1... bech32, generous */
|
||||
#define CR_URL_LEN 256
|
||||
#define CR_HEX_PUBKEY_LEN 65 /* 64 hex chars + NUL */
|
||||
|
||||
typedef struct {
|
||||
int enabled;
|
||||
int events_per_tick; /* per-query page size (default 500) */
|
||||
int tick_interval_seconds;
|
||||
} cr_backfill_config_t;
|
||||
|
||||
typedef struct {
|
||||
int enabled;
|
||||
int resubscribe_interval_seconds;
|
||||
} cr_live_config_t;
|
||||
|
||||
/* Persistent state. The per-author until-cursor drain model stores all
|
||||
* backfill progress in the caching_followed_pubkeys table, so the only
|
||||
* state kept here is the legacy-mode first-time flag (used to drive relay
|
||||
* discovery caching behavior). */
|
||||
typedef struct {
|
||||
long backfilled_until; /* legacy: 0 = first-time startup */
|
||||
} cr_state_t;
|
||||
|
||||
typedef struct {
|
||||
char root_npubs[CR_MAX_ROOT_NPUBS][CR_NPUB_LEN];
|
||||
int root_npub_count;
|
||||
/* Decoded hex pubkeys for root npubs (filled by follow_graph). */
|
||||
char root_hex[CR_MAX_ROOT_NPUBS][CR_HEX_PUBKEY_LEN];
|
||||
int root_hex_ready;
|
||||
|
||||
char upstream_relays[CR_MAX_UPSTREAM][CR_URL_LEN];
|
||||
int upstream_count;
|
||||
|
||||
char local_relay[CR_URL_LEN];
|
||||
|
||||
int kinds[CR_MAX_KINDS];
|
||||
int kind_count;
|
||||
|
||||
/* Kinds to follow specifically for the root (admin) npubs.
|
||||
* If admin_all_kinds is 1, grab everything (no kind filter). */
|
||||
int admin_kinds[CR_MAX_KINDS];
|
||||
int admin_kind_count;
|
||||
int admin_all_kinds;
|
||||
|
||||
cr_backfill_config_t backfill;
|
||||
cr_live_config_t live;
|
||||
int follow_graph_refresh_seconds;
|
||||
|
||||
cr_state_t state;
|
||||
|
||||
/* Path the config was loaded from (for state write-back). */
|
||||
char path[1024];
|
||||
} cr_config_t;
|
||||
|
||||
/* Load + validate config from a .jsonc file. Returns 0 on success, -1 on error.
|
||||
* On success cfg->path is set and cfg->state is populated from the file. */
|
||||
int cr_config_load(cr_config_t *cfg, const char *path);
|
||||
|
||||
/* Persist the state sub-object back into the config file (temp + rename).
|
||||
* Preserves all other config fields. Returns 0 on success, -1 on error. */
|
||||
int cr_config_save_state(cr_config_t *cfg);
|
||||
|
||||
/* Free any heap resources held by cfg (currently none, but kept for future). */
|
||||
void cr_config_free(cr_config_t *cfg);
|
||||
|
||||
#endif /* CACHING_RELAY_CONFIG_H */
|
||||
@@ -0,0 +1,71 @@
|
||||
#include "debug.h"
|
||||
#include <stdarg.h>
|
||||
#include <string.h>
|
||||
|
||||
/**
|
||||
* @file debug.c
|
||||
* @brief Debug and logging system implementation
|
||||
*
|
||||
* Provides a configurable logging system with timestamp formatting,
|
||||
* level-based filtering, and optional file:line information.
|
||||
*/
|
||||
|
||||
// Global debug level (default: no debug output)
|
||||
debug_level_t g_debug_level = DEBUG_LEVEL_NONE;
|
||||
|
||||
/**
|
||||
* @brief Initialize the debug system with a specific level
|
||||
* @param level Debug level (0-5, clamped to valid range)
|
||||
*/
|
||||
void debug_init(int level) {
|
||||
if (level < 0) level = 0;
|
||||
if (level > 5) level = 5;
|
||||
g_debug_level = (debug_level_t)level;
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief Core logging function
|
||||
* @param level Debug level for this message
|
||||
* @param file Source file name (__FILE__)
|
||||
* @param line Source line number (__LINE__)
|
||||
* @param format printf-style format string
|
||||
* @param ... Variable arguments for format string
|
||||
*/
|
||||
void debug_log(debug_level_t level, const char* file, int line, const char* format, ...) {
|
||||
// Get timestamp
|
||||
time_t now = time(NULL);
|
||||
struct tm* tm_info = localtime(&now);
|
||||
char timestamp[32];
|
||||
strftime(timestamp, sizeof(timestamp), "%Y-%m-%d %H:%M:%S", tm_info);
|
||||
|
||||
// Get level string
|
||||
const char* level_str = "UNKNOWN";
|
||||
switch (level) {
|
||||
case DEBUG_LEVEL_ERROR: level_str = "ERROR"; break;
|
||||
case DEBUG_LEVEL_WARN: level_str = "WARN "; break;
|
||||
case DEBUG_LEVEL_INFO: level_str = "INFO "; break;
|
||||
case DEBUG_LEVEL_DEBUG: level_str = "DEBUG"; break;
|
||||
case DEBUG_LEVEL_TRACE: level_str = "TRACE"; break;
|
||||
default: break;
|
||||
}
|
||||
|
||||
// Print prefix with timestamp and level
|
||||
printf("[%s] [%s] ", timestamp, level_str);
|
||||
|
||||
// Print source location when debug level is TRACE (5) or higher
|
||||
if (file && g_debug_level >= DEBUG_LEVEL_TRACE) {
|
||||
// Extract just the filename (not full path)
|
||||
const char* filename = strrchr(file, '/');
|
||||
filename = filename ? filename + 1 : file;
|
||||
printf("[%s:%d] ", filename, line);
|
||||
}
|
||||
|
||||
// Print message
|
||||
va_list args;
|
||||
va_start(args, format);
|
||||
vprintf(format, args);
|
||||
va_end(args);
|
||||
|
||||
printf("\n");
|
||||
fflush(stdout);
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
#ifndef C_UTILS_DEBUG_H
|
||||
#define C_UTILS_DEBUG_H
|
||||
|
||||
/**
|
||||
* @file debug.h
|
||||
* @brief Debug and logging system with configurable verbosity levels
|
||||
*
|
||||
* Provides a simple, efficient logging system with 5 levels:
|
||||
* - ERROR: Critical errors
|
||||
* - WARN: Warnings
|
||||
* - INFO: Informational messages
|
||||
* - DEBUG: Debug messages
|
||||
* - TRACE: Detailed trace with file:line info
|
||||
*/
|
||||
|
||||
#include <stdio.h>
|
||||
#include <time.h>
|
||||
|
||||
// Debug levels
|
||||
typedef enum {
|
||||
DEBUG_LEVEL_NONE = 0, /**< No debug output */
|
||||
DEBUG_LEVEL_ERROR = 1, /**< Critical errors only */
|
||||
DEBUG_LEVEL_WARN = 2, /**< Warnings and above */
|
||||
DEBUG_LEVEL_INFO = 3, /**< Informational messages and above */
|
||||
DEBUG_LEVEL_DEBUG = 4, /**< Debug messages and above */
|
||||
DEBUG_LEVEL_TRACE = 5 /**< Detailed trace with file:line info */
|
||||
} debug_level_t;
|
||||
|
||||
// Global debug level (set at runtime via CLI)
|
||||
extern debug_level_t g_debug_level;
|
||||
|
||||
/**
|
||||
* @brief Initialize the debug system
|
||||
* @param level Debug level (0-5, clamped to valid range)
|
||||
*/
|
||||
void debug_init(int level);
|
||||
|
||||
/**
|
||||
* @brief Core logging function
|
||||
* @param level Debug level for this message
|
||||
* @param file Source file name (__FILE__)
|
||||
* @param line Source line number (__LINE__)
|
||||
* @param format printf-style format string
|
||||
* @param ... Variable arguments for format string
|
||||
*/
|
||||
void debug_log(debug_level_t level, const char* file, int line, const char* format, ...);
|
||||
|
||||
// Convenience macros that check level before calling
|
||||
// Note: TRACE level (5) and above include file:line information for ALL messages
|
||||
#define DEBUG_ERROR(...) \
|
||||
do { if (g_debug_level >= DEBUG_LEVEL_ERROR) debug_log(DEBUG_LEVEL_ERROR, __FILE__, __LINE__, __VA_ARGS__); } while(0)
|
||||
|
||||
#define DEBUG_WARN(...) \
|
||||
do { if (g_debug_level >= DEBUG_LEVEL_WARN) debug_log(DEBUG_LEVEL_WARN, __FILE__, __LINE__, __VA_ARGS__); } while(0)
|
||||
|
||||
#define DEBUG_INFO(...) \
|
||||
do { if (g_debug_level >= DEBUG_LEVEL_INFO) debug_log(DEBUG_LEVEL_INFO, __FILE__, __LINE__, __VA_ARGS__); } while(0)
|
||||
|
||||
#define DEBUG_LOG(...) \
|
||||
do { if (g_debug_level >= DEBUG_LEVEL_DEBUG) debug_log(DEBUG_LEVEL_DEBUG, __FILE__, __LINE__, __VA_ARGS__); } while(0)
|
||||
|
||||
#define DEBUG_TRACE(...) \
|
||||
do { if (g_debug_level >= DEBUG_LEVEL_TRACE) debug_log(DEBUG_LEVEL_TRACE, __FILE__, __LINE__, __VA_ARGS__); } while(0)
|
||||
|
||||
#endif /* C_UTILS_DEBUG_H */
|
||||
@@ -0,0 +1,110 @@
|
||||
/*
|
||||
* caching_relay - follow graph resolution
|
||||
*/
|
||||
#define _GNU_SOURCE
|
||||
#include "follow_graph.h"
|
||||
#include "debug.h"
|
||||
|
||||
#include <string.h>
|
||||
#include <stdlib.h>
|
||||
|
||||
int cr_follow_is_root(const cr_config_t *cfg, const char *hex) {
|
||||
if (!cfg->root_hex_ready) return 0;
|
||||
for (int i = 0; i < cfg->root_npub_count; i++) {
|
||||
if (strcmp(cfg->root_hex[i], hex) == 0) return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
int cr_follow_decode_roots(cr_config_t *cfg) {
|
||||
for (int i = 0; i < cfg->root_npub_count; i++) {
|
||||
unsigned char pubkey[32];
|
||||
if (nostr_decode_npub(cfg->root_npubs[i], pubkey) != NOSTR_SUCCESS) {
|
||||
DEBUG_ERROR("follow: failed to decode npub '%s'", cfg->root_npubs[i]);
|
||||
return -1;
|
||||
}
|
||||
nostr_bytes_to_hex(pubkey, 32, cfg->root_hex[i]);
|
||||
}
|
||||
cfg->root_hex_ready = 1;
|
||||
DEBUG_INFO("follow: decoded %d root npubs", cfg->root_npub_count);
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* Parse "p" tags from a kind-3 event and add pubkeys to the set. */
|
||||
static int parse_p_tags(cJSON *event, cr_pubkey_set_t *followed) {
|
||||
cJSON *tags = cJSON_GetObjectItem(event, "tags");
|
||||
if (!tags || !cJSON_IsArray(tags)) return 0;
|
||||
int added = 0;
|
||||
cJSON *tag;
|
||||
cJSON_ArrayForEach(tag, tags) {
|
||||
if (!cJSON_IsArray(tag)) continue;
|
||||
cJSON *name = cJSON_GetArrayItem(tag, 0);
|
||||
if (!name || !cJSON_IsString(name)) continue;
|
||||
if (strcmp(cJSON_GetStringValue(name), "p") != 0) continue;
|
||||
cJSON *pk = cJSON_GetArrayItem(tag, 1);
|
||||
if (!pk || !cJSON_IsString(pk)) continue;
|
||||
const char *hex = cJSON_GetStringValue(pk);
|
||||
if (strlen(hex) != 64) continue;
|
||||
if (cr_pubkey_set_add(followed, hex) == 1) added++;
|
||||
}
|
||||
return added;
|
||||
}
|
||||
|
||||
int cr_follow_resolve(cr_config_t *cfg, nostr_relay_pool_t *upstream,
|
||||
cr_pubkey_set_t *followed) {
|
||||
if (!cfg->root_hex_ready) {
|
||||
if (cr_follow_decode_roots(cfg) != 0) return -1;
|
||||
}
|
||||
|
||||
/* Seed the followed set with the root pubkeys themselves. */
|
||||
for (int i = 0; i < cfg->root_npub_count; i++) {
|
||||
cr_pubkey_set_add(followed, cfg->root_hex[i]);
|
||||
}
|
||||
|
||||
int total_follows = 0;
|
||||
for (int i = 0; i < cfg->root_npub_count; i++) {
|
||||
/* Build filter: authors=[root], kinds=[3], limit=1 (most recent). */
|
||||
cJSON *filter = cJSON_CreateObject();
|
||||
cJSON *authors = cJSON_CreateArray();
|
||||
cJSON_AddItemToArray(authors, cJSON_CreateString(cfg->root_hex[i]));
|
||||
cJSON_AddItemToObject(filter, "authors", authors);
|
||||
cJSON *kinds = cJSON_CreateArray();
|
||||
cJSON_AddItemToArray(kinds, cJSON_CreateNumber(3));
|
||||
cJSON_AddItemToObject(filter, "kinds", kinds);
|
||||
cJSON_AddItemToObject(filter, "limit", cJSON_CreateNumber(1));
|
||||
|
||||
const char **urls = NULL;
|
||||
int n = 0;
|
||||
char **listed = NULL;
|
||||
nostr_pool_relay_status_t *statuses = NULL;
|
||||
int listed_count = nostr_relay_pool_list_relays(upstream, &listed, &statuses);
|
||||
if (listed_count > 0) {
|
||||
urls = malloc(listed_count * sizeof(char *));
|
||||
for (int j = 0; j < listed_count; j++) { urls[n++] = listed[j]; }
|
||||
}
|
||||
free(listed);
|
||||
free(statuses);
|
||||
|
||||
int ev_count = 0;
|
||||
cJSON **events = nostr_relay_pool_query_sync(upstream, urls, n,
|
||||
filter, &ev_count, 15000);
|
||||
free(urls);
|
||||
cJSON_Delete(filter);
|
||||
|
||||
if (events && ev_count > 0) {
|
||||
int added = parse_p_tags(events[0], followed);
|
||||
total_follows += added;
|
||||
DEBUG_INFO("follow: root %d: kind-3 found, +%d follows (total set %d)",
|
||||
i, added, followed->count);
|
||||
for (int k = 0; k < ev_count; k++) cJSON_Delete(events[k]);
|
||||
free(events);
|
||||
} else {
|
||||
DEBUG_WARN("follow: root %d: no kind-3 found", i);
|
||||
free(events);
|
||||
}
|
||||
}
|
||||
|
||||
DEBUG_INFO("follow: resolved %d total followed pubkeys (set size %d)",
|
||||
total_follows, followed->count);
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
/*
|
||||
* caching_relay - follow graph resolution.
|
||||
*
|
||||
* Decodes root npubs (NIP-19) to hex, queries their most recent kind-3
|
||||
* contact list from the upstream pool, parses the "p" tags, and populates
|
||||
* the in-memory followed-pubkey set (root pubkeys + their follows).
|
||||
*/
|
||||
#ifndef CACHING_RELAY_FOLLOW_GRAPH_H
|
||||
#define CACHING_RELAY_FOLLOW_GRAPH_H
|
||||
|
||||
#include "config.h"
|
||||
#include "state.h"
|
||||
#include "../nostr_core_lib/nostr_core/nostr_core.h"
|
||||
|
||||
/* Check if a hex pubkey is one of the root (admin) npubs. Returns 1 if yes. */
|
||||
int cr_follow_is_root(const cr_config_t *cfg, const char *hex);
|
||||
|
||||
/* Decode all root_npubs in cfg into cfg->root_hex[]. Returns 0 on success. */
|
||||
int cr_follow_decode_roots(cr_config_t *cfg);
|
||||
|
||||
/* Resolve the follow graph: for each root hex pubkey, query the most recent
|
||||
* kind-3 from the upstream pool, parse "p" tags, and add root + follows to
|
||||
* the followed set. Returns 0 on success, -1 on hard failure. */
|
||||
int cr_follow_resolve(cr_config_t *cfg, nostr_relay_pool_t *upstream,
|
||||
cr_pubkey_set_t *followed);
|
||||
|
||||
#endif /* CACHING_RELAY_FOLLOW_GRAPH_H */
|
||||
@@ -0,0 +1,69 @@
|
||||
/*
|
||||
* caching_relay - strip JSONC comments so cJSON can parse it.
|
||||
*/
|
||||
#define _GNU_SOURCE
|
||||
#include "jsonc_strip.h"
|
||||
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
char *jsonc_strip_comments(const char *input, size_t len) {
|
||||
if (!input) return NULL;
|
||||
if (len == 0) len = strlen(input);
|
||||
|
||||
/* Worst case: output is same size as input (no comments). +1 for NUL. */
|
||||
char *out = malloc(len + 1);
|
||||
if (!out) return NULL;
|
||||
|
||||
size_t i = 0, o = 0;
|
||||
int in_string = 0;
|
||||
int escape = 0;
|
||||
|
||||
while (i < len) {
|
||||
char c = input[i];
|
||||
|
||||
if (in_string) {
|
||||
out[o++] = c;
|
||||
if (escape) {
|
||||
escape = 0;
|
||||
} else if (c == '\\') {
|
||||
escape = 1;
|
||||
} else if (c == '"') {
|
||||
in_string = 0;
|
||||
}
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
/* Not in a string. */
|
||||
if (c == '"') {
|
||||
in_string = 1;
|
||||
out[o++] = c;
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
/* Line comment // */
|
||||
if (c == '/' && i + 1 < len && input[i + 1] == '/') {
|
||||
i += 2;
|
||||
while (i < len && input[i] != '\n') i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
/* Block comment / * ... * / */
|
||||
if (c == '/' && i + 1 < len && input[i + 1] == '*') {
|
||||
i += 2;
|
||||
while (i < len && !(input[i] == '*' && i + 1 < len && input[i + 1] == '/')) i++;
|
||||
if (i < len) i += 2; /* skip closing */
|
||||
/* Preserve a space so tokens don't merge. */
|
||||
out[o++] = ' ';
|
||||
continue;
|
||||
}
|
||||
|
||||
out[o++] = c;
|
||||
i++;
|
||||
}
|
||||
|
||||
out[o] = '\0';
|
||||
return out;
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
/*
|
||||
* caching_relay - strip JSONC comments so cJSON can parse it.
|
||||
*
|
||||
* cJSON does not natively understand JSONC. This produces a heap buffer the
|
||||
* caller must free() containing the comment-stripped JSON. String literals are
|
||||
* respected so a double-slash inside a string is preserved.
|
||||
*/
|
||||
#ifndef CACHING_RELAY_JSONC_STRIP_H
|
||||
#define CACHING_RELAY_JSONC_STRIP_H
|
||||
|
||||
#include <stddef.h>
|
||||
|
||||
/* Returns malloc'd buffer (null-terminated) or NULL on alloc failure. */
|
||||
char *jsonc_strip_comments(const char *input, size_t len);
|
||||
|
||||
#endif /* CACHING_RELAY_JSONC_STRIP_H */
|
||||
@@ -0,0 +1,189 @@
|
||||
/*
|
||||
* caching_relay - live subscription on the upstream pool
|
||||
*/
|
||||
#define _GNU_SOURCE
|
||||
#include "live_subscriber.h"
|
||||
#include "follow_graph.h"
|
||||
#include "debug.h"
|
||||
|
||||
#include <string.h>
|
||||
#include <stdlib.h>
|
||||
#include <time.h>
|
||||
|
||||
/* Context passed as user_data to the subscription callbacks. */
|
||||
typedef struct {
|
||||
cr_sink_t *sink;
|
||||
cr_live_t *live;
|
||||
cr_config_t *cfg;
|
||||
} cr_live_ctx_t;
|
||||
|
||||
static cr_live_ctx_t g_live_ctx;
|
||||
|
||||
static void live_on_event(cJSON *event, const char *relay_url, void *user_data) {
|
||||
cr_live_ctx_t *ctx = (cr_live_ctx_t *)user_data;
|
||||
(void)relay_url;
|
||||
ctx->live->events_received++;
|
||||
cr_sink_publish(ctx->sink, event);
|
||||
|
||||
/* Detect admin kind-3 (contact list) changes from a root pubkey and
|
||||
* signal the main loop to refresh the follow graph immediately. */
|
||||
if (ctx->cfg) {
|
||||
cJSON *kind = cJSON_GetObjectItem(event, "kind");
|
||||
cJSON *pubkey = cJSON_GetObjectItem(event, "pubkey");
|
||||
if (kind && cJSON_IsNumber(kind) && kind->valuedouble == 3.0 &&
|
||||
pubkey && cJSON_IsString(pubkey)) {
|
||||
if (cr_follow_is_root(ctx->cfg, cJSON_GetStringValue(pubkey))) {
|
||||
ctx->live->follow_graph_changed = 1;
|
||||
DEBUG_INFO("live: root kind-3 detected from %s, "
|
||||
"signaling follow graph refresh",
|
||||
cJSON_GetStringValue(pubkey));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static void live_on_eose(cJSON **events, int event_count, void *user_data) {
|
||||
(void)events; (void)event_count; (void)user_data;
|
||||
DEBUG_LOG("live: EOSE (stored events flushed), staying open");
|
||||
}
|
||||
|
||||
/* Build a filter for non-admin followed pubkeys with regular kinds. */
|
||||
static cJSON *build_follows_filter(cr_config_t *cfg, cr_pubkey_set_t *followed) {
|
||||
cJSON *filter = cJSON_CreateObject();
|
||||
cJSON *authors = cJSON_CreateArray();
|
||||
for (int i = 0; i < followed->count; i++) {
|
||||
/* Skip admin pubkeys - they get their own subscription. */
|
||||
if (cr_follow_is_root(cfg, followed->items[i])) continue;
|
||||
cJSON_AddItemToArray(authors, cJSON_CreateString(followed->items[i]));
|
||||
}
|
||||
cJSON_AddItemToObject(filter, "authors", authors);
|
||||
|
||||
cJSON *kinds = cJSON_CreateArray();
|
||||
for (int i = 0; i < cfg->kind_count; i++)
|
||||
cJSON_AddItemToArray(kinds, cJSON_CreateNumber(cfg->kinds[i]));
|
||||
cJSON_AddItemToObject(filter, "kinds", kinds);
|
||||
|
||||
cJSON_AddItemToObject(filter, "since", cJSON_CreateNumber((double)time(NULL)));
|
||||
return filter;
|
||||
}
|
||||
|
||||
/* Build a filter for admin (root) pubkeys with admin_kinds (or all kinds). */
|
||||
static cJSON *build_admin_filter(cr_config_t *cfg) {
|
||||
cJSON *filter = cJSON_CreateObject();
|
||||
cJSON *authors = cJSON_CreateArray();
|
||||
for (int i = 0; i < cfg->root_npub_count; i++)
|
||||
cJSON_AddItemToArray(authors, cJSON_CreateString(cfg->root_hex[i]));
|
||||
cJSON_AddItemToObject(filter, "authors", authors);
|
||||
|
||||
/* If admin_all_kinds, omit the kinds filter entirely (grab everything). */
|
||||
if (!cfg->admin_all_kinds && cfg->admin_kind_count > 0) {
|
||||
cJSON *kinds = cJSON_CreateArray();
|
||||
for (int i = 0; i < cfg->admin_kind_count; i++)
|
||||
cJSON_AddItemToArray(kinds, cJSON_CreateNumber(cfg->admin_kinds[i]));
|
||||
cJSON_AddItemToObject(filter, "kinds", kinds);
|
||||
}
|
||||
|
||||
cJSON_AddItemToObject(filter, "since", cJSON_CreateNumber((double)time(NULL)));
|
||||
return filter;
|
||||
}
|
||||
|
||||
static int get_relay_urls(nostr_relay_pool_t *upstream, const char ***urls_out) {
|
||||
char **listed = NULL;
|
||||
nostr_pool_relay_status_t *statuses = NULL;
|
||||
int n = nostr_relay_pool_list_relays(upstream, &listed, &statuses);
|
||||
const char **urls = NULL;
|
||||
if (n > 0) {
|
||||
urls = malloc(n * sizeof(char *));
|
||||
for (int j = 0; j < n; j++) urls[j] = listed[j];
|
||||
}
|
||||
free(listed);
|
||||
free(statuses);
|
||||
*urls_out = urls;
|
||||
return n;
|
||||
}
|
||||
|
||||
static nostr_pool_subscription_t *open_subscription(nostr_relay_pool_t *upstream,
|
||||
cJSON *filter) {
|
||||
const char **urls = NULL;
|
||||
int n = get_relay_urls(upstream, &urls);
|
||||
|
||||
nostr_pool_subscription_t *sub = nostr_relay_pool_subscribe(
|
||||
upstream, urls, n, filter,
|
||||
live_on_event, live_on_eose, &g_live_ctx,
|
||||
0, 1, NOSTR_POOL_EOSE_FULL_SET, 0, 0);
|
||||
free(urls);
|
||||
return sub;
|
||||
}
|
||||
|
||||
static int open_subs(cr_live_t *live, cr_config_t *cfg, nostr_relay_pool_t *upstream,
|
||||
cr_pubkey_set_t *followed, cr_sink_t *sink) {
|
||||
g_live_ctx.sink = sink;
|
||||
g_live_ctx.live = live;
|
||||
g_live_ctx.cfg = cfg;
|
||||
|
||||
int admin_count = 0;
|
||||
int follows_count = 0;
|
||||
for (int i = 0; i < followed->count; i++) {
|
||||
if (cr_follow_is_root(cfg, followed->items[i])) admin_count++;
|
||||
else follows_count++;
|
||||
}
|
||||
|
||||
/* Follows subscription (non-admin pubkeys, regular kinds). */
|
||||
if (follows_count > 0) {
|
||||
cJSON *filter = build_follows_filter(cfg, followed);
|
||||
live->follows_sub = open_subscription(upstream, filter);
|
||||
cJSON_Delete(filter);
|
||||
if (!live->follows_sub) {
|
||||
DEBUG_ERROR("live: follows subscribe failed");
|
||||
} else {
|
||||
DEBUG_INFO("live: follows sub - %d authors, %d kinds",
|
||||
follows_count, cfg->kind_count);
|
||||
}
|
||||
}
|
||||
|
||||
/* Admin subscription (root npubs, admin_kinds or all kinds). */
|
||||
if (admin_count > 0 || cfg->root_npub_count > 0) {
|
||||
cJSON *filter = build_admin_filter(cfg);
|
||||
live->admin_sub = open_subscription(upstream, filter);
|
||||
cJSON_Delete(filter);
|
||||
if (!live->admin_sub) {
|
||||
DEBUG_ERROR("live: admin subscribe failed");
|
||||
} else {
|
||||
if (cfg->admin_all_kinds) {
|
||||
DEBUG_INFO("live: admin sub - %d authors, ALL kinds",
|
||||
cfg->root_npub_count);
|
||||
} else {
|
||||
DEBUG_INFO("live: admin sub - %d authors, %d kinds",
|
||||
cfg->root_npub_count, cfg->admin_kind_count);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
live->last_resubscribe = time(NULL);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int cr_live_open(cr_live_t *live, cr_config_t *cfg, nostr_relay_pool_t *upstream,
|
||||
cr_pubkey_set_t *followed, cr_sink_t *sink) {
|
||||
memset(live, 0, sizeof(*live));
|
||||
return open_subs(live, cfg, upstream, followed, sink);
|
||||
}
|
||||
|
||||
int cr_live_resubscribe(cr_live_t *live, cr_config_t *cfg,
|
||||
nostr_relay_pool_t *upstream, cr_pubkey_set_t *followed,
|
||||
cr_sink_t *sink) {
|
||||
cr_live_close(live);
|
||||
DEBUG_INFO("live: resubscribing (events so far: %ld)", live->events_received);
|
||||
return open_subs(live, cfg, upstream, followed, sink);
|
||||
}
|
||||
|
||||
void cr_live_close(cr_live_t *live) {
|
||||
if (live->follows_sub) {
|
||||
nostr_pool_subscription_close(live->follows_sub);
|
||||
live->follows_sub = NULL;
|
||||
}
|
||||
if (live->admin_sub) {
|
||||
nostr_pool_subscription_close(live->admin_sub);
|
||||
live->admin_sub = NULL;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
/*
|
||||
* caching_relay - live subscription on the upstream pool.
|
||||
*
|
||||
* Builds filters from the followed-pubkey set + configured kinds, opens
|
||||
* long-lived subscriptions, and routes on_event to the relay sink.
|
||||
*
|
||||
* Two subscriptions are opened:
|
||||
* 1. "follows" sub: non-admin followed pubkeys + regular kinds
|
||||
* 2. "admin" sub: root (admin) npubs + admin_kinds (or all kinds if [*])
|
||||
*/
|
||||
#ifndef CACHING_RELAY_LIVE_SUBSCRIBER_H
|
||||
#define CACHING_RELAY_LIVE_SUBSCRIBER_H
|
||||
|
||||
#include "config.h"
|
||||
#include "state.h"
|
||||
#include "relay_sink.h"
|
||||
#include "../nostr_core_lib/nostr_core/nostr_core.h"
|
||||
|
||||
typedef struct {
|
||||
nostr_pool_subscription_t *follows_sub; /* non-admin pubkeys, regular kinds */
|
||||
nostr_pool_subscription_t *admin_sub; /* admin pubkeys, admin_kinds */
|
||||
long events_received;
|
||||
time_t last_resubscribe;
|
||||
int follow_graph_changed; /* set when a root npub publishes a kind-3 */
|
||||
} cr_live_t;
|
||||
|
||||
/* Open the live subscription(s). Returns 0 on success. */
|
||||
int cr_live_open(cr_live_t *live, cr_config_t *cfg, nostr_relay_pool_t *upstream,
|
||||
cr_pubkey_set_t *followed, cr_sink_t *sink);
|
||||
|
||||
/* Close + reopen the subscription(s). Returns 0 on success. */
|
||||
int cr_live_resubscribe(cr_live_t *live, cr_config_t *cfg,
|
||||
nostr_relay_pool_t *upstream, cr_pubkey_set_t *followed,
|
||||
cr_sink_t *sink);
|
||||
|
||||
void cr_live_close(cr_live_t *live);
|
||||
|
||||
#endif /* CACHING_RELAY_LIVE_SUBSCRIBER_H */
|
||||
@@ -0,0 +1,629 @@
|
||||
/*
|
||||
* caching_relay - a daemon that caches Nostr events from followed people
|
||||
* into a local relay.
|
||||
*
|
||||
* Architecture: see plans/plan.md. Two nostr_relay_pool_t instances:
|
||||
* - upstream_pool: query/subscribe to upstream relays
|
||||
* - sink_pool (in cr_sink_t): publish-only to the local relay
|
||||
*
|
||||
* Build: see Makefile. Statically linked C99 binary, c-relay style.
|
||||
*/
|
||||
#define _GNU_SOURCE
|
||||
#include "main.h"
|
||||
#include "debug.h"
|
||||
#include "config.h"
|
||||
#include "state.h"
|
||||
#include "follow_graph.h"
|
||||
#include "relay_sink.h"
|
||||
#include "live_subscriber.h"
|
||||
#include "backfill.h"
|
||||
#include "relay_discovery.h"
|
||||
#include "pg_inbox.h"
|
||||
#include "pg_config.h"
|
||||
|
||||
#include "../nostr_core_lib/nostr_core/nostr_core.h"
|
||||
#include "../nostr_core_lib/nostr_core/nostr_log.h"
|
||||
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <signal.h>
|
||||
#include <time.h>
|
||||
#include <unistd.h>
|
||||
#include <getopt.h>
|
||||
|
||||
/* Forward nostr_core_lib's internal logging to our debug system. */
|
||||
static void nostr_log_forwarder(int level, const char *component,
|
||||
const char *message, void *user_data) {
|
||||
(void)user_data;
|
||||
/* Map nostr log levels to our debug levels (they're the same 1-5). */
|
||||
if (level >= 5) {
|
||||
DEBUG_TRACE("[nostr:%s] %s", component ? component : "?", message ? message : "");
|
||||
} else if (level >= 4) {
|
||||
DEBUG_LOG("[nostr:%s] %s", component ? component : "?", message ? message : "");
|
||||
} else if (level >= 3) {
|
||||
DEBUG_INFO("[nostr:%s] %s", component ? component : "?", message ? message : "");
|
||||
} else if (level >= 2) {
|
||||
DEBUG_WARN("[nostr:%s] %s", component ? component : "?", message ? message : "");
|
||||
} else {
|
||||
DEBUG_ERROR("[nostr:%s] %s", component ? component : "?", message ? message : "");
|
||||
}
|
||||
}
|
||||
|
||||
/* Initialize per-relay backfill progress rows for every followed pubkey.
|
||||
* For each author, merges its discovered outbox relays (from relay_map)
|
||||
* with the bootstrap (upstream) relays from cfg, then inserts one
|
||||
* caching_backfill_relay_progress row per relay (ON CONFLICT DO NOTHING).
|
||||
* Safe to call repeatedly - existing rows are left untouched. */
|
||||
static void init_relay_progress_for_all(const cr_config_t *cfg,
|
||||
const cr_relay_map_t *relay_map,
|
||||
const cr_pubkey_set_t *followed) {
|
||||
if (!followed || followed->count <= 0) return;
|
||||
|
||||
/* Bootstrap relay URLs from cfg->upstream_relays. */
|
||||
const char **bootstrap = NULL;
|
||||
int bootstrap_n = cfg->upstream_count;
|
||||
if (bootstrap_n > 0) {
|
||||
bootstrap = malloc(bootstrap_n * sizeof(char *));
|
||||
if (!bootstrap) return;
|
||||
for (int i = 0; i < bootstrap_n; i++)
|
||||
bootstrap[i] = cfg->upstream_relays[i];
|
||||
}
|
||||
|
||||
for (int i = 0; i < followed->count; i++) {
|
||||
const char *pk = followed->items[i];
|
||||
if (!pk || !*pk) continue;
|
||||
|
||||
/* Collect this author's outbox relays. */
|
||||
const cr_outbox_entry_t *oe = NULL;
|
||||
int outbox_n = 0;
|
||||
if (relay_map) {
|
||||
oe = cr_relay_map_get_outbox(relay_map, pk);
|
||||
if (oe) outbox_n = oe->relay_count;
|
||||
}
|
||||
|
||||
int total = outbox_n + bootstrap_n;
|
||||
if (total <= 0) continue;
|
||||
|
||||
const char **urls = malloc(total * sizeof(char *));
|
||||
if (!urls) continue;
|
||||
int n = 0;
|
||||
if (oe) {
|
||||
for (int j = 0; j < oe->relay_count && n < total; j++)
|
||||
urls[n++] = oe->relays[j];
|
||||
}
|
||||
for (int j = 0; j < bootstrap_n && n < total; j++) {
|
||||
int dup = 0;
|
||||
for (int k = 0; k < n; k++) {
|
||||
if (strcmp(urls[k], bootstrap[j]) == 0) { dup = 1; break; }
|
||||
}
|
||||
if (!dup) urls[n++] = bootstrap[j];
|
||||
}
|
||||
|
||||
if (n > 0) {
|
||||
pg_inbox_init_relay_progress_for_author(pk, urls, n);
|
||||
}
|
||||
free(urls);
|
||||
}
|
||||
|
||||
free(bootstrap);
|
||||
}
|
||||
|
||||
volatile sig_atomic_t g_shutdown = 0;
|
||||
static volatile sig_atomic_t g_reload = 0;
|
||||
|
||||
static void on_signal(int sig) {
|
||||
if (sig == SIGINT || sig == SIGTERM) g_shutdown = 1;
|
||||
else if (sig == SIGHUP) g_reload = 1;
|
||||
}
|
||||
|
||||
static void usage(const char *prog) {
|
||||
fprintf(stderr,
|
||||
"caching_relay %s - cache Nostr events from followed people into a local relay\n"
|
||||
"\n"
|
||||
"Usage: %s [-c <config.jsonc>] [-p <pg-conn>] [options]\n"
|
||||
"\n"
|
||||
"Options:\n"
|
||||
" -c, --config <file> Path to .jsonc config file (default: ./caching_relay_config.jsonc)\n"
|
||||
" -p, --pg-conn <str> PostgreSQL connection string (libpq format). When provided,\n"
|
||||
" config is read from the c-relay-pg config table and fetched\n"
|
||||
" events are inserted into the caching_event_inbox table instead\n"
|
||||
" of being published via WebSocket to a local relay.\n"
|
||||
" -d, --debug <level> Log level 0-5 (0=none, 1=error, 2=warn, 3=info, 4=debug, 5=trace). Default 3.\n"
|
||||
" -r, --restart Reset state to first-time startup (ignore local relay cache,\n"
|
||||
" re-discover all kind-10002 from bootstrap relays, reset backfill)\n"
|
||||
" -h, --help Show this help\n"
|
||||
"\n"
|
||||
"Without -p, the daemon uses the .jsonc config file and publishes to a local relay\n"
|
||||
"via WebSocket (legacy mode). With -p, it uses PostgreSQL for config and inbox.\n"
|
||||
"The config file is also the persistent state store in legacy mode; the daemon\n"
|
||||
"rewrites it as backfill progresses. See plans/plan.md and caching_relay_config.jsonc.\n",
|
||||
CR_VERSION, prog);
|
||||
}
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
const char *config_path = NULL;
|
||||
const char *pg_conn = NULL;
|
||||
int log_level = DEBUG_LEVEL_INFO;
|
||||
int restart = 0;
|
||||
|
||||
static struct option longopts[] = {
|
||||
{"config", required_argument, 0, 'c'},
|
||||
{"pg-conn", required_argument, 0, 'p'},
|
||||
{"debug", required_argument, 0, 'd'},
|
||||
{"restart", no_argument, 0, 'r'},
|
||||
{"help", no_argument, 0, 'h'},
|
||||
{0, 0, 0, 0}
|
||||
};
|
||||
int opt;
|
||||
while ((opt = getopt_long(argc, argv, "c:p:d:rh", longopts, NULL)) != -1) {
|
||||
switch (opt) {
|
||||
case 'c': config_path = optarg; break;
|
||||
case 'p': pg_conn = optarg; break;
|
||||
case 'd': log_level = atoi(optarg); break;
|
||||
case 'r': restart = 1; break;
|
||||
case 'h': usage(argv[0]); return 0;
|
||||
default: usage(argv[0]); return 1;
|
||||
}
|
||||
}
|
||||
/* In PostgreSQL mode, the .jsonc config file is optional. */
|
||||
if (!config_path && !pg_conn) {
|
||||
/* Default: look for caching_relay.jsonc in the current directory. */
|
||||
config_path = "caching_relay_config.jsonc";
|
||||
if (access(config_path, F_OK) != 0) {
|
||||
fprintf(stderr, "ERROR: no config specified and default '%s' not found\n\n", config_path);
|
||||
usage(argv[0]);
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
if (log_level < 0) log_level = 0;
|
||||
if (log_level > 5) log_level = 5;
|
||||
|
||||
debug_init(log_level);
|
||||
|
||||
/* Enable nostr_core_lib internal logging and forward to our debug system.
|
||||
* This shows the actual WebSocket messages (REQ filters, EVENT responses)
|
||||
* at debug level 5 (trace). */
|
||||
nostr_set_log_callback(nostr_log_forwarder, NULL);
|
||||
nostr_set_log_level((nostr_log_level_t)log_level);
|
||||
|
||||
DEBUG_INFO("caching_relay %s starting (config=%s, pg-conn=%s, loglevel=%d%s)",
|
||||
CR_VERSION,
|
||||
config_path ? config_path : "(none)",
|
||||
pg_conn ? "yes" : "no",
|
||||
log_level, restart ? ", RESTART" : "");
|
||||
|
||||
/* Signals. */
|
||||
struct sigaction sa;
|
||||
memset(&sa, 0, sizeof(sa));
|
||||
sa.sa_handler = on_signal;
|
||||
sigaction(SIGINT, &sa, NULL);
|
||||
sigaction(SIGTERM, &sa, NULL);
|
||||
sigaction(SIGHUP, &sa, NULL);
|
||||
/* Ignore SIGPIPE - relay pool handles its own socket errors. */
|
||||
signal(SIGPIPE, SIG_IGN);
|
||||
|
||||
/* PostgreSQL inbox mode: connect and load config from the config table. */
|
||||
long config_generation = -1;
|
||||
if (pg_conn) {
|
||||
if (pg_inbox_init(pg_conn) != 0) {
|
||||
DEBUG_ERROR("failed to connect to PostgreSQL");
|
||||
return 1;
|
||||
}
|
||||
config_generation = pg_inbox_get_config_generation();
|
||||
if (config_generation < 0) {
|
||||
DEBUG_WARN("could not read caching_config_generation (defaulting to 0)");
|
||||
config_generation = 0;
|
||||
}
|
||||
DEBUG_INFO("config generation: %ld", config_generation);
|
||||
}
|
||||
|
||||
/* Load config. */
|
||||
cr_config_t cfg;
|
||||
if (pg_conn) {
|
||||
if (pg_config_load(&cfg) != 0) {
|
||||
DEBUG_ERROR("failed to load config from PostgreSQL");
|
||||
pg_inbox_shutdown();
|
||||
return 1;
|
||||
}
|
||||
} else {
|
||||
if (cr_config_load(&cfg, config_path) != 0) return 1;
|
||||
}
|
||||
|
||||
/* --restart: reset state to first-time startup. */
|
||||
if (restart) {
|
||||
DEBUG_INFO("RESTART: resetting state to first-time startup");
|
||||
cfg.state.backfilled_until = 0;
|
||||
if (pg_conn) {
|
||||
/* Reset per-author backfill progress (caching_followed_pubkeys)
|
||||
* so the next run starts fresh. The followed set itself is
|
||||
* preserved; only cursor/completion state is reset. */
|
||||
pg_inbox_reset_backfill_progress();
|
||||
} else {
|
||||
cr_config_save_state(&cfg);
|
||||
}
|
||||
}
|
||||
|
||||
/* Init crypto. */
|
||||
if (nostr_crypto_init() != NOSTR_SUCCESS) {
|
||||
DEBUG_ERROR("nostr_crypto_init failed");
|
||||
return 1;
|
||||
}
|
||||
|
||||
/* In-memory state. */
|
||||
cr_seen_ring_t seen;
|
||||
cr_seen_ring_init(&seen);
|
||||
cr_pubkey_set_t followed;
|
||||
cr_pubkey_set_init(&followed);
|
||||
|
||||
/* Upstream pool. */
|
||||
nostr_relay_pool_t *upstream = nostr_relay_pool_create(nostr_pool_reconnect_config_default());
|
||||
if (!upstream) {
|
||||
DEBUG_ERROR("failed to create upstream pool");
|
||||
return 1;
|
||||
}
|
||||
for (int i = 0; i < cfg.upstream_count; i++) {
|
||||
if (nostr_relay_pool_add_relay(upstream, cfg.upstream_relays[i]) != NOSTR_SUCCESS) {
|
||||
DEBUG_WARN("failed to add upstream relay %s (continuing)", cfg.upstream_relays[i]);
|
||||
} else {
|
||||
DEBUG_INFO("upstream: added %s", cfg.upstream_relays[i]);
|
||||
}
|
||||
}
|
||||
|
||||
/* Sink: WebSocket pool (legacy) or PostgreSQL inbox. */
|
||||
cr_sink_t sink;
|
||||
if (pg_conn) {
|
||||
if (cr_sink_init_pg(&sink, &seen) != 0) {
|
||||
DEBUG_ERROR("failed to init sink (pg)");
|
||||
return 1;
|
||||
}
|
||||
} else {
|
||||
if (cr_sink_init(&sink, cfg.local_relay, &seen) != 0) {
|
||||
DEBUG_ERROR("failed to init sink");
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
|
||||
/* Give the sink pool a moment to connect to the local relay before we
|
||||
* start publishing (relay discovery caches kind-10002 events to local).
|
||||
* No-op in PostgreSQL inbox mode (cr_sink_pump does nothing). */
|
||||
if (!pg_conn) {
|
||||
DEBUG_INFO("waiting for sink connection to establish...");
|
||||
for (int i = 0; i < 30 && !g_shutdown; i++) {
|
||||
cr_sink_pump(&sink, 100);
|
||||
}
|
||||
}
|
||||
|
||||
/* Resolve follow graph. */
|
||||
if (g_shutdown) goto shutdown;
|
||||
if (cr_follow_resolve(&cfg, upstream, &followed) != 0) {
|
||||
DEBUG_ERROR("follow graph resolution failed");
|
||||
return 1;
|
||||
}
|
||||
|
||||
/* Sync the followed set to the DB so the backfill tick can iterate
|
||||
* over it (per-author until-cursor drain). Also run the one-time
|
||||
* migration from the old caching_backfill_progress table. */
|
||||
if (pg_conn) {
|
||||
pg_inbox_migrate_backfill_progress();
|
||||
const char **root_pks = malloc(cfg.root_npub_count * sizeof(char *));
|
||||
for (int i = 0; i < cfg.root_npub_count; i++)
|
||||
root_pks[i] = cfg.root_hex[i];
|
||||
pg_inbox_sync_followed_pubkeys((const char **)followed.items,
|
||||
followed.count, root_pks,
|
||||
cfg.root_npub_count);
|
||||
free(root_pks);
|
||||
}
|
||||
|
||||
if (g_shutdown) goto shutdown;
|
||||
|
||||
/* Discover outbox relays (NIP-65) and compute minimum covering set. */
|
||||
int is_first_time = (cfg.state.backfilled_until == 0);
|
||||
cr_relay_map_t relay_map;
|
||||
if (cr_relay_discovery_run(&relay_map, &cfg, upstream, &sink, &followed,
|
||||
is_first_time) != 0) {
|
||||
DEBUG_WARN("relay discovery failed, continuing with bootstrap relays only");
|
||||
memset(&relay_map, 0, sizeof(relay_map));
|
||||
}
|
||||
|
||||
/* Add discovered outbox relays to the upstream pool. */
|
||||
for (int i = 0; i < relay_map.selected_count; i++) {
|
||||
/* Check if already in the pool (bootstrap relays may already be there). */
|
||||
char **listed = NULL;
|
||||
nostr_pool_relay_status_t *statuses = NULL;
|
||||
int n = nostr_relay_pool_list_relays(upstream, &listed, &statuses);
|
||||
int already = 0;
|
||||
for (int j = 0; j < n; j++) {
|
||||
if (strcmp(listed[j], relay_map.selected_relays[i]) == 0) {
|
||||
already = 1; break;
|
||||
}
|
||||
}
|
||||
free(listed);
|
||||
free(statuses);
|
||||
if (!already) {
|
||||
if (nostr_relay_pool_add_relay(upstream, relay_map.selected_relays[i]) == NOSTR_SUCCESS) {
|
||||
DEBUG_INFO("upstream: added outbox relay %s", relay_map.selected_relays[i]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* Log final upstream pool. */
|
||||
{
|
||||
char **listed = NULL;
|
||||
nostr_pool_relay_status_t *statuses = NULL;
|
||||
int n = nostr_relay_pool_list_relays(upstream, &listed, &statuses);
|
||||
DEBUG_INFO("upstream_pool: %d relays connected:", n);
|
||||
for (int j = 0; j < n; j++) {
|
||||
DEBUG_INFO(" %s", listed[j]);
|
||||
}
|
||||
free(listed);
|
||||
free(statuses);
|
||||
}
|
||||
|
||||
/* Create per-relay backfill progress rows for every followed pubkey
|
||||
* (outbox relays + bootstrap relays). Existing rows are left untouched
|
||||
* so cursor/completion state is preserved across restarts. */
|
||||
if (pg_conn) {
|
||||
init_relay_progress_for_all(&cfg, &relay_map, &followed);
|
||||
}
|
||||
|
||||
/* Open live subscription. */
|
||||
cr_live_t live;
|
||||
if (cfg.live.enabled) {
|
||||
if (cr_live_open(&live, &cfg, upstream, &followed, &sink) != 0) {
|
||||
DEBUG_WARN("live subscription failed to open (will retry on resubscribe)");
|
||||
}
|
||||
} else {
|
||||
memset(&live, 0, sizeof(live));
|
||||
}
|
||||
|
||||
/* Init backfill. */
|
||||
cr_backfill_t bf;
|
||||
cr_backfill_init(&bf, &cfg);
|
||||
|
||||
time_t last_follow_refresh = time(NULL);
|
||||
time_t last_state_save = time(NULL);
|
||||
time_t last_status_heartbeat = 0;
|
||||
|
||||
/* Initial status heartbeat in PostgreSQL mode. */
|
||||
if (pg_conn) {
|
||||
int bf_complete = 0, bf_total = 0;
|
||||
pg_inbox_count_backfill_progress(&bf_complete, &bf_total);
|
||||
pg_inbox_update_status("starting", config_generation, (long)time(NULL),
|
||||
followed.count, relay_map.selected_count, 0,
|
||||
bf_complete, bf_total, 0, 0, NULL, 0);
|
||||
}
|
||||
|
||||
DEBUG_INFO("entering main loop");
|
||||
|
||||
while (!g_shutdown) {
|
||||
if (g_reload) {
|
||||
g_reload = 0;
|
||||
DEBUG_INFO("SIGHUP: reloading config (state preserved)");
|
||||
cr_config_t newcfg;
|
||||
int reload_ok = 0;
|
||||
if (pg_conn) {
|
||||
if (pg_config_load(&newcfg) == 0 &&
|
||||
cr_follow_decode_roots(&newcfg) == 0) reload_ok = 1;
|
||||
} else if (config_path) {
|
||||
if (cr_config_load(&newcfg, config_path) == 0) reload_ok = 1;
|
||||
}
|
||||
if (reload_ok) {
|
||||
/* Preserve runtime state across reload. */
|
||||
newcfg.state = cfg.state;
|
||||
cr_config_free(&cfg);
|
||||
cfg = newcfg;
|
||||
DEBUG_INFO("config reloaded");
|
||||
/* Force an immediate follow-graph refresh so any new
|
||||
* root npubs are picked up right away. */
|
||||
last_follow_refresh = 0;
|
||||
} else {
|
||||
DEBUG_ERROR("config reload failed, keeping old config");
|
||||
}
|
||||
}
|
||||
|
||||
/* PostgreSQL: check for config generation change and reload. */
|
||||
if (pg_conn) {
|
||||
int changed = pg_config_generation_changed(config_generation);
|
||||
if (changed == 1) {
|
||||
long new_gen = pg_inbox_get_config_generation();
|
||||
DEBUG_INFO("config generation changed (%ld -> %ld), reloading",
|
||||
config_generation, new_gen);
|
||||
cr_config_t newcfg;
|
||||
if (pg_config_load(&newcfg) == 0) {
|
||||
/* Decode root npubs to hex — pg_config_load fills
|
||||
* root_npubs[] but NOT root_hex[] / root_hex_ready.
|
||||
* Without this, cr_follow_is_root() returns 0 for
|
||||
* everything and the new root's follows are never
|
||||
* resolved. */
|
||||
if (cr_follow_decode_roots(&newcfg) != 0) {
|
||||
DEBUG_ERROR("config reload: failed to decode root npubs, keeping old config");
|
||||
cr_config_free(&newcfg);
|
||||
} else {
|
||||
newcfg.state = cfg.state;
|
||||
cr_config_free(&cfg);
|
||||
cfg = newcfg;
|
||||
config_generation = new_gen;
|
||||
DEBUG_INFO("config reloaded from PostgreSQL");
|
||||
/* Force an immediate follow-graph refresh so the
|
||||
* new root npub's follows are picked up right
|
||||
* away (instead of waiting up to
|
||||
* follow_graph_refresh_seconds). */
|
||||
last_follow_refresh = 0;
|
||||
/* Resubscribe live with new config. */
|
||||
if (cfg.live.enabled) {
|
||||
cr_live_resubscribe(&live, &cfg, upstream, &followed, &sink);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
DEBUG_ERROR("PostgreSQL config reload failed, keeping old config");
|
||||
}
|
||||
} else if (changed < 0) {
|
||||
DEBUG_WARN("config generation check failed");
|
||||
}
|
||||
}
|
||||
|
||||
/* Pump upstream pool (drives live subscription callbacks). */
|
||||
nostr_relay_pool_run(upstream, 100);
|
||||
|
||||
/* Pump sink pool (flush publish callbacks). */
|
||||
cr_sink_pump(&sink, 50);
|
||||
|
||||
/* Backfill tick. */
|
||||
int brc = cr_backfill_tick(&bf, &cfg, upstream, &followed, &sink, &relay_map);
|
||||
(void)brc;
|
||||
|
||||
/* Immediate follow-graph refresh when a root npub publishes a new
|
||||
* kind-3 contact list (detected by the live subscriber). */
|
||||
if (cfg.live.enabled && live.follow_graph_changed) {
|
||||
live.follow_graph_changed = 0;
|
||||
DEBUG_INFO("live: admin kind-3 detected, refreshing follow graph immediately");
|
||||
cr_pubkey_set_t new_followed;
|
||||
cr_pubkey_set_init(&new_followed);
|
||||
if (cr_follow_resolve(&cfg, upstream, &new_followed) == 0) {
|
||||
int changed = (new_followed.count != followed.count);
|
||||
if (!changed) {
|
||||
for (int i = 0; i < followed.count; i++) {
|
||||
if (!cr_pubkey_set_contains(&new_followed, followed.items[i])) {
|
||||
changed = 1; break;
|
||||
}
|
||||
}
|
||||
}
|
||||
cr_pubkey_set_free(&followed);
|
||||
followed = new_followed;
|
||||
if (pg_conn) {
|
||||
const char **root_pks = malloc(cfg.root_npub_count * sizeof(char *));
|
||||
for (int i = 0; i < cfg.root_npub_count; i++)
|
||||
root_pks[i] = cfg.root_hex[i];
|
||||
pg_inbox_sync_followed_pubkeys((const char **)followed.items,
|
||||
followed.count, root_pks,
|
||||
cfg.root_npub_count);
|
||||
free(root_pks);
|
||||
/* Create relay progress rows for newly added follows.
|
||||
* Existing rows are preserved (ON CONFLICT DO NOTHING). */
|
||||
init_relay_progress_for_all(&cfg, &relay_map, &followed);
|
||||
}
|
||||
if (changed && cfg.live.enabled) {
|
||||
cr_live_resubscribe(&live, &cfg, upstream, &followed, &sink);
|
||||
}
|
||||
/* Reset the backfill in_progress flag in case we had entered
|
||||
* steady-state; new follows need to be drained. */
|
||||
bf.in_progress = 1;
|
||||
last_follow_refresh = time(NULL);
|
||||
} else {
|
||||
DEBUG_WARN("immediate follow graph refresh failed");
|
||||
cr_pubkey_set_free(&new_followed);
|
||||
}
|
||||
}
|
||||
|
||||
/* Periodic follow-graph refresh. */
|
||||
time_t now = time(NULL);
|
||||
if (cfg.follow_graph_refresh_seconds > 0 &&
|
||||
(now - last_follow_refresh) >= cfg.follow_graph_refresh_seconds) {
|
||||
DEBUG_INFO("refreshing follow graph");
|
||||
cr_pubkey_set_t new_followed;
|
||||
cr_pubkey_set_init(&new_followed);
|
||||
if (cr_follow_resolve(&cfg, upstream, &new_followed) == 0) {
|
||||
/* If the set changed, resubscribe live. */
|
||||
int changed = (new_followed.count != followed.count);
|
||||
if (!changed) {
|
||||
for (int i = 0; i < followed.count; i++) {
|
||||
if (!cr_pubkey_set_contains(&new_followed, followed.items[i])) {
|
||||
changed = 1; break;
|
||||
}
|
||||
}
|
||||
}
|
||||
cr_pubkey_set_free(&followed);
|
||||
followed = new_followed;
|
||||
/* Sync the refreshed followed set to the DB so newly added
|
||||
* follows get backfilled and dropped follows stop. Existing
|
||||
* cursor/completion state is preserved (upsert only touches
|
||||
* is_root and last_seen). */
|
||||
if (pg_conn) {
|
||||
const char **root_pks = malloc(cfg.root_npub_count * sizeof(char *));
|
||||
for (int i = 0; i < cfg.root_npub_count; i++)
|
||||
root_pks[i] = cfg.root_hex[i];
|
||||
pg_inbox_sync_followed_pubkeys((const char **)followed.items,
|
||||
followed.count, root_pks,
|
||||
cfg.root_npub_count);
|
||||
free(root_pks);
|
||||
/* Create relay progress rows for newly added follows.
|
||||
* Existing rows are preserved (ON CONFLICT DO NOTHING). */
|
||||
init_relay_progress_for_all(&cfg, &relay_map, &followed);
|
||||
}
|
||||
if (changed && cfg.live.enabled) {
|
||||
cr_live_resubscribe(&live, &cfg, upstream, &followed, &sink);
|
||||
}
|
||||
/* Reset the backfill in_progress flag in case we had entered
|
||||
* steady-state; new follows need to be drained. The backfill
|
||||
* tick will quickly re-enter steady-state if there are no
|
||||
* incomplete authors, so this is cheap. */
|
||||
bf.in_progress = 1;
|
||||
} else {
|
||||
DEBUG_WARN("follow graph refresh failed");
|
||||
cr_pubkey_set_free(&new_followed);
|
||||
}
|
||||
last_follow_refresh = now;
|
||||
}
|
||||
|
||||
/* Periodic live resubscribe. */
|
||||
if (cfg.live.enabled && cfg.live.resubscribe_interval_seconds > 0 &&
|
||||
(now - live.last_resubscribe) >= cfg.live.resubscribe_interval_seconds) {
|
||||
cr_live_resubscribe(&live, &cfg, upstream, &followed, &sink);
|
||||
}
|
||||
|
||||
/* Periodic state save (in case backfill didn't just save). */
|
||||
if ((now - last_state_save) >= 60) {
|
||||
if (!pg_conn) cr_config_save_state(&cfg);
|
||||
last_state_save = now;
|
||||
}
|
||||
|
||||
/* PostgreSQL: periodic status heartbeat. */
|
||||
if (pg_conn && (now - last_status_heartbeat) >= 15) {
|
||||
long events_fetched = live.events_received + bf.events_total;
|
||||
long inbox_inserts = sink.published_ok;
|
||||
int connected = 0;
|
||||
{
|
||||
char **listed = NULL;
|
||||
nostr_pool_relay_status_t *statuses = NULL;
|
||||
int n = nostr_relay_pool_list_relays(upstream, &listed, &statuses);
|
||||
for (int j = 0; j < n; j++) {
|
||||
if (statuses[j] == NOSTR_POOL_RELAY_CONNECTED) connected++;
|
||||
}
|
||||
free(listed);
|
||||
free(statuses);
|
||||
}
|
||||
int bf_complete = 0, bf_total = 0;
|
||||
pg_inbox_count_backfill_progress(&bf_complete, &bf_total);
|
||||
pg_inbox_update_status("running", config_generation, (long)now,
|
||||
followed.count, relay_map.selected_count,
|
||||
connected, bf_complete, bf_total,
|
||||
events_fetched, inbox_inserts, NULL, 0);
|
||||
last_status_heartbeat = now;
|
||||
}
|
||||
}
|
||||
|
||||
/* Graceful shutdown. */
|
||||
shutdown:
|
||||
DEBUG_INFO("shutting down...");
|
||||
cr_live_close(&live);
|
||||
if (!pg_conn) cr_config_save_state(&cfg);
|
||||
if (pg_conn) {
|
||||
int bf_complete = 0, bf_total = 0;
|
||||
pg_inbox_count_backfill_progress(&bf_complete, &bf_total);
|
||||
pg_inbox_update_status("stopped", config_generation, (long)time(NULL),
|
||||
followed.count, relay_map.selected_count, 0,
|
||||
bf_complete, bf_total,
|
||||
live.events_received + bf.events_total,
|
||||
sink.published_ok, NULL, 0);
|
||||
}
|
||||
cr_sink_destroy(&sink);
|
||||
nostr_relay_pool_destroy(upstream);
|
||||
cr_pubkey_set_free(&followed);
|
||||
cr_relay_map_free(&relay_map);
|
||||
cr_config_free(&cfg);
|
||||
if (pg_conn) pg_inbox_shutdown();
|
||||
nostr_crypto_cleanup();
|
||||
DEBUG_INFO("clean exit");
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
/*
|
||||
* caching_relay - main header
|
||||
*
|
||||
* Version information is auto-updated by increment_and_push.sh.
|
||||
* Git tags are the source of truth for versioning.
|
||||
*/
|
||||
#ifndef CACHING_RELAY_MAIN_H
|
||||
#define CACHING_RELAY_MAIN_H
|
||||
|
||||
// Version information (auto-updated by increment_and_push.sh)
|
||||
#define CR_VERSION_MAJOR 0
|
||||
#define CR_VERSION_MINOR 0
|
||||
#define CR_VERSION_PATCH 2
|
||||
#define CR_VERSION "v0.0.2"
|
||||
|
||||
#endif /* CACHING_RELAY_MAIN_H */
|
||||
@@ -0,0 +1,218 @@
|
||||
/*
|
||||
* caching_relay - read caching configuration from the c-relay-pg PostgreSQL
|
||||
* config table and populate the existing cr_config_t structure.
|
||||
*/
|
||||
#define _GNU_SOURCE
|
||||
#include "pg_config.h"
|
||||
#include "pg_inbox.h"
|
||||
#include "debug.h"
|
||||
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
/* ------------------------------------------------------------------ */
|
||||
/* Helpers */
|
||||
/* ------------------------------------------------------------------ */
|
||||
|
||||
/* Split a comma-separated string into a string array (fixed-size rows).
|
||||
* Trims leading/trailing whitespace from each token. */
|
||||
static void split_csv_str(const char *s, char *dst, int max, int elem_len,
|
||||
int *count_out) {
|
||||
int count = 0;
|
||||
if (!s) { *count_out = 0; return; }
|
||||
|
||||
/* Work on a mutable copy because strtok_r modifies its input. */
|
||||
char *copy = strdup(s);
|
||||
if (!copy) { *count_out = 0; return; }
|
||||
|
||||
char *saveptr = NULL;
|
||||
char *tok = strtok_r(copy, ",", &saveptr);
|
||||
while (tok && count < max) {
|
||||
/* Trim leading spaces. */
|
||||
while (*tok == ' ' || *tok == '\t') tok++;
|
||||
/* Trim trailing spaces. */
|
||||
char *end = tok + strlen(tok) - 1;
|
||||
while (end > tok && (*end == ' ' || *end == '\t' || *end == '\r' || *end == '\n')) {
|
||||
*end-- = '\0';
|
||||
}
|
||||
if (*tok != '\0') {
|
||||
strncpy(dst + count * elem_len, tok, elem_len - 1);
|
||||
dst[count * elem_len + (elem_len - 1)] = '\0';
|
||||
count++;
|
||||
}
|
||||
tok = strtok_r(NULL, ",", &saveptr);
|
||||
}
|
||||
free(copy);
|
||||
*count_out = count;
|
||||
}
|
||||
|
||||
/* Split a comma-separated string of integers into an int array. */
|
||||
static void split_csv_int(const char *s, int *dst, int max, int *count_out) {
|
||||
int count = 0;
|
||||
if (!s) { *count_out = 0; return; }
|
||||
|
||||
char *copy = strdup(s);
|
||||
if (!copy) { *count_out = 0; return; }
|
||||
|
||||
char *saveptr = NULL;
|
||||
char *tok = strtok_r(copy, ",", &saveptr);
|
||||
while (tok && count < max) {
|
||||
while (*tok == ' ' || *tok == '\t') tok++;
|
||||
if (*tok != '\0') {
|
||||
dst[count++] = atoi(tok);
|
||||
}
|
||||
tok = strtok_r(NULL, ",", &saveptr);
|
||||
}
|
||||
free(copy);
|
||||
*count_out = count;
|
||||
}
|
||||
|
||||
/* Split a comma-separated string of longs into a long array. */
|
||||
static void split_csv_long(const char *s, long *dst, int max, int *count_out) {
|
||||
int count = 0;
|
||||
if (!s) { *count_out = 0; return; }
|
||||
|
||||
char *copy = strdup(s);
|
||||
if (!copy) { *count_out = 0; return; }
|
||||
|
||||
char *saveptr = NULL;
|
||||
char *tok = strtok_r(copy, ",", &saveptr);
|
||||
while (tok && count < max) {
|
||||
while (*tok == ' ' || *tok == '\t') tok++;
|
||||
if (*tok != '\0') {
|
||||
dst[count++] = strtol(tok, NULL, 10);
|
||||
}
|
||||
tok = strtok_r(NULL, ",", &saveptr);
|
||||
}
|
||||
free(copy);
|
||||
*count_out = count;
|
||||
}
|
||||
|
||||
/* Read a config key as a freshly-allocated string. Returns NULL if missing. */
|
||||
static char *get_key(const char *key) {
|
||||
return pg_inbox_get_config_value(key);
|
||||
}
|
||||
|
||||
/* Read a config key as a boolean ("true"/"1"/"yes" -> 1). */
|
||||
static int get_bool(const char *key, int default_val) {
|
||||
char *v = get_key(key);
|
||||
if (!v) return default_val;
|
||||
int r = (strcmp(v, "true") == 0 || strcmp(v, "1") == 0 ||
|
||||
strcmp(v, "yes") == 0 || strcmp(v, "on") == 0);
|
||||
free(v);
|
||||
return r;
|
||||
}
|
||||
|
||||
/* Read a config key as a long. */
|
||||
static long get_long(const char *key, long default_val) {
|
||||
char *v = get_key(key);
|
||||
if (!v) return default_val;
|
||||
long r = strtol(v, NULL, 10);
|
||||
free(v);
|
||||
return r;
|
||||
}
|
||||
|
||||
/* Read a config key as an int. */
|
||||
static int get_int(const char *key, int default_val) {
|
||||
return (int)get_long(key, (long)default_val);
|
||||
}
|
||||
|
||||
/* ------------------------------------------------------------------ */
|
||||
/* Public API */
|
||||
/* ------------------------------------------------------------------ */
|
||||
|
||||
int pg_config_load(cr_config_t *cfg) {
|
||||
if (!cfg) return -1;
|
||||
memset(cfg, 0, sizeof(*cfg));
|
||||
|
||||
/* root_npubs */
|
||||
char *npubs = get_key("caching_root_npubs");
|
||||
split_csv_str(npubs, (char *)cfg->root_npubs, CR_MAX_ROOT_NPUBS,
|
||||
CR_NPUB_LEN, &cfg->root_npub_count);
|
||||
free(npubs);
|
||||
|
||||
/* bootstrap_relays -> upstream_relays */
|
||||
char *relays = get_key("caching_bootstrap_relays");
|
||||
split_csv_str(relays, (char *)cfg->upstream_relays, CR_MAX_UPSTREAM,
|
||||
CR_URL_LEN, &cfg->upstream_count);
|
||||
free(relays);
|
||||
|
||||
/* local_relay - no longer required for PostgreSQL mode, but keep a
|
||||
* placeholder for compatibility. The caller may override it. */
|
||||
cfg->local_relay[0] = '\0';
|
||||
|
||||
/* kinds */
|
||||
char *kinds = get_key("caching_kinds");
|
||||
split_csv_int(kinds, cfg->kinds, CR_MAX_KINDS, &cfg->kind_count);
|
||||
free(kinds);
|
||||
|
||||
/* admin_kinds - supports "*" to mean all kinds */
|
||||
char *akinds = get_key("caching_admin_kinds");
|
||||
if (akinds) {
|
||||
/* Check for "*" anywhere in the list. */
|
||||
if (strstr(akinds, "*") != NULL) {
|
||||
cfg->admin_all_kinds = 1;
|
||||
} else {
|
||||
split_csv_int(akinds, cfg->admin_kinds, CR_MAX_KINDS,
|
||||
&cfg->admin_kind_count);
|
||||
}
|
||||
free(akinds);
|
||||
}
|
||||
|
||||
/* live */
|
||||
cfg->live.enabled = get_bool("caching_live_enabled", 1);
|
||||
cfg->live.resubscribe_interval_seconds =
|
||||
get_int("caching_live_resubscribe_seconds", 300);
|
||||
|
||||
/* backfill */
|
||||
cfg->backfill.enabled = get_bool("caching_backfill_enabled", 1);
|
||||
cfg->backfill.events_per_tick =
|
||||
get_int("caching_backfill_page_size", 500);
|
||||
/* tick interval is stored in ms in the config table; convert to seconds. */
|
||||
int tick_ms = get_int("caching_backfill_tick_interval_ms", 5000);
|
||||
cfg->backfill.tick_interval_seconds = (tick_ms + 999) / 1000;
|
||||
if (cfg->backfill.tick_interval_seconds < 1)
|
||||
cfg->backfill.tick_interval_seconds = 1;
|
||||
|
||||
/* follow graph refresh */
|
||||
cfg->follow_graph_refresh_seconds =
|
||||
get_int("caching_follow_graph_refresh_seconds", 600);
|
||||
|
||||
/* Defaults if missing. */
|
||||
if (cfg->backfill.events_per_tick == 0)
|
||||
cfg->backfill.events_per_tick = 500;
|
||||
if (cfg->backfill.tick_interval_seconds == 0)
|
||||
cfg->backfill.tick_interval_seconds = 5;
|
||||
if (cfg->live.resubscribe_interval_seconds == 0)
|
||||
cfg->live.resubscribe_interval_seconds = 300;
|
||||
if (cfg->follow_graph_refresh_seconds == 0)
|
||||
cfg->follow_graph_refresh_seconds = 600;
|
||||
|
||||
/* State: per-author backfill progress lives in caching_followed_pubkeys.
|
||||
* backfilled_until is only used as a first-time flag in legacy mode. */
|
||||
cfg->state.backfilled_until = 0;
|
||||
|
||||
/* Validation. */
|
||||
if (cfg->root_npub_count == 0) {
|
||||
DEBUG_ERROR("pg_config: at least one caching_root_npubs required");
|
||||
return -1;
|
||||
}
|
||||
if (cfg->upstream_count == 0) {
|
||||
DEBUG_ERROR("pg_config: at least one caching_bootstrap_relays required");
|
||||
return -1;
|
||||
}
|
||||
if (cfg->kind_count == 0) {
|
||||
DEBUG_ERROR("pg_config: at least one caching_kinds required");
|
||||
return -1;
|
||||
}
|
||||
|
||||
DEBUG_INFO("pg_config loaded: %d root npubs, %d upstream relays, %d kinds",
|
||||
cfg->root_npub_count, cfg->upstream_count, cfg->kind_count);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int pg_config_generation_changed(long last_generation) {
|
||||
long cur = pg_inbox_get_config_generation();
|
||||
if (cur < 0) return -1;
|
||||
return (cur != last_generation) ? 1 : 0;
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
/*
|
||||
* caching_relay - read caching configuration from the c-relay-pg PostgreSQL
|
||||
* config table and populate the existing cr_config_t structure.
|
||||
*/
|
||||
#ifndef CACHING_RELAY_PG_CONFIG_H
|
||||
#define CACHING_RELAY_PG_CONFIG_H
|
||||
|
||||
#include "config.h" /* cr_config_t */
|
||||
|
||||
/* Load caching configuration from the c-relay-pg PostgreSQL config table.
|
||||
* Populates the cr_config_t fields from the config table.
|
||||
* Returns 0 on success, -1 on error. */
|
||||
int pg_config_load(cr_config_t *cfg);
|
||||
|
||||
/* Check if the config generation has changed since the last load.
|
||||
* Returns 1 if changed, 0 if not, -1 on error. */
|
||||
int pg_config_generation_changed(long last_generation);
|
||||
|
||||
#endif /* CACHING_RELAY_PG_CONFIG_H */
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,152 @@
|
||||
/*
|
||||
* caching_relay - PostgreSQL inbox connection and operations.
|
||||
*
|
||||
* Manages the libpq connection to the c-relay-pg PostgreSQL database and
|
||||
* provides functions to insert fetched events into the caching_event_inbox
|
||||
* table, update the caching_service_state singleton, and read configuration
|
||||
* from the shared config table.
|
||||
*/
|
||||
#ifndef CACHING_RELAY_PG_INBOX_H
|
||||
#define CACHING_RELAY_PG_INBOX_H
|
||||
|
||||
#include <stddef.h>
|
||||
|
||||
/* Forward declaration so the header does not pull in cJSON everywhere. */
|
||||
typedef struct cJSON cJSON;
|
||||
|
||||
/* Initialize PostgreSQL connection using the given connection string.
|
||||
* Returns 0 on success, -1 on error. */
|
||||
int pg_inbox_init(const char *connection_string);
|
||||
|
||||
/* Close the PostgreSQL connection. */
|
||||
void pg_inbox_shutdown(void);
|
||||
|
||||
/* Insert an event into the caching_event_inbox table.
|
||||
* event_json is the raw Nostr event JSON string.
|
||||
* source_relay may be NULL. source_class is "live", "discovery", or "backfill".
|
||||
* priority is 0 (live/discovery) or 1 (backfill).
|
||||
* Returns 0 on success (including conflict-skip), -1 on error. */
|
||||
int pg_inbox_insert_event(const char *event_json, const char *source_relay,
|
||||
const char *source_class, int priority);
|
||||
|
||||
/* Update the caching_service_state singleton row.
|
||||
* All string parameters may be NULL (left unchanged).
|
||||
* Returns 0 on success, -1 on error. */
|
||||
int pg_inbox_update_status(const char *service_state,
|
||||
long config_generation,
|
||||
long heartbeat_at,
|
||||
int followed_author_count,
|
||||
int selected_relay_count,
|
||||
int connected_relay_count,
|
||||
int backfill_authors_complete,
|
||||
int backfill_authors_total,
|
||||
long events_fetched,
|
||||
long inbox_inserts,
|
||||
const char *last_error,
|
||||
long last_error_at);
|
||||
|
||||
/* Read a config value from the c-relay-pg config table.
|
||||
* Returns a malloc'd string (caller must free) or NULL if not found/error. */
|
||||
char *pg_inbox_get_config_value(const char *key);
|
||||
|
||||
/* Read the current config generation.
|
||||
* Returns the generation number, or -1 on error. */
|
||||
long pg_inbox_get_config_generation(void);
|
||||
|
||||
/* ------------------------------------------------------------------ */
|
||||
/* Per-author until-cursor drain backfill model */
|
||||
/* ------------------------------------------------------------------ */
|
||||
|
||||
/* Reset all backfill progress: set until_cursor=0 and backfill_complete=FALSE
|
||||
* for all rows in caching_followed_pubkeys (the followed set itself is
|
||||
* preserved). Used by the "Reset Backfill" API button and --restart.
|
||||
* Returns 0 on success, -1 on error. */
|
||||
int pg_inbox_reset_backfill_progress(void);
|
||||
|
||||
/* Upsert all followed pubkeys into caching_followed_pubkeys.
|
||||
* For each pubkey: INSERT if new (until_cursor=0, backfill_complete=FALSE),
|
||||
* or UPDATE is_root and last_seen if existing (until_cursor and
|
||||
* backfill_complete are NOT touched). root_pubkeys marks which entries are
|
||||
* root follows. Returns 0 on success, -1 on error. */
|
||||
int pg_inbox_sync_followed_pubkeys(const char **pubkeys, int count,
|
||||
const char **root_pubkeys, int root_count);
|
||||
|
||||
/* Pick the next incomplete author for backfill, round-robin.
|
||||
* On success fills out_pk, out_until_cursor and out_is_root, advances
|
||||
* *round_cursor, and returns 0. Returns -1 if no incomplete authors or
|
||||
* on error. */
|
||||
int pg_inbox_pick_next_backfill_author(char *out_pk, int pk_len,
|
||||
long *out_until_cursor,
|
||||
int *out_is_root,
|
||||
int *round_cursor);
|
||||
|
||||
/* Update the cursor and completion state for an author.
|
||||
* events_this_page is added to the running events_fetched counter.
|
||||
* Returns 0 on success, -1 on error. */
|
||||
int pg_inbox_update_backfill_progress(const char *pk, long until_cursor,
|
||||
int complete, int events_this_page);
|
||||
|
||||
/* One-time migration from the old caching_backfill_progress table to the
|
||||
* new caching_followed_pubkeys table. Only runs if the new table is empty
|
||||
* and the old table has rows. Returns 0 on success (including no-op),
|
||||
* -1 on error. */
|
||||
int pg_inbox_migrate_backfill_progress(void);
|
||||
|
||||
/* Count complete/total authors for status reporting.
|
||||
* Returns 0 on success, -1 on error. */
|
||||
int pg_inbox_count_backfill_progress(int *out_complete, int *out_total);
|
||||
|
||||
/* ------------------------------------------------------------------ */
|
||||
/* Per-relay backfill progress (relay-by-relay drain model) */
|
||||
/* ------------------------------------------------------------------ */
|
||||
|
||||
/* Initialize relay progress rows for an author. Creates one row per relay
|
||||
* URL with until_cursor=0, complete=FALSE. Skips rows that already exist
|
||||
* (ON CONFLICT DO NOTHING). Returns 0 on success, -1 on error. */
|
||||
int pg_inbox_init_relay_progress_for_author(const char *pk,
|
||||
const char **relay_urls,
|
||||
int relay_count);
|
||||
|
||||
/* Pick the next author that has at least one incomplete relay progress row.
|
||||
* Round-robin via *round_cursor. On success fills out_pk and returns 0.
|
||||
* Returns -1 if no incomplete authors or on error. */
|
||||
int pg_inbox_pick_next_author_with_incomplete_relays(char *out_pk, int pk_len,
|
||||
int *round_cursor);
|
||||
|
||||
/* Get incomplete relays for an author. Returns a cJSON array of
|
||||
* {relay_url, until_cursor} objects (caller frees with cJSON_Delete).
|
||||
* Returns NULL on error or if no incomplete relays. */
|
||||
cJSON *pg_inbox_get_incomplete_relays(const char *pk);
|
||||
|
||||
/* Update relay progress: advance cursor and/or mark complete.
|
||||
* events_this_page is added to the running events_fetched counter.
|
||||
* last_status is a short string like "eose", "timeout", "error", or NULL
|
||||
* to leave unchanged. Returns 0 on success, -1 on error. */
|
||||
int pg_inbox_update_relay_progress(const char *pk, const char *relay_url,
|
||||
long until_cursor, int complete,
|
||||
int events_this_page,
|
||||
const char *last_status);
|
||||
|
||||
/* Check if all relays for an author are complete. If so, mark the author
|
||||
* as backfill_complete=TRUE in caching_followed_pubkeys.
|
||||
* Returns 1 if marked complete, 0 if still incomplete, -1 on error. */
|
||||
int pg_inbox_check_and_mark_author_complete(const char *pk);
|
||||
|
||||
/* ------------------------------------------------------------------ */
|
||||
/* Active backfill target (for UI status display) */
|
||||
/* ------------------------------------------------------------------ */
|
||||
|
||||
/* Set the currently-active backfill target (pubkey + relay being queried).
|
||||
* Either parameter may be NULL/empty to clear. Creates the
|
||||
* caching_backfill_active table if needed (single-row singleton).
|
||||
* Returns 0 on success, -1 on error. */
|
||||
int pg_inbox_set_active_target(const char *pubkey, const char *relay_url);
|
||||
|
||||
/* Read the active backfill target. Fills out_pubkey/out_relay with
|
||||
* the current values (or empty strings if none). Buffers must be
|
||||
* at least 65 and 256 bytes respectively.
|
||||
* Returns 0 on success, -1 on error. */
|
||||
int pg_inbox_get_active_target(char *out_pubkey, int pk_len,
|
||||
char *out_relay, int relay_len);
|
||||
|
||||
#endif /* CACHING_RELAY_PG_INBOX_H */
|
||||
@@ -0,0 +1,471 @@
|
||||
/*
|
||||
* caching_relay - NIP-65 outbox relay discovery
|
||||
*/
|
||||
#define _GNU_SOURCE
|
||||
#include "relay_discovery.h"
|
||||
#include "debug.h"
|
||||
|
||||
#include <string.h>
|
||||
#include <stdlib.h>
|
||||
#include <signal.h>
|
||||
|
||||
/* Access the shutdown flag from main.c so we can abort discovery early. */
|
||||
extern volatile sig_atomic_t g_shutdown;
|
||||
|
||||
/* Normalize a relay URL: strip trailing slash for consistency. */
|
||||
static void normalize_url(char *url, size_t maxlen) {
|
||||
size_t len = strlen(url);
|
||||
while (len > 0 && url[len - 1] == '/') {
|
||||
url[--len] = '\0';
|
||||
}
|
||||
(void)maxlen;
|
||||
}
|
||||
|
||||
/* ---- helpers ---- */
|
||||
|
||||
static int get_pool_urls(nostr_relay_pool_t *pool, const char ***urls_out) {
|
||||
char **listed = NULL;
|
||||
nostr_pool_relay_status_t *statuses = NULL;
|
||||
int n = nostr_relay_pool_list_relays(pool, &listed, &statuses);
|
||||
const char **urls = NULL;
|
||||
if (n > 0) {
|
||||
urls = malloc(n * sizeof(char *));
|
||||
for (int j = 0; j < n; j++) urls[j] = listed[j];
|
||||
}
|
||||
free(listed);
|
||||
free(statuses);
|
||||
*urls_out = urls;
|
||||
return n;
|
||||
}
|
||||
|
||||
/* Query the most recent kind 10002 for a single pubkey from a pool.
|
||||
* Returns the event (caller must cJSON_Delete) or NULL. */
|
||||
static cJSON *query_kind10002(nostr_relay_pool_t *pool, const char *hex) {
|
||||
cJSON *filter = cJSON_CreateObject();
|
||||
cJSON *authors = cJSON_CreateArray();
|
||||
cJSON_AddItemToArray(authors, cJSON_CreateString(hex));
|
||||
cJSON_AddItemToObject(filter, "authors", authors);
|
||||
cJSON *kinds = cJSON_CreateArray();
|
||||
cJSON_AddItemToArray(kinds, cJSON_CreateNumber(10002));
|
||||
cJSON_AddItemToObject(filter, "kinds", kinds);
|
||||
cJSON_AddItemToObject(filter, "limit", cJSON_CreateNumber(1));
|
||||
|
||||
const char **urls = NULL;
|
||||
int n = get_pool_urls(pool, &urls);
|
||||
int ev_count = 0;
|
||||
cJSON **events = nostr_relay_pool_query_sync(pool, urls, n, filter, &ev_count, 10000);
|
||||
free(urls);
|
||||
cJSON_Delete(filter);
|
||||
|
||||
cJSON *result = NULL;
|
||||
if (events && ev_count > 0) {
|
||||
result = events[0]; /* take first (most recent) */
|
||||
for (int k = 1; k < ev_count; k++) cJSON_Delete(events[k]);
|
||||
}
|
||||
free(events);
|
||||
return result;
|
||||
}
|
||||
|
||||
/* Parse "r" tags from a kind 10002 event into an outbox entry.
|
||||
* Only includes relays with "read" marker or no marker (assume both).
|
||||
* Skips "write" only relays. */
|
||||
static int parse_r_tags(cJSON *event, cr_outbox_entry_t *entry) {
|
||||
memset(entry, 0, sizeof(*entry));
|
||||
cJSON *pubkey = cJSON_GetObjectItem(event, "pubkey");
|
||||
if (pubkey && cJSON_IsString(pubkey)) {
|
||||
strncpy(entry->pubkey, cJSON_GetStringValue(pubkey), CR_HEX_LEN - 1);
|
||||
}
|
||||
|
||||
cJSON *tags = cJSON_GetObjectItem(event, "tags");
|
||||
if (!tags || !cJSON_IsArray(tags)) return 0;
|
||||
|
||||
int added = 0;
|
||||
cJSON *tag;
|
||||
cJSON_ArrayForEach(tag, tags) {
|
||||
if (!cJSON_IsArray(tag)) continue;
|
||||
cJSON *name = cJSON_GetArrayItem(tag, 0);
|
||||
if (!name || !cJSON_IsString(name)) continue;
|
||||
if (strcmp(cJSON_GetStringValue(name), "r") != 0) continue;
|
||||
cJSON *url = cJSON_GetArrayItem(tag, 1);
|
||||
if (!url || !cJSON_IsString(url)) continue;
|
||||
const char *relay_url = cJSON_GetStringValue(url);
|
||||
if (!relay_url || !relay_url[0]) continue;
|
||||
|
||||
/* Check marker (3rd element): "read", "write", or absent. */
|
||||
cJSON *marker = cJSON_GetArrayItem(tag, 2);
|
||||
if (marker && cJSON_IsString(marker)) {
|
||||
const char *m = cJSON_GetStringValue(marker);
|
||||
if (strcmp(m, "write") == 0) continue; /* skip write-only relays */
|
||||
}
|
||||
/* No marker or "read" => include. */
|
||||
|
||||
if (entry->relay_count >= CR_MAX_RELAYS_PER_PUBKEY) break;
|
||||
strncpy(entry->relays[entry->relay_count], relay_url, CR_URL_LEN - 1);
|
||||
entry->relays[entry->relay_count][CR_URL_LEN - 1] = '\0';
|
||||
normalize_url(entry->relays[entry->relay_count], CR_URL_LEN);
|
||||
entry->relay_count++;
|
||||
added++;
|
||||
}
|
||||
return added;
|
||||
}
|
||||
|
||||
/* ---- greedy set cover ---- */
|
||||
|
||||
/* Build relay -> pubkey coverage map and compute minimum covering set. */
|
||||
static void compute_covering_set(cr_relay_map_t *map, cr_config_t *cfg,
|
||||
const cr_pubkey_set_t *followed) {
|
||||
/* Build a temporary relay -> pubkeys map. */
|
||||
typedef struct {
|
||||
char url[CR_URL_LEN];
|
||||
char pubkeys[CR_MAX_PUBKEYS_PER_RELAY][CR_HEX_LEN];
|
||||
int pubkey_count;
|
||||
} relay_cov_t;
|
||||
|
||||
relay_cov_t *relays = calloc(CR_MAX_DISCOVERED_RELAYS, sizeof(relay_cov_t));
|
||||
int relay_count = 0;
|
||||
|
||||
/* Always include bootstrap relays in the coverage map. */
|
||||
for (int i = 0; i < cfg->upstream_count && relay_count < CR_MAX_DISCOVERED_RELAYS; i++) {
|
||||
strncpy(relays[relay_count].url, cfg->upstream_relays[i], CR_URL_LEN - 1);
|
||||
relays[relay_count].url[CR_URL_LEN - 1] = '\0';
|
||||
normalize_url(relays[relay_count].url, CR_URL_LEN);
|
||||
relay_count++;
|
||||
}
|
||||
|
||||
/* Add outbox relays and build coverage. */
|
||||
for (int i = 0; i < map->outbox_count; i++) {
|
||||
cr_outbox_entry_t *oe = &map->outboxes[i];
|
||||
for (int r = 0; r < oe->relay_count; r++) {
|
||||
/* Find or create relay entry. */
|
||||
int found = -1;
|
||||
for (int j = 0; j < relay_count; j++) {
|
||||
if (strcmp(relays[j].url, oe->relays[r]) == 0) { found = j; break; }
|
||||
}
|
||||
if (found < 0) {
|
||||
if (relay_count >= CR_MAX_DISCOVERED_RELAYS) break;
|
||||
strncpy(relays[relay_count].url, oe->relays[r], CR_URL_LEN - 1);
|
||||
found = relay_count++;
|
||||
}
|
||||
/* Add this pubkey to the relay's coverage. */
|
||||
if (relays[found].pubkey_count < CR_MAX_PUBKEYS_PER_RELAY) {
|
||||
strncpy(relays[found].pubkeys[relays[found].pubkey_count],
|
||||
oe->pubkey, CR_HEX_LEN - 1);
|
||||
relays[found].pubkey_count++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* Greedy set cover. */
|
||||
char *covered = calloc(followed->count, 1); /* 1 if pubkey is covered */
|
||||
map->selected_count = 0;
|
||||
map->coverage = calloc(map->outbox_count, sizeof(int));
|
||||
for (int i = 0; i < map->outbox_count; i++) map->coverage[i] = -1;
|
||||
|
||||
int uncovered_count = followed->count;
|
||||
|
||||
/* Helper: find index of a pubkey in the followed set. */
|
||||
/* (linear search, fine for our scale) */
|
||||
|
||||
while (uncovered_count > 0 && map->selected_count < CR_MAX_DISCOVERED_RELAYS) {
|
||||
int best_relay = -1;
|
||||
int best_count = 0;
|
||||
|
||||
for (int r = 0; r < relay_count; r++) {
|
||||
/* Count how many uncovered pubkeys this relay covers. */
|
||||
int count = 0;
|
||||
for (int p = 0; p < relays[r].pubkey_count; p++) {
|
||||
/* Find this pubkey in followed set. */
|
||||
for (int i = 0; i < followed->count; i++) {
|
||||
if (strcmp(followed->items[i], relays[r].pubkeys[p]) == 0) {
|
||||
if (!covered[i]) count++;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (count > best_count) {
|
||||
best_count = count;
|
||||
best_relay = r;
|
||||
}
|
||||
}
|
||||
|
||||
if (best_relay < 0 || best_count == 0) break; /* no more coverage possible */
|
||||
|
||||
/* Select this relay. */
|
||||
strncpy(map->selected_relays[map->selected_count],
|
||||
relays[best_relay].url, CR_URL_LEN - 1);
|
||||
map->selected_count++;
|
||||
|
||||
/* Mark covered pubkeys and record coverage. */
|
||||
for (int p = 0; p < relays[best_relay].pubkey_count; p++) {
|
||||
for (int i = 0; i < followed->count; i++) {
|
||||
if (strcmp(followed->items[i], relays[best_relay].pubkeys[p]) == 0) {
|
||||
if (!covered[i]) {
|
||||
covered[i] = 1;
|
||||
uncovered_count--;
|
||||
/* Record which selected relay covers this outbox entry. */
|
||||
for (int oe_i = 0; oe_i < map->outbox_count; oe_i++) {
|
||||
if (strcmp(map->outboxes[oe_i].pubkey,
|
||||
relays[best_relay].pubkeys[p]) == 0) {
|
||||
if (map->coverage[oe_i] < 0) {
|
||||
map->coverage[oe_i] = map->selected_count - 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* Log results. */
|
||||
DEBUG_INFO("relay_discovery: selected %d relays to cover %d/%d pubkeys",
|
||||
map->selected_count, followed->count - uncovered_count, followed->count);
|
||||
for (int i = 0; i < map->selected_count; i++) {
|
||||
DEBUG_INFO(" relay[%d]: %s", i, map->selected_relays[i]);
|
||||
}
|
||||
if (uncovered_count > 0) {
|
||||
DEBUG_WARN("relay_discovery: %d pubkeys have no outbox relays (covered by bootstrap)",
|
||||
uncovered_count);
|
||||
}
|
||||
|
||||
free(covered);
|
||||
free(relays);
|
||||
}
|
||||
|
||||
/* ---- batched kind-10002 query ---- */
|
||||
|
||||
/* Query kind 10002 for a batch of up to 100 pubkeys from a pool.
|
||||
* Returns a cJSON array of events (caller must cJSON_Delete each + free).
|
||||
* Sets *out_count to the number of events returned. */
|
||||
static cJSON **query_kind10002_batch(nostr_relay_pool_t *pool,
|
||||
const char **pubkeys, int pubkey_count,
|
||||
int *out_count) {
|
||||
cJSON *filter = cJSON_CreateObject();
|
||||
cJSON *authors = cJSON_CreateArray();
|
||||
for (int i = 0; i < pubkey_count; i++)
|
||||
cJSON_AddItemToArray(authors, cJSON_CreateString(pubkeys[i]));
|
||||
cJSON_AddItemToObject(filter, "authors", authors);
|
||||
cJSON *kinds = cJSON_CreateArray();
|
||||
cJSON_AddItemToArray(kinds, cJSON_CreateNumber(10002));
|
||||
cJSON_AddItemToObject(filter, "kinds", kinds);
|
||||
|
||||
const char **urls = NULL;
|
||||
int n = get_pool_urls(pool, &urls);
|
||||
|
||||
/* TRACE: log the query being sent. */
|
||||
if (g_debug_level >= DEBUG_LEVEL_TRACE) {
|
||||
char *filter_str = cJSON_PrintUnformatted(filter);
|
||||
DEBUG_TRACE("query_kind10002_batch: querying %d relays for %d pubkeys", n, pubkey_count);
|
||||
if (filter_str) {
|
||||
/* Only print first 200 chars of filter to avoid huge output */
|
||||
if (strlen(filter_str) > 200) filter_str[200] = '\0';
|
||||
DEBUG_TRACE(" filter: %s...", filter_str);
|
||||
free(filter_str);
|
||||
}
|
||||
}
|
||||
|
||||
int ev_count = 0;
|
||||
cJSON **events = nostr_relay_pool_query_sync(pool, urls, n, filter, &ev_count, 30000);
|
||||
free(urls);
|
||||
cJSON_Delete(filter);
|
||||
|
||||
DEBUG_TRACE("query_kind10002_batch: returned %d events", ev_count);
|
||||
|
||||
*out_count = ev_count;
|
||||
return events;
|
||||
}
|
||||
|
||||
/* Process a batch of kind-10002 events: parse r-tags, populate outbox entries,
|
||||
* publish to local relay. Returns count of events processed. */
|
||||
static int process_10002_batch(cJSON **events, int ev_count,
|
||||
cr_relay_map_t *map, cr_sink_t *sink,
|
||||
const char *source) {
|
||||
int processed = 0;
|
||||
for (int k = 0; k < ev_count; k++) {
|
||||
cJSON *event = events[k];
|
||||
if (!event) continue;
|
||||
|
||||
/* Get pubkey from event. */
|
||||
cJSON *pk = cJSON_GetObjectItem(event, "pubkey");
|
||||
if (!pk || !cJSON_IsString(pk)) { cJSON_Delete(event); continue; }
|
||||
const char *hex = cJSON_GetStringValue(pk);
|
||||
|
||||
/* Find or create outbox entry for this pubkey. */
|
||||
cr_outbox_entry_t *oe = NULL;
|
||||
for (int i = 0; i < map->outbox_count; i++) {
|
||||
if (strcmp(map->outboxes[i].pubkey, hex) == 0) { oe = &map->outboxes[i]; break; }
|
||||
}
|
||||
if (!oe) {
|
||||
if (map->outbox_count >= /* followed->count passed in via map size */ 99999) {
|
||||
cJSON_Delete(event); continue;
|
||||
}
|
||||
oe = &map->outboxes[map->outbox_count];
|
||||
strncpy(oe->pubkey, hex, CR_HEX_LEN - 1);
|
||||
map->outbox_count++;
|
||||
}
|
||||
|
||||
int n = parse_r_tags(event, oe);
|
||||
(void)n;
|
||||
|
||||
if (strcmp(source, "bootstrap") == 0) {
|
||||
cr_sink_publish(sink, event);
|
||||
/* Pump sink pool every 16 events to avoid pending publish overflow
|
||||
* (NOSTR_POOL_MAX_PENDING_PUBLISHES = 32). */
|
||||
if (processed > 0 && (processed % 16) == 0) {
|
||||
cr_sink_pump(sink, 200);
|
||||
}
|
||||
}
|
||||
|
||||
processed++;
|
||||
cJSON_Delete(event);
|
||||
}
|
||||
free(events);
|
||||
|
||||
/* Log sink stats after each batch. */
|
||||
DEBUG_INFO("relay_discovery: batch processed %d events (sink: ok=%ld reject=%ld dup=%ld err=%ld)",
|
||||
processed, sink->published_ok, sink->published_failed,
|
||||
sink->published_dup, 0L);
|
||||
|
||||
return processed;
|
||||
}
|
||||
|
||||
/* ---- main discovery function ---- */
|
||||
|
||||
#define CR_10002_BATCH_SIZE 100
|
||||
|
||||
int cr_relay_discovery_run(cr_relay_map_t *map, cr_config_t *cfg,
|
||||
nostr_relay_pool_t *upstream, cr_sink_t *sink,
|
||||
const cr_pubkey_set_t *followed, int is_first_time) {
|
||||
memset(map, 0, sizeof(*map));
|
||||
map->outboxes = calloc(followed->count, sizeof(cr_outbox_entry_t));
|
||||
if (!map->outboxes) return -1;
|
||||
map->outbox_count = 0;
|
||||
|
||||
DEBUG_INFO("relay_discovery: discovering outbox relays for %d pubkeys (%s)",
|
||||
followed->count, is_first_time ? "first-time" : "local-first");
|
||||
|
||||
int found_local = 0, found_bootstrap = 0, found_none = 0;
|
||||
|
||||
/* Pre-populate outbox entries for all followed pubkeys (empty, for coverage
|
||||
* tracking of pubkeys with no 10002). */
|
||||
for (int i = 0; i < followed->count; i++) {
|
||||
strncpy(map->outboxes[i].pubkey, followed->items[i], CR_HEX_LEN - 1);
|
||||
}
|
||||
map->outbox_count = followed->count;
|
||||
|
||||
/* Track which pubkeys we've found 10002 for. */
|
||||
char *found = calloc(followed->count, 1);
|
||||
|
||||
/* Phase 1: If subsequent startup, batch-query local relay first. */
|
||||
if (!is_first_time && sink && sink->pool) {
|
||||
DEBUG_INFO("relay_discovery: querying local relay for kind-10002 (batched)...");
|
||||
|
||||
for (int start = 0; start < followed->count && !g_shutdown; start += CR_10002_BATCH_SIZE) {
|
||||
int batch = followed->count - start;
|
||||
if (batch > CR_10002_BATCH_SIZE) batch = CR_10002_BATCH_SIZE;
|
||||
|
||||
int ev_count = 0;
|
||||
cJSON **events = query_kind10002_batch(sink->pool,
|
||||
(const char **)&followed->items[start],
|
||||
batch, &ev_count);
|
||||
if (events && ev_count > 0) {
|
||||
for (int k = 0; k < ev_count; k++) {
|
||||
cJSON *pk = cJSON_GetObjectItem(events[k], "pubkey");
|
||||
if (pk && cJSON_IsString(pk)) {
|
||||
const char *hex = cJSON_GetStringValue(pk);
|
||||
for (int i = 0; i < followed->count; i++) {
|
||||
if (strcmp(followed->items[i], hex) == 0) {
|
||||
if (!found[i]) { found[i] = 1; found_local++; }
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
process_10002_batch(events, ev_count, map, sink, "local");
|
||||
/* Pump sink pool to flush published events. */
|
||||
cr_sink_pump(sink, 500);
|
||||
} else {
|
||||
free(events);
|
||||
}
|
||||
}
|
||||
DEBUG_INFO("relay_discovery: local relay provided %d kind-10002 events", found_local);
|
||||
}
|
||||
|
||||
/* Phase 2: Batch-query bootstrap relays for pubkeys not found locally. */
|
||||
int remaining = 0;
|
||||
for (int i = 0; i < followed->count; i++) if (!found[i]) remaining++;
|
||||
|
||||
if (remaining > 0) {
|
||||
DEBUG_INFO("relay_discovery: querying bootstrap relays for %d remaining pubkeys (batched)...",
|
||||
remaining);
|
||||
|
||||
/* Build list of remaining pubkeys. */
|
||||
const char **remaining_pk = malloc(remaining * sizeof(char *));
|
||||
int ridx = 0;
|
||||
for (int i = 0; i < followed->count; i++) {
|
||||
if (!found[i]) remaining_pk[ridx++] = followed->items[i];
|
||||
}
|
||||
|
||||
for (int start = 0; start < remaining && !g_shutdown; start += CR_10002_BATCH_SIZE) {
|
||||
int batch = remaining - start;
|
||||
if (batch > CR_10002_BATCH_SIZE) batch = CR_10002_BATCH_SIZE;
|
||||
|
||||
int ev_count = 0;
|
||||
cJSON **events = query_kind10002_batch(upstream,
|
||||
&remaining_pk[start],
|
||||
batch, &ev_count);
|
||||
if (events && ev_count > 0) {
|
||||
for (int k = 0; k < ev_count; k++) {
|
||||
cJSON *pk = cJSON_GetObjectItem(events[k], "pubkey");
|
||||
if (pk && cJSON_IsString(pk)) {
|
||||
const char *hex = cJSON_GetStringValue(pk);
|
||||
for (int i = 0; i < followed->count; i++) {
|
||||
if (strcmp(followed->items[i], hex) == 0) {
|
||||
if (!found[i]) { found[i] = 1; found_bootstrap++; }
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
process_10002_batch(events, ev_count, map, sink, "bootstrap");
|
||||
/* Pump sink pool to flush published events. */
|
||||
cr_sink_pump(sink, 500);
|
||||
} else {
|
||||
free(events);
|
||||
}
|
||||
}
|
||||
free(remaining_pk);
|
||||
}
|
||||
|
||||
found_none = followed->count - found_local - found_bootstrap;
|
||||
DEBUG_INFO("relay_discovery: %d local, %d bootstrap, %d none",
|
||||
found_local, found_bootstrap, found_none);
|
||||
|
||||
free(found);
|
||||
|
||||
/* Compute minimum covering set. */
|
||||
compute_covering_set(map, cfg, followed);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
void cr_relay_map_free(cr_relay_map_t *map) {
|
||||
if (map->outboxes) {
|
||||
free(map->outboxes);
|
||||
map->outboxes = NULL;
|
||||
}
|
||||
if (map->coverage) {
|
||||
free(map->coverage);
|
||||
map->coverage = NULL;
|
||||
}
|
||||
map->outbox_count = 0;
|
||||
map->selected_count = 0;
|
||||
}
|
||||
|
||||
const cr_outbox_entry_t *cr_relay_map_get_outbox(const cr_relay_map_t *map, const char *hex) {
|
||||
for (int i = 0; i < map->outbox_count; i++) {
|
||||
if (strcmp(map->outboxes[i].pubkey, hex) == 0) return &map->outboxes[i];
|
||||
}
|
||||
return NULL;
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
/*
|
||||
* caching_relay - NIP-65 outbox relay discovery.
|
||||
*
|
||||
* For each followed pubkey, fetches their kind 10002 (relay list) and parses
|
||||
* the "r" tags to build a per-pubkey outbox relay map. Then computes the
|
||||
* minimum covering set of relays (greedy set cover) that covers all followed
|
||||
* pubkeys.
|
||||
*
|
||||
* On first-time startup, queries bootstrap relays for kind 10002.
|
||||
* On subsequent startups, queries the local relay first (fast), falls back
|
||||
* to bootstrap relays for any pubkey not found locally.
|
||||
*/
|
||||
#ifndef CACHING_RELAY_RELAY_DISCOVERY_H
|
||||
#define CACHING_RELAY_RELAY_DISCOVERY_H
|
||||
|
||||
#include "config.h"
|
||||
#include "state.h"
|
||||
#include "relay_sink.h"
|
||||
#include "../nostr_core_lib/nostr_core/nostr_core.h"
|
||||
|
||||
#define CR_MAX_RELAYS_PER_PUBKEY 8
|
||||
#define CR_MAX_PUBKEYS_PER_RELAY 512
|
||||
#define CR_MAX_DISCOVERED_RELAYS 128
|
||||
|
||||
/* Per-pubkey outbox relay list. */
|
||||
typedef struct {
|
||||
char pubkey[CR_HEX_LEN];
|
||||
char relays[CR_MAX_RELAYS_PER_PUBKEY][CR_URL_LEN];
|
||||
int relay_count;
|
||||
} cr_outbox_entry_t;
|
||||
|
||||
/* Result of relay discovery. */
|
||||
typedef struct {
|
||||
cr_outbox_entry_t *outboxes; /* per-pubkey relay lists (heap) */
|
||||
int outbox_count;
|
||||
|
||||
/* The selected minimum covering set of relay URLs. */
|
||||
char selected_relays[CR_MAX_DISCOVERED_RELAYS][CR_URL_LEN];
|
||||
int selected_count;
|
||||
|
||||
/* Per-pubkey: which selected relay covers this pubkey (index into selected_relays).
|
||||
* -1 means covered by bootstrap relays only. */
|
||||
int *coverage; /* heap, outbox_count entries */
|
||||
} cr_relay_map_t;
|
||||
|
||||
/* Discover outbox relays for all followed pubkeys.
|
||||
*
|
||||
* cfg - config (provides bootstrap relays, local_relay, root_hex)
|
||||
* upstream - upstream pool (has bootstrap relays; discovered relays will be added)
|
||||
* sink - sink pool (local relay, for publishing 10002 events + querying on subsequent startup)
|
||||
* followed - the followed pubkey set
|
||||
* is_first_time - 1 if first startup (query bootstrap only), 0 if subsequent (local-first)
|
||||
*
|
||||
* Returns 0 on success, -1 on hard error. On success, map is populated.
|
||||
* Caller must call cr_relay_map_free() when done.
|
||||
*/
|
||||
int cr_relay_discovery_run(cr_relay_map_t *map, cr_config_t *cfg,
|
||||
nostr_relay_pool_t *upstream, cr_sink_t *sink,
|
||||
const cr_pubkey_set_t *followed, int is_first_time);
|
||||
|
||||
/* Free heap resources in a relay map. */
|
||||
void cr_relay_map_free(cr_relay_map_t *map);
|
||||
|
||||
/* Get the outbox relays for a specific pubkey. Returns NULL if not found. */
|
||||
const cr_outbox_entry_t *cr_relay_map_get_outbox(const cr_relay_map_t *map, const char *hex);
|
||||
|
||||
#endif /* CACHING_RELAY_RELAY_DISCOVERY_H */
|
||||
@@ -0,0 +1,161 @@
|
||||
/*
|
||||
* caching_relay - relay sink: publishes fetched events to the local relay.
|
||||
*
|
||||
* See relay_sink.h for the dual-mode (WebSocket / PostgreSQL inbox) design.
|
||||
*/
|
||||
#define _GNU_SOURCE
|
||||
#include "relay_sink.h"
|
||||
#include "pg_inbox.h"
|
||||
#include "debug.h"
|
||||
|
||||
#include <string.h>
|
||||
#include <stdlib.h>
|
||||
|
||||
/* ------------------------------------------------------------------ */
|
||||
/* WebSocket mode callback */
|
||||
/* ------------------------------------------------------------------ */
|
||||
|
||||
static void sink_publish_cb(const char *relay_url, const char *event_id,
|
||||
int success, const char *message, void *user_data) {
|
||||
cr_sink_t *sink = (cr_sink_t *)user_data;
|
||||
if (success) {
|
||||
sink->published_ok++;
|
||||
DEBUG_TRACE("sink: OK %s from %s", event_id ? event_id : "?",
|
||||
relay_url ? relay_url : "?");
|
||||
} else {
|
||||
sink->published_failed++;
|
||||
/* REJECT is important - show at INFO level so it's visible at level 3. */
|
||||
DEBUG_INFO("sink: REJECT %s from %s: %s", event_id ? event_id : "?",
|
||||
relay_url ? relay_url : "?",
|
||||
message ? message : "(no message)");
|
||||
}
|
||||
}
|
||||
|
||||
/* ------------------------------------------------------------------ */
|
||||
/* Init */
|
||||
/* ------------------------------------------------------------------ */
|
||||
|
||||
int cr_sink_init(cr_sink_t *sink, const char *local_relay_url, cr_seen_ring_t *seen) {
|
||||
memset(sink, 0, sizeof(*sink));
|
||||
sink->seen = seen;
|
||||
sink->source_class = CR_SINK_CLASS_LIVE;
|
||||
strncpy(sink->url, local_relay_url, sizeof(sink->url) - 1);
|
||||
|
||||
sink->pool = nostr_relay_pool_create(nostr_pool_reconnect_config_default());
|
||||
if (!sink->pool) {
|
||||
DEBUG_ERROR("sink: failed to create pool");
|
||||
return -1;
|
||||
}
|
||||
if (nostr_relay_pool_add_relay(sink->pool, local_relay_url) != NOSTR_SUCCESS) {
|
||||
DEBUG_ERROR("sink: failed to add relay %s", local_relay_url);
|
||||
nostr_relay_pool_destroy(sink->pool);
|
||||
sink->pool = NULL;
|
||||
return -1;
|
||||
}
|
||||
DEBUG_INFO("sink: initialized (WebSocket) for %s", local_relay_url);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int cr_sink_init_pg(cr_sink_t *sink, cr_seen_ring_t *seen) {
|
||||
memset(sink, 0, sizeof(*sink));
|
||||
sink->seen = seen;
|
||||
sink->source_class = CR_SINK_CLASS_LIVE;
|
||||
sink->pool = NULL; /* PostgreSQL inbox mode - no WebSocket pool. */
|
||||
DEBUG_INFO("sink: initialized (PostgreSQL inbox)");
|
||||
return 0;
|
||||
}
|
||||
|
||||
void cr_sink_set_source_class(cr_sink_t *sink, const char *class_ptr) {
|
||||
if (sink) sink->source_class = class_ptr ? class_ptr : CR_SINK_CLASS_LIVE;
|
||||
}
|
||||
|
||||
void cr_sink_destroy(cr_sink_t *sink) {
|
||||
if (sink && sink->pool) {
|
||||
nostr_relay_pool_destroy(sink->pool);
|
||||
sink->pool = NULL;
|
||||
}
|
||||
}
|
||||
|
||||
/* ------------------------------------------------------------------ */
|
||||
/* Publish */
|
||||
/* ------------------------------------------------------------------ */
|
||||
|
||||
int cr_sink_publish_from(cr_sink_t *sink, cJSON *event, const char *source_relay) {
|
||||
if (!sink || !event) return -1;
|
||||
|
||||
cJSON *id = cJSON_GetObjectItem(event, "id");
|
||||
if (!id || !cJSON_IsString(id)) {
|
||||
DEBUG_WARN("sink: event missing id, skipping");
|
||||
return -1;
|
||||
}
|
||||
const char *eid = cJSON_GetStringValue(id);
|
||||
|
||||
if (cr_seen_ring_add(sink->seen, eid) == 0) {
|
||||
sink->published_dup++;
|
||||
return 0; /* already seen recently */
|
||||
}
|
||||
|
||||
/* PostgreSQL inbox mode. */
|
||||
if (!sink->pool) {
|
||||
char *json = cJSON_PrintUnformatted(event);
|
||||
if (!json) {
|
||||
DEBUG_WARN("sink: failed to serialize event %s", eid);
|
||||
sink->published_failed++;
|
||||
return -1;
|
||||
}
|
||||
const char *klass = sink->source_class ? sink->source_class
|
||||
: CR_SINK_CLASS_LIVE;
|
||||
int priority = (strcmp(klass, CR_SINK_CLASS_BACKFILL) == 0) ? 1 : 0;
|
||||
|
||||
cJSON *kind = cJSON_GetObjectItem(event, "kind");
|
||||
cJSON *pubkey = cJSON_GetObjectItem(event, "pubkey");
|
||||
int kind_num = (kind && cJSON_IsNumber(kind)) ? (int)cJSON_GetNumberValue(kind) : -1;
|
||||
DEBUG_TRACE("sink: INBOX event id=%s kind=%d pubkey=%s class=%s",
|
||||
eid, kind_num,
|
||||
pubkey && cJSON_IsString(pubkey) ? cJSON_GetStringValue(pubkey) : "?",
|
||||
klass);
|
||||
|
||||
int rc = pg_inbox_insert_event(json, source_relay, klass, priority);
|
||||
free(json);
|
||||
if (rc != 0) {
|
||||
sink->published_failed++;
|
||||
return -1;
|
||||
}
|
||||
sink->published_ok++;
|
||||
return 1;
|
||||
}
|
||||
|
||||
/* WebSocket mode. */
|
||||
const char *urls[1] = { sink->url };
|
||||
cJSON *kind = cJSON_GetObjectItem(event, "kind");
|
||||
cJSON *pubkey = cJSON_GetObjectItem(event, "pubkey");
|
||||
int kind_num = (kind && cJSON_IsNumber(kind)) ? (int)cJSON_GetNumberValue(kind) : -1;
|
||||
DEBUG_TRACE("sink: SEND event id=%s kind=%d pubkey=%s to %s",
|
||||
eid, kind_num,
|
||||
pubkey && cJSON_IsString(pubkey) ? cJSON_GetStringValue(pubkey) : "?",
|
||||
sink->url);
|
||||
|
||||
int rc = nostr_relay_pool_publish_async(sink->pool, urls, 1, event,
|
||||
sink_publish_cb, sink);
|
||||
if (rc < 0) {
|
||||
DEBUG_WARN("sink: publish_async error rc=%d for %s", rc, eid);
|
||||
return -1;
|
||||
}
|
||||
if (rc == 0) {
|
||||
DEBUG_INFO("sink: publish SKIPPED (not connected) for %s", eid);
|
||||
return 0;
|
||||
}
|
||||
DEBUG_TRACE("sink: publish_async rc=%d for %s", rc, eid);
|
||||
return 1;
|
||||
}
|
||||
|
||||
int cr_sink_publish(cr_sink_t *sink, cJSON *event) {
|
||||
return cr_sink_publish_from(sink, event, NULL);
|
||||
}
|
||||
|
||||
void cr_sink_pump(cr_sink_t *sink, int timeout_ms) {
|
||||
if (sink && sink->pool) {
|
||||
nostr_relay_pool_run(sink->pool, timeout_ms);
|
||||
}
|
||||
/* No-op in PostgreSQL inbox mode. */
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
/*
|
||||
* caching_relay - relay sink: publishes fetched events to the local relay.
|
||||
*
|
||||
* Two modes:
|
||||
* - WebSocket mode (legacy): a dedicated single-relay pool publishes via
|
||||
* nostr_relay_pool_publish_async(). Used when no PostgreSQL connection
|
||||
* string is provided.
|
||||
* - PostgreSQL inbox mode: events are inserted directly into the
|
||||
* caching_event_inbox table via pg_inbox_insert_event(). Used when the
|
||||
* daemon is started with --pg-conn.
|
||||
*
|
||||
* The public function signatures are identical in both modes so that callers
|
||||
* (live_subscriber, backfill, relay_discovery) do not need to change.
|
||||
*/
|
||||
#ifndef CACHING_RELAY_RELAY_SINK_H
|
||||
#define CACHING_RELAY_RELAY_SINK_H
|
||||
|
||||
#include "state.h"
|
||||
#include "../nostr_core_lib/nostr_core/nostr_core.h"
|
||||
|
||||
/* Source class labels for the inbox priority field. */
|
||||
#define CR_SINK_CLASS_LIVE "live"
|
||||
#define CR_SINK_CLASS_DISCOVERY "discovery"
|
||||
#define CR_SINK_CLASS_BACKFILL "backfill"
|
||||
|
||||
typedef struct {
|
||||
/* WebSocket mode only (NULL in PostgreSQL inbox mode). */
|
||||
nostr_relay_pool_t *pool;
|
||||
char url[256];
|
||||
|
||||
/* PostgreSQL inbox mode: source class used for priority assignment.
|
||||
* Defaults to "live". Set to "backfill" by the backfill module.
|
||||
* Ignored in WebSocket mode. */
|
||||
const char *source_class;
|
||||
|
||||
cr_seen_ring_t *seen; /* shared seen ring for dedup */
|
||||
long published_ok;
|
||||
long published_failed;
|
||||
long published_dup;
|
||||
} cr_sink_t;
|
||||
|
||||
/* Create the sink.
|
||||
*
|
||||
* In WebSocket mode (pg_mode = 0): creates a relay pool and adds
|
||||
* local_relay_url.
|
||||
* In PostgreSQL inbox mode (pg_mode != 0): no pool is created; the URL is
|
||||
* ignored. Events are inserted via pg_inbox_insert_event() (the pg_inbox
|
||||
* module must be initialized beforehand).
|
||||
*
|
||||
* The original two-argument signature is preserved via cr_sink_init() below
|
||||
* for backward compatibility; it defaults to WebSocket mode. */
|
||||
int cr_sink_init(cr_sink_t *sink, const char *local_relay_url, cr_seen_ring_t *seen);
|
||||
|
||||
/* Initialize the sink in PostgreSQL inbox mode. */
|
||||
int cr_sink_init_pg(cr_sink_t *sink, cr_seen_ring_t *seen);
|
||||
|
||||
/* Set the source class for priority assignment in PostgreSQL inbox mode.
|
||||
* class_ptr must point to a static string literal (e.g. CR_SINK_CLASS_LIVE). */
|
||||
void cr_sink_set_source_class(cr_sink_t *sink, const char *class_ptr);
|
||||
|
||||
void cr_sink_destroy(cr_sink_t *sink);
|
||||
|
||||
/* Publish a cJSON event. Dedups against the seen ring.
|
||||
* Returns 1 if enqueued/inserted, 0 if dup (skipped), -1 on error. */
|
||||
int cr_sink_publish(cr_sink_t *sink, cJSON *event);
|
||||
|
||||
/* Variant that records the originating relay URL in PostgreSQL inbox mode.
|
||||
* In WebSocket mode the source_relay is ignored. */
|
||||
int cr_sink_publish_from(cr_sink_t *sink, cJSON *event, const char *source_relay);
|
||||
|
||||
/* Pump the sink pool briefly to flush pending publish callbacks.
|
||||
* No-op in PostgreSQL inbox mode. */
|
||||
void cr_sink_pump(cr_sink_t *sink, int timeout_ms);
|
||||
|
||||
#endif /* CACHING_RELAY_RELAY_SINK_H */
|
||||
@@ -0,0 +1,76 @@
|
||||
/*
|
||||
* caching_relay - in-memory runtime state
|
||||
*/
|
||||
#define _GNU_SOURCE
|
||||
#include "state.h"
|
||||
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
/* ---- pubkey set ---- */
|
||||
|
||||
void cr_pubkey_set_init(cr_pubkey_set_t *s) {
|
||||
s->items = NULL;
|
||||
s->count = 0;
|
||||
s->capacity = 0;
|
||||
}
|
||||
|
||||
void cr_pubkey_set_free(cr_pubkey_set_t *s) {
|
||||
for (int i = 0; i < s->count; i++) free(s->items[i]);
|
||||
free(s->items);
|
||||
s->items = NULL;
|
||||
s->count = 0;
|
||||
s->capacity = 0;
|
||||
}
|
||||
|
||||
void cr_pubkey_set_clear(cr_pubkey_set_t *s) {
|
||||
cr_pubkey_set_free(s);
|
||||
cr_pubkey_set_init(s);
|
||||
}
|
||||
|
||||
int cr_pubkey_set_contains(const cr_pubkey_set_t *s, const char *hex) {
|
||||
for (int i = 0; i < s->count; i++) {
|
||||
if (strcmp(s->items[i], hex) == 0) return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
int cr_pubkey_set_add(cr_pubkey_set_t *s, const char *hex) {
|
||||
if (!hex || !hex[0]) return 0;
|
||||
if (cr_pubkey_set_contains(s, hex)) return 0;
|
||||
if (s->count >= s->capacity) {
|
||||
int newcap = s->capacity ? s->capacity * 2 : 64;
|
||||
char **ni = realloc(s->items, newcap * sizeof(char *));
|
||||
if (!ni) return -1;
|
||||
s->items = ni;
|
||||
s->capacity = newcap;
|
||||
}
|
||||
s->items[s->count] = strdup(hex);
|
||||
if (!s->items[s->count]) return -1;
|
||||
s->count++;
|
||||
return 1;
|
||||
}
|
||||
|
||||
/* ---- seen-event ring buffer ---- */
|
||||
|
||||
void cr_seen_ring_init(cr_seen_ring_t *r) {
|
||||
memset(r, 0, sizeof(*r));
|
||||
}
|
||||
|
||||
int cr_seen_ring_contains(const cr_seen_ring_t *r, const char *id) {
|
||||
for (int i = 0; i < r->count; i++) {
|
||||
int idx = (r->head - 1 - i + CR_SEEN_RING_SIZE) % CR_SEEN_RING_SIZE;
|
||||
if (strcmp(r->ids[idx], id) == 0) return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
int cr_seen_ring_add(cr_seen_ring_t *r, const char *id) {
|
||||
if (!id || !id[0]) return 0;
|
||||
if (cr_seen_ring_contains(r, id)) return 0;
|
||||
strncpy(r->ids[r->head], id, CR_HEX_LEN - 1);
|
||||
r->ids[r->head][CR_HEX_LEN - 1] = '\0';
|
||||
r->head = (r->head + 1) % CR_SEEN_RING_SIZE;
|
||||
if (r->count < CR_SEEN_RING_SIZE) r->count++;
|
||||
return 1;
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
/*
|
||||
* caching_relay - in-memory runtime state:
|
||||
* - followed-pubkey set (hex pubkeys)
|
||||
* - seen-event-id ring buffer (to avoid redundant publish attempts)
|
||||
*/
|
||||
#ifndef CACHING_RELAY_STATE_H
|
||||
#define CACHING_RELAY_STATE_H
|
||||
|
||||
#include <stddef.h>
|
||||
|
||||
/* Hex pubkey is 64 chars + NUL. Event id is 64 hex chars + NUL. */
|
||||
#define CR_HEX_LEN 65
|
||||
|
||||
/* A simple dynamic string set for followed pubkeys. */
|
||||
typedef struct {
|
||||
char **items;
|
||||
int count;
|
||||
int capacity;
|
||||
} cr_pubkey_set_t;
|
||||
|
||||
void cr_pubkey_set_init(cr_pubkey_set_t *s);
|
||||
void cr_pubkey_set_free(cr_pubkey_set_t *s);
|
||||
int cr_pubkey_set_add(cr_pubkey_set_t *s, const char *hex); /* 1 if added, 0 if dup */
|
||||
int cr_pubkey_set_contains(const cr_pubkey_set_t *s, const char *hex);
|
||||
void cr_pubkey_set_clear(cr_pubkey_set_t *s);
|
||||
|
||||
/* Fixed-size ring buffer of event ids for dedup. */
|
||||
#define CR_SEEN_RING_SIZE 4096
|
||||
|
||||
typedef struct {
|
||||
char ids[CR_SEEN_RING_SIZE][CR_HEX_LEN];
|
||||
int head;
|
||||
int count;
|
||||
} cr_seen_ring_t;
|
||||
|
||||
void cr_seen_ring_init(cr_seen_ring_t *r);
|
||||
/* Returns 1 if the id was newly inserted, 0 if it was already present. */
|
||||
int cr_seen_ring_add(cr_seen_ring_t *r, const char *id);
|
||||
int cr_seen_ring_contains(const cr_seen_ring_t *r, const char *id);
|
||||
|
||||
#endif /* CACHING_RELAY_STATE_H */
|
||||
@@ -0,0 +1 @@
|
||||
key,value,data_type,description,category,requires_restart,created_at,updated_at
|
||||
|
+4
-4
@@ -1,12 +1,12 @@
|
||||
#!/bin/bash
|
||||
# Restart the service
|
||||
sudo systemctl stop c-relay.service
|
||||
sudo systemctl stop c-relay-pg.service
|
||||
|
||||
# Copy the binary to the deployment location
|
||||
cp build/c_relay_static_x86_64 ~/Storage/c_relay/crelay
|
||||
cp build/c_relay_pg_static_x86_64 ~/Storage/c_relay_pg/crelay
|
||||
|
||||
# Restart the service
|
||||
sudo systemctl restart c-relay.service
|
||||
sudo systemctl restart c-relay-pg.service
|
||||
|
||||
# Show service status
|
||||
sudo systemctl status c-relay.service --no-pager -l
|
||||
sudo systemctl status c-relay-pg.service --no-pager -l
|
||||
|
||||
+125
-18
@@ -1,28 +1,135 @@
|
||||
#!/bin/bash
|
||||
|
||||
# C-Relay Static Binary Deployment Script
|
||||
# Deploys build/c_relay_static_x86_64 to server via ssh
|
||||
# C-Relay-PG PostgreSQL Deployment Script
|
||||
# Deploys static relay binary and configures PostgreSQL 18 + systemd service on remote server.
|
||||
|
||||
set -e
|
||||
set -euo pipefail
|
||||
|
||||
# Configuration
|
||||
LOCAL_BINARY="build/c_relay_static_x86_64"
|
||||
REMOTE_BINARY_PATH="/usr/local/bin/c_relay/c_relay"
|
||||
SERVICE_NAME="c-relay"
|
||||
REMOTE_HOST="ubuntu@laantungir.net"
|
||||
LOCAL_BINARY="build/c_relay_pg_static_x86_64"
|
||||
LOCAL_CACHING_BINARY="build/caching_relay"
|
||||
REMOTE_BINARY_DIR="/usr/local/bin/c_relay_pg"
|
||||
REMOTE_BINARY_PATH="/usr/local/bin/c_relay_pg/c_relay_pg"
|
||||
REMOTE_CACHING_BINARY_PATH="/opt/c-relay-pg/caching_relay"
|
||||
SERVICE_NAME="c-relay-pg"
|
||||
|
||||
# Create backup
|
||||
ssh ubuntu@laantungir.com "sudo cp '$REMOTE_BINARY_PATH' '${REMOTE_BINARY_PATH}.backup.$(date +%Y%m%d_%H%M%S)'" 2>/dev/null || true
|
||||
LOCAL_SERVICE_FILE="systemd/c-relay.service"
|
||||
LOCAL_PG_SETUP_SCRIPT="systemd/setup_postgres_18.sh"
|
||||
|
||||
# Upload binary to temp location
|
||||
scp "$LOCAL_BINARY" "ubuntu@laantungir.com:/tmp/c_relay.tmp"
|
||||
if [ ! -f "$LOCAL_BINARY" ]; then
|
||||
echo "ERROR: Binary not found: $LOCAL_BINARY"
|
||||
echo "Build it first (e.g. ./make_and_restart_relay.sh)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Install binary
|
||||
ssh ubuntu@laantungir.com "sudo mv '/tmp/c_relay.tmp' '$REMOTE_BINARY_PATH'"
|
||||
ssh ubuntu@laantungir.com "sudo chown c-relay:c-relay '$REMOTE_BINARY_PATH'"
|
||||
ssh ubuntu@laantungir.com "sudo chmod +x '$REMOTE_BINARY_PATH'"
|
||||
if [ ! -f "$LOCAL_SERVICE_FILE" ]; then
|
||||
echo "ERROR: Service file not found: $LOCAL_SERVICE_FILE"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Reload systemd and restart service
|
||||
ssh ubuntu@laantungir.com "sudo systemctl daemon-reload"
|
||||
ssh ubuntu@laantungir.com "sudo systemctl restart '$SERVICE_NAME'"
|
||||
if [ ! -f "$LOCAL_PG_SETUP_SCRIPT" ]; then
|
||||
echo "ERROR: PostgreSQL setup script not found: $LOCAL_PG_SETUP_SCRIPT"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Deployment complete!"
|
||||
echo "==> Uploading artifacts to $REMOTE_HOST"
|
||||
scp "$LOCAL_BINARY" "$REMOTE_HOST:/tmp/c_relay_pg.tmp"
|
||||
scp "$LOCAL_SERVICE_FILE" "$REMOTE_HOST:/tmp/c-relay-pg.service"
|
||||
scp "$LOCAL_PG_SETUP_SCRIPT" "$REMOTE_HOST:/tmp/setup_postgres_18.sh"
|
||||
# Upload the caching_relay binary if it was built locally. The caching
|
||||
# service is NOT started by this deploy (the systemd unit does not pass
|
||||
# --start-caching), but the binary is placed in the relay's WorkingDirectory
|
||||
# so it is ready for when caching is enabled via event-based config.
|
||||
if [ -f "$LOCAL_CACHING_BINARY" ]; then
|
||||
scp "$LOCAL_CACHING_BINARY" "$REMOTE_HOST:/tmp/caching_relay.tmp"
|
||||
else
|
||||
echo "WARNING: caching_relay binary not found locally ($LOCAL_CACHING_BINARY) — skipping caching binary upload"
|
||||
fi
|
||||
|
||||
echo "==> Running remote install/configuration"
|
||||
ssh "$REMOTE_HOST" 'bash -s' <<'EOF'
|
||||
set -euo pipefail
|
||||
|
||||
SERVICE_NAME="c-relay-pg"
|
||||
RELAY_USER="c-relay-pg"
|
||||
REMOTE_BINARY_DIR="/usr/local/bin/c_relay_pg"
|
||||
REMOTE_BINARY_PATH="/usr/local/bin/c_relay_pg/c_relay_pg"
|
||||
REMOTE_CACHING_BINARY_PATH="/opt/c-relay-pg/caching_relay"
|
||||
|
||||
echo "[remote] Ensuring service user exists"
|
||||
if ! id "$RELAY_USER" >/dev/null 2>&1; then
|
||||
sudo useradd --system --home-dir /opt/c-relay-pg --shell /usr/sbin/nologin "$RELAY_USER"
|
||||
fi
|
||||
|
||||
echo "[remote] Ensuring required directories exist"
|
||||
sudo mkdir -p "$REMOTE_BINARY_DIR" /opt/c-relay-pg /etc/c-relay-pg
|
||||
sudo chown "$RELAY_USER:$RELAY_USER" /opt/c-relay-pg
|
||||
|
||||
echo "[remote] Installing PostgreSQL 18 (PGDG) if missing"
|
||||
if ! dpkg -s postgresql-18 >/dev/null 2>&1; then
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y curl ca-certificates lsb-release gnupg
|
||||
sudo install -d /usr/share/postgresql-common/pgdg
|
||||
sudo curl -fsSL https://www.postgresql.org/media/keys/ACCC4CF8.asc -o /usr/share/postgresql-common/pgdg/apt.postgresql.org.asc
|
||||
echo "deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.asc] https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" | sudo tee /etc/apt/sources.list.d/pgdg.list >/dev/null
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y postgresql-18
|
||||
fi
|
||||
|
||||
sudo systemctl enable postgresql
|
||||
sudo systemctl restart postgresql
|
||||
|
||||
echo "[remote] Configuring PostgreSQL role/database"
|
||||
sudo chmod +x /tmp/setup_postgres_18.sh
|
||||
sudo /tmp/setup_postgres_18.sh
|
||||
|
||||
echo "[remote] Installing relay binary"
|
||||
if [ -f "$REMOTE_BINARY_PATH" ]; then
|
||||
sudo cp "$REMOTE_BINARY_PATH" "${REMOTE_BINARY_PATH}.backup.$(date +%Y%m%d_%H%M%S)"
|
||||
fi
|
||||
sudo mv /tmp/c_relay_pg.tmp "$REMOTE_BINARY_PATH"
|
||||
sudo chown "$RELAY_USER:$RELAY_USER" "$REMOTE_BINARY_PATH"
|
||||
sudo chmod +x "$REMOTE_BINARY_PATH"
|
||||
|
||||
echo "[remote] Installing caching_relay binary (if uploaded)"
|
||||
if [ -f /tmp/caching_relay.tmp ]; then
|
||||
sudo mv /tmp/caching_relay.tmp "$REMOTE_CACHING_BINARY_PATH"
|
||||
sudo chown "$RELAY_USER:$RELAY_USER" "$REMOTE_CACHING_BINARY_PATH"
|
||||
sudo chmod +x "$REMOTE_CACHING_BINARY_PATH"
|
||||
echo " -> caching_relay installed at $REMOTE_CACHING_BINARY_PATH (not started; enable via config when ready)"
|
||||
else
|
||||
echo " -> no caching_relay binary uploaded, skipping"
|
||||
fi
|
||||
|
||||
echo "[remote] Installing systemd unit"
|
||||
sudo mv /tmp/c-relay-pg.service /etc/systemd/system/c-relay-pg.service
|
||||
sudo chown root:root /etc/systemd/system/c-relay-pg.service
|
||||
sudo chmod 644 /etc/systemd/system/c-relay-pg.service
|
||||
|
||||
echo "[remote] Killing stale caching_relay processes (safety)"
|
||||
# The caching service is forked by the relay with setsid(), so it detaches
|
||||
# from the relay's process group and survives when the relay is killed.
|
||||
# Without this, every restart orphans the previous caching_relay child and
|
||||
# a new relay forks another, leading to many stale processes with dead PG
|
||||
# connections spamming errors. Kill them here so the new relay starts clean.
|
||||
CACHING_PIDS=$(pgrep -f "caching_relay" || echo "")
|
||||
if [ -n "$CACHING_PIDS" ]; then
|
||||
echo " -> killing stale caching_relay PIDs: $CACHING_PIDS"
|
||||
kill -9 $CACHING_PIDS 2>/dev/null || true
|
||||
sleep 1
|
||||
else
|
||||
echo " -> no stale caching_relay processes found"
|
||||
fi
|
||||
|
||||
echo "[remote] Reloading and restarting service"
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable "$SERVICE_NAME"
|
||||
sudo systemctl restart "$SERVICE_NAME"
|
||||
|
||||
echo "[remote] Health checks"
|
||||
sudo systemctl --no-pager --full status "$SERVICE_NAME" | sed -n '1,25p'
|
||||
sudo -u "$RELAY_USER" psql -d crelay -c "SELECT current_user, current_database();"
|
||||
EOF
|
||||
|
||||
echo "Deployment complete: $REMOTE_HOST"
|
||||
|
||||
Executable
+122
@@ -0,0 +1,122 @@
|
||||
#!/bin/bash
|
||||
|
||||
# C-Relay-PG Debug Binary Deployment Script
|
||||
# Deploys build/c_relay_pg_static_x86_64_debug to server for CPU profiling
|
||||
#
|
||||
# Usage:
|
||||
# ./deploy_lt_debug.sh -- deploy debug binary and restart
|
||||
# ./deploy_lt_debug.sh --profile -- deploy, then run perf and fetch results
|
||||
#
|
||||
# After deploying, profile with:
|
||||
# sudo perf record -g -p $(pgrep c_relay_pg) -- sleep 30
|
||||
# sudo perf report --stdio --sort=symbol --no-children -n 2>/dev/null | head -80
|
||||
#
|
||||
# Restore production binary:
|
||||
# ./deploy_lt.sh
|
||||
|
||||
set -e
|
||||
|
||||
LOCAL_DEBUG_BINARY="build/c_relay_pg_static_x86_64_debug"
|
||||
REMOTE_BINARY_PATH="/usr/local/bin/c_relay_pg/c_relay_pg"
|
||||
REMOTE_PROD_BACKUP="/usr/local/bin/c_relay_pg/c_relay_pg.production"
|
||||
SERVICE_NAME="c-relay-pg"
|
||||
REMOTE_HOST="ubuntu@laantungir.com"
|
||||
|
||||
# Check debug binary exists
|
||||
if [ ! -f "$LOCAL_DEBUG_BINARY" ]; then
|
||||
echo "ERROR: Debug binary not found: $LOCAL_DEBUG_BINARY"
|
||||
echo ""
|
||||
echo "Build it first with:"
|
||||
echo " ./build_static.sh --debug"
|
||||
echo ""
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "=========================================="
|
||||
echo "C-Relay-PG Debug Deployment"
|
||||
echo "=========================================="
|
||||
echo "Binary: $LOCAL_DEBUG_BINARY ($(du -h "$LOCAL_DEBUG_BINARY" | cut -f1))"
|
||||
echo "Target: $REMOTE_HOST:$REMOTE_BINARY_PATH"
|
||||
echo ""
|
||||
echo "WARNING: Debug binary has symbols and is larger than production."
|
||||
echo " It is safe to run but should not be left deployed long-term."
|
||||
echo ""
|
||||
|
||||
# Backup production binary (only if not already backed up)
|
||||
echo "Backing up production binary..."
|
||||
ssh "$REMOTE_HOST" "
|
||||
if [ ! -f '$REMOTE_PROD_BACKUP' ]; then
|
||||
sudo cp '$REMOTE_BINARY_PATH' '$REMOTE_PROD_BACKUP'
|
||||
echo 'Production binary backed up to $REMOTE_PROD_BACKUP'
|
||||
else
|
||||
echo 'Production backup already exists, skipping'
|
||||
fi
|
||||
"
|
||||
|
||||
# Upload debug binary
|
||||
echo "Uploading debug binary..."
|
||||
scp "$LOCAL_DEBUG_BINARY" "$REMOTE_HOST:/tmp/c_relay_pg_debug.tmp"
|
||||
|
||||
# Install debug binary
|
||||
echo "Installing debug binary..."
|
||||
ssh "$REMOTE_HOST" "
|
||||
sudo mv '/tmp/c_relay_pg_debug.tmp' '$REMOTE_BINARY_PATH'
|
||||
sudo chown c-relay-pg:c-relay-pg '$REMOTE_BINARY_PATH'
|
||||
sudo chmod +x '$REMOTE_BINARY_PATH'
|
||||
"
|
||||
|
||||
# Restart service
|
||||
echo "Restarting c-relay-pg service..."
|
||||
ssh "$REMOTE_HOST" "sudo systemctl daemon-reload && sudo systemctl restart '$SERVICE_NAME'"
|
||||
|
||||
echo ""
|
||||
echo "✓ Debug binary deployed and service restarted"
|
||||
echo ""
|
||||
|
||||
# If --profile flag, run perf automatically
|
||||
if [ "$1" = "--profile" ]; then
|
||||
echo "=========================================="
|
||||
echo "Running perf profile (30 seconds)..."
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
|
||||
# Wait for relay to start
|
||||
sleep 3
|
||||
|
||||
ssh "$REMOTE_HOST" "
|
||||
PID=\$(pgrep c_relay_pg)
|
||||
if [ -z \"\$PID\" ]; then
|
||||
echo 'ERROR: c_relay_pg not running'
|
||||
exit 1
|
||||
fi
|
||||
echo \"Profiling PID \$PID for 30 seconds...\"
|
||||
sudo perf record -g -p \$PID -- sleep 30
|
||||
echo ''
|
||||
echo '=== TOP FUNCTIONS BY CPU ==='
|
||||
sudo perf report --stdio --sort=symbol --no-children -n 2>/dev/null | head -60
|
||||
"
|
||||
else
|
||||
echo "=========================================="
|
||||
echo "Next Steps: Profile the relay"
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
echo "SSH to server and run:"
|
||||
echo " sudo perf record -g -p \$(pgrep c_relay_pg) -- sleep 30"
|
||||
echo " sudo perf report --stdio --sort=symbol --no-children -n 2>/dev/null | head -60"
|
||||
echo ""
|
||||
echo "Or run with --profile to do it automatically:"
|
||||
echo " ./deploy_lt_debug.sh --profile"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "=========================================="
|
||||
echo "Restore Production Binary When Done"
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
echo "Run this to restore the production binary:"
|
||||
echo " ./deploy_lt.sh"
|
||||
echo ""
|
||||
echo "Or manually on the server:"
|
||||
echo " sudo cp '$REMOTE_PROD_BACKUP' '$REMOTE_BINARY_PATH'"
|
||||
echo " sudo systemctl restart $SERVICE_NAME"
|
||||
echo ""
|
||||
@@ -1,4 +1,4 @@
|
||||
# C-Relay Administrator API Implementation Plan
|
||||
# C-Relay-PG Administrator API Implementation Plan
|
||||
|
||||
## Problem Analysis
|
||||
|
||||
@@ -161,7 +161,7 @@ Would require changing schema, migration scripts, and storage logic.
|
||||
#### README.md Documentation Format:
|
||||
|
||||
```markdown
|
||||
# C-Relay Administrator API
|
||||
# C-Relay-PG Administrator API
|
||||
|
||||
## Authentication
|
||||
All admin commands require signing with the admin private key generated during first startup.
|
||||
|
||||
@@ -0,0 +1,190 @@
|
||||
# Agent Browser Testing Guide for C-Relay-PG
|
||||
|
||||
This document explains how to use the `agent-browser` CLI tool to test the c-relay-pg admin web UI from an AI agent context (no physical display required).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **agent-browser** installed globally: `npm install -g agent-browser`
|
||||
- Binary location: `/home/user/.nvm/versions/node/v24.14.1/bin/agent-browser`
|
||||
- The relay must be running locally (default port 8888)
|
||||
- Test keys configured in `.test_keys`
|
||||
|
||||
## Starting the Relay for Local Testing
|
||||
|
||||
```bash
|
||||
./make_and_restart_relay.sh -t
|
||||
```
|
||||
|
||||
This reads `.test_keys` and starts the relay with:
|
||||
- `ADMIN_PUBKEY` — the hex public key of the admin account
|
||||
- `ADMIN_PRIVKEY` — the hex secret key (used for browser login, not by the relay itself)
|
||||
- `SERVER_PRIVKEY` — the relay's own private key
|
||||
|
||||
## Key URLs
|
||||
|
||||
| URL | Purpose |
|
||||
|-----|---------|
|
||||
| `http://127.0.0.1:8888/api/index.html` | Admin web UI (the correct entry point) |
|
||||
| `http://127.0.0.1:8888/` | Returns 406 without NIP-11 Accept header — **do not use for browser testing** |
|
||||
| `http://127.0.0.1:8888/` with `Accept: application/nostr+json` | NIP-11 relay info JSON |
|
||||
|
||||
**Important:** The root URL `/` is a WebSocket/NIP-11 endpoint, not an HTML page. Always use `/api/index.html` for browser testing.
|
||||
|
||||
## Admin Login Credentials
|
||||
|
||||
The admin nsec for the current `.test_keys` configuration:
|
||||
|
||||
```
|
||||
nsec: nsec1zkn0hlt4jvcvn9yt5p7ea4m7f82pf3ph0gj3d4rlcz4s64x5z86satv9qm
|
||||
hex: 15a6fbfd759330c9948ba07d9ed77e49d414c4377a2516d47fc0ab0d54d411f5
|
||||
npub: npub1tlyzk6slzt8d989f4pn3synk98fea64ea3jmrdc7dtd0kw8uxlas9a9fwr
|
||||
hex pubkey: 5fc82b6a1f12ced29ca9a86718127629d39eeab9ec65b1b71e6adafb38fc37fb
|
||||
```
|
||||
|
||||
## Complete Login Flow with agent-browser
|
||||
|
||||
### Step 1: Open the admin UI
|
||||
|
||||
```bash
|
||||
agent-browser open http://127.0.0.1:8888/api/index.html
|
||||
agent-browser wait --load networkidle
|
||||
```
|
||||
|
||||
### Step 2: Take an interactive snapshot to see what is on screen
|
||||
|
||||
```bash
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
You will see a login modal with buttons like:
|
||||
- `"Browser Extension"` — skip this
|
||||
- `"Local Key"` — **use this one**
|
||||
- `"Seed Phrase"` — skip
|
||||
- `"Nostr Connect"` — skip
|
||||
- `"Read Only"` — skip
|
||||
|
||||
### Step 3: Click "Local Key"
|
||||
|
||||
```bash
|
||||
agent-browser click @e15
|
||||
```
|
||||
|
||||
(The ref number may vary — use the ref from the snapshot output for the "Local Key" button.)
|
||||
|
||||
After clicking, a text input appears asking for the secret key.
|
||||
|
||||
### Step 4: Enter the admin nsec
|
||||
|
||||
```bash
|
||||
agent-browser fill @e14 "nsec1zkn0hlt4jvcvn9yt5p7ea4m7f82pf3ph0gj3d4rlcz4s64x5z86satv9qm"
|
||||
```
|
||||
|
||||
(Use the ref from the snapshot for the textbox element.)
|
||||
|
||||
### Step 5: Click "Import Key"
|
||||
|
||||
```bash
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
Check the snapshot — the "Import Key" button should now be enabled. Click it:
|
||||
|
||||
```bash
|
||||
agent-browser click @e15
|
||||
```
|
||||
|
||||
### Step 6: Click "Continue" on the success screen
|
||||
|
||||
After import, a success screen appears with "Continue" button:
|
||||
|
||||
```bash
|
||||
agent-browser wait 800
|
||||
agent-browser snapshot -i
|
||||
agent-browser click @e19
|
||||
```
|
||||
|
||||
(Use the ref from the snapshot for the "Continue" button.)
|
||||
|
||||
### Step 7: Wait for admin UI to load
|
||||
|
||||
```bash
|
||||
agent-browser wait 5000
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
You should now see the admin dashboard with:
|
||||
- Statistics table (Database Size, Total Events, PID, etc.)
|
||||
- Navigation buttons (Statistics, Subscriptions, Configuration, Authorization, etc.)
|
||||
- Admin profile area showing "admin" label
|
||||
|
||||
## Navigating Admin Sections
|
||||
|
||||
After login, use the sidebar navigation buttons:
|
||||
|
||||
```bash
|
||||
# View configuration
|
||||
agent-browser click @e8 # Configuration button ref
|
||||
|
||||
# View authorization rules
|
||||
agent-browser click @e9 # Authorization button ref
|
||||
|
||||
# View statistics
|
||||
agent-browser click @e6 # Statistics button ref
|
||||
|
||||
# Always snapshot after navigation to see results
|
||||
agent-browser wait 2500
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
## Checking for Errors
|
||||
|
||||
```bash
|
||||
# View browser console logs
|
||||
agent-browser console
|
||||
|
||||
# View JavaScript errors
|
||||
agent-browser errors
|
||||
|
||||
# View relay server logs
|
||||
tail -n 100 relay.log
|
||||
```
|
||||
|
||||
## Chained Command Example (Full Login in One Shot)
|
||||
|
||||
```bash
|
||||
agent-browser open http://127.0.0.1:8888/api/index.html && \
|
||||
agent-browser wait --load networkidle && \
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
Then use refs from the snapshot to complete login steps.
|
||||
|
||||
## Tips for AI Agents
|
||||
|
||||
1. **Always use `/api/index.html`** — never the root URL
|
||||
2. **Use `snapshot -i`** after every action to see the current interactive elements and their refs
|
||||
3. **Refs change** between snapshots — always re-snapshot before clicking
|
||||
4. **Wait after clicks** — use `agent-browser wait 2000` (milliseconds) between actions that trigger async operations
|
||||
5. **The login flow has 3 screens**: method selection → key input → success confirmation
|
||||
6. **Console logs are cumulative** — they show all logs since page load, which is useful for debugging admin API responses
|
||||
7. **The relay log** at `relay.log` shows server-side processing of admin commands
|
||||
8. **Command chaining** with `&&` works — the browser daemon persists between commands
|
||||
|
||||
## Verifying Admin API is Working
|
||||
|
||||
After login, the statistics page should show populated data:
|
||||
- Database Size (e.g., "4 KB")
|
||||
- Process ID (the relay PID)
|
||||
- WebSocket Connections count
|
||||
- Memory Usage
|
||||
|
||||
If these show "-" or "Loading...", check:
|
||||
1. `relay.log` for errors
|
||||
2. `agent-browser console` for JavaScript errors
|
||||
3. `agent-browser errors` for page-level errors
|
||||
|
||||
## Closing the Browser
|
||||
|
||||
```bash
|
||||
agent-browser close --all
|
||||
```
|
||||
@@ -324,7 +324,7 @@ int main() {
|
||||
}
|
||||
```
|
||||
|
||||
## Migration Plan for c-relay
|
||||
## Migration Plan for c-relay-pg
|
||||
|
||||
### Phase 1: Extract Debug System
|
||||
1. Create `c_utils_lib` repository
|
||||
@@ -333,16 +333,16 @@ int main() {
|
||||
4. Add basic tests
|
||||
|
||||
### Phase 2: Add Versioning System
|
||||
1. Extract version generation logic from c-relay
|
||||
1. Extract version generation logic from c-relay-pg
|
||||
2. Create reusable version utilities
|
||||
3. Update c-relay to use new system
|
||||
3. Update c-relay-pg to use new system
|
||||
4. Update nostr_core_lib to use new system
|
||||
|
||||
### Phase 3: Add as Submodule
|
||||
1. Add `c_utils_lib` as submodule to c-relay
|
||||
2. Update c-relay Makefile
|
||||
3. Update includes in c-relay source files
|
||||
4. Remove old debug files from c-relay
|
||||
1. Add `c_utils_lib` as submodule to c-relay-pg
|
||||
2. Update c-relay-pg Makefile
|
||||
3. Update includes in c-relay-pg source files
|
||||
4. Remove old debug files from c-relay-pg
|
||||
|
||||
### Phase 4: Documentation & Examples
|
||||
1. Create comprehensive README
|
||||
@@ -352,7 +352,7 @@ int main() {
|
||||
|
||||
## Benefits
|
||||
|
||||
### For c-relay
|
||||
### For c-relay-pg
|
||||
- Cleaner separation of concerns
|
||||
- Reusable utilities across projects
|
||||
- Easier to maintain and test
|
||||
@@ -379,7 +379,7 @@ int main() {
|
||||
- Memory leak detection (valgrind)
|
||||
|
||||
### Integration Tests
|
||||
- Test with real projects (c-relay, nostr_core_lib)
|
||||
- Test with real projects (c-relay-pg, nostr_core_lib)
|
||||
- Cross-platform testing
|
||||
- Performance benchmarks
|
||||
|
||||
@@ -433,7 +433,7 @@ MIT License - permissive and suitable for learning and commercial use.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. ✅ Successfully integrated into c-relay
|
||||
1. ✅ Successfully integrated into c-relay-pg
|
||||
2. ✅ Successfully integrated into nostr_core_lib
|
||||
3. ✅ All tests passing
|
||||
4. ✅ Documentation complete
|
||||
@@ -449,7 +449,7 @@ MIT License - permissive and suitable for learning and commercial use.
|
||||
4. Create build system
|
||||
5. Write tests
|
||||
6. Create documentation
|
||||
7. Integrate into c-relay
|
||||
7. Integrate into c-relay-pg
|
||||
8. Publish to GitHub
|
||||
|
||||
---
|
||||
|
||||
@@ -2,13 +2,13 @@
|
||||
|
||||
## Overview
|
||||
|
||||
This document provides a step-by-step implementation plan for creating the `c_utils_lib` library and integrating it into the c-relay project.
|
||||
This document provides a step-by-step implementation plan for creating the `c_utils_lib` library and integrating it into the c-relay-pg project.
|
||||
|
||||
## Phase 1: Repository Setup & Structure
|
||||
|
||||
### Step 1.1: Create Repository Structure
|
||||
|
||||
**Location**: Create outside c-relay project (sibling directory)
|
||||
**Location**: Create outside c-relay-pg project (sibling directory)
|
||||
|
||||
```bash
|
||||
# Create directory structure
|
||||
@@ -42,7 +42,7 @@ git branch -M main
|
||||
|
||||
### Step 2.1: Move Debug Files
|
||||
|
||||
**Source files** (from c-relay):
|
||||
**Source files** (from c-relay-pg):
|
||||
- `src/debug.c` → `c_utils_lib/src/debug.c`
|
||||
- `src/debug.h` → `c_utils_lib/include/c_utils/debug.h`
|
||||
|
||||
@@ -484,20 +484,20 @@ How to integrate into projects:
|
||||
3. Code examples
|
||||
4. Migration from standalone utilities
|
||||
|
||||
## Phase 7: Integration with c-relay
|
||||
## Phase 7: Integration with c-relay-pg
|
||||
|
||||
### Step 7.1: Add as Submodule
|
||||
|
||||
```bash
|
||||
cd /path/to/c-relay
|
||||
cd /path/to/c-relay-pg
|
||||
git submodule add <repo-url> c_utils_lib
|
||||
git submodule update --init --recursive
|
||||
```
|
||||
|
||||
### Step 7.2: Update c-relay Makefile
|
||||
### Step 7.2: Update c-relay-pg Makefile
|
||||
|
||||
```makefile
|
||||
# Add to c-relay Makefile
|
||||
# Add to c-relay-pg Makefile
|
||||
C_UTILS_LIB = c_utils_lib/libc_utils.a
|
||||
|
||||
# Update includes
|
||||
@@ -514,7 +514,7 @@ $(C_UTILS_LIB):
|
||||
$(TARGET): $(C_UTILS_LIB) ...
|
||||
```
|
||||
|
||||
### Step 7.3: Update c-relay Source Files
|
||||
### Step 7.3: Update c-relay-pg Source Files
|
||||
|
||||
**Changes needed**:
|
||||
|
||||
@@ -543,7 +543,7 @@ $(TARGET): $(C_UTILS_LIB) ...
|
||||
### Step 7.4: Test Integration
|
||||
|
||||
```bash
|
||||
cd c-relay
|
||||
cd c-relay-pg
|
||||
make clean
|
||||
make
|
||||
./make_and_restart_relay.sh
|
||||
@@ -556,7 +556,7 @@ Verify:
|
||||
|
||||
## Phase 8: Version System Integration
|
||||
|
||||
### Step 8.1: Update c-relay Makefile for Versioning
|
||||
### Step 8.1: Update c-relay-pg Makefile for Versioning
|
||||
|
||||
```makefile
|
||||
# Add version generation
|
||||
@@ -567,7 +567,7 @@ src/version.h: .git/refs/tags/*
|
||||
$(TARGET): src/version.h ...
|
||||
```
|
||||
|
||||
### Step 8.2: Update c-relay to Use Generated Version
|
||||
### Step 8.2: Update c-relay-pg to Use Generated Version
|
||||
|
||||
Replace hardcoded version in `src/main.h` with:
|
||||
```c
|
||||
@@ -583,7 +583,7 @@ Replace hardcoded version in `src/main.h` with:
|
||||
- **Phase 4**: Build System - 2 hours
|
||||
- **Phase 5**: Examples & Tests - 3 hours
|
||||
- **Phase 6**: Documentation - 3 hours
|
||||
- **Phase 7**: c-relay Integration - 2 hours
|
||||
- **Phase 7**: c-relay-pg Integration - 2 hours
|
||||
- **Phase 8**: Version Integration - 2 hours
|
||||
|
||||
**Total**: ~19 hours
|
||||
@@ -593,11 +593,11 @@ Replace hardcoded version in `src/main.h` with:
|
||||
- [ ] c_utils_lib builds successfully
|
||||
- [ ] All tests pass
|
||||
- [ ] Examples compile and run
|
||||
- [ ] c-relay integrates successfully
|
||||
- [ ] Debug output works in c-relay
|
||||
- [ ] c-relay-pg integrates successfully
|
||||
- [ ] Debug output works in c-relay-pg
|
||||
- [ ] Version generation works
|
||||
- [ ] Documentation complete
|
||||
- [ ] No regressions in c-relay functionality
|
||||
- [ ] No regressions in c-relay-pg functionality
|
||||
|
||||
## Next Steps
|
||||
|
||||
@@ -608,7 +608,7 @@ Replace hardcoded version in `src/main.h` with:
|
||||
5. Create build system
|
||||
6. Write tests and examples
|
||||
7. Create documentation
|
||||
8. Integrate into c-relay
|
||||
8. Integrate into c-relay-pg
|
||||
9. Test thoroughly
|
||||
10. Publish to GitHub
|
||||
|
||||
|
||||
@@ -78,9 +78,9 @@ Configuration events follow the standard Nostr event format with kind 33334:
|
||||
|
||||
#### `relay_software`
|
||||
- **Description**: Software identifier for NIP-11
|
||||
- **Default**: `"c-relay"`
|
||||
- **Default**: `"c-relay-pg"`
|
||||
- **Format**: String, max 64 characters
|
||||
- **Example**: `"c-relay v1.0.0"`
|
||||
- **Example**: `"c-relay-pg v1.0.0"`
|
||||
|
||||
#### `relay_version`
|
||||
- **Description**: Software version string
|
||||
@@ -366,7 +366,7 @@ sqlite3 relay.nrdb "SELECT json_pretty(json_object(
|
||||
#### Invalid Parameter Values
|
||||
```bash
|
||||
# Check relay logs for validation errors
|
||||
journalctl -u c-relay | grep "Configuration.*invalid\|Invalid.*configuration"
|
||||
journalctl -u c-relay-pg | grep "Configuration.*invalid\|Invalid.*configuration"
|
||||
|
||||
# Common issues:
|
||||
# - Numeric values outside valid ranges
|
||||
@@ -395,10 +395,10 @@ ORDER BY date DESC;"
|
||||
#### Resource Usage After Changes
|
||||
```bash
|
||||
# Monitor system resources after configuration updates
|
||||
top -p $(pgrep c_relay)
|
||||
top -p $(pgrep c_relay_pg)
|
||||
|
||||
# Check for memory leaks
|
||||
ps aux | grep c_relay | awk '{print $6}' # RSS memory
|
||||
ps aux | grep c_relay_pg | awk '{print $6}' # RSS memory
|
||||
```
|
||||
|
||||
### Emergency Recovery
|
||||
@@ -425,7 +425,7 @@ nostrtool event \
|
||||
# If database is corrupted, backup and recreate
|
||||
cp relay.nrdb relay.nrdb.backup
|
||||
rm relay.nrdb*
|
||||
./build/c_relay_x86 # Creates fresh database with new keys
|
||||
./build/c_relay_pg_x86 # Creates fresh database with new keys
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
+14
-14
@@ -21,22 +21,22 @@ typedef enum {
|
||||
|
||||
```bash
|
||||
# Production (default - no debug output)
|
||||
./c_relay_x86
|
||||
./c_relay_pg_x86
|
||||
|
||||
# Show errors only
|
||||
./c_relay_x86 --debug-level=1
|
||||
./c_relay_pg_x86 --debug-level=1
|
||||
|
||||
# Show errors and warnings
|
||||
./c_relay_x86 --debug-level=2
|
||||
./c_relay_pg_x86 --debug-level=2
|
||||
|
||||
# Show errors, warnings, and info (recommended for development)
|
||||
./c_relay_x86 --debug-level=3
|
||||
./c_relay_pg_x86 --debug-level=3
|
||||
|
||||
# Show all debug messages
|
||||
./c_relay_x86 --debug-level=4
|
||||
./c_relay_pg_x86 --debug-level=4
|
||||
|
||||
# Show everything including trace with file:line (very verbose)
|
||||
./c_relay_x86 --debug-level=5
|
||||
./c_relay_pg_x86 --debug-level=5
|
||||
```
|
||||
|
||||
## Implementation
|
||||
@@ -475,7 +475,7 @@ When `g_debug_level = 0` (constant), you'll see the compiler has removed all deb
|
||||
|
||||
### Level 3 (Errors + Warnings + Info)
|
||||
```
|
||||
[2025-01-12 14:30:15] [INFO ] Initializing C-Relay v0.4.6
|
||||
[2025-01-12 14:30:15] [INFO ] Initializing C-Relay-PG v0.4.6
|
||||
[2025-01-12 14:30:15] [INFO ] Loading configuration from database
|
||||
[2025-01-12 14:30:15] [ERROR] Failed to open database: permission denied
|
||||
[2025-01-12 14:30:16] [WARN ] Port 8888 unavailable, trying 8889
|
||||
@@ -484,7 +484,7 @@ When `g_debug_level = 0` (constant), you'll see the compiler has removed all deb
|
||||
|
||||
### Level 4 (All Debug Messages)
|
||||
```
|
||||
[2025-01-12 14:30:15] [INFO ] Initializing C-Relay v0.4.6
|
||||
[2025-01-12 14:30:15] [INFO ] Initializing C-Relay-PG v0.4.6
|
||||
[2025-01-12 14:30:15] [DEBUG] Opening database: build/abc123...def.db
|
||||
[2025-01-12 14:30:15] [DEBUG] Executing schema initialization
|
||||
[2025-01-12 14:30:15] [INFO ] SQLite WAL mode enabled
|
||||
@@ -496,7 +496,7 @@ When `g_debug_level = 0` (constant), you'll see the compiler has removed all deb
|
||||
|
||||
### Level 5 (Everything Including file:line for ALL messages)
|
||||
```
|
||||
[2025-01-12 14:30:15] [INFO ] [main.c:1607] Initializing C-Relay v0.4.6
|
||||
[2025-01-12 14:30:15] [INFO ] [main.c:1607] Initializing C-Relay-PG v0.4.6
|
||||
[2025-01-12 14:30:15] [DEBUG] [main.c:348] Opening database: build/abc123...def.db
|
||||
[2025-01-12 14:30:15] [TRACE] [main.c:330] Entering init_database()
|
||||
[2025-01-12 14:30:15] [ERROR] [config.c:125] Database locked
|
||||
@@ -525,11 +525,11 @@ Update the existing `log_*` functions to use the new debug macros
|
||||
make clean && make
|
||||
|
||||
# Test different levels
|
||||
./build/c_relay_x86 # No output
|
||||
./build/c_relay_x86 --debug-level=1 # Errors only
|
||||
./build/c_relay_x86 --debug-level=3 # Info + warnings + errors
|
||||
./build/c_relay_x86 --debug-level=4 # All debug messages
|
||||
./build/c_relay_x86 --debug-level=5 # Everything with file:line on TRACE
|
||||
./build/c_relay_pg_x86 # No output
|
||||
./build/c_relay_pg_x86 --debug-level=1 # Errors only
|
||||
./build/c_relay_pg_x86 --debug-level=3 # Info + warnings + errors
|
||||
./build/c_relay_pg_x86 --debug-level=4 # All debug messages
|
||||
./build/c_relay_pg_x86 --debug-level=5 # Everything with file:line on TRACE
|
||||
```
|
||||
|
||||
### Step 5: Gradual Migration (Ongoing)
|
||||
|
||||
@@ -34,7 +34,7 @@ static const struct {
|
||||
// NIP-11 Relay Information (relay keys will be populated at runtime)
|
||||
{"relay_description", "High-performance C Nostr relay with SQLite storage"},
|
||||
{"relay_contact", ""},
|
||||
{"relay_software", "https://git.laantungir.net/laantungir/c-relay.git"},
|
||||
{"relay_software", "https://git.laantungir.net/laantungir/c-relay-pg.git"},
|
||||
{"relay_version", "v1.0.0"},
|
||||
|
||||
// NIP-13 Proof of Work (pow_min_difficulty = 0 means PoW disabled)
|
||||
|
||||
+51
-51
@@ -52,11 +52,11 @@ sudo apt install -y build-essential git sqlite3 libsqlite3-dev \
|
||||
#### User and Directory Setup
|
||||
```bash
|
||||
# Create dedicated system user
|
||||
sudo useradd --system --home-dir /opt/c-relay --shell /bin/false c-relay
|
||||
sudo useradd --system --home-dir /opt/c-relay-pg --shell /bin/false c-relay-pg
|
||||
|
||||
# Create application directory
|
||||
sudo mkdir -p /opt/c-relay
|
||||
sudo chown c-relay:c-relay /opt/c-relay
|
||||
sudo mkdir -p /opt/c-relay-pg
|
||||
sudo chown c-relay-pg:c-relay-pg /opt/c-relay-pg
|
||||
```
|
||||
|
||||
### Build and Installation
|
||||
@@ -64,8 +64,8 @@ sudo chown c-relay:c-relay /opt/c-relay
|
||||
#### Automated Installation (Recommended)
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
git submodule update --init --recursive
|
||||
|
||||
# Build
|
||||
@@ -81,12 +81,12 @@ sudo systemd/install-service.sh
|
||||
make clean && make
|
||||
|
||||
# Install binary
|
||||
sudo cp build/c_relay_x86 /opt/c-relay/
|
||||
sudo chown c-relay:c-relay /opt/c-relay/c_relay_x86
|
||||
sudo chmod +x /opt/c-relay/c_relay_x86
|
||||
sudo cp build/c_relay_pg_x86 /opt/c-relay-pg/
|
||||
sudo chown c-relay-pg:c-relay-pg /opt/c-relay-pg/c_relay_pg_x86
|
||||
sudo chmod +x /opt/c-relay-pg/c_relay_pg_x86
|
||||
|
||||
# Install systemd service
|
||||
sudo cp systemd/c-relay.service /etc/systemd/system/
|
||||
sudo cp systemd/c-relay-pg.service /etc/systemd/system/
|
||||
sudo systemctl daemon-reload
|
||||
```
|
||||
|
||||
@@ -95,22 +95,22 @@ sudo systemctl daemon-reload
|
||||
#### Start and Enable Service
|
||||
```bash
|
||||
# Start the service
|
||||
sudo systemctl start c-relay
|
||||
sudo systemctl start c-relay-pg
|
||||
|
||||
# Enable auto-start on boot
|
||||
sudo systemctl enable c-relay
|
||||
sudo systemctl enable c-relay-pg
|
||||
|
||||
# Check status
|
||||
sudo systemctl status c-relay
|
||||
sudo systemctl status c-relay-pg
|
||||
```
|
||||
|
||||
#### Capture Admin Keys (CRITICAL)
|
||||
```bash
|
||||
# View startup logs to get admin keys
|
||||
sudo journalctl -u c-relay --since "5 minutes ago" | grep -A 10 "IMPORTANT: SAVE THIS ADMIN PRIVATE KEY"
|
||||
sudo journalctl -u c-relay-pg --since "5 minutes ago" | grep -A 10 "IMPORTANT: SAVE THIS ADMIN PRIVATE KEY"
|
||||
|
||||
# Or check the full log
|
||||
sudo journalctl -u c-relay --no-pager | grep "Admin Private Key"
|
||||
sudo journalctl -u c-relay-pg --no-pager | grep "Admin Private Key"
|
||||
```
|
||||
|
||||
⚠️ **CRITICAL**: Save the admin private key immediately - it's only shown once and is needed for all configuration updates!
|
||||
@@ -151,8 +151,8 @@ sudo iptables-save > /etc/iptables/rules.v4
|
||||
ssh -i your-key.pem ubuntu@your-instance-ip
|
||||
|
||||
# Use the simple deployment script
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
sudo examples/deployment/simple-vps/deploy.sh
|
||||
```
|
||||
|
||||
@@ -168,10 +168,10 @@ sudo examples/deployment/simple-vps/deploy.sh
|
||||
sudo mkfs.ext4 /dev/xvdf
|
||||
sudo mkdir /data
|
||||
sudo mount /dev/xvdf /data
|
||||
sudo chown c-relay:c-relay /data
|
||||
sudo chown c-relay-pg:c-relay-pg /data
|
||||
|
||||
# Update systemd service to use /data
|
||||
sudo sed -i 's/WorkingDirectory=\/opt\/c-relay/WorkingDirectory=\/data/' /etc/systemd/system/c-relay.service
|
||||
sudo sed -i 's/WorkingDirectory=\/opt\/c-relay-pg/WorkingDirectory=\/data/' /etc/systemd/system/c-relay-pg.service
|
||||
sudo systemctl daemon-reload
|
||||
```
|
||||
|
||||
@@ -180,7 +180,7 @@ sudo systemctl daemon-reload
|
||||
#### Compute Engine Setup
|
||||
```bash
|
||||
# Create VM instance (e2-micro or larger)
|
||||
gcloud compute instances create c-relay-instance \
|
||||
gcloud compute instances create c-relay-pg-instance \
|
||||
--image-family=ubuntu-2204-lts \
|
||||
--image-project=ubuntu-os-cloud \
|
||||
--machine-type=e2-micro \
|
||||
@@ -193,9 +193,9 @@ gcloud compute firewall-rules create allow-nostr-relay \
|
||||
--target-tags nostr-relay
|
||||
|
||||
# SSH and deploy
|
||||
gcloud compute ssh c-relay-instance
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
gcloud compute ssh c-relay-pg-instance
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
sudo examples/deployment/simple-vps/deploy.sh
|
||||
```
|
||||
|
||||
@@ -203,13 +203,13 @@ sudo examples/deployment/simple-vps/deploy.sh
|
||||
```bash
|
||||
# Create and attach persistent disk
|
||||
gcloud compute disks create relay-data --size=50GB
|
||||
gcloud compute instances attach-disk c-relay-instance --disk=relay-data
|
||||
gcloud compute instances attach-disk c-relay-pg-instance --disk=relay-data
|
||||
|
||||
# Format and mount
|
||||
sudo mkfs.ext4 /dev/sdb
|
||||
sudo mkdir /data
|
||||
sudo mount /dev/sdb /data
|
||||
sudo chown c-relay:c-relay /data
|
||||
sudo chown c-relay-pg:c-relay-pg /data
|
||||
```
|
||||
|
||||
### DigitalOcean
|
||||
@@ -223,8 +223,8 @@ sudo chown c-relay:c-relay /data
|
||||
ssh root@your-droplet-ip
|
||||
|
||||
# Deploy relay
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
examples/deployment/simple-vps/deploy.sh
|
||||
```
|
||||
|
||||
@@ -245,8 +245,8 @@ The `examples/deployment/` directory contains ready-to-use scripts:
|
||||
### Simple VPS Deployment
|
||||
```bash
|
||||
# Clone repository and run automated deployment
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
sudo examples/deployment/simple-vps/deploy.sh
|
||||
```
|
||||
|
||||
@@ -417,9 +417,9 @@ LOG_FILE="/var/log/relay-monitor.log"
|
||||
DATE=$(date '+%Y-%m-%d %H:%M:%S')
|
||||
|
||||
# Check if relay is running
|
||||
if ! pgrep -f "c_relay_x86" > /dev/null; then
|
||||
if ! pgrep -f "c_relay_pg_x86" > /dev/null; then
|
||||
echo "[$DATE] ERROR: Relay process not running" >> $LOG_FILE
|
||||
systemctl restart c-relay
|
||||
systemctl restart c-relay-pg
|
||||
fi
|
||||
|
||||
# Check port availability
|
||||
@@ -428,14 +428,14 @@ if ! netstat -tln | grep -q ":8888"; then
|
||||
fi
|
||||
|
||||
# Check database file
|
||||
RELAY_DB=$(find /opt/c-relay -name "*.nrdb" | head -1)
|
||||
RELAY_DB=$(find /opt/c-relay-pg -name "*.nrdb" | head -1)
|
||||
if [[ -n "$RELAY_DB" ]]; then
|
||||
DB_SIZE=$(du -h "$RELAY_DB" | cut -f1)
|
||||
echo "[$DATE] INFO: Database size: $DB_SIZE" >> $LOG_FILE
|
||||
fi
|
||||
|
||||
# Check memory usage
|
||||
MEM_USAGE=$(ps aux | grep c_relay_x86 | grep -v grep | awk '{print $6}')
|
||||
MEM_USAGE=$(ps aux | grep c_relay_pg_x86 | grep -v grep | awk '{print $6}')
|
||||
if [[ -n "$MEM_USAGE" ]]; then
|
||||
echo "[$DATE] INFO: Memory usage: ${MEM_USAGE}KB" >> $LOG_FILE
|
||||
fi
|
||||
@@ -454,8 +454,8 @@ sudo chmod +x /usr/local/bin/relay-monitor.sh
|
||||
|
||||
#### Centralized Logging with rsyslog
|
||||
```bash
|
||||
# /etc/rsyslog.d/50-c-relay.conf
|
||||
if $programname == 'c-relay' then /var/log/c-relay.log
|
||||
# /etc/rsyslog.d/50-c-relay-pg.conf
|
||||
if $programname == 'c-relay-pg' then /var/log/c-relay-pg.log
|
||||
& stop
|
||||
```
|
||||
|
||||
@@ -465,7 +465,7 @@ if $programname == 'c-relay' then /var/log/c-relay.log
|
||||
```yaml
|
||||
# /etc/prometheus/prometheus.yml
|
||||
scrape_configs:
|
||||
- job_name: 'c-relay'
|
||||
- job_name: 'c-relay-pg'
|
||||
static_configs:
|
||||
- targets: ['localhost:8888']
|
||||
metrics_path: '/metrics' # If implemented
|
||||
@@ -479,30 +479,30 @@ scrape_configs:
|
||||
#### Service User Restrictions
|
||||
```bash
|
||||
# Restrict service user
|
||||
sudo usermod -s /bin/false c-relay
|
||||
sudo usermod -d /opt/c-relay c-relay
|
||||
sudo usermod -s /bin/false c-relay-pg
|
||||
sudo usermod -d /opt/c-relay-pg c-relay-pg
|
||||
|
||||
# Set proper permissions
|
||||
sudo chmod 700 /opt/c-relay
|
||||
sudo chown -R c-relay:c-relay /opt/c-relay
|
||||
sudo chmod 700 /opt/c-relay-pg
|
||||
sudo chown -R c-relay-pg:c-relay-pg /opt/c-relay-pg
|
||||
```
|
||||
|
||||
#### File System Restrictions
|
||||
```bash
|
||||
# Mount data directory with appropriate options
|
||||
echo "/dev/sdb /opt/c-relay ext4 defaults,noexec,nosuid,nodev 0 2" >> /etc/fstab
|
||||
echo "/dev/sdb /opt/c-relay-pg ext4 defaults,noexec,nosuid,nodev 0 2" >> /etc/fstab
|
||||
```
|
||||
|
||||
### Network Security
|
||||
|
||||
#### Fail2Ban Configuration
|
||||
```ini
|
||||
# /etc/fail2ban/jail.d/c-relay.conf
|
||||
[c-relay-dos]
|
||||
# /etc/fail2ban/jail.d/c-relay-pg.conf
|
||||
[c-relay-pg-dos]
|
||||
enabled = true
|
||||
port = 8888
|
||||
filter = c-relay-dos
|
||||
logpath = /var/log/c-relay.log
|
||||
filter = c-relay-pg-dos
|
||||
logpath = /var/log/c-relay-pg.log
|
||||
maxretry = 10
|
||||
findtime = 60
|
||||
bantime = 300
|
||||
@@ -534,9 +534,9 @@ sudo mkfs.ext4 /dev/mapper/relay-data
|
||||
#!/bin/bash
|
||||
# /usr/local/bin/backup-relay.sh
|
||||
|
||||
BACKUP_DIR="/backup/c-relay"
|
||||
BACKUP_DIR="/backup/c-relay-pg"
|
||||
DATE=$(date +%Y%m%d_%H%M%S)
|
||||
RELAY_DB=$(find /opt/c-relay -name "*.nrdb" | head -1)
|
||||
RELAY_DB=$(find /opt/c-relay-pg -name "*.nrdb" | head -1)
|
||||
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
|
||||
@@ -574,7 +574,7 @@ sudo apt install -y awscli
|
||||
aws configure
|
||||
|
||||
# Sync backups to S3
|
||||
aws s3 sync /backup/c-relay/ s3://your-backup-bucket/c-relay/ --delete
|
||||
aws s3 sync /backup/c-relay-pg/ s3://your-backup-bucket/c-relay-pg/ --delete
|
||||
```
|
||||
|
||||
### Disaster Recovery
|
||||
@@ -583,16 +583,16 @@ aws s3 sync /backup/c-relay/ s3://your-backup-bucket/c-relay/ --delete
|
||||
```bash
|
||||
# 1. Restore from backup
|
||||
gunzip backup/relay_backup_20231201_020000.nrdb.gz
|
||||
cp backup/relay_backup_20231201_020000.nrdb /opt/c-relay/
|
||||
cp backup/relay_backup_20231201_020000.nrdb /opt/c-relay-pg/
|
||||
|
||||
# 2. Fix permissions
|
||||
sudo chown c-relay:c-relay /opt/c-relay/*.nrdb
|
||||
sudo chown c-relay-pg:c-relay-pg /opt/c-relay-pg/*.nrdb
|
||||
|
||||
# 3. Restart service
|
||||
sudo systemctl restart c-relay
|
||||
sudo systemctl restart c-relay-pg
|
||||
|
||||
# 4. Verify recovery
|
||||
sudo journalctl -u c-relay --since "1 minute ago"
|
||||
sudo journalctl -u c-relay-pg --since "1 minute ago"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
+25
-25
@@ -2,7 +2,7 @@
|
||||
|
||||
## Overview
|
||||
|
||||
This guide explains how to build truly portable MUSL-based static binaries of c-relay using Alpine Linux Docker containers. These binaries have **zero runtime dependencies** and work on any Linux distribution.
|
||||
This guide explains how to build truly portable MUSL-based static binaries of c-relay-pg using Alpine Linux Docker containers. These binaries have **zero runtime dependencies** and work on any Linux distribution.
|
||||
|
||||
## Why MUSL?
|
||||
|
||||
@@ -36,8 +36,8 @@ This guide explains how to build truly portable MUSL-based static binaries of c-
|
||||
./build_static.sh
|
||||
|
||||
# The binary will be created at:
|
||||
# build/c_relay_static_musl_x86_64 (on x86_64)
|
||||
# build/c_relay_static_musl_arm64 (on ARM64)
|
||||
# build/c_relay_pg_static_musl_x86_64 (on x86_64)
|
||||
# build/c_relay_pg_static_musl_arm64 (on ARM64)
|
||||
```
|
||||
|
||||
### What Happens During Build
|
||||
@@ -56,7 +56,7 @@ This guide explains how to build truly portable MUSL-based static binaries of c-
|
||||
- Includes required NIPs: 001, 006, 013, 017, 019, 044, 059
|
||||
- Produces static library (~316KB)
|
||||
|
||||
4. **c-relay Compilation**: Links everything statically:
|
||||
4. **c-relay-pg Compilation**: Links everything statically:
|
||||
- All source files compiled with `-static` flag
|
||||
- Fortification disabled to avoid `__*_chk` symbols
|
||||
- Results in ~7.6MB stripped binary
|
||||
@@ -78,7 +78,7 @@ FROM alpine:3.19 AS builder
|
||||
- Install build tools and static libraries
|
||||
- Build dependencies from source
|
||||
- Compile nostr_core_lib with MUSL flags
|
||||
- Compile c-relay with full static linking
|
||||
- Compile c-relay-pg with full static linking
|
||||
- Strip binary to reduce size
|
||||
|
||||
# Stage 2: Output (scratch)
|
||||
@@ -93,7 +93,7 @@ FROM scratch AS output
|
||||
CFLAGS="-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -Wall -Wextra -std=c99 -fPIC -O2"
|
||||
```
|
||||
|
||||
**For c-relay:**
|
||||
**For c-relay-pg:**
|
||||
```bash
|
||||
gcc -static -O2 -Wall -Wextra -std=c99 \
|
||||
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 \
|
||||
@@ -110,7 +110,7 @@ gcc -static -O2 -Wall -Wextra -std=c99 \
|
||||
- `-U_FORTIFY_SOURCE` (undefine any existing definition)
|
||||
- `-D_FORTIFY_SOURCE=0` (set to 0)
|
||||
|
||||
This must be applied to **both** nostr_core_lib and c-relay compilation.
|
||||
This must be applied to **both** nostr_core_lib and c-relay-pg compilation.
|
||||
|
||||
### NIP Dependencies
|
||||
|
||||
@@ -129,26 +129,26 @@ The build includes these NIPs in nostr_core_lib:
|
||||
|
||||
```bash
|
||||
# Should show "statically linked"
|
||||
file build/c_relay_static_musl_x86_64
|
||||
file build/c_relay_pg_static_musl_x86_64
|
||||
|
||||
# Should show "not a dynamic executable"
|
||||
ldd build/c_relay_static_musl_x86_64
|
||||
ldd build/c_relay_pg_static_musl_x86_64
|
||||
|
||||
# Check size (should be ~7.6MB)
|
||||
ls -lh build/c_relay_static_musl_x86_64
|
||||
ls -lh build/c_relay_pg_static_musl_x86_64
|
||||
```
|
||||
|
||||
### Test Execution
|
||||
|
||||
```bash
|
||||
# Show help
|
||||
./build/c_relay_static_musl_x86_64 --help
|
||||
./build/c_relay_pg_static_musl_x86_64 --help
|
||||
|
||||
# Show version
|
||||
./build/c_relay_static_musl_x86_64 --version
|
||||
./build/c_relay_pg_static_musl_x86_64 --version
|
||||
|
||||
# Run relay
|
||||
./build/c_relay_static_musl_x86_64 --port 8888
|
||||
./build/c_relay_pg_static_musl_x86_64 --port 8888
|
||||
```
|
||||
|
||||
### Cross-Distribution Testing
|
||||
@@ -157,16 +157,16 @@ Test the binary on different distributions to verify portability:
|
||||
|
||||
```bash
|
||||
# Alpine Linux
|
||||
docker run --rm -v $(pwd)/build:/app alpine:latest /app/c_relay_static_musl_x86_64 --version
|
||||
docker run --rm -v $(pwd)/build:/app alpine:latest /app/c_relay_pg_static_musl_x86_64 --version
|
||||
|
||||
# Ubuntu
|
||||
docker run --rm -v $(pwd)/build:/app ubuntu:latest /app/c_relay_static_musl_x86_64 --version
|
||||
docker run --rm -v $(pwd)/build:/app ubuntu:latest /app/c_relay_pg_static_musl_x86_64 --version
|
||||
|
||||
# Debian
|
||||
docker run --rm -v $(pwd)/build:/app debian:latest /app/c_relay_static_musl_x86_64 --version
|
||||
docker run --rm -v $(pwd)/build:/app debian:latest /app/c_relay_pg_static_musl_x86_64 --version
|
||||
|
||||
# CentOS
|
||||
docker run --rm -v $(pwd)/build:/app centos:latest /app/c_relay_static_musl_x86_64 --version
|
||||
docker run --rm -v $(pwd)/build:/app centos:latest /app/c_relay_pg_static_musl_x86_64 --version
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
@@ -187,7 +187,7 @@ newgrp docker # Or logout and login again
|
||||
|
||||
**Solution**: Ensure fortification is disabled in both:
|
||||
1. nostr_core_lib build.sh (line 534)
|
||||
2. c-relay compilation flags in Dockerfile
|
||||
2. c-relay-pg compilation flags in Dockerfile
|
||||
|
||||
### Binary Won't Execute
|
||||
|
||||
@@ -213,26 +213,26 @@ newgrp docker # Or logout and login again
|
||||
|
||||
```bash
|
||||
# Copy binary to server
|
||||
scp build/c_relay_static_musl_x86_64 user@server:/opt/c-relay/
|
||||
scp build/c_relay_pg_static_musl_x86_64 user@server:/opt/c-relay-pg/
|
||||
|
||||
# Run on server (no dependencies needed!)
|
||||
ssh user@server
|
||||
cd /opt/c-relay
|
||||
./c_relay_static_musl_x86_64 --port 8888
|
||||
cd /opt/c-relay-pg
|
||||
./c_relay_pg_static_musl_x86_64 --port 8888
|
||||
```
|
||||
|
||||
### SystemD Service
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=C-Relay Nostr Relay (MUSL Static)
|
||||
Description=C-Relay-PG Nostr Relay (MUSL Static)
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=c-relay
|
||||
WorkingDirectory=/opt/c-relay
|
||||
ExecStart=/opt/c-relay/c_relay_static_musl_x86_64
|
||||
User=c-relay-pg
|
||||
WorkingDirectory=/opt/c-relay-pg
|
||||
ExecStart=/opt/c-relay-pg/c_relay_pg_static_musl_x86_64
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
|
||||
@@ -23,7 +23,7 @@ We chose Option B because:
|
||||
|
||||
## Detailed Implementation Steps
|
||||
|
||||
### Phase 1: Configuration Setup in c-relay
|
||||
### Phase 1: Configuration Setup in c-relay-pg
|
||||
|
||||
#### 1.1 Add Configuration Parameter
|
||||
**File:** `src/default_config_event.h`
|
||||
@@ -293,7 +293,7 @@ int nostr_nip17_send_dm(cJSON* dm_event,
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Update c-relay Call Sites
|
||||
### Phase 4: Update c-relay-pg Call Sites
|
||||
|
||||
#### 4.1 Update src/api.c
|
||||
**Location:** Line 1319
|
||||
@@ -470,7 +470,7 @@ causing compatibility issues.
|
||||
- [ ] Update `nostr_nip17_send_dm()` signature and implementation
|
||||
- [ ] Update `nip017.h` function declaration and documentation
|
||||
|
||||
### c-relay Changes
|
||||
### c-relay-pg Changes
|
||||
- [ ] Add `nip59_timestamp_max_delay_sec` to `default_config_event.h`
|
||||
- [ ] Add validation in `config.c` for new parameter
|
||||
- [ ] Update `src/api.c` call site to pass `max_delay_sec`
|
||||
@@ -494,7 +494,7 @@ causing compatibility issues.
|
||||
|
||||
If issues arise:
|
||||
1. Revert nostr_core_lib changes (git revert in submodule)
|
||||
2. Revert c-relay changes
|
||||
2. Revert c-relay-pg changes
|
||||
3. Configuration parameter will be ignored if not used
|
||||
4. Default behavior (0) provides maximum compatibility
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## Overview
|
||||
|
||||
This document describes the design for a general-purpose SQL query interface for the C-Relay admin API. This allows administrators to execute read-only SQL queries against the relay database through cryptographically signed kind 23456 events with NIP-44 encrypted command arrays.
|
||||
This document describes the design for a general-purpose SQL query interface for the C-Relay-PG admin API. This allows administrators to execute read-only SQL queries against the relay database through cryptographically signed kind 23456 events with NIP-44 encrypted command arrays.
|
||||
|
||||
## Security Model
|
||||
|
||||
@@ -271,7 +271,7 @@ const SQL_QUERY_TEMPLATES = {
|
||||
};
|
||||
|
||||
// Query history management (localStorage)
|
||||
const QUERY_HISTORY_KEY = 'c_relay_sql_history';
|
||||
const QUERY_HISTORY_KEY = 'c_relay_pg_sql_history';
|
||||
const MAX_HISTORY_ITEMS = 20;
|
||||
|
||||
// Load query history from localStorage
|
||||
|
||||
@@ -224,7 +224,7 @@ The script should:
|
||||
- `curl` or `websocat` for WebSocket communication
|
||||
- `jq` for JSON parsing
|
||||
- Nostr CLI tools (optional, for event signing)
|
||||
- Running c-relay instance
|
||||
- Running c-relay-pg instance
|
||||
|
||||
## Example Output
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# C-Relay Complete Startup Flow Documentation
|
||||
# C-Relay-PG Complete Startup Flow Documentation
|
||||
|
||||
## Overview
|
||||
|
||||
C-Relay has two distinct startup paths:
|
||||
C-Relay-PG has two distinct startup paths:
|
||||
1. **First-Time Startup**: No database exists, generates keys and initializes system
|
||||
2. **Existing Relay Startup**: Database exists, loads configuration and resumes operation
|
||||
|
||||
@@ -220,7 +220,7 @@ int populate_default_config_values(void) {
|
||||
|
||||
struct config_default defaults[] = {
|
||||
{"relay_port", "8888", "integer", "WebSocket port", "network", 1},
|
||||
{"relay_name", "C-Relay", "string", "Relay name", "info", 0},
|
||||
{"relay_name", "C-Relay-PG", "string", "Relay name", "info", 0},
|
||||
{"relay_description", "High-performance C Nostr relay", "string", "Description", "info", 0},
|
||||
{"max_subscriptions_per_client", "25", "integer", "Max subs per client", "limits", 0},
|
||||
{"pow_min_difficulty", "0", "integer", "Minimum PoW difficulty", "security", 0},
|
||||
@@ -983,7 +983,7 @@ if (current_version < LATEST_VERSION) {
|
||||
|
||||
**Solutions**:
|
||||
- Use `--port <number>` to specify different port
|
||||
- Kill existing process: `pkill -f c_relay_`
|
||||
- Kill existing process: `pkill -f c_relay_pg_`
|
||||
- Force kill port: `fuser -k 8888/tcp`
|
||||
- Use `--strict-port` to fail fast instead of trying fallback ports
|
||||
|
||||
@@ -994,7 +994,7 @@ if (current_version < LATEST_VERSION) {
|
||||
**Solutions**:
|
||||
- Kill existing relay processes
|
||||
- Remove WAL files: `rm build/*.db-wal build/*.db-shm`
|
||||
- Check for stale processes: `ps aux | grep c_relay_`
|
||||
- Check for stale processes: `ps aux | grep c_relay_pg_`
|
||||
|
||||
#### 3. Missing Admin Private Key
|
||||
|
||||
@@ -1071,7 +1071,7 @@ Typical startup times (on modern hardware):
|
||||
|
||||
## Summary
|
||||
|
||||
The c-relay startup system is designed for:
|
||||
The c-relay-pg startup system is designed for:
|
||||
|
||||
1. **Security**: Admin keys never stored, relay keys encrypted
|
||||
2. **Reliability**: Automatic port fallback, schema migrations
|
||||
|
||||
@@ -18,10 +18,10 @@ The script now attempts to build with `musl-gcc` for truly portable static binar
|
||||
|
||||
SQLite is now built once and cached for future builds:
|
||||
|
||||
- **Cache location**: `~/.cache/c-relay-sqlite/`
|
||||
- **Cache location**: `~/.cache/c-relay-pg-sqlite/`
|
||||
- **Version-specific**: Each SQLite version gets its own cache directory
|
||||
- **Significant speedup**: Subsequent builds skip the SQLite compilation step
|
||||
- **Manual cleanup**: `rm -rf ~/.cache/c-relay-sqlite` to clear cache
|
||||
- **Manual cleanup**: `rm -rf ~/.cache/c-relay-pg-sqlite` to clear cache
|
||||
|
||||
### 3. Smart Package Installation
|
||||
|
||||
@@ -51,13 +51,13 @@ The script will:
|
||||
## Binary Types
|
||||
|
||||
### MUSL Static Binary (Ideal - Currently Not Achievable)
|
||||
- **Filename**: `build/c_relay_static_musl_x86_64`
|
||||
- **Filename**: `build/c_relay_pg_static_musl_x86_64`
|
||||
- **Dependencies**: None (truly static)
|
||||
- **Portability**: Works on any Linux distribution
|
||||
- **Status**: Requires MUSL-compiled libwebsockets and other dependencies (not available by default)
|
||||
|
||||
### Glibc Static Binary (Current Output)
|
||||
- **Filename**: `build/c_relay_static_x86_64` or `build/c_relay_static_glibc_x86_64`
|
||||
- **Filename**: `build/c_relay_pg_static_x86_64` or `build/c_relay_pg_static_glibc_x86_64`
|
||||
- **Dependencies**: None - fully statically linked with glibc
|
||||
- **Portability**: Works on most Linux distributions (glibc is statically included)
|
||||
- **Note**: Despite using glibc, this is a **fully static binary** with no runtime dependencies
|
||||
@@ -68,11 +68,11 @@ The script automatically verifies binaries using `ldd` and `file`:
|
||||
|
||||
```bash
|
||||
# For MUSL binary
|
||||
ldd build/c_relay_static_musl_x86_64
|
||||
ldd build/c_relay_pg_static_musl_x86_64
|
||||
# Output: "not a dynamic executable" (good!)
|
||||
|
||||
# For glibc binary
|
||||
ldd build/c_relay_static_glibc_x86_64
|
||||
ldd build/c_relay_pg_static_glibc_x86_64
|
||||
# Output: Shows glibc dependencies
|
||||
```
|
||||
|
||||
@@ -112,7 +112,7 @@ The script attempts MUSL compilation but falls back to glibc:
|
||||
|
||||
### Clear SQLite Cache
|
||||
```bash
|
||||
rm -rf ~/.cache/c-relay-sqlite
|
||||
rm -rf ~/.cache/c-relay-pg-sqlite
|
||||
```
|
||||
|
||||
### Force Package Reinstall
|
||||
@@ -127,8 +127,8 @@ cat /tmp/musl_build.log
|
||||
|
||||
### Verify Binary Type
|
||||
```bash
|
||||
file build/c_relay_static_*
|
||||
ldd build/c_relay_static_* 2>&1
|
||||
file build/c_relay_pg_static_*
|
||||
ldd build/c_relay_pg_static_* 2>&1
|
||||
```
|
||||
|
||||
## Performance Impact
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## Problem Summary
|
||||
|
||||
The c-relay Nostr relay experienced severe performance degradation (90-100% CPU) due to subscription accumulation in the database. Investigation revealed **323,644 orphaned subscriptions** that were never properly closed when WebSocket connections dropped.
|
||||
The c-relay-pg Nostr relay experienced severe performance degradation (90-100% CPU) due to subscription accumulation in the database. Investigation revealed **323,644 orphaned subscriptions** that were never properly closed when WebSocket connections dropped.
|
||||
|
||||
## Solution: Two-Component Approach
|
||||
|
||||
|
||||
@@ -586,7 +586,7 @@ int add_pubkeys_to_config_table(void) {
|
||||
rm -f *.db
|
||||
|
||||
# Start relay with defaults
|
||||
./build/c_relay_x86
|
||||
./build/c_relay_pg_x86
|
||||
|
||||
# Verify config table complete
|
||||
sqlite3 <relay_pubkey>.db "SELECT COUNT(*) FROM config;"
|
||||
@@ -602,7 +602,7 @@ int add_pubkeys_to_config_table(void) {
|
||||
rm -f *.db
|
||||
|
||||
# Start relay with port override
|
||||
./build/c_relay_x86 --port 9999
|
||||
./build/c_relay_pg_x86 --port 9999
|
||||
|
||||
# Verify port override applied
|
||||
sqlite3 <relay_pubkey>.db "SELECT value FROM config WHERE key='relay_port';"
|
||||
@@ -612,13 +612,13 @@ int add_pubkeys_to_config_table(void) {
|
||||
3. **Restart with Existing Database**
|
||||
```bash
|
||||
# Start relay (creates database)
|
||||
./build/c_relay_x86
|
||||
./build/c_relay_pg_x86
|
||||
|
||||
# Stop relay
|
||||
pkill -f c_relay_
|
||||
pkill -f c_relay_pg_
|
||||
|
||||
# Restart relay
|
||||
./build/c_relay_x86
|
||||
./build/c_relay_pg_x86
|
||||
|
||||
# Verify config unchanged
|
||||
# Check relay.log for validation message
|
||||
@@ -627,13 +627,13 @@ int add_pubkeys_to_config_table(void) {
|
||||
4. **Restart with CLI Overrides**
|
||||
```bash
|
||||
# Start relay (creates database)
|
||||
./build/c_relay_x86
|
||||
./build/c_relay_pg_x86
|
||||
|
||||
# Stop relay
|
||||
pkill -f c_relay_
|
||||
pkill -f c_relay_pg_
|
||||
|
||||
# Restart with port override
|
||||
./build/c_relay_x86 --port 9999
|
||||
./build/c_relay_pg_x86 --port 9999
|
||||
|
||||
# Verify port override applied
|
||||
sqlite3 <relay_pubkey>.db "SELECT value FROM config WHERE key='relay_port';"
|
||||
|
||||
+32
-32
@@ -19,12 +19,12 @@ Complete guide for deploying, configuring, and managing the C Nostr Relay with e
|
||||
```bash
|
||||
# Clone and build
|
||||
git clone <repository-url>
|
||||
cd c-relay
|
||||
cd c-relay-pg
|
||||
git submodule update --init --recursive
|
||||
make
|
||||
|
||||
# Start relay (zero configuration needed)
|
||||
./build/c_relay_x86
|
||||
./build/c_relay_pg_x86
|
||||
```
|
||||
|
||||
### 2. First Startup - Save Keys
|
||||
@@ -78,7 +78,7 @@ brew install git sqlite libwebsockets openssl libsecp256k1 curl zlib
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone <repository-url>
|
||||
cd c-relay
|
||||
cd c-relay-pg
|
||||
|
||||
# Initialize submodules
|
||||
git submodule update --init --recursive
|
||||
@@ -87,7 +87,7 @@ git submodule update --init --recursive
|
||||
make clean && make
|
||||
|
||||
# Verify build
|
||||
ls -la build/c_relay_x86
|
||||
ls -la build/c_relay_pg_x86
|
||||
```
|
||||
|
||||
### Production Deployment
|
||||
@@ -98,27 +98,27 @@ ls -la build/c_relay_x86
|
||||
sudo systemd/install-service.sh
|
||||
|
||||
# Start service
|
||||
sudo systemctl start c-relay
|
||||
sudo systemctl start c-relay-pg
|
||||
|
||||
# Enable auto-start
|
||||
sudo systemctl enable c-relay
|
||||
sudo systemctl enable c-relay-pg
|
||||
|
||||
# Check status
|
||||
sudo systemctl status c-relay
|
||||
sudo systemctl status c-relay-pg
|
||||
```
|
||||
|
||||
#### Manual Deployment
|
||||
```bash
|
||||
# Create dedicated user
|
||||
sudo useradd --system --home-dir /opt/c-relay --shell /bin/false c-relay
|
||||
sudo useradd --system --home-dir /opt/c-relay-pg --shell /bin/false c-relay-pg
|
||||
|
||||
# Install binary
|
||||
sudo mkdir -p /opt/c-relay
|
||||
sudo cp build/c_relay_x86 /opt/c-relay/
|
||||
sudo chown -R c-relay:c-relay /opt/c-relay
|
||||
sudo mkdir -p /opt/c-relay-pg
|
||||
sudo cp build/c_relay_pg_x86 /opt/c-relay-pg/
|
||||
sudo chown -R c-relay-pg:c-relay-pg /opt/c-relay-pg
|
||||
|
||||
# Run as service user
|
||||
sudo -u c-relay /opt/c-relay/c_relay_x86
|
||||
sudo -u c-relay-pg /opt/c-relay-pg/c_relay_pg_x86
|
||||
```
|
||||
|
||||
## Configuration Management
|
||||
@@ -188,7 +188,7 @@ Send this to your relay via WebSocket, and changes are applied immediately.
|
||||
|-----------|-------------|---------|---------|
|
||||
| `relay_description` | Relay description for NIP-11 | "C Nostr Relay" | "My awesome relay" |
|
||||
| `relay_contact` | Admin contact information | "" | "admin@example.com" |
|
||||
| `relay_software` | Software identifier | "c-relay" | "c-relay v1.0" |
|
||||
| `relay_software` | Software identifier | "c-relay-pg" | "c-relay-pg v1.0" |
|
||||
|
||||
#### Client Limits
|
||||
| Parameter | Description | Default | Range |
|
||||
@@ -273,7 +273,7 @@ chmod 600 admin_keys_backup_*.txt
|
||||
#### Key Recovery
|
||||
If you lose your admin private key:
|
||||
|
||||
1. **Stop the relay**: `pkill c_relay` or `sudo systemctl stop c-relay`
|
||||
1. **Stop the relay**: `pkill c_relay_pg` or `sudo systemctl stop c-relay-pg`
|
||||
2. **Backup events**: `cp <relay_pubkey>.nrdb backup_$(date +%Y%m%d).nrdb`
|
||||
3. **Remove database**: `rm <relay_pubkey>.nrdb*`
|
||||
4. **Restart relay**: This creates new database with new keys
|
||||
@@ -300,7 +300,7 @@ sudo ufw allow 8888/tcp
|
||||
```bash
|
||||
# Secure database file permissions
|
||||
chmod 600 <relay_pubkey>.nrdb
|
||||
chown c-relay:c-relay <relay_pubkey>.nrdb
|
||||
chown c-relay-pg:c-relay-pg <relay_pubkey>.nrdb
|
||||
|
||||
# Regular backups
|
||||
cp <relay_pubkey>.nrdb backup/relay_backup_$(date +%Y%m%d_%H%M%S).nrdb
|
||||
@@ -311,10 +311,10 @@ cp <relay_pubkey>.nrdb backup/relay_backup_$(date +%Y%m%d_%H%M%S).nrdb
|
||||
### Service Status
|
||||
```bash
|
||||
# Check if relay is running
|
||||
ps aux | grep c_relay
|
||||
ps aux | grep c_relay_pg
|
||||
|
||||
# SystemD status
|
||||
sudo systemctl status c-relay
|
||||
sudo systemctl status c-relay-pg
|
||||
|
||||
# Network connections
|
||||
netstat -tln | grep 8888
|
||||
@@ -324,16 +324,16 @@ sudo ss -tlpn | grep 8888
|
||||
### Log Monitoring
|
||||
```bash
|
||||
# Real-time logs (systemd)
|
||||
sudo journalctl -u c-relay -f
|
||||
sudo journalctl -u c-relay-pg -f
|
||||
|
||||
# Recent logs
|
||||
sudo journalctl -u c-relay --since "1 hour ago"
|
||||
sudo journalctl -u c-relay-pg --since "1 hour ago"
|
||||
|
||||
# Error logs only
|
||||
sudo journalctl -u c-relay -p err
|
||||
sudo journalctl -u c-relay-pg -p err
|
||||
|
||||
# Configuration changes
|
||||
sudo journalctl -u c-relay | grep "Configuration updated via kind 33334"
|
||||
sudo journalctl -u c-relay-pg | grep "Configuration updated via kind 33334"
|
||||
```
|
||||
|
||||
### Database Analytics
|
||||
@@ -365,13 +365,13 @@ ORDER BY created_at DESC;
|
||||
du -sh <relay_pubkey>.nrdb*
|
||||
|
||||
# Memory usage
|
||||
ps aux | grep c_relay | awk '{print $6}' # RSS memory in KB
|
||||
ps aux | grep c_relay_pg | awk '{print $6}' # RSS memory in KB
|
||||
|
||||
# Connection count (approximate)
|
||||
netstat -an | grep :8888 | grep ESTABLISHED | wc -l
|
||||
|
||||
# System resources
|
||||
top -p $(pgrep c_relay)
|
||||
top -p $(pgrep c_relay_pg)
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
@@ -385,11 +385,11 @@ netstat -tln | grep 8888
|
||||
# If port in use, find process: sudo lsof -i :8888
|
||||
|
||||
# Check binary permissions
|
||||
ls -la build/c_relay_x86
|
||||
chmod +x build/c_relay_x86
|
||||
ls -la build/c_relay_pg_x86
|
||||
chmod +x build/c_relay_pg_x86
|
||||
|
||||
# Check dependencies
|
||||
ldd build/c_relay_x86
|
||||
ldd build/c_relay_pg_x86
|
||||
```
|
||||
|
||||
#### Configuration Not Updating
|
||||
@@ -442,7 +442,7 @@ sqlite3 recovered.nrdb < recovered.sql
|
||||
|
||||
# If repair fails, start fresh (loses all events)
|
||||
mv <relay_pubkey>.nrdb <relay_pubkey>.nrdb.corrupted
|
||||
./build/c_relay_x86 # Creates new database
|
||||
./build/c_relay_pg_x86 # Creates new database
|
||||
```
|
||||
|
||||
#### Lost Configuration Recovery
|
||||
@@ -455,12 +455,12 @@ If configuration is lost but database is intact:
|
||||
#### Emergency Restart
|
||||
```bash
|
||||
# Quick restart with clean state
|
||||
sudo systemctl stop c-relay
|
||||
sudo systemctl stop c-relay-pg
|
||||
mv <relay_pubkey>.nrdb <relay_pubkey>.nrdb.backup
|
||||
sudo systemctl start c-relay
|
||||
sudo systemctl start c-relay-pg
|
||||
|
||||
# Check logs for new admin keys
|
||||
sudo journalctl -u c-relay --since "5 minutes ago" | grep "Admin Private Key"
|
||||
sudo journalctl -u c-relay-pg --since "5 minutes ago" | grep "Admin Private Key"
|
||||
```
|
||||
|
||||
## Advanced Usage
|
||||
@@ -503,7 +503,7 @@ ws.on('open', function() {
|
||||
# backup-relay.sh
|
||||
DATE=$(date +%Y%m%d_%H%M%S)
|
||||
DB_FILE=$(ls *.nrdb | head -1)
|
||||
BACKUP_DIR="/backup/c-relay"
|
||||
BACKUP_DIR="/backup/c-relay-pg"
|
||||
|
||||
mkdir -p $BACKUP_DIR
|
||||
cp $DB_FILE $BACKUP_DIR/relay_backup_$DATE.nrdb
|
||||
@@ -533,7 +533,7 @@ tar czf relay_migration.tar.gz *.nrdb* relay.log
|
||||
|
||||
# Target server
|
||||
tar xzf relay_migration.tar.gz
|
||||
./build/c_relay_x86 # Will detect existing database and continue
|
||||
./build/c_relay_pg_x86 # Will detect existing database and continue
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Script to embed web files into C headers for the C-Relay admin interface
|
||||
# Script to embed web files into C headers for the C-Relay-PG admin interface
|
||||
# Converts HTML, CSS, and JS files from api/ directory into C byte arrays
|
||||
|
||||
set -e
|
||||
|
||||
@@ -60,7 +60,7 @@ All examples assume the event-based configuration system where:
|
||||
- **Save Admin Keys**: All deployment examples emphasize capturing the admin private key on first startup
|
||||
- **Firewall Configuration**: Examples include proper firewall rules
|
||||
- **SSL/TLS**: Production examples include HTTPS configuration
|
||||
- **User Isolation**: Service runs as dedicated `c-relay` system user
|
||||
- **User Isolation**: Service runs as dedicated `c-relay-pg` system user
|
||||
|
||||
## Support
|
||||
|
||||
|
||||
@@ -13,8 +13,8 @@ BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Default configuration
|
||||
RELAY_DIR="/opt/c-relay"
|
||||
BACKUP_DIR="/backup/c-relay"
|
||||
RELAY_DIR="/opt/c-relay-pg"
|
||||
BACKUP_DIR="/backup/c-relay-pg"
|
||||
RETENTION_DAYS="30"
|
||||
COMPRESS="true"
|
||||
REMOTE_BACKUP=""
|
||||
@@ -47,8 +47,8 @@ show_help() {
|
||||
echo "Usage: $0 [OPTIONS]"
|
||||
echo
|
||||
echo "Options:"
|
||||
echo " -d, --relay-dir DIR Relay directory (default: /opt/c-relay)"
|
||||
echo " -b, --backup-dir DIR Backup directory (default: /backup/c-relay)"
|
||||
echo " -d, --relay-dir DIR Relay directory (default: /opt/c-relay-pg)"
|
||||
echo " -b, --backup-dir DIR Backup directory (default: /backup/c-relay-pg)"
|
||||
echo " -r, --retention DAYS Retention period in days (default: 30)"
|
||||
echo " -n, --no-compress Don't compress backups"
|
||||
echo " -s, --s3-bucket BUCKET Upload to S3 bucket"
|
||||
@@ -231,7 +231,7 @@ upload_to_s3() {
|
||||
|
||||
print_step "Uploading backup to S3..."
|
||||
|
||||
local s3_path="s3://$S3_BUCKET/c-relay/$(date +%Y)/$(date +%m)/"
|
||||
local s3_path="s3://$S3_BUCKET/c-relay-pg/$(date +%Y)/$(date +%m)/"
|
||||
|
||||
if aws s3 cp "$BACKUP_FILE" "$s3_path" --storage-class STANDARD_IA; then
|
||||
print_success "Backup uploaded to S3: $s3_path"
|
||||
@@ -264,7 +264,7 @@ cleanup_old_backups() {
|
||||
print_step "Cleaning S3 backups older than $cutoff_date..."
|
||||
|
||||
# Note: This is a simplified approach. In production, use S3 lifecycle policies
|
||||
aws s3 ls "s3://$S3_BUCKET/c-relay/" --recursive | \
|
||||
aws s3 ls "s3://$S3_BUCKET/c-relay-pg/" --recursive | \
|
||||
awk '$1 < "'$cutoff_date'" {print $4}' | \
|
||||
while read -r key; do
|
||||
aws s3 rm "s3://$S3_BUCKET/$key"
|
||||
|
||||
@@ -13,8 +13,8 @@ BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Configuration
|
||||
RELAY_DIR="/opt/c-relay"
|
||||
SERVICE_NAME="c-relay"
|
||||
RELAY_DIR="/opt/c-relay-pg"
|
||||
SERVICE_NAME="c-relay-pg"
|
||||
RELAY_PORT="8888"
|
||||
LOG_FILE="/var/log/relay-monitor.log"
|
||||
ALERT_EMAIL=""
|
||||
@@ -60,7 +60,7 @@ show_help() {
|
||||
echo "Usage: $0 [OPTIONS]"
|
||||
echo
|
||||
echo "Options:"
|
||||
echo " -d, --relay-dir DIR Relay directory (default: /opt/c-relay)"
|
||||
echo " -d, --relay-dir DIR Relay directory (default: /opt/c-relay-pg)"
|
||||
echo " -p, --port PORT Relay port (default: 8888)"
|
||||
echo " -i, --interval SECONDS Check interval (default: 60)"
|
||||
echo " -e, --email EMAIL Alert email address"
|
||||
@@ -134,7 +134,7 @@ parse_args() {
|
||||
check_process_running() {
|
||||
print_step "Checking if relay process is running..."
|
||||
|
||||
if pgrep -f "c_relay_x86" > /dev/null; then
|
||||
if pgrep -f "c_relay_pg_x86" > /dev/null; then
|
||||
print_success "Relay process is running"
|
||||
return 0
|
||||
else
|
||||
@@ -172,7 +172,7 @@ check_service_status() {
|
||||
check_memory_usage() {
|
||||
print_step "Checking memory usage..."
|
||||
|
||||
local memory_kb=$(ps aux | grep "c_relay_x86" | grep -v grep | awk '{sum+=$6} END {print sum}')
|
||||
local memory_kb=$(ps aux | grep "c_relay_pg_x86" | grep -v grep | awk '{sum+=$6} END {print sum}')
|
||||
|
||||
if [[ -z "$memory_kb" ]]; then
|
||||
print_warning "Could not determine memory usage"
|
||||
|
||||
@@ -56,7 +56,7 @@ http {
|
||||
}
|
||||
|
||||
# Upstream for the relay
|
||||
upstream c_relay_backend {
|
||||
upstream c_relay_pg_backend {
|
||||
server 127.0.0.1:8888;
|
||||
keepalive 32;
|
||||
}
|
||||
@@ -108,7 +108,7 @@ http {
|
||||
# Main proxy location for WebSocket and HTTP
|
||||
location / {
|
||||
# Proxy settings
|
||||
proxy_pass http://c_relay_backend;
|
||||
proxy_pass http://c_relay_pg_backend;
|
||||
proxy_http_version 1.1;
|
||||
proxy_cache_bypass $http_upgrade;
|
||||
|
||||
@@ -144,7 +144,7 @@ http {
|
||||
|
||||
# Health check endpoint (if implemented)
|
||||
location /health {
|
||||
proxy_pass http://c_relay_backend/health;
|
||||
proxy_pass http://c_relay_pg_backend/health;
|
||||
access_log off;
|
||||
}
|
||||
|
||||
@@ -155,9 +155,37 @@ http {
|
||||
log_not_found off;
|
||||
}
|
||||
|
||||
# PHP admin page for caching service (direct PostgreSQL access).
|
||||
# Bypasses the NIP-44 64KB encryption limit of the Nostr admin API.
|
||||
# Requires: php-fpm + php-pgsql installed and configured.
|
||||
location /admin/ {
|
||||
alias /opt/c-relay-pg/admin/;
|
||||
index index.php;
|
||||
|
||||
# HTTP Basic Auth
|
||||
auth_basic "Relay Admin";
|
||||
auth_basic_user_file /opt/c-relay-pg/admin/.htpasswd;
|
||||
|
||||
# Pass .php files to PHP-FPM
|
||||
location ~ \.php$ {
|
||||
# Adjust socket path for your distro:
|
||||
# Debian/Ubuntu: unix:/run/php/php8.2-fpm.sock
|
||||
# RHEL/Fedora: unix:/run/php-fpm/www.sock
|
||||
fastcgi_pass unix:/run/php/php8.2-fpm.sock;
|
||||
fastcgi_index index.php;
|
||||
include fastcgi_params;
|
||||
fastcgi_param SCRIPT_FILENAME $request_filename;
|
||||
}
|
||||
|
||||
# Deny access to the lib/ directory (contains DB credentials)
|
||||
location ^~ /admin/lib/ {
|
||||
deny all;
|
||||
}
|
||||
}
|
||||
|
||||
# Optional: Metrics endpoint (if implemented)
|
||||
location /metrics {
|
||||
proxy_pass http://c_relay_backend/metrics;
|
||||
proxy_pass http://c_relay_pg_backend/metrics;
|
||||
# Restrict access to monitoring systems
|
||||
allow 10.0.0.0/8;
|
||||
allow 172.16.0.0/12;
|
||||
|
||||
@@ -96,9 +96,9 @@ check_root() {
|
||||
check_relay_running() {
|
||||
print_step "Checking if C Nostr Relay is running..."
|
||||
|
||||
if ! pgrep -f "c_relay_x86" > /dev/null; then
|
||||
if ! pgrep -f "c_relay_pg_x86" > /dev/null; then
|
||||
print_error "C Nostr Relay is not running"
|
||||
print_error "Please start the relay first with: sudo systemctl start c-relay"
|
||||
print_error "Please start the relay first with: sudo systemctl start c-relay-pg"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
@@ -13,9 +13,9 @@ BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Configuration
|
||||
RELAY_USER="c-relay"
|
||||
INSTALL_DIR="/opt/c-relay"
|
||||
SERVICE_NAME="c-relay"
|
||||
RELAY_USER="c-relay-pg"
|
||||
INSTALL_DIR="/opt/c-relay-pg"
|
||||
SERVICE_NAME="c-relay-pg"
|
||||
RELAY_PORT="8888"
|
||||
|
||||
# Functions
|
||||
@@ -99,7 +99,7 @@ build_relay() {
|
||||
|
||||
# Check if we're in the source directory
|
||||
if [[ ! -f "Makefile" ]]; then
|
||||
print_error "Makefile not found. Please run this script from the c-relay source directory."
|
||||
print_error "Makefile not found. Please run this script from the c-relay-pg source directory."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -107,7 +107,7 @@ build_relay() {
|
||||
make clean
|
||||
make
|
||||
|
||||
if [[ ! -f "build/c_relay_x86" ]]; then
|
||||
if [[ ! -f "build/c_relay_pg_x86" ]]; then
|
||||
print_error "Build failed - binary not found"
|
||||
exit 1
|
||||
fi
|
||||
@@ -118,9 +118,9 @@ build_relay() {
|
||||
install_binary() {
|
||||
print_step "Installing relay binary..."
|
||||
|
||||
cp build/c_relay_x86 "$INSTALL_DIR/"
|
||||
chown "$RELAY_USER:$RELAY_USER" "$INSTALL_DIR/c_relay_x86"
|
||||
chmod +x "$INSTALL_DIR/c_relay_x86"
|
||||
cp build/c_relay_pg_x86 "$INSTALL_DIR/"
|
||||
chown "$RELAY_USER:$RELAY_USER" "$INSTALL_DIR/c_relay_pg_x86"
|
||||
chmod +x "$INSTALL_DIR/c_relay_pg_x86"
|
||||
|
||||
print_success "Binary installed to $INSTALL_DIR"
|
||||
}
|
||||
@@ -129,14 +129,14 @@ install_service() {
|
||||
print_step "Installing systemd service..."
|
||||
|
||||
# Use the existing systemd service file
|
||||
if [[ -f "systemd/c-relay.service" ]]; then
|
||||
cp systemd/c-relay.service /etc/systemd/system/
|
||||
if [[ -f "systemd/c-relay-pg.service" ]]; then
|
||||
cp systemd/c-relay-pg.service /etc/systemd/system/
|
||||
systemctl daemon-reload
|
||||
print_success "Systemd service installed"
|
||||
else
|
||||
print_warning "Systemd service file not found, creating basic one..."
|
||||
|
||||
cat > /etc/systemd/system/c-relay.service << EOF
|
||||
cat > /etc/systemd/system/c-relay-pg.service << EOF
|
||||
[Unit]
|
||||
Description=C Nostr Relay
|
||||
After=network.target
|
||||
@@ -146,7 +146,7 @@ Type=simple
|
||||
User=$RELAY_USER
|
||||
Group=$RELAY_USER
|
||||
WorkingDirectory=$INSTALL_DIR
|
||||
ExecStart=$INSTALL_DIR/c_relay_x86
|
||||
ExecStart=$INSTALL_DIR/c_relay_pg_x86
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# MUSL-based fully static C-Relay builder
|
||||
# MUSL-based fully static C-Relay-PG builder
|
||||
# Produces portable binaries with zero runtime dependencies
|
||||
|
||||
FROM alpine:latest AS builder
|
||||
@@ -125,7 +125,7 @@ RUN cd /tmp && \
|
||||
--prefix=/usr && \
|
||||
make && make install
|
||||
|
||||
# Copy c-relay source
|
||||
# Copy c-relay-pg source
|
||||
COPY . /build/
|
||||
|
||||
# Initialize submodules
|
||||
@@ -134,7 +134,7 @@ RUN git submodule update --init --recursive
|
||||
# Build nostr_core_lib
|
||||
RUN cd nostr_core_lib && ./build.sh
|
||||
|
||||
# Build c-relay static
|
||||
# Build c-relay-pg static
|
||||
RUN make clean && \
|
||||
CC="musl-gcc -static" \
|
||||
CFLAGS="-O2 -Wall -Wextra -std=c99 -g" \
|
||||
@@ -143,8 +143,8 @@ RUN make clean && \
|
||||
make
|
||||
|
||||
# Strip binary for size
|
||||
RUN strip build/c_relay_x86
|
||||
RUN strip build/c_relay_pg_x86
|
||||
|
||||
# Multi-stage build to produce minimal output
|
||||
FROM scratch AS output
|
||||
COPY --from=builder /build/build/c_relay_x86 /c_relay_static_musl_x86_64
|
||||
COPY --from=builder /build/build/c_relay_pg_x86 /c_relay_pg_static_musl_x86_64
|
||||
@@ -19,7 +19,7 @@ RELEASE_MODE=false
|
||||
VERSION_INCREMENT_TYPE="patch" # "patch", "minor", or "major"
|
||||
|
||||
show_usage() {
|
||||
echo "C-Relay Increment and Push Script"
|
||||
echo "C-Relay-PG Increment and Push Script"
|
||||
echo ""
|
||||
echo "USAGE:"
|
||||
echo " $0 [OPTIONS] \"commit message\""
|
||||
@@ -330,7 +330,7 @@ build_release_binary() {
|
||||
create_source_tarball() {
|
||||
print_status "Creating source tarball..."
|
||||
|
||||
local tarball_name="c-relay-${NEW_VERSION#v}.tar.gz"
|
||||
local tarball_name="c-relay-pg-${NEW_VERSION#v}.tar.gz"
|
||||
|
||||
# Create tarball excluding build artifacts and git files
|
||||
if tar -czf "$tarball_name" \
|
||||
@@ -365,7 +365,7 @@ upload_release_assets() {
|
||||
fi
|
||||
|
||||
local token=$(cat "$HOME/.gitea_token" | tr -d '\n\r')
|
||||
local api_url="https://git.laantungir.net/api/v1/repos/laantungir/c-relay"
|
||||
local api_url="https://git.laantungir.net/api/v1/repos/laantungir/c-relay-pg"
|
||||
local assets_url="$api_url/releases/$release_id/assets"
|
||||
print_status "Assets URL: $assets_url"
|
||||
|
||||
@@ -427,7 +427,7 @@ create_gitea_release() {
|
||||
fi
|
||||
|
||||
local token=$(cat "$HOME/.gitea_token" | tr -d '\n\r')
|
||||
local api_url="https://git.laantungir.net/api/v1/repos/laantungir/c-relay"
|
||||
local api_url="https://git.laantungir.net/api/v1/repos/laantungir/c-relay-pg"
|
||||
|
||||
# Create release
|
||||
print_status "Creating release $NEW_VERSION..."
|
||||
@@ -473,7 +473,7 @@ create_gitea_release() {
|
||||
|
||||
# Main execution
|
||||
main() {
|
||||
print_status "C-Relay Increment and Push Script"
|
||||
print_status "C-Relay-PG Increment and Push Script"
|
||||
|
||||
# Check prerequisites
|
||||
check_git_repo
|
||||
@@ -505,13 +505,13 @@ main() {
|
||||
|
||||
# Build release binary
|
||||
if build_release_binary; then
|
||||
local binary_path="build/c_relay_static_x86_64"
|
||||
local binary_path="build/c_relay_pg_static_x86_64"
|
||||
else
|
||||
print_warning "Binary build failed, continuing with release creation"
|
||||
# Check if binary exists from previous build
|
||||
if [[ -f "build/c_relay_static_x86_64" ]]; then
|
||||
if [[ -f "build/c_relay_pg_static_x86_64" ]]; then
|
||||
print_status "Using existing binary from previous build"
|
||||
binary_path="build/c_relay_static_x86_64"
|
||||
binary_path="build/c_relay_pg_static_x86_64"
|
||||
else
|
||||
binary_path=""
|
||||
fi
|
||||
|
||||
+312
-17
@@ -1,6 +1,6 @@
|
||||
#!/bin/bash
|
||||
|
||||
# C-Relay Build and Restart Script
|
||||
# C-Relay-PG Build and Restart Script
|
||||
# Builds the project first, then stops any running relay and starts a new one in the background
|
||||
|
||||
echo "=== C Nostr Relay Build and Restart Script ==="
|
||||
@@ -13,6 +13,15 @@ ADMIN_KEY=""
|
||||
RELAY_KEY=""
|
||||
PORT_OVERRIDE=""
|
||||
DEBUG_LEVEL="5"
|
||||
START_CACHING=false
|
||||
RESET_BACKFILL=false
|
||||
DB_BACKEND="postgres"
|
||||
DB_CONNSTRING=""
|
||||
DB_HOST=""
|
||||
DB_PORT=""
|
||||
DB_NAME=""
|
||||
DB_USER=""
|
||||
DB_PASSWORD=""
|
||||
|
||||
# Key validation function
|
||||
validate_hex_key() {
|
||||
@@ -68,8 +77,34 @@ while [[ $# -gt 0 ]]; do
|
||||
PRESERVE_DATABASE=true
|
||||
shift
|
||||
;;
|
||||
--start-caching)
|
||||
START_CACHING=true
|
||||
shift
|
||||
;;
|
||||
--reset-backfill)
|
||||
RESET_BACKFILL=true
|
||||
shift
|
||||
;;
|
||||
--test-keys|-t)
|
||||
USE_TEST_KEYS=true
|
||||
# Read keys from .test_keys file
|
||||
if [ -f ".test_keys" ]; then
|
||||
echo "Reading test keys from .test_keys file..."
|
||||
# Source the file to get the variables
|
||||
source .test_keys
|
||||
# Remove any single quotes from the values
|
||||
# Note: -a flag expects ADMIN_PUBKEY (public key), not ADMIN_PRIVKEY
|
||||
ADMIN_KEY=$(echo "$ADMIN_PUBKEY" | tr -d "'")
|
||||
RELAY_KEY=$(echo "$SERVER_PRIVKEY" | tr -d "'")
|
||||
echo "Using admin pubkey from .test_keys: ${ADMIN_KEY:0:16}..."
|
||||
echo "Using relay privkey from .test_keys: ${RELAY_KEY:0:16}..."
|
||||
else
|
||||
echo "ERROR: .test_keys file not found"
|
||||
echo "Please create a .test_keys file with the following format:"
|
||||
echo " ADMIN_PUBKEY='your_admin_public_key_hex'"
|
||||
echo " SERVER_PRIVKEY='your_relay_private_key_hex'"
|
||||
exit 1
|
||||
fi
|
||||
shift
|
||||
;;
|
||||
--debug-level=*)
|
||||
@@ -100,6 +135,84 @@ while [[ $# -gt 0 ]]; do
|
||||
shift 2
|
||||
fi
|
||||
;;
|
||||
--db-backend)
|
||||
if [ -z "$2" ]; then
|
||||
echo "ERROR: --db-backend requires a value (sqlite|postgres)"
|
||||
HELP=true
|
||||
shift
|
||||
else
|
||||
DB_BACKEND="$2"
|
||||
shift 2
|
||||
fi
|
||||
;;
|
||||
--db-backend=*)
|
||||
DB_BACKEND="${1#*=}"
|
||||
shift
|
||||
;;
|
||||
--db-connstring)
|
||||
if [ -z "$2" ]; then
|
||||
echo "ERROR: --db-connstring requires a value"
|
||||
HELP=true
|
||||
shift
|
||||
else
|
||||
DB_CONNSTRING="$2"
|
||||
shift 2
|
||||
fi
|
||||
;;
|
||||
--db-connstring=*)
|
||||
DB_CONNSTRING="${1#*=}"
|
||||
shift
|
||||
;;
|
||||
--db-host)
|
||||
if [ -z "$2" ]; then
|
||||
echo "ERROR: --db-host requires a value"
|
||||
HELP=true
|
||||
shift
|
||||
else
|
||||
DB_HOST="$2"
|
||||
shift 2
|
||||
fi
|
||||
;;
|
||||
--db-port)
|
||||
if [ -z "$2" ]; then
|
||||
echo "ERROR: --db-port requires a value"
|
||||
HELP=true
|
||||
shift
|
||||
else
|
||||
DB_PORT="$2"
|
||||
shift 2
|
||||
fi
|
||||
;;
|
||||
--db-name)
|
||||
if [ -z "$2" ]; then
|
||||
echo "ERROR: --db-name requires a value"
|
||||
HELP=true
|
||||
shift
|
||||
else
|
||||
DB_NAME="$2"
|
||||
shift 2
|
||||
fi
|
||||
;;
|
||||
--db-user)
|
||||
if [ -z "$2" ]; then
|
||||
echo "ERROR: --db-user requires a value"
|
||||
HELP=true
|
||||
shift
|
||||
else
|
||||
DB_USER="$2"
|
||||
shift 2
|
||||
fi
|
||||
;;
|
||||
--db-password)
|
||||
if [ -z "$2" ]; then
|
||||
echo "ERROR: --db-password requires a value"
|
||||
HELP=true
|
||||
shift
|
||||
else
|
||||
DB_PASSWORD="$2"
|
||||
shift 2
|
||||
fi
|
||||
;;
|
||||
--help|-h)
|
||||
HELP=true
|
||||
shift
|
||||
@@ -146,6 +259,98 @@ if [ -n "$DEBUG_LEVEL" ]; then
|
||||
fi
|
||||
fi
|
||||
|
||||
# Validate DB backend
|
||||
if [ "$DB_BACKEND" != "sqlite" ] && [ "$DB_BACKEND" != "postgres" ]; then
|
||||
echo "ERROR: Invalid --db-backend value '$DB_BACKEND'. Use sqlite or postgres."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Validate DB port if provided
|
||||
if [ -n "$DB_PORT" ]; then
|
||||
if ! [[ "$DB_PORT" =~ ^[0-9]+$ ]] || [ "$DB_PORT" -lt 1 ] || [ "$DB_PORT" -gt 65535 ]; then
|
||||
echo "ERROR: --db-port must be a number between 1 and 65535"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$DB_BACKEND" = "postgres" ] && [ -z "$DB_CONNSTRING" ]; then
|
||||
[ -z "$DB_HOST" ] && DB_HOST="localhost"
|
||||
[ -z "$DB_PORT" ] && DB_PORT="5432"
|
||||
[ -z "$DB_NAME" ] && DB_NAME="crelay"
|
||||
[ -z "$DB_USER" ] && DB_USER="crelay"
|
||||
[ -z "$DB_PASSWORD" ] && DB_PASSWORD="crelay"
|
||||
fi
|
||||
|
||||
ensure_postgres_database() {
|
||||
if ! command -v psql >/dev/null 2>&1; then
|
||||
echo "ERROR: psql not found. Install PostgreSQL client tools to continue."
|
||||
return 1
|
||||
fi
|
||||
|
||||
echo "Ensuring PostgreSQL database exists: host=$DB_HOST port=$DB_PORT dbname=$DB_NAME user=$DB_USER"
|
||||
|
||||
if [ -n "$DB_PASSWORD" ]; then
|
||||
export PGPASSWORD="$DB_PASSWORD"
|
||||
fi
|
||||
|
||||
# Prefer local postgres superuser for provisioning when available.
|
||||
if sudo -n -u postgres psql -d postgres -tAc "SELECT 1" >/dev/null 2>&1; then
|
||||
sudo -u postgres psql -d postgres -tAc "SELECT 1 FROM pg_roles WHERE rolname = '$DB_USER'" | grep -q 1 || \
|
||||
sudo -u postgres psql -d postgres -v ON_ERROR_STOP=1 -c "CREATE ROLE \"$DB_USER\" LOGIN PASSWORD '$DB_PASSWORD';" || return 1
|
||||
|
||||
sudo -u postgres psql -d postgres -v ON_ERROR_STOP=1 -c "ALTER ROLE \"$DB_USER\" WITH LOGIN PASSWORD '$DB_PASSWORD';" >/dev/null || return 1
|
||||
|
||||
if [ "$PRESERVE_DATABASE" = false ]; then
|
||||
echo "Resetting PostgreSQL database '$DB_NAME' for fresh start..."
|
||||
|
||||
# Ensure no active sessions block DROP DATABASE (common when relay is still running).
|
||||
sudo -u postgres psql -d postgres -v ON_ERROR_STOP=1 -c \
|
||||
"SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = '$DB_NAME' AND pid <> pg_backend_pid();" >/dev/null || return 1
|
||||
|
||||
# Prefer force drop when supported; fallback to regular dropdb.
|
||||
if ! sudo -u postgres dropdb --if-exists --force "$DB_NAME" >/dev/null 2>&1; then
|
||||
sudo -u postgres dropdb --if-exists "$DB_NAME" >/dev/null 2>&1 || return 1
|
||||
fi
|
||||
|
||||
sudo -u postgres createdb -O "$DB_USER" "$DB_NAME" >/dev/null 2>&1 || return 1
|
||||
echo "✓ Recreated PostgreSQL database '$DB_NAME' owned by '$DB_USER'"
|
||||
else
|
||||
DB_EXISTS=$(sudo -u postgres psql -d postgres -tAc "SELECT 1 FROM pg_database WHERE datname = '$DB_NAME'" 2>/dev/null | tr -d '[:space:]')
|
||||
if [ "$DB_EXISTS" != "1" ]; then
|
||||
sudo -u postgres createdb -O "$DB_USER" "$DB_NAME" >/dev/null 2>&1 || return 1
|
||||
echo "✓ Created PostgreSQL database '$DB_NAME'"
|
||||
else
|
||||
echo "✓ PostgreSQL database '$DB_NAME' already exists"
|
||||
fi
|
||||
fi
|
||||
|
||||
sudo -u postgres psql -d postgres -v ON_ERROR_STOP=1 -c "GRANT ALL PRIVILEGES ON DATABASE \"$DB_NAME\" TO \"$DB_USER\";" >/dev/null || return 1
|
||||
sudo -u postgres psql -d "$DB_NAME" -v ON_ERROR_STOP=1 -c "ALTER SCHEMA public OWNER TO \"$DB_USER\"; GRANT ALL ON SCHEMA public TO \"$DB_USER\";" >/dev/null || return 1
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Fallback: use configured DB user directly.
|
||||
if ! psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -v ON_ERROR_STOP=1 -tAc "SELECT 1" >/dev/null 2>&1; then
|
||||
echo "ERROR: Cannot connect to PostgreSQL server/database 'postgres' with current settings"
|
||||
echo " host=$DB_HOST port=$DB_PORT user=$DB_USER"
|
||||
return 1
|
||||
fi
|
||||
|
||||
DB_EXISTS=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -tAc "SELECT 1 FROM pg_database WHERE datname = '$DB_NAME'" 2>/dev/null | tr -d '[:space:]')
|
||||
if [ "$DB_EXISTS" != "1" ]; then
|
||||
echo "Database '$DB_NAME' not found. Creating..."
|
||||
if ! createdb -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" "$DB_NAME" >/dev/null 2>&1; then
|
||||
echo "ERROR: Failed to create PostgreSQL database '$DB_NAME'"
|
||||
return 1
|
||||
fi
|
||||
echo "✓ Created PostgreSQL database '$DB_NAME'"
|
||||
else
|
||||
echo "✓ PostgreSQL database '$DB_NAME' already exists"
|
||||
fi
|
||||
|
||||
return 0
|
||||
}
|
||||
|
||||
# Show help
|
||||
if [ "$HELP" = true ]; then
|
||||
echo "Usage: $0 [OPTIONS]"
|
||||
@@ -157,6 +362,15 @@ if [ "$HELP" = true ]; then
|
||||
echo " -d, --debug-level <0-5> Set debug level: 0=none, 1=errors, 2=warnings, 3=info, 4=debug, 5=trace"
|
||||
echo " --preserve-database Keep existing database files (don't delete for fresh start)"
|
||||
echo " --test-keys, -t Use deterministic test keys for development (admin: all 'a's, relay: all '1's)"
|
||||
echo " --db-backend <name> Database backend: postgres (default) or sqlite"
|
||||
echo " --db-connstring <str> PostgreSQL libpq connection string"
|
||||
echo " --db-host <host> PostgreSQL host"
|
||||
echo " --db-port <port> PostgreSQL port"
|
||||
echo " --db-name <name> PostgreSQL database name"
|
||||
echo " --db-user <user> PostgreSQL database user"
|
||||
echo " --db-password <pass> PostgreSQL database password"
|
||||
echo " --start-caching Auto-start the caching service on startup"
|
||||
echo " --reset-backfill Clear caching backfill progress (use with --start-caching)"
|
||||
echo " --help, -h Show this help message"
|
||||
echo ""
|
||||
echo "Event-Based Configuration:"
|
||||
@@ -176,6 +390,9 @@ if [ "$HELP" = true ]; then
|
||||
echo " $0 --test-keys # Use test keys for consistent development"
|
||||
echo " $0 -t --preserve-database # Use test keys and preserve database"
|
||||
echo ""
|
||||
echo "Default PostgreSQL connection (when no DB flags provided):"
|
||||
echo " host=localhost port=5432 dbname=crelay user=crelay password=crelay"
|
||||
echo ""
|
||||
echo "Key Format: Keys must be exactly 64 hexadecimal characters (0-9, a-f, A-F)"
|
||||
echo "Default behavior: Deletes existing database files to start fresh with new keys"
|
||||
echo " for development purposes"
|
||||
@@ -199,13 +416,17 @@ fi
|
||||
rm -rf dev-config/ 2>/dev/null
|
||||
rm -f db/c_nostr_relay.db* 2>/dev/null
|
||||
|
||||
if [ "$DB_BACKEND" = "postgres" ] && [ -z "$DB_CONNSTRING" ]; then
|
||||
ensure_postgres_database || exit 1
|
||||
fi
|
||||
|
||||
# Embed web files into C headers before building
|
||||
echo "Embedding web files..."
|
||||
./embed_web_files.sh
|
||||
|
||||
# Build the project - ONLY static build
|
||||
echo "Building project (static binary with SQLite JSON1 extension)..."
|
||||
./build_static.sh
|
||||
echo "Building project (static binary, backend: $DB_BACKEND)..."
|
||||
./build_static.sh --db-backend "$DB_BACKEND"
|
||||
|
||||
# Exit if static build fails - no fallback
|
||||
if [ $? -ne 0 ]; then
|
||||
@@ -224,13 +445,13 @@ fi
|
||||
ARCH=$(uname -m)
|
||||
case "$ARCH" in
|
||||
x86_64)
|
||||
BINARY_PATH="./build/c_relay_static_x86_64"
|
||||
BINARY_PATH="./build/c_relay_pg_static_x86_64"
|
||||
;;
|
||||
aarch64|arm64)
|
||||
BINARY_PATH="./build/c_relay_static_arm64"
|
||||
BINARY_PATH="./build/c_relay_pg_static_arm64"
|
||||
;;
|
||||
*)
|
||||
BINARY_PATH="./build/c_relay_static_$ARCH"
|
||||
BINARY_PATH="./build/c_relay_pg_static_$ARCH"
|
||||
;;
|
||||
esac
|
||||
|
||||
@@ -246,13 +467,26 @@ fi
|
||||
|
||||
echo "Using static binary: $BINARY_PATH"
|
||||
|
||||
# Build the caching_relay binary (from caching/ directory)
|
||||
# This is needed because the caching service launcher forks this binary.
|
||||
if [ -d "caching" ] && [ -f "caching/Makefile" ]; then
|
||||
echo "Building caching_relay binary..."
|
||||
(cd caching && make) > /dev/null 2>&1
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "WARNING: caching_relay build failed. The caching service will not be available."
|
||||
echo "You can build it manually with: cd caching && make"
|
||||
else
|
||||
echo "✓ caching_relay binary built successfully"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "Build successful. Proceeding with relay restart..."
|
||||
|
||||
# Kill existing relay if running - start aggressive immediately
|
||||
echo "Stopping any existing relay servers..."
|
||||
|
||||
# Get all relay processes and kill them immediately with -9
|
||||
RELAY_PIDS=$(pgrep -f "c_relay_" || echo "")
|
||||
RELAY_PIDS=$(pgrep -f "c_relay_pg_" || echo "")
|
||||
if [ -n "$RELAY_PIDS" ]; then
|
||||
echo "Force killing relay processes immediately: $RELAY_PIDS"
|
||||
kill -9 $RELAY_PIDS 2>/dev/null
|
||||
@@ -260,6 +494,21 @@ else
|
||||
echo "No existing relay processes found"
|
||||
fi
|
||||
|
||||
# Kill any stale caching_relay processes from previous runs.
|
||||
# The caching service is forked by the relay with setsid(), so it detaches
|
||||
# from the relay's process group and survives when the relay is killed -9.
|
||||
# Without this, every restart orphans the previous caching_relay child and
|
||||
# a new relay forks another, leading to many stale processes with dead PG
|
||||
# connections spamming errors. Kill them here so the new relay starts fresh.
|
||||
CACHING_PIDS=$(pgrep -f "caching_relay" || echo "")
|
||||
if [ -n "$CACHING_PIDS" ]; then
|
||||
echo "Killing stale caching_relay processes: $CACHING_PIDS"
|
||||
kill -9 $CACHING_PIDS 2>/dev/null || true
|
||||
sleep 1
|
||||
else
|
||||
echo "No existing caching_relay processes found"
|
||||
fi
|
||||
|
||||
# Ensure port 8888 is completely free with retry loop
|
||||
echo "Ensuring port 8888 is available..."
|
||||
for attempt in {1..15}; do
|
||||
@@ -273,7 +522,7 @@ for attempt in {1..15}; do
|
||||
fuser -k 8888/tcp 2>/dev/null || true
|
||||
|
||||
# Double-check for any remaining relay processes
|
||||
REMAINING_PIDS=$(pgrep -f "c_relay_" || echo "")
|
||||
REMAINING_PIDS=$(pgrep -f "c_relay_pg_" || echo "")
|
||||
if [ -n "$REMAINING_PIDS" ]; then
|
||||
echo "Killing remaining relay processes: $REMAINING_PIDS"
|
||||
kill -9 $REMAINING_PIDS 2>/dev/null || true
|
||||
@@ -290,14 +539,21 @@ for attempt in {1..15}; do
|
||||
fi
|
||||
done
|
||||
|
||||
# Final safety check - ensure no relay processes remain
|
||||
FINAL_PIDS=$(pgrep -f "c_relay_" || echo "")
|
||||
# Final safety check - ensure no relay or caching processes remain
|
||||
FINAL_PIDS=$(pgrep -f "c_relay_pg_" || echo "")
|
||||
if [ -n "$FINAL_PIDS" ]; then
|
||||
echo "Final cleanup: killing processes $FINAL_PIDS"
|
||||
echo "Final cleanup: killing relay processes $FINAL_PIDS"
|
||||
kill -9 $FINAL_PIDS 2>/dev/null || true
|
||||
sleep 1
|
||||
fi
|
||||
|
||||
FINAL_CACHING_PIDS=$(pgrep -f "caching_relay" || echo "")
|
||||
if [ -n "$FINAL_CACHING_PIDS" ]; then
|
||||
echo "Final cleanup: killing stale caching_relay processes $FINAL_CACHING_PIDS"
|
||||
kill -9 $FINAL_CACHING_PIDS 2>/dev/null || true
|
||||
sleep 1
|
||||
fi
|
||||
|
||||
# Clean up PID file
|
||||
rm -f relay.pid
|
||||
|
||||
@@ -307,7 +563,7 @@ echo "Database will be initialized automatically on startup if needed"
|
||||
|
||||
# Start relay in background with output redirection
|
||||
echo "Starting relay server..."
|
||||
echo "Debug: Current processes: $(ps aux | grep 'c_relay_' | grep -v grep || echo 'None')"
|
||||
echo "Debug: Current processes: $(ps aux | grep 'c_relay_pg_' | grep -v grep || echo 'None')"
|
||||
|
||||
# Build command line arguments for relay binary
|
||||
RELAY_ARGS=""
|
||||
@@ -332,15 +588,52 @@ if [ -n "$DEBUG_LEVEL" ]; then
|
||||
echo "Using debug level: $DEBUG_LEVEL"
|
||||
fi
|
||||
|
||||
if [ -n "$DB_CONNSTRING" ]; then
|
||||
RELAY_ARGS="$RELAY_ARGS --db-connstring '$DB_CONNSTRING'"
|
||||
echo "Using PostgreSQL connection string override"
|
||||
fi
|
||||
|
||||
if [ -n "$DB_HOST" ]; then
|
||||
RELAY_ARGS="$RELAY_ARGS --db-host '$DB_HOST'"
|
||||
fi
|
||||
|
||||
if [ -n "$DB_PORT" ]; then
|
||||
RELAY_ARGS="$RELAY_ARGS --db-port '$DB_PORT'"
|
||||
fi
|
||||
|
||||
if [ -n "$DB_NAME" ]; then
|
||||
RELAY_ARGS="$RELAY_ARGS --db-name '$DB_NAME'"
|
||||
fi
|
||||
|
||||
if [ -n "$DB_USER" ]; then
|
||||
RELAY_ARGS="$RELAY_ARGS --db-user '$DB_USER'"
|
||||
fi
|
||||
|
||||
if [ -n "$DB_PASSWORD" ]; then
|
||||
RELAY_ARGS="$RELAY_ARGS --db-password '$DB_PASSWORD'"
|
||||
fi
|
||||
|
||||
if [ "$START_CACHING" = true ]; then
|
||||
RELAY_ARGS="$RELAY_ARGS --start-caching"
|
||||
echo "Auto-starting caching service on startup"
|
||||
fi
|
||||
|
||||
if [ "$RESET_BACKFILL" = true ]; then
|
||||
RELAY_ARGS="$RELAY_ARGS --reset-backfill"
|
||||
echo "Resetting caching backfill progress before start"
|
||||
fi
|
||||
|
||||
# Change to build directory before starting relay so database files are created there
|
||||
cd build
|
||||
# Start relay in background and capture its PID
|
||||
if [ "$USE_TEST_KEYS" = true ]; then
|
||||
echo "Using deterministic test keys for development..."
|
||||
./$(basename $BINARY_PATH) -a 6a04ab98d9e4774ad806e302dddeb63bea16b5cb5f223ee77478e861bb583eb3 -r 1111111111111111111111111111111111111111111111111111111111111111 --debug-level=$DEBUG_LEVEL --strict-port > ../relay.log 2>&1 &
|
||||
echo "Using test keys from .test_keys file..."
|
||||
# shellcheck disable=SC2086
|
||||
eval ./$(basename $BINARY_PATH) -a "$ADMIN_KEY" -r "$RELAY_KEY" $RELAY_ARGS --strict-port > ../relay.log 2>&1 &
|
||||
elif [ -n "$RELAY_ARGS" ]; then
|
||||
echo "Starting relay with custom configuration..."
|
||||
./$(basename $BINARY_PATH) $RELAY_ARGS --debug-level=$DEBUG_LEVEL --strict-port > ../relay.log 2>&1 &
|
||||
# shellcheck disable=SC2086
|
||||
eval ./$(basename $BINARY_PATH) $RELAY_ARGS --strict-port > ../relay.log 2>&1 &
|
||||
else
|
||||
# No command line arguments needed for random key generation
|
||||
echo "Starting relay with random key generation..."
|
||||
@@ -410,9 +703,11 @@ if ps -p "$RELAY_PID" >/dev/null 2>&1; then
|
||||
echo "=== Event-Based Relay Server Running ==="
|
||||
echo "Configuration: Event-based (kind 33334 Nostr events)"
|
||||
echo "Database: Automatically created with relay pubkey naming"
|
||||
echo "To kill relay: pkill -f 'c_relay_'"
|
||||
echo "To check status: ps aux | grep c_relay_"
|
||||
echo "To kill relay: pkill -f 'c_relay_pg_'"
|
||||
echo "To kill caching service: pkill -f 'caching_relay'"
|
||||
echo "To check status: ps aux | grep c_relay_pg_"
|
||||
echo "To view logs: tail -f relay.log"
|
||||
echo "To view caching logs: tail -f build/caching_relay.log"
|
||||
echo "Binary: $BINARY_PATH (zero configuration needed)"
|
||||
echo "Ready for Nostr client connections!"
|
||||
else
|
||||
|
||||
+1
-1
@@ -1,3 +1,3 @@
|
||||
#!/bin/bash
|
||||
|
||||
curl -H "Accept: application/nostr+json" http://localhost:8888/
|
||||
curl -H "Accept: application/nostr+json" https://relay.laantungir.net
|
||||
|
||||
Submodule nostr_core_lib deleted from a8dc2ed046
@@ -12,11 +12,11 @@ After the next crash, analyze it:
|
||||
# List all core dumps (most recent first)
|
||||
sudo coredumpctl list
|
||||
|
||||
# View info about the most recent c-relay crash
|
||||
sudo coredumpctl info c-relay
|
||||
# View info about the most recent c-relay-pg crash
|
||||
sudo coredumpctl info c-relay-pg
|
||||
|
||||
# Load the core dump in gdb for detailed analysis
|
||||
sudo coredumpctl gdb c-relay
|
||||
sudo coredumpctl gdb c-relay-pg
|
||||
|
||||
Inside gdb, run these commands:
|
||||
(gdb) bt full # Full backtrace with all variables
|
||||
@@ -36,8 +36,8 @@ DEBUGGING
|
||||
|
||||
Even simpler: Use this one-liner
|
||||
# Start relay and immediately attach gdb
|
||||
cd /usr/local/bin/c_relay
|
||||
sudo -u c-relay ./c_relay --debug-level=5 & sleep 2 && sudo gdb -p $(pgrep c_relay)
|
||||
cd /usr/local/bin/c_relay_pg
|
||||
sudo -u c-relay-pg ./c_relay_pg --debug-level=5 & sleep 2 && sudo gdb -p $(pgrep c_relay_pg)
|
||||
|
||||
Inside gdb, after attaching:
|
||||
|
||||
@@ -48,17 +48,17 @@ Or shorter:
|
||||
|
||||
How to View the Logs
|
||||
Check systemd journal:
|
||||
# View all c-relay logs
|
||||
sudo journalctl -u c-relay
|
||||
# View all c-relay-pg logs
|
||||
sudo journalctl -u c-relay-pg
|
||||
|
||||
# View recent logs (last 50 lines)
|
||||
sudo journalctl -u c-relay -n 50
|
||||
sudo journalctl -u c-relay-pg -n 50
|
||||
|
||||
# Follow logs in real-time
|
||||
sudo journalctl -u c-relay -f
|
||||
sudo journalctl -u c-relay-pg -f
|
||||
|
||||
# View logs since last boot
|
||||
sudo journalctl -u c-relay -b
|
||||
sudo journalctl -u c-relay-pg -b
|
||||
|
||||
Check if service is running:
|
||||
|
||||
@@ -78,9 +78,9 @@ sudo systemctl start rsyslog
|
||||
sudo systemctl status rsyslog
|
||||
|
||||
|
||||
sudo -u c-relay ./c_relay --debug-level=5 -r 85d0b37e2ae822966dcadd06b2dc9368cde73865f90ea4d44f8b57d47ef0820a -a 1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139
|
||||
sudo -u c-relay-pg ./c_relay_pg --debug-level=5 -r 85d0b37e2ae822966dcadd06b2dc9368cde73865f90ea4d44f8b57d47ef0820a -a 1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139
|
||||
|
||||
./c_relay_static_x86_64 -p 7889 --debug-level=5 -r 85d0b37e2ae822966dcadd06b2dc9368cde73865f90ea4d44f8b57d47ef0820a -a 1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139
|
||||
./c_relay_pg_static_x86_64 -p 7889 --debug-level=5 -r 85d0b37e2ae822966dcadd06b2dc9368cde73865f90ea4d44f8b57d47ef0820a -a 1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139
|
||||
|
||||
|
||||
sudo ufw allow 8888/tcp
|
||||
|
||||
Generated
+1
-1
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"name": "c-relay",
|
||||
"name": "c-relay-pg",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
|
||||
@@ -0,0 +1,178 @@
|
||||
# API Upgrade Implementation Plan — Phases 1, 2, and 5
|
||||
|
||||
Goal: finish the remaining work from `plans/api_upgrade_plan.md`:
|
||||
- **Phase 1 (design fix):** convert the `api-worker` thread from job-queue-driven to **timer + subscriber-check** driven.
|
||||
- **Phase 2:** remove the per-event and per-subscription monitoring triggers so monitoring no longer runs on the event-processing path.
|
||||
- **Phase 5:** add **PostgreSQL `LISTEN`/`NOTIFY`** so the worker wakes on real data changes instead of pure polling.
|
||||
|
||||
Target test relay: **port 7777** (local).
|
||||
|
||||
---
|
||||
|
||||
## Current State (verified in code)
|
||||
|
||||
- `api_worker_main()` ([`src/api.c:172`](src/api.c:172)) blocks on `api_worker_pop_job_blocking()` and reacts to three job types: `API_WORK_JOB_EVENT_STORED`, `API_WORK_JOB_SUBSCRIPTION_CHANGE`, `API_WORK_JOB_STATUS_POST`.
|
||||
- Monitoring hooks `monitoring_on_event_stored()` ([`src/api.c:752`](src/api.c:752)) and `monitoring_on_subscription_change()` ([`src/api.c:759`](src/api.c:759)) are called from [`src/main.c:1169`](src/main.c:1169) and [`src/subscriptions.c:483`](src/subscriptions.c:483), [`src/subscriptions.c:536`](src/subscriptions.c:536). They enqueue jobs; sync fallback runs if the worker is down.
|
||||
- Throttling lives in `monitoring_on_event_stored_sync()` / `monitoring_on_subscription_change_sync()` ([`src/api.c:711`](src/api.c:711), [`src/api.c:731`](src/api.c:731)).
|
||||
- Round-robin selection in `generate_round_robin_monitoring_event()` ([`src/api.c:571`](src/api.c:571)); per-type build/sign/broadcast in `generate_monitoring_event_for_type()` ([`src/api.c:621`](src/api.c:621)).
|
||||
- `has_subscriptions_for_kind(int)` ([`src/subscriptions.c:980`](src/subscriptions.c:980)) checks only subscriptions with an explicit `kinds` filter — it does **not** consider the `no_kind_filter_subs` list, so it alone is insufficient for a "is anyone listening to kind 24567" check.
|
||||
- Worker DB connection is a `PGconn*` opened via `db_open_worker_connection()` ([`src/db_ops_postgres.c:226`](src/db_ops_postgres.c:226)). Schema is embedded in [`src/pg_schema.h`](src/pg_schema.h) (version `"4"`) and mirrored in [`src/pg_schema.sql`](src/pg_schema.sql).
|
||||
- No `LISTEN`/`NOTIFY`/`pg_notify`/`PQnotifies` usage exists anywhere in `src/`.
|
||||
|
||||
---
|
||||
|
||||
## Design
|
||||
|
||||
### Phase 1 + 2 + 5 combined worker loop
|
||||
|
||||
The three phases are coupled, so they are implemented together in one rewrite of `api_worker_main()` and its helpers.
|
||||
|
||||
### CRITICAL BUG FIX (why the web API isn't getting updates today)
|
||||
|
||||
`postgres_db_open_worker_connection()` ([`src/db_ops_postgres.c:226`](src/db_ops_postgres.c:226)) calls
|
||||
`postgres_db_set_thread_connection(conn)` at line 243 — but `g_thread_pg_conn` is declared `__thread`
|
||||
([`src/db_ops_postgres.c:45`](src/db_ops_postgres.c:45)), so the binding is set in the **caller's thread**
|
||||
(lws-main, which calls `start_api_worker`), NOT in the api-worker thread. When `api_worker_main` runs and
|
||||
the monitoring query functions call `postgres_db_active_connection()`, `g_thread_pg_conn` is NULL in that
|
||||
thread, so they fall back to the shared global `g_pg_conn` — concurrent access with the main thread. This
|
||||
causes monitoring queries to fail or interleave, which is why the dashboard stops receiving kind 24567
|
||||
events.
|
||||
|
||||
**Fix:** inside `api_worker_main()`, after opening `worker_db`, call
|
||||
`db_set_thread_connection(worker_db)` so the `__thread` pointer is bound in the api-worker thread itself.
|
||||
Call `db_clear_thread_connection()` before closing the connection on shutdown. This must be done in the
|
||||
new rewrite regardless of the loop design.
|
||||
|
||||
```
|
||||
api-worker thread:
|
||||
1. Open own PG connection.
|
||||
2. db_set_thread_connection(worker_db) // bind __thread pointer in THIS thread
|
||||
3. If PG backend: LISTEN event_stored on this connection.
|
||||
4. Loop while running:
|
||||
a. Determine if anyone is listening to kind 24567
|
||||
(has_subscriptions_for_kind(24567) OR no_kind_filter_subs non-empty).
|
||||
b. If NO subscribers:
|
||||
- condvar_timedwait(throttle_sec) // wakeable for shutdown / STATUS_POST job
|
||||
- on wake: process any pending STATUS_POST job, then continue.
|
||||
// Zero DB work, zero notify polling.
|
||||
c. If subscribers exist:
|
||||
- wait_for_notify_or_timeout(throttle_sec):
|
||||
select() on { PQsocket, self-pipe read fd } with timeout = throttle_sec.
|
||||
- If PQsocket readable: PQconsumeInput + drain PQnotifies.
|
||||
- If self-pipe readable: drain (shutdown or STATUS_POST signal).
|
||||
- If timeout: fall through.
|
||||
- generate_round_robin_monitoring_event() // one d-tag per tick
|
||||
- process any pending STATUS_POST job.
|
||||
4. On shutdown: UNLISTEN, close PG connection.
|
||||
```
|
||||
|
||||
Key properties:
|
||||
- **Zero overhead when no subscribers** — condvar sleep, no `select`, no `LISTEN` polling.
|
||||
- **Reactive when subscribers exist** — wakes within `throttle_sec` of an `event_stored` NOTIFY, but never more than once per `throttle_sec` (rate-limited by the select timeout + the round-robin cadence already in `generate_round_robin_monitoring_event`).
|
||||
- **STATUS_POST preserved** — still enqueued from [`src/websockets.c:3470`](src/websockets.c:3470); the self-pipe wakes the worker's `select` so status posts are not delayed by a pending notify wait.
|
||||
- **SQLite fallback** — `db_worker_poll_notify()` returns "not supported" on SQLite, so the worker falls back to pure timer behavior (condvar timed wait) — identical to the no-PG path.
|
||||
|
||||
### Self-pipe for job/shutdown signalling
|
||||
|
||||
A self-pipe (or `eventfd` on Linux) is added to `api.c`:
|
||||
- `api_worker_enqueue_job(STATUS_POST)` writes one byte to the pipe → wakes `select`.
|
||||
- `stop_api_worker()` sets `g_api_worker_running = 0`, writes a byte, and broadcasts the condvar (covers both the subscriber and no-subscriber wait paths).
|
||||
|
||||
### Subscriber-check helper
|
||||
|
||||
Add `int has_any_subscription_for_kind(int event_kind)` in [`src/subscriptions.c`](src/subscriptions.c) that returns true if `has_subscriptions_for_kind(event_kind)` OR the `no_kind_filter_subs` list is non-empty. Export it in [`src/subscriptions.h`](src/subscriptions.h). The worker calls `has_any_subscription_for_kind(24567)`.
|
||||
|
||||
### DB abstraction for LISTEN/NOTIFY
|
||||
|
||||
Add to [`src/db_ops.h`](src/db_ops.h) / [`src/db_ops_postgres.c`](src/db_ops_postgres.c) / [`src/db_ops_sqlite.c`](src/db_ops_sqlite.c):
|
||||
- `int db_worker_listen(void* conn, const char* channel);` — issues `LISTEN <channel>`.
|
||||
- `int db_worker_poll_notify(void* conn, int timeout_ms);` — returns `1` if a notification was consumed, `0` on timeout, `-1` on error/unsupported. Uses `PQsocket()`, `select()`, `PQconsumeInput()`, `PQnotifies()`.
|
||||
- SQLite stubs return `-1` (unsupported) so the worker falls back to timer mode.
|
||||
|
||||
### Schema change (Phase 5)
|
||||
|
||||
Add to both [`src/pg_schema.sql`](src/pg_schema.sql) and [`src/pg_schema.h`](src/pg_schema.h):
|
||||
```sql
|
||||
CREATE OR REPLACE FUNCTION notify_event_stored() RETURNS trigger AS $$
|
||||
BEGIN
|
||||
PERFORM pg_notify('event_stored', json_build_object(
|
||||
'kind', NEW.kind,
|
||||
'pubkey', substring(NEW.pubkey, 1, 8)
|
||||
)::text);
|
||||
RETURN NEW;
|
||||
END;
|
||||
$$ LANGUAGE plpgsql;
|
||||
|
||||
DROP TRIGGER IF EXISTS trg_notify_event_stored ON events;
|
||||
CREATE TRIGGER trg_notify_event_stored
|
||||
AFTER INSERT ON events
|
||||
FOR EACH ROW EXECUTE FUNCTION notify_event_stored();
|
||||
```
|
||||
Bump `EMBEDDED_PG_SCHEMA_VERSION` from `"4"` to `"5"` in [`src/pg_schema.h`](src/pg_schema.h) and the matching `schema_info` version write in [`src/pg_schema.sql`](src/pg_schema.sql). The trigger is idempotent (`DROP TRIGGER IF EXISTS` + `CREATE`), so existing databases upgrade automatically on next startup via `postgres_db_apply_schema()` ([`src/db_ops_postgres.c:134`](src/db_ops_postgres.c:134)).
|
||||
|
||||
### Phase 2 removals
|
||||
|
||||
- Delete the `monitoring_on_event_stored()` call at [`src/main.c:1169`](src/main.c:1169) (and its forward decl at [`src/main.c:459`](src/main.c:459)).
|
||||
- Delete the `monitoring_on_subscription_change()` calls at [`src/subscriptions.c:483`](src/subscriptions.c:483) and [`src/subscriptions.c:536`](src/subscriptions.c:536) (and its forward decl at [`src/subscriptions.c:58`](src/subscriptions.c:58)).
|
||||
- Remove from [`src/api.h`](src/api.h): `monitoring_on_event_stored`, `monitoring_on_subscription_change`.
|
||||
- Remove from [`src/api.c`](src/api.c): `API_WORK_JOB_EVENT_STORED`, `API_WORK_JOB_SUBSCRIPTION_CHANGE`, `monitoring_on_event_stored`, `monitoring_on_subscription_change`, `monitoring_on_event_stored_sync`, `monitoring_on_subscription_change_sync`. Keep `API_WORK_JOB_STATUS_POST`, `generate_round_robin_monitoring_event`, `generate_monitoring_event_for_type`, `generate_event_driven_monitoring`/`generate_subscription_driven_monitoring` (still used by `generate_monitoring_event` legacy path and tests).
|
||||
- The `generate_event_driven_monitoring` / `generate_subscription_driven_monitoring` wrappers become unused by the worker but remain as public helpers for any external/test callers; the worker calls `generate_round_robin_monitoring_event()` directly.
|
||||
|
||||
### Thread layout (unchanged from plan target)
|
||||
|
||||
```
|
||||
c_relay_pg process
|
||||
├── lws-main — WebSocket event loop (no monitoring DB work)
|
||||
├── db-read-1..N — Async REQ/COUNT queries
|
||||
├── event-worker — Async EVENT ingestion (no monitoring hook)
|
||||
├── db-write-1..N — Async sub logging, misc writes
|
||||
└── api-worker — Timer + LISTEN/NOTIFY driven monitoring (own PG conn)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Mermaid: api-worker state machine
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> OpenConn
|
||||
OpenConn --> BindThread: db_set_thread_connection
|
||||
BindThread --> Listen: PG backend
|
||||
BindThread --> Ready: SQLite/no-PG
|
||||
Listen --> Ready
|
||||
Ready --> CheckSubs
|
||||
CheckSubs --> NoSubSleep: no kind-24567 subs
|
||||
CheckSubs --> WaitNotify: subs present
|
||||
NoSubSleep --> CheckSubs: throttle_sec or wake
|
||||
WaitNotify --> Generate: NOTIFY or throttle_sec
|
||||
Generate --> CheckSubs: after round-robin tick
|
||||
WaitNotify --> NoSubSleep: subs dropped + wake
|
||||
Ready --> [*]: stop_api_worker
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Todo List
|
||||
|
||||
1. Add `has_any_subscription_for_kind()` to `subscriptions.c`/`subscriptions.h` (checks kind index + no-kind-filter list).
|
||||
2. Add `db_worker_listen()` and `db_worker_poll_notify()` to `db_ops.h`, `db_ops_postgres.c` (libpq), and `db_ops_sqlite.c` (stub).
|
||||
3. Add `notify_event_stored()` function + `trg_notify_event_stored` trigger to `pg_schema.sql` and `pg_schema.h`; bump schema version to 5.
|
||||
4. Rewrite `api_worker_main()` in `api.c`: bind the worker PG connection to the thread via `db_set_thread_connection()` (fixes the dashboard-not-updating bug), then run the timer + LISTEN/NOTIFY + subscriber-check loop; add self-pipe for STATUS_POST/shutdown wakeups; keep `start_api_worker`/`stop_api_worker`/`api_worker_process_completions`/`api_worker_enqueue_status_post` signatures.
|
||||
5. Remove `API_WORK_JOB_EVENT_STORED`/`API_WORK_JOB_SUBSCRIPTION_CHANGE` job types and the `monitoring_on_event_stored`/`monitoring_on_subscription_change` functions + sync wrappers from `api.c` and `api.h`.
|
||||
6. Remove the `monitoring_on_event_stored()` call from `main.c` (event storage path).
|
||||
7. Remove the `monitoring_on_subscription_change()` calls from `subscriptions.c` (subscription create/close paths).
|
||||
8. Build with `./make_and_restart_relay.sh` on port 7777; fix any compile errors.
|
||||
9. Test: subscribe to kind 24567 on ws://localhost:7777 → confirm monitoring events arrive on throttle cadence (this validates the thread-connection fix).
|
||||
10. Test: publish a kind-1 event → confirm a NOTIFY-driven monitoring event fires within throttle_sec.
|
||||
11. Test: with no kind-24567 subscribers → confirm no monitoring events generated (zero overhead) and event storage latency unaffected.
|
||||
12. Run `tests/quick_error_tests.sh` and `tests/subscribe_all.sh` against port 7777 to confirm no regressions.
|
||||
|
||||
---
|
||||
|
||||
## Risks & Mitigations
|
||||
|
||||
- **Self-pipe + condvar mix complexity.** Mitigation: the no-subscriber path uses only the condvar; the subscriber path uses `select` on the self-pipe + PQ socket. The condvar is only used to wake the no-subscriber sleep, so the two wait mechanisms never overlap.
|
||||
- **`LISTEN` on a worker connection that also runs monitoring queries.** `LISTEN` is connection-local and persists; running `SELECT`s on the same connection does not cancel it. `PQconsumeInput` must be called before `PQnotifies`. Mitigation: encapsulate all of this in `db_worker_poll_notify()`.
|
||||
- **Schema trigger on every insert adds per-event overhead.** `pg_notify` is cheap (in-memory queue) and the payload is tiny. The existing `sync_event_tags_from_events` AFTER INSERT trigger already does far more work per row, so this is negligible by comparison.
|
||||
- **SQLite has no LISTEN/NOTIFY.** Mitigation: `db_worker_poll_notify` returns `-1` on SQLite and the worker falls back to pure timer mode — functionally identical to the no-notify path.
|
||||
- **Removing the per-event monitoring hook changes dashboard freshness behavior.** Mitigation: with subscribers present, NOTIFY now drives near-real-time updates (better than before); with no subscribers, nothing is generated (matches the plan's "zero overhead" goal). Dashboard already handles event-driven updates via its kind-24567 subscription.
|
||||
@@ -0,0 +1,224 @@
|
||||
# API Upgrade Plan: Monitoring Thread + PostgreSQL Push
|
||||
|
||||
## Current API Architecture
|
||||
|
||||
### How Monitoring Works Today
|
||||
|
||||
The relay publishes monitoring data as **kind 24567 ephemeral Nostr events**. The admin dashboard is a standard Nostr client that subscribes to these events.
|
||||
|
||||
```
|
||||
Dashboard connects → subscribes to kind 24567 with d-tags
|
||||
Relay detects subscription → starts generating monitoring events
|
||||
Relay queries DB → builds JSON → signs event → broadcasts to subscribers
|
||||
Dashboard receives events → renders stats/charts
|
||||
```
|
||||
|
||||
### Current Monitoring Event Types
|
||||
|
||||
| d-tag | Query Function | What It Contains | Triggered By |
|
||||
|-------|---------------|-----------------|-------------|
|
||||
| `event_kinds` | `query_event_kind_distribution()` | Event count per kind, total events | Event storage |
|
||||
| `time_stats` | `query_time_based_statistics()` | Events in last 24h/7d/30d | Event storage |
|
||||
| `top_pubkeys` | `query_top_pubkeys()` | Top 10 pubkeys by event count | Event storage |
|
||||
| `subscription_details` | `query_subscription_details()` | Active subscriptions list | Subscription changes |
|
||||
| `cpu_metrics` | `query_cpu_metrics()` | PID, memory, CPU, connections | Both triggers |
|
||||
|
||||
### Current Admin Command System
|
||||
|
||||
Two parallel admin interfaces exist:
|
||||
|
||||
**1. Kind 23456 Admin Events (WebSocket)**
|
||||
- Admin sends NIP-44 encrypted commands as kind 23456 events
|
||||
- Relay decrypts, processes, responds with kind 23457 events
|
||||
- Commands: config get/set, auth rules, system commands, SQL queries
|
||||
|
||||
**2. NIP-17 DM Admin (Direct Messages)**
|
||||
- Admin sends gift-wrapped DMs (kind 1059) to relay
|
||||
- Relay unwraps, processes commands, responds via DM
|
||||
- Commands: stats, config, help, SQL queries
|
||||
|
||||
### Current Problems
|
||||
|
||||
1. **All monitoring runs on lws-main thread** — DB queries block the event loop
|
||||
2. **Monitoring is triggered by event storage** — `generate_event_driven_monitoring()` runs synchronously after each event insert
|
||||
3. **No dedicated monitoring thread** — monitoring piggybacks on the relay's main processing
|
||||
4. **Throttle is time-based only** — `kind_24567_reporting_throttle_sec` (default 5s) but queries still run on main thread
|
||||
5. **Dashboard polls via subscription** — no way to get data without the relay actively generating events
|
||||
|
||||
---
|
||||
|
||||
## Upgrade Plan
|
||||
|
||||
### Phase 1: Dedicated API Thread (`api-worker`)
|
||||
|
||||
**What:** Create a new thread that owns all monitoring event generation. It runs on a timer, queries PostgreSQL with its own connection, builds and signs monitoring events, and pushes them into the broadcast system.
|
||||
|
||||
**Thread behavior:**
|
||||
```
|
||||
api-worker thread:
|
||||
1. Open own PG connection
|
||||
2. Loop:
|
||||
a. Sleep for throttle_sec interval
|
||||
b. Check if anyone is subscribed to kind 24567
|
||||
c. If yes: run all monitoring queries
|
||||
d. Build kind 24567 events with results
|
||||
e. Sign events with relay key
|
||||
f. Push completed events to lws-main for broadcast
|
||||
g. Repeat
|
||||
```
|
||||
|
||||
**Key design decisions:**
|
||||
- The api-worker thread does ALL the heavy work (queries + signing)
|
||||
- It only hands off pre-built, signed events to lws-main for broadcast
|
||||
- lws-main never blocks on monitoring queries
|
||||
- The thread sleeps when no one is subscribed (zero overhead)
|
||||
|
||||
**Files to change:**
|
||||
- `src/api.c`: Refactor `generate_monitoring_event_for_type()` to be callable from any thread
|
||||
- `src/websockets.c`: Add monitor thread lifecycle (start/stop), completion queue for broadcast
|
||||
- `src/thread_pool.h`: Add api-worker thread config options
|
||||
|
||||
**New thread in the process:**
|
||||
```
|
||||
c_relay_pg process
|
||||
├── lws-main — WebSocket event loop
|
||||
├── db-read-1..N — Async REQ/COUNT queries
|
||||
├── event-worker — Async EVENT ingestion
|
||||
├── db-write-1..N — Async sub logging, misc writes
|
||||
└── api-worker — NEW: periodic monitoring event generation
|
||||
```
|
||||
|
||||
### Phase 2: Remove Monitoring from Main Thread Path
|
||||
|
||||
**What:** Remove `generate_event_driven_monitoring()` and `generate_subscription_driven_monitoring()` calls from the synchronous event processing path.
|
||||
|
||||
**Currently these are called from:**
|
||||
- After event storage (event-driven monitoring)
|
||||
- After subscription create/close (subscription-driven monitoring)
|
||||
|
||||
**After change:**
|
||||
- The api-worker thread handles all monitoring on its own timer
|
||||
- Event storage and subscription changes no longer trigger monitoring directly
|
||||
- The throttle interval controls how fresh the data is (default 5 seconds)
|
||||
|
||||
**Benefit:** Event processing becomes faster — no monitoring overhead per event.
|
||||
|
||||
### Phase 3: Dashboard Uses Normal Subscriptions for Real-Time Events
|
||||
|
||||
**What:** Instead of the relay generating special monitoring events for "recent events," the dashboard simply subscribes to all events using standard Nostr REQ filters.
|
||||
|
||||
**Current approach:** Dashboard subscribes to kind 24567 monitoring events that contain aggregated stats.
|
||||
|
||||
**New approach for real-time event feed:**
|
||||
```javascript
|
||||
// Dashboard subscribes to ALL events on the relay
|
||||
relayPool.subscribeMany([url], [
|
||||
{ kinds: [1, 3, 5, 7, ...], limit: 50 } // Recent events
|
||||
], {
|
||||
onevent(event) {
|
||||
// Render in real-time event feed
|
||||
addEventToFeed(event);
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
**What this replaces:** The "Relay Events" page in the dashboard currently shows events from monitoring. Instead, it would show live events as they arrive via normal Nostr subscription.
|
||||
|
||||
**What stays as monitoring events:** Aggregated stats (event_kinds, time_stats, top_pubkeys, cpu_metrics, subscription_details) still need to be generated by the relay because they require SQL aggregation queries that a client can't do.
|
||||
|
||||
### Phase 4: Evaluate Which Monitoring Events Can Be Replaced
|
||||
|
||||
For each current monitoring event type, here's whether it should stay as a relay-generated event or be replaced by dashboard-side logic:
|
||||
|
||||
| d-tag | Current: Relay generates | Could dashboard do it? | Recommendation |
|
||||
|-------|------------------------|----------------------|----------------|
|
||||
| `event_kinds` | SQL: `SELECT kind, COUNT(*) FROM events GROUP BY kind` | No — requires DB aggregation | **Keep as monitoring event** |
|
||||
| `time_stats` | SQL: `SELECT COUNT(*) FROM events WHERE created_at > ?` for 24h/7d/30d | No — requires DB aggregation | **Keep as monitoring event** |
|
||||
| `top_pubkeys` | SQL: `SELECT pubkey, COUNT(*) FROM events GROUP BY pubkey ORDER BY count DESC LIMIT 10` | No — requires DB aggregation | **Keep as monitoring event** |
|
||||
| `subscription_details` | SQL: `SELECT * FROM active_subscriptions_log` | No — requires DB access | **Keep as monitoring event** |
|
||||
| `cpu_metrics` | Reads `/proc/self/stat`, memory info | No — requires server-side access | **Keep as monitoring event** |
|
||||
| Real-time event feed | Currently via monitoring events | **Yes — normal Nostr subscription** | **Replace with REQ subscription** |
|
||||
| Auth rules list | Currently via admin command | No — requires DB access | **Keep as admin command** |
|
||||
| Config values | Currently via admin command | No — requires DB access | **Keep as admin command** |
|
||||
|
||||
**Conclusion:** All 5 monitoring event types should stay as relay-generated events (they require SQL aggregation or server-side data). The only thing that changes is the real-time event feed, which becomes a normal subscription.
|
||||
|
||||
### Phase 5: PostgreSQL LISTEN/NOTIFY for Monitoring Triggers (Future)
|
||||
|
||||
**What:** Instead of the monitor thread polling on a timer, PostgreSQL can push notifications when data changes.
|
||||
|
||||
**Example triggers:**
|
||||
```sql
|
||||
-- Notify when a new event is stored
|
||||
CREATE OR REPLACE FUNCTION notify_event_stored() RETURNS trigger AS $$
|
||||
BEGIN
|
||||
PERFORM pg_notify('event_stored', json_build_object(
|
||||
'kind', NEW.kind,
|
||||
'pubkey', substring(NEW.pubkey, 1, 8)
|
||||
)::text);
|
||||
RETURN NEW;
|
||||
END;
|
||||
$$ LANGUAGE plpgsql;
|
||||
|
||||
CREATE TRIGGER trg_notify_event_stored
|
||||
AFTER INSERT ON events
|
||||
FOR EACH ROW EXECUTE FUNCTION notify_event_stored();
|
||||
```
|
||||
|
||||
**How the monitor thread would use it:**
|
||||
```
|
||||
api-worker thread:
|
||||
1. LISTEN event_stored
|
||||
2. Wait for notification (blocks efficiently)
|
||||
3. On notification: batch up for throttle_sec, then generate monitoring events
|
||||
4. Repeat
|
||||
```
|
||||
|
||||
**Benefit:** Zero-polling. The monitor thread only wakes up when data actually changes.
|
||||
|
||||
**This is optional and can be added later.** The timer-based approach in Phase 1 works well and is simpler.
|
||||
|
||||
---
|
||||
|
||||
## Updated Thread Layout (After All Phases)
|
||||
|
||||
```
|
||||
c_relay_pg process
|
||||
├── lws-main — WebSocket event loop (never blocks on DB for monitoring)
|
||||
├── db-read-1..4 — Async REQ/COUNT queries
|
||||
├── event-worker — Async EVENT ingestion (validate + store)
|
||||
├── db-write-1..2 — Async sub logging, IP bans, misc writes
|
||||
└── api-worker — Periodic monitoring event generation (own PG conn)
|
||||
```
|
||||
|
||||
**Total: 9 threads, 9 PG connections** (with 4 readers + 2 writers)
|
||||
|
||||
---
|
||||
|
||||
## Implementation Priority
|
||||
|
||||
| Phase | Description | Complexity | Impact |
|
||||
|-------|-------------|-----------|--------|
|
||||
| 1 | Dedicated api-worker thread | Medium | High — unblocks main thread |
|
||||
| 2 | Remove monitoring from event processing path | Low | Medium — faster event processing |
|
||||
| 3 | Dashboard uses normal subscriptions for event feed | Low (frontend only) | Medium — simpler, more reliable |
|
||||
| 4 | Evaluate monitoring event types | Analysis only | Confirms architecture |
|
||||
| 5 | PostgreSQL LISTEN/NOTIFY triggers | Medium | Low (optimization) — add later |
|
||||
|
||||
**Recommended start:** Phase 1 + 2 together (they're coupled), then Phase 3 (frontend change), then Phase 5 when needed.
|
||||
|
||||
---
|
||||
|
||||
## Relationship to Thread Pool Plan
|
||||
|
||||
This plan complements `thread_pool_pg_optimization_plan.md`:
|
||||
|
||||
- Thread pool plan handles **reader/writer scaling** for client-facing operations
|
||||
- This plan handles **monitoring/API operations** on a dedicated thread
|
||||
- Both plans share the same PG connection model (one connection per thread)
|
||||
- The api-worker thread is independent from the thread pool — it has its own lifecycle
|
||||
|
||||
The combined target is:
|
||||
```
|
||||
lws-main (1) + readers (4) + event-worker (1) + writers (2) + api-worker (1) = 9 threads
|
||||
```
|
||||
@@ -0,0 +1,368 @@
|
||||
# Backfill Redesign: Per-Author Until-Cursor Drain (Option B)
|
||||
|
||||
## Problem
|
||||
|
||||
The current window-expansion backfill (`caching/src/backfill.c`) has four
|
||||
compounding structural defects that make cache coverage unacceptably poor
|
||||
(~10-28% of upstream availability even for a 7-pubkey follow set):
|
||||
|
||||
1. **Overlapping windows re-fetch the same events.** Window N+1's time range
|
||||
fully contains window N's range. The only dedup is a 4096-entry in-memory
|
||||
ring + inbox `ON CONFLICT DO NOTHING`, which overflows for high-volume
|
||||
authors, wasting query budget on duplicates.
|
||||
2. **Broken saturation detection.** When a relay caps at 500 events, the code
|
||||
retries with `limit=5000`, still gets 500, compares `500 == 5000` (false),
|
||||
and incorrectly concludes the page was not saturated — marking the author
|
||||
complete without paginating.
|
||||
3. **`until`-cursor walk-back never triggers** because it's gated on the
|
||||
broken saturation check.
|
||||
4. **Window 4 (full history) was skipped for 5/7 pubkeys** and the loop won't
|
||||
retry once `current_window_index` advances past the last window.
|
||||
|
||||
## Solution
|
||||
|
||||
Replace the window-expansion model with a **per-author `until`-cursor drain**:
|
||||
each followed author has a single persistent cursor (starting at `now`). Each
|
||||
backfill tick queries `since=0&until=cursor&limit=500` for one author,
|
||||
publishes the page, and sets `cursor = oldest_event_created_at - 1`. Repeat
|
||||
until a page returns fewer than the relay's cap (beginning of history reached).
|
||||
The live subscriber stays as-is (it's already correct).
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[cr_follow_resolve] --> B[caching_followed_pubkeys table]
|
||||
B --> C[Backfill tick: pick next incomplete author]
|
||||
C --> D[Query since=0 and until=cursor and limit=500]
|
||||
D --> E{ev_count == relay_cap?}
|
||||
E -- yes --> F[Publish page, set cursor = oldest - 1]
|
||||
F --> D
|
||||
E -- no < relay_cap --> G[Publish page, mark backfill_complete = true]
|
||||
G --> H{More incomplete authors?}
|
||||
H -- yes --> C
|
||||
H -- no --> I[Steady state: live sub only]
|
||||
J[Live subscriber] --> K[caching_event_inbox]
|
||||
K --> L[c-relay-pg inbox poller]
|
||||
L --> M[events table]
|
||||
```
|
||||
|
||||
## Changes
|
||||
|
||||
### 1. New table: `caching_followed_pubkeys`
|
||||
|
||||
Replaces both the ephemeral in-memory `cr_pubkey_set_t` (for persistence) and
|
||||
the window-coupled `caching_backfill_progress` (for cursor state).
|
||||
|
||||
**Schema** (add to `src/pg_schema.sql`):
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS caching_followed_pubkeys (
|
||||
pubkey TEXT PRIMARY KEY,
|
||||
is_root BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
until_cursor BIGINT NOT NULL DEFAULT 0,
|
||||
backfill_complete BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
events_fetched INTEGER NOT NULL DEFAULT 0,
|
||||
first_seen BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,
|
||||
last_seen BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,
|
||||
updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_caching_followed_incomplete
|
||||
ON caching_followed_pubkeys(backfill_complete, last_seen)
|
||||
WHERE backfill_complete = FALSE;
|
||||
```
|
||||
|
||||
- `until_cursor`: the `until` timestamp for the next backfill query. `0` means
|
||||
"not started yet" (treat as `now` on first query). After each page, set to
|
||||
`oldest_event_created_at - 1`.
|
||||
- `backfill_complete`: set true when a page returns fewer than the relay cap
|
||||
(we've reached the beginning of the author's history).
|
||||
- `is_root`: true for root/admin npubs (used to apply `admin_kinds`).
|
||||
- `events_fetched`: cumulative count for diagnostics.
|
||||
- `last_seen`: updated each follow-graph refresh (prune stale follows).
|
||||
|
||||
**Migration**: On startup, if `caching_followed_pubkeys` is empty but
|
||||
`caching_backfill_progress` has rows, migrate: for each distinct
|
||||
`author_pubkey` in the old table, insert a row with `until_cursor = 0`,
|
||||
`backfill_complete = FALSE` (re-backfill from scratch — the old data was
|
||||
incomplete anyway). Drop `caching_backfill_progress` after migration.
|
||||
|
||||
### 2. Follow graph resolver: persist to `caching_followed_pubkeys`
|
||||
|
||||
**File**: `caching/src/follow_graph.c` + `caching/src/pg_inbox.c`
|
||||
|
||||
After `cr_follow_resolve()` builds the in-memory set, sync it to the table:
|
||||
|
||||
```c
|
||||
/* For each pubkey in the followed set: */
|
||||
/* INSERT INTO caching_followed_pubkeys (pubkey, is_root, last_seen)
|
||||
* VALUES ($1, $2, now)
|
||||
* ON CONFLICT (pubkey) DO UPDATE SET
|
||||
* is_root = EXCLUDED.is_root,
|
||||
* last_seen = EXCLUDED.last_seen;
|
||||
*
|
||||
* Mark follows not seen this refresh as stale (for pruning / diagnostics).
|
||||
* Do NOT touch until_cursor or backfill_complete on existing rows. */
|
||||
```
|
||||
|
||||
New function: `pg_inbox_sync_followed_pubkeys(cr_pubkey_set_t *followed,
|
||||
cr_config_t *cfg)` — upserts all followed pubkeys, updates `last_seen`, sets
|
||||
`is_root` based on `cr_follow_is_root()`.
|
||||
|
||||
New function: `pg_inbox_prune_stale_follows(long threshold_seconds)` —
|
||||
optionally deletes follows not seen in `threshold_seconds` (e.g. 24h). This is
|
||||
off by default; stale follows just stop getting backfilled.
|
||||
|
||||
The in-memory `cr_pubkey_set_t followed` stays as the runtime working set
|
||||
(used by the live subscriber). The table is the durable source of truth for
|
||||
backfill iteration.
|
||||
|
||||
### 3. Rewrite `cr_backfill_tick` — per-author until-cursor drain
|
||||
|
||||
**File**: `caching/src/backfill.c`
|
||||
|
||||
Replace the entire window-expansion logic. New `cr_backfill_tick`:
|
||||
|
||||
```c
|
||||
int cr_backfill_tick(cr_backfill_t *bf, cr_config_t *cfg,
|
||||
nostr_relay_pool_t *upstream,
|
||||
cr_sink_t *sink, const cr_relay_map_t *relay_map) {
|
||||
if (!cfg->backfill.enabled) return -2;
|
||||
|
||||
/* Throttle: one query per tick_interval. */
|
||||
time_t now = time(NULL);
|
||||
if (bf->last_tick && (now - bf->last_tick) < cfg->backfill.tick_interval_seconds)
|
||||
return 0;
|
||||
bf->last_tick = now;
|
||||
|
||||
/* Pick the next incomplete author from the DB (round-robin via
|
||||
* last_seen ordering, or a cursor we persist in bf). */
|
||||
char pk[CR_HEX_LEN];
|
||||
long until_cursor = 0;
|
||||
int is_root = 0;
|
||||
if (pg_inbox_pick_next_backfill_author(pk, &until_cursor, &is_root,
|
||||
&bf->author_round_cursor) != 0) {
|
||||
/* No incomplete authors — steady state. */
|
||||
bf->in_progress = 0;
|
||||
return -2;
|
||||
}
|
||||
|
||||
/* First-time: cursor 0 means start from now. */
|
||||
if (until_cursor == 0) until_cursor = (long)now;
|
||||
|
||||
int page_size = 500; /* matches typical relay cap */
|
||||
int ev_count = 0;
|
||||
cJSON **events = query_author(upstream, relay_map, cfg, pk,
|
||||
0 /* since=0 full history */,
|
||||
until_cursor, page_size, &ev_count);
|
||||
|
||||
if (!events || ev_count == 0) {
|
||||
/* No events at all (or no more) — author is complete. */
|
||||
pg_inbox_update_backfill_progress(pk, 0, 1 /* complete */, 0);
|
||||
free(events);
|
||||
return 1;
|
||||
}
|
||||
|
||||
/* Find oldest timestamp BEFORE publishing. */
|
||||
long oldest_ts = 0;
|
||||
find_oldest_created_at(events, ev_count, &oldest_ts);
|
||||
|
||||
/* Publish all events. */
|
||||
cr_sink_set_source_class(sink, CR_SINK_CLASS_BACKFILL);
|
||||
for (int k = 0; k < ev_count; k++) {
|
||||
cr_sink_publish(sink, events[k]);
|
||||
cJSON_Delete(events[k]);
|
||||
}
|
||||
free(events);
|
||||
bf->events_total += ev_count;
|
||||
|
||||
/* Saturation check: if we got exactly page_size, the page may be
|
||||
* truncated. Set cursor = oldest - 1 and keep going next tick.
|
||||
* If we got fewer than page_size, we've reached the beginning. */
|
||||
if (ev_count >= page_size) {
|
||||
/* Saturated — more history likely exists. Advance cursor. */
|
||||
long next_cursor = (oldest_ts > 0) ? oldest_ts - 1 : 0;
|
||||
pg_inbox_update_backfill_progress(pk, next_cursor, 0 /* not complete */,
|
||||
ev_count);
|
||||
} else {
|
||||
/* Not saturated — beginning of history reached. */
|
||||
pg_inbox_update_backfill_progress(pk, until_cursor, 1 /* complete */,
|
||||
ev_count);
|
||||
}
|
||||
|
||||
return 1;
|
||||
}
|
||||
```
|
||||
|
||||
Key differences from the old code:
|
||||
- **No windows.** Single `since=0` query per author, always full history.
|
||||
- **Saturation check compares against `page_size` (500)**, not `max_cap`
|
||||
(5000). This fixes Defect 2 — if the relay returns 500, we paginate.
|
||||
- **Cursor persisted per-author** in `caching_followed_pubkeys.until_cursor`,
|
||||
not in a window-coupled table. This fixes Defect 4 — no window to skip.
|
||||
- **No overlapping queries.** Each event is fetched exactly once as the cursor
|
||||
walks backwards. This fixes Defect 1.
|
||||
- **`pg_inbox_pick_next_backfill_author`** selects the next
|
||||
`backfill_complete = FALSE` author, round-robin, so all authors get fair
|
||||
backfill time.
|
||||
|
||||
### 4. New PG inbox functions
|
||||
|
||||
**File**: `caching/src/pg_inbox.c` + `caching/src/pg_inbox.h`
|
||||
|
||||
```c
|
||||
/* Sync the followed set to the DB (upsert + update last_seen).
|
||||
* Does NOT touch until_cursor or backfill_complete on existing rows. */
|
||||
int pg_inbox_sync_followed_pubkeys(const cr_pubkey_set_t *followed,
|
||||
const cr_config_t *cfg);
|
||||
|
||||
/* Pick the next incomplete author for backfill (round-robin).
|
||||
* Sets out_pk, out_until_cursor, out_is_root.
|
||||
* Returns 0 on success, -1 if no incomplete authors remain. */
|
||||
int pg_inbox_pick_next_backfill_author(char *out_pk,
|
||||
long *out_until_cursor,
|
||||
int *out_is_root,
|
||||
int *round_cursor);
|
||||
|
||||
/* Update backfill progress for an author. */
|
||||
int pg_inbox_update_backfill_progress(const char *pk,
|
||||
long until_cursor,
|
||||
int complete,
|
||||
int events_this_page);
|
||||
|
||||
/* Migrate old caching_backfill_progress rows to caching_followed_pubkeys.
|
||||
* Called once on startup if the new table is empty and the old one has rows. */
|
||||
int pg_inbox_migrate_backfill_progress(void);
|
||||
|
||||
/* Reset all backfill progress (set until_cursor=0, backfill_complete=FALSE
|
||||
* for all followed pubkeys). Used by the "Reset Backfill" API button. */
|
||||
int pg_inbox_reset_backfill_progress(void);
|
||||
```
|
||||
|
||||
### 5. Simplify `cr_backfill_t` and `cr_config_t` state
|
||||
|
||||
**File**: `caching/src/backfill.h`, `caching/src/config.h`
|
||||
|
||||
Remove from `cr_backfill_t`:
|
||||
- `window_index`, `current_window_s`, `events_this_window`, `window_started`
|
||||
- `author_complete`, `blocked`, `blocked_until_ts`
|
||||
|
||||
Keep:
|
||||
- `cursor` (now: round-robin index into followed set, for fair scheduling)
|
||||
- `until_cursor` (transient, per-tick — not persisted here, read from DB)
|
||||
- `in_progress` (1 if any incomplete authors remain)
|
||||
- `events_total` (cumulative diagnostic)
|
||||
- `last_tick` (throttle)
|
||||
|
||||
Remove from `cr_config_t` state:
|
||||
- `state.backfilled_until` (no longer meaningful)
|
||||
- `state.current_window_index` (no windows)
|
||||
|
||||
Remove from `cr_config_t` backfill config:
|
||||
- `backfill.window_schedule_seconds[]`, `backfill.window_count`
|
||||
- `backfill.window_cooldown_seconds`
|
||||
- `backfill.events_per_tick` → rename to `backfill.page_size` (default 500)
|
||||
|
||||
Keep:
|
||||
- `backfill.enabled`
|
||||
- `backfill.page_size` (was `events_per_tick`)
|
||||
- `backfill.tick_interval_seconds`
|
||||
|
||||
### 6. Update `caching_service_state` table
|
||||
|
||||
**File**: `src/pg_schema.sql`
|
||||
|
||||
The `current_window_index` and `backfill_cursor` columns are no longer
|
||||
meaningful. Replace with:
|
||||
|
||||
```sql
|
||||
ALTER TABLE caching_service_state
|
||||
DROP COLUMN IF EXISTS current_window_index,
|
||||
DROP COLUMN IF EXISTS backfill_cursor,
|
||||
ADD COLUMN IF NOT EXISTS backfill_authors_complete INTEGER NOT NULL DEFAULT 0,
|
||||
ADD COLUMN IF NOT EXISTS backfill_authors_total INTEGER NOT NULL DEFAULT 0;
|
||||
```
|
||||
|
||||
Update `pg_inbox_update_status()` to report
|
||||
`backfill_authors_complete / backfill_authors_total` instead of
|
||||
`current_window_index / backfill_cursor`. The API status panel shows this as a
|
||||
progress bar (e.g. "Backfill: 3/7 authors complete").
|
||||
|
||||
### 7. Update main loop
|
||||
|
||||
**File**: `caching/src/main.c`
|
||||
|
||||
- Remove `advance_window` calls (no windows).
|
||||
- After `cr_follow_resolve()`, call `pg_inbox_sync_followed_pubkeys()`.
|
||||
- On startup, call `pg_inbox_migrate_backfill_progress()` if needed.
|
||||
- `cr_backfill_init` just sets `in_progress = 1` (check if any incomplete
|
||||
authors exist in the DB).
|
||||
- The follow-graph refresh at [`main.c:378`](caching/src/main.c:378) now also
|
||||
calls `pg_inbox_sync_followed_pubkeys()` so new follows get backfilled and
|
||||
unfollowed pubkeys stop getting backfilled.
|
||||
|
||||
### 8. Update API status display
|
||||
|
||||
**File**: `api/index.js`, `src/config.c` (caching_status handler)
|
||||
|
||||
- Show "Backfill: X/Y authors complete" instead of "Window N, cursor M".
|
||||
- The "Reset Backfill Progress" button now calls
|
||||
`pg_inbox_reset_backfill_progress()` which sets
|
||||
`until_cursor=0, backfill_complete=FALSE` for all rows in
|
||||
`caching_followed_pubkeys`.
|
||||
|
||||
### 9. Config changes
|
||||
|
||||
**File**: `src/pg_schema.sql` (default config inserts), `caching/src/pg_config.c`
|
||||
|
||||
Remove config keys:
|
||||
- `caching_backfill_windows` (no window schedule)
|
||||
- `caching_backfill_window_cooldown_seconds` (no windows)
|
||||
|
||||
Add config key:
|
||||
- `caching_backfill_page_size` already exists (value 500) — repurpose as the
|
||||
per-query limit (was already used this way, just rename in docs).
|
||||
|
||||
Keep:
|
||||
- `caching_backfill_enabled`
|
||||
- `caching_backfill_tick_interval_ms` (throttle between queries)
|
||||
- `caching_backfill_page_size` (per-query limit, default 500)
|
||||
|
||||
## Implementation Order
|
||||
|
||||
1. **Schema**: Add `caching_followed_pubkeys` table + alter
|
||||
`caching_service_state` in `src/pg_schema.sql`.
|
||||
2. **PG inbox functions**: Add `pg_inbox_sync_followed_pubkeys`,
|
||||
`pg_inbox_pick_next_backfill_author`, `pg_inbox_update_backfill_progress`,
|
||||
`pg_inbox_migrate_backfill_progress`, `pg_inbox_reset_backfill_progress`
|
||||
in `caching/src/pg_inbox.c` + `.h`.
|
||||
3. **Follow graph sync**: Call `pg_inbox_sync_followed_pubkeys()` after
|
||||
`cr_follow_resolve()` in `caching/src/main.c`.
|
||||
4. **Rewrite backfill**: Replace `cr_backfill_tick` in
|
||||
`caching/src/backfill.c`. Simplify `cr_backfill_t` in `backfill.h`.
|
||||
5. **Config cleanup**: Remove window config from `caching/src/pg_config.c`
|
||||
and `caching/src/config.c`. Update defaults in `src/pg_schema.sql`.
|
||||
6. **State reporting**: Update `pg_inbox_update_status()` in
|
||||
`caching/src/pg_inbox.c` to report authors complete/total.
|
||||
7. **API status**: Update `api/index.js` and `src/config.c` status handler
|
||||
to show backfill author progress.
|
||||
8. **Migration**: Implement `pg_inbox_migrate_backfill_progress()` and call
|
||||
on startup.
|
||||
9. **Test**: Rebuild, restart with `--reset-backfill`, verify 100% coverage
|
||||
for all 7 followed pubkeys via `nak req` comparison against upstream.
|
||||
|
||||
## Risk and Mitigation
|
||||
|
||||
- **Relays that don't honor `until`**: If a relay ignores the `until` filter,
|
||||
the cursor won't advance and we'll re-fetch the same page. Mitigation:
|
||||
detect when `oldest_ts` doesn't decrease between consecutive pages for the
|
||||
same author and mark the author complete (or skip to a different relay).
|
||||
- **High-volume authors take many ticks**: At 500 events/tick and 5s
|
||||
tick_interval, a 10,000-event author takes ~100s. This is acceptable — the
|
||||
live subscriber keeps you current, and the backfill makes steady progress.
|
||||
If faster backfill is needed later, Option C (parallel workers) can be added
|
||||
on top of this design.
|
||||
- **Follow graph churn**: If follows change frequently, the sync upsert is
|
||||
O(N) per refresh (every 10 min). Fine for personal-scale (hundreds of
|
||||
follows). For thousands, batch the upsert.
|
||||
@@ -0,0 +1,307 @@
|
||||
# Backfill v2: Relay-by-Relay Per-Author Until-Cursor Drain
|
||||
|
||||
## Overview
|
||||
|
||||
Refine the per-author until-cursor backfill to query each relay **one at a
|
||||
time** with **per-relay cursor tracking**, and pick up new follows
|
||||
**immediately** when the admin's kind-3 contact list changes.
|
||||
|
||||
## Problem with the current implementation
|
||||
|
||||
The current backfill (`caching/src/backfill.c`) calls `nostr_relay_pool_query_sync`
|
||||
with **all relay URLs at once**. This function:
|
||||
|
||||
1. Sends the same REQ to all connected relays
|
||||
2. Collects and deduplicates events from all relays into one merged set
|
||||
3. Waits until ALL relays send EOSE, or the timeout (now 30s) expires
|
||||
4. Returns the merged set — but **does not tell the caller whether it exited
|
||||
via all-EOSE or timeout**
|
||||
|
||||
If one relay is slow (e.g. nos.lol streaming 500+ events), the timeout can
|
||||
expire before that relay sends EOSE. The result is a partial event set, but
|
||||
the backfill code can't distinguish "partial due to timeout" from "complete
|
||||
because the author only has 225 events." This causes premature completion
|
||||
marking.
|
||||
|
||||
Additionally, the current design has a single `until_cursor` per author in
|
||||
`caching_followed_pubkeys`, which doesn't track per-relay progress. And new
|
||||
follows are only discovered on the 10-minute follow-graph refresh cycle.
|
||||
|
||||
## Solution
|
||||
|
||||
### 1. Per-relay cursor tracking
|
||||
|
||||
Add a new table to track the backfill cursor for each author on each relay
|
||||
independently:
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS caching_backfill_relay_progress (
|
||||
author_pubkey TEXT NOT NULL,
|
||||
relay_url TEXT NOT NULL,
|
||||
until_cursor BIGINT NOT NULL DEFAULT 0,
|
||||
complete BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
events_fetched INTEGER NOT NULL DEFAULT 0,
|
||||
updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,
|
||||
PRIMARY KEY (author_pubkey, relay_url)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_caching_relay_progress_incomplete
|
||||
ON caching_backfill_relay_progress(author_pubkey)
|
||||
WHERE complete = FALSE;
|
||||
```
|
||||
|
||||
- `until_cursor`: the `until` timestamp for the next query to this specific
|
||||
relay for this author. `0` means "not started" (treat as `now`).
|
||||
- `complete`: set true when this relay returns < page_size events for the
|
||||
current cursor range (this relay is drained for this author).
|
||||
- `events_fetched`: cumulative count for diagnostics.
|
||||
|
||||
The existing `caching_followed_pubkeys.backfill_complete` becomes a derived
|
||||
value: an author is complete when all their relay progress rows are complete
|
||||
(or when they have no relay progress rows at all — no relays to query).
|
||||
|
||||
### 2. Relay-by-relay querying
|
||||
|
||||
Instead of one `query_sync` call with N relays, make N `query_sync` calls
|
||||
with 1 relay each. Each call waits for that single relay's EOSE (or timeout).
|
||||
No merge/dedup needed — publish every event directly to the inbox; the
|
||||
database handles deduplication (`ON CONFLICT DO NOTHING`).
|
||||
|
||||
### 3. Immediate new-follow detection
|
||||
|
||||
When the admin publishes a new kind-3 contact list, the live subscriber's
|
||||
`on_event` callback receives it in real-time. We detect this and trigger an
|
||||
immediate follow-graph refresh, which:
|
||||
- Resolves the new follow set
|
||||
- Syncs new pubkeys to `caching_followed_pubkeys`
|
||||
- Creates relay progress rows for new follows (cursor=0, complete=false)
|
||||
- The backfill loop picks them up on the next tick
|
||||
|
||||
## Algorithm
|
||||
|
||||
### Per backfill tick:
|
||||
|
||||
```
|
||||
1. Pick next author that has at least one incomplete relay progress row
|
||||
(round-robin from caching_followed_pubkeys WHERE backfill_complete=FALSE)
|
||||
2. Get incomplete relays for this author:
|
||||
SELECT relay_url, until_cursor FROM caching_backfill_relay_progress
|
||||
WHERE author_pubkey = $1 AND complete = FALSE
|
||||
3. For each incomplete relay (one at a time):
|
||||
a. cursor = until_cursor (0 = now)
|
||||
b. query_sync(pool, &relay_url, 1, filter, &ev_count, 30000)
|
||||
filter = {authors: [pk], kinds: [configured], since: 0,
|
||||
until: cursor, limit: 500}
|
||||
c. Publish every event directly to inbox (cr_sink_publish)
|
||||
d. Find oldest_ts in this batch
|
||||
e. If ev_count == 0:
|
||||
- Mark this relay complete for this author
|
||||
(UPDATE caching_backfill_relay_progress SET complete=TRUE)
|
||||
f. If ev_count >= page_size (saturated):
|
||||
- Advance this relay's cursor: cursor = oldest_ts - 1
|
||||
(UPDATE caching_backfill_relay_progress SET until_cursor = oldest-1)
|
||||
g. If ev_count < page_size but > 0:
|
||||
- This relay is drained for this cursor range
|
||||
- Advance cursor to oldest_ts - 1 AND mark complete
|
||||
(the next query at this lower cursor would return 0 anyway,
|
||||
but we mark complete to avoid an extra round-trip)
|
||||
4. After querying all incomplete relays for this author:
|
||||
Check if ALL relays for this author are now complete:
|
||||
SELECT COUNT(*) WHERE author_pubkey=$1 AND complete=FALSE
|
||||
If 0: mark author complete in caching_followed_pubkeys
|
||||
```
|
||||
|
||||
### Why per-relay cursors are better
|
||||
|
||||
- **Relay A has 500+ events, relay B has 50**: Relay A gets paginated across
|
||||
multiple ticks (cursor walks back 500 at a time). Relay B is drained in one
|
||||
query and marked complete. The author stays incomplete until relay A is
|
||||
also drained. With a single shared cursor, we'd advance the cursor based on
|
||||
the oldest across both relays, potentially skipping events on relay A.
|
||||
|
||||
- **Resume after restart**: each relay's cursor is persisted independently.
|
||||
On restart, we resume each relay exactly where it left off.
|
||||
|
||||
- **Relay goes offline**: if a relay is unreachable, its progress row stays
|
||||
incomplete. The author stays incomplete. When the relay comes back, we
|
||||
resume querying it. No events are lost.
|
||||
|
||||
### Completion criteria
|
||||
|
||||
An author is marked `backfill_complete = TRUE` in `caching_followed_pubkeys`
|
||||
when ALL their relay progress rows in `caching_backfill_relay_progress` are
|
||||
marked `complete = TRUE`. This means every known relay has been queried down
|
||||
to the beginning of the author's history.
|
||||
|
||||
### New follow detection
|
||||
|
||||
In `caching/src/live_subscriber.c`, the `live_on_event` callback currently
|
||||
just publishes events. We add a check:
|
||||
|
||||
```c
|
||||
static void live_on_event(cJSON *event, const char *relay_url, void *user_data) {
|
||||
cr_live_ctx_t *ctx = (cr_live_ctx_t *)user_data;
|
||||
ctx->live->events_received++;
|
||||
cr_sink_publish(ctx->sink, event);
|
||||
|
||||
/* Detect admin kind-3 (contact list) changes → trigger follow refresh */
|
||||
cJSON *kind = cJSON_GetObjectItem(event, "kind");
|
||||
cJSON *pubkey = cJSON_GetObjectItem(event, "pubkey");
|
||||
if (kind && cJSON_IsNumber(kind) && kind->valuedouble == 3 &&
|
||||
pubkey && cJSON_IsString(pubkey)) {
|
||||
if (cr_follow_is_root(ctx->cfg, cJSON_GetStringValue(pubkey))) {
|
||||
ctx->live->follow_graph_changed = 1; /* signal to main loop */
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The main loop checks this flag each iteration:
|
||||
|
||||
```c
|
||||
if (live.follow_graph_changed) {
|
||||
live.follow_graph_changed = 0;
|
||||
/* Immediate follow-graph refresh */
|
||||
cr_pubkey_set_t new_followed;
|
||||
cr_pubkey_set_init(&new_followed);
|
||||
if (cr_follow_resolve(&cfg, upstream, &new_followed) == 0) {
|
||||
/* Sync to DB — new follows get relay progress rows automatically */
|
||||
pg_inbox_sync_followed_pubkeys(...);
|
||||
/* Create relay progress rows for new follows */
|
||||
pg_inbox_init_relay_progress_for_new_follows(&new_followed, &relay_map);
|
||||
/* Resubscribe live with new follow set */
|
||||
cr_live_resubscribe(&live, &cfg, upstream, &new_followed, &sink);
|
||||
cr_pubkey_set_free(&followed);
|
||||
followed = new_followed;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Relay progress row creation
|
||||
|
||||
When a new follow is discovered, we need to create relay progress rows for
|
||||
it. The relays to query are determined by the relay discovery phase (NIP-65
|
||||
outbox relays + bootstrap relays). New function:
|
||||
|
||||
```c
|
||||
int pg_inbox_init_relay_progress_for_author(const char *pk,
|
||||
const char **relay_urls,
|
||||
int relay_count);
|
||||
```
|
||||
|
||||
For each relay URL, insert a row with `until_cursor=0, complete=FALSE` (if
|
||||
not already present). This is called:
|
||||
- After follow-graph resolution for new follows
|
||||
- After relay discovery for newly discovered outbox relays
|
||||
|
||||
## Database schema changes
|
||||
|
||||
### New table: `caching_backfill_relay_progress`
|
||||
|
||||
(see SQL above)
|
||||
|
||||
### `caching_followed_pubkeys` — unchanged
|
||||
|
||||
The `until_cursor` and `backfill_complete` columns remain. `until_cursor`
|
||||
becomes less important (per-relay cursors are the source of truth), but we
|
||||
keep it for backward compatibility and as a quick "has this author started"
|
||||
flag. `backfill_complete` is updated when all relay progress rows are
|
||||
complete.
|
||||
|
||||
## New PG inbox functions
|
||||
|
||||
### `caching/src/pg_inbox.c` + `.h`
|
||||
|
||||
```c
|
||||
/* Initialize relay progress rows for an author. Creates one row per relay
|
||||
* with until_cursor=0, complete=FALSE. Skips rows that already exist. */
|
||||
int pg_inbox_init_relay_progress_for_author(const char *pk,
|
||||
const char **relay_urls,
|
||||
int relay_count);
|
||||
|
||||
/* Pick the next author that has at least one incomplete relay progress row.
|
||||
* Round-robin via round_cursor. Returns 0 on success, -1 if none. */
|
||||
int pg_inbox_pick_next_author_with_incomplete_relays(char *out_pk, int pk_len,
|
||||
int *round_cursor);
|
||||
|
||||
/* Get incomplete relays for an author. Returns a JSON array of
|
||||
* {relay_url, until_cursor} objects. Caller frees. */
|
||||
cJSON *pg_inbox_get_incomplete_relays(const char *pk);
|
||||
|
||||
/* Update relay progress: advance cursor and/or mark complete. */
|
||||
int pg_inbox_update_relay_progress(const char *pk, const char *relay_url,
|
||||
long until_cursor, int complete,
|
||||
int events_this_page);
|
||||
|
||||
/* Check if all relays for an author are complete. Returns 1 if all complete,
|
||||
* 0 if some incomplete, -1 on error. */
|
||||
int pg_inbox_check_author_relays_complete(const char *pk);
|
||||
|
||||
/* Mark author complete in caching_followed_pubkeys if all relay progress
|
||||
* rows are complete. Called after each relay update. */
|
||||
int pg_inbox_check_and_mark_author_complete(const char *pk);
|
||||
```
|
||||
|
||||
## Changes
|
||||
|
||||
### 1. `src/pg_schema.sql` + `src/pg_schema.h`
|
||||
|
||||
Add the `caching_backfill_relay_progress` table and index.
|
||||
|
||||
### 2. `caching/src/pg_inbox.c` + `.h`
|
||||
|
||||
Add the new relay progress functions listed above.
|
||||
|
||||
### 3. `caching/src/backfill.c` — rewrite `cr_backfill_tick`
|
||||
|
||||
Replace the single `query_author` call with a relay-by-relay loop using
|
||||
per-relay cursors from the database.
|
||||
|
||||
### 4. `caching/src/live_subscriber.c` — detect admin kind-3 changes
|
||||
|
||||
Add `follow_graph_changed` flag to `cr_live_t`. Set it in `live_on_event`
|
||||
when a root npub publishes a kind-3.
|
||||
|
||||
### 5. `caching/src/live_subscriber.h` — add flag to struct
|
||||
|
||||
Add `int follow_graph_changed;` to `cr_live_t`.
|
||||
|
||||
### 6. `caching/src/main.c` — handle follow_graph_changed signal
|
||||
|
||||
Check `live.follow_graph_changed` each main loop iteration. If set, trigger
|
||||
immediate follow-graph refresh + relay progress initialization for new
|
||||
follows.
|
||||
|
||||
### 7. `caching/src/main.c` — create relay progress rows after relay discovery
|
||||
|
||||
After `cr_relay_discovery_run()`, call `pg_inbox_init_relay_progress_for_author`
|
||||
for each followed pubkey with its discovered outbox relays + bootstrap relays.
|
||||
|
||||
## Implementation order
|
||||
|
||||
1. Schema: add `caching_backfill_relay_progress` table
|
||||
2. PG inbox functions: relay progress CRUD
|
||||
3. Backfill rewrite: relay-by-relay loop with per-relay cursors
|
||||
4. Live subscriber: kind-3 detection + `follow_graph_changed` flag
|
||||
5. Main loop: handle `follow_graph_changed`, create relay progress rows
|
||||
6. Build and test
|
||||
|
||||
## Testing
|
||||
|
||||
1. Rebuild: `cd caching && make`
|
||||
2. Reset backfill: restart relay with `--reset-backfill`
|
||||
3. Monitor relay progress:
|
||||
```sql
|
||||
SELECT author_pubkey, relay_url, until_cursor, complete, events_fetched
|
||||
FROM caching_backfill_relay_progress
|
||||
ORDER BY author_pubkey, relay_url;
|
||||
```
|
||||
4. Monitor author completion:
|
||||
```sql
|
||||
SELECT pubkey, backfill_complete
|
||||
FROM caching_followed_pubkeys ORDER BY pubkey;
|
||||
```
|
||||
5. Compare coverage: `nak req -k 1 -a <pk> ws://localhost:7777` vs upstream
|
||||
6. Test new-follow detection: follow a new pubkey from the admin account,
|
||||
verify it appears in `caching_followed_pubkeys` and gets backfilled within
|
||||
seconds (not 10 minutes)
|
||||
@@ -0,0 +1,240 @@
|
||||
# C-Relay-PG Admin Page Migration Plan
|
||||
|
||||
## Overview
|
||||
|
||||
Migrate the relay admin dashboard from an embedded HTTP-served page (`api/index.html` + `api/index.js`) to a full client-ndk page (`c-relay-pg.html`) built on the client-ndk template. The page will live in the client-ndk project (`~/lt/client-ndk/www/`) alongside other pages and communicate with the relay purely through Nostr protocol subscriptions.
|
||||
|
||||
## Why This Migration
|
||||
|
||||
The current `/api` page is served directly by the relay's `lws-main` thread, which causes:
|
||||
- **Slow page delivery** — HTTP file serving competes with WebSocket event processing on a single thread
|
||||
- **Incomplete page loads** — 10-second `timeout_secs` kills connections when the main thread is busy
|
||||
- **6+ sequential HTTP round-trips** — each asset requires a two-phase header/body delivery cycle
|
||||
- **Unnecessary complexity** — embedded file generation, `lws_set_wsi_user()` overwriting, session data type dispatch
|
||||
|
||||
The admin page already communicates entirely via Nostr protocol:
|
||||
- **Kind 23456** → Admin commands (encrypted, sent by admin)
|
||||
- **Kind 23457** → Admin responses (encrypted, sent by relay)
|
||||
- **Kind 24567** → Monitoring/stats events (ephemeral, broadcast by relay)
|
||||
- **NIP-11** → Relay info (fetched via HTTP, independent of file serving)
|
||||
- **NIP-17** → DM admin commands (gift-wrapped)
|
||||
|
||||
Moving to client-ndk means the relay serves zero HTML — it only speaks Nostr protocol.
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph client-ndk project
|
||||
A[c-relay-pg.html] --> B[init-ndk.mjs]
|
||||
B --> C[NDK SharedWorker]
|
||||
end
|
||||
|
||||
subgraph c-relay-pg
|
||||
D[WebSocket Server]
|
||||
E[api-worker thread]
|
||||
end
|
||||
|
||||
C -->|kind 23456 admin cmds| D
|
||||
D -->|kind 23457 responses| C
|
||||
E -->|kind 24567 monitoring| D
|
||||
D -->|kind 24567 events| C
|
||||
C -->|kind 1059 NIP-17 DMs| D
|
||||
```
|
||||
|
||||
## Current Admin Page Sections
|
||||
|
||||
| Section | Data Source | Communication |
|
||||
|---------|-----------|---------------|
|
||||
| Statistics | kind 24567 monitoring events with d-tags: event_kinds, time_stats, top_pubkeys, cpu_metrics | Subscribe to kind 24567 |
|
||||
| Subscriptions | kind 24567 monitoring events with d-tag: subscription_details | Subscribe to kind 24567 |
|
||||
| Configuration | kind 23457 admin responses | Send kind 23456 config_query, receive kind 23457 |
|
||||
| Authorization | kind 23457 admin responses | Send kind 23456 auth rule commands, receive kind 23457 |
|
||||
| IP Bans | kind 23457 admin responses | Send kind 23456 ip_ban_* commands, receive kind 23457 |
|
||||
| Relay Events | Live subscription to all events + kind 0/10050/10002 management | Standard Nostr subscriptions |
|
||||
| DM | NIP-17 gift-wrapped DMs kind 1059 | NIP-17 encrypt/wrap/publish |
|
||||
| SQL Query | kind 23457 admin responses | Send kind 23456 sql_query, receive kind 23457 |
|
||||
|
||||
## Design Decisions
|
||||
|
||||
1. **Full client-ndk page** — Uses `client.css`, `init-ndk.mjs`, `HamburgerMorphing`, `relay-ui.mjs`, `blossom-ui.mjs`, `ai-ui.mjs` — all standard template infrastructure stays intact
|
||||
2. **Sidenav preserved** — All existing sidenav sections (Relay, Blossom, AI) remain. Admin page navigation links are added to `divSideNavBody` as clickable items
|
||||
3. **Page-specific CSS** — Follows client-ndk convention: page styles in inline `<style>` block, using only CSS variables from `client.css`
|
||||
4. **Hardcoded relay URL** — `wss://relay.laantungir.net` for now; future enhancement to support relay selection
|
||||
5. **Auth mode: required** — Admin page requires authentication
|
||||
|
||||
## Migration Steps
|
||||
|
||||
### Step 1: Create `c-relay-pg.html` from Template
|
||||
|
||||
Start from `template.html` in client-ndk. Customize:
|
||||
|
||||
- **Title**: `C-Relay-PG`
|
||||
- **Header text**: `C-Relay-PG Admin`
|
||||
- **Auth mode**: `required` (default)
|
||||
- **`divBody` content**: All 8 admin sections, each in its own container div, all `display:none` except Statistics (default)
|
||||
- **`divSideNavBody`**: Add navigation links for each section:
|
||||
```html
|
||||
<div id="divAdminNav">
|
||||
<div class="adminNavItem active" data-section="statistics">Statistics</div>
|
||||
<div class="adminNavItem" data-section="subscriptions">Subscriptions</div>
|
||||
<div class="adminNavItem" data-section="configuration">Configuration</div>
|
||||
<div class="adminNavItem" data-section="authorization">Authorization</div>
|
||||
<div class="adminNavItem" data-section="ip-bans">IP Bans</div>
|
||||
<div class="adminNavItem" data-section="relay-events">Relay Events</div>
|
||||
<div class="adminNavItem" data-section="dm">DM</div>
|
||||
<div class="adminNavItem" data-section="database">Database Query</div>
|
||||
</div>
|
||||
```
|
||||
|
||||
### Step 2: Port Admin Communication Layer
|
||||
|
||||
Replace `SimplePool` + `nostr-lite.js` with NDK equivalents:
|
||||
|
||||
| Current index.js | New c-relay-pg.html |
|
||||
|-----------------|---------------------|
|
||||
| `new SimplePool()` | `subscribe()` from init-ndk.mjs |
|
||||
| `relayPool.subscribeMany()` | `subscribe(filter, opts)` |
|
||||
| `relayPool.publish()` | `publishEvent(event)` |
|
||||
| `window.nostr.signEvent()` | Handled by NDK MessageBasedSigner |
|
||||
| `window.NostrTools.nip42` | Handled by NDK NIP-42 support |
|
||||
| `window.NostrTools.nip44` | `window.NostrTools.nip44` from nostr.bundle.js — same as before |
|
||||
| `nlLite` nostr-login-lite | `initNDKPage()` + `getPubkey()` |
|
||||
|
||||
Key function — `sendAdminCommand(commandArray)`:
|
||||
```javascript
|
||||
async function sendAdminCommand(commandArray) {
|
||||
const conversationKey = window.NostrTools.nip44.v2.utils.getConversationKey(
|
||||
adminPrivateKey, relayPubkey
|
||||
);
|
||||
const encrypted = window.NostrTools.nip44.v2.encrypt(
|
||||
JSON.stringify(commandArray), conversationKey
|
||||
);
|
||||
|
||||
const event = {
|
||||
created_at: Math.floor(Date.now() / 1000),
|
||||
kind: 23456,
|
||||
tags: [['p', relayPubkey]],
|
||||
content: encrypted
|
||||
};
|
||||
|
||||
return await publishEvent(event);
|
||||
}
|
||||
```
|
||||
|
||||
Note: NIP-44 encryption requires the admin's private key. The current page uses `window.nostr.nip44.encrypt()` which delegates to the browser extension. NDK's `publishEvent` handles signing, but encryption must be done before publishing. We'll use `window.nostr.nip44.encrypt()` if available (NIP-07 extension), same as the current page does.
|
||||
|
||||
### Step 3: Port Subscription Setup
|
||||
|
||||
Using NDK's `subscribe()` and `window.addEventListener('ndkEvent', ...)`:
|
||||
|
||||
```javascript
|
||||
// Subscribe to admin responses + monitoring events from relay
|
||||
const adminSub = subscribe(
|
||||
{ kinds: [23457, 24567], authors: [RELAY_PUBKEY] },
|
||||
{ closeOnEose: false, cacheUsage: 'ONLY_RELAY' }
|
||||
);
|
||||
|
||||
// Live event feed
|
||||
const liveSub = subscribe(
|
||||
{ limit: 50 },
|
||||
{ closeOnEose: false, cacheUsage: 'ONLY_RELAY' }
|
||||
);
|
||||
|
||||
// Route events to handlers
|
||||
window.addEventListener('ndkEvent', (e) => {
|
||||
const evt = e.detail;
|
||||
if (evt.kind === 23457) processAdminResponse(evt);
|
||||
else if (evt.kind === 24567) processMonitoringEvent(evt);
|
||||
else addEventToLiveFeed(evt);
|
||||
});
|
||||
```
|
||||
|
||||
### Step 4: Port NIP-44 Encryption
|
||||
|
||||
Admin commands require NIP-44 encryption. The template already loads `nostr.bundle.js` which includes `window.NostrTools.nip44`. Additionally, `window.nostr.nip44.encrypt()` is available from NIP-07 extensions. Use the same approach as the current page.
|
||||
|
||||
### Step 5: Port UI Sections into divBody
|
||||
|
||||
Each section's HTML goes into `divBody` as a container div. Page-specific styles go in the inline `<style>` block following client-ndk conventions:
|
||||
- Use CSS variables only (no hardcoded colors)
|
||||
- Tables use `var(--primary-color)`, `var(--muted-color)`, `var(--accent-color)`
|
||||
- Buttons follow the `var(--button-*)` pattern
|
||||
- Font is always `var(--font-family)`
|
||||
|
||||
Sections to port:
|
||||
1. **Statistics** — DB overview table, event kinds table, time stats table, top pubkeys table, event rate chart area
|
||||
2. **Subscriptions** — Active subscription details table
|
||||
3. **Configuration** — Config key/value table with inline edit, refresh button
|
||||
4. **Authorization** — Auth rules table, whitelist/blacklist inputs, WoT level selector
|
||||
5. **IP Bans** — Ban stats, manual ban form, IP whitelist, ban list table with filters
|
||||
6. **Relay Events** — Live event feed table, kind 0/10050/10002 management forms
|
||||
7. **DM** — NIP-17 message textarea + send button, inbox display
|
||||
8. **SQL Query** — Query dropdown, SQL textarea, execute button, results table
|
||||
|
||||
### Step 6: Page-Specific CSS
|
||||
|
||||
Following client-ndk rules from `client.css`:
|
||||
- Override `#divBody` layout: `flex-direction: column; overflow-y: auto; padding: 20px;`
|
||||
- Admin section containers: hidden by default, shown when nav item clicked
|
||||
- Table styles: use `var(--primary-color)` for headers, `var(--muted-color)` for borders
|
||||
- Button styles: border + transparent background, hover with `var(--accent-color)`
|
||||
- All in inline `<style media="screen">` block in `<head>`
|
||||
|
||||
### Step 7: Relay URL — Hardcoded for Now
|
||||
|
||||
```javascript
|
||||
const RELAY_WS_URL = 'wss://relay.laantungir.net';
|
||||
const RELAY_HTTP_URL = 'https://relay.laantungir.net';
|
||||
```
|
||||
|
||||
NIP-11 fetch to get relay pubkey:
|
||||
```javascript
|
||||
const nip11 = await fetch(RELAY_HTTP_URL, {
|
||||
headers: { 'Accept': 'application/nostr+json' }
|
||||
}).then(r => r.json());
|
||||
const RELAY_PUBKEY = nip11.pubkey;
|
||||
```
|
||||
|
||||
### Step 8: Admin Verification
|
||||
|
||||
Same flow as current page:
|
||||
1. Get relay pubkey from NIP-11
|
||||
2. Send kind 23456 `system_status` command
|
||||
3. If relay responds with kind 23457, user is verified as admin
|
||||
4. Show admin sections; otherwise show access denied
|
||||
|
||||
### Step 9 (Optional, Later): Remove Embedded HTTP Serving from Relay
|
||||
|
||||
After migration is complete and tested:
|
||||
- Remove `handle_embedded_file_request()` from `src/api.c`
|
||||
- Remove `handle_embedded_file_writeable()` from `src/api.c`
|
||||
- Remove embedded file dispatch from `LWS_CALLBACK_HTTP` in `src/websockets.c`
|
||||
- Remove `embed_web_files.sh` and `src/embedded_web_content.c`
|
||||
- Keep NIP-11 serving (lightweight and required by protocol)
|
||||
|
||||
## File Changes Summary
|
||||
|
||||
### New File (in client-ndk project: ~/lt/client-ndk/www/)
|
||||
- `c-relay-pg.html` — Complete admin page, self-contained with inline `<style>` and `<script type="module">`
|
||||
|
||||
### Files Unchanged
|
||||
- All relay C code for kind 23456/23457/24567 processing — stays exactly the same
|
||||
- `src/dm_admin.c` — NIP-17 DM handling stays the same
|
||||
- `src/nip011.c` — NIP-11 stays the same
|
||||
- All client-ndk shared files (client.css, init-ndk.mjs, relay-ui.mjs, etc.) — no modifications needed
|
||||
|
||||
### Files to Modify Later (optional relay cleanup)
|
||||
- `src/api.c` — Remove embedded file serving functions
|
||||
- `src/websockets.c` — Remove embedded file dispatch from HTTP callback
|
||||
- `Makefile` — Remove embedded web content compilation step
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|-----------|
|
||||
| NIP-44 encryption via window.nostr.nip44 | Same mechanism as current page — no change in encryption approach |
|
||||
| NDK SharedWorker relay connection | Use ONLY_RELAY cache usage to ensure direct relay communication |
|
||||
| Large page size with all sections inline | Sections are display:none by default — no performance impact |
|
||||
| Event rate chart from text_graph.js | Port chart logic inline or skip initially |
|
||||
| Relay pubkey needed before subscriptions | Fetch NIP-11 first in initialization sequence |
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user