Compare commits

...
20 Commits
Author SHA1 Message Date
Laan Tungir 9a9365dcfa v2.1.10 - Fix NIP-50 PG search semantics and NIP-17 test compatibility 2026-04-02 13:00:34 -04:00
Laan Tungir 5577b57149 v2.1.9 - Rebrand repository, docs, scripts, and tooling from c-relay to c-relay-pg 2026-04-02 07:06:41 -04:00
Laan Tungir 22c943d495 v2.1.8 - Implement explicit WAL checkpoint scheduling and disable SQLite auto-checkpoint 2026-04-01 21:03:24 -04:00
Laan Tungir 7e3d9b6825 v2.1.7 - Refactor SQLite connection ownership into db_ops, remove direct sqlite usage from runtime modules, and document agent-browser admin testing flow 2026-04-01 19:48:29 -04:00
Laan Tungir 547d22d09f v2.1.6 - Fix strict-mode admin API key access by prewarming config and relay private key caches 2026-04-01 19:08:42 -04:00
Laan Tungir ce9ae5a488 v2.1.5 - Offload duplicate and COUNT paths, add config/NIP-11 caching, and reduce DB reader threads 2026-04-01 16:15:46 -04:00
Laan Tungir 9b248b740f v2.1.4 - Harden async EVENT fallback handling and verify relay/nginx 503 behavior 2026-04-01 15:19:28 -04:00
Laan Tungir 0f77aeb72b v2.1.3 - Fix async EVENT thread safety by splitting store core/post-actions and naming lws main thread 2026-04-01 11:29:14 -04:00
Laan Tungir bb64651a0c v2.1.2 - Fix async REQ callback leak and validate Phase 2+3 subscription offload path 2026-04-01 11:01:38 -04:00
Laan Tungir 17be9c2b03 v2.1.1 - Wire REQ/COUNT/EVENT paths through thread pool sync helpers and stabilize filter limit tests 2026-04-01 09:38:25 -04:00
Laan Tungir f396c622b6 v2.1.0 - Bump to v2.1.0 as SQLite baseline for PostgreSQL fork handoff 2026-04-01 09:14:23 -04:00
Laan Tungir cbd260c1ae v2.0.4 - Complete Phase 1 DB abstraction: opaque db_stmt API, ip_ban migration, and SQLite leakage cleanup 2026-04-01 09:10:42 -04:00
Laan Tungir bc9ac290ab v2.0.3 - Complete Phase 1 SQLite abstraction cleanup across config/api/websockets and add db path accessor 2026-04-01 07:47:21 -04:00
Laan Tungir 192eeb248d v2.0.2 - Stabilize websocket queue draining, add db config count helper, and align NIP-45/50 test expectations 2026-04-01 07:04:58 -04:00
Laan Tungir da6c505420 v2.0.1 - Fix test-mode build/restart pipeline and NIP-11 root path handling; validate core NIP tests 2026-04-01 05:43:57 -04:00
Laan Tungir c077c209e5 v2.0.0 - Major architecture refactor: db_ops abstraction, low-risk SQLite refactor, and thread-pool scaffolding 2026-04-01 05:25:03 -04:00
Laan Tungir 9bf02702c2 v1.2.56 - Update codebase and documentation 2026-04-01 04:51:44 -04:00
Your Name 0920cc092d v1.2.55 - Preserve relay version metadata for API header/title and accept JSON NIP-11 content type 2026-03-23 09:09:50 -04:00
Your Name a89460be5d v1.2.54 - Sync relay_version to compiled CRELAY_VERSION on startup and show version in API header/title 2026-03-23 09:04:49 -04:00
Your Name ff2a3aa335 v1.2.53 - Fix P0 config-value memory leaks in hot paths and add investigation plan 2026-03-19 07:59:40 -04:00
130 changed files with 19714 additions and 2763 deletions
+11 -11
View File
@@ -1,12 +1,12 @@
# AGENTS.md - AI Agent Integration Guide for Architect Mode
**Project-Specific Information for AI Agents Working with C-Relay in Architect Mode**
**Project-Specific Information for AI Agents Working with C-Relay-PG in Architect Mode**
## Critical Architecture Understanding
### System Architecture Overview
C-Relay implements a **unique event-based configuration architecture** that fundamentally differs from traditional Nostr relays:
C-Relay-PG implements a **unique event-based configuration architecture** that fundamentally differs from traditional Nostr relays:
```
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
@@ -51,9 +51,9 @@ C-Relay implements a **unique event-based configuration architecture** that fund
## Architectural Decision Analysis
### Configuration System Design
**Traditional Approach vs C-Relay:**
**Traditional Approach vs C-Relay-PG:**
```
Traditional: C-Relay:
Traditional: C-Relay-PG:
config.json → kind 33334 events
ENV variables → cryptographically signed tags
File watching → database polling/restart
@@ -119,18 +119,18 @@ File watching → database polling/restart
```
Developer Machine:
├── ./make_and_restart_relay.sh
├── build/c_relay_x86
├── build/c_relay_pg_x86
├── build/<relay_pubkey>.db
└── relay.log
```
### Production SystemD Deployment
```
/opt/c-relay/:
├── c_relay_x86
/opt/c-relay-pg/:
├── c_relay_pg_x86
├── <relay_pubkey>.db
├── systemd service (c-relay.service)
└── c-relay user isolation
├── systemd service (c-relay-pg.service)
└── c-relay-pg user isolation
```
### Container Deployment Architecture
@@ -144,7 +144,7 @@ Container:
### Reverse Proxy Architecture
```
Internet → Nginx/HAProxy → C-Relay
Internet → Nginx/HAProxy → C-Relay-PG
├── WebSocket upgrade handling
├── SSL termination
└── Rate limiting
@@ -292,7 +292,7 @@ Admin Signs Event → WebSocket Submit → Validate → Store → Restart Requir
**Decision**: Same port serves both protocols
**Consequences**: Simplified deployment, protocol detection overhead, libwebsockets dependency
These architectural decisions form the foundation of C-Relay's unique approach to Nostr relay implementation and should be carefully considered when planning extensions or modifications.
These architectural decisions form the foundation of C-Relay-PG's unique approach to Nostr relay implementation and should be carefully considered when planning extensions or modifications.
**
[Response interrupted by a tool use result. Only one tool may be used at a time and should be placed at the end of the message.]
Executable
BIN
View File
Binary file not shown.
+8 -8
View File
@@ -1,6 +1,6 @@
# AGENTS.md - AI Agent Integration Guide
**Project-Specific Information for AI Agents Working with C-Relay**
**Project-Specific Information for AI Agents Working with C-Relay-PG**
## Critical Build Commands
@@ -15,9 +15,9 @@
- Starts relay in background with proper logging
### Architecture-Specific Binary Outputs
- **x86_64**: `./build/c_relay_x86`
- **ARM64**: `./build/c_relay_arm64`
- **Other**: `./build/c_relay_$(ARCH)`
- **x86_64**: `./build/c_relay_pg_x86`
- **ARM64**: `./build/c_relay_pg_arm64`
- **Other**: `./build/c_relay_pg_$(ARCH)`
### Database File Naming Convention
- **Format**: `<relay_pubkey>.db` (NOT `.nrdb` as shown in docs)
@@ -75,10 +75,10 @@
### Process Management
```bash
# Kill existing relay processes
pkill -f "c_relay_"
pkill -f "c_relay_pg_"
# Check running processes
ps aux | grep c_relay_
ps aux | grep c_relay_pg_
# Force kill port binding
fuser -k 8888/tcp
@@ -95,7 +95,7 @@ fuser -k 8888/tcp
- Event configuration tests: `tests/event_config_tests.sh`
### SystemD Integration Considerations
- Service runs as `c-relay` user in `/opt/c-relay`
- Service runs as `c-relay-pg` user in `/opt/c-relay-pg`
- Database files created in WorkingDirectory automatically
- No environment variables needed (event-based config)
- Resource limits: 65536 file descriptors, 4096 processes
@@ -137,7 +137,7 @@ fuser -k 8888/tcp
## Quick Debugging Commands
```bash
# Check relay status
ps aux | grep c_relay_ && netstat -tln | grep 8888
ps aux | grep c_relay_pg_ && netstat -tln | grep 8888
# View logs
tail -f relay.log
+9 -9
View File
@@ -1,6 +1,6 @@
# C-Relay API Documentation
# C-Relay-PG API Documentation
Complete API reference for the C-Relay event-based administration system and advanced features.
Complete API reference for the C-Relay-PG event-based administration system and advanced features.
## Table of Contents
@@ -21,7 +21,7 @@ Complete API reference for the C-Relay event-based administration system and adv
## Overview
C-Relay uses an innovative **event-based administration system** where all configuration and management commands are sent as cryptographically signed Nostr events. This provides:
C-Relay-PG uses an innovative **event-based administration system** where all configuration and management commands are sent as cryptographically signed Nostr events. This provides:
- **Cryptographic security**: All commands must be signed with the admin private key
- **Audit trail**: Complete history of all administrative actions
@@ -63,8 +63,8 @@ Store the admin private key securely:
export C_RELAY_ADMIN_KEY="nsec1abc123..."
# Secure file
echo "nsec1abc123..." > ~/.c-relay-admin
chmod 600 ~/.c-relay-admin
echo "nsec1abc123..." > ~/.c-relay-pg-admin
chmod 600 ~/.c-relay-pg-admin
# Password manager (recommended)
# Store in 1Password, Bitwarden, etc.
@@ -167,7 +167,7 @@ Examples:
"data": [
{
"key": "relay_name",
"value": "C-Relay",
"value": "C-Relay-PG",
"data_type": "string",
"category": "relay",
"description": "Relay name displayed in NIP-11"
@@ -506,10 +506,10 @@ ORDER BY count DESC
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| `relay_name` | string | "C-Relay" | Relay name (NIP-11) |
| `relay_name` | string | "C-Relay-PG" | Relay name (NIP-11) |
| `relay_description` | string | "C Nostr Relay" | Relay description |
| `relay_contact` | string | "" | Admin contact info |
| `relay_software` | string | "c-relay" | Software identifier |
| `relay_software` | string | "c-relay-pg" | Software identifier |
| `relay_version` | string | auto | Software version |
| `supported_nips` | string | "1,9,11,13,15,20,33,40,42,45,50,70" | Supported NIPs |
| `language_tags` | string | "*" | Supported languages |
@@ -576,7 +576,7 @@ ORDER BY count DESC
## Real-time Monitoring
C-Relay provides subscription-based real-time monitoring using ephemeral events (kind 24567).
C-Relay-PG provides subscription-based real-time monitoring using ephemeral events (kind 24567).
### Activation
+37 -28
View File
@@ -1,12 +1,14 @@
# Alpine-based MUSL static binary builder for C-Relay
# Alpine-based MUSL static binary builder for C-Relay-PG
# Produces truly portable binaries with zero runtime dependencies
ARG DEBUG_BUILD=false
ARG DB_BACKEND=sqlite
FROM alpine:3.19 AS builder
# Re-declare build argument in this stage
# Re-declare build arguments in this stage
ARG DEBUG_BUILD=false
ARG DB_BACKEND=sqlite
# Install build dependencies
RUN apk add --no-cache \
@@ -26,9 +28,11 @@ RUN apk add --no-cache \
curl-static \
sqlite-dev \
sqlite-static \
postgresql-dev \
linux-headers \
wget \
bash
bash \
openssh-client
# Set working directory
WORKDIR /build
@@ -68,15 +72,8 @@ RUN cd /tmp && \
make install && \
rm -rf /tmp/libwebsockets
# Copy only submodule configuration and git directory
COPY .gitmodules /build/.gitmodules
COPY .git /build/.git
# Clean up any stale submodule references (nips directory is not a submodule)
RUN git rm --cached nips 2>/dev/null || true
# Initialize submodules (cached unless .gitmodules changes)
RUN git submodule update --init --recursive
# Submodules are provided in the local build context and copied explicitly below.
# Avoid network/SSH dependency inside Docker image build.
# Copy nostr_core_lib source files (cached unless nostr_core_lib changes)
COPY nostr_core_lib /build/nostr_core_lib/
@@ -84,11 +81,13 @@ COPY nostr_core_lib /build/nostr_core_lib/
# Copy c_utils_lib source files (cached unless c_utils_lib changes)
COPY c_utils_lib /build/c_utils_lib/
# Build c_utils_lib with MUSL-compatible flags (cached unless c_utils_lib changes)
# Build c_utils_lib static library for relay logging utilities
# (build only debug.c to avoid broken/missing version header surface in submodule revision)
RUN cd c_utils_lib && \
sed -i 's/CFLAGS = -Wall -Wextra -std=c99 -O2 -g/CFLAGS = -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -Wall -Wextra -std=c99 -O2 -g/' Makefile && \
make clean && \
make
mkdir -p build && \
gcc -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -Wall -Wextra -std=c99 -O2 -g \
-Isrc -Iinclude -c src/debug.c -o build/debug.o && \
ar rcs libc_utils.a build/debug.o
# Build nostr_core_lib with required NIPs (cached unless nostr_core_lib changes)
# Disable fortification in build.sh to prevent __*_chk symbol issues
@@ -99,44 +98,54 @@ RUN cd nostr_core_lib && \
rm -f *.o *.a 2>/dev/null || true && \
./build.sh --nips=1,6,13,17,19,44,59
# Copy c-relay source files LAST (only this layer rebuilds on source changes)
# Copy c-relay-pg source files LAST (only this layer rebuilds on source changes)
COPY src/ /build/src/
COPY Makefile /build/Makefile
# Build c-relay with full static linking (only rebuilds when src/ changes)
# Build c-relay-pg with full static linking (only rebuilds when src/ changes)
# Disable fortification to avoid __*_chk symbols that don't exist in MUSL
# Use conditional compilation flags based on DEBUG_BUILD argument
# Use conditional compilation flags based on DEBUG_BUILD and DB_BACKEND build args
RUN if [ "$DEBUG_BUILD" = "true" ]; then \
CFLAGS="-g -O2 -DDEBUG"; \
STRIP_CMD="echo 'Keeping debug symbols'"; \
echo "Building with DEBUG symbols enabled (optimized with -O2)"; \
else \
CFLAGS="-O2"; \
STRIP_CMD="strip /build/c_relay_static"; \
STRIP_CMD="strip /build/c_relay_pg_static"; \
echo "Building optimized production binary (symbols stripped)"; \
fi && \
gcc -static $CFLAGS -Wall -Wextra -std=c99 \
if [ "$DB_BACKEND" = "postgres" ]; then \
DB_FLAGS="-DDB_BACKEND_POSTGRES -DHAVE_LIBPQ"; \
DB_LIBS="-lpq -lpgcommon -lpgport"; \
echo "Compiling with PostgreSQL backend"; \
else \
DB_FLAGS=""; \
DB_LIBS=""; \
echo "Compiling with SQLite backend"; \
fi && \
gcc -static $CFLAGS $DB_FLAGS -Wall -Wextra -std=c99 \
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 \
-I. -Ic_utils_lib/src -Inostr_core_lib -Inostr_core_lib/nostr_core \
-Inostr_core_lib/cjson -Inostr_core_lib/nostr_websocket \
-Inostr_core_lib/cjson -Inostr_core_lib/nostr_websocket -I/usr/include/postgresql \
src/main.c src/config.c src/dm_admin.c src/request_validator.c \
src/nip009.c src/nip011.c src/nip013.c src/nip040.c src/nip042.c \
src/websockets.c src/subscriptions.c src/api.c src/embedded_web_content.c src/ip_ban.c \
-o /build/c_relay_static \
src/db_ops.c src/db_ops_sqlite.c src/db_ops_postgres.c src/thread_pool.c \
-o /build/c_relay_pg_static \
c_utils_lib/libc_utils.a \
nostr_core_lib/libnostr_core_x64.a \
-lwebsockets -lssl -lcrypto -lsqlite3 -lsecp256k1 \
-lcurl -lz -lpthread -lm -ldl && \
-lcurl -lz -lpthread -lm -ldl $DB_LIBS && \
eval "$STRIP_CMD"
# Verify it's truly static
RUN echo "=== Binary Information ===" && \
file /build/c_relay_static && \
ls -lh /build/c_relay_static && \
file /build/c_relay_pg_static && \
ls -lh /build/c_relay_pg_static && \
echo "=== Checking for dynamic dependencies ===" && \
(ldd /build/c_relay_static 2>&1 || echo "Binary is static") && \
(ldd /build/c_relay_pg_static 2>&1 || echo "Binary is static") && \
echo "=== Build complete ==="
# Output stage - just the binary
FROM scratch AS output
COPY --from=builder /build/c_relay_static /c_relay_static
COPY --from=builder /build/c_relay_pg_static /c_relay_pg_static
+37 -25
View File
@@ -1,28 +1,40 @@
# C-Relay Makefile
# C-Relay-PG Makefile
CC = gcc
CFLAGS = -Wall -Wextra -std=c99 -g -O2
INCLUDES = -I. -Ic_utils_lib/src -Inostr_core_lib -Inostr_core_lib/nostr_core -Inostr_core_lib/cjson -Inostr_core_lib/nostr_websocket
LIBS = -lsqlite3 -lwebsockets -lz -ldl -lpthread -lm -L/usr/local/lib -lsecp256k1 -lssl -lcrypto -L/usr/local/lib -lcurl -Lc_utils_lib -lc_utils
DB_BACKEND ?= sqlite
# Build directory
BUILD_DIR = build
# Source files
MAIN_SRC = src/main.c src/config.c src/dm_admin.c src/request_validator.c src/nip009.c src/nip011.c src/nip013.c src/nip040.c src/nip042.c src/websockets.c src/subscriptions.c src/api.c src/embedded_web_content.c src/ip_ban.c
MAIN_SRC = src/main.c src/config.c src/dm_admin.c src/request_validator.c src/nip009.c src/nip011.c src/nip013.c src/nip040.c src/nip042.c src/websockets.c src/subscriptions.c src/api.c src/embedded_web_content.c src/ip_ban.c src/thread_pool.c
DB_OPS_SRC = src/db_ops.c
ifeq ($(DB_BACKEND),postgres)
CFLAGS += -DDB_BACKEND_POSTGRES -DHAVE_LIBPQ
DB_OPS_SRC += src/db_ops_postgres.c
LIBS += -lpq
else
DB_OPS_SRC += src/db_ops_sqlite.c
endif
NOSTR_CORE_LIB = nostr_core_lib/libnostr_core_x64.a
C_UTILS_LIB = c_utils_lib/libc_utils.a
# Architecture detection
ARCH = $(shell uname -m)
ifeq ($(ARCH),x86_64)
TARGET = $(BUILD_DIR)/c_relay_x86
TARGET = $(BUILD_DIR)/c_relay_pg_x86
else ifeq ($(ARCH),aarch64)
TARGET = $(BUILD_DIR)/c_relay_arm64
TARGET = $(BUILD_DIR)/c_relay_pg_arm64
else ifeq ($(ARCH),arm64)
TARGET = $(BUILD_DIR)/c_relay_arm64
TARGET = $(BUILD_DIR)/c_relay_pg_arm64
else
TARGET = $(BUILD_DIR)/c_relay_$(ARCH)
TARGET = $(BUILD_DIR)/c_relay_pg_$(ARCH)
endif
# Default target
@@ -81,19 +93,19 @@ force-version:
@$(MAKE) src/main.h
# Build the relay
$(TARGET): $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
@echo "Compiling C-Relay for architecture: $(ARCH)"
$(CC) $(CFLAGS) $(INCLUDES) $(MAIN_SRC) -o $(TARGET) $(NOSTR_CORE_LIB) $(C_UTILS_LIB) $(LIBS)
$(TARGET): $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(DB_OPS_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
@echo "Compiling C-Relay-PG for architecture: $(ARCH) (backend: $(DB_BACKEND))"
$(CC) $(CFLAGS) $(INCLUDES) $(MAIN_SRC) $(DB_OPS_SRC) -o $(TARGET) $(NOSTR_CORE_LIB) $(C_UTILS_LIB) $(LIBS)
@echo "Build complete: $(TARGET)"
# Build for specific architectures
x86: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
@echo "Building C-Relay for x86_64..."
$(CC) $(CFLAGS) $(INCLUDES) $(MAIN_SRC) -o $(BUILD_DIR)/c_relay_x86 $(NOSTR_CORE_LIB) $(C_UTILS_LIB) $(LIBS)
@echo "Build complete: $(BUILD_DIR)/c_relay_x86"
x86: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(DB_OPS_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
@echo "Building C-Relay-PG for x86_64 (backend: $(DB_BACKEND))..."
$(CC) $(CFLAGS) $(INCLUDES) $(MAIN_SRC) $(DB_OPS_SRC) -o $(BUILD_DIR)/c_relay_pg_x86 $(NOSTR_CORE_LIB) $(C_UTILS_LIB) $(LIBS)
@echo "Build complete: $(BUILD_DIR)/c_relay_pg_x86"
arm64: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
@echo "Cross-compiling C-Relay for ARM64..."
arm64: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(DB_OPS_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
@echo "Cross-compiling C-Relay-PG for ARM64 (backend: $(DB_BACKEND))..."
@if ! command -v aarch64-linux-gnu-gcc >/dev/null 2>&1; then \
echo "ERROR: ARM64 cross-compiler not found."; \
echo "Install with: make install-cross-tools"; \
@@ -116,9 +128,9 @@ arm64: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(NOSTR_CORE_LIB) $(
fi
@echo "Using aarch64-linux-gnu-gcc with ARM64 libraries..."
PKG_CONFIG_PATH=/usr/lib/aarch64-linux-gnu/pkgconfig:/usr/share/pkgconfig \
aarch64-linux-gnu-gcc $(CFLAGS) $(INCLUDES) $(MAIN_SRC) -o $(BUILD_DIR)/c_relay_arm64 $(NOSTR_CORE_LIB) $(C_UTILS_LIB) \
aarch64-linux-gnu-gcc $(CFLAGS) $(INCLUDES) $(MAIN_SRC) $(DB_OPS_SRC) -o $(BUILD_DIR)/c_relay_pg_arm64 $(NOSTR_CORE_LIB) $(C_UTILS_LIB) \
-L/usr/lib/aarch64-linux-gnu $(LIBS)
@echo "Build complete: $(BUILD_DIR)/c_relay_arm64"
@echo "Build complete: $(BUILD_DIR)/c_relay_pg_arm64"
# Install ARM64 cross-compilation dependencies
install-arm64-deps:
@@ -160,7 +172,7 @@ test: $(TARGET)
init-db:
@echo "Database initialization is now handled automatically when the server starts."
@echo "The schema is embedded in the binary - no external files needed."
@echo "To manually recreate database: rm -f db/c_nostr_relay.db && ./build/c_relay_x86"
@echo "To manually recreate database: rm -f db/c_nostr_relay.db && ./build/c_relay_pg_x86"
# Clean build artifacts
clean:
@@ -180,7 +192,7 @@ install-deps:
# Help
help:
@echo "C-Relay Build System"
@echo "C-Relay-PG Build System"
@echo ""
@echo "Targets:"
@echo " all Build the relay for current architecture (default)"
@@ -209,15 +221,15 @@ help:
# Build fully static MUSL binaries using Docker
static-musl-x86_64:
@echo "Building fully static MUSL binary for x86_64..."
docker buildx build --platform linux/amd64 -f examples/deployment/static-builder.Dockerfile -t c-relay-static-builder-x86_64 --load .
docker run --rm -v $(PWD)/build:/output c-relay-static-builder-x86_64 sh -c "cp /c_relay_static_musl_x86_64 /output/"
@echo "Static binary created: build/c_relay_static_musl_x86_64"
docker buildx build --platform linux/amd64 -f examples/deployment/static-builder.Dockerfile -t c-relay-pg-static-builder-x86_64 --load .
docker run --rm -v $(PWD)/build:/output c-relay-pg-static-builder-x86_64 sh -c "cp /c_relay_pg_static_musl_x86_64 /output/"
@echo "Static binary created: build/c_relay_pg_static_musl_x86_64"
static-musl-arm64:
@echo "Building fully static MUSL binary for ARM64..."
docker buildx build --platform linux/arm64 -f examples/deployment/static-builder.Dockerfile -t c-relay-static-builder-arm64 --load .
docker run --rm -v $(PWD)/build:/output c-relay-static-builder-arm64 sh -c "cp /c_relay_static_musl_x86_64 /output/c_relay_static_musl_arm64"
@echo "Static binary created: build/c_relay_static_musl_arm64"
docker buildx build --platform linux/arm64 -f examples/deployment/static-builder.Dockerfile -t c-relay-pg-static-builder-arm64 --load .
docker run --rm -v $(PWD)/build:/output c-relay-pg-static-builder-arm64 sh -c "cp /c_relay_pg_static_musl_x86_64 /output/c_relay_pg_static_musl_arm64"
@echo "Static binary created: build/c_relay_pg_static_musl_arm64"
static-musl: static-musl-x86_64 static-musl-arm64
@echo "Built static MUSL binaries for both architectures"
+6 -6
View File
@@ -45,21 +45,21 @@ Also included is a more standard administrative web front end. This front end co
## Screenshots
![](https://git.laantungir.net/laantungir/c-relay/raw/branch/master/screenshots/main.png)
![](https://git.laantungir.net/laantungir/c-relay-pg/raw/branch/master/screenshots/main.png)
Main page with real time updates.
![](https://git.laantungir.net/laantungir/c-relay/raw/branch/master/screenshots/config.png)
![](https://git.laantungir.net/laantungir/c-relay-pg/raw/branch/master/screenshots/config.png)
Set your configuration preferences.
![](https://git.laantungir.net/laantungir/c-relay/raw/branch/master/screenshots/subscriptions.png)
![](https://git.laantungir.net/laantungir/c-relay-pg/raw/branch/master/screenshots/subscriptions.png)
View current subscriptions
![](https://git.laantungir.net/laantungir/c-relay/raw/branch/master/screenshots/white-blacklists.png)
![](https://git.laantungir.net/laantungir/c-relay-pg/raw/branch/master/screenshots/white-blacklists.png)
Add npubs to white or black lists.
![](https://git.laantungir.net/laantungir/c-relay/raw/branch/master/screenshots/sqlQuery.png)
![](https://git.laantungir.net/laantungir/c-relay-pg/raw/branch/master/screenshots/sqlQuery.png)
Run sql queries on the database.
![](https://git.laantungir.net/laantungir/c-relay/raw/branch/master/screenshots/main-light.png)
![](https://git.laantungir.net/laantungir/c-relay-pg/raw/branch/master/screenshots/main-light.png)
Light mode.
+24 -24
View File
@@ -1,11 +1,11 @@
# C-Relay: High-Performance Nostr Relay
# C-Relay-PG: High-Performance Nostr Relay
A blazingly fast, production-ready Nostr relay implemented in C with an innovative event-based configuration system. Built for performance, security, and ease of deployment.
## 🚀 Why C-Relay?
## 🚀 Why C-Relay-PG?
### Event-Based Configuration
Unlike traditional relays that require config files, C-Relay uses **cryptographically signed Nostr events** for all configuration. This means:
Unlike traditional relays that require config files, C-Relay-PG uses **cryptographically signed Nostr events** for all configuration. This means:
- **Zero config files** - Everything stored in the database
- **Real-time updates** - Changes applied instantly without restart
- **Cryptographic security** - All changes must be signed by admin
@@ -36,7 +36,7 @@ Control your relay by sending direct messages from any Nostr client:
## 📋 Supported NIPs
C-Relay implements a comprehensive set of Nostr Improvement Proposals:
C-Relay-PG implements a comprehensive set of Nostr Improvement Proposals:
-**NIP-01**: Basic protocol flow implementation
-**NIP-09**: Event deletion
@@ -89,12 +89,12 @@ Download and run - no dependencies required:
```bash
# Download the latest static release
wget https://git.laantungir.net/laantungir/c-relay/releases/download/v0.6.0/c-relay-v0.6.0-linux-x86_64-static
chmod +x c-relay-v0.6.0-linux-x86_64-static
mv c-relay-v0.6.0-linux-x86_64-static c-relay
wget https://git.laantungir.net/laantungir/c-relay-pg/releases/download/v0.6.0/c-relay-pg-v0.6.0-linux-x86_64-static
chmod +x c-relay-pg-v0.6.0-linux-x86_64-static
mv c-relay-pg-v0.6.0-linux-x86_64-static c-relay-pg
# Run the relay
./c-relay
./c-relay-pg
```
**Important**: On first startup, save the **Admin Private Key** displayed in the console. You'll need it for all administrative operations.
@@ -107,8 +107,8 @@ sudo apt install -y build-essential git sqlite3 libsqlite3-dev \
libwebsockets-dev libssl-dev libsecp256k1-dev libcurl4-openssl-dev zlib1g-dev
# Clone and build
git clone https://github.com/your-org/c-relay.git
cd c-relay
git clone https://github.com/your-org/c-relay-pg.git
cd c-relay-pg
git submodule update --init --recursive
./make_and_restart_relay.sh
```
@@ -136,8 +136,8 @@ The web interface provides:
```bash
# Clone repository
git clone https://github.com/your-org/c-relay.git
cd c-relay
git clone https://github.com/your-org/c-relay-pg.git
cd c-relay-pg
git submodule update --init --recursive
# Build
@@ -147,25 +147,25 @@ make clean && make
sudo systemd/install-service.sh
# Start and enable
sudo systemctl start c-relay
sudo systemctl enable c-relay
sudo systemctl start c-relay-pg
sudo systemctl enable c-relay-pg
# Capture admin keys from logs
sudo journalctl -u c-relay | grep "Admin Private Key"
sudo journalctl -u c-relay-pg | grep "Admin Private Key"
```
### Docker Deployment
```bash
# Build Docker image
docker build -f Dockerfile.alpine-musl -t c-relay .
docker build -f Dockerfile.alpine-musl -t c-relay-pg .
# Run container
docker run -d \
--name c-relay \
--name c-relay-pg \
-p 8888:8888 \
-v /path/to/data:/data \
c-relay
c-relay-pg
```
### Cloud Deployment
@@ -181,7 +181,7 @@ See [`docs/deployment_guide.md`](docs/deployment_guide.md) for detailed deployme
## 🔧 Configuration
C-Relay uses an innovative event-based configuration system. All settings are managed through signed Nostr events.
C-Relay-PG uses an innovative event-based configuration system. All settings are managed through signed Nostr events.
### Basic Configuration
@@ -239,8 +239,8 @@ The admin private key is displayed **only once** during first startup. Store it
```bash
# Save to secure location
echo "ADMIN_PRIVKEY=your_admin_private_key" > ~/.c-relay-admin
chmod 600 ~/.c-relay-admin
echo "ADMIN_PRIVKEY=your_admin_private_key" > ~/.c-relay-pg-admin
chmod 600 ~/.c-relay-pg-admin
```
### Production Security
@@ -269,9 +269,9 @@ Contributions are welcome! Please:
## 🔗 Links
- **Repository**: [https://github.com/your-org/c-relay](https://github.com/your-org/c-relay)
- **Releases**: [https://git.laantungir.net/laantungir/c-relay/releases](https://git.laantungir.net/laantungir/c-relay/releases)
- **Issues**: [https://github.com/your-org/c-relay/issues](https://github.com/your-org/c-relay/issues)
- **Repository**: [https://github.com/your-org/c-relay-pg](https://github.com/your-org/c-relay-pg)
- **Releases**: [https://git.laantungir.net/laantungir/c-relay-pg/releases](https://git.laantungir.net/laantungir/c-relay-pg/releases)
- **Issues**: [https://github.com/your-org/c-relay-pg/issues](https://github.com/your-org/c-relay-pg/issues)
- **Nostr Protocol**: [https://github.com/nostr-protocol/nostr](https://github.com/nostr-protocol/nostr)
## 💬 Support
+7 -7
View File
@@ -41,7 +41,7 @@ This guide is specifically tailored for C programs that use:
```bash
# 1. Copy the Dockerfile template (see below)
cp /path/to/c-relay/Dockerfile.alpine-musl ./Dockerfile.static
cp /path/to/c-relay-pg/Dockerfile.alpine-musl ./Dockerfile.static
# 2. Customize for your project (see Customization section)
vim Dockerfile.static
@@ -466,13 +466,13 @@ docker build -t my-app:latest .
docker run --rm my-app:latest --help
```
## Reusing c-relay Files
## Reusing c-relay-pg Files
You can directly copy these files from c-relay:
You can directly copy these files from c-relay-pg:
### 1. Dockerfile.alpine-musl
```bash
cp /path/to/c-relay/Dockerfile.alpine-musl ./Dockerfile.static
cp /path/to/c-relay-pg/Dockerfile.alpine-musl ./Dockerfile.static
```
Then customize:
@@ -482,7 +482,7 @@ Then customize:
### 2. build_static.sh
```bash
cp /path/to/c-relay/build_static.sh ./
cp /path/to/c-relay-pg/build_static.sh ./
```
Then customize:
@@ -492,7 +492,7 @@ Then customize:
### 3. .dockerignore (Optional)
```bash
cp /path/to/c-relay/.dockerignore ./
cp /path/to/c-relay-pg/.dockerignore ./
```
Helps speed up Docker builds by excluding unnecessary files.
@@ -522,7 +522,7 @@ Helps speed up Docker builds by excluding unnecessary files.
- [Alpine Linux](https://alpinelinux.org/)
- [nostr_core_lib](https://github.com/chebizarro/nostr_core_lib)
- [Static Linking Best Practices](https://www.musl-libc.org/faq.html)
- [c-relay Implementation](./docs/musl_static_build.md)
- [c-relay-pg Implementation](./docs/musl_static_build.md)
## Example: Minimal Nostr Client
+2 -2
View File
@@ -4,7 +4,7 @@
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>C-Relay Admin</title>
<title>C-Relay-PG Admin</title>
<link rel="stylesheet" href="/api/index.css">
</head>
@@ -42,7 +42,7 @@
<span class="relay-letter" data-letter="Y">Y</span>
</div>
<div class="relay-info">
<div id="relay-name" class="relay-name">C-Relay</div>
<div id="relay-name" class="relay-name">C-Relay-PG</div>
<div id="relay-description" class="relay-description">Loading...</div>
<div id="relay-pubkey-container" class="relay-pubkey-container">
<div id="relay-pubkey" class="relay-pubkey">Loading...</div>
+33 -18
View File
@@ -185,7 +185,7 @@ async function fetchRelayInfo(relayUrl) {
method: 'GET',
headers: {
'Accept': 'application/nostr+json',
'User-Agent': 'C-Relay-Admin-API/1.0'
'User-Agent': 'C-Relay-PG-Admin-API/1.0'
},
timeout: 10000 // 10 second timeout
});
@@ -194,9 +194,9 @@ async function fetchRelayInfo(relayUrl) {
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
}
const contentType = response.headers.get('content-type');
if (!contentType || !contentType.includes('application/nostr+json')) {
throw new Error(`Invalid content type: ${contentType}. Expected application/nostr+json`);
const contentType = response.headers.get('content-type') || '';
if (!contentType.includes('application/nostr+json') && !contentType.includes('application/json')) {
log(`Unexpected NIP-11 content type: ${contentType}; attempting JSON parse anyway`, 'WARN');
}
const relayInfo = await response.json();
@@ -448,17 +448,26 @@ async function setupAutomaticRelayConnection(showSections = false) {
const httpUrl = relayUrl.replace('ws', 'http').replace('wss', 'https');
const relayInfo = await fetchRelayInfo(httpUrl);
if (relayInfo && relayInfo.pubkey) {
relayPubkey = relayInfo.pubkey;
const fetchedPubkey = relayInfo && relayInfo.pubkey ? relayInfo.pubkey : null;
// Fallback must match the actual running relay pubkey when NIP-11 doesn't expose pubkey.
relayPubkey = fetchedPubkey || 'e9aa50decff01f2cec1ec2b2e0b34332cf9c92cafdac5a7cc0881a6d26b59854';
// Keep relay metadata (including version) even when pubkey is missing/fallback
relayInfoData = {
name: relayInfo?.name || 'C-Relay-PG',
version: relayInfo?.version || '',
description: relayInfo?.description || 'Nostr Relay',
pubkey: relayPubkey
};
if (fetchedPubkey) {
console.log('🔑 Auto-fetched relay pubkey:', relayPubkey.substring(0, 16) + '...');
} else {
// Use fallback pubkey
relayPubkey = '4f355bdcb7cc0af728ef3cceb9615d90684bb5b2ca5f859ab0f0b704075871aa';
console.log('⚠️ Using fallback relay pubkey');
}
} catch (error) {
console.log('⚠️ Could not fetch relay info, using fallback pubkey:', error.message);
relayPubkey = '4f355bdcb7cc0af728ef3cceb9615d90684bb5b2ca5f859ab0f0b704075871aa';
relayPubkey = 'e9aa50decff01f2cec1ec2b2e0b34332cf9c92cafdac5a7cc0881a6d26b59854';
}
// Set up subscription to receive admin API responses
@@ -3954,9 +3963,9 @@ async function testPostEvent() {
created_at: Math.floor(Date.now() / 1000),
tags: [
["t", "test"],
["client", "c-relay-admin-api"]
["client", "c-relay-pg-admin-api"]
],
content: `Test event from C-Relay Admin API at ${new Date().toISOString()}`
content: `Test event from C-Relay-PG Admin API at ${new Date().toISOString()}`
};
logTestEvent('SENT', `Test event (before signing): ${JSON.stringify(testEvent)}`, 'EVENT');
@@ -4238,9 +4247,10 @@ function updateRelayInfoInHeader() {
return;
}
// Get relay info from NIP-11 data or use defaults
// Get relay info from NIP-11/config data or use defaults
const relayInfo = getRelayInfo();
const relayName = relayInfo.name || 'C-Relay';
const relayName = relayInfo.name || 'C-Relay-PG';
const relayVersion = relayInfo.version || '';
const relayDescription = relayInfo.description || 'Nostr Relay';
// Convert relay pubkey to npub
@@ -4263,7 +4273,9 @@ function updateRelayInfoInHeader() {
formattedNpub = line1 + '\n' + line2 + '\n' + line3;
}
relayNameElement.textContent = relayName;
const displayName = relayVersion ? `${relayName} ${relayVersion}` : relayName;
relayNameElement.textContent = displayName;
document.title = `${displayName} Admin`;
relayPubkeyElement.textContent = formattedNpub;
relayDescriptionElement.textContent = relayDescription;
}
@@ -4280,7 +4292,8 @@ function getRelayInfo() {
// Default values
return {
name: 'C-Relay',
name: 'C-Relay-PG',
version: '',
description: 'Nostr Relay',
pubkey: relayPubkey
};
@@ -4290,11 +4303,13 @@ function getRelayInfo() {
function updateStoredRelayInfo(configData) {
if (configData && configData.data) {
// Extract relay info from config data
const relayName = configData.data.find(item => item.key === 'relay_name')?.value || 'C-Relay';
const relayName = configData.data.find(item => item.key === 'relay_name')?.value || 'C-Relay-PG';
const relayVersion = configData.data.find(item => item.key === 'relay_version')?.value || '';
const relayDescription = configData.data.find(item => item.key === 'relay_description')?.value || 'Nostr Relay';
relayInfoData = {
name: relayName,
version: relayVersion,
description: relayDescription,
pubkey: relayPubkey
};
@@ -5406,7 +5421,7 @@ function switchPage(pageName) {
// Initialize the app
document.addEventListener('DOMContentLoaded', () => {
console.log('C-Relay Admin API interface loaded');
console.log('C-Relay-PG Admin API interface loaded');
// Initialize dark mode
initializeDarkMode();
@@ -5508,7 +5523,7 @@ const SQL_QUERY_TEMPLATES = {
};
// Query history management (localStorage)
const QUERY_HISTORY_KEY = 'c_relay_sql_history';
const QUERY_HISTORY_KEY = 'c_relay_pg_sql_history';
const MAX_HISTORY_ITEMS = 20;
// Load query history from localStorage
+49 -13
View File
@@ -1,6 +1,6 @@
#!/bin/bash
# Build fully static MUSL binaries for C-Relay using Alpine Docker
# Build fully static MUSL binaries for C-Relay-PG using Alpine Docker
# Produces truly portable binaries with zero runtime dependencies
set -e
@@ -11,19 +11,52 @@ DOCKERFILE="$SCRIPT_DIR/Dockerfile.alpine-musl"
# Parse command line arguments
DEBUG_BUILD=false
if [[ "$1" == "--debug" ]]; then
DEBUG_BUILD=true
DB_BACKEND="${DB_BACKEND:-postgres}"
while [[ $# -gt 0 ]]; do
case "$1" in
--debug)
DEBUG_BUILD=true
shift
;;
--db-backend)
if [[ -z "$2" ]]; then
echo "ERROR: --db-backend requires a value (sqlite|postgres)"
exit 1
fi
DB_BACKEND="$2"
shift 2
;;
--db-backend=*)
DB_BACKEND="${1#*=}"
shift
;;
*)
echo "ERROR: Unknown argument: $1"
echo "Usage: $0 [--debug] [--db-backend postgres|sqlite]"
exit 1
;;
esac
done
if [[ "$DB_BACKEND" != "sqlite" && "$DB_BACKEND" != "postgres" ]]; then
echo "ERROR: Invalid DB backend '$DB_BACKEND'. Use sqlite or postgres."
exit 1
fi
if [[ "$DEBUG_BUILD" == "true" ]]; then
echo "=========================================="
echo "C-Relay MUSL Static Binary Builder (DEBUG MODE)"
echo "C-Relay-PG MUSL Static Binary Builder (DEBUG MODE)"
echo "=========================================="
else
echo "=========================================="
echo "C-Relay MUSL Static Binary Builder (PRODUCTION MODE)"
echo "C-Relay-PG MUSL Static Binary Builder (PRODUCTION MODE)"
echo "=========================================="
fi
echo "Project directory: $SCRIPT_DIR"
echo "Build directory: $BUILD_DIR"
echo "Debug build: $DEBUG_BUILD"
echo "DB backend: $DB_BACKEND"
echo ""
# Create build directory
@@ -63,17 +96,17 @@ ARCH=$(uname -m)
case "$ARCH" in
x86_64)
PLATFORM="linux/amd64"
OUTPUT_NAME="c_relay_static_x86_64"
OUTPUT_NAME="c_relay_pg_static_x86_64"
;;
aarch64|arm64)
PLATFORM="linux/arm64"
OUTPUT_NAME="c_relay_static_arm64"
OUTPUT_NAME="c_relay_pg_static_arm64"
;;
*)
echo "WARNING: Unknown architecture: $ARCH"
echo "Defaulting to linux/amd64"
PLATFORM="linux/amd64"
OUTPUT_NAME="c_relay_static_${ARCH}"
OUTPUT_NAME="c_relay_pg_static_${ARCH}"
;;
esac
@@ -116,14 +149,15 @@ echo "=========================================="
echo "This will:"
echo " - Use Alpine Linux (native MUSL)"
echo " - Build all dependencies statically"
echo " - Compile c-relay with full static linking"
echo " - Compile c-relay-pg with full static linking"
echo ""
$DOCKER_CMD build \
--platform "$PLATFORM" \
--build-arg DEBUG_BUILD=$DEBUG_BUILD \
--build-arg DB_BACKEND=$DB_BACKEND \
-f "$DOCKERFILE" \
-t c-relay-musl-builder:latest \
-t c-relay-pg-musl-builder:latest \
--progress=plain \
. || {
echo ""
@@ -145,16 +179,17 @@ echo "=========================================="
$DOCKER_CMD build \
--platform "$PLATFORM" \
--build-arg DEBUG_BUILD=$DEBUG_BUILD \
--build-arg DB_BACKEND=$DB_BACKEND \
--target builder \
-f "$DOCKERFILE" \
-t c-relay-static-builder-stage:latest \
-t c-relay-pg-static-builder-stage:latest \
. > /dev/null 2>&1
# Create a temporary container to copy the binary
CONTAINER_ID=$($DOCKER_CMD create c-relay-static-builder-stage:latest)
CONTAINER_ID=$($DOCKER_CMD create c-relay-pg-static-builder-stage:latest)
# Copy binary from container
$DOCKER_CMD cp "$CONTAINER_ID:/build/c_relay_static" "$BUILD_DIR/$OUTPUT_NAME" || {
$DOCKER_CMD cp "$CONTAINER_ID:/build/c_relay_pg_static" "$BUILD_DIR/$OUTPUT_NAME" || {
echo "ERROR: Failed to extract binary from container"
$DOCKER_CMD rm "$CONTAINER_ID" 2>/dev/null
exit 1
@@ -224,6 +259,7 @@ if [ "$DEBUG_BUILD" = true ]; then
else
echo "Build Type: PRODUCTION (optimized, stripped)"
fi
echo "DB Backend: $DB_BACKEND"
if [ "$TRULY_STATIC" = true ]; then
echo "Linkage: Fully static binary (Alpine MUSL-based)"
echo "Portability: Works on ANY Linux distribution"
+4 -4
View File
@@ -1,12 +1,12 @@
#!/bin/bash
# Restart the service
sudo systemctl stop c-relay.service
sudo systemctl stop c-relay-pg.service
# Copy the binary to the deployment location
cp build/c_relay_static_x86_64 ~/Storage/c_relay/crelay
cp build/c_relay_pg_static_x86_64 ~/Storage/c_relay_pg/crelay
# Restart the service
sudo systemctl restart c-relay.service
sudo systemctl restart c-relay-pg.service
# Show service status
sudo systemctl status c-relay.service --no-pager -l
sudo systemctl status c-relay-pg.service --no-pager -l
+8 -8
View File
@@ -1,24 +1,24 @@
#!/bin/bash
# C-Relay Static Binary Deployment Script
# Deploys build/c_relay_static_x86_64 to server via ssh
# C-Relay-PG Static Binary Deployment Script
# Deploys build/c_relay_pg_static_x86_64 to server via ssh
set -e
# Configuration
LOCAL_BINARY="build/c_relay_static_x86_64"
REMOTE_BINARY_PATH="/usr/local/bin/c_relay/c_relay"
SERVICE_NAME="c-relay"
LOCAL_BINARY="build/c_relay_pg_static_x86_64"
REMOTE_BINARY_PATH="/usr/local/bin/c_relay_pg/c_relay_pg"
SERVICE_NAME="c-relay-pg"
# Create backup
ssh ubuntu@laantungir.com "sudo cp '$REMOTE_BINARY_PATH' '${REMOTE_BINARY_PATH}.backup.$(date +%Y%m%d_%H%M%S)'" 2>/dev/null || true
# Upload binary to temp location
scp "$LOCAL_BINARY" "ubuntu@laantungir.com:/tmp/c_relay.tmp"
scp "$LOCAL_BINARY" "ubuntu@laantungir.com:/tmp/c_relay_pg.tmp"
# Install binary
ssh ubuntu@laantungir.com "sudo mv '/tmp/c_relay.tmp' '$REMOTE_BINARY_PATH'"
ssh ubuntu@laantungir.com "sudo chown c-relay:c-relay '$REMOTE_BINARY_PATH'"
ssh ubuntu@laantungir.com "sudo mv '/tmp/c_relay_pg.tmp' '$REMOTE_BINARY_PATH'"
ssh ubuntu@laantungir.com "sudo chown c-relay-pg:c-relay-pg '$REMOTE_BINARY_PATH'"
ssh ubuntu@laantungir.com "sudo chmod +x '$REMOTE_BINARY_PATH'"
# Reload systemd and restart service
+15 -15
View File
@@ -1,14 +1,14 @@
#!/bin/bash
# C-Relay Debug Binary Deployment Script
# Deploys build/c_relay_static_x86_64_debug to server for CPU profiling
# C-Relay-PG Debug Binary Deployment Script
# Deploys build/c_relay_pg_static_x86_64_debug to server for CPU profiling
#
# Usage:
# ./deploy_lt_debug.sh -- deploy debug binary and restart
# ./deploy_lt_debug.sh --profile -- deploy, then run perf and fetch results
#
# After deploying, profile with:
# sudo perf record -g -p $(pgrep c_relay) -- sleep 30
# sudo perf record -g -p $(pgrep c_relay_pg) -- sleep 30
# sudo perf report --stdio --sort=symbol --no-children -n 2>/dev/null | head -80
#
# Restore production binary:
@@ -16,10 +16,10 @@
set -e
LOCAL_DEBUG_BINARY="build/c_relay_static_x86_64_debug"
REMOTE_BINARY_PATH="/usr/local/bin/c_relay/c_relay"
REMOTE_PROD_BACKUP="/usr/local/bin/c_relay/c_relay.production"
SERVICE_NAME="c-relay"
LOCAL_DEBUG_BINARY="build/c_relay_pg_static_x86_64_debug"
REMOTE_BINARY_PATH="/usr/local/bin/c_relay_pg/c_relay_pg"
REMOTE_PROD_BACKUP="/usr/local/bin/c_relay_pg/c_relay_pg.production"
SERVICE_NAME="c-relay-pg"
REMOTE_HOST="ubuntu@laantungir.com"
# Check debug binary exists
@@ -33,7 +33,7 @@ if [ ! -f "$LOCAL_DEBUG_BINARY" ]; then
fi
echo "=========================================="
echo "C-Relay Debug Deployment"
echo "C-Relay-PG Debug Deployment"
echo "=========================================="
echo "Binary: $LOCAL_DEBUG_BINARY ($(du -h "$LOCAL_DEBUG_BINARY" | cut -f1))"
echo "Target: $REMOTE_HOST:$REMOTE_BINARY_PATH"
@@ -55,18 +55,18 @@ ssh "$REMOTE_HOST" "
# Upload debug binary
echo "Uploading debug binary..."
scp "$LOCAL_DEBUG_BINARY" "$REMOTE_HOST:/tmp/c_relay_debug.tmp"
scp "$LOCAL_DEBUG_BINARY" "$REMOTE_HOST:/tmp/c_relay_pg_debug.tmp"
# Install debug binary
echo "Installing debug binary..."
ssh "$REMOTE_HOST" "
sudo mv '/tmp/c_relay_debug.tmp' '$REMOTE_BINARY_PATH'
sudo chown c-relay:c-relay '$REMOTE_BINARY_PATH'
sudo mv '/tmp/c_relay_pg_debug.tmp' '$REMOTE_BINARY_PATH'
sudo chown c-relay-pg:c-relay-pg '$REMOTE_BINARY_PATH'
sudo chmod +x '$REMOTE_BINARY_PATH'
"
# Restart service
echo "Restarting c-relay service..."
echo "Restarting c-relay-pg service..."
ssh "$REMOTE_HOST" "sudo systemctl daemon-reload && sudo systemctl restart '$SERVICE_NAME'"
echo ""
@@ -84,9 +84,9 @@ if [ "$1" = "--profile" ]; then
sleep 3
ssh "$REMOTE_HOST" "
PID=\$(pgrep c_relay)
PID=\$(pgrep c_relay_pg)
if [ -z \"\$PID\" ]; then
echo 'ERROR: c_relay not running'
echo 'ERROR: c_relay_pg not running'
exit 1
fi
echo \"Profiling PID \$PID for 30 seconds...\"
@@ -101,7 +101,7 @@ else
echo "=========================================="
echo ""
echo "SSH to server and run:"
echo " sudo perf record -g -p \$(pgrep c_relay) -- sleep 30"
echo " sudo perf record -g -p \$(pgrep c_relay_pg) -- sleep 30"
echo " sudo perf report --stdio --sort=symbol --no-children -n 2>/dev/null | head -60"
echo ""
echo "Or run with --profile to do it automatically:"
+2 -2
View File
@@ -1,4 +1,4 @@
# C-Relay Administrator API Implementation Plan
# C-Relay-PG Administrator API Implementation Plan
## Problem Analysis
@@ -161,7 +161,7 @@ Would require changing schema, migration scripts, and storage logic.
#### README.md Documentation Format:
```markdown
# C-Relay Administrator API
# C-Relay-PG Administrator API
## Authentication
All admin commands require signing with the admin private key generated during first startup.
+190
View File
@@ -0,0 +1,190 @@
# Agent Browser Testing Guide for C-Relay-PG
This document explains how to use the `agent-browser` CLI tool to test the c-relay-pg admin web UI from an AI agent context (no physical display required).
## Prerequisites
- **agent-browser** installed globally: `npm install -g agent-browser`
- Binary location: `/home/user/.nvm/versions/node/v24.14.1/bin/agent-browser`
- The relay must be running locally (default port 8888)
- Test keys configured in `.test_keys`
## Starting the Relay for Local Testing
```bash
./make_and_restart_relay.sh -t
```
This reads `.test_keys` and starts the relay with:
- `ADMIN_PUBKEY` — the hex public key of the admin account
- `ADMIN_PRIVKEY` — the hex secret key (used for browser login, not by the relay itself)
- `SERVER_PRIVKEY` — the relay's own private key
## Key URLs
| URL | Purpose |
|-----|---------|
| `http://127.0.0.1:8888/api/index.html` | Admin web UI (the correct entry point) |
| `http://127.0.0.1:8888/` | Returns 406 without NIP-11 Accept header — **do not use for browser testing** |
| `http://127.0.0.1:8888/` with `Accept: application/nostr+json` | NIP-11 relay info JSON |
**Important:** The root URL `/` is a WebSocket/NIP-11 endpoint, not an HTML page. Always use `/api/index.html` for browser testing.
## Admin Login Credentials
The admin nsec for the current `.test_keys` configuration:
```
nsec: nsec1zkn0hlt4jvcvn9yt5p7ea4m7f82pf3ph0gj3d4rlcz4s64x5z86satv9qm
hex: 15a6fbfd759330c9948ba07d9ed77e49d414c4377a2516d47fc0ab0d54d411f5
npub: npub1tlyzk6slzt8d989f4pn3synk98fea64ea3jmrdc7dtd0kw8uxlas9a9fwr
hex pubkey: 5fc82b6a1f12ced29ca9a86718127629d39eeab9ec65b1b71e6adafb38fc37fb
```
## Complete Login Flow with agent-browser
### Step 1: Open the admin UI
```bash
agent-browser open http://127.0.0.1:8888/api/index.html
agent-browser wait --load networkidle
```
### Step 2: Take an interactive snapshot to see what is on screen
```bash
agent-browser snapshot -i
```
You will see a login modal with buttons like:
- `"Browser Extension"` — skip this
- `"Local Key"`**use this one**
- `"Seed Phrase"` — skip
- `"Nostr Connect"` — skip
- `"Read Only"` — skip
### Step 3: Click "Local Key"
```bash
agent-browser click @e15
```
(The ref number may vary — use the ref from the snapshot output for the "Local Key" button.)
After clicking, a text input appears asking for the secret key.
### Step 4: Enter the admin nsec
```bash
agent-browser fill @e14 "nsec1zkn0hlt4jvcvn9yt5p7ea4m7f82pf3ph0gj3d4rlcz4s64x5z86satv9qm"
```
(Use the ref from the snapshot for the textbox element.)
### Step 5: Click "Import Key"
```bash
agent-browser snapshot -i
```
Check the snapshot — the "Import Key" button should now be enabled. Click it:
```bash
agent-browser click @e15
```
### Step 6: Click "Continue" on the success screen
After import, a success screen appears with "Continue" button:
```bash
agent-browser wait 800
agent-browser snapshot -i
agent-browser click @e19
```
(Use the ref from the snapshot for the "Continue" button.)
### Step 7: Wait for admin UI to load
```bash
agent-browser wait 5000
agent-browser snapshot -i
```
You should now see the admin dashboard with:
- Statistics table (Database Size, Total Events, PID, etc.)
- Navigation buttons (Statistics, Subscriptions, Configuration, Authorization, etc.)
- Admin profile area showing "admin" label
## Navigating Admin Sections
After login, use the sidebar navigation buttons:
```bash
# View configuration
agent-browser click @e8 # Configuration button ref
# View authorization rules
agent-browser click @e9 # Authorization button ref
# View statistics
agent-browser click @e6 # Statistics button ref
# Always snapshot after navigation to see results
agent-browser wait 2500
agent-browser snapshot -i
```
## Checking for Errors
```bash
# View browser console logs
agent-browser console
# View JavaScript errors
agent-browser errors
# View relay server logs
tail -n 100 relay.log
```
## Chained Command Example (Full Login in One Shot)
```bash
agent-browser open http://127.0.0.1:8888/api/index.html && \
agent-browser wait --load networkidle && \
agent-browser snapshot -i
```
Then use refs from the snapshot to complete login steps.
## Tips for AI Agents
1. **Always use `/api/index.html`** — never the root URL
2. **Use `snapshot -i`** after every action to see the current interactive elements and their refs
3. **Refs change** between snapshots — always re-snapshot before clicking
4. **Wait after clicks** — use `agent-browser wait 2000` (milliseconds) between actions that trigger async operations
5. **The login flow has 3 screens**: method selection → key input → success confirmation
6. **Console logs are cumulative** — they show all logs since page load, which is useful for debugging admin API responses
7. **The relay log** at `relay.log` shows server-side processing of admin commands
8. **Command chaining** with `&&` works — the browser daemon persists between commands
## Verifying Admin API is Working
After login, the statistics page should show populated data:
- Database Size (e.g., "4 KB")
- Process ID (the relay PID)
- WebSocket Connections count
- Memory Usage
If these show "-" or "Loading...", check:
1. `relay.log` for errors
2. `agent-browser console` for JavaScript errors
3. `agent-browser errors` for page-level errors
## Closing the Browser
```bash
agent-browser close --all
```
+11 -11
View File
@@ -324,7 +324,7 @@ int main() {
}
```
## Migration Plan for c-relay
## Migration Plan for c-relay-pg
### Phase 1: Extract Debug System
1. Create `c_utils_lib` repository
@@ -333,16 +333,16 @@ int main() {
4. Add basic tests
### Phase 2: Add Versioning System
1. Extract version generation logic from c-relay
1. Extract version generation logic from c-relay-pg
2. Create reusable version utilities
3. Update c-relay to use new system
3. Update c-relay-pg to use new system
4. Update nostr_core_lib to use new system
### Phase 3: Add as Submodule
1. Add `c_utils_lib` as submodule to c-relay
2. Update c-relay Makefile
3. Update includes in c-relay source files
4. Remove old debug files from c-relay
1. Add `c_utils_lib` as submodule to c-relay-pg
2. Update c-relay-pg Makefile
3. Update includes in c-relay-pg source files
4. Remove old debug files from c-relay-pg
### Phase 4: Documentation & Examples
1. Create comprehensive README
@@ -352,7 +352,7 @@ int main() {
## Benefits
### For c-relay
### For c-relay-pg
- Cleaner separation of concerns
- Reusable utilities across projects
- Easier to maintain and test
@@ -379,7 +379,7 @@ int main() {
- Memory leak detection (valgrind)
### Integration Tests
- Test with real projects (c-relay, nostr_core_lib)
- Test with real projects (c-relay-pg, nostr_core_lib)
- Cross-platform testing
- Performance benchmarks
@@ -433,7 +433,7 @@ MIT License - permissive and suitable for learning and commercial use.
## Success Criteria
1. ✅ Successfully integrated into c-relay
1. ✅ Successfully integrated into c-relay-pg
2. ✅ Successfully integrated into nostr_core_lib
3. ✅ All tests passing
4. ✅ Documentation complete
@@ -449,7 +449,7 @@ MIT License - permissive and suitable for learning and commercial use.
4. Create build system
5. Write tests
6. Create documentation
7. Integrate into c-relay
7. Integrate into c-relay-pg
8. Publish to GitHub
---
+16 -16
View File
@@ -2,13 +2,13 @@
## Overview
This document provides a step-by-step implementation plan for creating the `c_utils_lib` library and integrating it into the c-relay project.
This document provides a step-by-step implementation plan for creating the `c_utils_lib` library and integrating it into the c-relay-pg project.
## Phase 1: Repository Setup & Structure
### Step 1.1: Create Repository Structure
**Location**: Create outside c-relay project (sibling directory)
**Location**: Create outside c-relay-pg project (sibling directory)
```bash
# Create directory structure
@@ -42,7 +42,7 @@ git branch -M main
### Step 2.1: Move Debug Files
**Source files** (from c-relay):
**Source files** (from c-relay-pg):
- `src/debug.c``c_utils_lib/src/debug.c`
- `src/debug.h``c_utils_lib/include/c_utils/debug.h`
@@ -484,20 +484,20 @@ How to integrate into projects:
3. Code examples
4. Migration from standalone utilities
## Phase 7: Integration with c-relay
## Phase 7: Integration with c-relay-pg
### Step 7.1: Add as Submodule
```bash
cd /path/to/c-relay
cd /path/to/c-relay-pg
git submodule add <repo-url> c_utils_lib
git submodule update --init --recursive
```
### Step 7.2: Update c-relay Makefile
### Step 7.2: Update c-relay-pg Makefile
```makefile
# Add to c-relay Makefile
# Add to c-relay-pg Makefile
C_UTILS_LIB = c_utils_lib/libc_utils.a
# Update includes
@@ -514,7 +514,7 @@ $(C_UTILS_LIB):
$(TARGET): $(C_UTILS_LIB) ...
```
### Step 7.3: Update c-relay Source Files
### Step 7.3: Update c-relay-pg Source Files
**Changes needed**:
@@ -543,7 +543,7 @@ $(TARGET): $(C_UTILS_LIB) ...
### Step 7.4: Test Integration
```bash
cd c-relay
cd c-relay-pg
make clean
make
./make_and_restart_relay.sh
@@ -556,7 +556,7 @@ Verify:
## Phase 8: Version System Integration
### Step 8.1: Update c-relay Makefile for Versioning
### Step 8.1: Update c-relay-pg Makefile for Versioning
```makefile
# Add version generation
@@ -567,7 +567,7 @@ src/version.h: .git/refs/tags/*
$(TARGET): src/version.h ...
```
### Step 8.2: Update c-relay to Use Generated Version
### Step 8.2: Update c-relay-pg to Use Generated Version
Replace hardcoded version in `src/main.h` with:
```c
@@ -583,7 +583,7 @@ Replace hardcoded version in `src/main.h` with:
- **Phase 4**: Build System - 2 hours
- **Phase 5**: Examples & Tests - 3 hours
- **Phase 6**: Documentation - 3 hours
- **Phase 7**: c-relay Integration - 2 hours
- **Phase 7**: c-relay-pg Integration - 2 hours
- **Phase 8**: Version Integration - 2 hours
**Total**: ~19 hours
@@ -593,11 +593,11 @@ Replace hardcoded version in `src/main.h` with:
- [ ] c_utils_lib builds successfully
- [ ] All tests pass
- [ ] Examples compile and run
- [ ] c-relay integrates successfully
- [ ] Debug output works in c-relay
- [ ] c-relay-pg integrates successfully
- [ ] Debug output works in c-relay-pg
- [ ] Version generation works
- [ ] Documentation complete
- [ ] No regressions in c-relay functionality
- [ ] No regressions in c-relay-pg functionality
## Next Steps
@@ -608,7 +608,7 @@ Replace hardcoded version in `src/main.h` with:
5. Create build system
6. Write tests and examples
7. Create documentation
8. Integrate into c-relay
8. Integrate into c-relay-pg
9. Test thoroughly
10. Publish to GitHub
+6 -6
View File
@@ -78,9 +78,9 @@ Configuration events follow the standard Nostr event format with kind 33334:
#### `relay_software`
- **Description**: Software identifier for NIP-11
- **Default**: `"c-relay"`
- **Default**: `"c-relay-pg"`
- **Format**: String, max 64 characters
- **Example**: `"c-relay v1.0.0"`
- **Example**: `"c-relay-pg v1.0.0"`
#### `relay_version`
- **Description**: Software version string
@@ -366,7 +366,7 @@ sqlite3 relay.nrdb "SELECT json_pretty(json_object(
#### Invalid Parameter Values
```bash
# Check relay logs for validation errors
journalctl -u c-relay | grep "Configuration.*invalid\|Invalid.*configuration"
journalctl -u c-relay-pg | grep "Configuration.*invalid\|Invalid.*configuration"
# Common issues:
# - Numeric values outside valid ranges
@@ -395,10 +395,10 @@ ORDER BY date DESC;"
#### Resource Usage After Changes
```bash
# Monitor system resources after configuration updates
top -p $(pgrep c_relay)
top -p $(pgrep c_relay_pg)
# Check for memory leaks
ps aux | grep c_relay | awk '{print $6}' # RSS memory
ps aux | grep c_relay_pg | awk '{print $6}' # RSS memory
```
### Emergency Recovery
@@ -425,7 +425,7 @@ nostrtool event \
# If database is corrupted, backup and recreate
cp relay.nrdb relay.nrdb.backup
rm relay.nrdb*
./build/c_relay_x86 # Creates fresh database with new keys
./build/c_relay_pg_x86 # Creates fresh database with new keys
```
---
+14 -14
View File
@@ -21,22 +21,22 @@ typedef enum {
```bash
# Production (default - no debug output)
./c_relay_x86
./c_relay_pg_x86
# Show errors only
./c_relay_x86 --debug-level=1
./c_relay_pg_x86 --debug-level=1
# Show errors and warnings
./c_relay_x86 --debug-level=2
./c_relay_pg_x86 --debug-level=2
# Show errors, warnings, and info (recommended for development)
./c_relay_x86 --debug-level=3
./c_relay_pg_x86 --debug-level=3
# Show all debug messages
./c_relay_x86 --debug-level=4
./c_relay_pg_x86 --debug-level=4
# Show everything including trace with file:line (very verbose)
./c_relay_x86 --debug-level=5
./c_relay_pg_x86 --debug-level=5
```
## Implementation
@@ -475,7 +475,7 @@ When `g_debug_level = 0` (constant), you'll see the compiler has removed all deb
### Level 3 (Errors + Warnings + Info)
```
[2025-01-12 14:30:15] [INFO ] Initializing C-Relay v0.4.6
[2025-01-12 14:30:15] [INFO ] Initializing C-Relay-PG v0.4.6
[2025-01-12 14:30:15] [INFO ] Loading configuration from database
[2025-01-12 14:30:15] [ERROR] Failed to open database: permission denied
[2025-01-12 14:30:16] [WARN ] Port 8888 unavailable, trying 8889
@@ -484,7 +484,7 @@ When `g_debug_level = 0` (constant), you'll see the compiler has removed all deb
### Level 4 (All Debug Messages)
```
[2025-01-12 14:30:15] [INFO ] Initializing C-Relay v0.4.6
[2025-01-12 14:30:15] [INFO ] Initializing C-Relay-PG v0.4.6
[2025-01-12 14:30:15] [DEBUG] Opening database: build/abc123...def.db
[2025-01-12 14:30:15] [DEBUG] Executing schema initialization
[2025-01-12 14:30:15] [INFO ] SQLite WAL mode enabled
@@ -496,7 +496,7 @@ When `g_debug_level = 0` (constant), you'll see the compiler has removed all deb
### Level 5 (Everything Including file:line for ALL messages)
```
[2025-01-12 14:30:15] [INFO ] [main.c:1607] Initializing C-Relay v0.4.6
[2025-01-12 14:30:15] [INFO ] [main.c:1607] Initializing C-Relay-PG v0.4.6
[2025-01-12 14:30:15] [DEBUG] [main.c:348] Opening database: build/abc123...def.db
[2025-01-12 14:30:15] [TRACE] [main.c:330] Entering init_database()
[2025-01-12 14:30:15] [ERROR] [config.c:125] Database locked
@@ -525,11 +525,11 @@ Update the existing `log_*` functions to use the new debug macros
make clean && make
# Test different levels
./build/c_relay_x86 # No output
./build/c_relay_x86 --debug-level=1 # Errors only
./build/c_relay_x86 --debug-level=3 # Info + warnings + errors
./build/c_relay_x86 --debug-level=4 # All debug messages
./build/c_relay_x86 --debug-level=5 # Everything with file:line on TRACE
./build/c_relay_pg_x86 # No output
./build/c_relay_pg_x86 --debug-level=1 # Errors only
./build/c_relay_pg_x86 --debug-level=3 # Info + warnings + errors
./build/c_relay_pg_x86 --debug-level=4 # All debug messages
./build/c_relay_pg_x86 --debug-level=5 # Everything with file:line on TRACE
```
### Step 5: Gradual Migration (Ongoing)
+1 -1
View File
@@ -34,7 +34,7 @@ static const struct {
// NIP-11 Relay Information (relay keys will be populated at runtime)
{"relay_description", "High-performance C Nostr relay with SQLite storage"},
{"relay_contact", ""},
{"relay_software", "https://git.laantungir.net/laantungir/c-relay.git"},
{"relay_software", "https://git.laantungir.net/laantungir/c-relay-pg.git"},
{"relay_version", "v1.0.0"},
// NIP-13 Proof of Work (pow_min_difficulty = 0 means PoW disabled)
+51 -51
View File
@@ -52,11 +52,11 @@ sudo apt install -y build-essential git sqlite3 libsqlite3-dev \
#### User and Directory Setup
```bash
# Create dedicated system user
sudo useradd --system --home-dir /opt/c-relay --shell /bin/false c-relay
sudo useradd --system --home-dir /opt/c-relay-pg --shell /bin/false c-relay-pg
# Create application directory
sudo mkdir -p /opt/c-relay
sudo chown c-relay:c-relay /opt/c-relay
sudo mkdir -p /opt/c-relay-pg
sudo chown c-relay-pg:c-relay-pg /opt/c-relay-pg
```
### Build and Installation
@@ -64,8 +64,8 @@ sudo chown c-relay:c-relay /opt/c-relay
#### Automated Installation (Recommended)
```bash
# Clone repository
git clone https://github.com/your-org/c-relay.git
cd c-relay
git clone https://github.com/your-org/c-relay-pg.git
cd c-relay-pg
git submodule update --init --recursive
# Build
@@ -81,12 +81,12 @@ sudo systemd/install-service.sh
make clean && make
# Install binary
sudo cp build/c_relay_x86 /opt/c-relay/
sudo chown c-relay:c-relay /opt/c-relay/c_relay_x86
sudo chmod +x /opt/c-relay/c_relay_x86
sudo cp build/c_relay_pg_x86 /opt/c-relay-pg/
sudo chown c-relay-pg:c-relay-pg /opt/c-relay-pg/c_relay_pg_x86
sudo chmod +x /opt/c-relay-pg/c_relay_pg_x86
# Install systemd service
sudo cp systemd/c-relay.service /etc/systemd/system/
sudo cp systemd/c-relay-pg.service /etc/systemd/system/
sudo systemctl daemon-reload
```
@@ -95,22 +95,22 @@ sudo systemctl daemon-reload
#### Start and Enable Service
```bash
# Start the service
sudo systemctl start c-relay
sudo systemctl start c-relay-pg
# Enable auto-start on boot
sudo systemctl enable c-relay
sudo systemctl enable c-relay-pg
# Check status
sudo systemctl status c-relay
sudo systemctl status c-relay-pg
```
#### Capture Admin Keys (CRITICAL)
```bash
# View startup logs to get admin keys
sudo journalctl -u c-relay --since "5 minutes ago" | grep -A 10 "IMPORTANT: SAVE THIS ADMIN PRIVATE KEY"
sudo journalctl -u c-relay-pg --since "5 minutes ago" | grep -A 10 "IMPORTANT: SAVE THIS ADMIN PRIVATE KEY"
# Or check the full log
sudo journalctl -u c-relay --no-pager | grep "Admin Private Key"
sudo journalctl -u c-relay-pg --no-pager | grep "Admin Private Key"
```
⚠️ **CRITICAL**: Save the admin private key immediately - it's only shown once and is needed for all configuration updates!
@@ -151,8 +151,8 @@ sudo iptables-save > /etc/iptables/rules.v4
ssh -i your-key.pem ubuntu@your-instance-ip
# Use the simple deployment script
git clone https://github.com/your-org/c-relay.git
cd c-relay
git clone https://github.com/your-org/c-relay-pg.git
cd c-relay-pg
sudo examples/deployment/simple-vps/deploy.sh
```
@@ -168,10 +168,10 @@ sudo examples/deployment/simple-vps/deploy.sh
sudo mkfs.ext4 /dev/xvdf
sudo mkdir /data
sudo mount /dev/xvdf /data
sudo chown c-relay:c-relay /data
sudo chown c-relay-pg:c-relay-pg /data
# Update systemd service to use /data
sudo sed -i 's/WorkingDirectory=\/opt\/c-relay/WorkingDirectory=\/data/' /etc/systemd/system/c-relay.service
sudo sed -i 's/WorkingDirectory=\/opt\/c-relay-pg/WorkingDirectory=\/data/' /etc/systemd/system/c-relay-pg.service
sudo systemctl daemon-reload
```
@@ -180,7 +180,7 @@ sudo systemctl daemon-reload
#### Compute Engine Setup
```bash
# Create VM instance (e2-micro or larger)
gcloud compute instances create c-relay-instance \
gcloud compute instances create c-relay-pg-instance \
--image-family=ubuntu-2204-lts \
--image-project=ubuntu-os-cloud \
--machine-type=e2-micro \
@@ -193,9 +193,9 @@ gcloud compute firewall-rules create allow-nostr-relay \
--target-tags nostr-relay
# SSH and deploy
gcloud compute ssh c-relay-instance
git clone https://github.com/your-org/c-relay.git
cd c-relay
gcloud compute ssh c-relay-pg-instance
git clone https://github.com/your-org/c-relay-pg.git
cd c-relay-pg
sudo examples/deployment/simple-vps/deploy.sh
```
@@ -203,13 +203,13 @@ sudo examples/deployment/simple-vps/deploy.sh
```bash
# Create and attach persistent disk
gcloud compute disks create relay-data --size=50GB
gcloud compute instances attach-disk c-relay-instance --disk=relay-data
gcloud compute instances attach-disk c-relay-pg-instance --disk=relay-data
# Format and mount
sudo mkfs.ext4 /dev/sdb
sudo mkdir /data
sudo mount /dev/sdb /data
sudo chown c-relay:c-relay /data
sudo chown c-relay-pg:c-relay-pg /data
```
### DigitalOcean
@@ -223,8 +223,8 @@ sudo chown c-relay:c-relay /data
ssh root@your-droplet-ip
# Deploy relay
git clone https://github.com/your-org/c-relay.git
cd c-relay
git clone https://github.com/your-org/c-relay-pg.git
cd c-relay-pg
examples/deployment/simple-vps/deploy.sh
```
@@ -245,8 +245,8 @@ The `examples/deployment/` directory contains ready-to-use scripts:
### Simple VPS Deployment
```bash
# Clone repository and run automated deployment
git clone https://github.com/your-org/c-relay.git
cd c-relay
git clone https://github.com/your-org/c-relay-pg.git
cd c-relay-pg
sudo examples/deployment/simple-vps/deploy.sh
```
@@ -417,9 +417,9 @@ LOG_FILE="/var/log/relay-monitor.log"
DATE=$(date '+%Y-%m-%d %H:%M:%S')
# Check if relay is running
if ! pgrep -f "c_relay_x86" > /dev/null; then
if ! pgrep -f "c_relay_pg_x86" > /dev/null; then
echo "[$DATE] ERROR: Relay process not running" >> $LOG_FILE
systemctl restart c-relay
systemctl restart c-relay-pg
fi
# Check port availability
@@ -428,14 +428,14 @@ if ! netstat -tln | grep -q ":8888"; then
fi
# Check database file
RELAY_DB=$(find /opt/c-relay -name "*.nrdb" | head -1)
RELAY_DB=$(find /opt/c-relay-pg -name "*.nrdb" | head -1)
if [[ -n "$RELAY_DB" ]]; then
DB_SIZE=$(du -h "$RELAY_DB" | cut -f1)
echo "[$DATE] INFO: Database size: $DB_SIZE" >> $LOG_FILE
fi
# Check memory usage
MEM_USAGE=$(ps aux | grep c_relay_x86 | grep -v grep | awk '{print $6}')
MEM_USAGE=$(ps aux | grep c_relay_pg_x86 | grep -v grep | awk '{print $6}')
if [[ -n "$MEM_USAGE" ]]; then
echo "[$DATE] INFO: Memory usage: ${MEM_USAGE}KB" >> $LOG_FILE
fi
@@ -454,8 +454,8 @@ sudo chmod +x /usr/local/bin/relay-monitor.sh
#### Centralized Logging with rsyslog
```bash
# /etc/rsyslog.d/50-c-relay.conf
if $programname == 'c-relay' then /var/log/c-relay.log
# /etc/rsyslog.d/50-c-relay-pg.conf
if $programname == 'c-relay-pg' then /var/log/c-relay-pg.log
& stop
```
@@ -465,7 +465,7 @@ if $programname == 'c-relay' then /var/log/c-relay.log
```yaml
# /etc/prometheus/prometheus.yml
scrape_configs:
- job_name: 'c-relay'
- job_name: 'c-relay-pg'
static_configs:
- targets: ['localhost:8888']
metrics_path: '/metrics' # If implemented
@@ -479,30 +479,30 @@ scrape_configs:
#### Service User Restrictions
```bash
# Restrict service user
sudo usermod -s /bin/false c-relay
sudo usermod -d /opt/c-relay c-relay
sudo usermod -s /bin/false c-relay-pg
sudo usermod -d /opt/c-relay-pg c-relay-pg
# Set proper permissions
sudo chmod 700 /opt/c-relay
sudo chown -R c-relay:c-relay /opt/c-relay
sudo chmod 700 /opt/c-relay-pg
sudo chown -R c-relay-pg:c-relay-pg /opt/c-relay-pg
```
#### File System Restrictions
```bash
# Mount data directory with appropriate options
echo "/dev/sdb /opt/c-relay ext4 defaults,noexec,nosuid,nodev 0 2" >> /etc/fstab
echo "/dev/sdb /opt/c-relay-pg ext4 defaults,noexec,nosuid,nodev 0 2" >> /etc/fstab
```
### Network Security
#### Fail2Ban Configuration
```ini
# /etc/fail2ban/jail.d/c-relay.conf
[c-relay-dos]
# /etc/fail2ban/jail.d/c-relay-pg.conf
[c-relay-pg-dos]
enabled = true
port = 8888
filter = c-relay-dos
logpath = /var/log/c-relay.log
filter = c-relay-pg-dos
logpath = /var/log/c-relay-pg.log
maxretry = 10
findtime = 60
bantime = 300
@@ -534,9 +534,9 @@ sudo mkfs.ext4 /dev/mapper/relay-data
#!/bin/bash
# /usr/local/bin/backup-relay.sh
BACKUP_DIR="/backup/c-relay"
BACKUP_DIR="/backup/c-relay-pg"
DATE=$(date +%Y%m%d_%H%M%S)
RELAY_DB=$(find /opt/c-relay -name "*.nrdb" | head -1)
RELAY_DB=$(find /opt/c-relay-pg -name "*.nrdb" | head -1)
mkdir -p "$BACKUP_DIR"
@@ -574,7 +574,7 @@ sudo apt install -y awscli
aws configure
# Sync backups to S3
aws s3 sync /backup/c-relay/ s3://your-backup-bucket/c-relay/ --delete
aws s3 sync /backup/c-relay-pg/ s3://your-backup-bucket/c-relay-pg/ --delete
```
### Disaster Recovery
@@ -583,16 +583,16 @@ aws s3 sync /backup/c-relay/ s3://your-backup-bucket/c-relay/ --delete
```bash
# 1. Restore from backup
gunzip backup/relay_backup_20231201_020000.nrdb.gz
cp backup/relay_backup_20231201_020000.nrdb /opt/c-relay/
cp backup/relay_backup_20231201_020000.nrdb /opt/c-relay-pg/
# 2. Fix permissions
sudo chown c-relay:c-relay /opt/c-relay/*.nrdb
sudo chown c-relay-pg:c-relay-pg /opt/c-relay-pg/*.nrdb
# 3. Restart service
sudo systemctl restart c-relay
sudo systemctl restart c-relay-pg
# 4. Verify recovery
sudo journalctl -u c-relay --since "1 minute ago"
sudo journalctl -u c-relay-pg --since "1 minute ago"
```
---
+25 -25
View File
@@ -2,7 +2,7 @@
## Overview
This guide explains how to build truly portable MUSL-based static binaries of c-relay using Alpine Linux Docker containers. These binaries have **zero runtime dependencies** and work on any Linux distribution.
This guide explains how to build truly portable MUSL-based static binaries of c-relay-pg using Alpine Linux Docker containers. These binaries have **zero runtime dependencies** and work on any Linux distribution.
## Why MUSL?
@@ -36,8 +36,8 @@ This guide explains how to build truly portable MUSL-based static binaries of c-
./build_static.sh
# The binary will be created at:
# build/c_relay_static_musl_x86_64 (on x86_64)
# build/c_relay_static_musl_arm64 (on ARM64)
# build/c_relay_pg_static_musl_x86_64 (on x86_64)
# build/c_relay_pg_static_musl_arm64 (on ARM64)
```
### What Happens During Build
@@ -56,7 +56,7 @@ This guide explains how to build truly portable MUSL-based static binaries of c-
- Includes required NIPs: 001, 006, 013, 017, 019, 044, 059
- Produces static library (~316KB)
4. **c-relay Compilation**: Links everything statically:
4. **c-relay-pg Compilation**: Links everything statically:
- All source files compiled with `-static` flag
- Fortification disabled to avoid `__*_chk` symbols
- Results in ~7.6MB stripped binary
@@ -78,7 +78,7 @@ FROM alpine:3.19 AS builder
- Install build tools and static libraries
- Build dependencies from source
- Compile nostr_core_lib with MUSL flags
- Compile c-relay with full static linking
- Compile c-relay-pg with full static linking
- Strip binary to reduce size
# Stage 2: Output (scratch)
@@ -93,7 +93,7 @@ FROM scratch AS output
CFLAGS="-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -Wall -Wextra -std=c99 -fPIC -O2"
```
**For c-relay:**
**For c-relay-pg:**
```bash
gcc -static -O2 -Wall -Wextra -std=c99 \
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 \
@@ -110,7 +110,7 @@ gcc -static -O2 -Wall -Wextra -std=c99 \
- `-U_FORTIFY_SOURCE` (undefine any existing definition)
- `-D_FORTIFY_SOURCE=0` (set to 0)
This must be applied to **both** nostr_core_lib and c-relay compilation.
This must be applied to **both** nostr_core_lib and c-relay-pg compilation.
### NIP Dependencies
@@ -129,26 +129,26 @@ The build includes these NIPs in nostr_core_lib:
```bash
# Should show "statically linked"
file build/c_relay_static_musl_x86_64
file build/c_relay_pg_static_musl_x86_64
# Should show "not a dynamic executable"
ldd build/c_relay_static_musl_x86_64
ldd build/c_relay_pg_static_musl_x86_64
# Check size (should be ~7.6MB)
ls -lh build/c_relay_static_musl_x86_64
ls -lh build/c_relay_pg_static_musl_x86_64
```
### Test Execution
```bash
# Show help
./build/c_relay_static_musl_x86_64 --help
./build/c_relay_pg_static_musl_x86_64 --help
# Show version
./build/c_relay_static_musl_x86_64 --version
./build/c_relay_pg_static_musl_x86_64 --version
# Run relay
./build/c_relay_static_musl_x86_64 --port 8888
./build/c_relay_pg_static_musl_x86_64 --port 8888
```
### Cross-Distribution Testing
@@ -157,16 +157,16 @@ Test the binary on different distributions to verify portability:
```bash
# Alpine Linux
docker run --rm -v $(pwd)/build:/app alpine:latest /app/c_relay_static_musl_x86_64 --version
docker run --rm -v $(pwd)/build:/app alpine:latest /app/c_relay_pg_static_musl_x86_64 --version
# Ubuntu
docker run --rm -v $(pwd)/build:/app ubuntu:latest /app/c_relay_static_musl_x86_64 --version
docker run --rm -v $(pwd)/build:/app ubuntu:latest /app/c_relay_pg_static_musl_x86_64 --version
# Debian
docker run --rm -v $(pwd)/build:/app debian:latest /app/c_relay_static_musl_x86_64 --version
docker run --rm -v $(pwd)/build:/app debian:latest /app/c_relay_pg_static_musl_x86_64 --version
# CentOS
docker run --rm -v $(pwd)/build:/app centos:latest /app/c_relay_static_musl_x86_64 --version
docker run --rm -v $(pwd)/build:/app centos:latest /app/c_relay_pg_static_musl_x86_64 --version
```
## Troubleshooting
@@ -187,7 +187,7 @@ newgrp docker # Or logout and login again
**Solution**: Ensure fortification is disabled in both:
1. nostr_core_lib build.sh (line 534)
2. c-relay compilation flags in Dockerfile
2. c-relay-pg compilation flags in Dockerfile
### Binary Won't Execute
@@ -213,26 +213,26 @@ newgrp docker # Or logout and login again
```bash
# Copy binary to server
scp build/c_relay_static_musl_x86_64 user@server:/opt/c-relay/
scp build/c_relay_pg_static_musl_x86_64 user@server:/opt/c-relay-pg/
# Run on server (no dependencies needed!)
ssh user@server
cd /opt/c-relay
./c_relay_static_musl_x86_64 --port 8888
cd /opt/c-relay-pg
./c_relay_pg_static_musl_x86_64 --port 8888
```
### SystemD Service
```ini
[Unit]
Description=C-Relay Nostr Relay (MUSL Static)
Description=C-Relay-PG Nostr Relay (MUSL Static)
After=network.target
[Service]
Type=simple
User=c-relay
WorkingDirectory=/opt/c-relay
ExecStart=/opt/c-relay/c_relay_static_musl_x86_64
User=c-relay-pg
WorkingDirectory=/opt/c-relay-pg
ExecStart=/opt/c-relay-pg/c_relay_pg_static_musl_x86_64
Restart=always
RestartSec=5
+4 -4
View File
@@ -23,7 +23,7 @@ We chose Option B because:
## Detailed Implementation Steps
### Phase 1: Configuration Setup in c-relay
### Phase 1: Configuration Setup in c-relay-pg
#### 1.1 Add Configuration Parameter
**File:** `src/default_config_event.h`
@@ -293,7 +293,7 @@ int nostr_nip17_send_dm(cJSON* dm_event,
---
### Phase 4: Update c-relay Call Sites
### Phase 4: Update c-relay-pg Call Sites
#### 4.1 Update src/api.c
**Location:** Line 1319
@@ -470,7 +470,7 @@ causing compatibility issues.
- [ ] Update `nostr_nip17_send_dm()` signature and implementation
- [ ] Update `nip017.h` function declaration and documentation
### c-relay Changes
### c-relay-pg Changes
- [ ] Add `nip59_timestamp_max_delay_sec` to `default_config_event.h`
- [ ] Add validation in `config.c` for new parameter
- [ ] Update `src/api.c` call site to pass `max_delay_sec`
@@ -494,7 +494,7 @@ causing compatibility issues.
If issues arise:
1. Revert nostr_core_lib changes (git revert in submodule)
2. Revert c-relay changes
2. Revert c-relay-pg changes
3. Configuration parameter will be ignored if not used
4. Default behavior (0) provides maximum compatibility
+2 -2
View File
@@ -2,7 +2,7 @@
## Overview
This document describes the design for a general-purpose SQL query interface for the C-Relay admin API. This allows administrators to execute read-only SQL queries against the relay database through cryptographically signed kind 23456 events with NIP-44 encrypted command arrays.
This document describes the design for a general-purpose SQL query interface for the C-Relay-PG admin API. This allows administrators to execute read-only SQL queries against the relay database through cryptographically signed kind 23456 events with NIP-44 encrypted command arrays.
## Security Model
@@ -271,7 +271,7 @@ const SQL_QUERY_TEMPLATES = {
};
// Query history management (localStorage)
const QUERY_HISTORY_KEY = 'c_relay_sql_history';
const QUERY_HISTORY_KEY = 'c_relay_pg_sql_history';
const MAX_HISTORY_ITEMS = 20;
// Load query history from localStorage
+1 -1
View File
@@ -224,7 +224,7 @@ The script should:
- `curl` or `websocat` for WebSocket communication
- `jq` for JSON parsing
- Nostr CLI tools (optional, for event signing)
- Running c-relay instance
- Running c-relay-pg instance
## Example Output
+6 -6
View File
@@ -1,8 +1,8 @@
# C-Relay Complete Startup Flow Documentation
# C-Relay-PG Complete Startup Flow Documentation
## Overview
C-Relay has two distinct startup paths:
C-Relay-PG has two distinct startup paths:
1. **First-Time Startup**: No database exists, generates keys and initializes system
2. **Existing Relay Startup**: Database exists, loads configuration and resumes operation
@@ -220,7 +220,7 @@ int populate_default_config_values(void) {
struct config_default defaults[] = {
{"relay_port", "8888", "integer", "WebSocket port", "network", 1},
{"relay_name", "C-Relay", "string", "Relay name", "info", 0},
{"relay_name", "C-Relay-PG", "string", "Relay name", "info", 0},
{"relay_description", "High-performance C Nostr relay", "string", "Description", "info", 0},
{"max_subscriptions_per_client", "25", "integer", "Max subs per client", "limits", 0},
{"pow_min_difficulty", "0", "integer", "Minimum PoW difficulty", "security", 0},
@@ -983,7 +983,7 @@ if (current_version < LATEST_VERSION) {
**Solutions**:
- Use `--port <number>` to specify different port
- Kill existing process: `pkill -f c_relay_`
- Kill existing process: `pkill -f c_relay_pg_`
- Force kill port: `fuser -k 8888/tcp`
- Use `--strict-port` to fail fast instead of trying fallback ports
@@ -994,7 +994,7 @@ if (current_version < LATEST_VERSION) {
**Solutions**:
- Kill existing relay processes
- Remove WAL files: `rm build/*.db-wal build/*.db-shm`
- Check for stale processes: `ps aux | grep c_relay_`
- Check for stale processes: `ps aux | grep c_relay_pg_`
#### 3. Missing Admin Private Key
@@ -1071,7 +1071,7 @@ Typical startup times (on modern hardware):
## Summary
The c-relay startup system is designed for:
The c-relay-pg startup system is designed for:
1. **Security**: Admin keys never stored, relay keys encrypted
2. **Reliability**: Automatic port fallback, schema migrations
+9 -9
View File
@@ -18,10 +18,10 @@ The script now attempts to build with `musl-gcc` for truly portable static binar
SQLite is now built once and cached for future builds:
- **Cache location**: `~/.cache/c-relay-sqlite/`
- **Cache location**: `~/.cache/c-relay-pg-sqlite/`
- **Version-specific**: Each SQLite version gets its own cache directory
- **Significant speedup**: Subsequent builds skip the SQLite compilation step
- **Manual cleanup**: `rm -rf ~/.cache/c-relay-sqlite` to clear cache
- **Manual cleanup**: `rm -rf ~/.cache/c-relay-pg-sqlite` to clear cache
### 3. Smart Package Installation
@@ -51,13 +51,13 @@ The script will:
## Binary Types
### MUSL Static Binary (Ideal - Currently Not Achievable)
- **Filename**: `build/c_relay_static_musl_x86_64`
- **Filename**: `build/c_relay_pg_static_musl_x86_64`
- **Dependencies**: None (truly static)
- **Portability**: Works on any Linux distribution
- **Status**: Requires MUSL-compiled libwebsockets and other dependencies (not available by default)
### Glibc Static Binary (Current Output)
- **Filename**: `build/c_relay_static_x86_64` or `build/c_relay_static_glibc_x86_64`
- **Filename**: `build/c_relay_pg_static_x86_64` or `build/c_relay_pg_static_glibc_x86_64`
- **Dependencies**: None - fully statically linked with glibc
- **Portability**: Works on most Linux distributions (glibc is statically included)
- **Note**: Despite using glibc, this is a **fully static binary** with no runtime dependencies
@@ -68,11 +68,11 @@ The script automatically verifies binaries using `ldd` and `file`:
```bash
# For MUSL binary
ldd build/c_relay_static_musl_x86_64
ldd build/c_relay_pg_static_musl_x86_64
# Output: "not a dynamic executable" (good!)
# For glibc binary
ldd build/c_relay_static_glibc_x86_64
ldd build/c_relay_pg_static_glibc_x86_64
# Output: Shows glibc dependencies
```
@@ -112,7 +112,7 @@ The script attempts MUSL compilation but falls back to glibc:
### Clear SQLite Cache
```bash
rm -rf ~/.cache/c-relay-sqlite
rm -rf ~/.cache/c-relay-pg-sqlite
```
### Force Package Reinstall
@@ -127,8 +127,8 @@ cat /tmp/musl_build.log
### Verify Binary Type
```bash
file build/c_relay_static_*
ldd build/c_relay_static_* 2>&1
file build/c_relay_pg_static_*
ldd build/c_relay_pg_static_* 2>&1
```
## Performance Impact
+1 -1
View File
@@ -2,7 +2,7 @@
## Problem Summary
The c-relay Nostr relay experienced severe performance degradation (90-100% CPU) due to subscription accumulation in the database. Investigation revealed **323,644 orphaned subscriptions** that were never properly closed when WebSocket connections dropped.
The c-relay-pg Nostr relay experienced severe performance degradation (90-100% CPU) due to subscription accumulation in the database. Investigation revealed **323,644 orphaned subscriptions** that were never properly closed when WebSocket connections dropped.
## Solution: Two-Component Approach
+8 -8
View File
@@ -586,7 +586,7 @@ int add_pubkeys_to_config_table(void) {
rm -f *.db
# Start relay with defaults
./build/c_relay_x86
./build/c_relay_pg_x86
# Verify config table complete
sqlite3 <relay_pubkey>.db "SELECT COUNT(*) FROM config;"
@@ -602,7 +602,7 @@ int add_pubkeys_to_config_table(void) {
rm -f *.db
# Start relay with port override
./build/c_relay_x86 --port 9999
./build/c_relay_pg_x86 --port 9999
# Verify port override applied
sqlite3 <relay_pubkey>.db "SELECT value FROM config WHERE key='relay_port';"
@@ -612,13 +612,13 @@ int add_pubkeys_to_config_table(void) {
3. **Restart with Existing Database**
```bash
# Start relay (creates database)
./build/c_relay_x86
./build/c_relay_pg_x86
# Stop relay
pkill -f c_relay_
pkill -f c_relay_pg_
# Restart relay
./build/c_relay_x86
./build/c_relay_pg_x86
# Verify config unchanged
# Check relay.log for validation message
@@ -627,13 +627,13 @@ int add_pubkeys_to_config_table(void) {
4. **Restart with CLI Overrides**
```bash
# Start relay (creates database)
./build/c_relay_x86
./build/c_relay_pg_x86
# Stop relay
pkill -f c_relay_
pkill -f c_relay_pg_
# Restart with port override
./build/c_relay_x86 --port 9999
./build/c_relay_pg_x86 --port 9999
# Verify port override applied
sqlite3 <relay_pubkey>.db "SELECT value FROM config WHERE key='relay_port';"
+32 -32
View File
@@ -19,12 +19,12 @@ Complete guide for deploying, configuring, and managing the C Nostr Relay with e
```bash
# Clone and build
git clone <repository-url>
cd c-relay
cd c-relay-pg
git submodule update --init --recursive
make
# Start relay (zero configuration needed)
./build/c_relay_x86
./build/c_relay_pg_x86
```
### 2. First Startup - Save Keys
@@ -78,7 +78,7 @@ brew install git sqlite libwebsockets openssl libsecp256k1 curl zlib
```bash
# Clone repository
git clone <repository-url>
cd c-relay
cd c-relay-pg
# Initialize submodules
git submodule update --init --recursive
@@ -87,7 +87,7 @@ git submodule update --init --recursive
make clean && make
# Verify build
ls -la build/c_relay_x86
ls -la build/c_relay_pg_x86
```
### Production Deployment
@@ -98,27 +98,27 @@ ls -la build/c_relay_x86
sudo systemd/install-service.sh
# Start service
sudo systemctl start c-relay
sudo systemctl start c-relay-pg
# Enable auto-start
sudo systemctl enable c-relay
sudo systemctl enable c-relay-pg
# Check status
sudo systemctl status c-relay
sudo systemctl status c-relay-pg
```
#### Manual Deployment
```bash
# Create dedicated user
sudo useradd --system --home-dir /opt/c-relay --shell /bin/false c-relay
sudo useradd --system --home-dir /opt/c-relay-pg --shell /bin/false c-relay-pg
# Install binary
sudo mkdir -p /opt/c-relay
sudo cp build/c_relay_x86 /opt/c-relay/
sudo chown -R c-relay:c-relay /opt/c-relay
sudo mkdir -p /opt/c-relay-pg
sudo cp build/c_relay_pg_x86 /opt/c-relay-pg/
sudo chown -R c-relay-pg:c-relay-pg /opt/c-relay-pg
# Run as service user
sudo -u c-relay /opt/c-relay/c_relay_x86
sudo -u c-relay-pg /opt/c-relay-pg/c_relay_pg_x86
```
## Configuration Management
@@ -188,7 +188,7 @@ Send this to your relay via WebSocket, and changes are applied immediately.
|-----------|-------------|---------|---------|
| `relay_description` | Relay description for NIP-11 | "C Nostr Relay" | "My awesome relay" |
| `relay_contact` | Admin contact information | "" | "admin@example.com" |
| `relay_software` | Software identifier | "c-relay" | "c-relay v1.0" |
| `relay_software` | Software identifier | "c-relay-pg" | "c-relay-pg v1.0" |
#### Client Limits
| Parameter | Description | Default | Range |
@@ -273,7 +273,7 @@ chmod 600 admin_keys_backup_*.txt
#### Key Recovery
If you lose your admin private key:
1. **Stop the relay**: `pkill c_relay` or `sudo systemctl stop c-relay`
1. **Stop the relay**: `pkill c_relay_pg` or `sudo systemctl stop c-relay-pg`
2. **Backup events**: `cp <relay_pubkey>.nrdb backup_$(date +%Y%m%d).nrdb`
3. **Remove database**: `rm <relay_pubkey>.nrdb*`
4. **Restart relay**: This creates new database with new keys
@@ -300,7 +300,7 @@ sudo ufw allow 8888/tcp
```bash
# Secure database file permissions
chmod 600 <relay_pubkey>.nrdb
chown c-relay:c-relay <relay_pubkey>.nrdb
chown c-relay-pg:c-relay-pg <relay_pubkey>.nrdb
# Regular backups
cp <relay_pubkey>.nrdb backup/relay_backup_$(date +%Y%m%d_%H%M%S).nrdb
@@ -311,10 +311,10 @@ cp <relay_pubkey>.nrdb backup/relay_backup_$(date +%Y%m%d_%H%M%S).nrdb
### Service Status
```bash
# Check if relay is running
ps aux | grep c_relay
ps aux | grep c_relay_pg
# SystemD status
sudo systemctl status c-relay
sudo systemctl status c-relay-pg
# Network connections
netstat -tln | grep 8888
@@ -324,16 +324,16 @@ sudo ss -tlpn | grep 8888
### Log Monitoring
```bash
# Real-time logs (systemd)
sudo journalctl -u c-relay -f
sudo journalctl -u c-relay-pg -f
# Recent logs
sudo journalctl -u c-relay --since "1 hour ago"
sudo journalctl -u c-relay-pg --since "1 hour ago"
# Error logs only
sudo journalctl -u c-relay -p err
sudo journalctl -u c-relay-pg -p err
# Configuration changes
sudo journalctl -u c-relay | grep "Configuration updated via kind 33334"
sudo journalctl -u c-relay-pg | grep "Configuration updated via kind 33334"
```
### Database Analytics
@@ -365,13 +365,13 @@ ORDER BY created_at DESC;
du -sh <relay_pubkey>.nrdb*
# Memory usage
ps aux | grep c_relay | awk '{print $6}' # RSS memory in KB
ps aux | grep c_relay_pg | awk '{print $6}' # RSS memory in KB
# Connection count (approximate)
netstat -an | grep :8888 | grep ESTABLISHED | wc -l
# System resources
top -p $(pgrep c_relay)
top -p $(pgrep c_relay_pg)
```
## Troubleshooting
@@ -385,11 +385,11 @@ netstat -tln | grep 8888
# If port in use, find process: sudo lsof -i :8888
# Check binary permissions
ls -la build/c_relay_x86
chmod +x build/c_relay_x86
ls -la build/c_relay_pg_x86
chmod +x build/c_relay_pg_x86
# Check dependencies
ldd build/c_relay_x86
ldd build/c_relay_pg_x86
```
#### Configuration Not Updating
@@ -442,7 +442,7 @@ sqlite3 recovered.nrdb < recovered.sql
# If repair fails, start fresh (loses all events)
mv <relay_pubkey>.nrdb <relay_pubkey>.nrdb.corrupted
./build/c_relay_x86 # Creates new database
./build/c_relay_pg_x86 # Creates new database
```
#### Lost Configuration Recovery
@@ -455,12 +455,12 @@ If configuration is lost but database is intact:
#### Emergency Restart
```bash
# Quick restart with clean state
sudo systemctl stop c-relay
sudo systemctl stop c-relay-pg
mv <relay_pubkey>.nrdb <relay_pubkey>.nrdb.backup
sudo systemctl start c-relay
sudo systemctl start c-relay-pg
# Check logs for new admin keys
sudo journalctl -u c-relay --since "5 minutes ago" | grep "Admin Private Key"
sudo journalctl -u c-relay-pg --since "5 minutes ago" | grep "Admin Private Key"
```
## Advanced Usage
@@ -503,7 +503,7 @@ ws.on('open', function() {
# backup-relay.sh
DATE=$(date +%Y%m%d_%H%M%S)
DB_FILE=$(ls *.nrdb | head -1)
BACKUP_DIR="/backup/c-relay"
BACKUP_DIR="/backup/c-relay-pg"
mkdir -p $BACKUP_DIR
cp $DB_FILE $BACKUP_DIR/relay_backup_$DATE.nrdb
@@ -533,7 +533,7 @@ tar czf relay_migration.tar.gz *.nrdb* relay.log
# Target server
tar xzf relay_migration.tar.gz
./build/c_relay_x86 # Will detect existing database and continue
./build/c_relay_pg_x86 # Will detect existing database and continue
```
---
+1 -1
View File
@@ -1,6 +1,6 @@
#!/bin/bash
# Script to embed web files into C headers for the C-Relay admin interface
# Script to embed web files into C headers for the C-Relay-PG admin interface
# Converts HTML, CSS, and JS files from api/ directory into C byte arrays
set -e
+1 -1
View File
@@ -60,7 +60,7 @@ All examples assume the event-based configuration system where:
- **Save Admin Keys**: All deployment examples emphasize capturing the admin private key on first startup
- **Firewall Configuration**: Examples include proper firewall rules
- **SSL/TLS**: Production examples include HTTPS configuration
- **User Isolation**: Service runs as dedicated `c-relay` system user
- **User Isolation**: Service runs as dedicated `c-relay-pg` system user
## Support
+6 -6
View File
@@ -13,8 +13,8 @@ BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Default configuration
RELAY_DIR="/opt/c-relay"
BACKUP_DIR="/backup/c-relay"
RELAY_DIR="/opt/c-relay-pg"
BACKUP_DIR="/backup/c-relay-pg"
RETENTION_DAYS="30"
COMPRESS="true"
REMOTE_BACKUP=""
@@ -47,8 +47,8 @@ show_help() {
echo "Usage: $0 [OPTIONS]"
echo
echo "Options:"
echo " -d, --relay-dir DIR Relay directory (default: /opt/c-relay)"
echo " -b, --backup-dir DIR Backup directory (default: /backup/c-relay)"
echo " -d, --relay-dir DIR Relay directory (default: /opt/c-relay-pg)"
echo " -b, --backup-dir DIR Backup directory (default: /backup/c-relay-pg)"
echo " -r, --retention DAYS Retention period in days (default: 30)"
echo " -n, --no-compress Don't compress backups"
echo " -s, --s3-bucket BUCKET Upload to S3 bucket"
@@ -231,7 +231,7 @@ upload_to_s3() {
print_step "Uploading backup to S3..."
local s3_path="s3://$S3_BUCKET/c-relay/$(date +%Y)/$(date +%m)/"
local s3_path="s3://$S3_BUCKET/c-relay-pg/$(date +%Y)/$(date +%m)/"
if aws s3 cp "$BACKUP_FILE" "$s3_path" --storage-class STANDARD_IA; then
print_success "Backup uploaded to S3: $s3_path"
@@ -264,7 +264,7 @@ cleanup_old_backups() {
print_step "Cleaning S3 backups older than $cutoff_date..."
# Note: This is a simplified approach. In production, use S3 lifecycle policies
aws s3 ls "s3://$S3_BUCKET/c-relay/" --recursive | \
aws s3 ls "s3://$S3_BUCKET/c-relay-pg/" --recursive | \
awk '$1 < "'$cutoff_date'" {print $4}' | \
while read -r key; do
aws s3 rm "s3://$S3_BUCKET/$key"
@@ -13,8 +13,8 @@ BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Configuration
RELAY_DIR="/opt/c-relay"
SERVICE_NAME="c-relay"
RELAY_DIR="/opt/c-relay-pg"
SERVICE_NAME="c-relay-pg"
RELAY_PORT="8888"
LOG_FILE="/var/log/relay-monitor.log"
ALERT_EMAIL=""
@@ -60,7 +60,7 @@ show_help() {
echo "Usage: $0 [OPTIONS]"
echo
echo "Options:"
echo " -d, --relay-dir DIR Relay directory (default: /opt/c-relay)"
echo " -d, --relay-dir DIR Relay directory (default: /opt/c-relay-pg)"
echo " -p, --port PORT Relay port (default: 8888)"
echo " -i, --interval SECONDS Check interval (default: 60)"
echo " -e, --email EMAIL Alert email address"
@@ -134,7 +134,7 @@ parse_args() {
check_process_running() {
print_step "Checking if relay process is running..."
if pgrep -f "c_relay_x86" > /dev/null; then
if pgrep -f "c_relay_pg_x86" > /dev/null; then
print_success "Relay process is running"
return 0
else
@@ -172,7 +172,7 @@ check_service_status() {
check_memory_usage() {
print_step "Checking memory usage..."
local memory_kb=$(ps aux | grep "c_relay_x86" | grep -v grep | awk '{sum+=$6} END {print sum}')
local memory_kb=$(ps aux | grep "c_relay_pg_x86" | grep -v grep | awk '{sum+=$6} END {print sum}')
if [[ -z "$memory_kb" ]]; then
print_warning "Could not determine memory usage"
+4 -4
View File
@@ -56,7 +56,7 @@ http {
}
# Upstream for the relay
upstream c_relay_backend {
upstream c_relay_pg_backend {
server 127.0.0.1:8888;
keepalive 32;
}
@@ -108,7 +108,7 @@ http {
# Main proxy location for WebSocket and HTTP
location / {
# Proxy settings
proxy_pass http://c_relay_backend;
proxy_pass http://c_relay_pg_backend;
proxy_http_version 1.1;
proxy_cache_bypass $http_upgrade;
@@ -144,7 +144,7 @@ http {
# Health check endpoint (if implemented)
location /health {
proxy_pass http://c_relay_backend/health;
proxy_pass http://c_relay_pg_backend/health;
access_log off;
}
@@ -157,7 +157,7 @@ http {
# Optional: Metrics endpoint (if implemented)
location /metrics {
proxy_pass http://c_relay_backend/metrics;
proxy_pass http://c_relay_pg_backend/metrics;
# Restrict access to monitoring systems
allow 10.0.0.0/8;
allow 172.16.0.0/12;
@@ -96,9 +96,9 @@ check_root() {
check_relay_running() {
print_step "Checking if C Nostr Relay is running..."
if ! pgrep -f "c_relay_x86" > /dev/null; then
if ! pgrep -f "c_relay_pg_x86" > /dev/null; then
print_error "C Nostr Relay is not running"
print_error "Please start the relay first with: sudo systemctl start c-relay"
print_error "Please start the relay first with: sudo systemctl start c-relay-pg"
exit 1
fi
+12 -12
View File
@@ -13,9 +13,9 @@ BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Configuration
RELAY_USER="c-relay"
INSTALL_DIR="/opt/c-relay"
SERVICE_NAME="c-relay"
RELAY_USER="c-relay-pg"
INSTALL_DIR="/opt/c-relay-pg"
SERVICE_NAME="c-relay-pg"
RELAY_PORT="8888"
# Functions
@@ -99,7 +99,7 @@ build_relay() {
# Check if we're in the source directory
if [[ ! -f "Makefile" ]]; then
print_error "Makefile not found. Please run this script from the c-relay source directory."
print_error "Makefile not found. Please run this script from the c-relay-pg source directory."
exit 1
fi
@@ -107,7 +107,7 @@ build_relay() {
make clean
make
if [[ ! -f "build/c_relay_x86" ]]; then
if [[ ! -f "build/c_relay_pg_x86" ]]; then
print_error "Build failed - binary not found"
exit 1
fi
@@ -118,9 +118,9 @@ build_relay() {
install_binary() {
print_step "Installing relay binary..."
cp build/c_relay_x86 "$INSTALL_DIR/"
chown "$RELAY_USER:$RELAY_USER" "$INSTALL_DIR/c_relay_x86"
chmod +x "$INSTALL_DIR/c_relay_x86"
cp build/c_relay_pg_x86 "$INSTALL_DIR/"
chown "$RELAY_USER:$RELAY_USER" "$INSTALL_DIR/c_relay_pg_x86"
chmod +x "$INSTALL_DIR/c_relay_pg_x86"
print_success "Binary installed to $INSTALL_DIR"
}
@@ -129,14 +129,14 @@ install_service() {
print_step "Installing systemd service..."
# Use the existing systemd service file
if [[ -f "systemd/c-relay.service" ]]; then
cp systemd/c-relay.service /etc/systemd/system/
if [[ -f "systemd/c-relay-pg.service" ]]; then
cp systemd/c-relay-pg.service /etc/systemd/system/
systemctl daemon-reload
print_success "Systemd service installed"
else
print_warning "Systemd service file not found, creating basic one..."
cat > /etc/systemd/system/c-relay.service << EOF
cat > /etc/systemd/system/c-relay-pg.service << EOF
[Unit]
Description=C Nostr Relay
After=network.target
@@ -146,7 +146,7 @@ Type=simple
User=$RELAY_USER
Group=$RELAY_USER
WorkingDirectory=$INSTALL_DIR
ExecStart=$INSTALL_DIR/c_relay_x86
ExecStart=$INSTALL_DIR/c_relay_pg_x86
Restart=always
RestartSec=5
@@ -1,4 +1,4 @@
# MUSL-based fully static C-Relay builder
# MUSL-based fully static C-Relay-PG builder
# Produces portable binaries with zero runtime dependencies
FROM alpine:latest AS builder
@@ -125,7 +125,7 @@ RUN cd /tmp && \
--prefix=/usr && \
make && make install
# Copy c-relay source
# Copy c-relay-pg source
COPY . /build/
# Initialize submodules
@@ -134,7 +134,7 @@ RUN git submodule update --init --recursive
# Build nostr_core_lib
RUN cd nostr_core_lib && ./build.sh
# Build c-relay static
# Build c-relay-pg static
RUN make clean && \
CC="musl-gcc -static" \
CFLAGS="-O2 -Wall -Wextra -std=c99 -g" \
@@ -143,8 +143,8 @@ RUN make clean && \
make
# Strip binary for size
RUN strip build/c_relay_x86
RUN strip build/c_relay_pg_x86
# Multi-stage build to produce minimal output
FROM scratch AS output
COPY --from=builder /build/build/c_relay_x86 /c_relay_static_musl_x86_64
COPY --from=builder /build/build/c_relay_pg_x86 /c_relay_pg_static_musl_x86_64
+8 -8
View File
@@ -19,7 +19,7 @@ RELEASE_MODE=false
VERSION_INCREMENT_TYPE="patch" # "patch", "minor", or "major"
show_usage() {
echo "C-Relay Increment and Push Script"
echo "C-Relay-PG Increment and Push Script"
echo ""
echo "USAGE:"
echo " $0 [OPTIONS] \"commit message\""
@@ -330,7 +330,7 @@ build_release_binary() {
create_source_tarball() {
print_status "Creating source tarball..."
local tarball_name="c-relay-${NEW_VERSION#v}.tar.gz"
local tarball_name="c-relay-pg-${NEW_VERSION#v}.tar.gz"
# Create tarball excluding build artifacts and git files
if tar -czf "$tarball_name" \
@@ -365,7 +365,7 @@ upload_release_assets() {
fi
local token=$(cat "$HOME/.gitea_token" | tr -d '\n\r')
local api_url="https://git.laantungir.net/api/v1/repos/laantungir/c-relay"
local api_url="https://git.laantungir.net/api/v1/repos/laantungir/c-relay-pg"
local assets_url="$api_url/releases/$release_id/assets"
print_status "Assets URL: $assets_url"
@@ -427,7 +427,7 @@ create_gitea_release() {
fi
local token=$(cat "$HOME/.gitea_token" | tr -d '\n\r')
local api_url="https://git.laantungir.net/api/v1/repos/laantungir/c-relay"
local api_url="https://git.laantungir.net/api/v1/repos/laantungir/c-relay-pg"
# Create release
print_status "Creating release $NEW_VERSION..."
@@ -473,7 +473,7 @@ create_gitea_release() {
# Main execution
main() {
print_status "C-Relay Increment and Push Script"
print_status "C-Relay-PG Increment and Push Script"
# Check prerequisites
check_git_repo
@@ -505,13 +505,13 @@ main() {
# Build release binary
if build_release_binary; then
local binary_path="build/c_relay_static_x86_64"
local binary_path="build/c_relay_pg_static_x86_64"
else
print_warning "Binary build failed, continuing with release creation"
# Check if binary exists from previous build
if [[ -f "build/c_relay_static_x86_64" ]]; then
if [[ -f "build/c_relay_pg_static_x86_64" ]]; then
print_status "Using existing binary from previous build"
binary_path="build/c_relay_static_x86_64"
binary_path="build/c_relay_pg_static_x86_64"
else
binary_path=""
fi
+232 -14
View File
@@ -1,6 +1,6 @@
#!/bin/bash
# C-Relay Build and Restart Script
# C-Relay-PG Build and Restart Script
# Builds the project first, then stops any running relay and starts a new one in the background
echo "=== C Nostr Relay Build and Restart Script ==="
@@ -13,6 +13,13 @@ ADMIN_KEY=""
RELAY_KEY=""
PORT_OVERRIDE=""
DEBUG_LEVEL="5"
DB_BACKEND="postgres"
DB_CONNSTRING=""
DB_HOST=""
DB_PORT=""
DB_NAME=""
DB_USER=""
DB_PASSWORD=""
# Key validation function
validate_hex_key() {
@@ -118,6 +125,84 @@ while [[ $# -gt 0 ]]; do
shift 2
fi
;;
--db-backend)
if [ -z "$2" ]; then
echo "ERROR: --db-backend requires a value (sqlite|postgres)"
HELP=true
shift
else
DB_BACKEND="$2"
shift 2
fi
;;
--db-backend=*)
DB_BACKEND="${1#*=}"
shift
;;
--db-connstring)
if [ -z "$2" ]; then
echo "ERROR: --db-connstring requires a value"
HELP=true
shift
else
DB_CONNSTRING="$2"
shift 2
fi
;;
--db-connstring=*)
DB_CONNSTRING="${1#*=}"
shift
;;
--db-host)
if [ -z "$2" ]; then
echo "ERROR: --db-host requires a value"
HELP=true
shift
else
DB_HOST="$2"
shift 2
fi
;;
--db-port)
if [ -z "$2" ]; then
echo "ERROR: --db-port requires a value"
HELP=true
shift
else
DB_PORT="$2"
shift 2
fi
;;
--db-name)
if [ -z "$2" ]; then
echo "ERROR: --db-name requires a value"
HELP=true
shift
else
DB_NAME="$2"
shift 2
fi
;;
--db-user)
if [ -z "$2" ]; then
echo "ERROR: --db-user requires a value"
HELP=true
shift
else
DB_USER="$2"
shift 2
fi
;;
--db-password)
if [ -z "$2" ]; then
echo "ERROR: --db-password requires a value"
HELP=true
shift
else
DB_PASSWORD="$2"
shift 2
fi
;;
--help|-h)
HELP=true
shift
@@ -164,6 +249,98 @@ if [ -n "$DEBUG_LEVEL" ]; then
fi
fi
# Validate DB backend
if [ "$DB_BACKEND" != "sqlite" ] && [ "$DB_BACKEND" != "postgres" ]; then
echo "ERROR: Invalid --db-backend value '$DB_BACKEND'. Use sqlite or postgres."
exit 1
fi
# Validate DB port if provided
if [ -n "$DB_PORT" ]; then
if ! [[ "$DB_PORT" =~ ^[0-9]+$ ]] || [ "$DB_PORT" -lt 1 ] || [ "$DB_PORT" -gt 65535 ]; then
echo "ERROR: --db-port must be a number between 1 and 65535"
exit 1
fi
fi
if [ "$DB_BACKEND" = "postgres" ] && [ -z "$DB_CONNSTRING" ]; then
[ -z "$DB_HOST" ] && DB_HOST="localhost"
[ -z "$DB_PORT" ] && DB_PORT="5432"
[ -z "$DB_NAME" ] && DB_NAME="crelay"
[ -z "$DB_USER" ] && DB_USER="crelay"
[ -z "$DB_PASSWORD" ] && DB_PASSWORD="crelay"
fi
ensure_postgres_database() {
if ! command -v psql >/dev/null 2>&1; then
echo "ERROR: psql not found. Install PostgreSQL client tools to continue."
return 1
fi
echo "Ensuring PostgreSQL database exists: host=$DB_HOST port=$DB_PORT dbname=$DB_NAME user=$DB_USER"
if [ -n "$DB_PASSWORD" ]; then
export PGPASSWORD="$DB_PASSWORD"
fi
# Prefer local postgres superuser for provisioning when available.
if sudo -n -u postgres psql -d postgres -tAc "SELECT 1" >/dev/null 2>&1; then
sudo -u postgres psql -d postgres -tAc "SELECT 1 FROM pg_roles WHERE rolname = '$DB_USER'" | grep -q 1 || \
sudo -u postgres psql -d postgres -v ON_ERROR_STOP=1 -c "CREATE ROLE \"$DB_USER\" LOGIN PASSWORD '$DB_PASSWORD';" || return 1
sudo -u postgres psql -d postgres -v ON_ERROR_STOP=1 -c "ALTER ROLE \"$DB_USER\" WITH LOGIN PASSWORD '$DB_PASSWORD';" >/dev/null || return 1
if [ "$PRESERVE_DATABASE" = false ]; then
echo "Resetting PostgreSQL database '$DB_NAME' for fresh start..."
# Ensure no active sessions block DROP DATABASE (common when relay is still running).
sudo -u postgres psql -d postgres -v ON_ERROR_STOP=1 -c \
"SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = '$DB_NAME' AND pid <> pg_backend_pid();" >/dev/null || return 1
# Prefer force drop when supported; fallback to regular dropdb.
if ! sudo -u postgres dropdb --if-exists --force "$DB_NAME" >/dev/null 2>&1; then
sudo -u postgres dropdb --if-exists "$DB_NAME" >/dev/null 2>&1 || return 1
fi
sudo -u postgres createdb -O "$DB_USER" "$DB_NAME" >/dev/null 2>&1 || return 1
echo "✓ Recreated PostgreSQL database '$DB_NAME' owned by '$DB_USER'"
else
DB_EXISTS=$(sudo -u postgres psql -d postgres -tAc "SELECT 1 FROM pg_database WHERE datname = '$DB_NAME'" 2>/dev/null | tr -d '[:space:]')
if [ "$DB_EXISTS" != "1" ]; then
sudo -u postgres createdb -O "$DB_USER" "$DB_NAME" >/dev/null 2>&1 || return 1
echo "✓ Created PostgreSQL database '$DB_NAME'"
else
echo "✓ PostgreSQL database '$DB_NAME' already exists"
fi
fi
sudo -u postgres psql -d postgres -v ON_ERROR_STOP=1 -c "GRANT ALL PRIVILEGES ON DATABASE \"$DB_NAME\" TO \"$DB_USER\";" >/dev/null || return 1
sudo -u postgres psql -d "$DB_NAME" -v ON_ERROR_STOP=1 -c "ALTER SCHEMA public OWNER TO \"$DB_USER\"; GRANT ALL ON SCHEMA public TO \"$DB_USER\";" >/dev/null || return 1
return 0
fi
# Fallback: use configured DB user directly.
if ! psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -v ON_ERROR_STOP=1 -tAc "SELECT 1" >/dev/null 2>&1; then
echo "ERROR: Cannot connect to PostgreSQL server/database 'postgres' with current settings"
echo " host=$DB_HOST port=$DB_PORT user=$DB_USER"
return 1
fi
DB_EXISTS=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -tAc "SELECT 1 FROM pg_database WHERE datname = '$DB_NAME'" 2>/dev/null | tr -d '[:space:]')
if [ "$DB_EXISTS" != "1" ]; then
echo "Database '$DB_NAME' not found. Creating..."
if ! createdb -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" "$DB_NAME" >/dev/null 2>&1; then
echo "ERROR: Failed to create PostgreSQL database '$DB_NAME'"
return 1
fi
echo "✓ Created PostgreSQL database '$DB_NAME'"
else
echo "✓ PostgreSQL database '$DB_NAME' already exists"
fi
return 0
}
# Show help
if [ "$HELP" = true ]; then
echo "Usage: $0 [OPTIONS]"
@@ -175,6 +352,13 @@ if [ "$HELP" = true ]; then
echo " -d, --debug-level <0-5> Set debug level: 0=none, 1=errors, 2=warnings, 3=info, 4=debug, 5=trace"
echo " --preserve-database Keep existing database files (don't delete for fresh start)"
echo " --test-keys, -t Use deterministic test keys for development (admin: all 'a's, relay: all '1's)"
echo " --db-backend <name> Database backend: postgres (default) or sqlite"
echo " --db-connstring <str> PostgreSQL libpq connection string"
echo " --db-host <host> PostgreSQL host"
echo " --db-port <port> PostgreSQL port"
echo " --db-name <name> PostgreSQL database name"
echo " --db-user <user> PostgreSQL database user"
echo " --db-password <pass> PostgreSQL database password"
echo " --help, -h Show this help message"
echo ""
echo "Event-Based Configuration:"
@@ -194,6 +378,9 @@ if [ "$HELP" = true ]; then
echo " $0 --test-keys # Use test keys for consistent development"
echo " $0 -t --preserve-database # Use test keys and preserve database"
echo ""
echo "Default PostgreSQL connection (when no DB flags provided):"
echo " host=localhost port=5432 dbname=crelay user=crelay password=crelay"
echo ""
echo "Key Format: Keys must be exactly 64 hexadecimal characters (0-9, a-f, A-F)"
echo "Default behavior: Deletes existing database files to start fresh with new keys"
echo " for development purposes"
@@ -217,13 +404,17 @@ fi
rm -rf dev-config/ 2>/dev/null
rm -f db/c_nostr_relay.db* 2>/dev/null
if [ "$DB_BACKEND" = "postgres" ] && [ -z "$DB_CONNSTRING" ]; then
ensure_postgres_database || exit 1
fi
# Embed web files into C headers before building
echo "Embedding web files..."
./embed_web_files.sh
# Build the project - ONLY static build
echo "Building project (static binary with SQLite JSON1 extension)..."
./build_static.sh
echo "Building project (static binary, backend: $DB_BACKEND)..."
./build_static.sh --db-backend "$DB_BACKEND"
# Exit if static build fails - no fallback
if [ $? -ne 0 ]; then
@@ -242,13 +433,13 @@ fi
ARCH=$(uname -m)
case "$ARCH" in
x86_64)
BINARY_PATH="./build/c_relay_static_x86_64"
BINARY_PATH="./build/c_relay_pg_static_x86_64"
;;
aarch64|arm64)
BINARY_PATH="./build/c_relay_static_arm64"
BINARY_PATH="./build/c_relay_pg_static_arm64"
;;
*)
BINARY_PATH="./build/c_relay_static_$ARCH"
BINARY_PATH="./build/c_relay_pg_static_$ARCH"
;;
esac
@@ -270,7 +461,7 @@ echo "Build successful. Proceeding with relay restart..."
echo "Stopping any existing relay servers..."
# Get all relay processes and kill them immediately with -9
RELAY_PIDS=$(pgrep -f "c_relay_" || echo "")
RELAY_PIDS=$(pgrep -f "c_relay_pg_" || echo "")
if [ -n "$RELAY_PIDS" ]; then
echo "Force killing relay processes immediately: $RELAY_PIDS"
kill -9 $RELAY_PIDS 2>/dev/null
@@ -291,7 +482,7 @@ for attempt in {1..15}; do
fuser -k 8888/tcp 2>/dev/null || true
# Double-check for any remaining relay processes
REMAINING_PIDS=$(pgrep -f "c_relay_" || echo "")
REMAINING_PIDS=$(pgrep -f "c_relay_pg_" || echo "")
if [ -n "$REMAINING_PIDS" ]; then
echo "Killing remaining relay processes: $REMAINING_PIDS"
kill -9 $REMAINING_PIDS 2>/dev/null || true
@@ -309,7 +500,7 @@ for attempt in {1..15}; do
done
# Final safety check - ensure no relay processes remain
FINAL_PIDS=$(pgrep -f "c_relay_" || echo "")
FINAL_PIDS=$(pgrep -f "c_relay_pg_" || echo "")
if [ -n "$FINAL_PIDS" ]; then
echo "Final cleanup: killing processes $FINAL_PIDS"
kill -9 $FINAL_PIDS 2>/dev/null || true
@@ -325,7 +516,7 @@ echo "Database will be initialized automatically on startup if needed"
# Start relay in background with output redirection
echo "Starting relay server..."
echo "Debug: Current processes: $(ps aux | grep 'c_relay_' | grep -v grep || echo 'None')"
echo "Debug: Current processes: $(ps aux | grep 'c_relay_pg_' | grep -v grep || echo 'None')"
# Build command line arguments for relay binary
RELAY_ARGS=""
@@ -350,15 +541,42 @@ if [ -n "$DEBUG_LEVEL" ]; then
echo "Using debug level: $DEBUG_LEVEL"
fi
if [ -n "$DB_CONNSTRING" ]; then
RELAY_ARGS="$RELAY_ARGS --db-connstring '$DB_CONNSTRING'"
echo "Using PostgreSQL connection string override"
fi
if [ -n "$DB_HOST" ]; then
RELAY_ARGS="$RELAY_ARGS --db-host '$DB_HOST'"
fi
if [ -n "$DB_PORT" ]; then
RELAY_ARGS="$RELAY_ARGS --db-port '$DB_PORT'"
fi
if [ -n "$DB_NAME" ]; then
RELAY_ARGS="$RELAY_ARGS --db-name '$DB_NAME'"
fi
if [ -n "$DB_USER" ]; then
RELAY_ARGS="$RELAY_ARGS --db-user '$DB_USER'"
fi
if [ -n "$DB_PASSWORD" ]; then
RELAY_ARGS="$RELAY_ARGS --db-password '$DB_PASSWORD'"
fi
# Change to build directory before starting relay so database files are created there
cd build
# Start relay in background and capture its PID
if [ "$USE_TEST_KEYS" = true ]; then
echo "Using test keys from .test_keys file..."
./$(basename $BINARY_PATH) -a "$ADMIN_KEY" -r "$RELAY_KEY" --debug-level=$DEBUG_LEVEL --strict-port > ../relay.log 2>&1 &
# shellcheck disable=SC2086
eval ./$(basename $BINARY_PATH) -a "$ADMIN_KEY" -r "$RELAY_KEY" $RELAY_ARGS --strict-port > ../relay.log 2>&1 &
elif [ -n "$RELAY_ARGS" ]; then
echo "Starting relay with custom configuration..."
./$(basename $BINARY_PATH) $RELAY_ARGS --debug-level=$DEBUG_LEVEL --strict-port > ../relay.log 2>&1 &
# shellcheck disable=SC2086
eval ./$(basename $BINARY_PATH) $RELAY_ARGS --strict-port > ../relay.log 2>&1 &
else
# No command line arguments needed for random key generation
echo "Starting relay with random key generation..."
@@ -428,8 +646,8 @@ if ps -p "$RELAY_PID" >/dev/null 2>&1; then
echo "=== Event-Based Relay Server Running ==="
echo "Configuration: Event-based (kind 33334 Nostr events)"
echo "Database: Automatically created with relay pubkey naming"
echo "To kill relay: pkill -f 'c_relay_'"
echo "To check status: ps aux | grep c_relay_"
echo "To kill relay: pkill -f 'c_relay_pg_'"
echo "To check status: ps aux | grep c_relay_pg_"
echo "To view logs: tail -f relay.log"
echo "Binary: $BINARY_PATH (zero configuration needed)"
echo "Ready for Nostr client connections!"
+12 -12
View File
@@ -12,11 +12,11 @@ After the next crash, analyze it:
# List all core dumps (most recent first)
sudo coredumpctl list
# View info about the most recent c-relay crash
sudo coredumpctl info c-relay
# View info about the most recent c-relay-pg crash
sudo coredumpctl info c-relay-pg
# Load the core dump in gdb for detailed analysis
sudo coredumpctl gdb c-relay
sudo coredumpctl gdb c-relay-pg
Inside gdb, run these commands:
(gdb) bt full # Full backtrace with all variables
@@ -36,8 +36,8 @@ DEBUGGING
Even simpler: Use this one-liner
# Start relay and immediately attach gdb
cd /usr/local/bin/c_relay
sudo -u c-relay ./c_relay --debug-level=5 & sleep 2 && sudo gdb -p $(pgrep c_relay)
cd /usr/local/bin/c_relay_pg
sudo -u c-relay-pg ./c_relay_pg --debug-level=5 & sleep 2 && sudo gdb -p $(pgrep c_relay_pg)
Inside gdb, after attaching:
@@ -48,17 +48,17 @@ Or shorter:
How to View the Logs
Check systemd journal:
# View all c-relay logs
sudo journalctl -u c-relay
# View all c-relay-pg logs
sudo journalctl -u c-relay-pg
# View recent logs (last 50 lines)
sudo journalctl -u c-relay -n 50
sudo journalctl -u c-relay-pg -n 50
# Follow logs in real-time
sudo journalctl -u c-relay -f
sudo journalctl -u c-relay-pg -f
# View logs since last boot
sudo journalctl -u c-relay -b
sudo journalctl -u c-relay-pg -b
Check if service is running:
@@ -78,9 +78,9 @@ sudo systemctl start rsyslog
sudo systemctl status rsyslog
sudo -u c-relay ./c_relay --debug-level=5 -r 85d0b37e2ae822966dcadd06b2dc9368cde73865f90ea4d44f8b57d47ef0820a -a 1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139
sudo -u c-relay-pg ./c_relay_pg --debug-level=5 -r 85d0b37e2ae822966dcadd06b2dc9368cde73865f90ea4d44f8b57d47ef0820a -a 1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139
./c_relay_static_x86_64 -p 7889 --debug-level=5 -r 85d0b37e2ae822966dcadd06b2dc9368cde73865f90ea4d44f8b57d47ef0820a -a 1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139
./c_relay_pg_static_x86_64 -p 7889 --debug-level=5 -r 85d0b37e2ae822966dcadd06b2dc9368cde73865f90ea4d44f8b57d47ef0820a -a 1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139
sudo ufw allow 8888/tcp
+1 -1
View File
@@ -1,5 +1,5 @@
{
"name": "c-relay",
"name": "c-relay-pg",
"lockfileVersion": 3,
"requires": true,
"packages": {
+632
View File
@@ -0,0 +1,632 @@
# c-relay-pg-pg Plan — PostgreSQL Backend + Multi-Instance + Dashboard
## Overview
**c-relay-pg-pg** is a new project (separate repository) forked from c-relay-pg after the `db_ops` abstraction layer is complete. It replaces the SQLite backend with PostgreSQL, enabling horizontal scaling, external dashboards, and production-grade deployments.
### Prerequisites
- The `db_ops.h` / `db_ops.c` abstraction layer must be complete in c-relay-pg first — see [c-relay-pg thread pool plan](c_relay_pg_thread_pool_plan.md) Phase 1.
- The fork happens after Phase 1 of that plan is done and tested.
### Why a Separate Project?
| | c-relay-pg | c-relay-pg-pg |
|---|---------|-----------|
| **Database** | SQLite (embedded) | PostgreSQL (client-server) |
| **Deployment** | Single binary + DB file | Binary + PostgreSQL server |
| **Scaling** | Single instance | Multiple instances + load balancer |
| **Dashboard** | Embedded web UI | Separate web server + Grafana |
| **Target user** | Personal relay, small community | Medium-large relay, production |
| **Complexity** | Minimal | More infrastructure |
For the full analysis of why PostgreSQL was chosen over MySQL/MariaDB and other databases, see the [database architecture analysis](database_architecture_analysis.md).
## Architecture
```mermaid
flowchart TD
subgraph c-relay-pg-pg - Production Deployment
LB[nginx - TLS + load balancing] -->|WebSocket| R1[c-relay-pg-pg Instance 1]
LB -->|WebSocket| R2[c-relay-pg-pg Instance 2]
LB -->|HTTP| DASH[Dashboard]
R1 -->|db_ops API| PGBACK[PostgreSQL Backend - db_ops_postgres.c]
R2 -->|db_ops API| PGBACK
PGBACK -->|libpq| PG[PostgreSQL Server]
DASH -->|SQL| PG
R1 <-->|LISTEN/NOTIFY| R2
end
```
---
## Phase 1: PostgreSQL Backend
### Goal
Fork c-relay-pg into a new repository called **c-relay-pg-pg**. Replace the SQLite `db_ops` implementation with PostgreSQL using `libpq`. The `db_ops.h` interface stays identical — only the backend changes.
### New Files
- `src/db_ops_sqlite.c` — Renamed from `db_ops.c` (the Phase 1 SQLite implementation from c-relay-pg)
- `src/db_ops_postgres.c` — New PostgreSQL implementation
- `src/db_ops.c` — Thin dispatcher that calls the active backend
### PostgreSQL Schema
```sql
-- PostgreSQL schema for c-relay-pg-pg
-- No event_tags table needed — JSONB + GIN handles everything
CREATE TABLE events (
id TEXT PRIMARY KEY,
pubkey TEXT NOT NULL,
created_at BIGINT NOT NULL,
kind INTEGER NOT NULL,
event_type TEXT NOT NULL CHECK (event_type IN ('regular', 'replaceable', 'ephemeral', 'addressable')),
content TEXT NOT NULL,
sig TEXT NOT NULL,
tags JSONB NOT NULL DEFAULT '[]',
event_json TEXT NOT NULL,
first_seen BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
);
-- Core indexes
CREATE INDEX idx_events_pubkey ON events(pubkey);
CREATE INDEX idx_events_kind ON events(kind);
CREATE INDEX idx_events_created_at ON events(created_at DESC);
CREATE INDEX idx_events_kind_created_at ON events(kind, created_at DESC);
CREATE INDEX idx_events_pubkey_created_at ON events(pubkey, created_at DESC);
CREATE INDEX idx_events_pubkey_kind ON events(pubkey, kind);
-- THE KEY INDEX: GIN on JSONB tags — replaces the entire event_tags table
CREATE INDEX idx_events_tags ON events USING GIN (tags);
-- Partial index: only non-ephemeral events (what REQ queries actually filter)
CREATE INDEX idx_events_non_ephemeral ON events(created_at DESC)
WHERE kind < 20000 OR kind >= 30000;
-- Config table
CREATE TABLE config (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
data_type TEXT NOT NULL CHECK (data_type IN ('string', 'integer', 'boolean', 'json')),
description TEXT,
category TEXT DEFAULT 'general',
requires_restart INTEGER DEFAULT 0,
created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,
updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
);
-- Auth rules table
CREATE TABLE auth_rules (
id SERIAL PRIMARY KEY,
rule_type TEXT NOT NULL CHECK (rule_type IN ('whitelist', 'blacklist', 'rate_limit', 'auth_required', 'wot_whitelist')),
pattern_type TEXT NOT NULL CHECK (pattern_type IN ('pubkey', 'kind', 'ip', 'global')),
pattern_value TEXT,
active INTEGER NOT NULL DEFAULT 1,
created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,
updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
);
CREATE INDEX idx_auth_rules_lookup ON auth_rules(rule_type, pattern_type, pattern_value) WHERE active = 1;
-- Relay private key storage
CREATE TABLE relay_seckey (
private_key_hex TEXT NOT NULL CHECK (length(private_key_hex) = 64),
created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
);
-- Subscription logging
CREATE TABLE subscriptions (
id SERIAL PRIMARY KEY,
subscription_id TEXT NOT NULL,
wsi_pointer TEXT NOT NULL,
client_ip TEXT NOT NULL,
event_type TEXT NOT NULL CHECK (event_type IN ('created', 'closed', 'expired', 'disconnected')),
filter_json TEXT,
events_sent INTEGER DEFAULT 0,
created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,
ended_at BIGINT,
duration INTEGER,
UNIQUE(subscription_id, wsi_pointer)
);
-- IP ban persistence
CREATE TABLE ip_bans (
ip TEXT PRIMARY KEY,
failure_count INTEGER NOT NULL DEFAULT 0,
ban_count INTEGER NOT NULL DEFAULT 0,
banned_until BIGINT NOT NULL DEFAULT 0,
first_failure BIGINT NOT NULL DEFAULT 0,
has_authed_successfully INTEGER NOT NULL DEFAULT 0,
last_success_at BIGINT NOT NULL DEFAULT 0,
total_connections INTEGER NOT NULL DEFAULT 0,
total_failures INTEGER NOT NULL DEFAULT 0,
total_successes INTEGER NOT NULL DEFAULT 0,
first_seen BIGINT NOT NULL DEFAULT 0,
idle_failure_count INTEGER NOT NULL DEFAULT 0,
idle_ban_count INTEGER NOT NULL DEFAULT 0,
idle_banned_until BIGINT NOT NULL DEFAULT 0,
idle_first_failure BIGINT NOT NULL DEFAULT 0
);
-- Materialized views for dashboard (refreshed periodically)
CREATE MATERIALIZED VIEW event_kinds_mv AS
SELECT kind, COUNT(*) as count,
ROUND(COUNT(*) * 100.0 / NULLIF((SELECT COUNT(*) FROM events), 0), 2) as percentage
FROM events GROUP BY kind;
CREATE MATERIALIZED VIEW time_stats_mv AS
SELECT 'total' as period, COUNT(*) as total_events, COUNT(DISTINCT pubkey) as unique_pubkeys
FROM events
UNION ALL
SELECT '24h', COUNT(*), COUNT(DISTINCT pubkey)
FROM events WHERE created_at >= EXTRACT(EPOCH FROM NOW())::BIGINT - 86400
UNION ALL
SELECT '7d', COUNT(*), COUNT(DISTINCT pubkey)
FROM events WHERE created_at >= EXTRACT(EPOCH FROM NOW())::BIGINT - 604800;
CREATE MATERIALIZED VIEW top_pubkeys_mv AS
SELECT pubkey, COUNT(*) as event_count,
ROUND(COUNT(*) * 100.0 / NULLIF((SELECT COUNT(*) FROM events), 0), 2) as percentage
FROM events GROUP BY pubkey ORDER BY event_count DESC LIMIT 20;
-- Unique indexes required for CONCURRENTLY refresh
CREATE UNIQUE INDEX idx_event_kinds_mv ON event_kinds_mv(kind);
CREATE UNIQUE INDEX idx_time_stats_mv ON time_stats_mv(period);
CREATE UNIQUE INDEX idx_top_pubkeys_mv ON top_pubkeys_mv(pubkey);
```
### Key Implementation Details
#### Tag Queries with JSONB
The biggest win — replacing the `event_tags` subquery with JSONB containment:
```c
// SQLite (current): subquery into event_tags table
// AND id IN (SELECT event_id FROM event_tags WHERE tag_name = ? AND tag_value IN (?))
// PostgreSQL: JSONB containment operator with GIN index
// AND tags @> '[["p", "pubkey_hex"]]'
// For multiple tag values:
// AND (tags @> '[["p", "val1"]]' OR tags @> '[["p", "val2"]]')
```
This eliminates the entire `event_tags` table (4 million rows, ~2 GB indexes in the current SQLite schema). The GIN index on JSONB handles everything in ~100200 MB.
#### LISTEN/NOTIFY Integration
For cross-instance event broadcasting:
```c
// In db_ops_postgres.c:
int db_notify_new_event(const char* event_id) {
char cmd[128];
snprintf(cmd, sizeof(cmd), "NOTIFY new_event, '%s'", event_id);
PGresult* res = PQexec(g_pg_conn, cmd);
PQclear(res);
return 0;
}
// Called from the lws event loop every iteration:
int db_check_notifications(char* event_id_out, size_t max_len) {
PQconsumeInput(g_pg_notify_conn);
PGnotify* notify = PQnotifies(g_pg_notify_conn);
if (notify) {
strncpy(event_id_out, notify->extra, max_len);
PQfreemem(notify);
return 1; // Got a notification
}
return 0; // No notifications
}
```
#### Connection Management
```c
// db_ops_postgres.c connection pool (simple version)
#define DB_POOL_SIZE 4
static PGconn* g_pg_read_pool[DB_POOL_SIZE]; // For REQ queries
static PGconn* g_pg_write_conn; // For EVENT inserts
static PGconn* g_pg_notify_conn; // For LISTEN/NOTIFY
static pthread_mutex_t g_pool_lock;
PGconn* db_get_read_connection(void) {
pthread_mutex_lock(&g_pool_lock);
// Round-robin or find idle connection
// ...
pthread_mutex_unlock(&g_pool_lock);
}
```
### Build System Changes
```makefile
# Makefile additions
DB_BACKEND ?= sqlite # Default to sqlite, override with: make DB_BACKEND=postgres
ifeq ($(DB_BACKEND),postgres)
MAIN_SRC += src/db_ops.c src/db_ops_postgres.c
LIBS += -lpq
CFLAGS += -DDB_BACKEND_POSTGRES
else
MAIN_SRC += src/db_ops.c src/db_ops_sqlite.c
CFLAGS += -DDB_BACKEND_SQLITE
endif
```
### Data Migration Tool
A standalone tool to migrate existing SQLite data to PostgreSQL:
```bash
# migrate_to_postgres.sh
# 1. Export events from SQLite
sqlite3 old_relay.db ".mode csv" "SELECT id,pubkey,created_at,kind,event_type,content,sig,tags,event_json,first_seen FROM events" > events.csv
# 2. Import into PostgreSQL
psql -d crelay -c "\COPY events FROM 'events.csv' WITH CSV"
# 3. Migrate config
sqlite3 old_relay.db ".mode csv" "SELECT key,value,data_type,description,category,requires_restart FROM config" > config.csv
psql -d crelay -c "\COPY config(key,value,data_type,description,category,requires_restart) FROM 'config.csv' WITH CSV"
# 4. Migrate auth rules
sqlite3 old_relay.db ".mode csv" "SELECT rule_type,pattern_type,pattern_value,active FROM auth_rules" > auth_rules.csv
psql -d crelay -c "\COPY auth_rules(rule_type,pattern_type,pattern_value,active) FROM 'auth_rules.csv' WITH CSV"
# 5. Refresh materialized views
psql -d crelay -c "REFRESH MATERIALIZED VIEW event_kinds_mv; REFRESH MATERIALIZED VIEW time_stats_mv; REFRESH MATERIALIZED VIEW top_pubkeys_mv;"
```
---
## Phase 2: Multi-Instance + Dashboard
### Goal
Run multiple c-relay-pg-pg instances behind a load balancer with a separate dashboard web server, all sharing the same PostgreSQL database.
### Components
1. **nginx config** — WebSocket load balancing with `ip_hash` stickiness
2. **systemd template**`c-relay-pg-pg@.service` for multiple instances
3. **LISTEN/NOTIFY integration** — Cross-instance event broadcasting in the `lws_service()` loop
4. **PgBouncer** — Connection pooling between c-relay-pg-pg instances and PostgreSQL
5. **Dashboard** — Separate web server querying PostgreSQL directly
6. **Materialized view refresh** — Background job to refresh analytics views
### Multi-Instance Architecture
```mermaid
flowchart TD
INTERNET[Internet - Nostr Clients] -->|wss://relay.example.com| NGINX[nginx reverse proxy - TLS termination + load balancing]
NGINX -->|ws://localhost:8888| R1[c-relay-pg-pg Instance 1 - port 8888]
NGINX -->|ws://localhost:8889| R2[c-relay-pg-pg Instance 2 - port 8889]
NGINX -->|ws://localhost:8890| R3[c-relay-pg-pg Instance 3 - port 8890]
NGINX -->|http://localhost:3000| DASHWEB[Dashboard Web Server]
R1 -->|libpq connection pool| PGPOOL[PgBouncer - connection pooler]
R2 -->|libpq connection pool| PGPOOL
R3 -->|libpq connection pool| PGPOOL
PGPOOL -->|pooled connections| PG[PostgreSQL Primary]
DASHWEB -->|read queries| PG
PG -->|streaming replication| REPLICA[Read Replica - optional]
DASHWEB -.->|heavy analytics| REPLICA
```
### nginx Load Balancing Config
```nginx
# nginx.conf - WebSocket load balancing for c-relay-pg-pg
upstream relay_backends {
# ip_hash ensures a client always hits the same instance
# (important for WebSocket session stickiness)
ip_hash;
server 127.0.0.1:8888;
server 127.0.0.1:8889;
server 127.0.0.1:8890;
}
server {
listen 443 ssl;
server_name relay.example.com;
# TLS config...
location / {
proxy_pass http://relay_backends;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header X-Real-IP $remote_addr;
proxy_read_timeout 86400; # Keep WebSocket alive for 24h
}
# Dashboard on separate path
location /dashboard {
proxy_pass http://127.0.0.1:3000;
}
}
```
**Session stickiness** (`ip_hash`) ensures that once a client connects to Instance 2, all their subsequent WebSocket frames go to Instance 2. This is important because subscriptions are held in-memory per instance.
### Starting Multiple Instances
Each instance is the same binary, just on a different port, all pointing to the same PostgreSQL:
```bash
# Instance 1
./build/c_relay_pg_x86 --port 8888 --db-host localhost --db-name crelay &
# Instance 2
./build/c_relay_pg_x86 --port 8889 --db-host localhost --db-name crelay &
# Instance 3
./build/c_relay_pg_x86 --port 8890 --db-host localhost --db-name crelay &
```
Or with systemd template units:
```ini
# /etc/systemd/system/c-relay-pg-pg@.service
[Unit]
Description=C-Relay-PG-PG Nostr Instance %i
After=postgresql.service
[Service]
ExecStart=/opt/c-relay-pg-pg/c_relay_pg_x86 --port %i --db-host localhost --db-name crelay
Restart=always
User=c-relay-pg
[Install]
WantedBy=multi-user.target
```
```bash
systemctl enable c-relay-pg-pg@8888 c-relay-pg-pg@8889 c-relay-pg-pg@8890
systemctl start c-relay-pg-pg@8888 c-relay-pg-pg@8889 c-relay-pg-pg@8890
```
### Cross-Instance Event Broadcasting
When Instance 1 receives a new EVENT and stores it in PostgreSQL, Instance 2 and Instance 3 need to know about it so they can broadcast to their connected subscribers.
```mermaid
sequenceDiagram
participant Client_A as Client A - connected to Instance 1
participant I1 as Instance 1
participant PG as PostgreSQL
participant I2 as Instance 2
participant I3 as Instance 3
participant Client_B as Client B - connected to Instance 2
participant Client_C as Client C - connected to Instance 3
Client_A->>I1: EVENT - new kind:1 note
I1->>PG: INSERT INTO events...
PG-->>I1: OK
I1->>I1: broadcast to local subscribers
I1->>PG: NOTIFY new_event with event_id
Note over PG: PostgreSQL delivers notification to all listeners
PG-->>I2: NOTIFY: new_event event_id
PG-->>I3: NOTIFY: new_event event_id
I2->>PG: SELECT event_json FROM events WHERE id = event_id
PG-->>I2: event JSON
I2->>I2: match against local subscriptions
I2->>Client_B: EVENT message - if subscription matches
I3->>PG: SELECT event_json FROM events WHERE id = event_id
PG-->>I3: event JSON
I3->>I3: match against local subscriptions
I3->>Client_C: EVENT message - if subscription matches
```
#### PostgreSQL LISTEN/NOTIFY Implementation
Built into PostgreSQL — no additional infrastructure needed:
```c
// === In the relay's event loop (modified lws_service loop) ===
// Setup: create a dedicated connection for LISTEN
PGconn* notify_conn = PQconnectdb("host=localhost dbname=crelay");
PQexec(notify_conn, "LISTEN new_event");
int notify_fd = PQsocket(notify_conn); // Get the socket fd for poll()
// After storing an event:
void on_event_stored(PGconn* write_conn, const char* event_id) {
char notify_cmd[128];
snprintf(notify_cmd, sizeof(notify_cmd),
"NOTIFY new_event, '%s'", event_id);
PQexec(write_conn, notify_cmd);
}
// In the main event loop (runs every lws_service iteration):
void check_cross_instance_events(PGconn* notify_conn) {
// Non-blocking check for notifications
PQconsumeInput(notify_conn);
PGnotify* notify;
while ((notify = PQnotifies(notify_conn)) != NULL) {
// Another instance stored a new event
const char* event_id = notify->extra;
// Fetch the event and check against local subscriptions
cJSON* event = db_get_event_by_id(event_id);
if (event) {
broadcast_event_to_subscriptions(event);
cJSON_Delete(event);
}
PQfreemem(notify);
}
}
```
**Performance**: LISTEN/NOTIFY adds ~15ms latency for cross-instance delivery. For a Nostr relay, this is imperceptible — clients already expect network latency.
**Payload limit**: NOTIFY payloads are limited to 8000 bytes. For event IDs (64 hex chars), this is fine.
#### Alternative: Redis Pub/Sub
If you later need even lower latency or more sophisticated routing:
```c
// Using hiredis (Redis C client)
redisContext* redis = redisConnect("127.0.0.1", 6379);
// After storing event:
redisCommand(redis, "PUBLISH new_event %s", event_json);
// Subscriber (in each instance):
redisCommand(redis, "SUBSCRIBE new_event");
// Then poll for messages in the event loop
```
Redis adds sub-millisecond pub/sub but requires running a Redis server. For most relays, PostgreSQL LISTEN/NOTIFY is sufficient.
### Connection Pooling with PgBouncer
Each relay instance needs multiple database connections. Without pooling, 3 instances × 10 connections = 30 PostgreSQL backend processes. With PgBouncer:
```ini
# /etc/pgbouncer/pgbouncer.ini
[databases]
crelay = host=127.0.0.1 port=5432 dbname=crelay
[pgbouncer]
listen_port = 6432
listen_addr = 127.0.0.1
auth_type = md5
pool_mode = transaction # Return connection to pool after each transaction
max_client_conn = 200 # Total connections from all relay instances
default_pool_size = 20 # Actual PostgreSQL connections
```
Relay instances connect to PgBouncer (port 6432) instead of PostgreSQL directly (port 5432). PgBouncer multiplexes 200 client connections onto 20 actual PostgreSQL connections.
### Zero-Downtime Deployment
```mermaid
sequenceDiagram
participant LB as nginx
participant I1 as Instance 1 - v1.0
participant I2 as Instance 2 - v1.0
participant I3 as Instance 3 - v1.0
participant I1_NEW as Instance 1 - v1.1
Note over LB,I3: Normal operation: 3 instances serving traffic
LB->>I1: Mark upstream as down
Note over I1: Drain: wait for existing connections to close or timeout
I1->>I1: Graceful shutdown
Note over LB: Traffic now goes to I2 and I3 only
I1_NEW->>I1_NEW: Start with new binary
I1_NEW->>LB: Health check passes
LB->>I1_NEW: Mark upstream as up
Note over LB,I1_NEW: Instance 1 now running v1.1, repeat for I2 and I3
```
Rolling deploy script:
```bash
#!/bin/bash
# rolling_deploy.sh - Zero-downtime deployment
for port in 8888 8889 8890; do
echo "Deploying instance on port $port..."
# 1. Tell nginx to stop sending new connections
sed -i "s/server 127.0.0.1:$port;/server 127.0.0.1:$port down;/" /etc/nginx/nginx.conf
nginx -s reload
# 2. Wait for existing connections to drain (30 seconds)
sleep 30
# 3. Stop old instance
systemctl stop c-relay-pg-pg@$port
# 4. Deploy new binary
cp ./build/c_relay_pg_x86 /opt/c-relay-pg-pg/c_relay_pg_x86
# 5. Start new instance
systemctl start c-relay-pg-pg@$port
# 6. Wait for health check
sleep 5
# 7. Re-enable in nginx
sed -i "s/server 127.0.0.1:$port down;/server 127.0.0.1:$port;/" /etc/nginx/nginx.conf
nginx -s reload
echo "Instance on port $port deployed successfully"
done
```
### Dashboard Options
With PostgreSQL, the dashboard can be built with any technology:
- **Grafana** — Point directly at PostgreSQL, zero custom code
- **Custom web app** — React/Vue frontend + any backend (Node, Python, Go) querying PostgreSQL
- **Embedded in c-relay-pg-pg** — Keep current approach but queries go through `db_ops` to PostgreSQL (no longer blocks event loop since PostgreSQL handles concurrency)
### Scaling Scenarios
| Scenario | Instances | Connections | Events/hour | Setup |
|----------|-----------|-------------|-------------|-------|
| **Current** | 1 | ~1,200 | 56 | Single process + SQLite |
| **Small upgrade** | 2 | ~2,500 | 500 | 2 instances + PostgreSQL |
| **Medium relay** | 4 | ~5,000 | 5,000 | 4 instances + PostgreSQL + PgBouncer |
| **Large relay** | 8 | ~10,000 | 50,000 | 8 instances + PostgreSQL + read replica |
| **Multi-region** | 24 per region | ~50,000 | 500,000 | Multiple servers + PostgreSQL replication |
### Cost Perspective
Running multiple relay instances with PostgreSQL on a single VPS:
- **PostgreSQL**: ~200500 MB RAM baseline
- **PgBouncer**: ~10 MB RAM
- **Each relay instance**: ~50100 MB RAM
- **Dashboard web server**: ~50100 MB RAM
- **4 instances + PostgreSQL + PgBouncer + dashboard**: ~12 GB total RAM
- A **$20/month VPS** with 4 cores and 4 GB RAM handles this easily
- A **$40/month VPS** with 8 cores and 8 GB RAM handles 8 instances comfortably
---
## Risk Mitigation
| Risk | Mitigation |
|------|-----------|
| PostgreSQL connection failures | `db_ops` returns error codes. Relay logs errors but doesn't crash. Reconnection logic with exponential backoff. |
| Performance regression | Benchmark before/after. PostgreSQL should be faster for complex queries, similar for simple ones. |
| Data loss during migration | Migration tool is read-only on SQLite. PostgreSQL import is idempotent (INSERT ON CONFLICT). |
| SQLite fallback needed | Keep `db_ops_sqlite.c` working. Compile-time flag switches backends. Can always go back. |
| LISTEN/NOTIFY message loss | NOTIFY is transactional — if the INSERT commits, the NOTIFY is guaranteed. Missed notifications during reconnect handled by periodic full-sync. |
| PgBouncer adds latency | Transaction pooling mode adds <0.1ms. Negligible compared to query time. |
---
## Relationship to c-relay-pg
This project depends on the `db_ops.h` abstraction layer built in [c-relay-pg thread pool plan](c_relay_pg_thread_pool_plan.md) Phase 1. The interface is identical — only the backend implementation changes:
- **c-relay-pg**: `db_ops.c` → SQLite calls via `sqlite3_*`
- **c-relay-pg-pg**: `db_ops_postgres.c` → PostgreSQL calls via `libpq` (`PQexec`, `PQgetvalue`, etc.)
The `db_ops.h` header file is shared between both projects. Changes to the interface should be coordinated.
+332
View File
@@ -0,0 +1,332 @@
# c-relay-pg Thread Pool Plan — db_ops Abstraction + SQLite Thread Pool
## Overview
This plan covers improvements to **c-relay-pg** (this repo) — the lean, single-binary, embedded-SQLite Nostr relay. Two phases:
1. **Phase 1: Database Abstraction Layer** — Consolidate all scattered `sqlite3_*` calls into a single `db_ops.h` / `db_ops.c` module. Pure refactor, no behavior change.
2. **Phase 2: SQLite Thread Pool** — Add worker threads for concurrent reads, unblocking the `lws_service()` event loop.
This is the **final form of c-relay-pg**: an embedded-SQLite relay with clean architecture and non-blocking database access. The abstraction layer also enables a future fork to PostgreSQL — see [c-relay-pg-pg plan](c_relay_pg_pg_plan.md).
### Why This First?
The current architecture has a fundamental bottleneck documented in the [database architecture analysis](database_architecture_analysis.md): the single-threaded event loop blocks on every database call. A 672ms REQ query freezes every connected client. The thread pool fixes this without changing the database engine or deployment model.
| Metric | Current | After Thread Pool |
|--------|---------|-------------------|
| **Event loop blocking** | 0.1672ms per query | Near-zero |
| **Concurrent reads** | 1 | 48 |
| **Deployment** | Single binary + DB file | Same |
| **Database** | SQLite | Same |
| **Code risk** | N/A | Low — additive change |
## Architecture
```mermaid
flowchart TD
subgraph c-relay-pg - Final Form
RELAY[c-relay-pg binary] -->|db_ops API| DBOPS[db_ops.c - abstraction layer]
DBOPS -->|sqlite3 calls| SQLITE[SQLite WAL database]
end
```
```mermaid
flowchart TD
subgraph c-relay-pg with Thread Pool
LWS[lws_service event loop] -->|REQ arrives| Q[Thread-safe job queue]
LWS -->|EVENT arrives| WQ[Write queue - single writer]
Q --> T1[Reader Thread 1 - own sqlite3*]
Q --> T2[Reader Thread 2 - own sqlite3*]
Q --> T3[Reader Thread 3 - own sqlite3*]
Q --> T4[Reader Thread 4 - own sqlite3*]
WQ --> TW[Writer Thread - own sqlite3*]
T1 -->|results| CB[Callback to lws event loop]
T2 -->|results| CB
T3 -->|results| CB
T4 -->|results| CB
TW -->|OK/error| CB
CB -->|queue_message| LWS
end
```
---
## Phase 1: Database Abstraction Layer
### Goal
Consolidate all 258 scattered `sqlite3_*` calls across 8 files into a single `db_ops.h` / `db_ops.c` module with a backend-agnostic interface. SQLite continues to work — this is a pure refactor.
### Files That Currently Touch SQLite Directly
| File | `sqlite3_*` calls | Operations |
|------|-------------------|------------|
| [`src/main.c`](../src/main.c) | ~80 | Event store/retrieve, tag storage, REQ queries, COUNT queries, DB init, schema migration |
| [`src/config.c`](../src/config.c) | ~60 | Config CRUD, auth rules CRUD, WoT sync, relay key storage, startup sequence |
| [`src/api.c`](../src/api.c) | ~40 | Stats queries, monitoring views, admin SQL execution, config management |
| [`src/dm_admin.c`](../src/dm_admin.c) | ~20 | Auth rule management, WoT whitelist operations |
| [`src/websockets.c`](../src/websockets.c) | ~15 | COUNT queries, IP ban stats, connection tracking |
| [`src/subscriptions.c`](../src/subscriptions.c) | ~15 | Subscription logging — create/close/disconnect |
| [`src/nip009.c`](../src/nip009.c) | ~10 | Event deletion — by ID and by address |
| [`src/request_validator.c`](../src/request_validator.c) | ~8 | Blacklist/whitelist checks |
| [`src/ip_ban.c`](../src/ip_ban.c) | ~15 | IP ban table CRUD, persistence |
### Abstraction Layer Design
New files: `src/db_ops.h` and `src/db_ops.c`
```c
// src/db_ops.h — Database operations abstraction layer
#ifndef DB_OPS_H
#define DB_OPS_H
#include "../nostr_core_lib/cjson/cJSON.h"
// ============================================================
// Lifecycle
// ============================================================
int db_init(const char* connection_string); // SQLite: file path, PG: connection string
void db_close(void);
int db_is_available(void);
// ============================================================
// Schema / Migration
// ============================================================
int db_ensure_schema(void);
int db_get_schema_version(void);
int db_execute_raw(const char* sql); // For schema DDL only
// ============================================================
// Event Operations
// ============================================================
int db_store_event(cJSON* event);
int db_event_exists(const char* event_id);
char* db_get_event_json(const char* event_id); // Caller must free
int db_delete_event_by_id(const char* event_id, const char* requester_pubkey);
int db_delete_events_by_address(const char* pubkey, int kind,
const char* d_tag, long before_timestamp);
// ============================================================
// Event Queries - REQ handling
// ============================================================
typedef struct {
int* kinds; int kind_count;
char** authors; int author_count;
char** ids; int id_count;
char** tag_names; // Parallel arrays: tag_names[i] has tag_values[i][]
char*** tag_values;
int* tag_value_counts;
int tag_filter_count;
long since; // 0 = not set
long until; // 0 = not set
int limit; // 0 = default 500
char* search; // NIP-50 search term, NULL = not set
} db_event_filter_t;
typedef struct db_result db_result_t;
db_result_t* db_query_events(const db_event_filter_t* filter);
const char* db_result_next_json(db_result_t* result); // Returns event_json, NULL when done
int db_result_row_count(db_result_t* result);
void db_result_free(db_result_t* result);
int db_count_events(const db_event_filter_t* filter);
// ============================================================
// Config Operations
// ============================================================
char* db_get_config(const char* key); // Caller must free, NULL if not found
int db_set_config(const char* key, const char* value, const char* data_type,
const char* description, const char* category, int requires_restart);
int db_update_config(const char* key, const char* value);
int db_config_exists(const char* key);
int db_get_config_count(void);
// ============================================================
// Auth Rules Operations
// ============================================================
int db_add_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
int db_remove_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
int db_auth_rule_exists(const char* rule_type, const char* pattern_type, const char* pattern_value);
int db_clear_auth_rules(const char* rule_type); // NULL = clear all
int db_is_blacklisted(const char* pubkey);
int db_is_whitelisted(const char* pubkey);
int db_has_any_whitelist(void);
// ============================================================
// Relay Key Storage
// ============================================================
int db_store_relay_private_key(const char* privkey_hex);
char* db_get_relay_private_key(void); // Caller must free
// ============================================================
// Subscription Logging
// ============================================================
int db_log_subscription_created(const char* sub_id, const char* wsi_ptr,
const char* client_ip, const char* filter_json);
int db_log_subscription_closed(const char* sub_id, const char* client_ip);
int db_log_subscription_disconnected(const char* client_ip);
int db_update_subscription_events_sent(const char* sub_id, int events_sent);
int db_cleanup_orphaned_subscriptions(void);
// ============================================================
// IP Ban Persistence
// ============================================================
int db_ensure_ip_ban_table(void);
int db_load_ip_bans(void* ban_table, int table_size); // Populates in-memory table
int db_save_ip_bans(const void* ban_table, int table_size);
// ============================================================
// Analytics / Stats - for dashboard
// ============================================================
cJSON* db_get_event_kind_distribution(void);
cJSON* db_get_time_based_stats(void);
cJSON* db_get_top_pubkeys(int limit);
cJSON* db_get_subscription_details(void);
int db_get_total_event_count(void);
// ============================================================
// Admin SQL Query
// ============================================================
cJSON* db_execute_admin_query(const char* sql, char* error_msg, size_t error_size);
#endif // DB_OPS_H
```
### Migration Strategy for Phase 1
The key principle: **change the interface, not the behavior**. Each function in `db_ops.c` initially just wraps the existing SQLite calls.
**Step-by-step for each file:**
1. **Create `db_ops.h` and `db_ops.c`** with the interface above
2. **Implement each `db_ops` function** by moving the existing SQLite code from the source files into `db_ops.c`
3. **Replace direct `sqlite3_*` calls** in each source file with `db_ops_*` calls
4. **Remove `extern sqlite3* g_db`** from each file — only `db_ops.c` knows about `g_db`
5. **Remove `#include <sqlite3.h>`** from each file — only `db_ops.c` includes it
6. **Update Makefile** to compile `db_ops.c`
### Order of Migration (by risk, lowest first)
1. **`src/ip_ban.c`** — Self-contained, simple CRUD. Good warmup.
2. **`src/subscriptions.c`** — Logging only, no critical path.
3. **`src/nip009.c`** — Event deletion, small file.
4. **`src/request_validator.c`** — Auth checks, small file.
5. **`src/api.c`** — Stats/monitoring queries. Larger but read-only.
6. **`src/dm_admin.c`** — Auth rules + WoT. Medium complexity.
7. **`src/config.c`** — Config CRUD. Large but well-structured.
8. **`src/websockets.c`** — COUNT queries, minimal DB usage.
9. **`src/main.c`** — Event store/retrieve, REQ queries. The big one — do last.
### Testing Phase 1
After Phase 1, the relay should behave **identically** to before. Run:
- `tests/run_all_tests.sh` — Full test suite
- `tests/performance_benchmarks.sh` — Verify no performance regression
- Manual testing with production-like traffic
---
## Phase 2: SQLite Thread Pool
### Goal
Add a pool of N worker threads, each with its own `sqlite3*` connection to the same database file. SQLite WAL mode (already enabled) supports **concurrent readers**. The event loop dispatches database work to the pool and remains responsive.
### Key Design Points
1. **Read path**: REQ queries dispatched to thread pool. Each worker opens its own `sqlite3*` connection. SQLite WAL allows unlimited concurrent readers.
2. **Write path**: EVENT inserts go through a single dedicated writer thread. SQLite only allows one writer at a time anyway — this serializes writes cleanly.
3. **Result delivery**: Worker threads cannot call `lws_write()` directly (libwebsockets is not thread-safe). Instead, they push results into a per-session message queue and call `lws_cancel_service()` to wake the event loop, which then drains the queue.
4. **Connection lifecycle**: Each thread opens its own connection with `PRAGMA journal_mode=WAL` and `PRAGMA busy_timeout=5000`.
### What Changes in the Codebase
| Component | Current | Thread Pool |
|-----------|---------|-------------|
| [`g_db`](../src/main.c:49) | Single global connection | One per thread + writer connection |
| [`handle_req_message()`](../src/main.c:1101) | Synchronous SQL in callback | Package filter into job, dispatch to pool |
| [`store_event()`](../src/main.c:787) | Synchronous INSERT in callback | Dispatch to writer thread |
| [`handle_count_message()`](../src/websockets.c:2863) | Synchronous COUNT in callback | Dispatch to pool |
| Result delivery | Direct `queue_message()` | Worker pushes to queue + `lws_cancel_service()` |
| Config reads | Direct `sqlite3_prepare` on `g_db` | Can stay synchronous with own connection or cache |
### New Files
- `src/thread_pool.h` — Thread pool interface
- `src/thread_pool.c` — Thread pool implementation (job queue, worker lifecycle, result delivery)
### Thread Pool Interface (sketch)
```c
// src/thread_pool.h
#ifndef THREAD_POOL_H
#define THREAD_POOL_H
typedef enum {
JOB_TYPE_REQ_QUERY,
JOB_TYPE_COUNT_QUERY,
JOB_TYPE_STORE_EVENT,
JOB_TYPE_DELETE_EVENT,
} job_type_t;
typedef struct {
job_type_t type;
void* session; // lws per-session data pointer
db_event_filter_t filter; // For REQ/COUNT jobs
cJSON* event; // For STORE jobs
char event_id[65]; // For DELETE jobs
} db_job_t;
int thread_pool_init(int num_readers, const char* db_path);
void thread_pool_shutdown(void);
int thread_pool_submit_read(db_job_t* job);
int thread_pool_submit_write(db_job_t* job);
#endif // THREAD_POOL_H
```
### Performance Characteristics
| Metric | Value |
|--------|-------|
| **Concurrent reads** | N readers in parallel (N = thread count, typically 48) |
| **Write throughput** | Same as current — SQLite serializes writes regardless |
| **Event loop latency** | Near-zero — REQ no longer blocks the loop |
| **Max theoretical read throughput** | ~48x current (limited by disk I/O, not CPU) |
| **Memory overhead** | ~50100 MB per connection (page cache) |
| **Latency per query** | Same as current per-query, but no head-of-line blocking |
### Limitations
- **Write contention**: SQLite still allows only ONE writer at a time. With WAL, readers don't block writers and writers don't block readers, but two simultaneous writes will serialize. At the current write rate (~56 events/hour), this is a non-issue.
- **Database size**: The 2.7 GB index bloat problem remains. Thread pool doesn't fix the schema — it fixes the concurrency.
- **Scaling ceiling**: Beyond ~8 reader threads, diminishing returns due to disk I/O contention on a single SQLite file.
- **Complexity**: Need a proper job queue, thread lifecycle management, and careful handling of the lws ↔ worker thread boundary.
---
## Risk Mitigation
| Risk | Mitigation |
|------|-----------|
| Phase 1 introduces bugs | Each file migrated independently, tested after each. Full test suite runs after each file. |
| Thread pool race conditions | Worker threads only touch their own `sqlite3*` connection. Result delivery uses a mutex-protected queue. `lws_cancel_service()` is documented as thread-safe. |
| Performance regression from abstraction | Abstraction layer is thin wrappers — no extra allocations or copies. Benchmark before/after. |
| lws thread safety issues | Workers never call `lws_write()` directly. They push to a queue and wake the event loop. This is the documented pattern for libwebsockets multi-threading. |
| Rollback needed | Phase 1 is a pure refactor — easy to revert. Phase 2 thread pool is additive — can be disabled with a compile flag. |
---
## Relationship to c-relay-pg-pg
After Phase 1 (abstraction layer) is complete, the codebase is ready to be forked into a separate **c-relay-pg-pg** project that replaces the SQLite backend with PostgreSQL. See [c-relay-pg-pg plan](c_relay_pg_pg_plan.md) for details.
The `db_ops.h` interface is designed to work with any backend:
- **SQLite** (this plan): `db_result_next_json()` copies from `sqlite3_column_text()`
- **PostgreSQL** (c-relay-pg-pg): `db_result_next_json()` returns from `PQgetvalue()`
- **LMDB** (future possibility): `db_result_next_json()` returns a zero-copy pointer into mmap'd memory
File diff suppressed because it is too large Load Diff
+186
View File
@@ -0,0 +1,186 @@
# Plan: Eliminate g_db from the Main Thread
## Problem Statement
All SQLite calls go through `db_ops.c`, but `db_active_connection()` falls back to `g_db` when `g_thread_db` is NULL. Since the main thread never sets `g_thread_db`, every `db_*` call from lws-main executes synchronous SQLite on the main thread. The thread pool workers have their own connections and work correctly — the problem is the **fallback path**.
Current perf data shows lws-main at **67.6% avg CPU**, dominated by SQLite symbols (`sqlite3BtreeTableMoveto`, `sqlite3VdbeExec`, `pcache1Fetch`).
## Goal
Remove `g_db` as a runtime connection used by the main thread. After this change:
- The main thread has **no SQLite connection** and cannot execute synchronous queries
- All DB work routes through thread pool workers (which have their own connections)
- `g_db` is only used during startup/shutdown (before/after the event loop)
- Any accidental `db_*` call from lws-main returns an error instead of silently blocking
## Architecture After Change
```
lws-main thread:
- WebSocket protocol handling
- Message parsing/routing
- Completion queue draining
- NO SQLite access during event loop
db-read-0 thread:
- REQ queries
- COUNT queries
- Config cache miss reads
- Auth rule checks
- Monitoring/stats queries
db-write thread:
- EVENT inserts
- Subscription logging
- Config updates
- Auth rule modifications
event-worker thread:
- Signature validation
- Duplicate check
- Store via db-write
```
## Categorized g_db Call Sites
### Category A: Startup-only — keep using g_db
These run before the event loop starts. They are fine.
| Function | File | Purpose |
|----------|------|---------|
| `db_init()` | db_ops.c:27 | Open database |
| `db_exec_sql()` for PRAGMAs | main.c:854-882 | WAL, mmap, cache setup |
| `db_table_exists()` | main.c:750 | Schema check |
| `db_get_schema_version_dup()` | main.c:754 | Migration check |
| `db_exec_sql()` for schema | main.c:842 | Create tables |
| `populate_all_config_values_atomic()` | config.c:4290 | First-time config |
| `populate_default_config_values()` | config.c:1657 | Default config |
| `add_pubkeys_to_config_table()` | config.c:1778 | Pubkey storage |
| `apply_cli_overrides_atomic()` | config.c:4211 | CLI overrides |
| `db_store_relay_private_key_hex()` | config.c:495 | Key storage |
| `db_populate_event_tags_from_existing()` | main.c:1156 | Tag migration |
| `cleanup_all_subscriptions_on_startup()` | subscriptions.c:1092 | Orphan cleanup |
### Category B: Shutdown-only — keep using g_db
| Function | File | Purpose |
|----------|------|---------|
| `db_exec_sql()` WAL checkpoint | main.c:900 | Clean shutdown |
| `db_close()` | main.c:904 | Close connection |
### Category C: Runtime hot path — must move off main thread
These are called during the event loop from lws-main context.
| Function | File | Called from | Fix strategy |
|----------|------|------------|--------------|
| `db_get_config_value_dup()` | db_ops.c:777 | config cache miss | Already cached with 5s TTL; pre-warm cache at startup so misses are rare |
| `store_event()` for kind 14/1059 | main.c:983 | websockets.c sync path | Route through async event worker |
| `store_event_post_actions()` | main.c | completion handler | Already runs on main; its DB calls need routing |
| `db_log_subscription_created()` | db_ops.c:145 | subscriptions.c | Fire-and-forget via write queue |
| `db_log_subscription_closed()` | db_ops.c:165 | subscriptions.c | Fire-and-forget via write queue |
| `db_log_subscription_disconnected()` | db_ops.c:193 | subscriptions.c | Fire-and-forget via write queue |
| `db_update_subscription_events_sent()` | db_ops.c:225 | subscriptions.c | Fire-and-forget via write queue |
| `db_cleanup_orphaned_subscriptions()` | db_ops.c:244 | subscriptions.c | Move to startup only |
| `generate_and_post_status_event()` | websockets.c:3408 | periodic timer | Submit to write worker |
| `ip_ban_cleanup()` / `ip_ban_log_stats()` | websockets.c:3196 | periodic timer | Submit to write worker |
| `db_get_total_event_count_ll()` | db_ops.c:589 | api.c monitoring | Submit to read worker |
| `db_get_event_count_since()` | db_ops.c:606 | api.c monitoring | Submit to read worker |
| `db_get_event_kind_distribution_rows()` | db_ops.c:625 | api.c monitoring | Submit to read worker |
| `db_get_top_pubkeys_rows()` | db_ops.c:663 | api.c monitoring | Submit to read worker |
| `db_get_subscription_details_rows()` | db_ops.c:697 | api.c monitoring | Submit to read worker |
| `db_get_all_config_rows()` | db_ops.c:745 | api.c config query | Submit to read worker |
| `db_execute_readonly_query_json()` | db_ops.c:442 | api.c SQL query | Submit to read worker |
| `db_event_id_exists()` | db_ops.c:1041 | main.c event check | Already moved to event worker |
| `db_retrieve_event_by_id()` | db_ops.c:1056 | main.c | Submit to read worker |
| `db_get_event_pubkey()` | db_ops.c:262 | NIP-09 deletion | Submit to read worker |
| `db_delete_event_by_id()` | db_ops.c:285 | NIP-09 deletion | Submit to write worker |
| `db_delete_older_replaceable_events()` | db_ops.c:305 | store_event_core | Already on write worker |
| `db_store_config_event()` | db_ops.c:888 | config.c | Submit to write worker |
| `db_add_auth_rule()` | db_ops.c:1226 | config.c admin | Submit to write worker |
| `db_remove_auth_rule()` | db_ops.c:1242 | config.c admin | Submit to write worker |
| `db_delete_wot_whitelist_rules()` | db_ops.c:1258 | config.c WoT | Submit to write worker |
| `db_count_wot_whitelist_rules()` | db_ops.c:1266 | config.c | Cache or read worker |
| `db_store_event_tags_cjson()` | db_ops.c:1140 | main.c post-actions | Already on write worker path |
| `db_get_config_row_count()` | db_ops.c:1121 | config.c diagnostics | Cache or read worker |
| `db_set_config_value_full()` | db_ops.c:796 | config.c | Submit to write worker |
| `db_update_config_value_only()` | db_ops.c:821 | config.c | Submit to write worker |
| `db_upsert_config_value()` | db_ops.c:838 | api.c | Submit to write worker |
| `db_exec_sql()` for transactions | config.c | admin events | Submit to write worker |
| `db_count_with_sql()` | db_ops.c:415 | config.c admin | Submit to read worker |
| `is_config_table_ready()` | config.c:4595 | config.c hybrid | Cache result at startup |
| `generate_config_event_from_table()` | config.c:4933 | config.c | Cache or read worker |
### Category D: Auth rule checks — already use separate connections
These functions in `db_ops.c` open their own temporary read-only connections:
| Function | Line | Notes |
|----------|------|-------|
| `db_is_pubkey_blacklisted()` | 347 | Opens own connection |
| `db_is_hash_blacklisted()` | 362 | Opens own connection |
| `db_is_pubkey_whitelisted()` | 377 | Opens own connection |
| `db_count_active_whitelist_rules()` | 392 | Opens own connection |
These are already safe — they don't use `g_db`. No change needed.
## Implementation Strategy
### Phase 1: Pre-warm caches and eliminate cache-miss DB reads
1. Pre-warm the config cache at startup by loading all config values into the in-memory cache before the event loop starts
2. Pre-warm the NIP-11 cache at startup
3. Pre-warm the auth rules fast cache at startup
4. This eliminates the most frequent cache-miss `db_get_config_value_dup()` calls
### Phase 2: Route subscription logging through write worker
1. Add a `THREAD_POOL_JOB_FIRE_AND_FORGET` job type to thread_pool
2. Create async wrappers: `db_log_subscription_created_async()`, etc.
3. These submit SQL to the write worker queue and return immediately
4. No completion callback needed — fire and forget
### Phase 3: Route special-kind EVENT store through async worker
1. Remove the `event_is_async_eligible()` exclusion for kinds 14, 1059, 23456
2. For kind 23456: process admin command in completion handler on main thread after store
3. For kind 14/1059: process NIP-17 DM in completion handler after store
### Phase 4: Route periodic timer DB work through workers
1. `generate_and_post_status_event()` — submit event creation to write worker
2. `ip_ban_cleanup()` / `ip_ban_log_stats()` — submit to write worker
3. Monitoring queries — submit to read worker with completion callback
### Phase 5: Null out g_db before event loop
1. After startup is complete and thread pool is initialized, set `g_db = NULL`
2. Change `db_active_connection()` to assert/warn if both `g_thread_db` and `g_db` are NULL
3. Any accidental main-thread DB call will now fail loudly instead of silently blocking
4. Restore `g_db` briefly for shutdown checkpoint
## Implementation Order
1. Phase 1 — lowest risk, immediate benefit from eliminating cache misses
2. Phase 2 — straightforward fire-and-forget pattern
3. Phase 3 — moderate complexity, needs careful completion handler design
4. Phase 4 — moderate complexity, periodic tasks need async patterns
5. Phase 5 — the final enforcement step, only safe after phases 1-4
## Risk Assessment
- **Phase 1**: Very low risk — just pre-warming existing caches
- **Phase 2**: Low risk — subscription logging is non-critical, fire-and-forget is safe
- **Phase 3**: Medium risk — admin/DM event processing has complex state; needs careful testing
- **Phase 4**: Medium risk — periodic tasks have side effects that need to complete
- **Phase 5**: High risk if done prematurely — must verify ALL runtime paths are covered first
## Expected Impact
After all phases:
- lws-main CPU should drop from ~67% to near 0% SQLite overhead
- All SQLite work happens on db-read and db-write threads
- Main thread only does WebSocket I/O, JSON parsing, and completion queue draining
- Thread model becomes: lws-main = pure I/O, workers = all DB
+194
View File
@@ -0,0 +1,194 @@
# Main Thread CPU Offload Plan
## Problem Statement
The main `c_relay_pg` thread consumes ~56% CPU while the DB worker threads (`db-read-1..4`, `db-write`) sit near 0%. All protocol handling, JSON parsing, event validation, subscription matching, and message queueing happens synchronously inside `nostr_relay_callback()` on the main libwebsockets event-loop thread.
## Current Architecture
```mermaid
flowchart TD
LWS[lws_service - main thread] --> CB[nostr_relay_callback]
CB --> PARSE[JSON parse - cJSON_Parse]
CB --> VALIDATE[Signature verify - nostr_validate_unified_request]
CB --> STORE[store_event - builds payload]
STORE --> SYNC_WRITE[thread_pool_execute_store_event_sync]
SYNC_WRITE --> |blocks main thread| DB_WRITE[db-write thread]
DB_WRITE --> |signal| SYNC_WRITE
CB --> REQ[handle_req_message - builds SQL]
REQ --> SYNC_READ[thread_pool_execute_req_sync]
SYNC_READ --> |blocks main thread| DB_READ[db-read-N thread]
DB_READ --> |signal| SYNC_READ
CB --> BROADCAST[broadcast_event_to_subscriptions]
BROADCAST --> QUEUE[queue_message per subscriber]
CB --> WRITEABLE[LWS_CALLBACK_SERVER_WRITEABLE]
WRITEABLE --> DRAIN[process_message_queue - lws_write]
```
### Why the main thread is hot
| Work item | Where | Cost |
|-----------|-------|------|
| JSON parsing | `cJSON_Parse` in `LWS_CALLBACK_RECEIVE` | Medium - per message |
| Signature verification | `nostr_validate_unified_request` - ed25519 crypto | **High** - per EVENT |
| SQL query building | `handle_req_message` filter-to-SQL loop | Low-Medium |
| Sync DB wait | `thread_pool_execute_*_sync` - pthread_cond_wait | Blocks but yields CPU |
| Result iteration + expiration check | Row loop in `handle_req_message` with `cJSON_Parse` per row | Medium-High for large result sets |
| Subscription matching | `broadcast_event_to_subscriptions` - filter matching | Medium - scales with subscriber count |
| Message serialization + queueing | `snprintf` + `queue_message_take_ownership` per subscriber | Medium |
| Config lookups | `get_config_int/bool` called repeatedly in hot paths | Low but frequent |
### Key constraint: libwebsockets is single-threaded
libwebsockets requires that **all `lws_write`, `lws_callback_on_writable`, and `lws_close_reason` calls happen from the service thread** (the thread running `lws_service`). This means we cannot directly write to WebSocket connections from worker threads. However, we **can** do computation on worker threads and post results back to the main thread for I/O.
## Offload Strategy
### Phase 1: Async EVENT Processing (Highest Impact)
Convert EVENT handling from synchronous to async. Currently the main thread does: parse → validate → store → broadcast, all blocking. Instead:
1. **Main thread**: Parse JSON (fast), extract event ID for dedup check, then submit a job to a new "event processing" worker thread
2. **Worker thread**: Signature verification (expensive crypto), store_event (already goes to DB thread), prepare broadcast payload
3. **Main thread callback**: Receive result via `wake_loop_cb` + `lws_cancel_service`, send OK response and broadcast to subscribers
```mermaid
flowchart TD
LWS[lws_service - main thread] --> RECEIVE[LWS_CALLBACK_RECEIVE]
RECEIVE --> PARSE[JSON parse + dedup check]
PARSE --> SUBMIT[Submit to event-worker queue]
SUBMIT --> LWS
WORKER[event-worker thread] --> VERIFY[Signature verification]
VERIFY --> STORE_DB[store_event via DB pool]
STORE_DB --> PREP[Prepare broadcast payload]
PREP --> RESULT_Q[Push result to completion queue]
RESULT_Q --> WAKE[lws_cancel_service]
LWS2[lws_service wakes] --> POLL[Poll completion queue]
POLL --> OK[Send OK response via queue_message]
POLL --> BCAST[broadcast_event_to_subscriptions]
```
**What this offloads**: ed25519 signature verification (~the most expensive per-event operation), event classification, tag serialization, and the synchronous DB store wait.
**What stays on main thread**: JSON parse (needed to extract event ID for dedup), OK response writing, broadcast fan-out (requires lws access).
### Phase 2: Async REQ Query Execution (Medium Impact)
Convert REQ handling from sync to async:
1. **Main thread**: Parse filters, build SQL, create subscription, submit query job
2. **DB reader thread**: Execute query (already happens, but currently blocks main thread via `_sync`)
3. **Main thread callback**: Iterate results, queue EVENT messages, send EOSE
This is simpler than Phase 1 because the thread pool already supports async submission via `thread_pool_submit_read` with a `result_cb`. The `_sync` wrappers just add a condvar wait on top. We need to:
- Use `thread_pool_submit_read` directly instead of `thread_pool_execute_req_sync`
- Store pending REQ context (sub_id, wsi, pss) so the callback can complete the work
- In the result callback, push results to a completion queue and call `lws_cancel_service`
- On the main thread, drain the completion queue and send EVENT + EOSE messages
### Phase 3: Reduce Per-Row Overhead in REQ Results (Low-Medium Impact)
Currently each row from a REQ query gets `cJSON_Parse` just to check NIP-40 expiration. This is wasteful:
- Option A: Add an `expiration` column to the events table so expiration filtering can be done in SQL
- Option B: Store expiration timestamp in a fast-parse format (extract during INSERT, store as integer column)
- Option C: Use string search on the raw JSON for the expiration tag instead of full parse
### Phase 4: Config Value Caching (Low Impact, Easy Win)
`get_config_int` and `get_config_bool` are called on every message in hot paths. These do SQLite queries. Cache config values in memory with a TTL or invalidation signal, so the main loop only refreshes them periodically (already done for `debug_level` every 60s — extend to all hot-path config values).
## Implementation Priority
| Phase | Impact | Risk | Complexity |
|-------|--------|------|------------|
| Phase 1: Async EVENT | **High** - removes crypto from main thread | Medium - async state management | Medium-High |
| Phase 2: Async REQ | **Medium** - unblocks main thread during queries | Low - infrastructure exists | Medium |
| Phase 3: Expiration optimization | **Low-Medium** - reduces per-row parse cost | Low | Low |
| Phase 4: Config caching | **Low** - reduces DB round-trips | Very Low | Low |
## Detailed Design: Phase 1 (Async EVENT Processing)
### New Components
#### Completion Queue (`src/completion_queue.h/.c`)
A thread-safe FIFO queue for posting results from worker threads back to the main thread:
```c
typedef struct {
int type; // COMPLETION_TYPE_EVENT_RESULT, COMPLETION_TYPE_REQ_RESULT, etc.
void* data; // Type-specific result data
struct lws* wsi; // Target WebSocket connection
void* pss; // Per-session data
} completion_item_t;
int completion_queue_init(void);
int completion_queue_push(completion_item_t* item);
completion_item_t* completion_queue_pop(void); // Non-blocking
void completion_queue_shutdown(void);
```
#### Event Worker Thread
A dedicated pthread that processes EVENT validation/storage:
```c
typedef struct {
cJSON* event; // Parsed event JSON - ownership transferred
cJSON* full_message; // Full message JSON for context
struct lws* wsi;
void* pss;
char sub_id[64]; // For response routing
} event_work_item_t;
```
#### Main Loop Integration
Add a completion queue drain step to the main event loop. After `lws_service` returns (either from timeout or `lws_cancel_service` wake), check the completion queue:
```c
while (g_server_running && !g_shutdown_flag) {
int result = lws_service(ws_context, 1000);
// NEW: Drain completion queue
completion_item_t* item;
while ((item = completion_queue_pop()) != NULL) {
process_completion(item); // Send OK, broadcast, etc.
free(item);
}
// ... existing periodic checks ...
}
```
### Changes to Existing Code
1. **`nostr_relay_callback` EVENT path**: After JSON parse and dedup check, instead of calling `nostr_validate_unified_request` + `store_event` + `broadcast_event_to_subscriptions` synchronously, submit an `event_work_item_t` to the event worker queue and return 0 immediately.
2. **`store_event`**: No changes needed — it already uses `thread_pool_execute_store_event_sync` which will run on the DB writer thread. The event worker thread will call it.
3. **`broadcast_event_to_subscriptions`**: No changes needed — it will be called from the main thread when processing the completion item, which is correct since it calls `queue_message_take_ownership` (requires lws thread).
### Thread Safety Considerations
- The `cJSON* event` object must be fully owned by the worker thread during processing. The main thread must not access it after submission.
- The `wsi` and `pss` pointers could become invalid if the client disconnects while the event is being processed. The completion handler must validate that the connection is still alive before sending the OK response.
- A generation counter or epoch on `pss` can detect stale references.
## What Cannot Be Offloaded
- **`lws_write` / `queue_message`**: Must happen on the lws service thread
- **`lws_callback_on_writable`**: Must happen on the lws service thread
- **`lws_close_reason`**: Must happen on the lws service thread
- **Subscription list iteration for broadcast**: Accesses `lws_wsi_user` which is lws-internal
These are fundamental libwebsockets constraints. The pattern is always: do computation off-thread, post result to completion queue, wake main thread, do I/O on main thread.
## Expected Impact
With Phase 1 alone, the main thread would no longer perform:
- ed25519 signature verification (~100-500μs per event depending on CPU)
- Synchronous DB store wait (~50-200μs per event)
- Event classification, tag serialization, JSON serialization for storage
This should reduce main-thread CPU by roughly 30-50% for EVENT-heavy workloads, shifting that work to the event worker thread and DB threads.
+207
View File
@@ -0,0 +1,207 @@
# C-Relay-PG Memory Leak Investigation & Fix Plan
## Problem Statement
c-relay-pg reached **1.56 GB** of its 1.5 GB `MemoryHigh` cgroup limit after only **1 hour 37 minutes** of runtime. The kernel is actively throttling the process with swap pressure (1.8M swap used). This strongly indicates one or more memory leaks causing unbounded growth.
## Root Cause Analysis
After thorough code review, I identified **three categories** of memory issues:
1. **Confirmed `get_config_value()` leaks** — the most pervasive issue
2. **Potential structural leaks** in subscription/connection lifecycle
3. **Stack pressure** from large stack-allocated arrays
---
## Category 1: `get_config_value()` Leaks (HIGH SEVERITY — Most Likely Root Cause)
### The Core Problem
[`get_config_value()`](src/config.c:293) calls [`get_config_value_from_table()`](src/config.c:1690) which returns `strdup(value)` — a **heap-allocated string that the caller must free**. Many call sites treat the return value as a borrowed pointer and never free it.
**Every unfree'd call leaks ~64-256 bytes per invocation.** On a busy relay processing hundreds of events/second, this accumulates to GB-scale leaks within hours.
### Confirmed Leak Sites
#### `websockets.c` — Called on EVERY incoming event
| Line | Code | Frequency | Severity |
|------|------|-----------|----------|
| [1527](src/websockets.c:1527) | `get_config_value("relay_pubkey")` — auth bypass check for kind 14 DMs | Every kind-14 event | **HIGH** |
| [1549](src/websockets.c:1549) | `get_config_value("admin_pubkey")` — auth bypass check for kind 23456 | Every kind-23456 event | **MEDIUM** |
| [2704](src/websockets.c:2704) | `get_config_value("relay_pubkey")` — DM stats command processing | Every DM to relay | **MEDIUM** |
| [2742](src/websockets.c:2742) | `get_config_value("admin_pubkey")` — DM admin check | Every DM to relay | **MEDIUM** |
#### `main.c` — Called on EVERY admin event check
| Line | Code | Frequency | Severity |
|------|------|-----------|----------|
| [1718](src/main.c:1718) | `get_config_value("relay_pubkey")` — inside a loop iterating tags | Every admin event, per tag | **CRITICAL** |
| [1742](src/main.c:1742) | `get_config_value("admin_pubkey")` — admin authorization | Every admin event | **HIGH** |
#### `config.c` — Called on EVERY event kind check
| Line | Code | Frequency | Severity |
|------|------|-----------|----------|
| [377](src/config.c:377) | `get_config_value("nip42_auth_required_kinds")` — NIP-42 kind check | Every incoming event | **CRITICAL** |
#### `ip_ban.c` — Called on EVERY ban check
| Line | Code | Frequency | Severity |
|------|------|-----------|----------|
| [255](src/ip_ban.c:255) | `get_config_value("idle_ban_whitelist")` — whitelist check | Every connection check | **HIGH** |
#### `nip042.c` — Called on every auth verification
| Line | Code | Frequency | Severity |
|------|------|-----------|----------|
| [99](src/nip042.c:99) | `get_config_value("relay_url")` — relay URL for auth verification | Every NIP-42 auth | **MEDIUM** |
#### `request_validator.c` — Called on every auth rules check
| Line | Code | Frequency | Severity |
|------|------|-----------|----------|
| [208](src/request_validator.c:208) | `get_config_value("auth_enabled")` — properly freed | N/A | OK |
| [216](src/request_validator.c:216) | `get_config_value("auth_rules_enabled")` — properly freed | N/A | OK |
| [304](src/request_validator.c:304) | `get_config_value("admin_pubkey")` — NOT freed | Every event validation | **HIGH** |
| [320](src/request_validator.c:320) | `get_config_value("nip42_auth_enabled")` — NOT freed | Every event validation | **HIGH** |
### Files That DO Free Correctly (for reference)
- [`nip011.c`](src/nip011.c:127) — Properly frees all `get_config_value()` results
- [`nip013.c`](src/nip013.c:43) — Properly frees `pow_mode`
- [`request_validator.c`](src/request_validator.c:191) — Properly frees `nip42_timeout` and `nip42_tolerance`
### Estimated Impact
On a relay processing ~100 events/second:
- `is_nip42_auth_required_for_kind()` leaks ~100-200 bytes × 100/sec = **~10-20 KB/sec**
- `ip_is_whitelisted()` leaks ~100 bytes × connections/sec
- `is_authorized_admin_event()` leaks inside tag loop — potentially **multiple leaks per event**
- Combined: **~50-100 KB/sec → ~180-360 MB/hour** — consistent with the observed 1.56 GB in 97 minutes
---
## Category 2: Structural Lifecycle Leaks (MEDIUM SEVERITY)
### 2a. `LWS_CALLBACK_CLOSED` — Inactive Subscription Skip
In [`websockets.c:2339`](src/websockets.c:2339), the cleanup handler only collects subscription IDs where `sub->active` is true:
```c
if (sub->active) { // Only process active subscriptions
temp_sub_id_t* temp = malloc(sizeof(temp_sub_id_t));
```
If a subscription was marked inactive by another thread between the time it was added and the connection close, it will be **skipped** — never removed from the global manager, never freed. The subscription object, its filters, and all cJSON objects within leak permanently.
### 2b. `connection_list_remove` Potential
The `connection_list_remove(wsi)` call at [`websockets.c:2252`](src/websockets.c:2252) happens before pss cleanup. Need to verify the connection list implementation doesn't hold references that prevent cleanup.
### 2c. `ip_connection_info_t` Leak on Disconnect
The per-IP connection tracking in [`subscriptions.h:79`](src/subscriptions.h:79) creates `ip_connection_info_t` nodes via `calloc`. These are only removed by `remove_ip_connection()` — need to verify this is called on every disconnect path.
---
## Category 3: Stack Pressure (LOW SEVERITY but notable)
### 3a. `candidates_to_check` Stack Array
In [`subscriptions.c:810`](src/subscriptions.c:810):
```c
subscription_t* candidates_to_check[MAX_TOTAL_SUBSCRIPTIONS]; // 5000 × 8 bytes = 40 KB
```
This allocates **40 KB on the stack** for every `broadcast_event_to_subscriptions()` call. While not a heap leak, it contributes to high memory pressure under concurrent load.
### 3b. `added_kinds` Bitmap
In [`subscriptions.c:68`](src/subscriptions.c:68):
```c
unsigned char added_kinds[8192] = {0}; // 8 KB on stack
```
---
## Fix Implementation Plan
### Phase 1: Fix `get_config_value()` Leaks (Highest Impact)
**Strategy A — Preferred: Add a config cache layer**
Instead of fixing 20+ call sites, add a **cached config system** that reads values once and caches them in memory, invalidating on config change events. This eliminates both the leak AND the repeated SQLite queries per event.
```
Config Cache Architecture:
- Static hash table of key-value pairs
- Populated at startup and on config change events
- get_config_value_cached() returns const pointer to cached value (no allocation)
- Existing get_config_value() kept for dynamic/rare lookups
```
**Strategy B — Quick fix: Free at every call site**
Add `free((char*)result)` after every `get_config_value()` call that doesn't already free. This is more error-prone but faster to implement.
**Recommendation: Implement Strategy A for hot-path values (relay_pubkey, admin_pubkey, auth settings), Strategy B for cold-path values.**
### Phase 2: Fix Structural Leaks
1. **Fix inactive subscription skip in CLOSED handler** — Remove the `if (sub->active)` guard, or add a second pass that also collects inactive subscriptions for cleanup
2. **Verify `remove_ip_connection()` is called on all disconnect paths**
3. **Add defensive cleanup** — periodic sweep of orphaned subscriptions
### Phase 3: Add Memory Monitoring
1. **Add a `/stats` endpoint** that reports:
- Current RSS/VSZ from `/proc/self/status`
- Active subscription count
- Message queue depth across all sessions
- Config cache hit/miss ratio
2. **Add periodic memory logging** to the service loop
3. **Consider integrating jemalloc** for better allocation tracking in production
### Phase 4: Reduce Stack Pressure
1. Move `candidates_to_check` to heap allocation with `malloc`/`free`
2. Consider reducing `MAX_TOTAL_SUBSCRIPTIONS` or using a dynamic array
---
## Verification Strategy
1. **Build with AddressSanitizer** (`-fsanitize=address`) for development testing
2. **Run under Valgrind** with `--leak-check=full` for comprehensive leak detection
3. **Monitor RSS over time** after fixes — should plateau rather than grow linearly
4. **Load test** with `tests/load_tests.sh` while monitoring memory
---
## Immediate Mitigation
While fixes are being implemented:
1. **Raise `MemoryHigh`** to 2 GB in the systemd unit to prevent throttling
2. **Add a cron job** to restart the service every 12-24 hours
3. **Monitor with**: `watch -n 5 'cat /proc/$(pgrep c_relay_pg)/status | grep -E "VmRSS|VmSwap"'`
---
## Implementation Priority Order
| Priority | Task | Impact |
|----------|------|--------|
| P0 | Fix `is_nip42_auth_required_for_kind()` leak in config.c:377 | Called on every event |
| P0 | Fix `ip_is_whitelisted()` leak in ip_ban.c:255 | Called on every connection check |
| P0 | Fix `is_authorized_admin_event()` leaks in main.c:1718,1742 | Called per admin event, leaks in loop |
| P1 | Fix websockets.c:1527,1549 auth bypass leaks | Called on every event with auth |
| P1 | Fix request_validator.c:304,320 leaks | Called on every event validation |
| P1 | Fix nip042.c:99 relay_url leak | Called on every NIP-42 auth |
| P2 | Implement config cache for hot-path values | Eliminates leak class entirely |
| P2 | Fix inactive subscription cleanup in CLOSED handler | Prevents slow subscription leak |
| P3 | Add memory monitoring endpoint | Ongoing visibility |
| P3 | Move large stack arrays to heap | Reduces stack pressure |
+410
View File
@@ -0,0 +1,410 @@
# C-Relay-PG: PostgreSQL Implementation Plan
## Current State Assessment
### What We Have
- **PostgreSQL 18.3** installed and running on localhost:5432
- **libpq development headers** available via pkg-config (`/usr/include/postgresql`, `/usr/lib/x86_64-linux-gnu`)
- **`db_ops.h`** abstraction layer with 40+ functions already defined
- **`db_ops.c`** — single 1,398-line file implementing all functions against SQLite
- **`ip_ban.c`** — already uses `db_ops` API (not raw SQLite) for persistence
- **`config.c`** — uses `db_ops` API for all database access
### SQLite Coupling Points (Must Fix)
| Location | Issue |
|----------|-------|
| `src/main.c:11` | `#include <sqlite3.h>` |
| `src/main.c:51` | `sqlite3* g_db = NULL;` — global handle owned by main.c |
| `src/main.c:897` | `if (g_db)` — direct handle check |
| `src/main.c:2228` | `if (!g_db)` — direct handle check |
| `src/main.c:2358` | `g_db = NULL;` — strict mode nullification |
| `src/main.c:743` | Log message references `sqlite3_open()` |
| `src/main.c:854-885` | SQLite PRAGMAs (WAL, mmap_size, cache_size) |
| `src/main.c:795-824` | SQLite-specific DDL in migration code |
| `src/main.c:900` | `PRAGMA wal_checkpoint(TRUNCATE)` |
| `src/config.c:4727` | `sqlite_master` query |
| `src/config.c:4388` | `INSERT OR REPLACE` SQL |
| `src/db_ops.c:13` | `extern sqlite3* g_db;` |
| `src/db_ops.c:871` | `strftime('%s', 'now')` — SQLite-specific |
| `src/db_ops.c:889,909,956` | `INSERT OR REPLACE` — SQLite-specific |
| `src/db_ops.c:1341` | `sqlite_master` — SQLite-specific |
| `src/db_ops.c:1387-1395` | `sqlite3_wal_checkpoint_v2()` — SQLite-only |
| `src/ip_ban.c:122,205` | `strftime('%s', 'now')` and `INSERT OR REPLACE` in SQL strings |
| `src/sql_schema.h` | Entire embedded schema is SQLite DDL |
| `src/subscriptions.c:1187-1197` | Commented-out raw SQLite code |
### What Already Works Through `db_ops`
- `ip_ban.c` — uses `db_exec_sql()`, `db_prepare()`, `db_step_stmt()`, etc.
- `config.c` — uses `db_get_config_value_dup()`, `db_set_config_value_full()`, etc.
- `subscriptions.c` — uses `db_log_subscription_*()` functions
- `api.c` — uses `db_execute_readonly_query_json()`, stat helpers
- `nip009.c` — uses `db_delete_event_by_id()`, `db_get_event_pubkey()`
---
## Implementation Phases
### Phase 0: PostgreSQL Setup & Smoke Test
Create the PostgreSQL role, database, and verify connectivity from C.
```
Tasks:
0.1 Create PostgreSQL role "crelay" with password
0.2 Create database "crelay" owned by role "crelay"
0.3 Write a minimal C test program that connects via libpq and runs SELECT 1
0.4 Verify the test compiles and links with: gcc test_pg.c -lpq -o test_pg
```
### Phase 1: Internalize g_db — Move Database Ownership Into db_ops
**Goal**: Eliminate `extern sqlite3* g_db` from main.c. The `db_ops` module must own its connection handle internally. This is the critical prerequisite — without it, swapping backends is impossible.
```mermaid
flowchart LR
subgraph Before
MAIN[main.c owns sqlite3* g_db] --> DBOPS[db_ops.c uses extern g_db]
end
subgraph After
MAIN2[main.c calls db_init/db_close] --> DBOPS2[db_ops.c owns static g_db internally]
end
```
```
Tasks:
1.1 Move `sqlite3* g_db` from main.c into db_ops.c as a static variable
1.2 Move `char g_database_path[512]` from config.c into db_ops.c as static
1.3 Remove `#include <sqlite3.h>` from main.c
1.4 Replace all `g_db` references in main.c with db_ops API calls:
- `if (g_db)``if (db_is_available())`
- `g_db = NULL``db_close()` (or new `db_detach_main()`)
1.5 Replace SQLite PRAGMA calls in main.c init_database() with new db_ops functions:
- `db_set_journal_mode_wal()`
- `db_set_mmap_size(long size)`
- `db_set_cache_size(int kb)`
1.6 Move schema migration logic from main.c into db_ops (new `db_apply_schema()`)
1.7 Replace `sqlite_master` query in config.c:4727 with `db_table_exists()`
1.8 Replace `INSERT OR REPLACE` in config.c:4388 with `db_prepare()`/`db_step_stmt()`
1.9 Build and run existing test suite to confirm no regressions
```
### Phase 2: Rename db_ops.c → db_ops_sqlite.c, Create Dispatcher
**Goal**: Split the single `db_ops.c` into a backend-specific file and a thin dispatcher, preparing the architecture for a second backend.
```
New file layout:
src/db_ops.h — unchanged public API
src/db_ops_sqlite.c — renamed from db_ops.c (all SQLite implementation)
src/db_ops_postgres.c — new file (stub initially)
src/db_ops.c — thin dispatcher that forwards to active backend
```
```
Tasks:
2.1 Rename src/db_ops.c → src/db_ops_sqlite.c
2.2 Prefix all function names in db_ops_sqlite.c with `sqlite_` (e.g., `sqlite_db_init()`)
2.3 Create src/db_ops.c dispatcher with compile-time backend selection:
- #ifdef DB_BACKEND_POSTGRES → call postgres_* functions
- #else → call sqlite_* functions (default)
2.4 Create src/db_ops_postgres.c with stub implementations that return DB_ERROR
2.5 Update Makefile with DB_BACKEND variable:
- `DB_BACKEND ?= sqlite`
- Conditional source file inclusion and -lpq linking
2.6 Build with DB_BACKEND=sqlite — verify identical behavior
2.7 Build with DB_BACKEND=postgres — verify it compiles (stubs return errors)
```
### Phase 3: PostgreSQL Schema
**Goal**: Create the PostgreSQL schema file and apply it to the database.
```
Tasks:
3.1 Create src/pg_schema.sql with PostgreSQL DDL from the plan:
- events table with JSONB tags column
- GIN index on tags
- config, auth_rules, relay_seckey, subscriptions, ip_bans tables
- Materialized views for dashboard
3.2 Create src/pg_schema.h — embedded schema as C string (like sql_schema.h)
3.3 Add `postgres_db_apply_schema()` function in db_ops_postgres.c
3.4 Apply schema to the crelay database and verify with psql
```
### Phase 4: Implement db_ops_postgres.c — Core Functions
**Goal**: Implement the PostgreSQL backend for all `db_ops.h` functions using libpq.
This is the largest phase. Functions are grouped by dependency order.
#### 4A: Connection Management
```
Tasks:
4A.1 Implement postgres_db_init() — PQconnectdb() with connection string
4A.2 Implement postgres_db_close() — PQfinish()
4A.3 Implement postgres_db_is_available() — PQstatus() check
4A.4 Implement postgres_db_last_error() — PQerrorMessage()
4A.5 Implement postgres_db_get_database_path() — return connection string
4A.6 Implement thread-local connection pool:
- postgres_db_set_thread_connection()
- postgres_db_clear_thread_connection()
- postgres_db_open_worker_connection() — new PQconnectdb per worker
- postgres_db_close_worker_connection() — PQfinish per worker
```
#### 4B: Statement API (db_stmt_t wrapper for PGresult)
```
Tasks:
4B.1 Define postgres db_stmt_t struct wrapping PGresult + cursor state
4B.2 Implement postgres_db_prepare() — PQprepare() with auto-generated name
4B.3 Implement postgres_db_bind_text_param/int_param/int64_param
Note: libpq uses $1,$2 params, not ?. Need parameter array approach.
4B.4 Implement postgres_db_step_stmt() — PQexecPrepared() + row iteration
4B.5 Implement postgres_db_column_text_value/int_value/int64_value/double_value
4B.6 Implement postgres_db_reset_stmt() and postgres_db_finalize_stmt()
4B.7 Implement postgres_db_exec_sql() — PQexec() for DDL/DML
```
**Key design decision**: libpq doesn't have a step-based cursor like SQLite. Two approaches:
1. **PQexecPrepared** returns all rows at once — iterate with row index
2. **DECLARE CURSOR / FETCH** for streaming large result sets
For this relay's workload (most queries return <1000 rows), approach 1 is simpler and sufficient.
#### 4C: Config & Key Storage
```
Tasks:
4C.1 Implement postgres_db_get_all_config_rows()
4C.2 Implement postgres_db_get_config_value_dup()
4C.3 Implement postgres_db_set_config_value_full()
— Use INSERT ... ON CONFLICT (key) DO UPDATE instead of INSERT OR REPLACE
4C.4 Implement postgres_db_update_config_value_only()
— Use EXTRACT(EPOCH FROM NOW()) instead of strftime('%s','now')
4C.5 Implement postgres_db_upsert_config_value()
4C.6 Implement postgres_db_store_relay_private_key_hex()
4C.7 Implement postgres_db_get_relay_private_key_hex_dup()
4C.8 Implement postgres_db_store_config_event()
4C.9 Implement postgres_db_get_config_row_count()
```
#### 4D: Event Storage & Retrieval
```
Tasks:
4D.1 Implement postgres_db_insert_event_with_json()
— INSERT ... ON CONFLICT (id) DO NOTHING
— Map extended_errcode to DB_CONSTRAINT for duplicates
4D.2 Implement postgres_db_event_id_exists()
4D.3 Implement postgres_db_retrieve_event_by_id()
4D.4 Implement postgres_db_get_event_pubkey()
4D.5 Implement postgres_db_get_event_time_bounds()
4D.6 Implement postgres_db_get_latest_event_pubkey_for_kind_dup()
```
#### 4E: Event Deletion (NIP-09)
```
Tasks:
4E.1 Implement postgres_db_delete_event_by_id()
4E.2 Implement postgres_db_delete_events_by_address()
```
#### 4F: Tag Operations
```
Tasks:
4F.1 Implement postgres_db_store_event_tags_cjson()
— Note: With JSONB+GIN, this may become a no-op for PostgreSQL
— Tags are already stored in the events.tags JSONB column
4F.2 Implement postgres_db_populate_event_tags_from_existing()
— No-op for PostgreSQL (no separate event_tags table needed)
```
#### 4G: Auth Rules
```
Tasks:
4G.1 Implement postgres_db_is_pubkey_blacklisted()
4G.2 Implement postgres_db_is_hash_blacklisted()
4G.3 Implement postgres_db_is_pubkey_whitelisted()
4G.4 Implement postgres_db_count_active_whitelist_rules()
4G.5 Implement postgres_db_add_auth_rule()
4G.6 Implement postgres_db_remove_auth_rule()
4G.7 Implement postgres_db_delete_wot_whitelist_rules()
4G.8 Implement postgres_db_count_wot_whitelist_rules()
```
#### 4H: Subscription Logging
```
Tasks:
4H.1 Implement postgres_db_log_subscription_created()
— Use INSERT ... ON CONFLICT DO UPDATE
4H.2 Implement postgres_db_log_subscription_closed()
— Use EXTRACT(EPOCH FROM NOW()) for timestamps
4H.3 Implement postgres_db_log_subscription_disconnected()
4H.4 Implement postgres_db_update_subscription_events_sent()
4H.5 Implement postgres_db_cleanup_orphaned_subscriptions()
```
#### 4I: Monitoring & Stats
```
Tasks:
4I.1 Implement postgres_db_get_total_event_count_ll()
4I.2 Implement postgres_db_get_event_count_since()
4I.3 Implement postgres_db_get_event_kind_distribution_rows()
4I.4 Implement postgres_db_get_top_pubkeys_rows()
4I.5 Implement postgres_db_get_subscription_details_rows()
4I.6 Implement postgres_db_count_with_sql()
4I.7 Implement postgres_db_execute_readonly_query_json()
— Map PG column types instead of SQLITE_INTEGER/SQLITE_TEXT/etc.
```
#### 4J: Schema & DDL Helpers
```
Tasks:
4J.1 Implement postgres_db_table_exists()
— Use information_schema.tables instead of sqlite_master
4J.2 Implement postgres_db_get_schema_version_dup()
4J.3 Implement postgres_db_wal_checkpoint_passive() — no-op for PostgreSQL
4J.4 Implement postgres_db_wal_checkpoint_truncate() — no-op for PostgreSQL
```
### Phase 5: Fix SQLite-Specific SQL in Callers
**Goal**: Ensure SQL strings passed through `db_prepare()` / `db_exec_sql()` from outside `db_ops` are portable.
```
Tasks:
5.1 Audit ip_ban.c SQL strings:
— Replace `strftime('%s', 'now')` with `EXTRACT(EPOCH FROM NOW())::BIGINT`
or pass timestamp as bind parameter from C (time(NULL))
— Replace `INSERT OR REPLACE` with `INSERT ... ON CONFLICT DO UPDATE`
5.2 Audit config.c SQL strings:
— Replace `sqlite_master` query with db_table_exists()
— Replace `INSERT OR REPLACE` with ON CONFLICT syntax
5.3 Audit main.c init_database():
— Move all PRAGMA calls behind db_ops backend-specific init
— Move schema migration behind db_apply_schema()
5.4 Strategy decision: use C-side timestamps (time(NULL)) as bind params
instead of database-side timestamp functions for portability
```
### Phase 6: CLI Connection String Support
**Goal**: Accept PostgreSQL connection parameters from the command line.
```
Tasks:
6.1 Add --db-host, --db-port, --db-name, --db-user, --db-password CLI options
6.2 Add --db-connstring option for full libpq connection string
6.3 Construct connection string from individual params if --db-connstring not given
6.4 Pass connection string to db_init() (works for both backends)
6.5 Update make_and_restart_relay.sh to support PostgreSQL connection params
6.6 Update AGENTS.md with new CLI options
```
### Phase 7: Integration Testing
**Goal**: Verify the PostgreSQL backend passes all existing tests.
```
Tasks:
7.1 Build with DB_BACKEND=postgres
7.2 Start relay with PostgreSQL connection string
7.3 Run tests/run_nip_tests.sh — verify NIP compliance
7.4 Run tests/run_all_tests.sh — verify full test suite
7.5 Run tests/performance_benchmarks.sh — compare with SQLite baseline
7.6 Test first-time startup flow (key generation, schema creation)
7.7 Test config event storage and retrieval
7.8 Test auth rules (whitelist/blacklist)
7.9 Test IP ban persistence across restarts
7.10 Test thread pool worker connections with PostgreSQL
```
### Phase 8: Query Builder — JSONB Tag Queries
**Goal**: Replace the `event_tags` JOIN queries with PostgreSQL JSONB containment queries for tag filtering in REQ handling.
```
Tasks:
8.1 Identify where REQ filter → SQL translation happens (websockets.c / subscriptions.c)
8.2 Create backend-aware query builder or use #ifdef for tag filter SQL:
— SQLite: `id IN (SELECT event_id FROM event_tags WHERE tag_name=? AND tag_value IN (?))`
— PostgreSQL: `tags @> '[["p", "value"]]'::jsonb`
8.3 Test tag-filtered REQ queries against PostgreSQL
8.4 Verify GIN index is being used (EXPLAIN ANALYZE)
```
---
## SQL Dialect Mapping Reference
| SQLite | PostgreSQL |
|--------|-----------|
| `INSERT OR REPLACE INTO t ...` | `INSERT INTO t ... ON CONFLICT (pk) DO UPDATE SET ...` |
| `strftime('%s', 'now')` | `EXTRACT(EPOCH FROM NOW())::BIGINT` |
| `SELECT 1 FROM sqlite_master WHERE type='table' AND name=?` | `SELECT 1 FROM information_schema.tables WHERE table_name=?` |
| `PRAGMA journal_mode=WAL` | N/A (PostgreSQL handles WAL internally) |
| `PRAGMA mmap_size=N` | N/A |
| `PRAGMA cache_size=-N` | `SET shared_buffers` (server-level, not per-connection) |
| `sqlite3_wal_checkpoint_v2()` | N/A (no-op) |
| `sqlite3_changes()` | `PQcmdTuples(result)` |
| `sqlite3_extended_errcode()` | `PQresultErrorField(result, PG_DIAG_SQLSTATE)` |
| `?` bind parameter | `$1`, `$2`, ... positional parameters |
| `JSON` column type | `JSONB` column type |
| `json_each()` for tag queries | `@>` containment operator with GIN index |
| `INTEGER PRIMARY KEY` auto-increment | `SERIAL` or `BIGSERIAL` |
---
## Architecture After Implementation
```mermaid
flowchart TD
subgraph Application Layer
MAIN[main.c] --> DBOPS_H[db_ops.h - public API]
CONFIG[config.c] --> DBOPS_H
IPBAN[ip_ban.c] --> DBOPS_H
SUBS[subscriptions.c] --> DBOPS_H
API[api.c] --> DBOPS_H
NIP09[nip009.c] --> DBOPS_H
WS[websockets.c] --> DBOPS_H
end
subgraph Database Abstraction
DBOPS_H --> DISPATCH[db_ops.c - dispatcher]
DISPATCH -->|DB_BACKEND_SQLITE| SQLITE[db_ops_sqlite.c]
DISPATCH -->|DB_BACKEND_POSTGRES| PG[db_ops_postgres.c]
end
subgraph Backends
SQLITE --> SQLITE3[libsqlite3]
PG --> LIBPQ[libpq]
LIBPQ --> PGSERVER[PostgreSQL Server]
end
```
---
## Risk Mitigation
| Risk | Mitigation |
|------|-----------|
| Breaking SQLite backend during refactor | Phase 1-2 are pure refactors — test suite must pass after each |
| libpq statement API mismatch | Use PQexecParams with row-index iteration, not cursor-based |
| SQL dialect differences in caller code | Phase 5 audits all SQL outside db_ops; use C timestamps for portability |
| Connection failures | Implement reconnection with exponential backoff in postgres_db_init |
| Thread safety | Each worker thread gets its own PGconn via db_open_worker_connection |
| Performance regression | Phase 7 benchmarks compare SQLite vs PostgreSQL on same workload |
---
## Build Commands
```bash
# SQLite backend (default, unchanged behavior)
make
# PostgreSQL backend
make DB_BACKEND=postgres
# Run with PostgreSQL
./build/c_relay_pg_x86 --db-connstring "host=localhost dbname=crelay user=crelay password=secret"
```
+257
View File
@@ -0,0 +1,257 @@
# PostgreSQL Nostr Relay Architecture Analysis
This document summarizes common approaches and best practices for building Nostr relays backed by PostgreSQL, based on industry-standard designs and existing open-source relay implementations.
## Identified Projects
* **Nostrss**: High-performance Rust-based relay.
* **Nostrgres**: Focused on PostgreSQL integration with complex event filtering.
* **Relay-rs (Postgres branch)**: General purpose high-throughput relay.
* **Nostream** ([github.com/Cameri/nostream](https://github.com/Cameri/nostream)): Production-ready TypeScript relay backed by PostgreSQL and Redis.
* **Nostr-Relay-NestJS** ([github.com/CodyTseng/nostr-relay-nestjs](https://github.com/CodyTseng/nostr-relay-nestjs)): High-performance relay built with NestJS and PostgreSQL, utilizing Kysely for type-safe database interactions.
## Detailed Analysis
### 1. Common Schema Patterns
* **Event Storage (JSONB)**: Most PostgreSQL-backed relays store the raw event as a JSONB object in a central `events` table. This provides flexibility for the evolving Nostr spec while allowing efficient extraction of fields.
* **Tag Denormalization**: While the raw event is in JSONB, performant relays extract tags (like `p`, `e`, `t`, `d`) into a separate `tags` table with foreign key relationships to the `events` table. This avoids slow JSONB traversal during complex filter queries.
* **Author/Publisher Tracking**: A dedicated `authors` or `pubkeys` table is typically used to maintain indexing on the `pubkey` field, enabling quick lookups of user activity.
### 2. Performance Optimization
* **GIN Indexes on JSONB**: Crucial for filtering on specific event tags or custom properties stored within the event object. GIN indexes with `jsonb_path_ops` are generally preferred for equality checks.
* **Time-Series Partitioning**: Given the append-only nature of Nostr, partitioning the `events` table by time (e.g., daily or weekly chunks) is highly recommended. This significantly improves query performance for recent data and simplifies data expiration/deletion.
* **Clustering**: Clustering the `events` table by the timestamp index can reduce disk I/O, as it keeps temporally related events physically adjacent on the disk.
### 3. Schema Management Approaches
* **Migrations**: Most mature relays utilize migration tools (like `Flyway`, `Diesel migrations`, or `Golang-migrate`) to version control database schema changes. This is critical for production stability.
* **Auto-generation**: Some lightweight prototypes auto-generate schemas at startup. This is generally discouraged for production due to the risk of destructive changes, data loss, or blocking DDL operations on large tables.
## Recommendations for Building a New Relay
1. **Prioritize Denormalization**: Do not rely solely on JSONB queries for high-traffic filters. Extract indexed tags into dedicated columns or tables.
2. **Use Partitioning from Day One**: Implementing native PostgreSQL partitioning (e.g., using `pg_partman`) is much easier before the dataset grows to millions of rows.
3. **Connection Pooling**: PostgreSQL requires aggressive connection management. Use a high-performance pooler like `PgBouncer` to handle the large number of concurrent, short-lived connections common in WebSocket-based relay traffic.
4. **Asynchronous Writes**: Decouple the WebSocket ingestion thread from the database writer thread to ensure that slow database writes do not impact the relay's responsiveness.
## Nostream ([github.com/Cameri/nostream](https://github.com/Cameri/nostream))
A production-ready Nostr relay written in TypeScript (Node.js), backed by PostgreSQL 14+ and Redis. Uses Knex.js for versioned migrations.
### PostgreSQL Schema
The schema is fully normalized — no JSONB blob for the whole event. Tags are stored separately and populated by a trigger.
**`events` table** — one row per event, tags kept as a JSONB column for the trigger to process:
```sql
CREATE TABLE events (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
event_id BYTEA UNIQUE NOT NULL, -- 32-byte hash
event_pubkey BYTEA NOT NULL,
event_kind INTEGER UNSIGNED NOT NULL,
event_created_at INTEGER UNSIGNED NOT NULL,
event_content TEXT NOT NULL,
event_tags JSONB, -- raw tags array, source of truth for trigger
event_signature BYTEA NOT NULL,
remote_address TEXT,
expires_at INTEGER,
deleted_at TIMESTAMP,
event_deduplication JSONB, -- used for replaceable event conflict key
first_seen TIMESTAMP DEFAULT now()
);
-- Indexes
CREATE INDEX ON events (event_pubkey);
CREATE INDEX ON events (event_kind);
CREATE INDEX ON events (event_created_at);
CREATE INDEX ON events (expires_at);
CREATE INDEX event_tags_idx ON events USING GIN (event_tags); -- GIN on raw JSONB
-- Unique index enforcing NIP-16 replaceable event semantics
CREATE UNIQUE INDEX replaceable_events_idx ON events (event_pubkey, event_kind)
WHERE event_kind = 0 OR event_kind = 3
OR (event_kind >= 10000 AND event_kind < 20000);
```
**`event_tags` table** — denormalized single-letter tags, populated by a PostgreSQL trigger on `events`:
```sql
CREATE TABLE event_tags (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
event_id BYTEA NOT NULL,
tag_name TEXT NOT NULL, -- single-letter only (e.g. 'e', 'p', 't')
tag_value TEXT NOT NULL
);
CREATE INDEX ON event_tags (event_id);
CREATE INDEX ON event_tags (tag_name, tag_value);
-- Trigger: on INSERT/UPDATE/DELETE to events, repopulate event_tags
CREATE OR REPLACE FUNCTION process_event_tags() RETURNS TRIGGER AS $$
DECLARE
tag_element jsonb;
BEGIN
DELETE FROM event_tags WHERE event_id = OLD.event_id;
IF TG_OP = 'INSERT' OR TG_OP = 'UPDATE' THEN
FOR tag_element IN SELECT jsonb_array_elements(NEW.event_tags) LOOP
IF length(trim('"' FROM (tag_element->0)::text)) = 1
AND (tag_element->1)::text IS NOT NULL THEN
INSERT INTO event_tags (event_id, tag_name, tag_value)
VALUES (NEW.event_id,
trim('"' FROM (tag_element->0)::text),
trim('"' FROM (tag_element->1)::text));
END IF;
END LOOP;
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER insert_event_tags
AFTER INSERT OR UPDATE OR DELETE ON events
FOR EACH ROW EXECUTE FUNCTION process_event_tags();
```
Tag queries (`#e`, `#p`, etc.) join `event_tags` rather than traversing the JSONB array.
**`users` and `invoices` tables** — for paid relay admission (Lightning payments):
```sql
CREATE TABLE users (
pubkey BYTEA PRIMARY KEY,
is_admitted BOOLEAN DEFAULT FALSE,
balance BIGINT DEFAULT 0, -- in msats
tos_accepted_at DATETIME
);
CREATE TABLE invoices (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
pubkey BYTEA NOT NULL,
bolt11 TEXT NOT NULL,
amount_requested BIGINT UNSIGNED NOT NULL,
amount_paid BIGINT UNSIGNED,
unit ENUM('msats','sats','btc'),
status ENUM('pending','completed','expired'),
description TEXT,
confirmed_at DATETIME,
expires_at DATETIME
);
```
### How Redis Is Used
Redis is **not** used for subscription fan-out. Subscription delivery is handled entirely in-process: each `WebSocketAdapter` holds a `Map<subscriptionId, filters[]>`, and a broadcast event walks all connected clients and filters locally.
Redis has two actual roles:
1. **Sliding-window rate limiting** — The `SlidingWindowRateLimiter` uses Redis sorted sets to track request timestamps per key (pubkey, IP, etc.) within a rolling time window:
- `ZREMRANGEBYSCORE key 0 (now - period)` — evict old entries
- `ZADD key timestamp member` — record this hit
- `ZRANGE key 0 -1` — count hits in window
- `EXPIRE key period` — auto-cleanup
- Keys follow the pattern `<pubkey>:events:<period>` or `<ip>:message:<period>`
2. **General-purpose cache** — A `RedisAdapter` wraps the client with `get`/`set`/`hasKey` helpers, used for caching admission checks (e.g. `<pubkey>:is-admitted`). The event handler has a `TODO` comment noting that the `userRepository.findByPubkey()` call should be replaced with a cache lookup — meaning this is partially implemented.
---
## Schema Definitions
### Relay-rs (Postgres)
```sql
-- Events table
CREATE TABLE "event" (
id bytea NOT NULL,
pub_key bytea NOT NULL,
created_at timestamp with time zone NOT NULL,
kind integer NOT NULL,
"content" bytea NOT NULL,
hidden bit(1) NOT NULL DEFAULT 0::bit(1),
delegated_by bytea NULL,
first_seen timestamp with time zone NOT NULL DEFAULT now(),
expires_at timestamp(0) with time zone,
CONSTRAINT event_pkey PRIMARY KEY (id)
);
CREATE INDEX event_created_at_idx ON "event" (created_at,kind);
CREATE INDEX event_pub_key_idx ON "event" (pub_key);
CREATE INDEX event_delegated_by_idx ON "event" (delegated_by);
CREATE INDEX event_expires_at_idx ON "event" (expires_at);
-- Tags table
CREATE TABLE "tag" (
id int8 NOT NULL GENERATED BY DEFAULT AS IDENTITY,
event_id bytea NOT NULL,
"name" varchar NOT NULL,
value bytea NULL,
value_hex bytea NULL,
CONSTRAINT tag_fk FOREIGN KEY (event_id) REFERENCES "event"(id) ON DELETE CASCADE,
CONSTRAINT unique_constraint_name UNIQUE (event_id, "name", value, value_hex)
);
CREATE INDEX tag_event_id_idx ON tag USING btree (event_id, name);
CREATE INDEX tag_value_idx ON tag USING btree (value);
CREATE INDEX tag_value_hex_idx ON tag USING btree (value_hex);
-- Account table
CREATE TABLE "account" (
pubkey varchar NOT NULL,
is_admitted BOOLEAN NOT NULL DEFAULT FALSE,
balance BIGINT NOT NULL DEFAULT 0,
tos_accepted_at TIMESTAMP,
CONSTRAINT account_pkey PRIMARY KEY (pubkey)
);
-- Invoice table
CREATE TYPE status AS ENUM ('Paid', 'Unpaid', 'Expired');
CREATE TABLE "invoice" (
payment_hash varchar NOT NULL,
pubkey varchar NOT NULL,
invoice varchar NOT NULL,
amount BIGINT NOT NULL,
status status NOT NULL DEFAULT 'Unpaid',
description varchar,
created_at timestamp,
confirmed_at timestamp,
CONSTRAINT invoice_payment_hash PRIMARY KEY (payment_hash),
CONSTRAINT invoice_pubkey_fkey FOREIGN KEY (pubkey) REFERENCES account (pubkey) ON DELETE CASCADE
);
```
### Nostr-Relay-NestJS (PostgreSQL via Kysely)
```sql
-- Events table
CREATE TABLE events (
id CHAR(64) PRIMARY KEY,
pubkey CHAR(64) NOT NULL,
created_at BIGINT NOT NULL,
kind INTEGER NOT NULL,
tags JSONB NOT NULL DEFAULT '[]',
generic_tags TEXT[] NOT NULL DEFAULT '{}',
content TEXT NOT NULL DEFAULT '',
sig CHAR(128) NOT NULL,
expired_at BIGINT,
d_tag_value TEXT,
delegator CHAR(64),
create_date TIMESTAMP NOT NULL DEFAULT now(),
update_date TIMESTAMP NOT NULL DEFAULT now(),
delete_date TIMESTAMP
);
-- Indices
CREATE EXTENSION IF NOT EXISTS btree_gin;
CREATE INDEX generic_tags_kind_idx ON events USING gin (generic_tags, kind);
CREATE INDEX pubkey_kind_created_at_idx ON events (pubkey, kind, created_at);
CREATE INDEX delegator_kind_created_at_idx ON events (delegator, kind, created_at);
CREATE INDEX created_at_kind_idx ON events (created_at, kind);
-- Generic tags table (for optimized tag queries)
CREATE TABLE generic_tags (
id SERIAL PRIMARY KEY,
tag TEXT NOT NULL,
author CHAR(64) NOT NULL,
kind INTEGER NOT NULL,
event_id CHAR(64) NOT NULL REFERENCES events(id),
created_at BIGINT NOT NULL
);
-- Indices for tag filtering
CREATE UNIQUE INDEX g_tag_event_id_idx ON generic_tags (tag, event_id);
CREATE INDEX g_tag_kind_created_at_idx ON generic_tags (tag, kind, created_at);
CREATE INDEX g_tag_created_at_idx ON generic_tags (tag, created_at);
```
Nostr-Relay-NestJS features a hybrid storage model where tags are stored both in a JSONB field (for general lookup) and a normalized `generic_tags` table (for optimized tag query performance). The schema uses Kysely for type-safe interaction.
+204
View File
@@ -0,0 +1,204 @@
# Remaining Implementation Plan — Zero SQLite on Main Thread + PG Fork Readiness
## Overview
This plan covers all remaining work to achieve two goals:
1. **Zero direct SQLite calls on lws-main** during the event loop
2. **Clean `db_ops` abstraction boundary** so the PG fork has no raw `sqlite3_*` calls outside `db_ops.c`
## Current State
### Completed (Fixes 1-5, 7 + Phases 1-3, 5)
- Global config cache with 5s TTL
- Dedup check moved to event worker
- Async COUNT queries via thread pool
- WoT auth rules cached per-session
- Special-kind EVENT store through async worker
- Subscription logging via fire-and-forget write queue
- IP ban periodic save via write queue
- g_db nulled before event loop
### Remaining
- Fix 6: Reduce reader threads (trivial)
- Fix 8: Move periodic tasks off main thread (partial — IP ban save done, monitoring/status not)
- Fix 9: Cache NIP-11 response
- Sync `store_event()` calls from admin/DM/NIP-09 paths
- Raw `sqlite3_*` calls in `thread_pool.c` and `websockets.c`
- COUNT query path in `websockets.c` still uses `json_each()` instead of `event_tags`
---
## Phase A: Move Remaining Periodic Tasks Off Main Thread
**Goal:** Eliminate the last regular SQLite calls from the lws-main event loop timer.
### A1: Move `generate_and_post_status_event()` to write worker
Currently called synchronously from the main event loop at `websockets.c:3455`.
**Changes:**
- Add `THREAD_POOL_JOB_STATUS_POST` to `thread_pool.h` job type enum
- Add executor in `thread_pool.c` that calls `generate_and_post_status_event()`
- In `websockets.c` main loop, replace direct call with `thread_pool_submit_write()` job
- The write worker has a DB connection, so `generate_stats_json()``store_event()``broadcast_event_to_subscriptions()` all work naturally
- Note: `broadcast_event_to_subscriptions()` is thread-safe (uses its own mutex)
**Files:** `thread_pool.h`, `thread_pool.c`, `websockets.c`
### A2: Move monitoring queries to read/write worker
`monitoring_on_event_stored()` and `monitoring_on_subscription_change()` in `api.c` call `generate_event_driven_monitoring()` / `generate_subscription_driven_monitoring()` which do DB queries and then broadcast ephemeral events.
**Changes:**
- Add `THREAD_POOL_JOB_MONITORING` job type
- Submit monitoring generation as a read job (it mostly does SELECTs)
- The completion broadcasts the ephemeral event via `lws_cancel_service()` + completion queue
- Alternative simpler approach: since these are throttled (default 5s) and only fire when kind 24567 subscribers exist, they could run on the write worker as fire-and-forget
**Files:** `thread_pool.h`, `thread_pool.c`, `api.c`, `websockets.c`
### A3: Move `ip_ban_cleanup()` and `ip_ban_log_stats()` to write worker
Currently called from the 60-second periodic timer at `websockets.c:3479-3480`.
**Changes:**
- Add `THREAD_POOL_JOB_IP_BAN_CLEANUP` job type (or reuse a generic callback job)
- Submit as fire-and-forget write job from the periodic timer
- `ip_ban_cleanup()` does in-memory cleanup + optional DB writes
- `ip_ban_log_stats()` is purely logging, no DB
**Files:** `thread_pool.h`, `thread_pool.c`, `ip_ban.c`, `websockets.c`
---
## Phase B: Route Remaining Sync `store_event()` Calls Through Async Worker
**Goal:** Eliminate all synchronous `store_event()` calls from lws-main context.
### Current sync `store_event()` call sites on main thread
| Call site | File | Context | Frequency |
|-----------|------|---------|-----------|
| Kind 1059 gift wrap store | `websockets.c:1795,1803,1819` | NIP-17 DM processing | Low |
| Kind 14 DM store | `websockets.c:1843` | NIP-17 DM processing | Low |
| Regular event fallback | `websockets.c:1862,1876` | Non-async-eligible events | Low |
| Kind 1059 (auth path) | `websockets.c:2594,2602,2618` | NIP-42 auth + DM | Low |
| Kind 14 (auth path) | `websockets.c:2642` | NIP-42 auth + DM | Low |
| Regular (auth path) | `websockets.c:2661,2675` | NIP-42 auth fallback | Low |
| Admin response store | `config.c:2585` | Admin API response | Very low |
| DM admin gift wrap | `dm_admin.c:389` | Admin DM response | Very low |
| NIP-09 deletion store | `nip009.c:135` | Deletion events | Low |
| Status event store | `api.c:595` | Periodic status post | Very low (handled by Phase A1) |
| Admin response event | `api.c:908` | Admin API | Very low |
| DM chat response | `api.c:1377` | Admin DM chat | Very low |
| Relay-generated event | `api.c:2350` | Relay metadata | Very low |
| DM stats response | `websockets.c:3648` | DM stats command | Very low |
### Strategy
Rather than converting each call site individually, create a **synchronous-to-async bridge**:
**Changes:**
- Create `store_event_async(cJSON* event, store_event_callback_t cb, void* ctx)` that submits the event to the write worker
- For call sites that need the result (e.g., to broadcast after store), use a completion callback
- For fire-and-forget sites (admin responses, DM responses), use NULL callback
- The existing `store_event()` wrapper becomes: submit to write worker + block on completion (for backward compat during transition)
- Eventually, all callers migrate to the async version
**Alternative simpler approach:** Since these are all low-frequency paths, and the write worker already has a DB connection, we can submit them as generic write jobs with a callback that does the store + broadcast. The main thread just needs to hand off the event JSON.
**Files:** `main.c`, `websockets.c`, `api.c`, `config.c`, `dm_admin.c`, `nip009.c`, `thread_pool.h`, `thread_pool.c`
---
## Phase C: `db_ops` Abstraction Cleanup for PG Fork
**Goal:** Ensure all raw `sqlite3_*` calls are inside `db_ops.c` only. External code uses only `db_ops.h` functions.
### C1: Add `db_open_worker_connection()` / `db_close_worker_connection()` to `db_ops.h`
Currently, `thread_pool.c:208` and `websockets.c:537` call `sqlite3_open_v2()` directly.
**Changes:**
- Add to `db_ops.h`:
```c
void* db_open_worker_connection(void); // Returns opaque connection handle
void db_close_worker_connection(void* conn);
```
- Implementation in `db_ops.c`: opens a new SQLite connection with WAL + busy timeout
- `thread_pool.c` worker init calls `db_open_worker_connection()` instead of raw `sqlite3_open_v2()`
- `websockets.c` event-worker calls `db_open_worker_connection()` instead of raw `sqlite3_open_v2()`
- Remove `#include <sqlite3.h>` from `thread_pool.c` and `websockets.c`
**Files:** `db_ops.h`, `db_ops.c`, `thread_pool.c`, `websockets.c`
### C2: Audit and remove remaining `sqlite3` includes outside `db_ops.c`
After C1, grep for any remaining `sqlite3` references outside `db_ops.c` and eliminate them.
**Files:** All `.c` files
### C3: Unify COUNT query tag path
The COUNT query builder in `websockets.c:3849` still uses `json_each(json(tags))` while the REQ query builder in `main.c:1604` uses the indexed `event_tags` table. Unify to use `event_tags` for consistency.
**Files:** `websockets.c`
---
## Phase D: Quick Wins
### D1: Fix 6 — Reduce reader threads from 4 to 1
Change default in `thread_pool_init()` or make configurable. With all reads going through the pool, a single reader is sufficient unless profiling shows otherwise.
**Files:** `thread_pool.c` or config default
### D2: Fix 9 — Cache NIP-11 response
Cache the serialized NIP-11 JSON string with a 60-second TTL. Invalidate on admin config change.
**Files:** `nip011.c`
---
## Implementation Order
| # | Task | Risk | Dependencies |
|---|------|------|-------------|
| 1 | A3: ip_ban_cleanup to write worker | Very Low | None |
| 2 | A1: status post to write worker | Low | None |
| 3 | A2: monitoring to worker | Low | None |
| 4 | D1: reduce reader threads to 1 | Very Low | None |
| 5 | D2: cache NIP-11 response | Very Low | None |
| 6 | C1: db_open/close_worker_connection | Low | None |
| 7 | C3: unify COUNT tag query path | Low | None |
| 8 | C2: audit/remove sqlite3 includes | Very Low | C1 |
| 9 | B: route sync store_event through async | Medium | A1 (status post already handled) |
## Expected Outcome
After all phases:
- **lws-main has zero SQLite calls** during the event loop
- **All `sqlite3_*` calls are inside `db_ops.c`** — clean abstraction boundary
- **PG fork can replace `db_ops.c`** without touching any other file
- Thread model is fully realized:
- `lws-main` = WebSocket I/O + JSON parse + subscription match + completion drain
- `event-worker` = signature verify + dedup + store submission
- `db-read` = REQ queries + COUNT queries + monitoring reads
- `db-write` = event INSERTs + subscription logging + periodic tasks + admin writes
```mermaid
flowchart LR
CLIENT[Nostr Clients] --> LWS[lws-main<br/>Zero SQLite<br/>WS I/O + JSON + Sub Match]
LWS -->|EVENT| EW[event-worker<br/>Verify + Dedup]
EW -->|store job| DW[db-write<br/>INSERT events<br/>Sub logging<br/>Periodic tasks<br/>Admin writes]
EW -->|completion| LWS
DW -->|completion| LWS
LWS -->|REQ/COUNT| DR[db-read<br/>SELECT queries<br/>Monitoring reads]
DR -->|completion| LWS
LWS -->|OK/EVENT/EOSE/COUNT| CLIENT
```
+379
View File
@@ -0,0 +1,379 @@
# Plan: Remove All SQLite from lws-main Thread
## Problem Statement
After implementing the 4 offload fixes (v2.1.4), `lws-main` still consumes **53.87% avg CPU** (down from 68.71%). Perf profiling shows **100% of the top symbols are SQLite** — B-tree traversal, page cache, VFS reads. The worker threads (`db-read`, `db-write`, `event-worker`) are nearly idle at 0.00.1%.
The goal is to eliminate all synchronous SQLite access from `lws-main` so it only does: JSON parsing, WebSocket I/O, subscription matching (in-memory), and message queuing.
## Thread Architecture Change
Reduce from 7 threads to 4:
```
Current: lws-main, db-read-1, db-read-2, db-read-3, db-read-4, db-write, event-worker
Proposed: lws-main, db-read, db-write, event-worker
```
The 4 reader threads are overkill — they average 0.070.10% CPU each. A single `db-read` thread is sufficient for the current workload. This can be made configurable later if needed.
## Remaining SQLite Calls on lws-main
Every synchronous SQLite call that currently runs on `lws-main`:
### Hot Path — Per-Event (every incoming EVENT message)
| Call | Location | Frequency | Cost |
|------|----------|-----------|------|
| `event_id_exists_in_db()` | websockets.c:1324, 2134 | Every EVENT | ~10-50us per B-tree lookup |
| `get_config_value("relay_pubkey")` | websockets.c:2050 | Every kind-14 EVENT | SQLite SELECT |
| `get_config_value("admin_pubkey")` | websockets.c:2073 | Every kind-23456 EVENT | SQLite SELECT |
### Hot Path — Per-Connection (every new WebSocket connection)
| Call | Location | Frequency | Cost |
|------|----------|-----------|------|
| `get_config_bool("trust_proxy_headers")` | websockets.c:1071 | Every connection | SQLite SELECT |
| `get_config_bool("nip42_auth_required_events")` | websockets.c:1134 | Every connection | SQLite SELECT |
| `get_config_bool("nip42_auth_required_subscriptions")` | websockets.c:1135 | Every connection | SQLite SELECT |
### Hot Path — Per-REQ (every subscription request)
| Call | Location | Frequency | Cost |
|------|----------|-----------|------|
| `get_config_bool("expiration_enabled")` | main.c:1424 | Every REQ | SQLite SELECT |
| `get_config_bool("expiration_filter")` | main.c:1425 | Every REQ | SQLite SELECT |
| `check_database_auth_rules()` | websockets.c:1706 | Every REQ when WoT enabled | SQLite SELECT |
### Medium Path — Per-COUNT (NIP-45)
| Call | Location | Frequency | Cost |
|------|----------|-----------|------|
| `thread_pool_execute_count_sync()` | websockets.c:3689 | Every COUNT message | Blocks main thread waiting for db-read |
### Low Path — Periodic (every 60s or on timer)
| Call | Location | Frequency | Cost |
|------|----------|-----------|------|
| `refresh_hot_config_if_needed()` | websockets.c:228-247 | Every 5s | 8 SQLite SELECTs |
| `generate_and_post_status_event()` | websockets.c:3213 | Every N hours | store_event + broadcast |
| `ip_ban_cleanup/log_stats` | websockets.c:3004-3005 | Every 60s | SQLite reads/writes |
### Per-EVENT — Validator Path (called from event-worker AND sync path)
| Call | Location | Frequency | Cost |
|------|----------|-----------|------|
| `get_config_value("admin_pubkey")` | request_validator.c:308 | Every kind-23456 EVENT | SQLite SELECT |
| `get_config_value("nip42_auth_enabled")` | request_validator.c:326 | Every EVENT | SQLite SELECT |
| `get_config_bool("pow_enabled")` | request_validator.c:370 | Every EVENT | SQLite SELECT |
| `get_config_int("pow_min_difficulty")` | request_validator.c:371 | Every EVENT | SQLite SELECT |
| `get_config_int("pow_validation_flags")` | request_validator.c:372 | Every EVENT | SQLite SELECT |
| `get_config_int("nip40_expiration_grace_period")` | request_validator.c:432 | Every EVENT with expiration | SQLite SELECT |
| `get_config_value("auth_enabled")` | request_validator.c:212 | Every EVENT (reload_auth_config) | SQLite SELECT |
| `get_config_value("auth_rules_enabled")` | request_validator.c:220 | Every EVENT (reload_auth_config) | SQLite SELECT |
| `db_count_active_whitelist_rules()` | request_validator.c:536 | Every 5s (cached) | Opens separate SQLite connection |
**Note:** These run on the **event-worker** thread for async-eligible events, but on **lws-main** for special kinds (14, 1059, 23456) that go through the sync validation path. They also run on lws-main for the `nostr_validate_unified_request()` call in the sync fallback.
### Per-EVENT — NIP-13 PoW Validation
| Call | Location | Frequency | Cost |
|------|----------|-----------|------|
| `get_config_bool("pow_enabled")` | nip013.c:27 | Every EVENT | SQLite SELECT |
| `get_config_int("pow_min_difficulty")` | nip013.c:28 | Every EVENT | SQLite SELECT |
| `get_config_value("pow_mode")` | nip013.c:29 | Every EVENT | SQLite SELECT |
### Per-Broadcast — NIP-40 Expiration Check
| Call | Location | Frequency | Cost |
|------|----------|-----------|------|
| `get_config_bool("expiration_enabled")` | subscriptions.c:764 | Every broadcast | SQLite SELECT |
| `get_config_bool("expiration_filter")` | subscriptions.c:765 | Every broadcast | SQLite SELECT |
### Per-NIP-11 Request (HTTP, not WebSocket)
| Call | Location | Frequency | Cost |
|------|----------|-----------|------|
| 15+ `get_config_value()` calls | nip011.c:96-110 | Every NIP-11 HTTP request | 15 SQLite SELECTs |
| 10+ `get_config_int()`/`get_config_bool()` calls | nip011.c:113-123 | Every NIP-11 HTTP request | 10 SQLite SELECTs |
### Per-NIP-42 Auth Event
| Call | Location | Frequency | Cost |
|------|----------|-----------|------|
| `get_config_value("relay_url")` | nip042.c:99 | Every AUTH event | SQLite SELECT |
| `get_config_int("relay_port")` | nip042.c:103 | Every AUTH event (fallback) | SQLite SELECT |
### Per-NIP-40 Expiration Check (EVENT submission)
| Call | Location | Frequency | Cost |
|------|----------|-----------|------|
| 5 `get_config_bool()`/`get_config_int()` calls | nip040.c:37-41 | Every EVENT with expiration tag | 5 SQLite SELECTs |
### Subscription Lifecycle (per sub create/close/disconnect)
| Call | Location | Frequency | Cost |
|------|----------|-----------|------|
| `db_log_subscription_created()` | subscriptions.c:1043 | Every REQ | SQLite INSERT |
| `db_log_subscription_closed()` | subscriptions.c:1053 | Every CLOSE | SQLite INSERT + UPDATE |
| `db_log_subscription_disconnected()` | subscriptions.c:1060 | Every disconnect | SQLite UPDATE + INSERT |
| `db_update_subscription_events_sent()` | subscriptions.c:1088 | Every broadcast match | SQLite UPDATE |
| `db_cleanup_orphaned_subscriptions()` | subscriptions.c:1100 | Startup only | SQLite DELETE |
### IP Ban System (per-connection and periodic)
| Call | Location | Frequency | Cost |
|------|----------|-----------|------|
| `get_config_value("idle_ban_whitelist")` | ip_ban.c:254 | Every ban check | SQLite SELECT |
| `get_config_bool("auth_fail_ban_enabled")` | ip_ban.c:301, 343 | Every ban check + failure | SQLite SELECT |
| `get_config_bool("idle_ban_enabled")` | ip_ban.c:307, 394 | Every ban check + idle failure | SQLite SELECT |
| 6+ `get_config_int()` calls | ip_ban.c:345-399, 464-465 | Every failure recording | SQLite SELECTs |
| `db_prepare()`/`db_step_stmt()` for ip_bans | ip_ban.c:141-238 | Load at startup + periodic save | SQLite reads/writes |
### Special Kind Sync Path (kinds 14, 1059, 23456)
| Call | Location | Frequency | Cost |
|------|----------|-----------|------|
| `store_event()` | websockets.c:1548-1629 | Every special-kind EVENT | Full sync store |
| `process_admin_event_in_config()` | websockets.c:1468 | Every kind-23456 | Config DB writes |
| `process_nip17_admin_message()` | websockets.c:1523 | Every kind-1059 when enabled | Decrypt + DB |
| `process_dm_stats_command()` | websockets.c:3240 | Admin DM stats | Decrypt + query + encrypt |
| `generate_stats_json()` | api.c:1167 (via dm_admin.c:805) | Admin stats command | Multiple SQLite queries |
### DM Admin Path (called from lws-main for kind 1059/14)
| Call | Location | Frequency | Cost |
|------|----------|-----------|------|
| `get_config_value("relay_pubkey")` | dm_admin.c:358, 451 | Every admin DM | SQLite SELECT |
| `get_config_value("admin_pubkey")` | dm_admin.c:504 | Every admin DM | SQLite SELECT |
| `get_config_int("nip59_timestamp_max_delay_sec")` | dm_admin.c:373 | Every admin DM | SQLite SELECT |
| `get_config_int("wot_enabled")` | dm_admin.c:613, 640, 660 | WoT commands | SQLite SELECT |
### API/Monitoring (called from lws-main periodic timer)
| Call | Location | Frequency | Cost |
|------|----------|-----------|------|
| `generate_stats_json()` | api.c:1167 | Status post + admin stats | ~10 SQLite queries |
| `query_time_based_statistics()` | api.c:102 | Monitoring events | SQLite queries |
| `query_subscription_details()` | api.c:197 | Monitoring events | SQLite queries |
| `get_config_value("relay_pubkey")` | api.c:419, 821, 1291 | Every monitoring event | SQLite SELECT |
| `get_config_int("kind_1_status_posts_hours")` | api.c:547 | Status post check | SQLite SELECT |
| `get_config_int("kind_24567_reporting_throttle_sec")` | api.c:60 | Monitoring throttle | SQLite SELECT |
## Implementation Plan
### Fix 1: Create Global Config Cache — Eliminate ALL get_config_* SQLite Calls
**Impact: Very High** — eliminates 50+ distinct `get_config_value/int/bool` call sites across 8 source files
The audit found `get_config_value/int/bool` calls in: `websockets.c`, `main.c`, `subscriptions.c`, `request_validator.c`, `nip011.c`, `nip013.c`, `nip040.c`, `nip042.c`, `ip_ban.c`, `dm_admin.c`, `api.c`, `config.c`. Every single one does a `SELECT value FROM config WHERE key=?` SQLite query.
**Approach:** Replace `get_config_value_from_table()` with an in-memory hash map that is loaded once at startup and refreshed periodically (every 5s) or on admin config change events.
**Changes:**
1. **New: `config_cache` in config.c** — A simple hash map or fixed array of key-value pairs loaded from the config table:
```c
typedef struct {
char key[64];
char value[512];
} config_cache_entry_t;
static config_cache_entry_t g_config_cache[256];
static int g_config_cache_count = 0;
static time_t g_config_cache_last_refresh = 0;
static pthread_mutex_t g_config_cache_mutex;
#define CONFIG_CACHE_TTL_SEC 5
```
2. **Modify `get_config_value_from_table()`** — Instead of querying SQLite, look up in the in-memory cache. If cache is stale, refresh from SQLite (but only once per TTL period, not per call).
3. **Add `config_cache_refresh()`** — Loads all rows from `SELECT key, value FROM config` into the hash map. Called:
- Once at startup after config table is populated
- Every 5 seconds from the existing `refresh_hot_config_if_needed()` timer
- Immediately after `process_admin_event_in_config()` modifies config
4. **Thread safety:** The cache is read from lws-main and event-worker threads. Use a read-write lock or double-buffer pattern. Since config changes are rare, a simple mutex with short hold time is fine.
5. **Remove the existing `hot_config_cache_t`** in websockets.c — it becomes redundant since the global config cache serves the same purpose with broader coverage.
This single change eliminates SQLite calls from:
- Per-connection: `trust_proxy_headers`, `nip42_auth_required_*` (websockets.c)
- Per-REQ: `expiration_enabled`, `expiration_filter` (main.c)
- Per-EVENT: `admin_pubkey`, `relay_pubkey`, `nip42_auth_enabled`, `pow_*` (request_validator.c, nip013.c)
- Per-broadcast: `expiration_enabled`, `expiration_filter` (subscriptions.c)
- Per-NIP-11: 25+ config values (nip011.c)
- Per-NIP-42: `relay_url`, `relay_port` (nip042.c)
- Per-NIP-40: 5 expiration config values (nip040.c)
- Per-IP-ban: `idle_ban_*`, `auth_fail_*` config values (ip_ban.c)
- Per-admin-DM: `relay_pubkey`, `admin_pubkey`, `wot_enabled` (dm_admin.c)
- Per-monitoring: `relay_pubkey`, throttle values (api.c)
### Fix 2: Move Duplicate Check to Event Worker Thread
**Impact: High** — eliminates the most frequent per-EVENT SQLite call from lws-main
Currently `event_id_exists_in_db()` runs on lws-main before async submission. Move it into the `async_event_worker_main()` function, which already has its own SQLite connection.
**Changes:**
- Remove `event_id_exists_in_db()` calls from both EVENT paths in websockets.c (lines 1324 and 2134)
- Add the duplicate check at the start of `async_event_worker_main()` before `nostr_validate_unified_request()`
- If duplicate found, set `completion->success = 1` and `completion->error_message = "duplicate: already have this event"` and skip crypto
- The completion handler already sends OK responses, so this works seamlessly
**Trade-off:** Without the early dedup check on lws-main, duplicate events will be submitted to the event worker queue and consume a queue slot + thread wakeup before being detected. This is acceptable because:
- The worker thread dedup check is still fast (B-tree index lookup)
- It avoids the much larger cost of running SQLite on lws-main for every event
- Queue depth is 4096, so even under heavy duplicate traffic this won't overflow
### Fix 3: Make COUNT Queries Async
**Impact: Medium** — eliminates blocking wait on lws-main for NIP-45 COUNT
Currently `handle_count_message()` calls `thread_pool_execute_count_sync()` which blocks lws-main waiting for the db-read worker. Convert to the same async pattern used for REQ queries.
**Changes:**
- Create `count_async_state_t` similar to `req_async_state_t`
- Create `count_async_completion_t` with count result
- Add `submit_count_query_async()` that uses `thread_pool_submit_read()` with a callback
- The callback pushes to a count completion queue
- Add `process_count_async_completions()` to drain the queue on lws-main and send COUNT responses
- Call it from the main event loop alongside `process_req_async_completions()`
### Fix 4: Cache Auth Rules Check Result
**Impact: Low-Medium** — eliminates per-REQ SQLite when WoT is enabled
`check_database_auth_rules()` at websockets.c:1706 runs a SQLite query for every REQ when `wot_enabled == 2`. Cache the result per-session.
**Changes:**
- Add `wot_checked` and `wot_allowed` fields to `per_session_data`
- On first REQ, call `check_database_auth_rules()` and cache the result
- On subsequent REQs from the same session, use the cached value
- Reset cache when auth state changes (e.g., after NIP-42 AUTH)
### Fix 5: Move Special-Kind EVENT Store to Event Worker
**Impact: Low** — special kinds are rare but currently do full sync store on lws-main
Kinds 14, 1059, and 23456 are currently excluded from async processing because they have special handling. However, the `store_event()` call within those paths still blocks lws-main.
**Changes:**
- For kind 1059 and kind 14: after special processing completes, submit the store to the event worker instead of calling `store_event()` synchronously
- For kind 23456: admin events are not stored (processed by admin API), so no change needed
- This requires the special processing (decryption, admin check) to still happen on lws-main (it needs `pss` context), but the final DB write goes async
### Fix 6: Reduce Reader Threads from 4 to 1
**Impact: Resource savings** — 3 fewer threads, 3 fewer SQLite connections
**Changes:**
- Change default `reader_threads` in `thread_pool_init()` from 4 to 1
- Or make it configurable via config with default 1
- The single `db-read` thread handles all REQ and COUNT queries sequentially
- If future profiling shows the read thread becoming a bottleneck, increase back to 2+
### Fix 7: Move Subscription Logging Off Main Thread
**Impact: Medium** — eliminates per-REQ INSERT, per-CLOSE UPDATE, per-broadcast UPDATE
The subscription lifecycle functions in subscriptions.c do synchronous SQLite writes on lws-main:
- `db_log_subscription_created()` — INSERT on every REQ
- `db_log_subscription_closed()` — INSERT + UPDATE on every CLOSE
- `db_log_subscription_disconnected()` — UPDATE + INSERT on every disconnect
- `db_update_subscription_events_sent()` — UPDATE on every broadcast match
**Changes:**
- Submit these as fire-and-forget write jobs to the db-write thread via `thread_pool_submit_write()`
- Create a new job type `THREAD_POOL_JOB_SUBSCRIPTION_LOG` or simply use a generic callback job
- The main thread doesn't need to wait for these to complete — they're purely observational logging
### Fix 8: Move Periodic Tasks Off Main Thread
**Impact: Low** — these run every 60s but can still cause latency spikes
- `generate_and_post_status_event()`: Submit as an async job to the event worker
- `ip_ban_cleanup()`/`ip_ban_log_stats()`/`ip_ban_save()`: These do SQLite reads/writes. Move to a periodic job on the db-write thread
- Config cache refresh (Fix 1) replaces the old `refresh_hot_config_if_needed()` SQLite queries
### Fix 9: Cache NIP-11 Response
**Impact: Low-Medium** — eliminates 25+ SQLite SELECTs per NIP-11 HTTP request
`generate_relay_info_json()` in nip011.c calls `get_config_value()` 15 times and `get_config_int()`/`get_config_bool()` 10 times. After Fix 1 (global config cache), these become in-memory lookups. But we can go further:
**Changes:**
- Cache the serialized NIP-11 JSON response string with a TTL (e.g., 60 seconds)
- On NIP-11 request, return the cached string directly without rebuilding
- Invalidate on admin config change events
## Implementation Order
| # | Fix | Risk | Complexity |
|---|-----|------|------------|
| 1 | Global config cache (replaces all get_config_* SQLite) | Low | Medium |
| 2 | Move dedup check to event worker | Low | Low |
| 3 | Async COUNT queries | Low | Medium |
| 4 | Cache WoT auth rules per-session | Very Low | Low |
| 5 | Async store for special kinds | Medium | Medium |
| 6 | Reduce reader threads to 1 | Very Low | Trivial |
| 7 | Move subscription logging off main thread | Low | Low-Medium |
| 8 | Move periodic tasks off main thread | Low | Medium |
| 9 | Cache NIP-11 response | Very Low | Low |
## Expected Impact
After all 9 fixes, `lws-main` should have **zero direct SQLite calls**. Its work becomes:
- JSON parse incoming messages (~fast)
- In-memory subscription matching (~fast)
- In-memory config lookups (~fast, hash map)
- Message formatting and queuing (~fast)
- `lws_service()` I/O (~fast)
- Draining completion queues (~fast)
### Impact by fix
| Fix | SQLite calls eliminated | Frequency |
|-----|------------------------|-----------|
| Fix 1: Global config cache | ~50+ `get_config_*` call sites across 8 files | Per-event, per-connection, per-REQ, per-broadcast, per-NIP-11 |
| Fix 2: Dedup to event worker | 2 `event_id_exists_in_db()` calls | Per-EVENT |
| Fix 3: Async COUNT | 1 `thread_pool_execute_count_sync()` | Per-COUNT message |
| Fix 4: WoT cache per-session | 1 `check_database_auth_rules()` | Per-REQ when WoT enabled |
| Fix 5: Async special-kind store | ~6 `store_event()` calls | Per special-kind EVENT |
| Fix 6: Reduce readers | N/A (resource savings) | N/A |
| Fix 7: Async subscription logging | 4 `db_log_*`/`db_update_*` calls | Per-REQ, per-CLOSE, per-broadcast |
| Fix 8: Async periodic tasks | `generate_stats_json()`, `ip_ban_*` | Every 60s |
| Fix 9: Cache NIP-11 | 25+ config lookups per request | Per NIP-11 HTTP request |
Estimated main-thread CPU reduction: from **54% to ~5-10%** (mostly WebSocket I/O, subscription matching, and JSON parsing).
## Thread Model After Changes
```
lws-main — WebSocket I/O, JSON parse, subscription match, message queue
NO SQLite. Pure compute + I/O.
db-read — All SELECT queries: REQ results, COUNT results, config refresh,
auth rule checks, event dedup checks via completion queue
db-write — All INSERT/UPDATE: event storage, tag inserts, IP ban persistence,
status event generation
event-worker — EVENT validation: signature verification, then submits store to
db-write via completion queue, results flow back to lws-main
```
```mermaid
flowchart LR
CLIENT[Nostr Clients] --> LWS[lws-main<br/>WebSocket I/O<br/>JSON parse<br/>Sub matching<br/>NO SQLite]
LWS -->|EVENT submit| EW[event-worker<br/>Sig verify<br/>Dedup check]
EW -->|store job| DW[db-write<br/>INSERT events<br/>INSERT tags<br/>IP ban save]
EW -->|completion| LWS
DW -->|completion| LWS
LWS -->|REQ/COUNT submit| DR[db-read<br/>SELECT queries<br/>Config refresh<br/>Auth checks]
DR -->|completion| LWS
LWS -->|OK/EVENT/EOSE| CLIENT
```
+107
View File
@@ -0,0 +1,107 @@
# WAL Checkpoint Fix — db-write Thread CPU Plan
## Problem
Fresh profiling of the production server shows the `db-write` thread consuming **77% average CPU** (peaking at 100%) while `lws-main` sits at 0.3%.
The `perf` callgraph confirms 100% of the hot symbols are SQLite **B-tree read operations** running on the `db-write` thread:
| % CPU | Symbol | Role |
|-------|--------|------|
| 13.88 | `sqlite3BtreeTableMoveto` | B-tree seek |
| 10.05 | `sqlite3VdbeExec` | VM execution |
| 7.77 | `rep_movs_alternative` | kernel memcpy in pread |
| 7.76 | `sqlite3GetVarint` | varint decode |
| 6.66 | `pcache1Fetch` | page cache lookup |
| 6.02 | `memcpy` | data copy |
These are **read-path** functions, not write-path. The `event-worker` thread accumulated only 3 CPU ticks over 5 minutes, meaning very few events are actually being stored.
## Root Cause
SQLite WAL auto-checkpoint. By default SQLite triggers a checkpoint every 1000 WAL pages. The checkpoint is executed by the **next writer** — which is always the `db-write` thread since all writes are funnelled through it.
A WAL checkpoint reads every dirty page from the WAL file and writes it back to the main database file. This involves B-tree traversals to locate pages, which explains the `BtreeTableMoveto` dominance.
The current `db_open_worker_connection` in `src/db_ops.c` sets `PRAGMA journal_mode=WAL` and `busy_timeout=5000` but does **not** configure `wal_autocheckpoint`. SQLite's default of 1000 pages applies.
In earlier profiling runs the same CPU burn appeared on `lws-main` at 55-67% — because that thread was doing the writes directly before the thread-pool refactoring. The work simply moved threads; the total cost is unchanged.
## Implementation Plan
### Step 1 — Disable auto-checkpoint on the write connection
In `db_open_worker_connection` add:
```c
sqlite3_exec(db, "PRAGMA wal_autocheckpoint=0;", NULL, NULL, NULL);
```
This prevents the write thread from ever running a checkpoint inline with normal writes.
### Step 2 — Add a periodic checkpoint job type to the thread pool
Add a new job type `THREAD_POOL_JOB_WAL_CHECKPOINT` to `thread_pool.h`.
The handler in `thread_pool.c` calls:
```c
sqlite3_wal_checkpoint_v2(db, NULL, SQLITE_CHECKPOINT_PASSIVE, NULL, NULL);
```
`PASSIVE` mode checkpoints only pages that are not currently being read, so it never blocks readers.
### Step 3 — Submit the checkpoint job on a timer
In the existing 60-second periodic timer in `websockets.c`, add a call to submit a WAL checkpoint job to the write queue:
```c
thread_pool_submit_wal_checkpoint();
```
This runs the checkpoint once per minute on the write thread, but as a **bounded, predictable** operation rather than being triggered unpredictably by every INSERT.
### Step 4 — Also disable auto-checkpoint on reader connections
In `db_open_worker_connection`, the same `wal_autocheckpoint=0` applies to reader connections too. Readers can also trigger checkpoints in SQLite; disabling it on all connections ensures only the explicit periodic job does checkpointing.
### Step 5 — Run a TRUNCATE checkpoint at shutdown
In `thread_pool_shutdown`, before closing the write connection, run:
```c
sqlite3_wal_checkpoint_v2(db, NULL, SQLITE_CHECKPOINT_TRUNCATE, NULL, NULL);
```
This ensures the WAL is fully flushed and truncated on clean shutdown, keeping the database file compact.
## Files Changed
| File | Change |
|------|--------|
| `src/db_ops.c` | Add `PRAGMA wal_autocheckpoint=0` in `db_open_worker_connection` |
| `src/thread_pool.h` | Add `THREAD_POOL_JOB_WAL_CHECKPOINT` enum value |
| `src/thread_pool.c` | Add `execute_wal_checkpoint_job` handler; add checkpoint-at-shutdown in `thread_pool_shutdown`; add `thread_pool_submit_wal_checkpoint` helper |
| `src/websockets.c` | Call `thread_pool_submit_wal_checkpoint()` in the 60-second periodic timer |
## Expected Impact
- `db-write` CPU should drop from ~77% to near 0% when idle (no events to store)
- Periodic checkpoint bursts of a few seconds every 60s instead of continuous burn
- No change to data durability — WAL still protects against crashes; checkpoint just moves data from WAL to main DB file
- Readers are unaffected — PASSIVE checkpoint never blocks them
## Risk
- If the relay crashes between checkpoints, the WAL file may be larger than before (up to 60s of accumulated writes). This is safe — SQLite replays the WAL on next open. The WAL file size is bounded by the write rate, which is low.
- PASSIVE checkpoint may not checkpoint all pages if readers hold them. This is fine — the next checkpoint will catch up.
## Verification
After deploying, re-run the profiler:
```bash
./deploy_lt_debug.sh && DURATION=300 INTERVAL=5 ./tests/thread_cpu_profile.sh
```
Expected: `db-write` avg CPU drops below 5%.
+10
View File
@@ -0,0 +1,10 @@
https://github.com/rushmi0/Fenrir-s
https://github.com/barkyq/gnost-relay
https://github.com/lpicanco/knostr
https://github.com/bezysoftware/netstr
https://github.com/lebrunel/nex
https://github.com/CodyTseng/nostr-relay-nestjs
https://github.com/mattn/nostr-relay
https://github.com/Cameri/nostream
https://github.com/Giszmo/NostrPostr/tree/master/NostrRelay
https://github.com/fiatjaf/relayer/tree/master/examples/basic
+1 -1
View File
@@ -1 +1 @@
3134018
1048978
+168 -342
View File
@@ -14,7 +14,6 @@ extern void log_query_execution(const char* query_type, const char* sub_id,
#include <libwebsockets.h>
#include <cjson/cJSON.h>
int get_active_connection_count(void);
#include <sqlite3.h>
#include <time.h>
#include <sys/stat.h>
#include <unistd.h>
@@ -28,6 +27,7 @@ int get_active_connection_count(void);
#include "../nostr_core_lib/nostr_core/nip017.h"
#include "../nostr_core_lib/nostr_core/nip044.h"
#include "subscriptions.h"
#include "db_ops.h"
// External subscription manager (from main.c via subscriptions.c)
extern subscription_manager_t g_subscription_manager;
@@ -55,6 +55,14 @@ int generate_monitoring_event_for_type(const char* d_tag_value, cJSON* (*query_f
// Forward declaration for CPU metrics query function
cJSON* query_cpu_metrics(void);
static int api_open_temp_db_connection(void** out_db) {
return db_open_worker_connection(db_get_database_path(), out_db);
}
static void api_close_temp_db_connection(void* db) {
db_close_worker_connection(db);
}
// Monitoring system helper functions
int get_monitoring_throttle_seconds(void) {
return get_config_int("kind_24567_reporting_throttle_sec", 5);
@@ -62,22 +70,22 @@ int get_monitoring_throttle_seconds(void) {
// Query event kind distribution from database
cJSON* query_event_kind_distribution(void) {
extern sqlite3* g_db;
if (!g_db) {
void* temp_db = NULL;
if (!db_is_available() && api_open_temp_db_connection(&temp_db) != 0) {
DEBUG_ERROR("Database not available for monitoring query");
return NULL;
}
// Query event kinds distribution with total count
sqlite3_stmt* stmt;
const char* sql = "SELECT kind, COUNT(*) as count FROM events GROUP BY kind ORDER BY count DESC";
// Start timing
struct timespec query_start, query_end;
clock_gettime(CLOCK_MONOTONIC, &query_start);
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) {
DEBUG_ERROR("Failed to prepare event kind distribution query");
long long total_events = 0;
cJSON* kinds_array = db_get_event_kind_distribution_rows(&total_events);
if (!kinds_array) {
DEBUG_ERROR("Failed to query event kind distribution rows");
return NULL;
}
@@ -85,41 +93,25 @@ cJSON* query_event_kind_distribution(void) {
cJSON_AddStringToObject(distribution, "data_type", "event_kinds");
cJSON_AddNumberToObject(distribution, "timestamp", (double)time(NULL));
cJSON* kinds_array = cJSON_CreateArray();
long long total_events = 0;
int row_count = 0;
while (sqlite3_step(stmt) == SQLITE_ROW) {
row_count++;
int kind = sqlite3_column_int(stmt, 0);
long long count = sqlite3_column_int64(stmt, 1);
total_events += count;
cJSON* kind_obj = cJSON_CreateObject();
cJSON_AddNumberToObject(kind_obj, "kind", kind);
cJSON_AddNumberToObject(kind_obj, "count", count);
cJSON_AddItemToArray(kinds_array, kind_obj);
}
sqlite3_finalize(stmt);
// Stop timing and log
clock_gettime(CLOCK_MONOTONIC, &query_end);
long elapsed_us = (query_end.tv_sec - query_start.tv_sec) * 1000000L +
(query_end.tv_nsec - query_start.tv_nsec) / 1000L;
int row_count = cJSON_GetArraySize(kinds_array);
log_query_execution("MONITOR", "event_kinds", NULL, sql, elapsed_us, row_count);
cJSON_AddNumberToObject(distribution, "total_events", total_events);
cJSON_AddItemToObject(distribution, "kinds", kinds_array);
api_close_temp_db_connection(temp_db);
return distribution;
}
// Query time-based statistics from database
cJSON* query_time_based_statistics(void) {
extern sqlite3* g_db;
if (!g_db) {
void* temp_db = NULL;
if (!db_is_available() && api_open_temp_db_connection(&temp_db) != 0) {
DEBUG_ERROR("Database not available for time stats query");
return NULL;
}
@@ -143,38 +135,18 @@ cJSON* query_time_based_statistics(void) {
{NULL, 0, NULL}
};
// Get total events count
sqlite3_stmt* total_stmt;
const char* total_sql = "SELECT COUNT(*) FROM events";
long long total_events = 0;
if (sqlite3_prepare_v2(g_db, total_sql, -1, &total_stmt, NULL) == SQLITE_OK) {
if (sqlite3_step(total_stmt) == SQLITE_ROW) {
total_events = sqlite3_column_int64(total_stmt, 0);
}
sqlite3_finalize(total_stmt);
}
(void)db_get_total_event_count_ll(&total_events);
// Query each time period
for (int i = 0; periods[i].period != NULL; i++) {
sqlite3_stmt* stmt;
const char* sql = "SELECT COUNT(*) FROM events WHERE created_at >= ?";
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) {
DEBUG_ERROR("Failed to prepare time stats query");
time_t cutoff = now - periods[i].seconds;
long long count = 0;
if (db_get_event_count_since(cutoff, &count) != 0) {
DEBUG_ERROR("Failed to query time stats count");
continue;
}
time_t cutoff = now - periods[i].seconds;
sqlite3_bind_int64(stmt, 1, cutoff);
long long count = 0;
if (sqlite3_step(stmt) == SQLITE_ROW) {
count = sqlite3_column_int64(stmt, 0);
}
sqlite3_finalize(stmt);
cJSON* period_obj = cJSON_CreateObject();
cJSON_AddStringToObject(period_obj, "period", periods[i].period);
cJSON_AddNumberToObject(period_obj, "count", count);
@@ -185,60 +157,52 @@ cJSON* query_time_based_statistics(void) {
cJSON_AddItemToObject(time_stats, "periods", periods_array);
cJSON_AddNumberToObject(time_stats, "total_events", total_events);
api_close_temp_db_connection(temp_db);
return time_stats;
}
// Query top pubkeys by event count from database
cJSON* query_top_pubkeys(void) {
extern sqlite3* g_db;
if (!g_db) {
void* temp_db = NULL;
if (!db_is_available() && api_open_temp_db_connection(&temp_db) != 0) {
DEBUG_ERROR("Database not available for top pubkeys query");
return NULL;
}
// Query top 10 pubkeys by event count
sqlite3_stmt* stmt;
const char* sql = "SELECT pubkey, COUNT(*) as count FROM events GROUP BY pubkey ORDER BY count DESC LIMIT 10";
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) {
DEBUG_ERROR("Failed to prepare top pubkeys query");
return NULL;
}
cJSON* top_pubkeys = cJSON_CreateObject();
cJSON_AddStringToObject(top_pubkeys, "data_type", "top_pubkeys");
cJSON_AddNumberToObject(top_pubkeys, "timestamp", (double)time(NULL));
cJSON* pubkeys_array = cJSON_CreateArray();
// Get total events count for percentage calculation
sqlite3_stmt* total_stmt;
const char* total_sql = "SELECT COUNT(*) FROM events";
long long total_events = 0;
if (sqlite3_prepare_v2(g_db, total_sql, -1, &total_stmt, NULL) == SQLITE_OK) {
if (sqlite3_step(total_stmt) == SQLITE_ROW) {
total_events = sqlite3_column_int64(total_stmt, 0);
}
sqlite3_finalize(total_stmt);
cJSON* rows = db_get_top_pubkeys_rows(10);
if (!rows) {
cJSON_Delete(top_pubkeys);
api_close_temp_db_connection(temp_db);
DEBUG_ERROR("Failed to query top pubkeys rows");
return NULL;
}
while (sqlite3_step(stmt) == SQLITE_ROW) {
const char* pubkey = (const char*)sqlite3_column_text(stmt, 0);
long long count = sqlite3_column_int64(stmt, 1);
cJSON* pubkeys_array = cJSON_CreateArray();
cJSON* row = NULL;
cJSON_ArrayForEach(row, rows) {
cJSON* pubkey_obj = cJSON_CreateObject();
cJSON_AddStringToObject(pubkey_obj, "pubkey", pubkey ? pubkey : "");
cJSON_AddNumberToObject(pubkey_obj, "event_count", count);
// Percentage will be calculated by frontend using total_events
cJSON* pubkey_item = cJSON_GetObjectItemCaseSensitive(row, "pubkey");
cJSON* count_item = cJSON_GetObjectItemCaseSensitive(row, "count");
cJSON_AddStringToObject(pubkey_obj, "pubkey",
cJSON_IsString(pubkey_item) ? pubkey_item->valuestring : "");
cJSON_AddNumberToObject(pubkey_obj, "event_count",
cJSON_IsNumber(count_item) ? count_item->valuedouble : 0.0);
cJSON_AddItemToArray(pubkeys_array, pubkey_obj);
}
cJSON_Delete(rows);
sqlite3_finalize(stmt);
long long total_events = 0;
(void)db_get_total_event_count_ll(&total_events);
cJSON_AddItemToObject(top_pubkeys, "pubkeys", pubkeys_array);
cJSON_AddNumberToObject(top_pubkeys, "total_events", total_events);
api_close_temp_db_connection(temp_db);
return top_pubkeys;
}
@@ -246,15 +210,12 @@ cJSON* query_top_pubkeys(void) {
// Query detailed subscription information from database log (ADMIN ONLY)
// Uses subscriptions table instead of in-memory iteration to avoid mutex contention
cJSON* query_subscription_details(void) {
extern sqlite3* g_db;
if (!g_db) {
void* temp_db = NULL;
if (!db_is_available() && api_open_temp_db_connection(&temp_db) != 0) {
DEBUG_ERROR("Database not available for subscription details query");
return NULL;
}
// Query active subscriptions from the active_subscriptions_log view
// This view properly handles deduplication of closed/expired subscriptions
sqlite3_stmt* stmt;
const char* sql =
"SELECT * "
"FROM active_subscriptions_log "
@@ -268,11 +229,6 @@ cJSON* query_subscription_details(void) {
struct timespec query_start, query_end;
clock_gettime(CLOCK_MONOTONIC, &query_start);
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) {
DEBUG_ERROR("Failed to prepare subscription details query");
return NULL;
}
time_t current_time = time(NULL);
cJSON* subscriptions_data = cJSON_CreateObject();
cJSON_AddStringToObject(subscriptions_data, "data_type", "subscription_details");
@@ -281,34 +237,52 @@ cJSON* query_subscription_details(void) {
cJSON* data = cJSON_CreateObject();
cJSON* subscriptions_array = cJSON_CreateArray();
cJSON* rows = db_get_subscription_details_rows();
if (!rows) {
cJSON_Delete(subscriptions_data);
cJSON_Delete(data);
cJSON_Delete(subscriptions_array);
api_close_temp_db_connection(temp_db);
DEBUG_ERROR("Failed to query subscription details rows");
return NULL;
}
// Iterate through query results
int row_count = 0;
while (sqlite3_step(stmt) == SQLITE_ROW) {
cJSON* row = NULL;
cJSON_ArrayForEach(row, rows) {
row_count++;
cJSON* sub_obj = cJSON_CreateObject();
// Extract subscription data from database
const char* sub_id = (const char*)sqlite3_column_text(stmt, 0);
const char* client_ip = (const char*)sqlite3_column_text(stmt, 1);
const char* filter_json = (const char*)sqlite3_column_text(stmt, 2);
long long events_sent = sqlite3_column_int64(stmt, 3);
long long created_at = sqlite3_column_int64(stmt, 4);
long long duration_seconds = sqlite3_column_int64(stmt, 5);
const char* wsi_pointer = (const char*)sqlite3_column_text(stmt, 6);
cJSON* sub_id_item = cJSON_GetObjectItemCaseSensitive(row, "id");
cJSON* client_ip_item = cJSON_GetObjectItemCaseSensitive(row, "client_ip");
cJSON* filter_json_item = cJSON_GetObjectItemCaseSensitive(row, "filter_json");
cJSON* events_sent_item = cJSON_GetObjectItemCaseSensitive(row, "events_sent");
cJSON* created_at_item = cJSON_GetObjectItemCaseSensitive(row, "created_at");
cJSON* duration_item = cJSON_GetObjectItemCaseSensitive(row, "duration_seconds");
cJSON* wsi_pointer_item = cJSON_GetObjectItemCaseSensitive(row, "wsi_pointer");
const char* sub_id = cJSON_IsString(sub_id_item) ? sub_id_item->valuestring : "";
const char* client_ip = cJSON_IsString(client_ip_item) ? client_ip_item->valuestring : "";
const char* filter_json = cJSON_IsString(filter_json_item) ? filter_json_item->valuestring : "[]";
long long events_sent = cJSON_IsNumber(events_sent_item) ? (long long)events_sent_item->valuedouble : 0;
long long created_at = cJSON_IsNumber(created_at_item) ? (long long)created_at_item->valuedouble : 0;
long long duration_seconds = cJSON_IsNumber(duration_item) ? (long long)duration_item->valuedouble : 0;
const char* wsi_pointer = cJSON_IsString(wsi_pointer_item) ? wsi_pointer_item->valuestring : "";
// DEBUG: Log each subscription found
DEBUG_LOG("Row %d: sub_id=%s, client_ip=%s, events_sent=%lld, created_at=%lld",
row_count, sub_id ? sub_id : "NULL", client_ip ? client_ip : "NULL",
row_count, sub_id[0] ? sub_id : "NULL", client_ip[0] ? client_ip : "NULL",
events_sent, created_at);
// Add basic subscription info
cJSON_AddStringToObject(sub_obj, "id", sub_id ? sub_id : "");
cJSON_AddStringToObject(sub_obj, "client_ip", client_ip ? client_ip : "");
cJSON_AddStringToObject(sub_obj, "id", sub_id);
cJSON_AddStringToObject(sub_obj, "client_ip", client_ip);
cJSON_AddNumberToObject(sub_obj, "created_at", (double)created_at);
cJSON_AddNumberToObject(sub_obj, "duration_seconds", (double)duration_seconds);
cJSON_AddNumberToObject(sub_obj, "events_sent", events_sent);
cJSON_AddBoolToObject(sub_obj, "active", 1); // All from this view are active
cJSON_AddStringToObject(sub_obj, "wsi_pointer", wsi_pointer ? wsi_pointer : "N/A");
cJSON_AddStringToObject(sub_obj, "wsi_pointer", wsi_pointer[0] ? wsi_pointer : "N/A");
// Extract query stats from per_session_data if wsi is still valid
int db_queries = 0;
@@ -317,16 +291,16 @@ cJSON* query_subscription_details(void) {
double row_rate = 0.0;
double avg_rows_per_query = 0.0;
if (wsi_pointer && strlen(wsi_pointer) > 2) { // Check for valid pointer string
if (wsi_pointer[0] != '\0' && strlen(wsi_pointer) > 2) { // Check for valid pointer string
// Parse wsi pointer from hex string
struct lws* wsi = NULL;
if (sscanf(wsi_pointer, "%p", (void**)&wsi) == 1 && wsi != NULL) {
// Get per_session_data from wsi
struct per_session_data* pss = (struct per_session_data*)lws_wsi_user(wsi);
if (pss) {
// Resolve live session data only if this wsi is still tracked.
struct per_session_data* pss = NULL;
if (websocket_get_live_pss(wsi, &pss) && pss) {
db_queries = pss->db_queries_executed;
db_rows = pss->db_rows_returned;
// Calculate rates (per minute)
time_t connection_duration = current_time - pss->query_tracking_start;
if (connection_duration > 0) {
@@ -334,7 +308,7 @@ cJSON* query_subscription_details(void) {
query_rate = db_queries / minutes;
row_rate = db_rows / minutes;
}
// Calculate average rows per query
if (db_queries > 0) {
avg_rows_per_query = (double)db_rows / (double)db_queries;
@@ -365,8 +339,7 @@ cJSON* query_subscription_details(void) {
cJSON_AddItemToArray(subscriptions_array, sub_obj);
}
sqlite3_finalize(stmt);
cJSON_Delete(rows);
// Add subscriptions array and count to data
cJSON_AddItemToObject(data, "subscriptions", subscriptions_array);
@@ -378,13 +351,14 @@ cJSON* query_subscription_details(void) {
clock_gettime(CLOCK_MONOTONIC, &query_end);
long elapsed_us = (query_end.tv_sec - query_start.tv_sec) * 1000000L +
(query_end.tv_nsec - query_start.tv_nsec) / 1000L;
log_query_execution("MONITOR", "subscription_details", NULL, sql, elapsed_us, row_count);
// DEBUG: Log final summary
DEBUG_LOG("Total subscriptions found: %d", row_count);
DEBUG_LOG("=== END SUBSCRIPTION_DETAILS QUERY DEBUG ===");
api_close_temp_db_connection(temp_db);
return subscriptions_data;
}
@@ -731,7 +705,6 @@ static const config_definition_t known_configs[] = {
{NULL, NULL, 0, 0}
};
// External database connection (from main.c)
extern sqlite3* g_db;
extern char g_database_path[512];
// Forward declarations for database functions
@@ -1036,138 +1009,12 @@ char* execute_sql_query(const char* query, const char* request_id, char* error_m
return NULL;
}
if (!g_db) {
if (!db_is_available()) {
snprintf(error_message, error_size, "Database not available");
return NULL;
}
// Set busy timeout to prevent long-running queries (5 seconds)
sqlite3_busy_timeout(g_db, 5000);
// Prepare statement
sqlite3_stmt* stmt;
int rc = sqlite3_prepare_v2(g_db, query, -1, &stmt, NULL);
if (rc != SQLITE_OK) {
const char* err_msg = sqlite3_errmsg(g_db);
snprintf(error_message, error_size, "SQL prepare failed: %s", err_msg);
return NULL;
}
// Execute query and collect results
cJSON* response = cJSON_CreateObject();
cJSON_AddStringToObject(response, "query_type", "sql_query");
cJSON_AddStringToObject(response, "request_id", request_id);
cJSON_AddNumberToObject(response, "timestamp", (double)time(NULL));
cJSON_AddStringToObject(response, "query", query);
// Get column information
int col_count = sqlite3_column_count(stmt);
cJSON* columns = cJSON_CreateArray();
for (int i = 0; i < col_count; i++) {
const char* col_name = sqlite3_column_name(stmt, i);
cJSON_AddItemToArray(columns, cJSON_CreateString(col_name ? col_name : ""));
}
cJSON_AddItemToObject(response, "columns", columns);
// Execute and collect rows (with limit)
cJSON* rows = cJSON_CreateArray();
int row_count = 0;
const int MAX_ROWS = 1000; // Configurable limit
struct timespec start_time;
clock_gettime(CLOCK_MONOTONIC, &start_time);
while ((rc = sqlite3_step(stmt)) == SQLITE_ROW && row_count < MAX_ROWS) {
cJSON* row = cJSON_CreateArray();
for (int i = 0; i < col_count; i++) {
int col_type = sqlite3_column_type(stmt, i);
switch (col_type) {
case SQLITE_INTEGER:
cJSON_AddItemToArray(row, cJSON_CreateNumber((double)sqlite3_column_int64(stmt, i)));
break;
case SQLITE_FLOAT:
cJSON_AddItemToArray(row, cJSON_CreateNumber(sqlite3_column_double(stmt, i)));
break;
case SQLITE_TEXT: {
const char* text = (const char*)sqlite3_column_text(stmt, i);
cJSON_AddItemToArray(row, cJSON_CreateString(text ? text : ""));
break;
}
case SQLITE_BLOB: {
// Convert blob to hex string for JSON compatibility
const void* blob = sqlite3_column_blob(stmt, i);
int blob_size = sqlite3_column_bytes(stmt, i);
if (blob && blob_size > 0) {
char* hex_str = malloc(blob_size * 2 + 1);
if (hex_str) {
for (int j = 0; j < blob_size; j++) {
sprintf(hex_str + j * 2, "%02x", ((unsigned char*)blob)[j]);
}
hex_str[blob_size * 2] = '\0';
cJSON_AddItemToArray(row, cJSON_CreateString(hex_str));
free(hex_str);
} else {
cJSON_AddItemToArray(row, cJSON_CreateString("[BLOB]"));
}
} else {
cJSON_AddItemToArray(row, cJSON_CreateString(""));
}
break;
}
case SQLITE_NULL:
cJSON_AddItemToArray(row, cJSON_CreateNull());
break;
default:
cJSON_AddItemToArray(row, cJSON_CreateString("[UNKNOWN]"));
break;
}
}
cJSON_AddItemToArray(rows, row);
row_count++;
// Check timeout (additional safety check)
struct timespec current_time;
clock_gettime(CLOCK_MONOTONIC, &current_time);
double elapsed = (current_time.tv_sec - start_time.tv_sec) +
(current_time.tv_nsec - start_time.tv_nsec) / 1e9;
if (elapsed > 4.5) { // 4.5 seconds to allow for cleanup
break;
}
}
sqlite3_finalize(stmt);
// Check for execution errors
if (rc != SQLITE_DONE && rc != SQLITE_ROW) {
const char* err_msg = sqlite3_errmsg(g_db);
snprintf(error_message, error_size, "SQL execution failed: %s", err_msg);
cJSON_Delete(response);
return NULL;
}
// Check row limit
if (row_count >= MAX_ROWS) {
cJSON_AddStringToObject(response, "warning", "Result truncated to maximum row limit");
}
// Add metadata
cJSON_AddNumberToObject(response, "row_count", row_count);
cJSON_AddNumberToObject(response, "execution_time_ms", 0); // Will be set by caller
cJSON_AddItemToObject(response, "rows", rows);
// Convert to JSON string
char* json_result = cJSON_Print(response);
cJSON_Delete(response);
if (!json_result) {
snprintf(error_message, error_size, "Failed to generate JSON response");
return NULL;
}
return json_result;
return db_execute_readonly_query_json(query, request_id, error_message, error_size, 1000, 5000);
}
// Unified handler for SQL query commands
@@ -1336,8 +1183,8 @@ cJSON* query_cpu_metrics(void) {
// Generate stats JSON from database queries
char* generate_stats_json(void) {
extern sqlite3* g_db;
if (!g_db) {
void* temp_db = NULL;
if (!db_is_available() && api_open_temp_db_connection(&temp_db) != 0) {
DEBUG_ERROR("Database not available for stats generation");
return NULL;
}
@@ -1348,10 +1195,10 @@ char* generate_stats_json(void) {
cJSON_AddNumberToObject(response, "timestamp", (double)time(NULL));
// Get database file size
extern char g_database_path[512];
const char* db_path = db_get_database_path();
struct stat db_stat;
long long db_size = 0;
if (stat(g_database_path, &db_stat) == 0) {
if (db_path && db_path[0] != '\0' && stat(db_path, &db_stat) == 0) {
db_size = db_stat.st_size;
}
cJSON_AddNumberToObject(response, "database_size_bytes", db_size);
@@ -1362,85 +1209,80 @@ char* generate_stats_json(void) {
pthread_mutex_unlock(&g_subscription_manager.subscriptions_lock);
cJSON_AddNumberToObject(response, "active_subscriptions", active_subs);
// Query total events count
sqlite3_stmt* stmt;
if (sqlite3_prepare_v2(g_db, "SELECT COUNT(*) FROM events", -1, &stmt, NULL) == SQLITE_OK) {
if (sqlite3_step(stmt) == SQLITE_ROW) {
cJSON_AddNumberToObject(response, "total_events", sqlite3_column_int64(stmt, 0));
}
sqlite3_finalize(stmt);
long long total_events = 0;
if (db_get_total_event_count_ll(&total_events) == 0) {
cJSON_AddNumberToObject(response, "total_events", total_events);
}
// Query event kinds distribution
cJSON* event_kinds = cJSON_CreateArray();
if (sqlite3_prepare_v2(g_db, "SELECT kind, count, percentage FROM event_kinds_view ORDER BY count DESC", -1, &stmt, NULL) == SQLITE_OK) {
while (sqlite3_step(stmt) == SQLITE_ROW) {
long long kinds_total = 0;
cJSON* kind_rows = db_get_event_kind_distribution_rows(&kinds_total);
if (kind_rows) {
cJSON* kind_row = NULL;
cJSON_ArrayForEach(kind_row, kind_rows) {
cJSON* kind_obj = cJSON_CreateObject();
cJSON_AddNumberToObject(kind_obj, "kind", sqlite3_column_int(stmt, 0));
cJSON_AddNumberToObject(kind_obj, "count", sqlite3_column_int64(stmt, 1));
cJSON_AddNumberToObject(kind_obj, "percentage", sqlite3_column_double(stmt, 2));
cJSON* kind = cJSON_GetObjectItemCaseSensitive(kind_row, "kind");
cJSON* count = cJSON_GetObjectItemCaseSensitive(kind_row, "count");
double count_val = cJSON_IsNumber(count) ? count->valuedouble : 0.0;
double pct = (kinds_total > 0) ? ((count_val * 100.0) / (double)kinds_total) : 0.0;
cJSON_AddNumberToObject(kind_obj, "kind", cJSON_IsNumber(kind) ? kind->valuedouble : 0.0);
cJSON_AddNumberToObject(kind_obj, "count", count_val);
cJSON_AddNumberToObject(kind_obj, "percentage", pct);
cJSON_AddItemToArray(event_kinds, kind_obj);
}
sqlite3_finalize(stmt);
cJSON_Delete(kind_rows);
}
cJSON_AddItemToObject(response, "event_kinds", event_kinds);
// Query time-based statistics
cJSON* time_stats = cJSON_CreateObject();
if (sqlite3_prepare_v2(g_db, "SELECT period, total_events FROM time_stats_view", -1, &stmt, NULL) == SQLITE_OK) {
while (sqlite3_step(stmt) == SQLITE_ROW) {
const char* period = (const char*)sqlite3_column_text(stmt, 0);
sqlite3_int64 count = sqlite3_column_int64(stmt, 1);
if (strcmp(period, "total") == 0) {
cJSON_AddNumberToObject(time_stats, "total", count);
} else if (strcmp(period, "24h") == 0) {
cJSON_AddNumberToObject(time_stats, "last_24h", count);
} else if (strcmp(period, "7d") == 0) {
cJSON_AddNumberToObject(time_stats, "last_7d", count);
} else if (strcmp(period, "30d") == 0) {
cJSON_AddNumberToObject(time_stats, "last_30d", count);
}
}
sqlite3_finalize(stmt);
time_t now = time(NULL);
long long c24h = 0, c7d = 0, c30d = 0;
if (db_get_event_count_since(now - 86400, &c24h) == 0) {
cJSON_AddNumberToObject(time_stats, "last_24h", c24h);
}
if (db_get_event_count_since(now - 604800, &c7d) == 0) {
cJSON_AddNumberToObject(time_stats, "last_7d", c7d);
}
if (db_get_event_count_since(now - 2592000, &c30d) == 0) {
cJSON_AddNumberToObject(time_stats, "last_30d", c30d);
}
cJSON_AddNumberToObject(time_stats, "total", total_events);
cJSON_AddItemToObject(response, "time_stats", time_stats);
// Query top pubkeys
cJSON* top_pubkeys = cJSON_CreateArray();
if (sqlite3_prepare_v2(g_db, "SELECT pubkey, event_count, percentage FROM top_pubkeys_view ORDER BY event_count DESC LIMIT 10", -1, &stmt, NULL) == SQLITE_OK) {
while (sqlite3_step(stmt) == SQLITE_ROW) {
cJSON* pubkey_rows = db_get_top_pubkeys_rows(10);
if (pubkey_rows) {
cJSON* row = NULL;
cJSON_ArrayForEach(row, pubkey_rows) {
cJSON* pubkey_obj = cJSON_CreateObject();
const char* pubkey = (const char*)sqlite3_column_text(stmt, 0);
cJSON_AddStringToObject(pubkey_obj, "pubkey", pubkey ? pubkey : "");
cJSON_AddNumberToObject(pubkey_obj, "event_count", sqlite3_column_int64(stmt, 1));
cJSON_AddNumberToObject(pubkey_obj, "percentage", sqlite3_column_double(stmt, 2));
cJSON* pubkey_item = cJSON_GetObjectItemCaseSensitive(row, "pubkey");
cJSON* count_item = cJSON_GetObjectItemCaseSensitive(row, "count");
double count_val = cJSON_IsNumber(count_item) ? count_item->valuedouble : 0.0;
double pct = (total_events > 0) ? ((count_val * 100.0) / (double)total_events) : 0.0;
cJSON_AddStringToObject(pubkey_obj, "pubkey", cJSON_IsString(pubkey_item) ? pubkey_item->valuestring : "");
cJSON_AddNumberToObject(pubkey_obj, "event_count", count_val);
cJSON_AddNumberToObject(pubkey_obj, "percentage", pct);
cJSON_AddItemToArray(top_pubkeys, pubkey_obj);
}
sqlite3_finalize(stmt);
cJSON_Delete(pubkey_rows);
}
cJSON_AddItemToObject(response, "top_pubkeys", top_pubkeys);
// Get database creation timestamp (oldest event)
if (sqlite3_prepare_v2(g_db, "SELECT MIN(created_at) FROM events", -1, &stmt, NULL) == SQLITE_OK) {
if (sqlite3_step(stmt) == SQLITE_ROW) {
sqlite3_int64 oldest_timestamp = sqlite3_column_int64(stmt, 0);
if (oldest_timestamp > 0) {
cJSON_AddNumberToObject(response, "database_created_at", (double)oldest_timestamp);
}
// Get database creation/latest timestamps
long long min_created_at = 0;
long long max_created_at = 0;
if (db_get_event_time_bounds(&min_created_at, &max_created_at) == 0) {
if (min_created_at > 0) {
cJSON_AddNumberToObject(response, "database_created_at", (double)min_created_at);
}
sqlite3_finalize(stmt);
}
// Get latest event timestamp
if (sqlite3_prepare_v2(g_db, "SELECT MAX(created_at) FROM events", -1, &stmt, NULL) == SQLITE_OK) {
if (sqlite3_step(stmt) == SQLITE_ROW) {
sqlite3_int64 latest_timestamp = sqlite3_column_int64(stmt, 0);
if (latest_timestamp > 0) {
cJSON_AddNumberToObject(response, "latest_event_at", (double)latest_timestamp);
}
if (max_created_at > 0) {
cJSON_AddNumberToObject(response, "latest_event_at", (double)max_created_at);
}
sqlite3_finalize(stmt);
}
// Convert to JSON string
@@ -1451,6 +1293,7 @@ char* generate_stats_json(void) {
DEBUG_ERROR("Failed to generate stats JSON");
}
api_close_temp_db_connection(temp_db);
return json_string;
}
@@ -1567,8 +1410,7 @@ int send_nip17_response(const char* sender_pubkey, const char* response_content,
// Generate config text from database
char* generate_config_text(void) {
extern sqlite3* g_db;
if (!g_db) {
if (!db_is_available()) {
DEBUG_ERROR("NIP-17: Database not available for config query");
return NULL;
}
@@ -1587,25 +1429,29 @@ char* generate_config_text(void) {
"🔧 Relay Configuration\n"
"━━━━━━━━━━━━━━━━━━━━━━━━\n");
// Query all config values from database
sqlite3_stmt* stmt;
if (sqlite3_prepare_v2(g_db, "SELECT key, value FROM config ORDER BY key", -1, &stmt, NULL) == SQLITE_OK) {
while (sqlite3_step(stmt) == SQLITE_ROW && offset < 8192 - 200) {
const char* key = (const char*)sqlite3_column_text(stmt, 0);
const char* value = (const char*)sqlite3_column_text(stmt, 1);
if (key && value) {
offset += snprintf(config_text + offset, 8192 - offset,
"%s: %s\n", key, value);
}
}
sqlite3_finalize(stmt);
} else {
cJSON* rows = db_get_all_config_rows();
if (!rows) {
free(config_text);
DEBUG_ERROR("NIP-17: Failed to query config from database");
return NULL;
}
cJSON* row = NULL;
cJSON_ArrayForEach(row, rows) {
if (offset >= 8192 - 200) break;
cJSON* key_item = cJSON_GetObjectItemCaseSensitive(row, "key");
cJSON* value_item = cJSON_GetObjectItemCaseSensitive(row, "value");
const char* key = cJSON_IsString(key_item) ? key_item->valuestring : NULL;
const char* value = cJSON_IsString(value_item) ? value_item->valuestring : NULL;
if (key && value) {
offset += snprintf(config_text + offset, 8192 - offset,
"%s: %s\n", key, value);
}
}
cJSON_Delete(rows);
// Footer
offset += snprintf(config_text + offset, 8192 - offset, "\n");
@@ -2166,8 +2012,7 @@ int apply_config_change(const char* key, const char* value) {
return -1;
}
extern sqlite3* g_db;
if (!g_db) {
if (!db_is_available()) {
DEBUG_ERROR("Database not available for config change");
return -1;
}
@@ -2200,30 +2045,11 @@ int apply_config_change(const char* key, const char* value) {
}
// Update or insert the configuration value
sqlite3_stmt* stmt;
const char* sql = "INSERT OR REPLACE INTO config (key, value, data_type) VALUES (?, ?, ?)";
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) {
DEBUG_ERROR("Failed to prepare config update statement");
const char* err_msg = sqlite3_errmsg(g_db);
DEBUG_ERROR(err_msg);
return -1;
}
sqlite3_bind_text(stmt, 1, key, -1, SQLITE_STATIC);
sqlite3_bind_text(stmt, 2, normalized_value, -1, SQLITE_STATIC);
sqlite3_bind_text(stmt, 3, data_type, -1, SQLITE_STATIC);
int result = sqlite3_step(stmt);
if (result != SQLITE_DONE) {
if (db_upsert_config_value(key, normalized_value, data_type) != 0) {
DEBUG_ERROR("Failed to update configuration in database");
const char* err_msg = sqlite3_errmsg(g_db);
DEBUG_ERROR(err_msg);
sqlite3_finalize(stmt);
return -1;
}
sqlite3_finalize(stmt);
return 0;
}
+598 -658
View File
File diff suppressed because it is too large Load Diff
+7 -1
View File
@@ -1,7 +1,6 @@
#ifndef CONFIG_H
#define CONFIG_H
#include <sqlite3.h>
#include <cjson/cJSON.h>
#include <time.h>
#include <pthread.h>
@@ -34,6 +33,12 @@ typedef struct {
char relay_privkey_override[65]; // Empty string = not set, 64-char hex = override
int strict_port; // 0 = allow port increment, 1 = fail if exact port unavailable
int debug_level; // 0-5, default 0 (no debug output)
char db_connstring_override[1024];
char db_host[128];
int db_port; // 0 = not set
char db_name[128];
char db_user[128];
char db_password[128];
} cli_options_t;
// Core configuration functions (temporary compatibility)
@@ -78,6 +83,7 @@ char* extract_pubkey_from_filename(const char* filename);
int store_relay_private_key(const char* relay_privkey_hex);
char* get_relay_private_key(void);
const char* get_temp_relay_private_key(void); // For first-time startup only
int preload_relay_private_key_cache(void); // Preload relay private key before strict DB mode
// NIP-42 authentication configuration functions
int parse_auth_required_kinds(const char* kinds_str, int* kinds_array, int max_kinds);
+315
View File
@@ -0,0 +1,315 @@
#define _GNU_SOURCE
#include "db_ops.h"
#include "sqlite_db_ops.h"
#include "db_ops_postgres.h"
#ifdef DB_BACKEND_POSTGRES
int db_init(const char* connection_string) { return postgres_db_init(connection_string); }
void db_close(void) { postgres_db_close(); }
int db_is_available(void) { return postgres_db_is_available(); }
const char* db_last_error(void) { return postgres_db_last_error(); }
const char* db_get_database_path(void) { return postgres_db_get_database_path(); }
int db_set_thread_connection(void* connection) { return postgres_db_set_thread_connection(connection); }
void db_clear_thread_connection(void) { postgres_db_clear_thread_connection(); }
int db_open_worker_connection(const char* db_path, void** out_connection) {
return postgres_db_open_worker_connection(db_path, out_connection);
}
void db_close_worker_connection(void* connection) { postgres_db_close_worker_connection(connection); }
int db_prepare(const char* sql, db_stmt_t** out_stmt) {
return postgres_db_prepare(sql, (postgres_db_stmt_t**)out_stmt);
}
int db_bind_text_param(db_stmt_t* stmt, int index, const char* value) {
return postgres_db_bind_text_param((postgres_db_stmt_t*)stmt, index, value);
}
int db_bind_int_param(db_stmt_t* stmt, int index, int value) {
return postgres_db_bind_int_param((postgres_db_stmt_t*)stmt, index, value);
}
int db_bind_int64_param(db_stmt_t* stmt, int index, long long value) {
return postgres_db_bind_int64_param((postgres_db_stmt_t*)stmt, index, value);
}
int db_step_stmt(db_stmt_t* stmt) { return postgres_db_step_stmt((postgres_db_stmt_t*)stmt); }
int db_reset_stmt(db_stmt_t* stmt) { return postgres_db_reset_stmt((postgres_db_stmt_t*)stmt); }
const char* db_column_text_value(db_stmt_t* stmt, int col) {
return postgres_db_column_text_value((postgres_db_stmt_t*)stmt, col);
}
int db_column_int_value(db_stmt_t* stmt, int col) {
return postgres_db_column_int_value((postgres_db_stmt_t*)stmt, col);
}
long long db_column_int64_value(db_stmt_t* stmt, int col) {
return postgres_db_column_int64_value((postgres_db_stmt_t*)stmt, col);
}
double db_column_double_value(db_stmt_t* stmt, int col) {
return postgres_db_column_double_value((postgres_db_stmt_t*)stmt, col);
}
void db_finalize_stmt(db_stmt_t* stmt) { postgres_db_finalize_stmt((postgres_db_stmt_t*)stmt); }
int db_log_subscription_created(const char* sub_id, const char* wsi_ptr,
const char* client_ip, const char* filter_json) {
return postgres_db_log_subscription_created(sub_id, wsi_ptr, client_ip, filter_json);
}
int db_log_subscription_closed(const char* sub_id, const char* client_ip) {
return postgres_db_log_subscription_closed(sub_id, client_ip);
}
int db_log_subscription_disconnected(const char* client_ip) {
return postgres_db_log_subscription_disconnected(client_ip);
}
int db_update_subscription_events_sent(const char* sub_id, int events_sent) {
return postgres_db_update_subscription_events_sent(sub_id, events_sent);
}
int db_cleanup_orphaned_subscriptions(void) { return postgres_db_cleanup_orphaned_subscriptions(); }
int db_get_event_pubkey(const char* event_id, char* pubkey_out, size_t pubkey_out_size) {
return postgres_db_get_event_pubkey(event_id, pubkey_out, pubkey_out_size);
}
int db_delete_event_by_id(const char* event_id, const char* requester_pubkey) {
return postgres_db_delete_event_by_id(event_id, requester_pubkey);
}
int db_delete_events_by_address(const char* pubkey, int kind, const char* d_tag, long before_timestamp) {
return postgres_db_delete_events_by_address(pubkey, kind, d_tag, before_timestamp);
}
int db_is_pubkey_blacklisted(const char* pubkey) { return postgres_db_is_pubkey_blacklisted(pubkey); }
int db_is_hash_blacklisted(const char* resource_hash) { return postgres_db_is_hash_blacklisted(resource_hash); }
int db_is_pubkey_whitelisted(const char* pubkey) { return postgres_db_is_pubkey_whitelisted(pubkey); }
int db_count_active_whitelist_rules(void) { return postgres_db_count_active_whitelist_rules(); }
int db_count_with_sql(const char* sql, const char** bind_params, int bind_param_count, int* out_count) {
return postgres_db_count_with_sql(sql, bind_params, bind_param_count, out_count);
}
char* db_execute_readonly_query_json(const char* query, const char* request_id,
char* error_message, size_t error_size,
int max_rows, int timeout_ms) {
return postgres_db_execute_readonly_query_json(query, request_id, error_message, error_size,
max_rows, timeout_ms);
}
int db_get_total_event_count_ll(long long* out_count) { return postgres_db_get_total_event_count_ll(out_count); }
int db_get_event_count_since(time_t cutoff, long long* out_count) {
return postgres_db_get_event_count_since(cutoff, out_count);
}
cJSON* db_get_event_kind_distribution_rows(long long* out_total_events) {
return postgres_db_get_event_kind_distribution_rows(out_total_events);
}
cJSON* db_get_top_pubkeys_rows(int limit) { return postgres_db_get_top_pubkeys_rows(limit); }
cJSON* db_get_subscription_details_rows(void) { return postgres_db_get_subscription_details_rows(); }
cJSON* db_get_all_config_rows(void) { return postgres_db_get_all_config_rows(); }
char* db_get_config_value_dup(const char* key) { return postgres_db_get_config_value_dup(key); }
int db_set_config_value_full(const char* key, const char* value, const char* data_type,
const char* description, const char* category, int requires_restart) {
return postgres_db_set_config_value_full(key, value, data_type, description, category, requires_restart);
}
int db_update_config_value_only(const char* key, const char* value) {
return postgres_db_update_config_value_only(key, value);
}
int db_upsert_config_value(const char* key, const char* value, const char* data_type) {
return postgres_db_upsert_config_value(key, value, data_type);
}
int db_store_relay_private_key_hex(const char* relay_privkey_hex) {
return postgres_db_store_relay_private_key_hex(relay_privkey_hex);
}
char* db_get_relay_private_key_hex_dup(void) { return postgres_db_get_relay_private_key_hex_dup(); }
int db_store_config_event(const cJSON* event) { return postgres_db_store_config_event(event); }
int db_insert_event_with_json(const char* id, const char* pubkey, long long created_at,
int kind, const char* event_type, const char* content,
const char* sig, const char* tags_json, const char* event_json,
int* out_step_rc, int* out_extended_errcode) {
return postgres_db_insert_event_with_json(id, pubkey, created_at, kind, event_type, content,
sig, tags_json, event_json, out_step_rc, out_extended_errcode);
}
int db_get_event_time_bounds(long long* out_min_created_at, long long* out_max_created_at) {
return postgres_db_get_event_time_bounds(out_min_created_at, out_max_created_at);
}
int db_event_id_exists(const char* event_id, int* out_exists) {
return postgres_db_event_id_exists(event_id, out_exists);
}
cJSON* db_retrieve_event_by_id(const char* event_id) { return postgres_db_retrieve_event_by_id(event_id); }
char* db_get_latest_event_pubkey_for_kind_dup(int kind) {
return postgres_db_get_latest_event_pubkey_for_kind_dup(kind);
}
int db_get_config_row_count(int* out_count) { return postgres_db_get_config_row_count(out_count); }
int db_store_event_tags_cjson(const char* event_id, const cJSON* tags) {
return postgres_db_store_event_tags_cjson(event_id, tags);
}
int db_populate_event_tags_from_existing(void) { return postgres_db_populate_event_tags_from_existing(); }
int db_add_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value) {
return postgres_db_add_auth_rule(rule_type, pattern_type, pattern_value);
}
int db_remove_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value) {
return postgres_db_remove_auth_rule(rule_type, pattern_type, pattern_value);
}
int db_delete_wot_whitelist_rules(void) { return postgres_db_delete_wot_whitelist_rules(); }
int db_count_wot_whitelist_rules(void) { return postgres_db_count_wot_whitelist_rules(); }
int db_table_exists(const char* table_name, int* out_exists) {
return postgres_db_table_exists(table_name, out_exists);
}
char* db_get_schema_version_dup(void) { return postgres_db_get_schema_version_dup(); }
int db_exec_sql(const char* sql) { return postgres_db_exec_sql(sql); }
int db_wal_checkpoint_passive(void) { return postgres_db_wal_checkpoint_passive(); }
int db_wal_checkpoint_truncate(void) { return postgres_db_wal_checkpoint_truncate(); }
#else
int db_init(const char* connection_string) { return sqlite_db_init(connection_string); }
void db_close(void) { sqlite_db_close(); }
int db_is_available(void) { return sqlite_db_is_available(); }
const char* db_last_error(void) { return sqlite_db_last_error(); }
const char* db_get_database_path(void) { return sqlite_db_get_database_path(); }
int db_set_thread_connection(void* connection) { return sqlite_db_set_thread_connection(connection); }
void db_clear_thread_connection(void) { sqlite_db_clear_thread_connection(); }
int db_open_worker_connection(const char* db_path, void** out_connection) {
return sqlite_db_open_worker_connection(db_path, out_connection);
}
void db_close_worker_connection(void* connection) { sqlite_db_close_worker_connection(connection); }
int db_prepare(const char* sql, db_stmt_t** out_stmt) {
return sqlite_db_prepare(sql, (sqlite_db_stmt_t**)out_stmt);
}
int db_bind_text_param(db_stmt_t* stmt, int index, const char* value) {
return sqlite_db_bind_text_param((sqlite_db_stmt_t*)stmt, index, value);
}
int db_bind_int_param(db_stmt_t* stmt, int index, int value) {
return sqlite_db_bind_int_param((sqlite_db_stmt_t*)stmt, index, value);
}
int db_bind_int64_param(db_stmt_t* stmt, int index, long long value) {
return sqlite_db_bind_int64_param((sqlite_db_stmt_t*)stmt, index, value);
}
int db_step_stmt(db_stmt_t* stmt) { return sqlite_db_step_stmt((sqlite_db_stmt_t*)stmt); }
int db_reset_stmt(db_stmt_t* stmt) { return sqlite_db_reset_stmt((sqlite_db_stmt_t*)stmt); }
const char* db_column_text_value(db_stmt_t* stmt, int col) {
return sqlite_db_column_text_value((sqlite_db_stmt_t*)stmt, col);
}
int db_column_int_value(db_stmt_t* stmt, int col) {
return sqlite_db_column_int_value((sqlite_db_stmt_t*)stmt, col);
}
long long db_column_int64_value(db_stmt_t* stmt, int col) {
return sqlite_db_column_int64_value((sqlite_db_stmt_t*)stmt, col);
}
double db_column_double_value(db_stmt_t* stmt, int col) {
return sqlite_db_column_double_value((sqlite_db_stmt_t*)stmt, col);
}
void db_finalize_stmt(db_stmt_t* stmt) { sqlite_db_finalize_stmt((sqlite_db_stmt_t*)stmt); }
int db_log_subscription_created(const char* sub_id, const char* wsi_ptr,
const char* client_ip, const char* filter_json) {
return sqlite_db_log_subscription_created(sub_id, wsi_ptr, client_ip, filter_json);
}
int db_log_subscription_closed(const char* sub_id, const char* client_ip) {
return sqlite_db_log_subscription_closed(sub_id, client_ip);
}
int db_log_subscription_disconnected(const char* client_ip) {
return sqlite_db_log_subscription_disconnected(client_ip);
}
int db_update_subscription_events_sent(const char* sub_id, int events_sent) {
return sqlite_db_update_subscription_events_sent(sub_id, events_sent);
}
int db_cleanup_orphaned_subscriptions(void) { return sqlite_db_cleanup_orphaned_subscriptions(); }
int db_get_event_pubkey(const char* event_id, char* pubkey_out, size_t pubkey_out_size) {
return sqlite_db_get_event_pubkey(event_id, pubkey_out, pubkey_out_size);
}
int db_delete_event_by_id(const char* event_id, const char* requester_pubkey) {
return sqlite_db_delete_event_by_id(event_id, requester_pubkey);
}
int db_delete_events_by_address(const char* pubkey, int kind, const char* d_tag, long before_timestamp) {
return sqlite_db_delete_events_by_address(pubkey, kind, d_tag, before_timestamp);
}
int db_is_pubkey_blacklisted(const char* pubkey) { return sqlite_db_is_pubkey_blacklisted(pubkey); }
int db_is_hash_blacklisted(const char* resource_hash) { return sqlite_db_is_hash_blacklisted(resource_hash); }
int db_is_pubkey_whitelisted(const char* pubkey) { return sqlite_db_is_pubkey_whitelisted(pubkey); }
int db_count_active_whitelist_rules(void) { return sqlite_db_count_active_whitelist_rules(); }
int db_count_with_sql(const char* sql, const char** bind_params, int bind_param_count, int* out_count) {
return sqlite_db_count_with_sql(sql, bind_params, bind_param_count, out_count);
}
char* db_execute_readonly_query_json(const char* query, const char* request_id,
char* error_message, size_t error_size,
int max_rows, int timeout_ms) {
return sqlite_db_execute_readonly_query_json(query, request_id, error_message, error_size,
max_rows, timeout_ms);
}
int db_get_total_event_count_ll(long long* out_count) { return sqlite_db_get_total_event_count_ll(out_count); }
int db_get_event_count_since(time_t cutoff, long long* out_count) {
return sqlite_db_get_event_count_since(cutoff, out_count);
}
cJSON* db_get_event_kind_distribution_rows(long long* out_total_events) {
return sqlite_db_get_event_kind_distribution_rows(out_total_events);
}
cJSON* db_get_top_pubkeys_rows(int limit) { return sqlite_db_get_top_pubkeys_rows(limit); }
cJSON* db_get_subscription_details_rows(void) { return sqlite_db_get_subscription_details_rows(); }
cJSON* db_get_all_config_rows(void) { return sqlite_db_get_all_config_rows(); }
char* db_get_config_value_dup(const char* key) { return sqlite_db_get_config_value_dup(key); }
int db_set_config_value_full(const char* key, const char* value, const char* data_type,
const char* description, const char* category, int requires_restart) {
return sqlite_db_set_config_value_full(key, value, data_type, description, category, requires_restart);
}
int db_update_config_value_only(const char* key, const char* value) {
return sqlite_db_update_config_value_only(key, value);
}
int db_upsert_config_value(const char* key, const char* value, const char* data_type) {
return sqlite_db_upsert_config_value(key, value, data_type);
}
int db_store_relay_private_key_hex(const char* relay_privkey_hex) {
return sqlite_db_store_relay_private_key_hex(relay_privkey_hex);
}
char* db_get_relay_private_key_hex_dup(void) { return sqlite_db_get_relay_private_key_hex_dup(); }
int db_store_config_event(const cJSON* event) { return sqlite_db_store_config_event(event); }
int db_insert_event_with_json(const char* id, const char* pubkey, long long created_at,
int kind, const char* event_type, const char* content,
const char* sig, const char* tags_json, const char* event_json,
int* out_step_rc, int* out_extended_errcode) {
return sqlite_db_insert_event_with_json(id, pubkey, created_at, kind, event_type, content,
sig, tags_json, event_json, out_step_rc, out_extended_errcode);
}
int db_get_event_time_bounds(long long* out_min_created_at, long long* out_max_created_at) {
return sqlite_db_get_event_time_bounds(out_min_created_at, out_max_created_at);
}
int db_event_id_exists(const char* event_id, int* out_exists) {
return sqlite_db_event_id_exists(event_id, out_exists);
}
cJSON* db_retrieve_event_by_id(const char* event_id) { return sqlite_db_retrieve_event_by_id(event_id); }
char* db_get_latest_event_pubkey_for_kind_dup(int kind) {
return sqlite_db_get_latest_event_pubkey_for_kind_dup(kind);
}
int db_get_config_row_count(int* out_count) { return sqlite_db_get_config_row_count(out_count); }
int db_store_event_tags_cjson(const char* event_id, const cJSON* tags) {
return sqlite_db_store_event_tags_cjson(event_id, tags);
}
int db_populate_event_tags_from_existing(void) { return sqlite_db_populate_event_tags_from_existing(); }
int db_add_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value) {
return sqlite_db_add_auth_rule(rule_type, pattern_type, pattern_value);
}
int db_remove_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value) {
return sqlite_db_remove_auth_rule(rule_type, pattern_type, pattern_value);
}
int db_delete_wot_whitelist_rules(void) { return sqlite_db_delete_wot_whitelist_rules(); }
int db_count_wot_whitelist_rules(void) { return sqlite_db_count_wot_whitelist_rules(); }
int db_table_exists(const char* table_name, int* out_exists) {
return sqlite_db_table_exists(table_name, out_exists);
}
char* db_get_schema_version_dup(void) { return sqlite_db_get_schema_version_dup(); }
int db_exec_sql(const char* sql) { return sqlite_db_exec_sql(sql); }
int db_wal_checkpoint_passive(void) { return sqlite_db_wal_checkpoint_passive(); }
int db_wal_checkpoint_truncate(void) { return sqlite_db_wal_checkpoint_truncate(); }
#endif
+120
View File
@@ -0,0 +1,120 @@
#ifndef DB_OPS_H
#define DB_OPS_H
#include <stddef.h>
#include <time.h>
#include <cjson/cJSON.h>
// Generic helpers
int db_init(const char* connection_string);
void db_close(void);
int db_is_available(void);
const char* db_last_error(void);
const char* db_get_database_path(void);
// Per-thread connection override (used by thread pool workers)
int db_set_thread_connection(void* connection);
void db_clear_thread_connection(void);
// Worker/runtime SQLite connection lifecycle (kept internal to db_ops.c)
int db_open_worker_connection(const char* db_path, void** out_connection);
void db_close_worker_connection(void* connection);
// DB result codes (backend-agnostic)
#define DB_OK 0
#define DB_ERROR 1
#define DB_CONSTRAINT 19
#define DB_MISUSE 21
#define DB_ROW 100
#define DB_DONE 101
typedef struct db_stmt db_stmt_t;
// Generic statement helpers (Phase 1 migration)
int db_prepare(const char* sql, db_stmt_t** out_stmt);
int db_bind_text_param(db_stmt_t* stmt, int index, const char* value);
int db_bind_int_param(db_stmt_t* stmt, int index, int value);
int db_bind_int64_param(db_stmt_t* stmt, int index, long long value);
int db_step_stmt(db_stmt_t* stmt);
int db_reset_stmt(db_stmt_t* stmt);
const char* db_column_text_value(db_stmt_t* stmt, int col);
int db_column_int_value(db_stmt_t* stmt, int col);
long long db_column_int64_value(db_stmt_t* stmt, int col);
double db_column_double_value(db_stmt_t* stmt, int col);
void db_finalize_stmt(db_stmt_t* stmt);
// Subscription logging
int db_log_subscription_created(const char* sub_id, const char* wsi_ptr,
const char* client_ip, const char* filter_json);
int db_log_subscription_closed(const char* sub_id, const char* client_ip);
int db_log_subscription_disconnected(const char* client_ip);
int db_update_subscription_events_sent(const char* sub_id, int events_sent);
int db_cleanup_orphaned_subscriptions(void);
// NIP-09 event deletion helpers
int db_get_event_pubkey(const char* event_id, char* pubkey_out, size_t pubkey_out_size);
int db_delete_event_by_id(const char* event_id, const char* requester_pubkey);
int db_delete_events_by_address(const char* pubkey, int kind,
const char* d_tag, long before_timestamp);
// Auth rule checks for request validator
int db_is_pubkey_blacklisted(const char* pubkey);
int db_is_hash_blacklisted(const char* resource_hash);
int db_is_pubkey_whitelisted(const char* pubkey);
int db_count_active_whitelist_rules(void);
// Generic prepared COUNT helper
int db_count_with_sql(const char* sql, const char** bind_params, int bind_param_count, int* out_count);
char* db_execute_readonly_query_json(const char* query, const char* request_id,
char* error_message, size_t error_size,
int max_rows, int timeout_ms);
// Monitoring/stat helpers (Phase 1 api.c migration)
int db_get_total_event_count_ll(long long* out_count);
int db_get_event_count_since(time_t cutoff, long long* out_count);
cJSON* db_get_event_kind_distribution_rows(long long* out_total_events);
cJSON* db_get_top_pubkeys_rows(int limit);
cJSON* db_get_subscription_details_rows(void);
// Config helpers
cJSON* db_get_all_config_rows(void);
char* db_get_config_value_dup(const char* key);
int db_set_config_value_full(const char* key, const char* value, const char* data_type,
const char* description, const char* category, int requires_restart);
int db_update_config_value_only(const char* key, const char* value);
int db_upsert_config_value(const char* key, const char* value, const char* data_type);
int db_store_relay_private_key_hex(const char* relay_privkey_hex);
char* db_get_relay_private_key_hex_dup(void);
int db_store_config_event(const cJSON* event);
// Event storage/timestamp/retrieval helpers
int db_insert_event_with_json(const char* id, const char* pubkey, long long created_at,
int kind, const char* event_type, const char* content,
const char* sig, const char* tags_json, const char* event_json,
int* out_step_rc, int* out_extended_errcode);
int db_get_event_time_bounds(long long* out_min_created_at, long long* out_max_created_at);
int db_event_id_exists(const char* event_id, int* out_exists);
cJSON* db_retrieve_event_by_id(const char* event_id);
char* db_get_latest_event_pubkey_for_kind_dup(int kind);
// Config table helpers
int db_get_config_row_count(int* out_count);
// Event tag helpers
int db_store_event_tags_cjson(const char* event_id, const cJSON* tags);
int db_populate_event_tags_from_existing(void);
// Auth/WoT rule helpers
int db_add_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
int db_remove_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
int db_delete_wot_whitelist_rules(void);
int db_count_wot_whitelist_rules(void);
// Schema/DDL helpers
int db_table_exists(const char* table_name, int* out_exists);
char* db_get_schema_version_dup(void);
int db_exec_sql(const char* sql);
int db_wal_checkpoint_passive(void);
int db_wal_checkpoint_truncate(void);
#endif // DB_OPS_H
File diff suppressed because it is too large Load Diff
+97
View File
@@ -0,0 +1,97 @@
#ifndef DB_OPS_POSTGRES_H
#define DB_OPS_POSTGRES_H
#include "db_ops.h"
typedef struct postgres_db_stmt postgres_db_stmt_t;
int postgres_db_init(const char* connection_string);
void postgres_db_close(void);
int postgres_db_is_available(void);
const char* postgres_db_last_error(void);
const char* postgres_db_get_database_path(void);
int postgres_db_apply_schema(void);
int postgres_db_set_thread_connection(void* connection);
void postgres_db_clear_thread_connection(void);
int postgres_db_open_worker_connection(const char* db_path, void** out_connection);
void postgres_db_close_worker_connection(void* connection);
int postgres_db_prepare(const char* sql, postgres_db_stmt_t** out_stmt);
int postgres_db_bind_text_param(postgres_db_stmt_t* stmt, int index, const char* value);
int postgres_db_bind_int_param(postgres_db_stmt_t* stmt, int index, int value);
int postgres_db_bind_int64_param(postgres_db_stmt_t* stmt, int index, long long value);
int postgres_db_step_stmt(postgres_db_stmt_t* stmt);
int postgres_db_reset_stmt(postgres_db_stmt_t* stmt);
const char* postgres_db_column_text_value(postgres_db_stmt_t* stmt, int col);
int postgres_db_column_int_value(postgres_db_stmt_t* stmt, int col);
long long postgres_db_column_int64_value(postgres_db_stmt_t* stmt, int col);
double postgres_db_column_double_value(postgres_db_stmt_t* stmt, int col);
void postgres_db_finalize_stmt(postgres_db_stmt_t* stmt);
int postgres_db_log_subscription_created(const char* sub_id, const char* wsi_ptr,
const char* client_ip, const char* filter_json);
int postgres_db_log_subscription_closed(const char* sub_id, const char* client_ip);
int postgres_db_log_subscription_disconnected(const char* client_ip);
int postgres_db_update_subscription_events_sent(const char* sub_id, int events_sent);
int postgres_db_cleanup_orphaned_subscriptions(void);
int postgres_db_get_event_pubkey(const char* event_id, char* pubkey_out, size_t pubkey_out_size);
int postgres_db_delete_event_by_id(const char* event_id, const char* requester_pubkey);
int postgres_db_delete_events_by_address(const char* pubkey, int kind,
const char* d_tag, long before_timestamp);
int postgres_db_is_pubkey_blacklisted(const char* pubkey);
int postgres_db_is_hash_blacklisted(const char* resource_hash);
int postgres_db_is_pubkey_whitelisted(const char* pubkey);
int postgres_db_count_active_whitelist_rules(void);
int postgres_db_count_with_sql(const char* sql, const char** bind_params, int bind_param_count, int* out_count);
char* postgres_db_execute_readonly_query_json(const char* query, const char* request_id,
char* error_message, size_t error_size,
int max_rows, int timeout_ms);
int postgres_db_get_total_event_count_ll(long long* out_count);
int postgres_db_get_event_count_since(time_t cutoff, long long* out_count);
cJSON* postgres_db_get_event_kind_distribution_rows(long long* out_total_events);
cJSON* postgres_db_get_top_pubkeys_rows(int limit);
cJSON* postgres_db_get_subscription_details_rows(void);
cJSON* postgres_db_get_all_config_rows(void);
char* postgres_db_get_config_value_dup(const char* key);
int postgres_db_set_config_value_full(const char* key, const char* value, const char* data_type,
const char* description, const char* category, int requires_restart);
int postgres_db_update_config_value_only(const char* key, const char* value);
int postgres_db_upsert_config_value(const char* key, const char* value, const char* data_type);
int postgres_db_store_relay_private_key_hex(const char* relay_privkey_hex);
char* postgres_db_get_relay_private_key_hex_dup(void);
int postgres_db_store_config_event(const cJSON* event);
int postgres_db_insert_event_with_json(const char* id, const char* pubkey, long long created_at,
int kind, const char* event_type, const char* content,
const char* sig, const char* tags_json, const char* event_json,
int* out_step_rc, int* out_extended_errcode);
int postgres_db_get_event_time_bounds(long long* out_min_created_at, long long* out_max_created_at);
int postgres_db_event_id_exists(const char* event_id, int* out_exists);
cJSON* postgres_db_retrieve_event_by_id(const char* event_id);
char* postgres_db_get_latest_event_pubkey_for_kind_dup(int kind);
int postgres_db_get_config_row_count(int* out_count);
int postgres_db_store_event_tags_cjson(const char* event_id, const cJSON* tags);
int postgres_db_populate_event_tags_from_existing(void);
int postgres_db_add_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
int postgres_db_remove_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
int postgres_db_delete_wot_whitelist_rules(void);
int postgres_db_count_wot_whitelist_rules(void);
int postgres_db_table_exists(const char* table_name, int* out_exists);
char* postgres_db_get_schema_version_dup(void);
int postgres_db_exec_sql(const char* sql);
int postgres_db_wal_checkpoint_passive(void);
int postgres_db_wal_checkpoint_truncate(void);
#endif // DB_OPS_POSTGRES_H
+1397
View File
File diff suppressed because it is too large Load Diff
+3 -15
View File
@@ -5,6 +5,7 @@
#include "config.h"
#include "debug.h"
#include "api.h"
#include "db_ops.h"
#include "../nostr_core_lib/nostr_core/nostr_core.h"
#include "../nostr_core_lib/nostr_core/nip017.h"
#include "../nostr_core_lib/nostr_core/nip044.h"
@@ -17,9 +18,6 @@
#include <cjson/cJSON.h>
#include <libwebsockets.h>
// External database connection (from main.c)
extern sqlite3* g_db;
// Forward declarations for unified handlers
extern int handle_auth_query_unified(cJSON* event, const char* query_type, char* error_message, size_t error_size, struct lws* wsi);
extern int handle_config_query_unified(cJSON* event, const char* query_type, char* error_message, size_t error_size, struct lws* wsi);
@@ -622,8 +620,7 @@ int process_nip17_admin_command(cJSON* dm_event, char* error_message, size_t err
update_config_in_table("nip42_auth_required_subscriptions", "false");
// Clear wot_whitelist rules
const char* delete_sql = "DELETE FROM auth_rules WHERE rule_type = 'wot_whitelist'";
sqlite3_exec(g_db, delete_sql, NULL, NULL, NULL);
db_delete_wot_whitelist_rules();
snprintf(response_msg, sizeof(response_msg),
"🔓 Web of Trust Disabled\n"
@@ -695,16 +692,7 @@ int process_nip17_admin_command(cJSON* dm_event, char* error_message, size_t err
else if (strcmp(wot_cmd, "status") == 0 || strlen(wot_cmd) == 0) {
// Show status
// Count whitelisted pubkeys
sqlite3_stmt* stmt;
const char* count_sql = "SELECT COUNT(*) FROM auth_rules WHERE rule_type = 'wot_whitelist' AND active = 1";
int whitelist_count = 0;
if (sqlite3_prepare_v2(g_db, count_sql, -1, &stmt, NULL) == SQLITE_OK) {
if (sqlite3_step(stmt) == SQLITE_ROW) {
whitelist_count = sqlite3_column_int(stmt, 0);
}
sqlite3_finalize(stmt);
}
int whitelist_count = db_count_wot_whitelist_rules();
const char* level_str;
if (wot_level == 0) level_str = "Off (open relay)";
File diff suppressed because one or more lines are too long
+107 -62
View File
@@ -2,6 +2,8 @@
#include "ip_ban.h"
#include "debug.h"
#include "config.h"
#include "db_ops.h"
#include "thread_pool.h"
#include <string.h>
#include <stdlib.h>
#include <pthread.h>
@@ -100,8 +102,8 @@ void ip_ban_init(void) {
DEBUG_LOG("IP ban table initialized (%d slots)", IP_BAN_TABLE_SIZE);
}
void ip_ban_load_from_db(sqlite3* db) {
if (!db || !g_initialized) return;
void ip_ban_load_from_db(void) {
if (!db_is_available() || !g_initialized) return;
// Create table if it doesn't exist (handles existing databases)
const char* create_sql =
@@ -117,20 +119,18 @@ void ip_ban_load_from_db(sqlite3* db) {
" total_failures INTEGER NOT NULL DEFAULT 0,"
" total_successes INTEGER NOT NULL DEFAULT 0,"
" first_seen INTEGER NOT NULL DEFAULT 0,"
" updated_at INTEGER NOT NULL DEFAULT (strftime('%s', 'now'))"
" updated_at INTEGER NOT NULL DEFAULT 0"
");";
char* err = NULL;
if (sqlite3_exec(db, create_sql, NULL, NULL, &err) != SQLITE_OK) {
DEBUG_ERROR("Failed to create ip_bans table: %s", err ? err : "unknown");
if (err) sqlite3_free(err);
if (db_exec_sql(create_sql) != 0) {
DEBUG_ERROR("Failed to create ip_bans table: %s", db_last_error());
return;
}
// Migration: Add idle_* columns if they don't exist (ignore errors if already exists)
sqlite3_exec(db, "ALTER TABLE ip_bans ADD COLUMN idle_failure_count INTEGER NOT NULL DEFAULT 0", NULL, NULL, NULL);
sqlite3_exec(db, "ALTER TABLE ip_bans ADD COLUMN idle_ban_count INTEGER NOT NULL DEFAULT 0", NULL, NULL, NULL);
sqlite3_exec(db, "ALTER TABLE ip_bans ADD COLUMN idle_banned_until INTEGER NOT NULL DEFAULT 0", NULL, NULL, NULL);
sqlite3_exec(db, "ALTER TABLE ip_bans ADD COLUMN idle_first_failure INTEGER NOT NULL DEFAULT 0", NULL, NULL, NULL);
(void)db_exec_sql("ALTER TABLE ip_bans ADD COLUMN idle_failure_count INTEGER NOT NULL DEFAULT 0");
(void)db_exec_sql("ALTER TABLE ip_bans ADD COLUMN idle_ban_count INTEGER NOT NULL DEFAULT 0");
(void)db_exec_sql("ALTER TABLE ip_bans ADD COLUMN idle_banned_until INTEGER NOT NULL DEFAULT 0");
(void)db_exec_sql("ALTER TABLE ip_bans ADD COLUMN idle_first_failure INTEGER NOT NULL DEFAULT 0");
const char* sql =
"SELECT ip, failure_count, ban_count, banned_until, first_failure,"
@@ -139,16 +139,16 @@ void ip_ban_load_from_db(sqlite3* db) {
" idle_failure_count, idle_ban_count, idle_banned_until, idle_first_failure"
" FROM ip_bans";
sqlite3_stmt* stmt;
if (sqlite3_prepare_v2(db, sql, -1, &stmt, NULL) != SQLITE_OK) {
db_stmt_t* stmt;
if (db_prepare(sql, &stmt) != DB_OK) {
DEBUG_ERROR("Failed to prepare ip_bans load query");
return;
}
int loaded = 0;
pthread_mutex_lock(&g_ban_mutex);
while (sqlite3_step(stmt) == SQLITE_ROW) {
const char* ip = (const char*)sqlite3_column_text(stmt, 0);
while (db_step_stmt(stmt) == DB_ROW) {
const char* ip = db_column_text_value(stmt, 0);
if (!ip) continue;
int idx = find_slot(ip);
@@ -158,25 +158,25 @@ void ip_ban_load_from_db(sqlite3* db) {
entry->state = IP_BAN_ACTIVE;
strncpy(entry->ip, ip, sizeof(entry->ip) - 1);
entry->ip[sizeof(entry->ip) - 1] = '\0';
entry->failure_count = sqlite3_column_int(stmt, 1);
entry->ban_count = sqlite3_column_int(stmt, 2);
entry->banned_until = (time_t)sqlite3_column_int64(stmt, 3);
entry->first_failure = (time_t)sqlite3_column_int64(stmt, 4);
entry->has_authed_successfully = sqlite3_column_int(stmt, 5);
entry->last_success_at = (time_t)sqlite3_column_int64(stmt, 6);
entry->total_connections = sqlite3_column_int(stmt, 7);
entry->total_failures = sqlite3_column_int(stmt, 8);
entry->total_successes = sqlite3_column_int(stmt, 9);
entry->first_seen = (time_t)sqlite3_column_int64(stmt, 10);
entry->failure_count = db_column_int_value(stmt, 1);
entry->ban_count = db_column_int_value(stmt, 2);
entry->banned_until = (time_t)db_column_int64_value(stmt, 3);
entry->first_failure = (time_t)db_column_int64_value(stmt, 4);
entry->has_authed_successfully = db_column_int_value(stmt, 5);
entry->last_success_at = (time_t)db_column_int64_value(stmt, 6);
entry->total_connections = db_column_int_value(stmt, 7);
entry->total_failures = db_column_int_value(stmt, 8);
entry->total_successes = db_column_int_value(stmt, 9);
entry->first_seen = (time_t)db_column_int64_value(stmt, 10);
// Load idle tracking fields (default to 0 if columns don't exist yet)
entry->idle_failure_count = sqlite3_column_int(stmt, 11);
entry->idle_ban_count = sqlite3_column_int(stmt, 12);
entry->idle_banned_until = (time_t)sqlite3_column_int64(stmt, 13);
entry->idle_first_failure = (time_t)sqlite3_column_int64(stmt, 14);
entry->idle_failure_count = db_column_int_value(stmt, 11);
entry->idle_ban_count = db_column_int_value(stmt, 12);
entry->idle_banned_until = (time_t)db_column_int64_value(stmt, 13);
entry->idle_first_failure = (time_t)db_column_int64_value(stmt, 14);
loaded++;
}
pthread_mutex_unlock(&g_ban_mutex);
sqlite3_finalize(stmt);
db_finalize_stmt(stmt);
// Count how many are still actively banned
time_t now = time(NULL);
@@ -193,24 +193,40 @@ void ip_ban_load_from_db(sqlite3* db) {
DEBUG_WARN("IP ban table loaded: %d IPs restored (%d still banned)", loaded, still_banned);
}
void ip_ban_save_to_db(sqlite3* db) {
if (!db || !g_initialized) return;
void ip_ban_save_to_db(void) {
if (!db_is_available() || !g_initialized) return;
const char* upsert_sql =
"INSERT OR REPLACE INTO ip_bans"
"INSERT INTO ip_bans"
" (ip, failure_count, ban_count, banned_until, first_failure,"
" has_authed_successfully, last_success_at, total_connections,"
" total_failures, total_successes, first_seen, updated_at,"
" idle_failure_count, idle_ban_count, idle_banned_until, idle_first_failure)"
" VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, strftime('%s', 'now'), ?, ?, ?, ?)";
" VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"
" ON CONFLICT(ip) DO UPDATE SET"
" failure_count=excluded.failure_count,"
" ban_count=excluded.ban_count,"
" banned_until=excluded.banned_until,"
" first_failure=excluded.first_failure,"
" has_authed_successfully=excluded.has_authed_successfully,"
" last_success_at=excluded.last_success_at,"
" total_connections=excluded.total_connections,"
" total_failures=excluded.total_failures,"
" total_successes=excluded.total_successes,"
" first_seen=excluded.first_seen,"
" updated_at=excluded.updated_at,"
" idle_failure_count=excluded.idle_failure_count,"
" idle_ban_count=excluded.idle_ban_count,"
" idle_banned_until=excluded.idle_banned_until,"
" idle_first_failure=excluded.idle_first_failure";
sqlite3_stmt* stmt;
if (sqlite3_prepare_v2(db, upsert_sql, -1, &stmt, NULL) != SQLITE_OK) {
db_stmt_t* stmt;
if (db_prepare(upsert_sql, &stmt) != DB_OK) {
DEBUG_ERROR("Failed to prepare ip_bans save query");
return;
}
sqlite3_exec(db, "BEGIN TRANSACTION", NULL, NULL, NULL);
(void)db_exec_sql("BEGIN TRANSACTION");
int saved = 0;
pthread_mutex_lock(&g_ban_mutex);
@@ -218,25 +234,26 @@ void ip_ban_save_to_db(sqlite3* db) {
ip_ban_entry_t* entry = &g_ban_table[i];
if (entry->state != IP_BAN_ACTIVE) continue;
sqlite3_reset(stmt);
sqlite3_bind_text(stmt, 1, entry->ip, -1, SQLITE_STATIC);
sqlite3_bind_int(stmt, 2, entry->failure_count);
sqlite3_bind_int(stmt, 3, entry->ban_count);
sqlite3_bind_int64(stmt, 4, (sqlite3_int64)entry->banned_until);
sqlite3_bind_int64(stmt, 5, (sqlite3_int64)entry->first_failure);
sqlite3_bind_int(stmt, 6, entry->has_authed_successfully);
sqlite3_bind_int64(stmt, 7, (sqlite3_int64)entry->last_success_at);
sqlite3_bind_int(stmt, 8, entry->total_connections);
sqlite3_bind_int(stmt, 9, entry->total_failures);
sqlite3_bind_int(stmt, 10, entry->total_successes);
sqlite3_bind_int64(stmt, 11, (sqlite3_int64)entry->first_seen);
(void)db_reset_stmt(stmt);
db_bind_text_param(stmt, 1, entry->ip);
db_bind_int_param(stmt, 2, entry->failure_count);
db_bind_int_param(stmt, 3, entry->ban_count);
db_bind_int64_param(stmt, 4, (long long)entry->banned_until);
db_bind_int64_param(stmt, 5, (long long)entry->first_failure);
db_bind_int_param(stmt, 6, entry->has_authed_successfully);
db_bind_int64_param(stmt, 7, (long long)entry->last_success_at);
db_bind_int_param(stmt, 8, entry->total_connections);
db_bind_int_param(stmt, 9, entry->total_failures);
db_bind_int_param(stmt, 10, entry->total_successes);
db_bind_int64_param(stmt, 11, (long long)entry->first_seen);
db_bind_int64_param(stmt, 12, (long long)time(NULL));
// Idle tracking fields
sqlite3_bind_int(stmt, 12, entry->idle_failure_count);
sqlite3_bind_int(stmt, 13, entry->idle_ban_count);
sqlite3_bind_int64(stmt, 14, (sqlite3_int64)entry->idle_banned_until);
sqlite3_bind_int64(stmt, 15, (sqlite3_int64)entry->idle_first_failure);
db_bind_int_param(stmt, 13, entry->idle_failure_count);
db_bind_int_param(stmt, 14, entry->idle_ban_count);
db_bind_int64_param(stmt, 15, (long long)entry->idle_banned_until);
db_bind_int64_param(stmt, 16, (long long)entry->idle_first_failure);
if (sqlite3_step(stmt) != SQLITE_DONE) {
if (db_step_stmt(stmt) != DB_DONE) {
DEBUG_WARN("Failed to save ip_ban entry for %s", entry->ip);
} else {
saved++;
@@ -244,32 +261,50 @@ void ip_ban_save_to_db(sqlite3* db) {
}
pthread_mutex_unlock(&g_ban_mutex);
sqlite3_finalize(stmt);
sqlite3_exec(db, "COMMIT", NULL, NULL, NULL);
db_finalize_stmt(stmt);
(void)db_exec_sql("COMMIT");
DEBUG_TRACE("IP ban table saved: %d entries written to DB", saved);
}
// Check if an IP is in the idle_ban_whitelist config (comma-separated list)
static int ip_is_whitelisted(const char* ip) {
if (!ip) {
return 0;
}
// Always trust loopback to avoid local test/dev self-bans.
if (strcmp(ip, "127.0.0.1") == 0 || strcmp(ip, "::1") == 0) {
return 1;
}
const char* whitelist = get_config_value("idle_ban_whitelist");
if (!whitelist || whitelist[0] == '\0') return 0;
if (!whitelist || whitelist[0] == '\0') {
if (whitelist) free((char*)whitelist);
return 0;
}
// Make a mutable copy to tokenize
char buf[1024];
strncpy(buf, whitelist, sizeof(buf) - 1);
buf[sizeof(buf) - 1] = '\0';
int is_match = 0;
char* token = strtok(buf, ",");
while (token) {
// Trim leading/trailing spaces
while (*token == ' ') token++;
char* end = token + strlen(token) - 1;
while (end > token && *end == ' ') { *end = '\0'; end--; }
if (strcmp(token, ip) == 0) return 1;
if (strcmp(token, ip) == 0) {
is_match = 1;
break;
}
token = strtok(NULL, ",");
}
return 0;
free((char*)whitelist);
return is_match;
}
int ip_ban_is_banned(const char* ip) {
@@ -538,7 +573,7 @@ int ip_ban_get_tracked_count(void) {
return count;
}
void ip_ban_log_stats(sqlite3* db) {
void ip_ban_log_stats(void) {
if (!g_initialized) return;
static time_t last_log = 0;
@@ -548,8 +583,18 @@ void ip_ban_log_stats(sqlite3* db) {
last_log = now;
// Save to DB every 5 minutes
if (db) {
ip_ban_save_to_db(db);
if (thread_pool_is_running()) {
thread_pool_job_t job;
memset(&job, 0, sizeof(job));
job.type = THREAD_POOL_JOB_IP_BAN_SAVE;
thread_pool_status_t rc = thread_pool_submit_write(&job, NULL);
if (rc != THREAD_POOL_STATUS_OK) {
DEBUG_WARN("Failed to queue IP_BAN_SAVE job (%d); saving synchronously", rc);
ip_ban_save_to_db();
}
} else {
ip_ban_save_to_db();
}
pthread_mutex_lock(&g_ban_mutex);
+3 -4
View File
@@ -15,7 +15,6 @@
// auth_fail_ban_duration_sec int default: 300 (initial ban duration, doubles each time)
#include <time.h>
#include <sqlite3.h>
// Maximum number of IPs tracked simultaneously (fixed-size, no malloc)
#define IP_BAN_TABLE_SIZE 4096
@@ -25,11 +24,11 @@ void ip_ban_init(void);
// Load ban state from the ip_bans database table.
// Call after ip_ban_init() and after the database is open.
void ip_ban_load_from_db(sqlite3* db);
void ip_ban_load_from_db(void);
// Save current ban state to the ip_bans database table.
// Called every 5 minutes from the maintenance timer.
void ip_ban_save_to_db(sqlite3* db);
void ip_ban_save_to_db(void);
// Check if an IP is currently banned.
// Returns 1 if banned (connection should be rejected), 0 if allowed.
@@ -57,7 +56,7 @@ void ip_ban_cleanup(void);
// Emit a periodic WARN-level log summary of banned IPs (every 5 minutes).
// Also saves state to DB.
void ip_ban_log_stats(sqlite3* db);
void ip_ban_log_stats(void);
// Get stats for logging/monitoring
int ip_ban_get_banned_count(void);
+799 -487
View File
File diff suppressed because it is too large Load Diff
+25 -7
View File
@@ -1,5 +1,5 @@
/*
* C-Relay Main Header - Version and Metadata Information
* C-Relay-PG Main Header - Version and Metadata Information
*
* This header contains version information and relay metadata.
* Version macros are auto-updated by the build system.
@@ -11,16 +11,16 @@
// Version information (auto-updated by build system)
// Using CRELAY_ prefix to avoid conflicts with nostr_core_lib VERSION macros
#define CRELAY_VERSION_MAJOR 1
#define CRELAY_VERSION_MINOR 2
#define CRELAY_VERSION_PATCH 52
#define CRELAY_VERSION "v1.2.52"
#define CRELAY_VERSION_MAJOR 2
#define CRELAY_VERSION_MINOR 1
#define CRELAY_VERSION_PATCH 10
#define CRELAY_VERSION "v2.1.10"
// Relay metadata (authoritative source for NIP-11 information)
#define RELAY_NAME "C-Relay"
#define RELAY_NAME "C-Relay-PG"
#define RELAY_DESCRIPTION "High-performance C Nostr relay with SQLite storage"
#define RELAY_CONTACT ""
#define RELAY_SOFTWARE "https://git.laantungir.net/laantungir/c-relay.git"
#define RELAY_SOFTWARE "https://git.laantungir.net/laantungir/c-relay-pg.git"
#define RELAY_VERSION CRELAY_VERSION // Use the same version as the build
#define SUPPORTED_NIPS "1,2,4,9,11,12,13,15,16,20,22,33,40,42,50,70"
#define LANGUAGE_TAGS ""
@@ -28,4 +28,22 @@
#define POSTING_POLICY ""
#define PAYMENTS_URL ""
// Forward declaration to avoid pulling cJSON headers into all includers.
typedef struct cJSON cJSON;
// Async REQ handler status used by websocket callback to defer EOSE until worker completion
#define HANDLE_REQ_ASYNC_PENDING (-2)
// Async EVENT storage split:
// - store_event_core(): worker-safe core DB write path
// returns 0 when inserted, 1 when handled without insert (duplicate/ephemeral), -1 on error
// - store_event_post_actions(): main-thread-only follow-up actions (monitoring/tags/WoT sync)
// - store_event(): legacy wrapper preserving original behavior for synchronous call sites
int store_event_core(cJSON* event);
void store_event_post_actions(cJSON* event);
int store_event(cJSON* event);
// Drains completed async REQ jobs and queues EVENT/EOSE on the lws service thread.
void process_req_async_completions(void);
#endif /* MAIN_H */
+33 -89
View File
@@ -7,7 +7,7 @@
/////////////////////////////////////////////////////////////////////////////////////////
#include <cjson/cJSON.h>
#include "debug.h"
#include <sqlite3.h>
#include "db_ops.h"
#include <string.h>
#include <stdlib.h>
#include <time.h>
@@ -21,10 +21,6 @@ int store_event(cJSON* event);
int delete_events_by_id(const char* requester_pubkey, cJSON* event_ids);
int delete_events_by_address(const char* requester_pubkey, cJSON* addresses, long deletion_timestamp);
// Global database variable
extern sqlite3* g_db;
// Handle NIP-09 deletion request event (kind 5)
int handle_deletion_request(cJSON* event, char* error_message, size_t error_size) {
if (!event) {
@@ -146,97 +142,72 @@ int handle_deletion_request(cJSON* event, char* error_message, size_t error_size
// Delete events by ID (with pubkey authorization)
int delete_events_by_id(const char* requester_pubkey, cJSON* event_ids) {
if (!g_db || !requester_pubkey || !event_ids || !cJSON_IsArray(event_ids)) {
if (!db_is_available() || !requester_pubkey || !event_ids || !cJSON_IsArray(event_ids)) {
return 0;
}
int deleted_count = 0;
cJSON* event_id = NULL;
cJSON_ArrayForEach(event_id, event_ids) {
if (!cJSON_IsString(event_id)) {
continue;
}
const char* id = cJSON_GetStringValue(event_id);
// First check if event exists and if requester is authorized
const char* check_sql = "SELECT pubkey FROM events WHERE id = ?";
sqlite3_stmt* check_stmt;
int rc = sqlite3_prepare_v2(g_db, check_sql, -1, &check_stmt, NULL);
if (rc != SQLITE_OK) {
char event_pubkey[65] = {0};
int exists = db_get_event_pubkey(id, event_pubkey, sizeof(event_pubkey));
if (exists <= 0) {
continue;
}
sqlite3_bind_text(check_stmt, 1, id, -1, SQLITE_STATIC);
if (sqlite3_step(check_stmt) == SQLITE_ROW) {
const char* event_pubkey = (char*)sqlite3_column_text(check_stmt, 0);
// Only delete if the requester is the author
if (event_pubkey && strcmp(event_pubkey, requester_pubkey) == 0) {
sqlite3_finalize(check_stmt);
// Delete the event
const char* delete_sql = "DELETE FROM events WHERE id = ? AND pubkey = ?";
sqlite3_stmt* delete_stmt;
rc = sqlite3_prepare_v2(g_db, delete_sql, -1, &delete_stmt, NULL);
if (rc == SQLITE_OK) {
sqlite3_bind_text(delete_stmt, 1, id, -1, SQLITE_STATIC);
sqlite3_bind_text(delete_stmt, 2, requester_pubkey, -1, SQLITE_STATIC);
if (sqlite3_step(delete_stmt) == SQLITE_DONE && sqlite3_changes(g_db) > 0) {
deleted_count++;
}
sqlite3_finalize(delete_stmt);
}
} else {
sqlite3_finalize(check_stmt);
char warning_msg[128];
snprintf(warning_msg, sizeof(warning_msg), "Unauthorized deletion attempt for event: %.16s...", id);
DEBUG_WARN(warning_msg);
// Only delete if the requester is the author
if (strcmp(event_pubkey, requester_pubkey) == 0) {
int changes = db_delete_event_by_id(id, requester_pubkey);
if (changes > 0) {
deleted_count += changes;
}
} else {
sqlite3_finalize(check_stmt);
char warning_msg[128];
snprintf(warning_msg, sizeof(warning_msg), "Unauthorized deletion attempt for event: %.16s...", id);
DEBUG_WARN(warning_msg);
}
}
return deleted_count;
}
// Delete events by addressable reference (kind:pubkey:d-identifier)
int delete_events_by_address(const char* requester_pubkey, cJSON* addresses, long deletion_timestamp) {
if (!g_db || !requester_pubkey || !addresses || !cJSON_IsArray(addresses)) {
if (!db_is_available() || !requester_pubkey || !addresses || !cJSON_IsArray(addresses)) {
return 0;
}
int deleted_count = 0;
cJSON* address = NULL;
cJSON_ArrayForEach(address, addresses) {
if (!cJSON_IsString(address)) {
continue;
}
const char* addr = cJSON_GetStringValue(address);
// Parse address format: kind:pubkey:d-identifier
char* addr_copy = strdup(addr);
if (!addr_copy) continue;
char* kind_str = strtok(addr_copy, ":");
char* pubkey_str = strtok(NULL, ":");
char* d_identifier = strtok(NULL, ":");
if (!kind_str || !pubkey_str) {
free(addr_copy);
continue;
}
int kind = atoi(kind_str);
// Only delete if the requester is the author
if (strcmp(pubkey_str, requester_pubkey) != 0) {
free(addr_copy);
@@ -245,42 +216,15 @@ int delete_events_by_address(const char* requester_pubkey, cJSON* addresses, lon
DEBUG_WARN(warning_msg);
continue;
}
// Build deletion query based on whether we have d-identifier
const char* delete_sql;
sqlite3_stmt* delete_stmt;
if (d_identifier && strlen(d_identifier) > 0) {
// Delete specific addressable event with d-tag
delete_sql = "DELETE FROM events WHERE kind = ? AND pubkey = ? AND created_at <= ? "
"AND json_extract(tags, '$[*]') LIKE '%[\"d\",\"' || ? || '\"]%'";
} else {
// Delete all events of this kind by this author up to deletion timestamp
delete_sql = "DELETE FROM events WHERE kind = ? AND pubkey = ? AND created_at <= ?";
int changes = db_delete_events_by_address(requester_pubkey, kind, d_identifier, deletion_timestamp);
if (changes > 0) {
deleted_count += changes;
}
int rc = sqlite3_prepare_v2(g_db, delete_sql, -1, &delete_stmt, NULL);
if (rc == SQLITE_OK) {
sqlite3_bind_int(delete_stmt, 1, kind);
sqlite3_bind_text(delete_stmt, 2, requester_pubkey, -1, SQLITE_STATIC);
sqlite3_bind_int64(delete_stmt, 3, deletion_timestamp);
if (d_identifier && strlen(d_identifier) > 0) {
sqlite3_bind_text(delete_stmt, 4, d_identifier, -1, SQLITE_STATIC);
}
if (sqlite3_step(delete_stmt) == SQLITE_DONE) {
int changes = sqlite3_changes(g_db);
if (changes > 0) {
deleted_count += changes;
}
}
sqlite3_finalize(delete_stmt);
}
free(addr_copy);
}
return deleted_count;
}
+71 -36
View File
@@ -8,7 +8,7 @@
#include <libwebsockets.h>
#include "../nostr_core_lib/cjson/cJSON.h"
#include "config.h"
#include "main.h"
// Forward declarations for configuration functions
const char* get_config_value(const char* key);
@@ -23,6 +23,67 @@ int get_config_bool(const char* key, int default_value);
#define HTTP_STATUS_INTERNAL_SERVER_ERROR 500
#define NIP11_CACHE_TTL_SEC 5
static pthread_mutex_t g_nip11_cache_mutex = PTHREAD_MUTEX_INITIALIZER;
static char* g_nip11_cached_json = NULL;
static size_t g_nip11_cached_json_len = 0;
static time_t g_nip11_cache_expires_at = 0;
cJSON* generate_relay_info_json();
static void invalidate_nip11_cache_locked(void) {
if (g_nip11_cached_json) {
free(g_nip11_cached_json);
g_nip11_cached_json = NULL;
}
g_nip11_cached_json_len = 0;
g_nip11_cache_expires_at = 0;
}
static char* get_nip11_json_cached_dup(size_t* out_len) {
time_t now = time(NULL);
pthread_mutex_lock(&g_nip11_cache_mutex);
if (g_nip11_cached_json && now < g_nip11_cache_expires_at) {
char* copy = strdup(g_nip11_cached_json);
if (copy && out_len) {
*out_len = g_nip11_cached_json_len;
}
pthread_mutex_unlock(&g_nip11_cache_mutex);
return copy;
}
pthread_mutex_unlock(&g_nip11_cache_mutex);
cJSON* info_json = generate_relay_info_json();
if (!info_json) {
return NULL;
}
char* fresh_json = cJSON_Print(info_json);
cJSON_Delete(info_json);
if (!fresh_json) {
return NULL;
}
size_t fresh_len = strlen(fresh_json);
pthread_mutex_lock(&g_nip11_cache_mutex);
invalidate_nip11_cache_locked();
g_nip11_cached_json = strdup(fresh_json);
if (g_nip11_cached_json) {
g_nip11_cached_json_len = fresh_len;
g_nip11_cache_expires_at = now + NIP11_CACHE_TTL_SEC;
}
pthread_mutex_unlock(&g_nip11_cache_mutex);
if (out_len) {
*out_len = fresh_len;
}
return fresh_json;
}
/////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////
// NIP-11 RELAY INFORMATION DOCUMENT
@@ -80,8 +141,9 @@ void init_relay_info() {
// Clean up relay information JSON objects
void cleanup_relay_info() {
// NIP-11 relay information is now generated dynamically from config table
// No cleanup needed - data is fetched directly from database when requested
pthread_mutex_lock(&g_nip11_cache_mutex);
invalidate_nip11_cache_locked();
pthread_mutex_unlock(&g_nip11_cache_mutex);
}
// Generate NIP-11 compliant JSON document
@@ -184,14 +246,14 @@ cJSON* generate_relay_info_json() {
cJSON_AddStringToObject(info, "software", relay_software);
free((char*)relay_software);
} else {
cJSON_AddStringToObject(info, "software", "https://git.laantungir.net/laantungir/c-relay.git");
cJSON_AddStringToObject(info, "software", RELAY_SOFTWARE);
}
if (relay_version && strlen(relay_version) > 0) {
cJSON_AddStringToObject(info, "version", relay_version);
free((char*)relay_version);
} else {
cJSON_AddStringToObject(info, "version", "0.2.0");
cJSON_AddStringToObject(info, "version", RELAY_VERSION);
}
// Add policies
@@ -348,9 +410,10 @@ int handle_nip11_http_request(struct lws* wsi, const char* accept_header) {
return -1; // Close connection
}
// Generate relay information JSON
cJSON* info_json = generate_relay_info_json();
if (!info_json) {
// Generate (or fetch cached) relay information JSON
size_t json_len = 0;
char* json_string = get_nip11_json_cached_dup(&json_len);
if (!json_string) {
DEBUG_ERROR("Failed to generate relay info JSON");
unsigned char buf[LWS_PRE + 256];
unsigned char *p = &buf[LWS_PRE];
@@ -373,34 +436,6 @@ int handle_nip11_http_request(struct lws* wsi, const char* accept_header) {
return -1;
}
char* json_string = cJSON_Print(info_json);
cJSON_Delete(info_json);
if (!json_string) {
DEBUG_ERROR("Failed to serialize relay info JSON");
unsigned char buf[LWS_PRE + 256];
unsigned char *p = &buf[LWS_PRE];
unsigned char *start = p;
unsigned char *end = &buf[sizeof(buf) - 1];
if (lws_add_http_header_status(wsi, HTTP_STATUS_INTERNAL_SERVER_ERROR, &p, end)) {
return -1;
}
if (lws_add_http_header_by_token(wsi, WSI_TOKEN_HTTP_CONTENT_TYPE, (unsigned char*)"text/plain", 10, &p, end)) {
return -1;
}
if (lws_add_http_header_content_length(wsi, 0, &p, end)) {
return -1;
}
if (lws_finalize_http_header(wsi, &p, end)) {
return -1;
}
lws_write(wsi, start, p - start, LWS_WRITE_HTTP_HEADERS);
return -1;
}
size_t json_len = strlen(json_string);
// Allocate session data to manage buffer lifetime across callbacks
struct nip11_session_data* session_data = malloc(sizeof(struct nip11_session_data));
if (!session_data) {
+249
View File
@@ -0,0 +1,249 @@
#ifndef PG_SCHEMA_H
#define PG_SCHEMA_H
#define EMBEDDED_PG_SCHEMA_VERSION "2"
static const char* const EMBEDDED_PG_SCHEMA_SQL =
"-- C-Relay-PG PostgreSQL Schema\n"
"-- Initial PostgreSQL backend schema\n"
"\n"
"BEGIN;\n"
"\n"
"CREATE TABLE IF NOT EXISTS schema_info (\n"
" key TEXT PRIMARY KEY,\n"
" value TEXT NOT NULL,\n"
" updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT\n"
");\n"
"\n"
"CREATE TABLE IF NOT EXISTS events (\n"
" id TEXT PRIMARY KEY,\n"
" pubkey TEXT NOT NULL,\n"
" created_at BIGINT NOT NULL,\n"
" kind INTEGER NOT NULL,\n"
" event_type TEXT NOT NULL CHECK (event_type IN ('regular', 'replaceable', 'ephemeral', 'addressable')),\n"
" content TEXT NOT NULL,\n"
" sig TEXT NOT NULL,\n"
" tags JSONB NOT NULL DEFAULT '[]'::jsonb,\n"
" d_tag_value TEXT GENERATED ALWAYS AS (\n"
" COALESCE(\n"
" NULLIF(\n"
" jsonb_path_query_first(tags, '$[*] ? (@[0] == \"d\")[1]') #>> '{}',\n"
" ''\n"
" ),\n"
" ''\n"
" )\n"
" ) STORED,\n"
" expires_at BIGINT,\n"
" event_json TEXT NOT NULL,\n"
" first_seen BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT\n"
");\n"
"\n"
"ALTER TABLE events\n"
" ADD COLUMN IF NOT EXISTS expires_at BIGINT;\n"
"\n"
"DO $$\n"
"BEGIN\n"
" IF NOT EXISTS (\n"
" SELECT 1\n"
" FROM information_schema.columns\n"
" WHERE table_schema = current_schema()\n"
" AND table_name = 'events'\n"
" AND column_name = 'd_tag_value'\n"
" ) THEN\n"
" ALTER TABLE events\n"
" ADD COLUMN d_tag_value TEXT GENERATED ALWAYS AS (\n"
" COALESCE(\n"
" NULLIF(\n"
" jsonb_path_query_first(tags, '$[*] ? (@[0] == \"d\")[1]') #>> '{}',\n"
" ''\n"
" ),\n"
" ''\n"
" )\n"
" ) STORED;\n"
" END IF;\n"
"END\n"
"$$;\n"
"\n"
"CREATE INDEX IF NOT EXISTS idx_events_pubkey ON events(pubkey);\n"
"CREATE INDEX IF NOT EXISTS idx_events_kind ON events(kind);\n"
"CREATE INDEX IF NOT EXISTS idx_events_created_at ON events(created_at DESC);\n"
"CREATE INDEX IF NOT EXISTS idx_events_event_type ON events(event_type);\n"
"CREATE INDEX IF NOT EXISTS idx_events_kind_created_at ON events(kind, created_at DESC);\n"
"CREATE INDEX IF NOT EXISTS idx_events_pubkey_created_at ON events(pubkey, created_at DESC);\n"
"CREATE INDEX IF NOT EXISTS idx_events_pubkey_kind ON events(pubkey, kind);\n"
"CREATE INDEX IF NOT EXISTS idx_events_tags_gin ON events USING GIN(tags);\n"
"CREATE INDEX IF NOT EXISTS idx_events_expires_at ON events(expires_at DESC) WHERE expires_at IS NOT NULL;\n"
"CREATE INDEX IF NOT EXISTS idx_events_d_tag_value ON events(d_tag_value) WHERE d_tag_value <> '';\n"
"CREATE UNIQUE INDEX IF NOT EXISTS uq_events_replaceable_pubkey_kind\n"
" ON events(pubkey, kind)\n"
" WHERE event_type = 'replaceable';\n"
"CREATE UNIQUE INDEX IF NOT EXISTS uq_events_addressable_pubkey_kind_dtag\n"
" ON events(pubkey, kind, d_tag_value)\n"
" WHERE event_type = 'addressable';\n"
"\n"
"-- Keep this table for compatibility with existing query paths during migration.\n"
"CREATE TABLE IF NOT EXISTS event_tags (\n"
" event_id TEXT NOT NULL REFERENCES events(id) ON DELETE CASCADE,\n"
" tag_name TEXT NOT NULL,\n"
" tag_value TEXT NOT NULL,\n"
" tag_index INTEGER NOT NULL DEFAULT 0,\n"
" PRIMARY KEY (event_id, tag_name, tag_value, tag_index)\n"
");\n"
"\n"
"CREATE INDEX IF NOT EXISTS idx_event_tags_lookup ON event_tags(tag_name, tag_value);\n"
"CREATE INDEX IF NOT EXISTS idx_event_tags_event ON event_tags(event_id);\n"
"CREATE INDEX IF NOT EXISTS idx_event_tags_value_name ON event_tags(tag_value, tag_name);\n"
"\n"
"CREATE OR REPLACE FUNCTION set_event_derived_fields()\n"
"RETURNS TRIGGER AS $$\n"
"DECLARE\n"
" expiration_value TEXT;\n"
"BEGIN\n"
" NEW.expires_at := NULL;\n"
"\n"
" SELECT tag->>1\n"
" INTO expiration_value\n"
" FROM jsonb_array_elements(NEW.tags) AS tag\n"
" WHERE jsonb_typeof(tag) = 'array'\n"
" AND jsonb_array_length(tag) >= 2\n"
" AND tag->>0 = 'expiration'\n"
" LIMIT 1;\n"
"\n"
" IF expiration_value IS NOT NULL AND expiration_value ~ '^[0-9]+$' THEN\n"
" NEW.expires_at := expiration_value::BIGINT;\n"
" END IF;\n"
"\n"
" RETURN NEW;\n"
"END;\n"
"$$ LANGUAGE plpgsql;\n"
"\n"
"CREATE OR REPLACE FUNCTION sync_event_tags_from_events()\n"
"RETURNS TRIGGER AS $$\n"
"BEGIN\n"
" DELETE FROM event_tags WHERE event_id = NEW.id;\n"
"\n"
" INSERT INTO event_tags (event_id, tag_name, tag_value, tag_index)\n"
" SELECT NEW.id,\n"
" tag->>0 AS tag_name,\n"
" tag->>1 AS tag_value,\n"
" (ord - 1)::INTEGER AS tag_index\n"
" FROM jsonb_array_elements(NEW.tags) WITH ORDINALITY AS expanded(tag, ord)\n"
" WHERE jsonb_typeof(tag) = 'array'\n"
" AND jsonb_array_length(tag) >= 2\n"
" AND COALESCE(tag->>0, '') <> ''\n"
" AND COALESCE(tag->>1, '') <> '';\n"
"\n"
" RETURN NEW;\n"
"END;\n"
"$$ LANGUAGE plpgsql;\n"
"\n"
"DROP TRIGGER IF EXISTS trg_events_set_derived_fields ON events;\n"
"CREATE TRIGGER trg_events_set_derived_fields\n"
"BEFORE INSERT OR UPDATE OF tags ON events\n"
"FOR EACH ROW\n"
"EXECUTE FUNCTION set_event_derived_fields();\n"
"\n"
"DROP TRIGGER IF EXISTS trg_events_sync_event_tags ON events;\n"
"CREATE TRIGGER trg_events_sync_event_tags\n"
"AFTER INSERT OR UPDATE OF tags ON events\n"
"FOR EACH ROW\n"
"EXECUTE FUNCTION sync_event_tags_from_events();\n"
"\n"
"CREATE TABLE IF NOT EXISTS relay_seckey (\n"
" id SMALLINT PRIMARY KEY DEFAULT 1,\n"
" private_key_hex TEXT NOT NULL CHECK (char_length(private_key_hex) = 64),\n"
" created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,\n"
" CONSTRAINT relay_seckey_singleton CHECK (id = 1)\n"
");\n"
"\n"
"CREATE TABLE IF NOT EXISTS auth_rules (\n"
" id BIGSERIAL PRIMARY KEY,\n"
" rule_type TEXT NOT NULL CHECK (rule_type IN ('whitelist', 'blacklist', 'rate_limit', 'auth_required', 'wot_whitelist')),\n"
" pattern_type TEXT NOT NULL CHECK (pattern_type IN ('pubkey', 'kind', 'ip', 'global', 'event_id', 'content', 'hash')),\n"
" pattern_value TEXT,\n"
" active INTEGER NOT NULL DEFAULT 1,\n"
" created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,\n"
" updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT\n"
");\n"
"\n"
"CREATE INDEX IF NOT EXISTS idx_auth_rules_pattern ON auth_rules(pattern_type, pattern_value);\n"
"CREATE INDEX IF NOT EXISTS idx_auth_rules_type ON auth_rules(rule_type);\n"
"CREATE INDEX IF NOT EXISTS idx_auth_rules_active ON auth_rules(active);\n"
"\n"
"CREATE TABLE IF NOT EXISTS config (\n"
" key TEXT PRIMARY KEY,\n"
" value TEXT NOT NULL,\n"
" data_type TEXT NOT NULL CHECK (data_type IN ('string', 'integer', 'boolean', 'json')),\n"
" description TEXT,\n"
" category TEXT DEFAULT 'general',\n"
" requires_restart INTEGER DEFAULT 0,\n"
" created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,\n"
" updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT\n"
");\n"
"\n"
"CREATE INDEX IF NOT EXISTS idx_config_category ON config(category);\n"
"CREATE INDEX IF NOT EXISTS idx_config_restart ON config(requires_restart);\n"
"CREATE INDEX IF NOT EXISTS idx_config_updated ON config(updated_at DESC);\n"
"\n"
"CREATE TABLE IF NOT EXISTS subscriptions (\n"
" id BIGSERIAL PRIMARY KEY,\n"
" subscription_id TEXT NOT NULL,\n"
" wsi_pointer TEXT NOT NULL,\n"
" client_ip TEXT NOT NULL,\n"
" event_type TEXT NOT NULL CHECK (event_type IN ('created', 'closed', 'expired', 'disconnected')),\n"
" filter_json TEXT,\n"
" events_sent INTEGER DEFAULT 0,\n"
" created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,\n"
" ended_at BIGINT,\n"
" duration BIGINT,\n"
" UNIQUE(subscription_id, wsi_pointer)\n"
");\n"
"\n"
"CREATE TABLE IF NOT EXISTS subscription_metrics (\n"
" id BIGSERIAL PRIMARY KEY,\n"
" date TEXT NOT NULL UNIQUE,\n"
" total_created INTEGER DEFAULT 0,\n"
" total_closed INTEGER DEFAULT 0,\n"
" total_events_broadcast INTEGER DEFAULT 0,\n"
" avg_duration DOUBLE PRECISION DEFAULT 0,\n"
" peak_concurrent INTEGER DEFAULT 0,\n"
" updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT\n"
");\n"
"\n"
"CREATE INDEX IF NOT EXISTS idx_subscriptions_id ON subscriptions(subscription_id);\n"
"CREATE INDEX IF NOT EXISTS idx_subscriptions_type ON subscriptions(event_type);\n"
"CREATE INDEX IF NOT EXISTS idx_subscriptions_created ON subscriptions(created_at DESC);\n"
"CREATE INDEX IF NOT EXISTS idx_subscriptions_client ON subscriptions(client_ip);\n"
"CREATE INDEX IF NOT EXISTS idx_subscriptions_wsi ON subscriptions(wsi_pointer);\n"
"CREATE INDEX IF NOT EXISTS idx_subscriptions_active_log ON subscriptions(event_type, ended_at, created_at DESC);\n"
"CREATE INDEX IF NOT EXISTS idx_subscription_metrics_date ON subscription_metrics(date DESC);\n"
"\n"
"CREATE TABLE IF NOT EXISTS ip_bans (\n"
" ip TEXT PRIMARY KEY,\n"
" failure_count INTEGER NOT NULL DEFAULT 0,\n"
" ban_count INTEGER NOT NULL DEFAULT 0,\n"
" banned_until BIGINT NOT NULL DEFAULT 0,\n"
" first_failure BIGINT NOT NULL DEFAULT 0,\n"
" has_authed_successfully INTEGER NOT NULL DEFAULT 0,\n"
" last_success_at BIGINT NOT NULL DEFAULT 0,\n"
" total_connections INTEGER NOT NULL DEFAULT 0,\n"
" total_failures INTEGER NOT NULL DEFAULT 0,\n"
" total_successes INTEGER NOT NULL DEFAULT 0,\n"
" first_seen BIGINT NOT NULL DEFAULT 0,\n"
" updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,\n"
" idle_failure_count INTEGER NOT NULL DEFAULT 0,\n"
" idle_ban_count INTEGER NOT NULL DEFAULT 0,\n"
" idle_banned_until BIGINT NOT NULL DEFAULT 0,\n"
" idle_first_failure BIGINT NOT NULL DEFAULT 0\n"
");\n"
"\n"
"INSERT INTO schema_info(key, value, updated_at)\n"
"VALUES ('version', '2', EXTRACT(EPOCH FROM NOW())::BIGINT)\n"
"ON CONFLICT (key) DO UPDATE SET\n"
" value = EXCLUDED.value,\n"
" updated_at = EXCLUDED.updated_at;\n"
"\n"
"COMMIT;\n"
;
#endif // PG_SCHEMA_H
+240
View File
@@ -0,0 +1,240 @@
-- C-Relay-PG PostgreSQL Schema
-- Initial PostgreSQL backend schema
BEGIN;
CREATE TABLE IF NOT EXISTS schema_info (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
);
CREATE TABLE IF NOT EXISTS events (
id TEXT PRIMARY KEY,
pubkey TEXT NOT NULL,
created_at BIGINT NOT NULL,
kind INTEGER NOT NULL,
event_type TEXT NOT NULL CHECK (event_type IN ('regular', 'replaceable', 'ephemeral', 'addressable')),
content TEXT NOT NULL,
sig TEXT NOT NULL,
tags JSONB NOT NULL DEFAULT '[]'::jsonb,
d_tag_value TEXT GENERATED ALWAYS AS (
COALESCE(
NULLIF(
jsonb_path_query_first(tags, '$[*] ? (@[0] == "d")[1]') #>> '{}',
''
),
''
)
) STORED,
expires_at BIGINT,
event_json TEXT NOT NULL,
first_seen BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
);
ALTER TABLE events
ADD COLUMN IF NOT EXISTS expires_at BIGINT;
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1
FROM information_schema.columns
WHERE table_schema = current_schema()
AND table_name = 'events'
AND column_name = 'd_tag_value'
) THEN
ALTER TABLE events
ADD COLUMN d_tag_value TEXT GENERATED ALWAYS AS (
COALESCE(
NULLIF(
jsonb_path_query_first(tags, '$[*] ? (@[0] == "d")[1]') #>> '{}',
''
),
''
)
) STORED;
END IF;
END
$$;
CREATE INDEX IF NOT EXISTS idx_events_pubkey ON events(pubkey);
CREATE INDEX IF NOT EXISTS idx_events_kind ON events(kind);
CREATE INDEX IF NOT EXISTS idx_events_created_at ON events(created_at DESC);
CREATE INDEX IF NOT EXISTS idx_events_event_type ON events(event_type);
CREATE INDEX IF NOT EXISTS idx_events_kind_created_at ON events(kind, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_events_pubkey_created_at ON events(pubkey, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_events_pubkey_kind ON events(pubkey, kind);
CREATE INDEX IF NOT EXISTS idx_events_tags_gin ON events USING GIN(tags);
CREATE INDEX IF NOT EXISTS idx_events_expires_at ON events(expires_at DESC) WHERE expires_at IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_events_d_tag_value ON events(d_tag_value) WHERE d_tag_value <> '';
CREATE UNIQUE INDEX IF NOT EXISTS uq_events_replaceable_pubkey_kind
ON events(pubkey, kind)
WHERE event_type = 'replaceable';
CREATE UNIQUE INDEX IF NOT EXISTS uq_events_addressable_pubkey_kind_dtag
ON events(pubkey, kind, d_tag_value)
WHERE event_type = 'addressable';
-- Keep this table for compatibility with existing query paths during migration.
CREATE TABLE IF NOT EXISTS event_tags (
event_id TEXT NOT NULL REFERENCES events(id) ON DELETE CASCADE,
tag_name TEXT NOT NULL,
tag_value TEXT NOT NULL,
tag_index INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (event_id, tag_name, tag_value, tag_index)
);
CREATE INDEX IF NOT EXISTS idx_event_tags_lookup ON event_tags(tag_name, tag_value);
CREATE INDEX IF NOT EXISTS idx_event_tags_event ON event_tags(event_id);
CREATE INDEX IF NOT EXISTS idx_event_tags_value_name ON event_tags(tag_value, tag_name);
CREATE OR REPLACE FUNCTION set_event_derived_fields()
RETURNS TRIGGER AS $$
DECLARE
expiration_value TEXT;
BEGIN
NEW.expires_at := NULL;
SELECT tag->>1
INTO expiration_value
FROM jsonb_array_elements(NEW.tags) AS tag
WHERE jsonb_typeof(tag) = 'array'
AND jsonb_array_length(tag) >= 2
AND tag->>0 = 'expiration'
LIMIT 1;
IF expiration_value IS NOT NULL AND expiration_value ~ '^[0-9]+$' THEN
NEW.expires_at := expiration_value::BIGINT;
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION sync_event_tags_from_events()
RETURNS TRIGGER AS $$
BEGIN
DELETE FROM event_tags WHERE event_id = NEW.id;
INSERT INTO event_tags (event_id, tag_name, tag_value, tag_index)
SELECT NEW.id,
tag->>0 AS tag_name,
tag->>1 AS tag_value,
(ord - 1)::INTEGER AS tag_index
FROM jsonb_array_elements(NEW.tags) WITH ORDINALITY AS expanded(tag, ord)
WHERE jsonb_typeof(tag) = 'array'
AND jsonb_array_length(tag) >= 2
AND COALESCE(tag->>0, '') <> ''
AND COALESCE(tag->>1, '') <> '';
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
DROP TRIGGER IF EXISTS trg_events_set_derived_fields ON events;
CREATE TRIGGER trg_events_set_derived_fields
BEFORE INSERT OR UPDATE OF tags ON events
FOR EACH ROW
EXECUTE FUNCTION set_event_derived_fields();
DROP TRIGGER IF EXISTS trg_events_sync_event_tags ON events;
CREATE TRIGGER trg_events_sync_event_tags
AFTER INSERT OR UPDATE OF tags ON events
FOR EACH ROW
EXECUTE FUNCTION sync_event_tags_from_events();
CREATE TABLE IF NOT EXISTS relay_seckey (
id SMALLINT PRIMARY KEY DEFAULT 1,
private_key_hex TEXT NOT NULL CHECK (char_length(private_key_hex) = 64),
created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,
CONSTRAINT relay_seckey_singleton CHECK (id = 1)
);
CREATE TABLE IF NOT EXISTS auth_rules (
id BIGSERIAL PRIMARY KEY,
rule_type TEXT NOT NULL CHECK (rule_type IN ('whitelist', 'blacklist', 'rate_limit', 'auth_required', 'wot_whitelist')),
pattern_type TEXT NOT NULL CHECK (pattern_type IN ('pubkey', 'kind', 'ip', 'global', 'event_id', 'content', 'hash')),
pattern_value TEXT,
active INTEGER NOT NULL DEFAULT 1,
created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,
updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
);
CREATE INDEX IF NOT EXISTS idx_auth_rules_pattern ON auth_rules(pattern_type, pattern_value);
CREATE INDEX IF NOT EXISTS idx_auth_rules_type ON auth_rules(rule_type);
CREATE INDEX IF NOT EXISTS idx_auth_rules_active ON auth_rules(active);
CREATE TABLE IF NOT EXISTS config (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
data_type TEXT NOT NULL CHECK (data_type IN ('string', 'integer', 'boolean', 'json')),
description TEXT,
category TEXT DEFAULT 'general',
requires_restart INTEGER DEFAULT 0,
created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,
updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
);
CREATE INDEX IF NOT EXISTS idx_config_category ON config(category);
CREATE INDEX IF NOT EXISTS idx_config_restart ON config(requires_restart);
CREATE INDEX IF NOT EXISTS idx_config_updated ON config(updated_at DESC);
CREATE TABLE IF NOT EXISTS subscriptions (
id BIGSERIAL PRIMARY KEY,
subscription_id TEXT NOT NULL,
wsi_pointer TEXT NOT NULL,
client_ip TEXT NOT NULL,
event_type TEXT NOT NULL CHECK (event_type IN ('created', 'closed', 'expired', 'disconnected')),
filter_json TEXT,
events_sent INTEGER DEFAULT 0,
created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,
ended_at BIGINT,
duration BIGINT,
UNIQUE(subscription_id, wsi_pointer)
);
CREATE TABLE IF NOT EXISTS subscription_metrics (
id BIGSERIAL PRIMARY KEY,
date TEXT NOT NULL UNIQUE,
total_created INTEGER DEFAULT 0,
total_closed INTEGER DEFAULT 0,
total_events_broadcast INTEGER DEFAULT 0,
avg_duration DOUBLE PRECISION DEFAULT 0,
peak_concurrent INTEGER DEFAULT 0,
updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
);
CREATE INDEX IF NOT EXISTS idx_subscriptions_id ON subscriptions(subscription_id);
CREATE INDEX IF NOT EXISTS idx_subscriptions_type ON subscriptions(event_type);
CREATE INDEX IF NOT EXISTS idx_subscriptions_created ON subscriptions(created_at DESC);
CREATE INDEX IF NOT EXISTS idx_subscriptions_client ON subscriptions(client_ip);
CREATE INDEX IF NOT EXISTS idx_subscriptions_wsi ON subscriptions(wsi_pointer);
CREATE INDEX IF NOT EXISTS idx_subscriptions_active_log ON subscriptions(event_type, ended_at, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_subscription_metrics_date ON subscription_metrics(date DESC);
CREATE TABLE IF NOT EXISTS ip_bans (
ip TEXT PRIMARY KEY,
failure_count INTEGER NOT NULL DEFAULT 0,
ban_count INTEGER NOT NULL DEFAULT 0,
banned_until BIGINT NOT NULL DEFAULT 0,
first_failure BIGINT NOT NULL DEFAULT 0,
has_authed_successfully INTEGER NOT NULL DEFAULT 0,
last_success_at BIGINT NOT NULL DEFAULT 0,
total_connections INTEGER NOT NULL DEFAULT 0,
total_failures INTEGER NOT NULL DEFAULT 0,
total_successes INTEGER NOT NULL DEFAULT 0,
first_seen BIGINT NOT NULL DEFAULT 0,
updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,
idle_failure_count INTEGER NOT NULL DEFAULT 0,
idle_ban_count INTEGER NOT NULL DEFAULT 0,
idle_banned_until BIGINT NOT NULL DEFAULT 0,
idle_first_failure BIGINT NOT NULL DEFAULT 0
);
INSERT INTO schema_info(key, value, updated_at)
VALUES ('version', '2', EXTRACT(EPOCH FROM NOW())::BIGINT)
ON CONFLICT (key) DO UPDATE SET
value = EXCLUDED.value,
updated_at = EXCLUDED.updated_at;
COMMIT;
+42 -97
View File
@@ -1,5 +1,5 @@
/*
* C-Relay Request Validator - Integrated Authentication System
* C-Relay-PG Request Validator - Integrated Authentication System
*
* Provides complete request validation including:
* - Protocol validation via nostr_core_lib (signatures, pubkey extraction,
@@ -17,17 +17,13 @@
#include "../nostr_core_lib/nostr_core/utils.h"
#include "debug.h" // C-relay debug system
#include "config.h" // C-relay configuration system
#include <sqlite3.h>
#include "db_ops.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <strings.h>
#include <time.h>
// External references to C-relay global state
extern sqlite3* g_db;
extern char g_database_path[512];
// Forward declaration for C-relay event storage function
extern int store_event(cJSON* event);
@@ -133,6 +129,14 @@ typedef struct {
static nip42_challenge_manager_t g_challenge_manager = {0};
static int g_validator_initialized = 0;
typedef struct {
int has_active_whitelist_rules;
time_t last_refresh;
} auth_rules_fast_cache_t;
static auth_rules_fast_cache_t g_auth_rules_fast_cache = {0};
#define AUTH_RULES_CACHE_TTL_SEC 5
// Last rule violation details for status code mapping
struct {
char violation_type[100]; // "pubkey_blacklist", "hash_blacklist",
@@ -304,9 +308,11 @@ int nostr_validate_unified_request(const char* json_string, size_t json_length)
const char* admin_pubkey = get_config_value("admin_pubkey");
if (admin_pubkey && strcmp(event_pubkey, admin_pubkey) == 0) {
// Valid admin event - bypass remaining validation
free((char*)admin_pubkey);
cJSON_Delete(event);
return NOSTR_SUCCESS;
}
if (admin_pubkey) free((char*)admin_pubkey);
// Not from admin - continue with normal validation
}
@@ -504,10 +510,10 @@ void nostr_request_result_free_file_data(nostr_request_result_t *result) {
/**
* Force cache refresh - cache no longer exists, function kept for compatibility
* Force cache refresh for auth rule fast cache.
*/
void nostr_request_validator_force_cache_refresh(void) {
// Cache no longer exists - direct database queries are used
g_auth_rules_fast_cache.last_refresh = 0;
}
/**
@@ -522,118 +528,57 @@ static int reload_auth_config(void) {
// Note: Blossom protocol validation removed - C-relay uses standard Nostr events only
static int has_active_whitelist_rules_cached(void) {
time_t now = time(NULL);
if (g_auth_rules_fast_cache.last_refresh == 0 ||
(now - g_auth_rules_fast_cache.last_refresh) >= AUTH_RULES_CACHE_TTL_SEC) {
g_auth_rules_fast_cache.has_active_whitelist_rules =
(db_count_active_whitelist_rules() > 0) ? 1 : 0;
g_auth_rules_fast_cache.last_refresh = now;
}
return g_auth_rules_fast_cache.has_active_whitelist_rules;
}
/**
* Check database authentication rules for the request
* Implements the 6-step rule evaluation engine from AUTH_API.md
*/
int check_database_auth_rules(const char *pubkey, const char *operation __attribute__((unused)),
const char *resource_hash) {
sqlite3 *db = NULL;
sqlite3_stmt *stmt = NULL;
int rc;
DEBUG_TRACE("Checking auth rules for pubkey: %s", pubkey);
if (!pubkey) {
return NOSTR_ERROR_INVALID_INPUT;
}
// Open database using global database path
if (strlen(g_database_path) == 0) {
return NOSTR_SUCCESS; // Default allow on DB error
}
rc = sqlite3_open_v2(g_database_path, &db, SQLITE_OPEN_READONLY, NULL);
if (rc != SQLITE_OK) {
return NOSTR_SUCCESS; // Default allow on DB error
}
// Step 1: Check pubkey blacklist (highest priority)
const char *blacklist_sql =
"SELECT rule_type FROM auth_rules WHERE rule_type = "
"'blacklist' AND pattern_type = 'pubkey' AND pattern_value = ? AND active = 1 LIMIT 1";
DEBUG_TRACE("Blacklist SQL: %s", blacklist_sql);
rc = sqlite3_prepare_v2(db, blacklist_sql, -1, &stmt, NULL);
if (rc == SQLITE_OK) {
sqlite3_bind_text(stmt, 1, pubkey, -1, SQLITE_STATIC);
int step_result = sqlite3_step(stmt);
DEBUG_TRACE("Blacklist query result: %s", step_result == SQLITE_ROW ? "FOUND" : "NOT_FOUND");
if (step_result == SQLITE_ROW) {
DEBUG_TRACE("BLACKLIST HIT: Denying access for pubkey: %s", pubkey);
// Set specific violation details for status code mapping
strcpy(g_last_rule_violation.violation_type, "pubkey_blacklist");
sprintf(g_last_rule_violation.reason, "Public key blacklisted");
sqlite3_finalize(stmt);
sqlite3_close(db);
return NOSTR_ERROR_AUTH_REQUIRED;
}
sqlite3_finalize(stmt);
if (db_is_pubkey_blacklisted(pubkey)) {
DEBUG_TRACE("BLACKLIST HIT: Denying access for pubkey: %s", pubkey);
strcpy(g_last_rule_violation.violation_type, "pubkey_blacklist");
sprintf(g_last_rule_violation.reason, "Public key blacklisted");
return NOSTR_ERROR_AUTH_REQUIRED;
}
// Step 2: Check hash blacklist
if (resource_hash) {
const char *hash_blacklist_sql =
"SELECT rule_type FROM auth_rules WHERE rule_type = "
"'blacklist' AND pattern_type = 'hash' AND pattern_value = ? AND active = 1 LIMIT 1";
rc = sqlite3_prepare_v2(db, hash_blacklist_sql, -1, &stmt, NULL);
if (rc == SQLITE_OK) {
sqlite3_bind_text(stmt, 1, resource_hash, -1, SQLITE_STATIC);
if (sqlite3_step(stmt) == SQLITE_ROW) {
// Set specific violation details for status code mapping
strcpy(g_last_rule_violation.violation_type, "hash_blacklist");
sprintf(g_last_rule_violation.reason, "File hash blacklisted");
sqlite3_finalize(stmt);
sqlite3_close(db);
return NOSTR_ERROR_AUTH_REQUIRED;
}
sqlite3_finalize(stmt);
}
if (resource_hash && db_is_hash_blacklisted(resource_hash)) {
strcpy(g_last_rule_violation.violation_type, "hash_blacklist");
sprintf(g_last_rule_violation.reason, "File hash blacklisted");
return NOSTR_ERROR_AUTH_REQUIRED;
}
// Step 3: Check pubkey whitelist (includes wot_whitelist)
const char *whitelist_sql =
"SELECT rule_type FROM auth_rules WHERE rule_type IN "
"('whitelist', 'wot_whitelist') AND pattern_type = 'pubkey' AND pattern_value = ? AND active = 1 LIMIT 1";
rc = sqlite3_prepare_v2(db, whitelist_sql, -1, &stmt, NULL);
if (rc == SQLITE_OK) {
sqlite3_bind_text(stmt, 1, pubkey, -1, SQLITE_STATIC);
if (sqlite3_step(stmt) == SQLITE_ROW) {
sqlite3_finalize(stmt);
sqlite3_close(db);
return NOSTR_SUCCESS; // Allow whitelisted pubkey
}
sqlite3_finalize(stmt);
if (db_is_pubkey_whitelisted(pubkey)) {
return NOSTR_SUCCESS; // Allow whitelisted pubkey
}
// Step 4: Check if any whitelist rules exist - if yes, deny by default (includes wot_whitelist)
const char *whitelist_exists_sql =
"SELECT COUNT(*) FROM auth_rules WHERE rule_type IN ('whitelist', 'wot_whitelist') "
"AND pattern_type = 'pubkey' AND active = 1 LIMIT 1";
rc = sqlite3_prepare_v2(db, whitelist_exists_sql, -1, &stmt, NULL);
if (rc == SQLITE_OK) {
if (sqlite3_step(stmt) == SQLITE_ROW) {
int whitelist_count = sqlite3_column_int(stmt, 0);
if (whitelist_count > 0) {
// Set specific violation details for status code mapping
strcpy(g_last_rule_violation.violation_type, "whitelist_violation");
strcpy(g_last_rule_violation.reason,
"Public key not whitelisted for this operation");
sqlite3_finalize(stmt);
sqlite3_close(db);
return NOSTR_ERROR_AUTH_REQUIRED;
}
}
sqlite3_finalize(stmt);
// Step 4: If any whitelist rules exist, deny by default
if (has_active_whitelist_rules_cached()) {
strcpy(g_last_rule_violation.violation_type, "whitelist_violation");
strcpy(g_last_rule_violation.reason,
"Public key not whitelisted for this operation");
return NOSTR_ERROR_AUTH_REQUIRED;
}
sqlite3_close(db);
return NOSTR_SUCCESS; // Default allow if no restrictive rules matched
}
+95
View File
@@ -0,0 +1,95 @@
#ifndef SQLITE_DB_OPS_H
#define SQLITE_DB_OPS_H
#include "db_ops.h"
typedef struct sqlite_db_stmt sqlite_db_stmt_t;
int sqlite_db_init(const char* connection_string);
void sqlite_db_close(void);
int sqlite_db_is_available(void);
const char* sqlite_db_last_error(void);
const char* sqlite_db_get_database_path(void);
int sqlite_db_set_thread_connection(void* connection);
void sqlite_db_clear_thread_connection(void);
int sqlite_db_open_worker_connection(const char* db_path, void** out_connection);
void sqlite_db_close_worker_connection(void* connection);
int sqlite_db_prepare(const char* sql, sqlite_db_stmt_t** out_stmt);
int sqlite_db_bind_text_param(sqlite_db_stmt_t* stmt, int index, const char* value);
int sqlite_db_bind_int_param(sqlite_db_stmt_t* stmt, int index, int value);
int sqlite_db_bind_int64_param(sqlite_db_stmt_t* stmt, int index, long long value);
int sqlite_db_step_stmt(sqlite_db_stmt_t* stmt);
int sqlite_db_reset_stmt(sqlite_db_stmt_t* stmt);
const char* sqlite_db_column_text_value(sqlite_db_stmt_t* stmt, int col);
int sqlite_db_column_int_value(sqlite_db_stmt_t* stmt, int col);
long long sqlite_db_column_int64_value(sqlite_db_stmt_t* stmt, int col);
double sqlite_db_column_double_value(sqlite_db_stmt_t* stmt, int col);
void sqlite_db_finalize_stmt(sqlite_db_stmt_t* stmt);
int sqlite_db_log_subscription_created(const char* sub_id, const char* wsi_ptr,
const char* client_ip, const char* filter_json);
int sqlite_db_log_subscription_closed(const char* sub_id, const char* client_ip);
int sqlite_db_log_subscription_disconnected(const char* client_ip);
int sqlite_db_update_subscription_events_sent(const char* sub_id, int events_sent);
int sqlite_db_cleanup_orphaned_subscriptions(void);
int sqlite_db_get_event_pubkey(const char* event_id, char* pubkey_out, size_t pubkey_out_size);
int sqlite_db_delete_event_by_id(const char* event_id, const char* requester_pubkey);
int sqlite_db_delete_events_by_address(const char* pubkey, int kind,
const char* d_tag, long before_timestamp);
int sqlite_db_is_pubkey_blacklisted(const char* pubkey);
int sqlite_db_is_hash_blacklisted(const char* resource_hash);
int sqlite_db_is_pubkey_whitelisted(const char* pubkey);
int sqlite_db_count_active_whitelist_rules(void);
int sqlite_db_count_with_sql(const char* sql, const char** bind_params, int bind_param_count, int* out_count);
char* sqlite_db_execute_readonly_query_json(const char* query, const char* request_id,
char* error_message, size_t error_size,
int max_rows, int timeout_ms);
int sqlite_db_get_total_event_count_ll(long long* out_count);
int sqlite_db_get_event_count_since(time_t cutoff, long long* out_count);
cJSON* sqlite_db_get_event_kind_distribution_rows(long long* out_total_events);
cJSON* sqlite_db_get_top_pubkeys_rows(int limit);
cJSON* sqlite_db_get_subscription_details_rows(void);
cJSON* sqlite_db_get_all_config_rows(void);
char* sqlite_db_get_config_value_dup(const char* key);
int sqlite_db_set_config_value_full(const char* key, const char* value, const char* data_type,
const char* description, const char* category, int requires_restart);
int sqlite_db_update_config_value_only(const char* key, const char* value);
int sqlite_db_upsert_config_value(const char* key, const char* value, const char* data_type);
int sqlite_db_store_relay_private_key_hex(const char* relay_privkey_hex);
char* sqlite_db_get_relay_private_key_hex_dup(void);
int sqlite_db_store_config_event(const cJSON* event);
int sqlite_db_insert_event_with_json(const char* id, const char* pubkey, long long created_at,
int kind, const char* event_type, const char* content,
const char* sig, const char* tags_json, const char* event_json,
int* out_step_rc, int* out_extended_errcode);
int sqlite_db_get_event_time_bounds(long long* out_min_created_at, long long* out_max_created_at);
int sqlite_db_event_id_exists(const char* event_id, int* out_exists);
cJSON* sqlite_db_retrieve_event_by_id(const char* event_id);
char* sqlite_db_get_latest_event_pubkey_for_kind_dup(int kind);
int sqlite_db_get_config_row_count(int* out_count);
int sqlite_db_store_event_tags_cjson(const char* event_id, const cJSON* tags);
int sqlite_db_populate_event_tags_from_existing(void);
int sqlite_db_add_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
int sqlite_db_remove_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
int sqlite_db_delete_wot_whitelist_rules(void);
int sqlite_db_count_wot_whitelist_rules(void);
int sqlite_db_table_exists(const char* table_name, int* out_exists);
char* sqlite_db_get_schema_version_dup(void);
int sqlite_db_exec_sql(const char* sql);
int sqlite_db_wal_checkpoint_passive(void);
int sqlite_db_wal_checkpoint_truncate(void);
#endif // SQLITE_DB_OPS_H
+160 -117
View File
@@ -1,7 +1,8 @@
#define _GNU_SOURCE
#include <cjson/cJSON.h>
#include "debug.h"
#include <sqlite3.h>
#include "db_ops.h"
#include "thread_pool.h"
#include <string.h>
#include <stdlib.h>
#include <time.h>
@@ -10,6 +11,34 @@
#include <libwebsockets.h>
#include "subscriptions.h"
static void free_sub_log_created_payload_local(thread_pool_sub_log_created_payload_t* payload) {
if (!payload) return;
free(payload->sub_id);
free(payload->wsi_ptr);
free(payload->client_ip);
free(payload->filter_json);
free(payload);
}
static void free_sub_log_closed_payload_local(thread_pool_sub_log_closed_payload_t* payload) {
if (!payload) return;
free(payload->sub_id);
free(payload->client_ip);
free(payload);
}
static void free_sub_log_disconnected_payload_local(thread_pool_sub_log_disconnected_payload_t* payload) {
if (!payload) return;
free(payload->client_ip);
free(payload);
}
static void free_sub_update_events_payload_local(thread_pool_sub_update_events_payload_t* payload) {
if (!payload) return;
free(payload->sub_id);
free(payload);
}
// Forward declarations for logging functions
// Forward declarations for configuration functions
@@ -28,9 +57,6 @@ int validate_search_term(const char* search_term, char* error_message, size_t er
// Forward declaration for monitoring function
void monitoring_on_subscription_change(void);
// Global database variable
extern sqlite3* g_db;
// Configuration functions from config.c
extern int get_config_bool(const char* key, int default_value);
@@ -991,21 +1017,21 @@ int has_subscriptions_for_kind(int event_kind) {
// Log subscription creation to database
void log_subscription_created(const subscription_t* sub) {
if (!g_db || !sub) return;
if (!sub) return;
// Convert wsi pointer to string
char wsi_str[32];
snprintf(wsi_str, sizeof(wsi_str), "%p", (void*)sub->wsi);
// Create filter JSON for logging
char* filter_json = NULL;
if (sub->filters) {
cJSON* filters_array = cJSON_CreateArray();
subscription_filter_t* filter = sub->filters;
while (filter) {
cJSON* filter_obj = cJSON_CreateObject();
if (filter->kinds) {
cJSON_AddItemToObject(filter_obj, "kinds", cJSON_Duplicate(filter->kinds, 1));
}
@@ -1034,99 +1060,119 @@ void log_subscription_created(const subscription_t* sub) {
}
cJSON_Delete(tags_obj);
}
cJSON_AddItemToArray(filters_array, filter_obj);
filter = filter->next;
}
filter_json = cJSON_Print(filters_array);
cJSON_Delete(filters_array);
}
// Use INSERT OR REPLACE to handle duplicates automatically
const char* sql =
"INSERT OR REPLACE INTO subscriptions (subscription_id, wsi_pointer, client_ip, event_type, filter_json) "
"VALUES (?, ?, ?, 'created', ?)";
sqlite3_stmt* stmt;
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
if (rc == SQLITE_OK) {
sqlite3_bind_text(stmt, 1, sub->id, -1, SQLITE_STATIC);
sqlite3_bind_text(stmt, 2, wsi_str, -1, SQLITE_TRANSIENT);
sqlite3_bind_text(stmt, 3, sub->client_ip, -1, SQLITE_STATIC);
sqlite3_bind_text(stmt, 4, filter_json ? filter_json : "[]", -1, SQLITE_TRANSIENT);
sqlite3_step(stmt);
sqlite3_finalize(stmt);
if (!thread_pool_is_running()) {
db_log_subscription_created(sub->id, wsi_str, sub->client_ip, filter_json ? filter_json : "[]");
if (filter_json) free(filter_json);
return;
}
thread_pool_sub_log_created_payload_t* payload = calloc(1, sizeof(*payload));
if (!payload) {
if (filter_json) free(filter_json);
return;
}
payload->sub_id = strdup(sub->id);
payload->wsi_ptr = strdup(wsi_str);
payload->client_ip = strdup(sub->client_ip);
payload->filter_json = strdup(filter_json ? filter_json : "[]");
if (!payload->sub_id || !payload->wsi_ptr || !payload->client_ip || !payload->filter_json) {
free(payload->sub_id);
free(payload->wsi_ptr);
free(payload->client_ip);
free(payload->filter_json);
free(payload);
if (filter_json) free(filter_json);
return;
}
thread_pool_job_t job;
memset(&job, 0, sizeof(job));
job.type = THREAD_POOL_JOB_LOG_SUB_CREATED;
job.payload = payload;
job.payload_free = (thread_pool_payload_free_cb)free_sub_log_created_payload_local;
thread_pool_status_t submit_rc = thread_pool_submit_write(&job, NULL);
if (submit_rc != THREAD_POOL_STATUS_OK) {
free_sub_log_created_payload_local(payload);
}
if (filter_json) free(filter_json);
}
// Log subscription closure to database
void log_subscription_closed(const char* sub_id, const char* client_ip, const char* reason) {
(void)reason; // Mark as intentionally unused
if (!g_db || !sub_id) return;
const char* sql =
"INSERT INTO subscriptions (subscription_id, wsi_pointer, client_ip, event_type) "
"VALUES (?, '', ?, 'closed')";
sqlite3_stmt* stmt;
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
if (rc == SQLITE_OK) {
sqlite3_bind_text(stmt, 1, sub_id, -1, SQLITE_STATIC);
sqlite3_bind_text(stmt, 2, client_ip ? client_ip : "unknown", -1, SQLITE_STATIC);
sqlite3_step(stmt);
sqlite3_finalize(stmt);
if (!sub_id) return;
if (!thread_pool_is_running()) {
db_log_subscription_closed(sub_id, client_ip);
return;
}
// Update the corresponding 'created' entry with end time and events sent
const char* update_sql =
"UPDATE subscriptions "
"SET ended_at = strftime('%s', 'now') "
"WHERE subscription_id = ? AND event_type = 'created' AND ended_at IS NULL";
rc = sqlite3_prepare_v2(g_db, update_sql, -1, &stmt, NULL);
if (rc == SQLITE_OK) {
sqlite3_bind_text(stmt, 1, sub_id, -1, SQLITE_STATIC);
sqlite3_step(stmt);
sqlite3_finalize(stmt);
thread_pool_sub_log_closed_payload_t* payload = calloc(1, sizeof(*payload));
if (!payload) {
return;
}
payload->sub_id = strdup(sub_id);
payload->client_ip = strdup(client_ip ? client_ip : "unknown");
if (!payload->sub_id || !payload->client_ip) {
free_sub_log_closed_payload_local(payload);
return;
}
thread_pool_job_t job;
memset(&job, 0, sizeof(job));
job.type = THREAD_POOL_JOB_LOG_SUB_CLOSED;
job.payload = payload;
job.payload_free = (thread_pool_payload_free_cb)free_sub_log_closed_payload_local;
thread_pool_status_t submit_rc = thread_pool_submit_write(&job, NULL);
if (submit_rc != THREAD_POOL_STATUS_OK) {
free_sub_log_closed_payload_local(payload);
}
}
// Log subscription disconnection to database
void log_subscription_disconnected(const char* client_ip) {
if (!g_db || !client_ip) return;
// Mark all active subscriptions for this client as disconnected
const char* sql =
"UPDATE subscriptions "
"SET ended_at = strftime('%s', 'now') "
"WHERE client_ip = ? AND event_type = 'created' AND ended_at IS NULL";
sqlite3_stmt* stmt;
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
if (rc == SQLITE_OK) {
sqlite3_bind_text(stmt, 1, client_ip, -1, SQLITE_STATIC);
int changes = sqlite3_changes(g_db);
sqlite3_step(stmt);
sqlite3_finalize(stmt);
if (changes > 0) {
// Log a disconnection event
const char* insert_sql =
"INSERT INTO subscriptions (subscription_id, wsi_pointer, client_ip, event_type) "
"VALUES ('disconnect', '', ?, 'disconnected')";
rc = sqlite3_prepare_v2(g_db, insert_sql, -1, &stmt, NULL);
if (rc == SQLITE_OK) {
sqlite3_bind_text(stmt, 1, client_ip, -1, SQLITE_STATIC);
sqlite3_step(stmt);
sqlite3_finalize(stmt);
}
}
if (!client_ip) return;
if (!thread_pool_is_running()) {
db_log_subscription_disconnected(client_ip);
return;
}
thread_pool_sub_log_disconnected_payload_t* payload = calloc(1, sizeof(*payload));
if (!payload) {
return;
}
payload->client_ip = strdup(client_ip);
if (!payload->client_ip) {
free_sub_log_disconnected_payload_local(payload);
return;
}
thread_pool_job_t job;
memset(&job, 0, sizeof(job));
job.type = THREAD_POOL_JOB_LOG_SUB_DISCONNECTED;
job.payload = payload;
job.payload_free = (thread_pool_payload_free_cb)free_sub_log_disconnected_payload_local;
thread_pool_status_t submit_rc = thread_pool_submit_write(&job, NULL);
if (submit_rc != THREAD_POOL_STATUS_OK) {
free_sub_log_disconnected_payload_local(payload);
}
}
@@ -1153,55 +1199,52 @@ void log_subscription_disconnected(const char* client_ip) {
// Update events sent counter for a subscription
void update_subscription_events_sent(const char* sub_id, int events_sent) {
if (!g_db || !sub_id) return;
if (!sub_id) return;
const char* sql =
"UPDATE subscriptions "
"SET events_sent = ? "
"WHERE subscription_id = ? AND event_type = 'created'";
if (!thread_pool_is_running()) {
db_update_subscription_events_sent(sub_id, events_sent);
return;
}
sqlite3_stmt* stmt;
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
if (rc == SQLITE_OK) {
sqlite3_bind_int(stmt, 1, events_sent);
sqlite3_bind_text(stmt, 2, sub_id, -1, SQLITE_STATIC);
thread_pool_sub_update_events_payload_t* payload = calloc(1, sizeof(*payload));
if (!payload) {
return;
}
sqlite3_step(stmt);
sqlite3_finalize(stmt);
payload->sub_id = strdup(sub_id);
payload->events_sent = events_sent;
if (!payload->sub_id) {
free_sub_update_events_payload_local(payload);
return;
}
thread_pool_job_t job;
memset(&job, 0, sizeof(job));
job.type = THREAD_POOL_JOB_UPDATE_SUB_EVENTS_SENT;
job.payload = payload;
job.payload_free = (thread_pool_payload_free_cb)free_sub_update_events_payload_local;
thread_pool_status_t submit_rc = thread_pool_submit_write(&job, NULL);
if (submit_rc != THREAD_POOL_STATUS_OK) {
free_sub_update_events_payload_local(payload);
}
}
// Cleanup all subscriptions on startup
void cleanup_all_subscriptions_on_startup(void) {
if (!g_db) {
if (!db_is_available()) {
DEBUG_ERROR("Database not available for startup cleanup");
return;
}
DEBUG_LOG("Performing startup subscription cleanup");
// Mark all active subscriptions as disconnected
const char* sql =
"UPDATE subscriptions "
"SET ended_at = strftime('%s', 'now') "
"WHERE event_type = 'created' AND ended_at IS NULL";
sqlite3_stmt* stmt;
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
if (rc != SQLITE_OK) {
DEBUG_ERROR("Failed to prepare startup cleanup query");
return;
}
rc = sqlite3_step(stmt);
int changes = sqlite3_changes(g_db);
sqlite3_finalize(stmt);
if (rc != SQLITE_DONE) {
int changes = db_cleanup_orphaned_subscriptions();
if (changes < 0) {
DEBUG_ERROR("Failed to execute startup cleanup");
return;
}
if (changes > 0) {
DEBUG_LOG("Startup cleanup: marked %d orphaned subscriptions as disconnected", changes);
} else {
+1 -1
View File
@@ -1,4 +1,4 @@
// Subscription system structures and functions for C-Relay
// Subscription system structures and functions for C-Relay-PG
// This header defines subscription management functionality
#ifndef SUBSCRIPTIONS_H
+953
View File
@@ -0,0 +1,953 @@
#define _GNU_SOURCE
#include "thread_pool.h"
#include "debug.h"
#include "db_ops.h"
#include "ip_ban.h"
#include <pthread.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
typedef struct thread_pool_job_node {
uint64_t job_id;
thread_pool_job_t job;
struct thread_pool_job_node* next;
} thread_pool_job_node_t;
typedef struct {
thread_pool_job_node_t* head;
thread_pool_job_node_t* tail;
int size;
int max_size;
pthread_mutex_t mutex;
pthread_cond_t cond;
} thread_pool_queue_t;
typedef struct {
int running;
uint64_t next_job_id;
int reader_count;
pthread_t* readers;
pthread_t writer;
thread_pool_queue_t read_q;
thread_pool_queue_t write_q;
thread_pool_wake_loop_cb wake_loop_cb;
void* wake_loop_ctx;
char db_path[512];
pthread_mutex_t state_mutex;
} thread_pool_state_t;
static thread_pool_state_t g_pool = {0};
typedef struct {
pthread_mutex_t mutex;
pthread_cond_t cond;
int done;
thread_pool_status_t status;
void* result_data;
size_t result_size;
} thread_pool_wait_ctx_t;
static void wait_ctx_init(thread_pool_wait_ctx_t* ctx) {
memset(ctx, 0, sizeof(*ctx));
pthread_mutex_init(&ctx->mutex, NULL);
pthread_cond_init(&ctx->cond, NULL);
}
static void wait_ctx_destroy(thread_pool_wait_ctx_t* ctx) {
pthread_mutex_destroy(&ctx->mutex);
pthread_cond_destroy(&ctx->cond);
}
static void wait_ctx_result_cb(const thread_pool_result_t* result, void* user_ctx) {
thread_pool_wait_ctx_t* ctx = (thread_pool_wait_ctx_t*)user_ctx;
if (!ctx || !result) return;
pthread_mutex_lock(&ctx->mutex);
ctx->status = result->status;
ctx->result_data = result->result_data;
ctx->result_size = result->result_size;
ctx->done = 1;
pthread_cond_signal(&ctx->cond);
pthread_mutex_unlock(&ctx->mutex);
}
static void wait_ctx_wait(thread_pool_wait_ctx_t* ctx) {
pthread_mutex_lock(&ctx->mutex);
while (!ctx->done) {
pthread_cond_wait(&ctx->cond, &ctx->mutex);
}
pthread_mutex_unlock(&ctx->mutex);
}
static void free_count_payload(void* p) {
thread_pool_count_payload_t* payload = (thread_pool_count_payload_t*)p;
if (!payload) return;
free(payload->sql);
if (payload->bind_params) {
for (int i = 0; i < payload->bind_param_count; i++) {
free(payload->bind_params[i]);
}
free(payload->bind_params);
}
free(payload);
}
static void free_req_payload(void* p) {
thread_pool_req_payload_t* payload = (thread_pool_req_payload_t*)p;
if (!payload) return;
free(payload->sql);
if (payload->bind_params) {
for (int i = 0; i < payload->bind_param_count; i++) {
free(payload->bind_params[i]);
}
free(payload->bind_params);
}
free(payload);
}
static void free_store_event_payload(void* p) {
thread_pool_store_event_payload_t* payload = (thread_pool_store_event_payload_t*)p;
if (!payload) return;
free(payload->id);
free(payload->pubkey);
free(payload->event_type);
free(payload->content);
free(payload->sig);
free(payload->tags_json);
free(payload->event_json);
free(payload);
}
static void queue_init(thread_pool_queue_t* q, int max_size) {
memset(q, 0, sizeof(*q));
q->max_size = (max_size > 0) ? max_size : 4096;
pthread_mutex_init(&q->mutex, NULL);
pthread_cond_init(&q->cond, NULL);
}
static void queue_destroy(thread_pool_queue_t* q) {
pthread_mutex_lock(&q->mutex);
thread_pool_job_node_t* cur = q->head;
while (cur) {
thread_pool_job_node_t* next = cur->next;
if (cur->job.payload_free && cur->job.payload) {
cur->job.payload_free(cur->job.payload);
}
free(cur);
cur = next;
}
q->head = q->tail = NULL;
q->size = 0;
pthread_mutex_unlock(&q->mutex);
pthread_mutex_destroy(&q->mutex);
pthread_cond_destroy(&q->cond);
}
static int queue_push(thread_pool_queue_t* q, thread_pool_job_node_t* node) {
pthread_mutex_lock(&q->mutex);
if (q->size >= q->max_size) {
pthread_mutex_unlock(&q->mutex);
return 0;
}
node->next = NULL;
if (!q->tail) {
q->head = q->tail = node;
} else {
q->tail->next = node;
q->tail = node;
}
q->size++;
pthread_cond_signal(&q->cond);
pthread_mutex_unlock(&q->mutex);
return 1;
}
static thread_pool_job_node_t* queue_pop(thread_pool_queue_t* q) {
pthread_mutex_lock(&q->mutex);
while (g_pool.running && q->size == 0) {
pthread_cond_wait(&q->cond, &q->mutex);
}
if (!g_pool.running && q->size == 0) {
pthread_mutex_unlock(&q->mutex);
return NULL;
}
thread_pool_job_node_t* node = q->head;
q->head = node->next;
if (!q->head) q->tail = NULL;
q->size--;
pthread_mutex_unlock(&q->mutex);
return node;
}
static void wake_event_loop(void) {
if (g_pool.wake_loop_cb) {
g_pool.wake_loop_cb(g_pool.wake_loop_ctx);
}
}
static void* open_worker_connection(void) {
void* db_conn = NULL;
if (db_open_worker_connection(g_pool.db_path, &db_conn) != 0) {
return NULL;
}
return db_conn;
}
static void complete_job_with_result(thread_pool_job_node_t* node,
thread_pool_status_t status,
const char* message,
void* result_data,
size_t result_size) {
thread_pool_result_t result;
memset(&result, 0, sizeof(result));
result.job_id = node->job_id;
result.type = node->job.type;
result.status = status;
result.session = node->job.session;
result.result_data = result_data;
result.result_size = result_size;
result.message = message;
if (node->job.result_cb) {
node->job.result_cb(&result, node->job.result_cb_ctx);
}
wake_event_loop();
if (node->job.payload_free && node->job.payload) {
node->job.payload_free(node->job.payload);
}
free(node);
}
static void execute_count_job(thread_pool_job_node_t* node) {
thread_pool_count_payload_t* payload = (thread_pool_count_payload_t*)node->job.payload;
if (!payload || !payload->sql) {
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"COUNT payload missing", NULL, 0);
return;
}
int count = 0;
if (db_count_with_sql(payload->sql, (const char**)payload->bind_params,
payload->bind_param_count, &count) != 0) {
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"COUNT query failed", NULL, 0);
return;
}
thread_pool_count_result_t* out = calloc(1, sizeof(*out));
if (!out) {
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"COUNT result allocation failed", NULL, 0);
return;
}
out->count = count;
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
"COUNT query completed", out, sizeof(*out));
}
static void execute_req_job(thread_pool_job_node_t* node) {
thread_pool_req_payload_t* payload = (thread_pool_req_payload_t*)node->job.payload;
if (!payload || !payload->sql) {
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"REQ payload missing", NULL, 0);
return;
}
db_stmt_t* stmt = NULL;
if (db_prepare(payload->sql, &stmt) != DB_OK) {
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"REQ prepare failed", NULL, 0);
return;
}
for (int i = 0; i < payload->bind_param_count; i++) {
const char* v = (payload->bind_params && payload->bind_params[i]) ? payload->bind_params[i] : "";
(void)db_bind_text_param(stmt, i + 1, v);
}
int cap = (payload->row_limit > 0) ? payload->row_limit : 500;
if (cap < 1) cap = 1;
thread_pool_req_result_t* out = calloc(1, sizeof(*out));
if (!out) {
db_finalize_stmt(stmt);
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"REQ result allocation failed", NULL, 0);
return;
}
out->event_json_rows = calloc((size_t)cap, sizeof(char*));
if (!out->event_json_rows) {
db_finalize_stmt(stmt);
free(out);
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"REQ row array allocation failed", NULL, 0);
return;
}
int rc = DB_OK;
while ((rc = db_step_stmt(stmt)) == DB_ROW) {
if (out->row_count >= cap) break;
const char* row = db_column_text_value(stmt, 0);
if (!row) continue;
out->event_json_rows[out->row_count] = strdup(row);
if (!out->event_json_rows[out->row_count]) {
break;
}
out->row_count++;
}
db_finalize_stmt(stmt);
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
"REQ query completed", out, sizeof(*out));
}
static void execute_store_event_job(thread_pool_job_node_t* node) {
thread_pool_store_event_payload_t* payload = (thread_pool_store_event_payload_t*)node->job.payload;
if (!payload || !payload->id || !payload->pubkey || !payload->content || !payload->sig) {
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"STORE_EVENT payload missing required fields", NULL, 0);
return;
}
thread_pool_store_event_result_t* out = calloc(1, sizeof(*out));
if (!out) {
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"STORE_EVENT result allocation failed", NULL, 0);
return;
}
int step_rc = DB_ERROR;
int extended_errcode = 0;
if (db_insert_event_with_json(payload->id,
payload->pubkey,
payload->created_at,
payload->kind,
payload->event_type ? payload->event_type : "regular",
payload->content,
payload->sig,
payload->tags_json ? payload->tags_json : "[]",
payload->event_json ? payload->event_json : "{}",
&step_rc,
&extended_errcode) != 0) {
free(out);
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"STORE_EVENT execution failed", NULL, 0);
return;
}
out->step_rc = step_rc;
out->extended_errcode = extended_errcode;
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
"STORE_EVENT completed", out, sizeof(*out));
}
static void execute_sub_log_created_job(thread_pool_job_node_t* node) {
thread_pool_sub_log_created_payload_t* payload = (thread_pool_sub_log_created_payload_t*)node->job.payload;
if (!payload || !payload->sub_id || !payload->wsi_ptr || !payload->client_ip) {
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"SUB_LOG_CREATED payload missing", NULL, 0);
return;
}
if (db_log_subscription_created(payload->sub_id,
payload->wsi_ptr,
payload->client_ip,
payload->filter_json ? payload->filter_json : "[]") != 0) {
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"SUB_LOG_CREATED failed", NULL, 0);
return;
}
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
"SUB_LOG_CREATED completed", NULL, 0);
}
static void execute_sub_log_closed_job(thread_pool_job_node_t* node) {
thread_pool_sub_log_closed_payload_t* payload = (thread_pool_sub_log_closed_payload_t*)node->job.payload;
if (!payload || !payload->sub_id) {
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"SUB_LOG_CLOSED payload missing", NULL, 0);
return;
}
if (db_log_subscription_closed(payload->sub_id, payload->client_ip) != 0) {
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"SUB_LOG_CLOSED failed", NULL, 0);
return;
}
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
"SUB_LOG_CLOSED completed", NULL, 0);
}
static void execute_sub_log_disconnected_job(thread_pool_job_node_t* node) {
thread_pool_sub_log_disconnected_payload_t* payload = (thread_pool_sub_log_disconnected_payload_t*)node->job.payload;
if (!payload || !payload->client_ip) {
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"SUB_LOG_DISCONNECTED payload missing", NULL, 0);
return;
}
if (db_log_subscription_disconnected(payload->client_ip) < 0) {
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"SUB_LOG_DISCONNECTED failed", NULL, 0);
return;
}
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
"SUB_LOG_DISCONNECTED completed", NULL, 0);
}
static void execute_sub_update_events_job(thread_pool_job_node_t* node) {
thread_pool_sub_update_events_payload_t* payload = (thread_pool_sub_update_events_payload_t*)node->job.payload;
if (!payload || !payload->sub_id) {
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"SUB_UPDATE_EVENTS payload missing", NULL, 0);
return;
}
if (db_update_subscription_events_sent(payload->sub_id, payload->events_sent) != 0) {
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"SUB_UPDATE_EVENTS failed", NULL, 0);
return;
}
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
"SUB_UPDATE_EVENTS completed", NULL, 0);
}
static void execute_ip_ban_save_job(thread_pool_job_node_t* node) {
ip_ban_save_to_db();
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
"IP_BAN_SAVE completed", NULL, 0);
}
static void execute_wal_checkpoint_job(thread_pool_job_node_t* node) {
if (db_wal_checkpoint_passive() != 0) {
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
"WAL_CHECKPOINT failed", NULL, 0);
return;
}
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
"WAL_CHECKPOINT completed", NULL, 0);
}
static void execute_read_job(thread_pool_job_node_t* node) {
switch (node->job.type) {
case THREAD_POOL_JOB_REQ_QUERY:
execute_req_job(node);
break;
case THREAD_POOL_JOB_COUNT_QUERY:
execute_count_job(node);
break;
default:
complete_job_with_result(node, THREAD_POOL_STATUS_NOT_IMPLEMENTED,
"Read job type not implemented", NULL, 0);
break;
}
}
static void execute_write_job(thread_pool_job_node_t* node) {
switch (node->job.type) {
case THREAD_POOL_JOB_STORE_EVENT:
execute_store_event_job(node);
break;
case THREAD_POOL_JOB_LOG_SUB_CREATED:
execute_sub_log_created_job(node);
break;
case THREAD_POOL_JOB_LOG_SUB_CLOSED:
execute_sub_log_closed_job(node);
break;
case THREAD_POOL_JOB_LOG_SUB_DISCONNECTED:
execute_sub_log_disconnected_job(node);
break;
case THREAD_POOL_JOB_UPDATE_SUB_EVENTS_SENT:
execute_sub_update_events_job(node);
break;
case THREAD_POOL_JOB_IP_BAN_SAVE:
execute_ip_ban_save_job(node);
break;
case THREAD_POOL_JOB_WAL_CHECKPOINT:
execute_wal_checkpoint_job(node);
break;
default:
complete_job_with_result(node, THREAD_POOL_STATUS_NOT_IMPLEMENTED,
"Write job type not implemented", NULL, 0);
break;
}
}
static void* reader_worker_main(void* arg) {
int reader_index = (int)(intptr_t)arg;
char thread_name[16];
snprintf(thread_name, sizeof(thread_name), "db-read-%d", reader_index);
pthread_setname_np(pthread_self(), thread_name);
void* worker_db = open_worker_connection();
if (!worker_db) {
DEBUG_ERROR("Reader worker failed to open SQLite connection");
return NULL;
}
while (g_pool.running) {
thread_pool_job_node_t* node = queue_pop(&g_pool.read_q);
if (!node) break;
execute_read_job(node);
}
db_close_worker_connection(worker_db);
return NULL;
}
static void* writer_worker_main(void* arg) {
(void)arg;
pthread_setname_np(pthread_self(), "db-write");
void* worker_db = open_worker_connection();
if (!worker_db) {
DEBUG_ERROR("Writer worker failed to open SQLite connection");
return NULL;
}
while (g_pool.running) {
thread_pool_job_node_t* node = queue_pop(&g_pool.write_q);
if (!node) break;
execute_write_job(node);
}
if (db_wal_checkpoint_truncate() != 0) {
DEBUG_WARN("Writer shutdown TRUNCATE checkpoint failed");
}
db_close_worker_connection(worker_db);
return NULL;
}
int thread_pool_init(const thread_pool_config_t* config) {
if (!config) return -1;
pthread_mutex_lock(&g_pool.state_mutex);
if (g_pool.running) {
pthread_mutex_unlock(&g_pool.state_mutex);
return 0;
}
g_pool.reader_count = (config->reader_threads > 0) ? config->reader_threads : 4;
g_pool.readers = calloc((size_t)g_pool.reader_count, sizeof(pthread_t));
if (!g_pool.readers) {
pthread_mutex_unlock(&g_pool.state_mutex);
return -1;
}
queue_init(&g_pool.read_q, config->max_queue_depth);
queue_init(&g_pool.write_q, config->max_queue_depth);
g_pool.next_job_id = 1;
g_pool.wake_loop_cb = config->wake_loop_cb;
g_pool.wake_loop_ctx = config->wake_loop_ctx;
const char* db_path = (config->db_path && config->db_path[0] != '\0') ? config->db_path : db_get_database_path();
if (!db_path || db_path[0] == '\0') {
free(g_pool.readers);
g_pool.readers = NULL;
pthread_mutex_unlock(&g_pool.state_mutex);
return -1;
}
strncpy(g_pool.db_path, db_path, sizeof(g_pool.db_path) - 1);
g_pool.db_path[sizeof(g_pool.db_path) - 1] = '\0';
g_pool.running = 1;
for (int i = 0; i < g_pool.reader_count; i++) {
if (pthread_create(&g_pool.readers[i], NULL, reader_worker_main, (void*)(intptr_t)(i + 1)) != 0) {
g_pool.running = 0;
pthread_cond_broadcast(&g_pool.read_q.cond);
pthread_cond_broadcast(&g_pool.write_q.cond);
pthread_mutex_unlock(&g_pool.state_mutex);
return -1;
}
}
if (pthread_create(&g_pool.writer, NULL, writer_worker_main, NULL) != 0) {
g_pool.running = 0;
pthread_cond_broadcast(&g_pool.read_q.cond);
pthread_cond_broadcast(&g_pool.write_q.cond);
pthread_mutex_unlock(&g_pool.state_mutex);
return -1;
}
pthread_mutex_unlock(&g_pool.state_mutex);
DEBUG_LOG("Thread pool initialized: %d readers + 1 writer", g_pool.reader_count);
return 0;
}
void thread_pool_shutdown(void) {
pthread_mutex_lock(&g_pool.state_mutex);
if (!g_pool.running) {
pthread_mutex_unlock(&g_pool.state_mutex);
return;
}
g_pool.running = 0;
pthread_cond_broadcast(&g_pool.read_q.cond);
pthread_cond_broadcast(&g_pool.write_q.cond);
pthread_mutex_unlock(&g_pool.state_mutex);
for (int i = 0; i < g_pool.reader_count; i++) {
pthread_join(g_pool.readers[i], NULL);
}
pthread_join(g_pool.writer, NULL);
free(g_pool.readers);
g_pool.readers = NULL;
g_pool.reader_count = 0;
queue_destroy(&g_pool.read_q);
queue_destroy(&g_pool.write_q);
DEBUG_LOG("Thread pool shutdown complete");
}
int thread_pool_is_running(void) {
return g_pool.running;
}
static thread_pool_status_t submit_to_queue(thread_pool_queue_t* q, const thread_pool_job_t* job, uint64_t* out_job_id) {
if (!g_pool.running) return THREAD_POOL_STATUS_SHUTTING_DOWN;
if (!job) return THREAD_POOL_STATUS_INTERNAL_ERROR;
thread_pool_job_node_t* node = calloc(1, sizeof(*node));
if (!node) return THREAD_POOL_STATUS_INTERNAL_ERROR;
node->job = *job;
pthread_mutex_lock(&g_pool.state_mutex);
node->job_id = g_pool.next_job_id++;
pthread_mutex_unlock(&g_pool.state_mutex);
if (out_job_id) *out_job_id = node->job_id;
if (!queue_push(q, node)) {
if (job->payload_free && job->payload) {
job->payload_free(job->payload);
}
free(node);
return THREAD_POOL_STATUS_QUEUE_FULL;
}
return THREAD_POOL_STATUS_OK;
}
thread_pool_status_t thread_pool_submit_read(const thread_pool_job_t* job, uint64_t* out_job_id) {
return submit_to_queue(&g_pool.read_q, job, out_job_id);
}
thread_pool_status_t thread_pool_submit_write(const thread_pool_job_t* job, uint64_t* out_job_id) {
return submit_to_queue(&g_pool.write_q, job, out_job_id);
}
int thread_pool_submit_wal_checkpoint(void) {
if (!thread_pool_is_running()) {
return -1;
}
thread_pool_job_t job;
memset(&job, 0, sizeof(job));
job.type = THREAD_POOL_JOB_WAL_CHECKPOINT;
thread_pool_status_t rc = thread_pool_submit_write(&job, NULL);
return (rc == THREAD_POOL_STATUS_OK) ? 0 : -1;
}
int thread_pool_execute_count_sync(const char* sql, const char** bind_params, int bind_param_count, int* out_count) {
if (!sql || !out_count) {
return -1;
}
if (!thread_pool_is_running()) {
return db_count_with_sql(sql, bind_params, bind_param_count, out_count);
}
thread_pool_count_payload_t* payload = calloc(1, sizeof(*payload));
if (!payload) return -1;
payload->sql = strdup(sql);
payload->bind_param_count = bind_param_count;
if (!payload->sql) {
free_count_payload(payload);
return -1;
}
if (bind_param_count > 0) {
payload->bind_params = calloc((size_t)bind_param_count, sizeof(char*));
if (!payload->bind_params) {
free_count_payload(payload);
return -1;
}
for (int i = 0; i < bind_param_count; i++) {
const char* v = (bind_params && bind_params[i]) ? bind_params[i] : "";
payload->bind_params[i] = strdup(v);
if (!payload->bind_params[i]) {
free_count_payload(payload);
return -1;
}
}
}
thread_pool_wait_ctx_t wait_ctx;
wait_ctx_init(&wait_ctx);
thread_pool_job_t job;
memset(&job, 0, sizeof(job));
job.type = THREAD_POOL_JOB_COUNT_QUERY;
job.payload = payload;
job.payload_free = free_count_payload;
job.result_cb = wait_ctx_result_cb;
job.result_cb_ctx = &wait_ctx;
thread_pool_status_t submit_rc = thread_pool_submit_read(&job, NULL);
if (submit_rc != THREAD_POOL_STATUS_OK) {
free_count_payload(payload);
wait_ctx_destroy(&wait_ctx);
return -1;
}
wait_ctx_wait(&wait_ctx);
int rc = -1;
if (wait_ctx.status == THREAD_POOL_STATUS_OK && wait_ctx.result_data) {
thread_pool_count_result_t* result = (thread_pool_count_result_t*)wait_ctx.result_data;
*out_count = result->count;
free(result);
rc = 0;
}
wait_ctx_destroy(&wait_ctx);
return rc;
}
int thread_pool_execute_req_sync(const char* sql, const char** bind_params, int bind_param_count,
int row_limit, thread_pool_req_result_t** out_result) {
if (!sql || !out_result) {
return -1;
}
if (!thread_pool_is_running()) {
db_stmt_t* stmt = NULL;
if (db_prepare(sql, &stmt) != DB_OK) {
return -1;
}
for (int i = 0; i < bind_param_count; i++) {
const char* v = (bind_params && bind_params[i]) ? bind_params[i] : "";
(void)db_bind_text_param(stmt, i + 1, v);
}
int cap = (row_limit > 0) ? row_limit : 500;
if (cap < 1) cap = 1;
thread_pool_req_result_t* result = calloc(1, sizeof(*result));
if (!result) {
db_finalize_stmt(stmt);
return -1;
}
result->event_json_rows = calloc((size_t)cap, sizeof(char*));
if (!result->event_json_rows) {
free(result);
db_finalize_stmt(stmt);
return -1;
}
int step_rc = DB_OK;
while ((step_rc = db_step_stmt(stmt)) == DB_ROW) {
if (result->row_count >= cap) break;
const char* row = db_column_text_value(stmt, 0);
if (!row) continue;
result->event_json_rows[result->row_count] = strdup(row);
if (!result->event_json_rows[result->row_count]) {
break;
}
result->row_count++;
}
db_finalize_stmt(stmt);
*out_result = result;
return 0;
}
thread_pool_req_payload_t* payload = calloc(1, sizeof(*payload));
if (!payload) return -1;
payload->sql = strdup(sql);
payload->bind_param_count = bind_param_count;
payload->row_limit = row_limit;
if (!payload->sql) {
free_req_payload(payload);
return -1;
}
if (bind_param_count > 0) {
payload->bind_params = calloc((size_t)bind_param_count, sizeof(char*));
if (!payload->bind_params) {
free_req_payload(payload);
return -1;
}
for (int i = 0; i < bind_param_count; i++) {
const char* v = (bind_params && bind_params[i]) ? bind_params[i] : "";
payload->bind_params[i] = strdup(v);
if (!payload->bind_params[i]) {
free_req_payload(payload);
return -1;
}
}
}
thread_pool_wait_ctx_t wait_ctx;
wait_ctx_init(&wait_ctx);
thread_pool_job_t job;
memset(&job, 0, sizeof(job));
job.type = THREAD_POOL_JOB_REQ_QUERY;
job.payload = payload;
job.payload_free = free_req_payload;
job.result_cb = wait_ctx_result_cb;
job.result_cb_ctx = &wait_ctx;
thread_pool_status_t submit_rc = thread_pool_submit_read(&job, NULL);
if (submit_rc != THREAD_POOL_STATUS_OK) {
free_req_payload(payload);
wait_ctx_destroy(&wait_ctx);
return -1;
}
wait_ctx_wait(&wait_ctx);
int rc = -1;
if (wait_ctx.status == THREAD_POOL_STATUS_OK && wait_ctx.result_data) {
*out_result = (thread_pool_req_result_t*)wait_ctx.result_data;
rc = 0;
}
wait_ctx_destroy(&wait_ctx);
return rc;
}
void thread_pool_free_req_result(thread_pool_req_result_t* result) {
if (!result) return;
if (result->event_json_rows) {
for (int i = 0; i < result->row_count; i++) {
free(result->event_json_rows[i]);
}
free(result->event_json_rows);
}
free(result);
}
int thread_pool_execute_store_event_sync(const thread_pool_store_event_payload_t* payload,
thread_pool_store_event_result_t* out_result) {
if (!payload || !out_result || !payload->id || !payload->pubkey || !payload->content || !payload->sig) {
return -1;
}
if (!thread_pool_is_running()) {
int step_rc = DB_ERROR;
int extended_errcode = 0;
if (db_insert_event_with_json(payload->id,
payload->pubkey,
payload->created_at,
payload->kind,
payload->event_type ? payload->event_type : "regular",
payload->content,
payload->sig,
payload->tags_json ? payload->tags_json : "[]",
payload->event_json ? payload->event_json : "{}",
&step_rc,
&extended_errcode) != 0) {
return -1;
}
out_result->step_rc = step_rc;
out_result->extended_errcode = extended_errcode;
return 0;
}
thread_pool_store_event_payload_t* payload_copy = calloc(1, sizeof(*payload_copy));
if (!payload_copy) return -1;
payload_copy->id = payload->id ? strdup(payload->id) : NULL;
payload_copy->pubkey = payload->pubkey ? strdup(payload->pubkey) : NULL;
payload_copy->created_at = payload->created_at;
payload_copy->kind = payload->kind;
payload_copy->event_type = payload->event_type ? strdup(payload->event_type) : strdup("regular");
payload_copy->content = payload->content ? strdup(payload->content) : NULL;
payload_copy->sig = payload->sig ? strdup(payload->sig) : NULL;
payload_copy->tags_json = payload->tags_json ? strdup(payload->tags_json) : strdup("[]");
payload_copy->event_json = payload->event_json ? strdup(payload->event_json) : strdup("{}");
if (!payload_copy->id || !payload_copy->pubkey || !payload_copy->event_type ||
!payload_copy->content || !payload_copy->sig || !payload_copy->tags_json || !payload_copy->event_json) {
free_store_event_payload(payload_copy);
return -1;
}
thread_pool_wait_ctx_t wait_ctx;
wait_ctx_init(&wait_ctx);
thread_pool_job_t job;
memset(&job, 0, sizeof(job));
job.type = THREAD_POOL_JOB_STORE_EVENT;
job.payload = payload_copy;
job.payload_free = free_store_event_payload;
job.result_cb = wait_ctx_result_cb;
job.result_cb_ctx = &wait_ctx;
thread_pool_status_t submit_rc = thread_pool_submit_write(&job, NULL);
if (submit_rc != THREAD_POOL_STATUS_OK) {
free_store_event_payload(payload_copy);
wait_ctx_destroy(&wait_ctx);
return -1;
}
wait_ctx_wait(&wait_ctx);
int rc = -1;
if (wait_ctx.status == THREAD_POOL_STATUS_OK && wait_ctx.result_data) {
thread_pool_store_event_result_t* result = (thread_pool_store_event_result_t*)wait_ctx.result_data;
out_result->step_rc = result->step_rc;
out_result->extended_errcode = result->extended_errcode;
free(result);
rc = 0;
}
wait_ctx_destroy(&wait_ctx);
return rc;
}
__attribute__((constructor))
static void thread_pool_state_init_once(void) {
pthread_mutex_init(&g_pool.state_mutex, NULL);
}
+145
View File
@@ -0,0 +1,145 @@
#ifndef THREAD_POOL_H
#define THREAD_POOL_H
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef enum {
THREAD_POOL_JOB_REQ_QUERY = 0,
THREAD_POOL_JOB_COUNT_QUERY,
THREAD_POOL_JOB_STORE_EVENT,
THREAD_POOL_JOB_DELETE_EVENT,
THREAD_POOL_JOB_LOG_SUB_CREATED,
THREAD_POOL_JOB_LOG_SUB_CLOSED,
THREAD_POOL_JOB_LOG_SUB_DISCONNECTED,
THREAD_POOL_JOB_UPDATE_SUB_EVENTS_SENT,
THREAD_POOL_JOB_IP_BAN_SAVE,
THREAD_POOL_JOB_WAL_CHECKPOINT,
THREAD_POOL_JOB_CUSTOM
} thread_pool_job_type_t;
typedef enum {
THREAD_POOL_STATUS_OK = 0,
THREAD_POOL_STATUS_NOT_IMPLEMENTED,
THREAD_POOL_STATUS_QUEUE_FULL,
THREAD_POOL_STATUS_SHUTTING_DOWN,
THREAD_POOL_STATUS_INTERNAL_ERROR
} thread_pool_status_t;
typedef struct {
uint64_t job_id;
thread_pool_job_type_t type;
thread_pool_status_t status;
void* session;
void* result_data;
size_t result_size;
const char* message;
} thread_pool_result_t;
typedef void (*thread_pool_result_cb)(const thread_pool_result_t* result, void* user_ctx);
typedef void (*thread_pool_payload_free_cb)(void* payload);
typedef void (*thread_pool_wake_loop_cb)(void* wake_ctx);
typedef struct {
thread_pool_job_type_t type;
void* session;
void* payload;
size_t payload_size;
thread_pool_payload_free_cb payload_free;
thread_pool_result_cb result_cb;
void* result_cb_ctx;
} thread_pool_job_t;
typedef struct {
int reader_threads;
int max_queue_depth;
const char* db_path;
thread_pool_wake_loop_cb wake_loop_cb;
void* wake_loop_ctx;
} thread_pool_config_t;
typedef struct {
char* sql;
char** bind_params;
int bind_param_count;
} thread_pool_count_payload_t;
typedef struct {
char* sql;
char** bind_params;
int bind_param_count;
int row_limit;
} thread_pool_req_payload_t;
typedef struct {
char* id;
char* pubkey;
long long created_at;
int kind;
char* event_type;
char* content;
char* sig;
char* tags_json;
char* event_json;
} thread_pool_store_event_payload_t;
typedef struct {
int count;
} thread_pool_count_result_t;
typedef struct {
char** event_json_rows;
int row_count;
} thread_pool_req_result_t;
typedef struct {
int step_rc;
int extended_errcode;
} thread_pool_store_event_result_t;
typedef struct {
char* sub_id;
char* wsi_ptr;
char* client_ip;
char* filter_json;
} thread_pool_sub_log_created_payload_t;
typedef struct {
char* sub_id;
char* client_ip;
} thread_pool_sub_log_closed_payload_t;
typedef struct {
char* client_ip;
} thread_pool_sub_log_disconnected_payload_t;
typedef struct {
char* sub_id;
int events_sent;
} thread_pool_sub_update_events_payload_t;
int thread_pool_init(const thread_pool_config_t* config);
void thread_pool_shutdown(void);
int thread_pool_is_running(void);
thread_pool_status_t thread_pool_submit_read(const thread_pool_job_t* job, uint64_t* out_job_id);
thread_pool_status_t thread_pool_submit_write(const thread_pool_job_t* job, uint64_t* out_job_id);
int thread_pool_submit_wal_checkpoint(void);
int thread_pool_execute_count_sync(const char* sql, const char** bind_params, int bind_param_count, int* out_count);
int thread_pool_execute_req_sync(const char* sql, const char** bind_params, int bind_param_count,
int row_limit, thread_pool_req_result_t** out_result);
void thread_pool_free_req_result(thread_pool_req_result_t* result);
int thread_pool_execute_store_event_sync(const thread_pool_store_event_payload_t* payload,
thread_pool_store_event_result_t* out_result);
#ifdef __cplusplus
}
#endif
#endif // THREAD_POOL_H
+1046 -223
View File
File diff suppressed because it is too large Load Diff
+5 -1
View File
@@ -1,4 +1,4 @@
// WebSocket protocol structures and constants for C-Relay
// WebSocket protocol structures and constants for C-Relay-PG
// This header defines structures shared between main.c and websockets.c
#ifndef WEBSOCKETS_H
@@ -99,6 +99,10 @@ struct per_session_data {
// Get current active WebSocket connection count
int get_active_connection_count(void);
// Safely resolve per-session data only for currently tracked live WebSocket sessions.
// Returns 1 and sets *out_pss on success, otherwise returns 0.
int websocket_get_live_pss(struct lws* wsi, struct per_session_data** out_pss);
// Actual relay port bound by libwebsockets at runtime (after fallback/retry logic)
extern int g_relay_port;
+34 -34
View File
@@ -8,7 +8,7 @@ The C Nostr Relay now uses a revolutionary **zero-configuration** approach where
## Files
- **`c-relay.service`** - SystemD service unit file
- **`c-relay-pg.service`** - SystemD service unit file
- **`install-service.sh`** - Automated installation script
- **`uninstall-service.sh`** - Automated uninstall script
- **`README.md`** - This documentation
@@ -27,12 +27,12 @@ The C Nostr Relay now uses a revolutionary **zero-configuration** approach where
3. **Start the service:**
```bash
sudo systemctl start c-relay
sudo systemctl start c-relay-pg
```
4. **Check admin keys (IMPORTANT!):**
```bash
sudo journalctl -u c-relay --since="1 hour ago" | grep "Admin Private Key"
sudo journalctl -u c-relay-pg --since="1 hour ago" | grep "Admin Private Key"
```
## Event-Based Configuration System
@@ -61,7 +61,7 @@ On first startup, the relay will:
```bash
# View first startup logs to get admin private key
sudo journalctl -u c-relay --since="1 hour ago" | grep -A 5 "IMPORTANT: SAVE THIS ADMIN PRIVATE KEY"
sudo journalctl -u c-relay-pg --since="1 hour ago" | grep -A 5 "IMPORTANT: SAVE THIS ADMIN PRIVATE KEY"
```
The admin private key is needed to update relay configuration by sending signed kind 33334 events.
@@ -72,10 +72,10 @@ The admin private key is needed to update relay configuration by sending signed
```bash
# Find the database file
ls /opt/c-relay/*.nrdb
ls /opt/c-relay-pg/*.nrdb
# View configuration event
sqlite3 /opt/c-relay/<relay_pubkey>.nrdb "SELECT content, tags FROM events WHERE kind = 33334;"
sqlite3 /opt/c-relay-pg/<relay_pubkey>.nrdb "SELECT content, tags FROM events WHERE kind = 33334;"
```
### Updating Configuration
@@ -93,57 +93,57 @@ Send a new kind 33334 event to the relay via WebSocket:
```bash
# Start service
sudo systemctl start c-relay
sudo systemctl start c-relay-pg
# Stop service
sudo systemctl stop c-relay
sudo systemctl stop c-relay-pg
# Restart service
sudo systemctl restart c-relay
sudo systemctl restart c-relay-pg
# Enable auto-start on boot
sudo systemctl enable c-relay
sudo systemctl enable c-relay-pg
# Check status
sudo systemctl status c-relay
sudo systemctl status c-relay-pg
# View logs (live)
sudo journalctl -u c-relay -f
sudo journalctl -u c-relay-pg -f
# View recent logs
sudo journalctl -u c-relay --since="1 hour ago"
sudo journalctl -u c-relay-pg --since="1 hour ago"
```
### Log Analysis
```bash
# Check for successful startup
sudo journalctl -u c-relay | grep "First-time startup sequence completed"
sudo journalctl -u c-relay-pg | grep "First-time startup sequence completed"
# Find admin keys
sudo journalctl -u c-relay | grep "Admin Private Key"
sudo journalctl -u c-relay-pg | grep "Admin Private Key"
# Check configuration updates
sudo journalctl -u c-relay | grep "Configuration updated via kind 33334"
sudo journalctl -u c-relay-pg | grep "Configuration updated via kind 33334"
# Monitor real-time activity
sudo journalctl -u c-relay -f | grep -E "(INFO|SUCCESS|ERROR)"
sudo journalctl -u c-relay-pg -f | grep -E "(INFO|SUCCESS|ERROR)"
```
## File Locations
After installation:
- **Binary:** `/opt/c-relay/c_relay_x86`
- **Database:** `/opt/c-relay/<relay_pubkey>.nrdb` (created automatically)
- **Service File:** `/etc/systemd/system/c-relay.service`
- **User:** `c-relay` (system user created automatically)
- **Binary:** `/opt/c-relay-pg/c_relay_pg_x86`
- **Database:** `/opt/c-relay-pg/<relay_pubkey>.nrdb` (created automatically)
- **Service File:** `/etc/systemd/system/c-relay-pg.service`
- **User:** `c-relay-pg` (system user created automatically)
## Security Features
The systemd service includes security hardening:
- Runs as dedicated system user `c-relay`
- Runs as dedicated system user `c-relay-pg`
- `NoNewPrivileges=true`
- `ProtectSystem=strict`
- `ProtectHome=true`
@@ -165,7 +165,7 @@ The database file contains everything:
```bash
# Backup database file
sudo cp /opt/c-relay/*.nrdb /backup/location/
sudo cp /opt/c-relay-pg/*.nrdb /backup/location/
# The .nrdb file contains:
# - All Nostr events
@@ -177,7 +177,7 @@ sudo cp /opt/c-relay/*.nrdb /backup/location/
To migrate to new server:
1. Copy `.nrdb` file to new server's `/opt/c-relay/` directory
1. Copy `.nrdb` file to new server's `/opt/c-relay-pg/` directory
2. Install service with `install-service.sh`
3. Start service - it will automatically detect existing configuration
@@ -187,39 +187,39 @@ To migrate to new server:
```bash
# Check service status
sudo systemctl status c-relay
sudo systemctl status c-relay-pg
# Check logs for errors
sudo journalctl -u c-relay --no-pager
sudo journalctl -u c-relay-pg --no-pager
# Check if binary exists and is executable
ls -la /opt/c-relay/c_relay_x86
ls -la /opt/c-relay-pg/c_relay_pg_x86
# Check permissions
sudo -u c-relay ls -la /opt/c-relay/
sudo -u c-relay-pg ls -la /opt/c-relay-pg/
```
### Database Issues
```bash
# Check if database file exists
ls -la /opt/c-relay/*.nrdb*
ls -la /opt/c-relay-pg/*.nrdb*
# Check database integrity
sqlite3 /opt/c-relay/*.nrdb "PRAGMA integrity_check;"
sqlite3 /opt/c-relay-pg/*.nrdb "PRAGMA integrity_check;"
# View database schema
sqlite3 /opt/c-relay/*.nrdb ".schema"
sqlite3 /opt/c-relay-pg/*.nrdb ".schema"
```
### Configuration Issues
```bash
# Check if configuration event exists
sqlite3 /opt/c-relay/*.nrdb "SELECT COUNT(*) FROM events WHERE kind = 33334;"
sqlite3 /opt/c-relay-pg/*.nrdb "SELECT COUNT(*) FROM events WHERE kind = 33334;"
# View configuration event
sqlite3 /opt/c-relay/*.nrdb "SELECT id, created_at, LENGTH(tags) FROM events WHERE kind = 33334;"
sqlite3 /opt/c-relay-pg/*.nrdb "SELECT id, created_at, LENGTH(tags) FROM events WHERE kind = 33334;"
```
## Uninstallation
@@ -238,7 +238,7 @@ The uninstall script will:
For issues with the event-based configuration system:
1. Check service logs: `sudo journalctl -u c-relay -f`
1. Check service logs: `sudo journalctl -u c-relay-pg -f`
2. Verify database integrity
3. Ensure admin private key is saved securely
4. Check WebSocket connectivity on port 8888
+7 -7
View File
@@ -1,27 +1,27 @@
[Unit]
Description=C Nostr Relay Server (Event-Based Configuration)
Documentation=https://github.com/your-repo/c-relay
Documentation=https://github.com/your-repo/c-relay-pg
After=network.target
Wants=network-online.target
[Service]
Type=simple
User=c-relay
Group=c-relay
WorkingDirectory=/opt/c-relay
User=c-relay-pg
Group=c-relay-pg
WorkingDirectory=/opt/c-relay-pg
Environment=DEBUG_LEVEL=0
ExecStart=/opt/c-relay/c_relay_x86 --debug-level=$DEBUG_LEVEL
ExecStart=/opt/c-relay-pg/c_relay_pg_x86 --debug-level=$DEBUG_LEVEL
Restart=always
RestartSec=5
StandardOutput=journal
StandardError=journal
SyslogIdentifier=c-relay
SyslogIdentifier=c-relay-pg
# Security settings
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/opt/c-relay
ReadWritePaths=/opt/c-relay-pg
PrivateTmp=true
ProtectKernelTunables=true
ProtectKernelModules=true
+4 -4
View File
@@ -6,10 +6,10 @@
set -e
# Configuration
SERVICE_NAME="c-relay"
SERVICE_USER="c-relay"
INSTALL_DIR="/opt/c-relay"
BINARY_NAME="c_relay_x86"
SERVICE_NAME="c-relay-pg"
SERVICE_USER="c-relay-pg"
INSTALL_DIR="/opt/c-relay-pg"
BINARY_NAME="c_relay_pg_x86"
# Colors for output
RED='\033[0;31m'
+13 -13
View File
@@ -1,15 +1,15 @@
#!/bin/bash
# C-Relay Systemd Service Installation Script
# This script installs the C-Relay as a systemd service
# C-Relay-PG Systemd Service Installation Script
# This script installs the C-Relay-PG as a systemd service
set -e
# Configuration
INSTALL_DIR="/opt/c-relay"
SERVICE_NAME="c-relay"
SERVICE_FILE="c-relay.service"
BINARY_NAME="c_relay_x86"
INSTALL_DIR="/opt/c-relay-pg"
SERVICE_NAME="c-relay-pg"
SERVICE_FILE="c-relay-pg.service"
BINARY_NAME="c_relay_pg_x86"
# Colors for output
RED='\033[0;31m'
@@ -17,7 +17,7 @@ GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
echo -e "${GREEN}=== C-Relay Systemd Service Installation ===${NC}"
echo -e "${GREEN}=== C-Relay-PG Systemd Service Installation ===${NC}"
# Check if running as root
if [[ $EUID -ne 0 ]]; then
@@ -39,12 +39,12 @@ if [ ! -f "$SERVICE_FILE" ]; then
exit 1
fi
# Create c-relay user if it doesn't exist
if ! id "c-relay" &>/dev/null; then
echo -e "${YELLOW}Creating c-relay user...${NC}"
useradd --system --shell /bin/false --home-dir $INSTALL_DIR --create-home c-relay
# Create c-relay-pg user if it doesn't exist
if ! id "c-relay-pg" &>/dev/null; then
echo -e "${YELLOW}Creating c-relay-pg user...${NC}"
useradd --system --shell /bin/false --home-dir $INSTALL_DIR --create-home c-relay-pg
else
echo -e "${GREEN}User c-relay already exists${NC}"
echo -e "${GREEN}User c-relay-pg already exists${NC}"
fi
# Create installation directory
@@ -59,7 +59,7 @@ chmod +x $INSTALL_DIR/$BINARY_NAME
# Set permissions
echo -e "${YELLOW}Setting permissions...${NC}"
chown -R c-relay:c-relay $INSTALL_DIR
chown -R c-relay-pg:c-relay-pg $INSTALL_DIR
# Install systemd service
echo -e "${YELLOW}Installing systemd service...${NC}"
+3 -3
View File
@@ -6,9 +6,9 @@
set -e
# Configuration
SERVICE_NAME="c-relay"
SERVICE_USER="c-relay"
INSTALL_DIR="/opt/c-relay"
SERVICE_NAME="c-relay-pg"
SERVICE_USER="c-relay-pg"
INSTALL_DIR="/opt/c-relay-pg"
# Colors for output
RED='\033[0;31m'
+15 -15
View File
@@ -1,14 +1,14 @@
#!/bin/bash
# C-Relay Systemd Service Uninstallation Script
# This script removes the C-Relay systemd service
# C-Relay-PG Systemd Service Uninstallation Script
# This script removes the C-Relay-PG systemd service
set -e
# Configuration
INSTALL_DIR="/opt/c-relay"
SERVICE_NAME="c-relay"
SERVICE_FILE="c-relay.service"
INSTALL_DIR="/opt/c-relay-pg"
SERVICE_NAME="c-relay-pg"
SERVICE_FILE="c-relay-pg.service"
# Colors for output
RED='\033[0;31m'
@@ -16,7 +16,7 @@ GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
echo -e "${GREEN}=== C-Relay Systemd Service Uninstallation ===${NC}"
echo -e "${GREEN}=== C-Relay-PG Systemd Service Uninstallation ===${NC}"
# Check if running as root
if [[ $EUID -ne 0 ]]; then
@@ -65,22 +65,22 @@ else
echo -e "${GREEN}Installation directory preserved${NC}"
fi
# Ask about removing c-relay user
# Ask about removing c-relay-pg user
echo
echo -e "${YELLOW}Do you want to remove the c-relay user? (y/N)${NC}"
echo -e "${YELLOW}Do you want to remove the c-relay-pg user? (y/N)${NC}"
read -r response
if [[ "$response" =~ ^([yY][eE][sS]|[yY])$ ]]; then
echo -e "${YELLOW}Removing c-relay user...${NC}"
if id "c-relay" &>/dev/null; then
userdel c-relay
echo -e "${GREEN}User c-relay removed${NC}"
echo -e "${YELLOW}Removing c-relay-pg user...${NC}"
if id "c-relay-pg" &>/dev/null; then
userdel c-relay-pg
echo -e "${GREEN}User c-relay-pg removed${NC}"
else
echo -e "${GREEN}User c-relay was not found${NC}"
echo -e "${GREEN}User c-relay-pg was not found${NC}"
fi
else
echo -e "${GREEN}User c-relay preserved${NC}"
echo -e "${GREEN}User c-relay-pg preserved${NC}"
fi
echo
echo -e "${GREEN}=== Uninstallation Complete ===${NC}"
echo -e "${GREEN}C-Relay systemd service has been removed${NC}"
echo -e "${GREEN}C-Relay-PG systemd service has been removed${NC}"
+18
View File
@@ -0,0 +1,18 @@
2026-04-02 11:34:06 - ==========================================
2026-04-02 11:34:06 - C-Relay-PG Comprehensive Test Suite Runner
2026-04-02 11:34:06 - ==========================================
2026-04-02 11:34:06 - Relay URL: ws://127.0.0.1:8888
2026-04-02 11:34:06 - Log file: test_results_20260402_113406.log
2026-04-02 11:34:06 - Report file: test_report_20260402_113406.html
2026-04-02 11:34:06 -
2026-04-02 11:34:06 - Checking relay status at ws://127.0.0.1:8888...
2026-04-02 11:34:07 - \033[0;32m✓ Relay HTTP endpoint is accessible\033[0m
2026-04-02 11:34:07 -
2026-04-02 11:34:07 - Starting comprehensive test execution...
2026-04-02 11:34:07 -
2026-04-02 11:34:07 - \033[0;34m=== SECURITY TEST SUITES ===\033[0m
2026-04-02 11:34:07 - ==========================================
2026-04-02 11:34:07 - Running Test Suite: SQL Injection Tests
2026-04-02 11:34:07 - Description: Comprehensive SQL injection vulnerability testing
2026-04-02 11:34:07 - ==========================================
2026-04-02 11:34:07 - \033[0;31mERROR: Test script sql_injection_tests.sh not found\033[0m
+1 -1
View File
@@ -421,7 +421,7 @@ echo
if run_nip11_tests; then
echo
print_success "All NIP-11 tests completed successfully!"
print_info "The C-Relay NIP-11 implementation is fully compliant"
print_info "The C-Relay-PG NIP-11 implementation is fully compliant"
print_info "✅ HTTP endpoint, Accept header validation, CORS, and JSON structure all working"
echo
exit 0
+24 -3
View File
@@ -28,7 +28,28 @@ echo "✓ nak command found"
# Check if relay is running by testing connection
echo "Testing relay connection..."
if ! timeout 5 nc -z localhost 8888 2>/dev/null; then
relay_up=0
if command -v nc &> /dev/null; then
if timeout 5 nc -z localhost 8888 2>/dev/null; then
relay_up=1
fi
fi
# Fallback for environments where netcat probing is unreliable/unsupported
if [ "$relay_up" -eq 0 ] && command -v websocat &> /dev/null; then
if timeout 5 bash -c "echo 'ping' | websocat -n1 $RELAY_URL >/dev/null 2>&1"; then
relay_up=1
fi
fi
# Additional fallback: POSIX TCP open check without external tools
if [ "$relay_up" -eq 0 ]; then
if timeout 5 bash -c "exec 3<>/dev/tcp/localhost/8888" 2>/dev/null; then
relay_up=1
fi
fi
if [ "$relay_up" -eq 0 ]; then
echo "ERROR: Relay does not appear to be running on localhost:8888"
echo "Please start the relay first with: ./make_and_restart_relay.sh"
exit 1
@@ -91,10 +112,10 @@ sleep 3
# Query for gift wrap responses from the relay (kind 1059 events authored by relay)
echo ""
echo "Querying for gift wrap responses from relay..."
echo "Command: nak req -k 1059 --authors $RELAY_PUBLIC_KEY $RELAY_URL"
echo "Command: nak req -k 1059 -a $RELAY_PUBLIC_KEY $RELAY_URL"
# Capture the output and filter for events
RESPONSE_OUTPUT=$(nak req -k 1059 --authors "$RELAY_PUBLIC_KEY" "$RELAY_URL" 2>&1)
RESPONSE_OUTPUT=$(nak req -k 1059 -a "$RELAY_PUBLIC_KEY" "$RELAY_URL" 2>&1)
REQ_EXIT_CODE=$?
echo ""
+6 -6
View File
@@ -1,6 +1,6 @@
#!/bin/bash
# Comprehensive C-Relay Test - Test event types and subscriptions
# Comprehensive C-Relay-PG Test - Test event types and subscriptions
# Uses nak to generate and publish various event types, then tests subscriptions
set -e # Exit on any error
@@ -284,7 +284,7 @@ test_subscription() {
# Main test function
run_comprehensive_test() {
print_header "C-Relay Comprehensive Test"
print_header "C-Relay-PG Comprehensive Test"
# Check dependencies
print_step "Checking dependencies..."
@@ -317,7 +317,7 @@ run_comprehensive_test() {
# Test 2: Replaceable Events (kind 0 - metadata)
print_step "Creating replaceable events (kind 0)..."
local replaceable1=$(nak event --sec "$TEST_PRIVATE_KEY" -c '{"name":"Test User","about":"Testing C-Relay"}' -k 0 --ts $(($(date +%s) - 80)) -t "type=replaceable" 2>/dev/null)
local replaceable1=$(nak event --sec "$TEST_PRIVATE_KEY" -c '{"name":"Test User","about":"Testing C-Relay-PG"}' -k 0 --ts $(($(date +%s) - 80)) -t "type=replaceable" 2>/dev/null)
local replaceable2=$(nak event --sec "$TEST_PRIVATE_KEY" -c '{"name":"Test User Updated","about":"Updated profile"}' -k 0 --ts $(($(date +%s) - 70)) -t "type=replaceable" 2>/dev/null)
publish_event "$replaceable1" "replaceable" "Replaceable event #1 (metadata)"
@@ -473,20 +473,20 @@ run_comprehensive_test() {
}
# Run the comprehensive test
print_header "Starting C-Relay Comprehensive Test Suite with NIP-01 Validation"
print_header "Starting C-Relay-PG Comprehensive Test Suite with NIP-01 Validation"
echo
if run_comprehensive_test; then
echo
print_success "All tests completed successfully!"
print_info "The C-Relay with full NIP-01 validation is working correctly"
print_info "The C-Relay-PG with full NIP-01 validation is working correctly"
print_info "✅ Event validation, signature verification, and error handling all working"
echo
exit 0
else
echo
print_error "❌ TESTS FAILED: Protocol violations detected!"
print_error "The C-Relay has critical issues that need to be fixed:"
print_error "The C-Relay-PG has critical issues that need to be fixed:"
print_error " - Server-side filtering is not implemented properly"
print_error " - Events are sent to clients regardless of subscription filters"
print_error " - This violates the Nostr protocol specification"
+16 -7
View File
@@ -54,6 +54,8 @@ BASELINE_KIND1=0
BASELINE_KIND0=0
BASELINE_KIND30001=0
BASELINE_AUTHOR=0
BASELINE_AUTHOR_KIND0=0
BASELINE_AUTHOR_KIND30001=0
BASELINE_TYPE_REGULAR=0
BASELINE_TEST_NIP45=0
BASELINE_KINDS_01=0
@@ -253,9 +255,11 @@ run_count_test() {
publish_event "$regular1" "regular" "Regular event #1"
# Now that we have the pubkey, get the author baseline
# Now that we have the pubkey, get author baselines
local test_pubkey=$(echo "$regular1" | jq -r '.pubkey' 2>/dev/null)
BASELINE_AUTHOR=$(get_baseline_count "{\"authors\":[\"$test_pubkey\"]}" "events by test author")
BASELINE_AUTHOR_KIND0=$(get_baseline_count "{\"authors\":[\"$test_pubkey\"],\"kinds\":[0]}" "kind 0 events by test author")
BASELINE_AUTHOR_KIND30001=$(get_baseline_count "{\"authors\":[\"$test_pubkey\"],\"kinds\":[30001]}" "kind 30001 events by test author")
publish_event "$regular2" "regular" "Regular event #2"
publish_event "$regular3" "regular" "Regular event #3"
@@ -314,10 +318,13 @@ run_count_test() {
fi
# Test 3: Count events by author (pubkey)
# BASELINE_AUTHOR includes the first regular event, we add 2 more regular
# Replaceable and addressable replace existing events from this author
# BASELINE_AUTHOR is measured after regular1 is already published.
# Net additions after baseline:
# +2 regular (regular2, regular3)
# +(1 - BASELINE_AUTHOR_KIND0) from replaceable upsert behavior
# +(1 - BASELINE_AUTHOR_KIND30001) from addressable replacement behavior
local test_pubkey=$(echo "$regular1" | jq -r '.pubkey' 2>/dev/null)
local expected_author=$((2 + BASELINE_AUTHOR))
local expected_author=$((BASELINE_AUTHOR + 2 + (1 - BASELINE_AUTHOR_KIND0) + (1 - BASELINE_AUTHOR_KIND30001)))
if ! test_count "count_author" "{\"authors\":[\"$test_pubkey\"]}" "Count events by specific author" "$expected_author"; then
((test_failures++))
fi
@@ -340,8 +347,10 @@ run_count_test() {
fi
# Test 6: Count multiple kinds
# BASELINE_KINDS_01 + 3 regular events = total for kinds 0+1
local expected_kinds_01=$((3 + BASELINE_KINDS_01))
# Net additions for kinds 0+1 after baseline:
# +3 kind-1 regular events
# +(1 - BASELINE_AUTHOR_KIND0) for kind-0 replaceable upsert behavior
local expected_kinds_01=$((BASELINE_KINDS_01 + 3 + (1 - BASELINE_AUTHOR_KIND0)))
if ! test_count "count_multi_kinds" '{"kinds":[0,1]}' "Count multiple kinds (0,1)" "$expected_kinds_01"; then
((test_failures++))
fi
@@ -452,7 +461,7 @@ echo
if run_count_test; then
echo
print_success "All NIP-45 COUNT tests completed successfully!"
print_info "The C-Relay COUNT functionality is working correctly"
print_info "The C-Relay-PG COUNT functionality is working correctly"
print_info "✅ COUNT messages are processed and return correct event counts"
echo
exit 0
+1 -1
View File
@@ -405,7 +405,7 @@ echo
if run_search_test; then
echo
print_success "All NIP-50 SEARCH tests completed successfully!"
print_info "The C-Relay SEARCH functionality is working correctly"
print_info "The C-Relay-PG SEARCH functionality is working correctly"
print_info "✅ Search field in filter objects works for both REQ and COUNT messages"
print_info "✅ Search works across event content and tag values"
print_info "✅ Search is case-insensitive and supports partial matches"
+1 -1
View File
@@ -220,7 +220,7 @@ echo
if run_protected_events_test; then
echo
print_success "All NIP-70 PROTECTED EVENTS tests completed successfully!"
print_info "The C-Relay PROTECTED EVENTS functionality is working correctly"
print_info "The C-Relay-PG PROTECTED EVENTS functionality is working correctly"
print_info "✅ Protected events are rejected when feature is disabled"
print_info "✅ Protected events are rejected when enabled but not authenticated"
print_info "✅ Normal events work regardless of protected events setting"

Some files were not shown because too many files have changed in this diff Show More