Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5577b57149 | ||
|
|
22c943d495 | ||
|
|
7e3d9b6825 | ||
|
|
547d22d09f | ||
|
|
ce9ae5a488 | ||
|
|
9b248b740f | ||
|
|
0f77aeb72b | ||
|
|
bb64651a0c | ||
|
|
17be9c2b03 | ||
|
|
f396c622b6 | ||
|
|
cbd260c1ae | ||
|
|
bc9ac290ab | ||
|
|
192eeb248d | ||
|
|
da6c505420 | ||
|
|
c077c209e5 | ||
|
|
9bf02702c2 | ||
|
|
0920cc092d | ||
|
|
a89460be5d | ||
|
|
ff2a3aa335 | ||
|
|
94b61e8a7c | ||
|
|
b0c0754e83 | ||
|
|
3265e3d114 | ||
|
|
927659ece1 | ||
|
|
b6ff4150b4 | ||
|
|
63bc526163 | ||
|
|
a1f712236a | ||
|
|
06e6c17b7b | ||
|
|
0751a7c55c | ||
|
|
f1728932a9 | ||
|
|
ef8bdef2a8 | ||
|
|
c11a8ba292 | ||
|
|
0f124fe575 | ||
|
|
6b20452fab | ||
|
|
10c19bc243 | ||
|
|
3d7aa2196f | ||
|
|
a416c3f275 | ||
|
|
bba9baabc3 | ||
|
|
31187c4c4f | ||
|
|
55f862b879 | ||
|
|
76c9b3fcf0 | ||
|
|
929bd09164 | ||
|
|
d17f1dd8d5 | ||
|
|
207a949835 | ||
|
|
d08f4e4221 | ||
|
|
c96736fa6a | ||
|
|
f8ec4ae924 | ||
|
|
fe7304ac7f | ||
|
|
e5d39c984b | ||
|
|
5e45f21e35 | ||
|
|
5321a238b8 | ||
|
|
083bc14972 | ||
|
|
11aaccba9b | ||
|
|
bd1bbd763d | ||
|
|
2bd7aa5a10 | ||
|
|
361912ec85 | ||
|
|
0de491382e | ||
|
|
3148bbbee7 | ||
|
|
3965ba04d8 | ||
|
|
b96af938bd | ||
|
|
89c8248013 | ||
|
|
d8f477c6cf | ||
|
|
040eeadb13 | ||
|
|
83f8b0ab88 | ||
|
|
b82f7eaaf3 | ||
|
|
0dc5b75d7c | ||
|
|
e94b1a81e3 | ||
|
|
1adabdbc4e | ||
|
|
81d44c3d8c | ||
|
|
7acc0bdd90 | ||
|
|
c4ef71d673 |
@@ -11,3 +11,4 @@ copy_executable_local.sh
|
||||
nostr_login_lite/
|
||||
style_guide/
|
||||
nostr-tools
|
||||
.test_keys
|
||||
+11
-11
@@ -1,12 +1,12 @@
|
||||
|
||||
# AGENTS.md - AI Agent Integration Guide for Architect Mode
|
||||
|
||||
**Project-Specific Information for AI Agents Working with C-Relay in Architect Mode**
|
||||
**Project-Specific Information for AI Agents Working with C-Relay-PG in Architect Mode**
|
||||
|
||||
## Critical Architecture Understanding
|
||||
|
||||
### System Architecture Overview
|
||||
C-Relay implements a **unique event-based configuration architecture** that fundamentally differs from traditional Nostr relays:
|
||||
C-Relay-PG implements a **unique event-based configuration architecture** that fundamentally differs from traditional Nostr relays:
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
|
||||
@@ -51,9 +51,9 @@ C-Relay implements a **unique event-based configuration architecture** that fund
|
||||
## Architectural Decision Analysis
|
||||
|
||||
### Configuration System Design
|
||||
**Traditional Approach vs C-Relay:**
|
||||
**Traditional Approach vs C-Relay-PG:**
|
||||
```
|
||||
Traditional: C-Relay:
|
||||
Traditional: C-Relay-PG:
|
||||
config.json → kind 33334 events
|
||||
ENV variables → cryptographically signed tags
|
||||
File watching → database polling/restart
|
||||
@@ -119,18 +119,18 @@ File watching → database polling/restart
|
||||
```
|
||||
Developer Machine:
|
||||
├── ./make_and_restart_relay.sh
|
||||
├── build/c_relay_x86
|
||||
├── build/c_relay_pg_x86
|
||||
├── build/<relay_pubkey>.db
|
||||
└── relay.log
|
||||
```
|
||||
|
||||
### Production SystemD Deployment
|
||||
```
|
||||
/opt/c-relay/:
|
||||
├── c_relay_x86
|
||||
/opt/c-relay-pg/:
|
||||
├── c_relay_pg_x86
|
||||
├── <relay_pubkey>.db
|
||||
├── systemd service (c-relay.service)
|
||||
└── c-relay user isolation
|
||||
├── systemd service (c-relay-pg.service)
|
||||
└── c-relay-pg user isolation
|
||||
```
|
||||
|
||||
### Container Deployment Architecture
|
||||
@@ -144,7 +144,7 @@ Container:
|
||||
|
||||
### Reverse Proxy Architecture
|
||||
```
|
||||
Internet → Nginx/HAProxy → C-Relay
|
||||
Internet → Nginx/HAProxy → C-Relay-PG
|
||||
├── WebSocket upgrade handling
|
||||
├── SSL termination
|
||||
└── Rate limiting
|
||||
@@ -292,7 +292,7 @@ Admin Signs Event → WebSocket Submit → Validate → Store → Restart Requir
|
||||
**Decision**: Same port serves both protocols
|
||||
**Consequences**: Simplified deployment, protocol detection overhead, libwebsockets dependency
|
||||
|
||||
These architectural decisions form the foundation of C-Relay's unique approach to Nostr relay implementation and should be carefully considered when planning extensions or modifications.
|
||||
These architectural decisions form the foundation of C-Relay-PG's unique approach to Nostr relay implementation and should be carefully considered when planning extensions or modifications.
|
||||
**
|
||||
|
||||
[Response interrupted by a tool use result. Only one tool may be used at a time and should be placed at the end of the message.]
|
||||
Executable
BIN
Binary file not shown.
@@ -1,6 +1,6 @@
|
||||
# AGENTS.md - AI Agent Integration Guide
|
||||
|
||||
**Project-Specific Information for AI Agents Working with C-Relay**
|
||||
**Project-Specific Information for AI Agents Working with C-Relay-PG**
|
||||
|
||||
## Critical Build Commands
|
||||
|
||||
@@ -15,9 +15,9 @@
|
||||
- Starts relay in background with proper logging
|
||||
|
||||
### Architecture-Specific Binary Outputs
|
||||
- **x86_64**: `./build/c_relay_x86`
|
||||
- **ARM64**: `./build/c_relay_arm64`
|
||||
- **Other**: `./build/c_relay_$(ARCH)`
|
||||
- **x86_64**: `./build/c_relay_pg_x86`
|
||||
- **ARM64**: `./build/c_relay_pg_arm64`
|
||||
- **Other**: `./build/c_relay_pg_$(ARCH)`
|
||||
|
||||
### Database File Naming Convention
|
||||
- **Format**: `<relay_pubkey>.db` (NOT `.nrdb` as shown in docs)
|
||||
@@ -75,10 +75,10 @@
|
||||
### Process Management
|
||||
```bash
|
||||
# Kill existing relay processes
|
||||
pkill -f "c_relay_"
|
||||
pkill -f "c_relay_pg_"
|
||||
|
||||
# Check running processes
|
||||
ps aux | grep c_relay_
|
||||
ps aux | grep c_relay_pg_
|
||||
|
||||
# Force kill port binding
|
||||
fuser -k 8888/tcp
|
||||
@@ -95,7 +95,7 @@ fuser -k 8888/tcp
|
||||
- Event configuration tests: `tests/event_config_tests.sh`
|
||||
|
||||
### SystemD Integration Considerations
|
||||
- Service runs as `c-relay` user in `/opt/c-relay`
|
||||
- Service runs as `c-relay-pg` user in `/opt/c-relay-pg`
|
||||
- Database files created in WorkingDirectory automatically
|
||||
- No environment variables needed (event-based config)
|
||||
- Resource limits: 65536 file descriptors, 4096 processes
|
||||
@@ -137,7 +137,7 @@ fuser -k 8888/tcp
|
||||
## Quick Debugging Commands
|
||||
```bash
|
||||
# Check relay status
|
||||
ps aux | grep c_relay_ && netstat -tln | grep 8888
|
||||
ps aux | grep c_relay_pg_ && netstat -tln | grep 8888
|
||||
|
||||
# View logs
|
||||
tail -f relay.log
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# C-Relay API Documentation
|
||||
# C-Relay-PG API Documentation
|
||||
|
||||
Complete API reference for the C-Relay event-based administration system and advanced features.
|
||||
Complete API reference for the C-Relay-PG event-based administration system and advanced features.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
@@ -21,7 +21,7 @@ Complete API reference for the C-Relay event-based administration system and adv
|
||||
|
||||
## Overview
|
||||
|
||||
C-Relay uses an innovative **event-based administration system** where all configuration and management commands are sent as cryptographically signed Nostr events. This provides:
|
||||
C-Relay-PG uses an innovative **event-based administration system** where all configuration and management commands are sent as cryptographically signed Nostr events. This provides:
|
||||
|
||||
- **Cryptographic security**: All commands must be signed with the admin private key
|
||||
- **Audit trail**: Complete history of all administrative actions
|
||||
@@ -63,8 +63,8 @@ Store the admin private key securely:
|
||||
export C_RELAY_ADMIN_KEY="nsec1abc123..."
|
||||
|
||||
# Secure file
|
||||
echo "nsec1abc123..." > ~/.c-relay-admin
|
||||
chmod 600 ~/.c-relay-admin
|
||||
echo "nsec1abc123..." > ~/.c-relay-pg-admin
|
||||
chmod 600 ~/.c-relay-pg-admin
|
||||
|
||||
# Password manager (recommended)
|
||||
# Store in 1Password, Bitwarden, etc.
|
||||
@@ -167,7 +167,7 @@ Examples:
|
||||
"data": [
|
||||
{
|
||||
"key": "relay_name",
|
||||
"value": "C-Relay",
|
||||
"value": "C-Relay-PG",
|
||||
"data_type": "string",
|
||||
"category": "relay",
|
||||
"description": "Relay name displayed in NIP-11"
|
||||
@@ -506,10 +506,10 @@ ORDER BY count DESC
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `relay_name` | string | "C-Relay" | Relay name (NIP-11) |
|
||||
| `relay_name` | string | "C-Relay-PG" | Relay name (NIP-11) |
|
||||
| `relay_description` | string | "C Nostr Relay" | Relay description |
|
||||
| `relay_contact` | string | "" | Admin contact info |
|
||||
| `relay_software` | string | "c-relay" | Software identifier |
|
||||
| `relay_software` | string | "c-relay-pg" | Software identifier |
|
||||
| `relay_version` | string | auto | Software version |
|
||||
| `supported_nips` | string | "1,9,11,13,15,20,33,40,42,45,50,70" | Supported NIPs |
|
||||
| `language_tags` | string | "*" | Supported languages |
|
||||
@@ -576,7 +576,7 @@ ORDER BY count DESC
|
||||
|
||||
## Real-time Monitoring
|
||||
|
||||
C-Relay provides subscription-based real-time monitoring using ephemeral events (kind 24567).
|
||||
C-Relay-PG provides subscription-based real-time monitoring using ephemeral events (kind 24567).
|
||||
|
||||
### Activation
|
||||
|
||||
|
||||
+21
-24
@@ -1,4 +1,4 @@
|
||||
# Alpine-based MUSL static binary builder for C-Relay
|
||||
# Alpine-based MUSL static binary builder for C-Relay-PG
|
||||
# Produces truly portable binaries with zero runtime dependencies
|
||||
|
||||
ARG DEBUG_BUILD=false
|
||||
@@ -28,7 +28,8 @@ RUN apk add --no-cache \
|
||||
sqlite-static \
|
||||
linux-headers \
|
||||
wget \
|
||||
bash
|
||||
bash \
|
||||
openssh-client
|
||||
|
||||
# Set working directory
|
||||
WORKDIR /build
|
||||
@@ -68,15 +69,8 @@ RUN cd /tmp && \
|
||||
make install && \
|
||||
rm -rf /tmp/libwebsockets
|
||||
|
||||
# Copy only submodule configuration and git directory
|
||||
COPY .gitmodules /build/.gitmodules
|
||||
COPY .git /build/.git
|
||||
|
||||
# Clean up any stale submodule references (nips directory is not a submodule)
|
||||
RUN git rm --cached nips 2>/dev/null || true
|
||||
|
||||
# Initialize submodules (cached unless .gitmodules changes)
|
||||
RUN git submodule update --init --recursive
|
||||
# Submodules are provided in the local build context and copied explicitly below.
|
||||
# Avoid network/SSH dependency inside Docker image build.
|
||||
|
||||
# Copy nostr_core_lib source files (cached unless nostr_core_lib changes)
|
||||
COPY nostr_core_lib /build/nostr_core_lib/
|
||||
@@ -84,11 +78,13 @@ COPY nostr_core_lib /build/nostr_core_lib/
|
||||
# Copy c_utils_lib source files (cached unless c_utils_lib changes)
|
||||
COPY c_utils_lib /build/c_utils_lib/
|
||||
|
||||
# Build c_utils_lib with MUSL-compatible flags (cached unless c_utils_lib changes)
|
||||
# Build c_utils_lib static library for relay logging utilities
|
||||
# (build only debug.c to avoid broken/missing version header surface in submodule revision)
|
||||
RUN cd c_utils_lib && \
|
||||
sed -i 's/CFLAGS = -Wall -Wextra -std=c99 -O2 -g/CFLAGS = -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -Wall -Wextra -std=c99 -O2 -g/' Makefile && \
|
||||
make clean && \
|
||||
make
|
||||
mkdir -p build && \
|
||||
gcc -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -Wall -Wextra -std=c99 -O2 -g \
|
||||
-Isrc -Iinclude -c src/debug.c -o build/debug.o && \
|
||||
ar rcs libc_utils.a build/debug.o
|
||||
|
||||
# Build nostr_core_lib with required NIPs (cached unless nostr_core_lib changes)
|
||||
# Disable fortification in build.sh to prevent __*_chk symbol issues
|
||||
@@ -99,11 +95,11 @@ RUN cd nostr_core_lib && \
|
||||
rm -f *.o *.a 2>/dev/null || true && \
|
||||
./build.sh --nips=1,6,13,17,19,44,59
|
||||
|
||||
# Copy c-relay source files LAST (only this layer rebuilds on source changes)
|
||||
# Copy c-relay-pg source files LAST (only this layer rebuilds on source changes)
|
||||
COPY src/ /build/src/
|
||||
COPY Makefile /build/Makefile
|
||||
|
||||
# Build c-relay with full static linking (only rebuilds when src/ changes)
|
||||
# Build c-relay-pg with full static linking (only rebuilds when src/ changes)
|
||||
# Disable fortification to avoid __*_chk symbols that don't exist in MUSL
|
||||
# Use conditional compilation flags based on DEBUG_BUILD argument
|
||||
RUN if [ "$DEBUG_BUILD" = "true" ]; then \
|
||||
@@ -112,7 +108,7 @@ RUN if [ "$DEBUG_BUILD" = "true" ]; then \
|
||||
echo "Building with DEBUG symbols enabled (optimized with -O2)"; \
|
||||
else \
|
||||
CFLAGS="-O2"; \
|
||||
STRIP_CMD="strip /build/c_relay_static"; \
|
||||
STRIP_CMD="strip /build/c_relay_pg_static"; \
|
||||
echo "Building optimized production binary (symbols stripped)"; \
|
||||
fi && \
|
||||
gcc -static $CFLAGS -Wall -Wextra -std=c99 \
|
||||
@@ -121,8 +117,9 @@ RUN if [ "$DEBUG_BUILD" = "true" ]; then \
|
||||
-Inostr_core_lib/cjson -Inostr_core_lib/nostr_websocket \
|
||||
src/main.c src/config.c src/dm_admin.c src/request_validator.c \
|
||||
src/nip009.c src/nip011.c src/nip013.c src/nip040.c src/nip042.c \
|
||||
src/websockets.c src/subscriptions.c src/api.c src/embedded_web_content.c \
|
||||
-o /build/c_relay_static \
|
||||
src/websockets.c src/subscriptions.c src/api.c src/embedded_web_content.c src/ip_ban.c \
|
||||
src/db_ops.c src/thread_pool.c \
|
||||
-o /build/c_relay_pg_static \
|
||||
c_utils_lib/libc_utils.a \
|
||||
nostr_core_lib/libnostr_core_x64.a \
|
||||
-lwebsockets -lssl -lcrypto -lsqlite3 -lsecp256k1 \
|
||||
@@ -131,12 +128,12 @@ RUN if [ "$DEBUG_BUILD" = "true" ]; then \
|
||||
|
||||
# Verify it's truly static
|
||||
RUN echo "=== Binary Information ===" && \
|
||||
file /build/c_relay_static && \
|
||||
ls -lh /build/c_relay_static && \
|
||||
file /build/c_relay_pg_static && \
|
||||
ls -lh /build/c_relay_pg_static && \
|
||||
echo "=== Checking for dynamic dependencies ===" && \
|
||||
(ldd /build/c_relay_static 2>&1 || echo "Binary is static") && \
|
||||
(ldd /build/c_relay_pg_static 2>&1 || echo "Binary is static") && \
|
||||
echo "=== Build complete ==="
|
||||
|
||||
# Output stage - just the binary
|
||||
FROM scratch AS output
|
||||
COPY --from=builder /build/c_relay_static /c_relay_static
|
||||
COPY --from=builder /build/c_relay_pg_static /c_relay_pg_static
|
||||
@@ -1,4 +1,4 @@
|
||||
# C-Relay Makefile
|
||||
# C-Relay-PG Makefile
|
||||
|
||||
CC = gcc
|
||||
CFLAGS = -Wall -Wextra -std=c99 -g -O2
|
||||
@@ -9,20 +9,20 @@ LIBS = -lsqlite3 -lwebsockets -lz -ldl -lpthread -lm -L/usr/local/lib -lsecp256k
|
||||
BUILD_DIR = build
|
||||
|
||||
# Source files
|
||||
MAIN_SRC = src/main.c src/config.c src/dm_admin.c src/request_validator.c src/nip009.c src/nip011.c src/nip013.c src/nip040.c src/nip042.c src/websockets.c src/subscriptions.c src/api.c src/embedded_web_content.c
|
||||
MAIN_SRC = src/main.c src/config.c src/dm_admin.c src/request_validator.c src/nip009.c src/nip011.c src/nip013.c src/nip040.c src/nip042.c src/websockets.c src/subscriptions.c src/api.c src/embedded_web_content.c src/ip_ban.c src/db_ops.c src/thread_pool.c
|
||||
NOSTR_CORE_LIB = nostr_core_lib/libnostr_core_x64.a
|
||||
C_UTILS_LIB = c_utils_lib/libc_utils.a
|
||||
|
||||
# Architecture detection
|
||||
ARCH = $(shell uname -m)
|
||||
ifeq ($(ARCH),x86_64)
|
||||
TARGET = $(BUILD_DIR)/c_relay_x86
|
||||
TARGET = $(BUILD_DIR)/c_relay_pg_x86
|
||||
else ifeq ($(ARCH),aarch64)
|
||||
TARGET = $(BUILD_DIR)/c_relay_arm64
|
||||
TARGET = $(BUILD_DIR)/c_relay_pg_arm64
|
||||
else ifeq ($(ARCH),arm64)
|
||||
TARGET = $(BUILD_DIR)/c_relay_arm64
|
||||
TARGET = $(BUILD_DIR)/c_relay_pg_arm64
|
||||
else
|
||||
TARGET = $(BUILD_DIR)/c_relay_$(ARCH)
|
||||
TARGET = $(BUILD_DIR)/c_relay_pg_$(ARCH)
|
||||
endif
|
||||
|
||||
# Default target
|
||||
@@ -82,18 +82,18 @@ force-version:
|
||||
|
||||
# Build the relay
|
||||
$(TARGET): $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
|
||||
@echo "Compiling C-Relay for architecture: $(ARCH)"
|
||||
@echo "Compiling C-Relay-PG for architecture: $(ARCH)"
|
||||
$(CC) $(CFLAGS) $(INCLUDES) $(MAIN_SRC) -o $(TARGET) $(NOSTR_CORE_LIB) $(C_UTILS_LIB) $(LIBS)
|
||||
@echo "Build complete: $(TARGET)"
|
||||
|
||||
# Build for specific architectures
|
||||
x86: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
|
||||
@echo "Building C-Relay for x86_64..."
|
||||
$(CC) $(CFLAGS) $(INCLUDES) $(MAIN_SRC) -o $(BUILD_DIR)/c_relay_x86 $(NOSTR_CORE_LIB) $(C_UTILS_LIB) $(LIBS)
|
||||
@echo "Build complete: $(BUILD_DIR)/c_relay_x86"
|
||||
@echo "Building C-Relay-PG for x86_64..."
|
||||
$(CC) $(CFLAGS) $(INCLUDES) $(MAIN_SRC) -o $(BUILD_DIR)/c_relay_pg_x86 $(NOSTR_CORE_LIB) $(C_UTILS_LIB) $(LIBS)
|
||||
@echo "Build complete: $(BUILD_DIR)/c_relay_pg_x86"
|
||||
|
||||
arm64: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
|
||||
@echo "Cross-compiling C-Relay for ARM64..."
|
||||
@echo "Cross-compiling C-Relay-PG for ARM64..."
|
||||
@if ! command -v aarch64-linux-gnu-gcc >/dev/null 2>&1; then \
|
||||
echo "ERROR: ARM64 cross-compiler not found."; \
|
||||
echo "Install with: make install-cross-tools"; \
|
||||
@@ -116,9 +116,9 @@ arm64: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(NOSTR_CORE_LIB) $(
|
||||
fi
|
||||
@echo "Using aarch64-linux-gnu-gcc with ARM64 libraries..."
|
||||
PKG_CONFIG_PATH=/usr/lib/aarch64-linux-gnu/pkgconfig:/usr/share/pkgconfig \
|
||||
aarch64-linux-gnu-gcc $(CFLAGS) $(INCLUDES) $(MAIN_SRC) -o $(BUILD_DIR)/c_relay_arm64 $(NOSTR_CORE_LIB) $(C_UTILS_LIB) \
|
||||
aarch64-linux-gnu-gcc $(CFLAGS) $(INCLUDES) $(MAIN_SRC) -o $(BUILD_DIR)/c_relay_pg_arm64 $(NOSTR_CORE_LIB) $(C_UTILS_LIB) \
|
||||
-L/usr/lib/aarch64-linux-gnu $(LIBS)
|
||||
@echo "Build complete: $(BUILD_DIR)/c_relay_arm64"
|
||||
@echo "Build complete: $(BUILD_DIR)/c_relay_pg_arm64"
|
||||
|
||||
# Install ARM64 cross-compilation dependencies
|
||||
install-arm64-deps:
|
||||
@@ -160,7 +160,7 @@ test: $(TARGET)
|
||||
init-db:
|
||||
@echo "Database initialization is now handled automatically when the server starts."
|
||||
@echo "The schema is embedded in the binary - no external files needed."
|
||||
@echo "To manually recreate database: rm -f db/c_nostr_relay.db && ./build/c_relay_x86"
|
||||
@echo "To manually recreate database: rm -f db/c_nostr_relay.db && ./build/c_relay_pg_x86"
|
||||
|
||||
# Clean build artifacts
|
||||
clean:
|
||||
@@ -180,7 +180,7 @@ install-deps:
|
||||
|
||||
# Help
|
||||
help:
|
||||
@echo "C-Relay Build System"
|
||||
@echo "C-Relay-PG Build System"
|
||||
@echo ""
|
||||
@echo "Targets:"
|
||||
@echo " all Build the relay for current architecture (default)"
|
||||
@@ -209,15 +209,15 @@ help:
|
||||
# Build fully static MUSL binaries using Docker
|
||||
static-musl-x86_64:
|
||||
@echo "Building fully static MUSL binary for x86_64..."
|
||||
docker buildx build --platform linux/amd64 -f examples/deployment/static-builder.Dockerfile -t c-relay-static-builder-x86_64 --load .
|
||||
docker run --rm -v $(PWD)/build:/output c-relay-static-builder-x86_64 sh -c "cp /c_relay_static_musl_x86_64 /output/"
|
||||
@echo "Static binary created: build/c_relay_static_musl_x86_64"
|
||||
docker buildx build --platform linux/amd64 -f examples/deployment/static-builder.Dockerfile -t c-relay-pg-static-builder-x86_64 --load .
|
||||
docker run --rm -v $(PWD)/build:/output c-relay-pg-static-builder-x86_64 sh -c "cp /c_relay_pg_static_musl_x86_64 /output/"
|
||||
@echo "Static binary created: build/c_relay_pg_static_musl_x86_64"
|
||||
|
||||
static-musl-arm64:
|
||||
@echo "Building fully static MUSL binary for ARM64..."
|
||||
docker buildx build --platform linux/arm64 -f examples/deployment/static-builder.Dockerfile -t c-relay-static-builder-arm64 --load .
|
||||
docker run --rm -v $(PWD)/build:/output c-relay-static-builder-arm64 sh -c "cp /c_relay_static_musl_x86_64 /output/c_relay_static_musl_arm64"
|
||||
@echo "Static binary created: build/c_relay_static_musl_arm64"
|
||||
docker buildx build --platform linux/arm64 -f examples/deployment/static-builder.Dockerfile -t c-relay-pg-static-builder-arm64 --load .
|
||||
docker run --rm -v $(PWD)/build:/output c-relay-pg-static-builder-arm64 sh -c "cp /c_relay_pg_static_musl_x86_64 /output/c_relay_pg_static_musl_arm64"
|
||||
@echo "Static binary created: build/c_relay_pg_static_musl_arm64"
|
||||
|
||||
static-musl: static-musl-x86_64 static-musl-arm64
|
||||
@echo "Built static MUSL binaries for both architectures"
|
||||
|
||||
+6
-6
@@ -45,21 +45,21 @@ Also included is a more standard administrative web front end. This front end co
|
||||
|
||||
## Screenshots
|
||||
|
||||

|
||||

|
||||
Main page with real time updates.
|
||||
|
||||

|
||||

|
||||
Set your configuration preferences.
|
||||
|
||||

|
||||

|
||||
View current subscriptions
|
||||
|
||||

|
||||

|
||||
Add npubs to white or black lists.
|
||||
|
||||

|
||||

|
||||
Run sql queries on the database.
|
||||
|
||||

|
||||

|
||||
Light mode.
|
||||
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
# C-Relay: High-Performance Nostr Relay
|
||||
# C-Relay-PG: High-Performance Nostr Relay
|
||||
|
||||
A blazingly fast, production-ready Nostr relay implemented in C with an innovative event-based configuration system. Built for performance, security, and ease of deployment.
|
||||
|
||||
## 🚀 Why C-Relay?
|
||||
## 🚀 Why C-Relay-PG?
|
||||
|
||||
### Event-Based Configuration
|
||||
Unlike traditional relays that require config files, C-Relay uses **cryptographically signed Nostr events** for all configuration. This means:
|
||||
Unlike traditional relays that require config files, C-Relay-PG uses **cryptographically signed Nostr events** for all configuration. This means:
|
||||
- **Zero config files** - Everything stored in the database
|
||||
- **Real-time updates** - Changes applied instantly without restart
|
||||
- **Cryptographic security** - All changes must be signed by admin
|
||||
@@ -36,7 +36,7 @@ Control your relay by sending direct messages from any Nostr client:
|
||||
|
||||
## 📋 Supported NIPs
|
||||
|
||||
C-Relay implements a comprehensive set of Nostr Improvement Proposals:
|
||||
C-Relay-PG implements a comprehensive set of Nostr Improvement Proposals:
|
||||
|
||||
- ✅ **NIP-01**: Basic protocol flow implementation
|
||||
- ✅ **NIP-09**: Event deletion
|
||||
@@ -89,12 +89,12 @@ Download and run - no dependencies required:
|
||||
|
||||
```bash
|
||||
# Download the latest static release
|
||||
wget https://git.laantungir.net/laantungir/c-relay/releases/download/v0.6.0/c-relay-v0.6.0-linux-x86_64-static
|
||||
chmod +x c-relay-v0.6.0-linux-x86_64-static
|
||||
mv c-relay-v0.6.0-linux-x86_64-static c-relay
|
||||
wget https://git.laantungir.net/laantungir/c-relay-pg/releases/download/v0.6.0/c-relay-pg-v0.6.0-linux-x86_64-static
|
||||
chmod +x c-relay-pg-v0.6.0-linux-x86_64-static
|
||||
mv c-relay-pg-v0.6.0-linux-x86_64-static c-relay-pg
|
||||
|
||||
# Run the relay
|
||||
./c-relay
|
||||
./c-relay-pg
|
||||
```
|
||||
|
||||
**Important**: On first startup, save the **Admin Private Key** displayed in the console. You'll need it for all administrative operations.
|
||||
@@ -107,8 +107,8 @@ sudo apt install -y build-essential git sqlite3 libsqlite3-dev \
|
||||
libwebsockets-dev libssl-dev libsecp256k1-dev libcurl4-openssl-dev zlib1g-dev
|
||||
|
||||
# Clone and build
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
git submodule update --init --recursive
|
||||
./make_and_restart_relay.sh
|
||||
```
|
||||
@@ -136,8 +136,8 @@ The web interface provides:
|
||||
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
git submodule update --init --recursive
|
||||
|
||||
# Build
|
||||
@@ -147,25 +147,25 @@ make clean && make
|
||||
sudo systemd/install-service.sh
|
||||
|
||||
# Start and enable
|
||||
sudo systemctl start c-relay
|
||||
sudo systemctl enable c-relay
|
||||
sudo systemctl start c-relay-pg
|
||||
sudo systemctl enable c-relay-pg
|
||||
|
||||
# Capture admin keys from logs
|
||||
sudo journalctl -u c-relay | grep "Admin Private Key"
|
||||
sudo journalctl -u c-relay-pg | grep "Admin Private Key"
|
||||
```
|
||||
|
||||
### Docker Deployment
|
||||
|
||||
```bash
|
||||
# Build Docker image
|
||||
docker build -f Dockerfile.alpine-musl -t c-relay .
|
||||
docker build -f Dockerfile.alpine-musl -t c-relay-pg .
|
||||
|
||||
# Run container
|
||||
docker run -d \
|
||||
--name c-relay \
|
||||
--name c-relay-pg \
|
||||
-p 8888:8888 \
|
||||
-v /path/to/data:/data \
|
||||
c-relay
|
||||
c-relay-pg
|
||||
```
|
||||
|
||||
### Cloud Deployment
|
||||
@@ -181,7 +181,7 @@ See [`docs/deployment_guide.md`](docs/deployment_guide.md) for detailed deployme
|
||||
|
||||
## 🔧 Configuration
|
||||
|
||||
C-Relay uses an innovative event-based configuration system. All settings are managed through signed Nostr events.
|
||||
C-Relay-PG uses an innovative event-based configuration system. All settings are managed through signed Nostr events.
|
||||
|
||||
### Basic Configuration
|
||||
|
||||
@@ -239,8 +239,8 @@ The admin private key is displayed **only once** during first startup. Store it
|
||||
|
||||
```bash
|
||||
# Save to secure location
|
||||
echo "ADMIN_PRIVKEY=your_admin_private_key" > ~/.c-relay-admin
|
||||
chmod 600 ~/.c-relay-admin
|
||||
echo "ADMIN_PRIVKEY=your_admin_private_key" > ~/.c-relay-pg-admin
|
||||
chmod 600 ~/.c-relay-pg-admin
|
||||
```
|
||||
|
||||
### Production Security
|
||||
@@ -269,9 +269,9 @@ Contributions are welcome! Please:
|
||||
|
||||
## 🔗 Links
|
||||
|
||||
- **Repository**: [https://github.com/your-org/c-relay](https://github.com/your-org/c-relay)
|
||||
- **Releases**: [https://git.laantungir.net/laantungir/c-relay/releases](https://git.laantungir.net/laantungir/c-relay/releases)
|
||||
- **Issues**: [https://github.com/your-org/c-relay/issues](https://github.com/your-org/c-relay/issues)
|
||||
- **Repository**: [https://github.com/your-org/c-relay-pg](https://github.com/your-org/c-relay-pg)
|
||||
- **Releases**: [https://git.laantungir.net/laantungir/c-relay-pg/releases](https://git.laantungir.net/laantungir/c-relay-pg/releases)
|
||||
- **Issues**: [https://github.com/your-org/c-relay-pg/issues](https://github.com/your-org/c-relay-pg/issues)
|
||||
- **Nostr Protocol**: [https://github.com/nostr-protocol/nostr](https://github.com/nostr-protocol/nostr)
|
||||
|
||||
## 💬 Support
|
||||
|
||||
@@ -41,7 +41,7 @@ This guide is specifically tailored for C programs that use:
|
||||
|
||||
```bash
|
||||
# 1. Copy the Dockerfile template (see below)
|
||||
cp /path/to/c-relay/Dockerfile.alpine-musl ./Dockerfile.static
|
||||
cp /path/to/c-relay-pg/Dockerfile.alpine-musl ./Dockerfile.static
|
||||
|
||||
# 2. Customize for your project (see Customization section)
|
||||
vim Dockerfile.static
|
||||
@@ -466,13 +466,13 @@ docker build -t my-app:latest .
|
||||
docker run --rm my-app:latest --help
|
||||
```
|
||||
|
||||
## Reusing c-relay Files
|
||||
## Reusing c-relay-pg Files
|
||||
|
||||
You can directly copy these files from c-relay:
|
||||
You can directly copy these files from c-relay-pg:
|
||||
|
||||
### 1. Dockerfile.alpine-musl
|
||||
```bash
|
||||
cp /path/to/c-relay/Dockerfile.alpine-musl ./Dockerfile.static
|
||||
cp /path/to/c-relay-pg/Dockerfile.alpine-musl ./Dockerfile.static
|
||||
```
|
||||
|
||||
Then customize:
|
||||
@@ -482,7 +482,7 @@ Then customize:
|
||||
|
||||
### 2. build_static.sh
|
||||
```bash
|
||||
cp /path/to/c-relay/build_static.sh ./
|
||||
cp /path/to/c-relay-pg/build_static.sh ./
|
||||
```
|
||||
|
||||
Then customize:
|
||||
@@ -492,7 +492,7 @@ Then customize:
|
||||
|
||||
### 3. .dockerignore (Optional)
|
||||
```bash
|
||||
cp /path/to/c-relay/.dockerignore ./
|
||||
cp /path/to/c-relay-pg/.dockerignore ./
|
||||
```
|
||||
|
||||
Helps speed up Docker builds by excluding unnecessary files.
|
||||
@@ -522,7 +522,7 @@ Helps speed up Docker builds by excluding unnecessary files.
|
||||
- [Alpine Linux](https://alpinelinux.org/)
|
||||
- [nostr_core_lib](https://github.com/chebizarro/nostr_core_lib)
|
||||
- [Static Linking Best Practices](https://www.musl-libc.org/faq.html)
|
||||
- [c-relay Implementation](./docs/musl_static_build.md)
|
||||
- [c-relay-pg Implementation](./docs/musl_static_build.md)
|
||||
|
||||
## Example: Minimal Nostr Client
|
||||
|
||||
|
||||
+191
@@ -1291,6 +1291,184 @@ body.dark-mode .sql-results-table tbody tr:nth-child(even) {
|
||||
/* background-color: var(--secondary-color); */
|
||||
}
|
||||
|
||||
/* ================================
|
||||
WEB OF TRUST (WoT) STYLES
|
||||
================================ */
|
||||
|
||||
.wot-status-row, .wot-stats-row {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: 8px 0;
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.wot-indicator {
|
||||
padding: 2px 10px;
|
||||
border-radius: 4px;
|
||||
font-weight: bold;
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
.wot-indicator.wot-found { background: #28a745; color: white; }
|
||||
.wot-indicator.wot-missing { background: #dc3545; color: white; }
|
||||
.wot-indicator.wot-unknown { background: #6c757d; color: white; }
|
||||
|
||||
.wot-level-selector { padding: 10px 0; }
|
||||
.wot-level-selector label { display: block; margin-bottom: 5px; font-weight: bold; }
|
||||
|
||||
.wot-level-btn { min-width: 100px; }
|
||||
.wot-level-btn.active {
|
||||
background: var(--accent-color, #ff0000);
|
||||
color: white;
|
||||
border-color: var(--accent-color, #ff0000);
|
||||
}
|
||||
|
||||
.wot-level-description {
|
||||
font-size: 12px;
|
||||
color: var(--primary-color);
|
||||
margin-top: 5px;
|
||||
font-style: italic;
|
||||
opacity: 0.8;
|
||||
}
|
||||
|
||||
/* Dark mode adjustments for WoT */
|
||||
body.dark-mode .wot-indicator.wot-found { background: #28a745; }
|
||||
body.dark-mode .wot-indicator.wot-missing { background: #dc3545; }
|
||||
body.dark-mode .wot-indicator.wot-unknown { background: #6c757d; }
|
||||
|
||||
/* ================================
|
||||
ADMIN ACCESS GATE STYLES
|
||||
================================ */
|
||||
|
||||
/* Access Denied Overlay */
|
||||
.access-denied-overlay {
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: 0;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
background: rgba(0, 0, 0, 0.85);
|
||||
z-index: 9999;
|
||||
display: none;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.access-denied-content {
|
||||
background: var(--card-bg);
|
||||
border: 2px solid #dc3545;
|
||||
border-radius: 12px;
|
||||
padding: 40px 60px;
|
||||
text-align: center;
|
||||
max-width: 500px;
|
||||
box-shadow: 0 10px 40px rgba(220, 53, 69, 0.3);
|
||||
}
|
||||
|
||||
.access-denied-icon {
|
||||
font-size: 64px;
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
|
||||
.access-denied-content h2 {
|
||||
color: #dc3545;
|
||||
font-size: 32px;
|
||||
margin-bottom: 20px;
|
||||
letter-spacing: 2px;
|
||||
}
|
||||
|
||||
.access-denied-message {
|
||||
font-size: 16px;
|
||||
color: var(--primary-color);
|
||||
margin-bottom: 10px;
|
||||
line-height: 1.5;
|
||||
}
|
||||
|
||||
.access-denied-submessage {
|
||||
font-size: 14px;
|
||||
color: var(--muted-color);
|
||||
margin-bottom: 30px;
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
.access-denied-logout-btn {
|
||||
background: #dc3545;
|
||||
color: white;
|
||||
border: none;
|
||||
padding: 12px 40px;
|
||||
font-size: 16px;
|
||||
font-weight: bold;
|
||||
border-radius: 6px;
|
||||
cursor: pointer;
|
||||
transition: all 0.3s ease;
|
||||
}
|
||||
|
||||
.access-denied-logout-btn:hover {
|
||||
background: #c82333;
|
||||
transform: translateY(-2px);
|
||||
box-shadow: 0 4px 12px rgba(220, 53, 69, 0.4);
|
||||
}
|
||||
|
||||
/* Admin Verification Loading Overlay */
|
||||
.admin-verification-overlay {
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: 0;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
background: rgba(0, 0, 0, 0.8);
|
||||
z-index: 9998;
|
||||
display: none;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.admin-verification-content {
|
||||
background: var(--card-bg);
|
||||
border: 1px solid var(--border-color);
|
||||
border-radius: 12px;
|
||||
padding: 40px 60px;
|
||||
text-align: center;
|
||||
max-width: 450px;
|
||||
}
|
||||
|
||||
.admin-verification-content h3 {
|
||||
color: var(--accent-color);
|
||||
font-size: 20px;
|
||||
margin-bottom: 15px;
|
||||
}
|
||||
|
||||
.admin-verification-content p {
|
||||
color: var(--muted-color);
|
||||
font-size: 14px;
|
||||
margin-top: 15px;
|
||||
}
|
||||
|
||||
/* Spinner Animation */
|
||||
.spinner {
|
||||
width: 50px;
|
||||
height: 50px;
|
||||
border: 4px solid var(--border-color);
|
||||
border-top: 4px solid var(--accent-color);
|
||||
border-radius: 50%;
|
||||
animation: spin 1s linear infinite;
|
||||
margin: 0 auto 20px;
|
||||
}
|
||||
|
||||
@keyframes spin {
|
||||
0% { transform: rotate(0deg); }
|
||||
100% { transform: rotate(360deg); }
|
||||
}
|
||||
|
||||
/* Dark mode adjustments */
|
||||
body.dark-mode .access-denied-content {
|
||||
background: var(--card-bg);
|
||||
}
|
||||
|
||||
body.dark-mode .admin-verification-content {
|
||||
background: var(--card-bg);
|
||||
}
|
||||
|
||||
.subscription-detail-row:hover {
|
||||
background-color: var(--muted-color);
|
||||
}
|
||||
@@ -1308,3 +1486,16 @@ body.dark-mode .sql-results-table tbody tr:nth-child(even) {
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
|
||||
/* ================================
|
||||
IP BANS TABLE - compact rows
|
||||
================================ */
|
||||
#ip-bans-table td, #ip-bans-table th {
|
||||
padding: 4px 8px;
|
||||
line-height: 1.3;
|
||||
font-size: 13px;
|
||||
}
|
||||
#ip-bans-table button {
|
||||
padding: 2px 8px;
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
+175
-15
@@ -4,7 +4,7 @@
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>C-Relay Admin</title>
|
||||
<title>C-Relay-PG Admin</title>
|
||||
<link rel="stylesheet" href="/api/index.css">
|
||||
</head>
|
||||
|
||||
@@ -16,6 +16,7 @@
|
||||
<li><button class="nav-item" data-page="subscriptions">Subscriptions</button></li>
|
||||
<li><button class="nav-item" data-page="configuration">Configuration</button></li>
|
||||
<li><button class="nav-item" data-page="authorization">Authorization</button></li>
|
||||
<li><button class="nav-item" data-page="ip-bans">IP BANS</button></li>
|
||||
<li><button class="nav-item" data-page="relay-events">Relay Events</button></li>
|
||||
<li><button class="nav-item" data-page="dm">DM</button></li>
|
||||
<li><button class="nav-item" data-page="database">Database Query</button></li>
|
||||
@@ -41,7 +42,7 @@
|
||||
<span class="relay-letter" data-letter="Y">Y</span>
|
||||
</div>
|
||||
<div class="relay-info">
|
||||
<div id="relay-name" class="relay-name">C-Relay</div>
|
||||
<div id="relay-name" class="relay-name">C-Relay-PG</div>
|
||||
<div id="relay-description" class="relay-description">Loading...</div>
|
||||
<div id="relay-pubkey-container" class="relay-pubkey-container">
|
||||
<div id="relay-pubkey" class="relay-pubkey">Loading...</div>
|
||||
@@ -67,6 +68,17 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Access Denied Overlay (shown when non-admin user logs in) -->
|
||||
<div id="access-denied-overlay" class="access-denied-overlay" style="display: none;">
|
||||
<div class="access-denied-content">
|
||||
<div class="access-denied-icon">⛔</div>
|
||||
<h2>ACCESS DENIED</h2>
|
||||
<p class="access-denied-message">This interface is restricted to the relay administrator.</p>
|
||||
<p class="access-denied-submessage">The logged-in account does not have admin privileges.</p>
|
||||
<button type="button" class="access-denied-logout-btn" onclick="logout()">LOGOUT</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- DATABASE STATISTICS Section -->
|
||||
<!-- Subscribe to kind 24567 events to receive real-time monitoring data -->
|
||||
<div class="section flex-section" id="databaseStatisticsSection" style="display: none;">
|
||||
@@ -100,6 +112,10 @@
|
||||
<td>Process ID</td>
|
||||
<td id="process-id">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>WebSocket Connections</td>
|
||||
<td id="websocket-connections">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Active Subscriptions</td>
|
||||
<td id="active-subscriptions">-</td>
|
||||
@@ -108,14 +124,14 @@
|
||||
<td>Memory Usage</td>
|
||||
<td id="memory-usage">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>CPU Core</td>
|
||||
<td id="cpu-core">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>CPU Usage</td>
|
||||
<td id="cpu-usage">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>CPU Core</td>
|
||||
<td id="cpu-core">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Oldest Event</td>
|
||||
<td id="oldest-event">-</td>
|
||||
@@ -251,6 +267,7 @@
|
||||
</div>
|
||||
|
||||
<!-- Auth Rules Management - Moved after configuration -->
|
||||
<!-- AUTH RULES MANAGEMENT SECTION -->
|
||||
<div class="section flex-section" id="authRulesSection" style="display: none;">
|
||||
<div class="section-header">
|
||||
AUTH RULES MANAGEMENT
|
||||
@@ -273,16 +290,11 @@ AUTH RULES MANAGEMENT
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<!-- Simplified Auth Rule Input Section -->
|
||||
<!-- Auth Rule Input Section -->
|
||||
<div id="authRuleInputSections" style="display: block;">
|
||||
|
||||
<!-- Combined Pubkey Auth Rule Section -->
|
||||
|
||||
|
||||
<div class="input-group">
|
||||
<label for="authRulePubkey">Pubkey (nsec or hex):</label>
|
||||
<input type="text" id="authRulePubkey" placeholder="nsec1... or 64-character hex pubkey">
|
||||
|
||||
<label for="authRulePubkey">Public Key (npub or hex):</label>
|
||||
<input type="text" id="authRulePubkey" placeholder="npub1... or 64-character hex pubkey">
|
||||
</div>
|
||||
<div id="whitelistWarning" class="warning-box" style="display: none;">
|
||||
<strong>⚠️ WARNING:</strong> Adding whitelist rules changes relay behavior to whitelist-only
|
||||
@@ -296,10 +308,51 @@ AUTH RULES MANAGEMENT
|
||||
BLACKLIST</button>
|
||||
<button type="button" id="refreshAuthRulesBtn">REFRESH</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<!-- WEB OF TRUST SECTION -->
|
||||
<div class="section flex-section" id="wotSection" style="display: none;">
|
||||
<div class="section-header">
|
||||
WEB OF TRUST
|
||||
</div>
|
||||
|
||||
<!-- Kind 3 Status Indicator -->
|
||||
<div id="wotKind3Status" class="wot-status-row">
|
||||
<span>Admin Contact List (kind 3):</span>
|
||||
<span id="wotKind3Indicator" class="wot-indicator wot-unknown">Checking...</span>
|
||||
</div>
|
||||
|
||||
<!-- WoT Level Selector -->
|
||||
<div class="wot-level-selector">
|
||||
<label>WoT Level:</label>
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="wotLevel0Btn" class="wot-level-btn" onclick="setWotLevel(0)">
|
||||
OFF
|
||||
</button>
|
||||
<button type="button" id="wotLevel1Btn" class="wot-level-btn" onclick="setWotLevel(1)">
|
||||
WRITE ONLY
|
||||
</button>
|
||||
<button type="button" id="wotLevel2Btn" class="wot-level-btn" onclick="setWotLevel(2)">
|
||||
FULL
|
||||
</button>
|
||||
</div>
|
||||
<div class="wot-level-description" id="wotLevelDescription">
|
||||
Level 0: Open relay — anyone can read and write
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- WoT Stats -->
|
||||
<div class="wot-stats-row">
|
||||
<span>Whitelisted Pubkeys:</span>
|
||||
<span id="wotWhitelistCount">—</span>
|
||||
</div>
|
||||
|
||||
<!-- Sync Button -->
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="wotSyncBtn" onclick="syncWot()">SYNC FROM KIND 3</button>
|
||||
<button type="button" id="wotRefreshBtn" onclick="loadWotStatus()">REFRESH STATUS</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@@ -334,6 +387,109 @@ AUTH RULES MANAGEMENT
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- IP BANS Section -->
|
||||
<div class="section" id="ipBansSection" style="display: none;">
|
||||
<div class="section-header">
|
||||
IP BAN MANAGEMENT
|
||||
</div>
|
||||
|
||||
<!-- Statistics Cards -->
|
||||
<div class="input-group">
|
||||
<div class="config-table-container">
|
||||
<table class="config-table" id="ip-bans-stats-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Total IPs Tracked</th>
|
||||
<th>Currently Banned</th>
|
||||
<th>Total Bans Issued</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td id="ip-bans-total">-</td>
|
||||
<td id="ip-bans-active">-</td>
|
||||
<td id="ip-bans-issued">-</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Add Ban Form -->
|
||||
<div class="input-group">
|
||||
<h3>Manually Ban IP Address</h3>
|
||||
<div class="form-group">
|
||||
<label for="ban-ip-input">IP Address:</label>
|
||||
<input type="text" id="ban-ip-input" placeholder="192.168.1.100">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="ban-duration-select">Ban Duration:</label>
|
||||
<select id="ban-duration-select">
|
||||
<option value="3600">1 Hour</option>
|
||||
<option value="86400" selected>24 Hours</option>
|
||||
<option value="604800">7 Days</option>
|
||||
<option value="2592000">30 Days</option>
|
||||
<option value="31536000">1 Year</option>
|
||||
<option value="999999999">Permanent</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="add-ban-btn">BAN IP</button>
|
||||
</div>
|
||||
<div id="add-ban-status" class="status-message"></div>
|
||||
</div>
|
||||
|
||||
<!-- Whitelist Management -->
|
||||
<div class="input-group">
|
||||
<h3>IP Whitelist (Never Banned)</h3>
|
||||
<p style="font-size:13px;opacity:0.8;">IPs in this list are never idle-banned. Comma-separated.</p>
|
||||
<div class="form-group">
|
||||
<label for="whitelist-ip-input">Add IP to Whitelist:</label>
|
||||
<input type="text" id="whitelist-ip-input" placeholder="103.81.231.220">
|
||||
</div>
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="add-whitelist-btn">ADD TO WHITELIST</button>
|
||||
</div>
|
||||
<div id="whitelist-status" class="status-message"></div>
|
||||
<div id="whitelist-current" style="margin-top:8px;font-size:13px;"></div>
|
||||
</div>
|
||||
|
||||
<!-- Filter Controls -->
|
||||
<div class="input-group">
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="ip-ban-filter-all" class="active">All IPs</button>
|
||||
<button type="button" id="ip-ban-filter-banned">Currently Banned</button>
|
||||
<button type="button" id="ip-ban-filter-expired">Expired</button>
|
||||
<button type="button" id="refresh-ip-bans-btn">REFRESH</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- IP Bans List -->
|
||||
<div class="input-group">
|
||||
<label>Banned IP Addresses:</label>
|
||||
<div class="config-table-container">
|
||||
<table class="config-table" id="ip-bans-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>IP Address</th>
|
||||
<th>Status</th>
|
||||
<th>Banned Until</th>
|
||||
<th>Failures</th>
|
||||
<th>Authed Successfully</th>
|
||||
<th>Connection Attempts</th>
|
||||
<th>Actions</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody id="ip-bans-tbody">
|
||||
<tr>
|
||||
<td colspan="7" style="text-align: center;">Click REFRESH to load IP bans</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- RELAY EVENTS Section -->
|
||||
<div class="section" id="relayEventsSection" style="display: none;">
|
||||
<div class="section-header">
|
||||
@@ -419,6 +575,10 @@ AUTH RULES MANAGEMENT
|
||||
<option value="event_kinds">Event Kinds Distribution</option>
|
||||
<option value="time_stats">Time-based Statistics</option>
|
||||
</optgroup>
|
||||
<optgroup label="IP Ban Queries">
|
||||
<option value="banned_ips_summary">Banned IPs Summary</option>
|
||||
<option value="banned_ips_list">All Banned IP Addresses</option>
|
||||
</optgroup>
|
||||
<optgroup label="Query History" id="history-group">
|
||||
<!-- Dynamically populated from localStorage -->
|
||||
</optgroup>
|
||||
|
||||
+1060
-101
File diff suppressed because it is too large
Load Diff
+95
-1
@@ -3297,8 +3297,18 @@ var NostrTools = (() => {
|
||||
this.enablePing = opts.enablePing;
|
||||
}
|
||||
async ensureRelay(url, params) {
|
||||
const rawUrl = url;
|
||||
const debugEnabled = typeof window !== "undefined" && !!window.__SIMPLE_POOL_DEBUG__;
|
||||
url = normalizeURL(url);
|
||||
let relay = this.relays.get(url);
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL ensureRelay START", {
|
||||
rawUrl,
|
||||
normalizedUrl: url,
|
||||
hadRelay: !!relay,
|
||||
relayMapKeysBefore: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
if (!relay) {
|
||||
relay = new AbstractRelay(url, {
|
||||
verifyEvent: this.trustedRelayURLs.has(url) ? alwaysTrue : this.verifyEvent,
|
||||
@@ -3306,13 +3316,38 @@ var NostrTools = (() => {
|
||||
enablePing: this.enablePing
|
||||
});
|
||||
relay.onclose = () => {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL ensureRelay onclose", {
|
||||
normalizedUrl: url,
|
||||
relayMapKeysBeforeDelete: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
this.relays.delete(url);
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL ensureRelay onclose complete", {
|
||||
normalizedUrl: url,
|
||||
relayMapKeysAfterDelete: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
};
|
||||
if (params?.connectionTimeout)
|
||||
relay.connectionTimeout = params.connectionTimeout;
|
||||
this.relays.set(url, relay);
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL ensureRelay created relay", {
|
||||
normalizedUrl: url,
|
||||
relayMapKeysAfterCreate: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
}
|
||||
await relay.connect();
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL ensureRelay connected", {
|
||||
normalizedUrl: url,
|
||||
relayConnected: relay.connected,
|
||||
relayMapKeysAfterConnect: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
return relay;
|
||||
}
|
||||
close(relays) {
|
||||
@@ -3334,16 +3369,26 @@ var NostrTools = (() => {
|
||||
}
|
||||
subscribeMany(relays, filters, params) {
|
||||
params.onauth = params.onauth || params.doauth;
|
||||
const debugEnabled = typeof window !== "undefined" && !!window.__SIMPLE_POOL_DEBUG__;
|
||||
const request = [];
|
||||
const uniqUrls = [];
|
||||
for (let i2 = 0; i2 < relays.length; i2++) {
|
||||
const url = normalizeURL(relays[i2]);
|
||||
if (uniqUrls.indexOf(url) === -1) {
|
||||
uniqUrls.push(url);
|
||||
for (let f2 = 0; f2 < filters.length; f2++) {
|
||||
request.push({ url, filter: filters[f2] });
|
||||
}
|
||||
}
|
||||
}
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL subscribeMany", {
|
||||
relaysInput: relays,
|
||||
uniqUrls,
|
||||
filtersCount: filters.length,
|
||||
requestsCount: request.length
|
||||
});
|
||||
}
|
||||
return this.subscribeMap(request, params);
|
||||
}
|
||||
subscribeMap(requests, params) {
|
||||
@@ -3391,14 +3436,31 @@ var NostrTools = (() => {
|
||||
_knownIds.add(id);
|
||||
return have;
|
||||
};
|
||||
const debugEnabled = typeof window !== "undefined" && !!window.__SIMPLE_POOL_DEBUG__;
|
||||
const allOpened = Promise.all(
|
||||
requests.map(async ({ url, filter }, i2) => {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL subscribeMap request", {
|
||||
index: i2,
|
||||
url,
|
||||
filterKinds: filter?.kinds,
|
||||
hasAuthorFilter: !!filter?.authors,
|
||||
hasPTagFilter: !!filter?.["#p"]
|
||||
});
|
||||
}
|
||||
let relay;
|
||||
try {
|
||||
relay = await this.ensureRelay(url, {
|
||||
connectionTimeout: params.maxWait ? Math.max(params.maxWait * 0.8, params.maxWait - 1e3) : void 0
|
||||
});
|
||||
} catch (err) {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL subscribeMap ensureRelay FAILED", {
|
||||
index: i2,
|
||||
url,
|
||||
error: err?.message || String(err)
|
||||
});
|
||||
}
|
||||
handleClose(i2, err?.message || String(err));
|
||||
return;
|
||||
}
|
||||
@@ -3480,18 +3542,50 @@ var NostrTools = (() => {
|
||||
return events[0] || null;
|
||||
}
|
||||
publish(relays, event, options) {
|
||||
return relays.map(normalizeURL).map(async (url, i2, arr) => {
|
||||
const debugEnabled = typeof window !== "undefined" && !!window.__SIMPLE_POOL_DEBUG__;
|
||||
const normalizedRelays = relays.map(normalizeURL);
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL publish START", {
|
||||
relaysInput: relays,
|
||||
normalizedRelays,
|
||||
eventKind: event?.kind,
|
||||
eventId: event?.id
|
||||
});
|
||||
}
|
||||
return normalizedRelays.map(async (url, i2, arr) => {
|
||||
if (arr.indexOf(url) !== i2) {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL publish duplicate URL", { url, index: i2 });
|
||||
}
|
||||
return Promise.reject("duplicate url");
|
||||
}
|
||||
let r = await this.ensureRelay(url);
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL publish ensured relay", {
|
||||
url,
|
||||
relayConnected: r?.connected,
|
||||
relayMapKeys: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
return r.publish(event).catch(async (err) => {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL publish ERROR", {
|
||||
url,
|
||||
error: err?.message || String(err)
|
||||
});
|
||||
}
|
||||
if (err instanceof Error && err.message.startsWith("auth-required: ") && options?.onauth) {
|
||||
await r.auth(options.onauth);
|
||||
return r.publish(event);
|
||||
}
|
||||
throw err;
|
||||
}).then((reason) => {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL publish SUCCESS", {
|
||||
url,
|
||||
reason
|
||||
});
|
||||
}
|
||||
if (this.trackRelays) {
|
||||
let set = this.seenOn.get(event.id);
|
||||
if (!set) {
|
||||
|
||||
+16
-11
@@ -1,6 +1,6 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Build fully static MUSL binaries for C-Relay using Alpine Docker
|
||||
# Build fully static MUSL binaries for C-Relay-PG using Alpine Docker
|
||||
# Produces truly portable binaries with zero runtime dependencies
|
||||
|
||||
set -e
|
||||
@@ -14,11 +14,11 @@ DEBUG_BUILD=false
|
||||
if [[ "$1" == "--debug" ]]; then
|
||||
DEBUG_BUILD=true
|
||||
echo "=========================================="
|
||||
echo "C-Relay MUSL Static Binary Builder (DEBUG MODE)"
|
||||
echo "C-Relay-PG MUSL Static Binary Builder (DEBUG MODE)"
|
||||
echo "=========================================="
|
||||
else
|
||||
echo "=========================================="
|
||||
echo "C-Relay MUSL Static Binary Builder (PRODUCTION MODE)"
|
||||
echo "C-Relay-PG MUSL Static Binary Builder (PRODUCTION MODE)"
|
||||
echo "=========================================="
|
||||
fi
|
||||
echo "Project directory: $SCRIPT_DIR"
|
||||
@@ -63,20 +63,25 @@ ARCH=$(uname -m)
|
||||
case "$ARCH" in
|
||||
x86_64)
|
||||
PLATFORM="linux/amd64"
|
||||
OUTPUT_NAME="c_relay_static_x86_64"
|
||||
OUTPUT_NAME="c_relay_pg_static_x86_64"
|
||||
;;
|
||||
aarch64|arm64)
|
||||
PLATFORM="linux/arm64"
|
||||
OUTPUT_NAME="c_relay_static_arm64"
|
||||
OUTPUT_NAME="c_relay_pg_static_arm64"
|
||||
;;
|
||||
*)
|
||||
echo "WARNING: Unknown architecture: $ARCH"
|
||||
echo "Defaulting to linux/amd64"
|
||||
PLATFORM="linux/amd64"
|
||||
OUTPUT_NAME="c_relay_static_${ARCH}"
|
||||
OUTPUT_NAME="c_relay_pg_static_${ARCH}"
|
||||
;;
|
||||
esac
|
||||
|
||||
# Append _debug suffix to output name for debug builds so production binary is never overwritten
|
||||
if [ "$DEBUG_BUILD" = true ]; then
|
||||
OUTPUT_NAME="${OUTPUT_NAME}_debug"
|
||||
fi
|
||||
|
||||
echo "Building for platform: $PLATFORM"
|
||||
echo "Output binary: $OUTPUT_NAME"
|
||||
echo ""
|
||||
@@ -111,14 +116,14 @@ echo "=========================================="
|
||||
echo "This will:"
|
||||
echo " - Use Alpine Linux (native MUSL)"
|
||||
echo " - Build all dependencies statically"
|
||||
echo " - Compile c-relay with full static linking"
|
||||
echo " - Compile c-relay-pg with full static linking"
|
||||
echo ""
|
||||
|
||||
$DOCKER_CMD build \
|
||||
--platform "$PLATFORM" \
|
||||
--build-arg DEBUG_BUILD=$DEBUG_BUILD \
|
||||
-f "$DOCKERFILE" \
|
||||
-t c-relay-musl-builder:latest \
|
||||
-t c-relay-pg-musl-builder:latest \
|
||||
--progress=plain \
|
||||
. || {
|
||||
echo ""
|
||||
@@ -142,14 +147,14 @@ $DOCKER_CMD build \
|
||||
--build-arg DEBUG_BUILD=$DEBUG_BUILD \
|
||||
--target builder \
|
||||
-f "$DOCKERFILE" \
|
||||
-t c-relay-static-builder-stage:latest \
|
||||
-t c-relay-pg-static-builder-stage:latest \
|
||||
. > /dev/null 2>&1
|
||||
|
||||
# Create a temporary container to copy the binary
|
||||
CONTAINER_ID=$($DOCKER_CMD create c-relay-static-builder-stage:latest)
|
||||
CONTAINER_ID=$($DOCKER_CMD create c-relay-pg-static-builder-stage:latest)
|
||||
|
||||
# Copy binary from container
|
||||
$DOCKER_CMD cp "$CONTAINER_ID:/build/c_relay_static" "$BUILD_DIR/$OUTPUT_NAME" || {
|
||||
$DOCKER_CMD cp "$CONTAINER_ID:/build/c_relay_pg_static" "$BUILD_DIR/$OUTPUT_NAME" || {
|
||||
echo "ERROR: Failed to extract binary from container"
|
||||
$DOCKER_CMD rm "$CONTAINER_ID" 2>/dev/null
|
||||
exit 1
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
key,value,data_type,description,category,requires_restart,created_at,updated_at
|
||||
|
+9970
File diff suppressed because it is too large
Load Diff
+5
-12
@@ -1,19 +1,12 @@
|
||||
#!/bin/bash
|
||||
# Restart the service
|
||||
sudo systemctl stop c-relay-pg.service
|
||||
|
||||
# Copy the binary to the deployment location
|
||||
cp build/c_relay_x86 ~/Storage/c_relay/crelay
|
||||
|
||||
# Copy the service file to systemd (use the main service file)
|
||||
sudo cp systemd/c-relay.service /etc/systemd/system/c-relay-local.service
|
||||
|
||||
# Reload systemd daemon to pick up the new service
|
||||
sudo systemctl daemon-reload
|
||||
|
||||
# Enable the service (if not already enabled)
|
||||
sudo systemctl enable c-relay-local.service
|
||||
cp build/c_relay_pg_static_x86_64 ~/Storage/c_relay_pg/crelay
|
||||
|
||||
# Restart the service
|
||||
sudo systemctl restart c-relay-local.service
|
||||
sudo systemctl restart c-relay-pg.service
|
||||
|
||||
# Show service status
|
||||
sudo systemctl status c-relay-local.service --no-pager -l
|
||||
sudo systemctl status c-relay-pg.service --no-pager -l
|
||||
|
||||
+8
-8
@@ -1,24 +1,24 @@
|
||||
#!/bin/bash
|
||||
|
||||
# C-Relay Static Binary Deployment Script
|
||||
# Deploys build/c_relay_static_x86_64 to server via ssh
|
||||
# C-Relay-PG Static Binary Deployment Script
|
||||
# Deploys build/c_relay_pg_static_x86_64 to server via ssh
|
||||
|
||||
set -e
|
||||
|
||||
# Configuration
|
||||
LOCAL_BINARY="build/c_relay_static_x86_64"
|
||||
REMOTE_BINARY_PATH="/usr/local/bin/c_relay/c_relay"
|
||||
SERVICE_NAME="c-relay"
|
||||
LOCAL_BINARY="build/c_relay_pg_static_x86_64"
|
||||
REMOTE_BINARY_PATH="/usr/local/bin/c_relay_pg/c_relay_pg"
|
||||
SERVICE_NAME="c-relay-pg"
|
||||
|
||||
# Create backup
|
||||
ssh ubuntu@laantungir.com "sudo cp '$REMOTE_BINARY_PATH' '${REMOTE_BINARY_PATH}.backup.$(date +%Y%m%d_%H%M%S)'" 2>/dev/null || true
|
||||
|
||||
# Upload binary to temp location
|
||||
scp "$LOCAL_BINARY" "ubuntu@laantungir.com:/tmp/c_relay.tmp"
|
||||
scp "$LOCAL_BINARY" "ubuntu@laantungir.com:/tmp/c_relay_pg.tmp"
|
||||
|
||||
# Install binary
|
||||
ssh ubuntu@laantungir.com "sudo mv '/tmp/c_relay.tmp' '$REMOTE_BINARY_PATH'"
|
||||
ssh ubuntu@laantungir.com "sudo chown c-relay:c-relay '$REMOTE_BINARY_PATH'"
|
||||
ssh ubuntu@laantungir.com "sudo mv '/tmp/c_relay_pg.tmp' '$REMOTE_BINARY_PATH'"
|
||||
ssh ubuntu@laantungir.com "sudo chown c-relay-pg:c-relay-pg '$REMOTE_BINARY_PATH'"
|
||||
ssh ubuntu@laantungir.com "sudo chmod +x '$REMOTE_BINARY_PATH'"
|
||||
|
||||
# Reload systemd and restart service
|
||||
|
||||
Executable
+122
@@ -0,0 +1,122 @@
|
||||
#!/bin/bash
|
||||
|
||||
# C-Relay-PG Debug Binary Deployment Script
|
||||
# Deploys build/c_relay_pg_static_x86_64_debug to server for CPU profiling
|
||||
#
|
||||
# Usage:
|
||||
# ./deploy_lt_debug.sh -- deploy debug binary and restart
|
||||
# ./deploy_lt_debug.sh --profile -- deploy, then run perf and fetch results
|
||||
#
|
||||
# After deploying, profile with:
|
||||
# sudo perf record -g -p $(pgrep c_relay_pg) -- sleep 30
|
||||
# sudo perf report --stdio --sort=symbol --no-children -n 2>/dev/null | head -80
|
||||
#
|
||||
# Restore production binary:
|
||||
# ./deploy_lt.sh
|
||||
|
||||
set -e
|
||||
|
||||
LOCAL_DEBUG_BINARY="build/c_relay_pg_static_x86_64_debug"
|
||||
REMOTE_BINARY_PATH="/usr/local/bin/c_relay_pg/c_relay_pg"
|
||||
REMOTE_PROD_BACKUP="/usr/local/bin/c_relay_pg/c_relay_pg.production"
|
||||
SERVICE_NAME="c-relay-pg"
|
||||
REMOTE_HOST="ubuntu@laantungir.com"
|
||||
|
||||
# Check debug binary exists
|
||||
if [ ! -f "$LOCAL_DEBUG_BINARY" ]; then
|
||||
echo "ERROR: Debug binary not found: $LOCAL_DEBUG_BINARY"
|
||||
echo ""
|
||||
echo "Build it first with:"
|
||||
echo " ./build_static.sh --debug"
|
||||
echo ""
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "=========================================="
|
||||
echo "C-Relay-PG Debug Deployment"
|
||||
echo "=========================================="
|
||||
echo "Binary: $LOCAL_DEBUG_BINARY ($(du -h "$LOCAL_DEBUG_BINARY" | cut -f1))"
|
||||
echo "Target: $REMOTE_HOST:$REMOTE_BINARY_PATH"
|
||||
echo ""
|
||||
echo "WARNING: Debug binary has symbols and is larger than production."
|
||||
echo " It is safe to run but should not be left deployed long-term."
|
||||
echo ""
|
||||
|
||||
# Backup production binary (only if not already backed up)
|
||||
echo "Backing up production binary..."
|
||||
ssh "$REMOTE_HOST" "
|
||||
if [ ! -f '$REMOTE_PROD_BACKUP' ]; then
|
||||
sudo cp '$REMOTE_BINARY_PATH' '$REMOTE_PROD_BACKUP'
|
||||
echo 'Production binary backed up to $REMOTE_PROD_BACKUP'
|
||||
else
|
||||
echo 'Production backup already exists, skipping'
|
||||
fi
|
||||
"
|
||||
|
||||
# Upload debug binary
|
||||
echo "Uploading debug binary..."
|
||||
scp "$LOCAL_DEBUG_BINARY" "$REMOTE_HOST:/tmp/c_relay_pg_debug.tmp"
|
||||
|
||||
# Install debug binary
|
||||
echo "Installing debug binary..."
|
||||
ssh "$REMOTE_HOST" "
|
||||
sudo mv '/tmp/c_relay_pg_debug.tmp' '$REMOTE_BINARY_PATH'
|
||||
sudo chown c-relay-pg:c-relay-pg '$REMOTE_BINARY_PATH'
|
||||
sudo chmod +x '$REMOTE_BINARY_PATH'
|
||||
"
|
||||
|
||||
# Restart service
|
||||
echo "Restarting c-relay-pg service..."
|
||||
ssh "$REMOTE_HOST" "sudo systemctl daemon-reload && sudo systemctl restart '$SERVICE_NAME'"
|
||||
|
||||
echo ""
|
||||
echo "✓ Debug binary deployed and service restarted"
|
||||
echo ""
|
||||
|
||||
# If --profile flag, run perf automatically
|
||||
if [ "$1" = "--profile" ]; then
|
||||
echo "=========================================="
|
||||
echo "Running perf profile (30 seconds)..."
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
|
||||
# Wait for relay to start
|
||||
sleep 3
|
||||
|
||||
ssh "$REMOTE_HOST" "
|
||||
PID=\$(pgrep c_relay_pg)
|
||||
if [ -z \"\$PID\" ]; then
|
||||
echo 'ERROR: c_relay_pg not running'
|
||||
exit 1
|
||||
fi
|
||||
echo \"Profiling PID \$PID for 30 seconds...\"
|
||||
sudo perf record -g -p \$PID -- sleep 30
|
||||
echo ''
|
||||
echo '=== TOP FUNCTIONS BY CPU ==='
|
||||
sudo perf report --stdio --sort=symbol --no-children -n 2>/dev/null | head -60
|
||||
"
|
||||
else
|
||||
echo "=========================================="
|
||||
echo "Next Steps: Profile the relay"
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
echo "SSH to server and run:"
|
||||
echo " sudo perf record -g -p \$(pgrep c_relay_pg) -- sleep 30"
|
||||
echo " sudo perf report --stdio --sort=symbol --no-children -n 2>/dev/null | head -60"
|
||||
echo ""
|
||||
echo "Or run with --profile to do it automatically:"
|
||||
echo " ./deploy_lt_debug.sh --profile"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "=========================================="
|
||||
echo "Restore Production Binary When Done"
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
echo "Run this to restore the production binary:"
|
||||
echo " ./deploy_lt.sh"
|
||||
echo ""
|
||||
echo "Or manually on the server:"
|
||||
echo " sudo cp '$REMOTE_PROD_BACKUP' '$REMOTE_BINARY_PATH'"
|
||||
echo " sudo systemctl restart $SERVICE_NAME"
|
||||
echo ""
|
||||
@@ -1,4 +1,4 @@
|
||||
# C-Relay Administrator API Implementation Plan
|
||||
# C-Relay-PG Administrator API Implementation Plan
|
||||
|
||||
## Problem Analysis
|
||||
|
||||
@@ -161,7 +161,7 @@ Would require changing schema, migration scripts, and storage logic.
|
||||
#### README.md Documentation Format:
|
||||
|
||||
```markdown
|
||||
# C-Relay Administrator API
|
||||
# C-Relay-PG Administrator API
|
||||
|
||||
## Authentication
|
||||
All admin commands require signing with the admin private key generated during first startup.
|
||||
|
||||
@@ -0,0 +1,190 @@
|
||||
# Agent Browser Testing Guide for C-Relay-PG
|
||||
|
||||
This document explains how to use the `agent-browser` CLI tool to test the c-relay-pg admin web UI from an AI agent context (no physical display required).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **agent-browser** installed globally: `npm install -g agent-browser`
|
||||
- Binary location: `/home/user/.nvm/versions/node/v24.14.1/bin/agent-browser`
|
||||
- The relay must be running locally (default port 8888)
|
||||
- Test keys configured in `.test_keys`
|
||||
|
||||
## Starting the Relay for Local Testing
|
||||
|
||||
```bash
|
||||
./make_and_restart_relay.sh -t
|
||||
```
|
||||
|
||||
This reads `.test_keys` and starts the relay with:
|
||||
- `ADMIN_PUBKEY` — the hex public key of the admin account
|
||||
- `ADMIN_PRIVKEY` — the hex secret key (used for browser login, not by the relay itself)
|
||||
- `SERVER_PRIVKEY` — the relay's own private key
|
||||
|
||||
## Key URLs
|
||||
|
||||
| URL | Purpose |
|
||||
|-----|---------|
|
||||
| `http://127.0.0.1:8888/api/index.html` | Admin web UI (the correct entry point) |
|
||||
| `http://127.0.0.1:8888/` | Returns 406 without NIP-11 Accept header — **do not use for browser testing** |
|
||||
| `http://127.0.0.1:8888/` with `Accept: application/nostr+json` | NIP-11 relay info JSON |
|
||||
|
||||
**Important:** The root URL `/` is a WebSocket/NIP-11 endpoint, not an HTML page. Always use `/api/index.html` for browser testing.
|
||||
|
||||
## Admin Login Credentials
|
||||
|
||||
The admin nsec for the current `.test_keys` configuration:
|
||||
|
||||
```
|
||||
nsec: nsec1zkn0hlt4jvcvn9yt5p7ea4m7f82pf3ph0gj3d4rlcz4s64x5z86satv9qm
|
||||
hex: 15a6fbfd759330c9948ba07d9ed77e49d414c4377a2516d47fc0ab0d54d411f5
|
||||
npub: npub1tlyzk6slzt8d989f4pn3synk98fea64ea3jmrdc7dtd0kw8uxlas9a9fwr
|
||||
hex pubkey: 5fc82b6a1f12ced29ca9a86718127629d39eeab9ec65b1b71e6adafb38fc37fb
|
||||
```
|
||||
|
||||
## Complete Login Flow with agent-browser
|
||||
|
||||
### Step 1: Open the admin UI
|
||||
|
||||
```bash
|
||||
agent-browser open http://127.0.0.1:8888/api/index.html
|
||||
agent-browser wait --load networkidle
|
||||
```
|
||||
|
||||
### Step 2: Take an interactive snapshot to see what is on screen
|
||||
|
||||
```bash
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
You will see a login modal with buttons like:
|
||||
- `"Browser Extension"` — skip this
|
||||
- `"Local Key"` — **use this one**
|
||||
- `"Seed Phrase"` — skip
|
||||
- `"Nostr Connect"` — skip
|
||||
- `"Read Only"` — skip
|
||||
|
||||
### Step 3: Click "Local Key"
|
||||
|
||||
```bash
|
||||
agent-browser click @e15
|
||||
```
|
||||
|
||||
(The ref number may vary — use the ref from the snapshot output for the "Local Key" button.)
|
||||
|
||||
After clicking, a text input appears asking for the secret key.
|
||||
|
||||
### Step 4: Enter the admin nsec
|
||||
|
||||
```bash
|
||||
agent-browser fill @e14 "nsec1zkn0hlt4jvcvn9yt5p7ea4m7f82pf3ph0gj3d4rlcz4s64x5z86satv9qm"
|
||||
```
|
||||
|
||||
(Use the ref from the snapshot for the textbox element.)
|
||||
|
||||
### Step 5: Click "Import Key"
|
||||
|
||||
```bash
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
Check the snapshot — the "Import Key" button should now be enabled. Click it:
|
||||
|
||||
```bash
|
||||
agent-browser click @e15
|
||||
```
|
||||
|
||||
### Step 6: Click "Continue" on the success screen
|
||||
|
||||
After import, a success screen appears with "Continue" button:
|
||||
|
||||
```bash
|
||||
agent-browser wait 800
|
||||
agent-browser snapshot -i
|
||||
agent-browser click @e19
|
||||
```
|
||||
|
||||
(Use the ref from the snapshot for the "Continue" button.)
|
||||
|
||||
### Step 7: Wait for admin UI to load
|
||||
|
||||
```bash
|
||||
agent-browser wait 5000
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
You should now see the admin dashboard with:
|
||||
- Statistics table (Database Size, Total Events, PID, etc.)
|
||||
- Navigation buttons (Statistics, Subscriptions, Configuration, Authorization, etc.)
|
||||
- Admin profile area showing "admin" label
|
||||
|
||||
## Navigating Admin Sections
|
||||
|
||||
After login, use the sidebar navigation buttons:
|
||||
|
||||
```bash
|
||||
# View configuration
|
||||
agent-browser click @e8 # Configuration button ref
|
||||
|
||||
# View authorization rules
|
||||
agent-browser click @e9 # Authorization button ref
|
||||
|
||||
# View statistics
|
||||
agent-browser click @e6 # Statistics button ref
|
||||
|
||||
# Always snapshot after navigation to see results
|
||||
agent-browser wait 2500
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
## Checking for Errors
|
||||
|
||||
```bash
|
||||
# View browser console logs
|
||||
agent-browser console
|
||||
|
||||
# View JavaScript errors
|
||||
agent-browser errors
|
||||
|
||||
# View relay server logs
|
||||
tail -n 100 relay.log
|
||||
```
|
||||
|
||||
## Chained Command Example (Full Login in One Shot)
|
||||
|
||||
```bash
|
||||
agent-browser open http://127.0.0.1:8888/api/index.html && \
|
||||
agent-browser wait --load networkidle && \
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
Then use refs from the snapshot to complete login steps.
|
||||
|
||||
## Tips for AI Agents
|
||||
|
||||
1. **Always use `/api/index.html`** — never the root URL
|
||||
2. **Use `snapshot -i`** after every action to see the current interactive elements and their refs
|
||||
3. **Refs change** between snapshots — always re-snapshot before clicking
|
||||
4. **Wait after clicks** — use `agent-browser wait 2000` (milliseconds) between actions that trigger async operations
|
||||
5. **The login flow has 3 screens**: method selection → key input → success confirmation
|
||||
6. **Console logs are cumulative** — they show all logs since page load, which is useful for debugging admin API responses
|
||||
7. **The relay log** at `relay.log` shows server-side processing of admin commands
|
||||
8. **Command chaining** with `&&` works — the browser daemon persists between commands
|
||||
|
||||
## Verifying Admin API is Working
|
||||
|
||||
After login, the statistics page should show populated data:
|
||||
- Database Size (e.g., "4 KB")
|
||||
- Process ID (the relay PID)
|
||||
- WebSocket Connections count
|
||||
- Memory Usage
|
||||
|
||||
If these show "-" or "Loading...", check:
|
||||
1. `relay.log` for errors
|
||||
2. `agent-browser console` for JavaScript errors
|
||||
3. `agent-browser errors` for page-level errors
|
||||
|
||||
## Closing the Browser
|
||||
|
||||
```bash
|
||||
agent-browser close --all
|
||||
```
|
||||
@@ -324,7 +324,7 @@ int main() {
|
||||
}
|
||||
```
|
||||
|
||||
## Migration Plan for c-relay
|
||||
## Migration Plan for c-relay-pg
|
||||
|
||||
### Phase 1: Extract Debug System
|
||||
1. Create `c_utils_lib` repository
|
||||
@@ -333,16 +333,16 @@ int main() {
|
||||
4. Add basic tests
|
||||
|
||||
### Phase 2: Add Versioning System
|
||||
1. Extract version generation logic from c-relay
|
||||
1. Extract version generation logic from c-relay-pg
|
||||
2. Create reusable version utilities
|
||||
3. Update c-relay to use new system
|
||||
3. Update c-relay-pg to use new system
|
||||
4. Update nostr_core_lib to use new system
|
||||
|
||||
### Phase 3: Add as Submodule
|
||||
1. Add `c_utils_lib` as submodule to c-relay
|
||||
2. Update c-relay Makefile
|
||||
3. Update includes in c-relay source files
|
||||
4. Remove old debug files from c-relay
|
||||
1. Add `c_utils_lib` as submodule to c-relay-pg
|
||||
2. Update c-relay-pg Makefile
|
||||
3. Update includes in c-relay-pg source files
|
||||
4. Remove old debug files from c-relay-pg
|
||||
|
||||
### Phase 4: Documentation & Examples
|
||||
1. Create comprehensive README
|
||||
@@ -352,7 +352,7 @@ int main() {
|
||||
|
||||
## Benefits
|
||||
|
||||
### For c-relay
|
||||
### For c-relay-pg
|
||||
- Cleaner separation of concerns
|
||||
- Reusable utilities across projects
|
||||
- Easier to maintain and test
|
||||
@@ -379,7 +379,7 @@ int main() {
|
||||
- Memory leak detection (valgrind)
|
||||
|
||||
### Integration Tests
|
||||
- Test with real projects (c-relay, nostr_core_lib)
|
||||
- Test with real projects (c-relay-pg, nostr_core_lib)
|
||||
- Cross-platform testing
|
||||
- Performance benchmarks
|
||||
|
||||
@@ -433,7 +433,7 @@ MIT License - permissive and suitable for learning and commercial use.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. ✅ Successfully integrated into c-relay
|
||||
1. ✅ Successfully integrated into c-relay-pg
|
||||
2. ✅ Successfully integrated into nostr_core_lib
|
||||
3. ✅ All tests passing
|
||||
4. ✅ Documentation complete
|
||||
@@ -449,7 +449,7 @@ MIT License - permissive and suitable for learning and commercial use.
|
||||
4. Create build system
|
||||
5. Write tests
|
||||
6. Create documentation
|
||||
7. Integrate into c-relay
|
||||
7. Integrate into c-relay-pg
|
||||
8. Publish to GitHub
|
||||
|
||||
---
|
||||
|
||||
@@ -2,13 +2,13 @@
|
||||
|
||||
## Overview
|
||||
|
||||
This document provides a step-by-step implementation plan for creating the `c_utils_lib` library and integrating it into the c-relay project.
|
||||
This document provides a step-by-step implementation plan for creating the `c_utils_lib` library and integrating it into the c-relay-pg project.
|
||||
|
||||
## Phase 1: Repository Setup & Structure
|
||||
|
||||
### Step 1.1: Create Repository Structure
|
||||
|
||||
**Location**: Create outside c-relay project (sibling directory)
|
||||
**Location**: Create outside c-relay-pg project (sibling directory)
|
||||
|
||||
```bash
|
||||
# Create directory structure
|
||||
@@ -42,7 +42,7 @@ git branch -M main
|
||||
|
||||
### Step 2.1: Move Debug Files
|
||||
|
||||
**Source files** (from c-relay):
|
||||
**Source files** (from c-relay-pg):
|
||||
- `src/debug.c` → `c_utils_lib/src/debug.c`
|
||||
- `src/debug.h` → `c_utils_lib/include/c_utils/debug.h`
|
||||
|
||||
@@ -484,20 +484,20 @@ How to integrate into projects:
|
||||
3. Code examples
|
||||
4. Migration from standalone utilities
|
||||
|
||||
## Phase 7: Integration with c-relay
|
||||
## Phase 7: Integration with c-relay-pg
|
||||
|
||||
### Step 7.1: Add as Submodule
|
||||
|
||||
```bash
|
||||
cd /path/to/c-relay
|
||||
cd /path/to/c-relay-pg
|
||||
git submodule add <repo-url> c_utils_lib
|
||||
git submodule update --init --recursive
|
||||
```
|
||||
|
||||
### Step 7.2: Update c-relay Makefile
|
||||
### Step 7.2: Update c-relay-pg Makefile
|
||||
|
||||
```makefile
|
||||
# Add to c-relay Makefile
|
||||
# Add to c-relay-pg Makefile
|
||||
C_UTILS_LIB = c_utils_lib/libc_utils.a
|
||||
|
||||
# Update includes
|
||||
@@ -514,7 +514,7 @@ $(C_UTILS_LIB):
|
||||
$(TARGET): $(C_UTILS_LIB) ...
|
||||
```
|
||||
|
||||
### Step 7.3: Update c-relay Source Files
|
||||
### Step 7.3: Update c-relay-pg Source Files
|
||||
|
||||
**Changes needed**:
|
||||
|
||||
@@ -543,7 +543,7 @@ $(TARGET): $(C_UTILS_LIB) ...
|
||||
### Step 7.4: Test Integration
|
||||
|
||||
```bash
|
||||
cd c-relay
|
||||
cd c-relay-pg
|
||||
make clean
|
||||
make
|
||||
./make_and_restart_relay.sh
|
||||
@@ -556,7 +556,7 @@ Verify:
|
||||
|
||||
## Phase 8: Version System Integration
|
||||
|
||||
### Step 8.1: Update c-relay Makefile for Versioning
|
||||
### Step 8.1: Update c-relay-pg Makefile for Versioning
|
||||
|
||||
```makefile
|
||||
# Add version generation
|
||||
@@ -567,7 +567,7 @@ src/version.h: .git/refs/tags/*
|
||||
$(TARGET): src/version.h ...
|
||||
```
|
||||
|
||||
### Step 8.2: Update c-relay to Use Generated Version
|
||||
### Step 8.2: Update c-relay-pg to Use Generated Version
|
||||
|
||||
Replace hardcoded version in `src/main.h` with:
|
||||
```c
|
||||
@@ -583,7 +583,7 @@ Replace hardcoded version in `src/main.h` with:
|
||||
- **Phase 4**: Build System - 2 hours
|
||||
- **Phase 5**: Examples & Tests - 3 hours
|
||||
- **Phase 6**: Documentation - 3 hours
|
||||
- **Phase 7**: c-relay Integration - 2 hours
|
||||
- **Phase 7**: c-relay-pg Integration - 2 hours
|
||||
- **Phase 8**: Version Integration - 2 hours
|
||||
|
||||
**Total**: ~19 hours
|
||||
@@ -593,11 +593,11 @@ Replace hardcoded version in `src/main.h` with:
|
||||
- [ ] c_utils_lib builds successfully
|
||||
- [ ] All tests pass
|
||||
- [ ] Examples compile and run
|
||||
- [ ] c-relay integrates successfully
|
||||
- [ ] Debug output works in c-relay
|
||||
- [ ] c-relay-pg integrates successfully
|
||||
- [ ] Debug output works in c-relay-pg
|
||||
- [ ] Version generation works
|
||||
- [ ] Documentation complete
|
||||
- [ ] No regressions in c-relay functionality
|
||||
- [ ] No regressions in c-relay-pg functionality
|
||||
|
||||
## Next Steps
|
||||
|
||||
@@ -608,7 +608,7 @@ Replace hardcoded version in `src/main.h` with:
|
||||
5. Create build system
|
||||
6. Write tests and examples
|
||||
7. Create documentation
|
||||
8. Integrate into c-relay
|
||||
8. Integrate into c-relay-pg
|
||||
9. Test thoroughly
|
||||
10. Publish to GitHub
|
||||
|
||||
|
||||
@@ -78,9 +78,9 @@ Configuration events follow the standard Nostr event format with kind 33334:
|
||||
|
||||
#### `relay_software`
|
||||
- **Description**: Software identifier for NIP-11
|
||||
- **Default**: `"c-relay"`
|
||||
- **Default**: `"c-relay-pg"`
|
||||
- **Format**: String, max 64 characters
|
||||
- **Example**: `"c-relay v1.0.0"`
|
||||
- **Example**: `"c-relay-pg v1.0.0"`
|
||||
|
||||
#### `relay_version`
|
||||
- **Description**: Software version string
|
||||
@@ -366,7 +366,7 @@ sqlite3 relay.nrdb "SELECT json_pretty(json_object(
|
||||
#### Invalid Parameter Values
|
||||
```bash
|
||||
# Check relay logs for validation errors
|
||||
journalctl -u c-relay | grep "Configuration.*invalid\|Invalid.*configuration"
|
||||
journalctl -u c-relay-pg | grep "Configuration.*invalid\|Invalid.*configuration"
|
||||
|
||||
# Common issues:
|
||||
# - Numeric values outside valid ranges
|
||||
@@ -395,10 +395,10 @@ ORDER BY date DESC;"
|
||||
#### Resource Usage After Changes
|
||||
```bash
|
||||
# Monitor system resources after configuration updates
|
||||
top -p $(pgrep c_relay)
|
||||
top -p $(pgrep c_relay_pg)
|
||||
|
||||
# Check for memory leaks
|
||||
ps aux | grep c_relay | awk '{print $6}' # RSS memory
|
||||
ps aux | grep c_relay_pg | awk '{print $6}' # RSS memory
|
||||
```
|
||||
|
||||
### Emergency Recovery
|
||||
@@ -425,7 +425,7 @@ nostrtool event \
|
||||
# If database is corrupted, backup and recreate
|
||||
cp relay.nrdb relay.nrdb.backup
|
||||
rm relay.nrdb*
|
||||
./build/c_relay_x86 # Creates fresh database with new keys
|
||||
./build/c_relay_pg_x86 # Creates fresh database with new keys
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
+14
-14
@@ -21,22 +21,22 @@ typedef enum {
|
||||
|
||||
```bash
|
||||
# Production (default - no debug output)
|
||||
./c_relay_x86
|
||||
./c_relay_pg_x86
|
||||
|
||||
# Show errors only
|
||||
./c_relay_x86 --debug-level=1
|
||||
./c_relay_pg_x86 --debug-level=1
|
||||
|
||||
# Show errors and warnings
|
||||
./c_relay_x86 --debug-level=2
|
||||
./c_relay_pg_x86 --debug-level=2
|
||||
|
||||
# Show errors, warnings, and info (recommended for development)
|
||||
./c_relay_x86 --debug-level=3
|
||||
./c_relay_pg_x86 --debug-level=3
|
||||
|
||||
# Show all debug messages
|
||||
./c_relay_x86 --debug-level=4
|
||||
./c_relay_pg_x86 --debug-level=4
|
||||
|
||||
# Show everything including trace with file:line (very verbose)
|
||||
./c_relay_x86 --debug-level=5
|
||||
./c_relay_pg_x86 --debug-level=5
|
||||
```
|
||||
|
||||
## Implementation
|
||||
@@ -475,7 +475,7 @@ When `g_debug_level = 0` (constant), you'll see the compiler has removed all deb
|
||||
|
||||
### Level 3 (Errors + Warnings + Info)
|
||||
```
|
||||
[2025-01-12 14:30:15] [INFO ] Initializing C-Relay v0.4.6
|
||||
[2025-01-12 14:30:15] [INFO ] Initializing C-Relay-PG v0.4.6
|
||||
[2025-01-12 14:30:15] [INFO ] Loading configuration from database
|
||||
[2025-01-12 14:30:15] [ERROR] Failed to open database: permission denied
|
||||
[2025-01-12 14:30:16] [WARN ] Port 8888 unavailable, trying 8889
|
||||
@@ -484,7 +484,7 @@ When `g_debug_level = 0` (constant), you'll see the compiler has removed all deb
|
||||
|
||||
### Level 4 (All Debug Messages)
|
||||
```
|
||||
[2025-01-12 14:30:15] [INFO ] Initializing C-Relay v0.4.6
|
||||
[2025-01-12 14:30:15] [INFO ] Initializing C-Relay-PG v0.4.6
|
||||
[2025-01-12 14:30:15] [DEBUG] Opening database: build/abc123...def.db
|
||||
[2025-01-12 14:30:15] [DEBUG] Executing schema initialization
|
||||
[2025-01-12 14:30:15] [INFO ] SQLite WAL mode enabled
|
||||
@@ -496,7 +496,7 @@ When `g_debug_level = 0` (constant), you'll see the compiler has removed all deb
|
||||
|
||||
### Level 5 (Everything Including file:line for ALL messages)
|
||||
```
|
||||
[2025-01-12 14:30:15] [INFO ] [main.c:1607] Initializing C-Relay v0.4.6
|
||||
[2025-01-12 14:30:15] [INFO ] [main.c:1607] Initializing C-Relay-PG v0.4.6
|
||||
[2025-01-12 14:30:15] [DEBUG] [main.c:348] Opening database: build/abc123...def.db
|
||||
[2025-01-12 14:30:15] [TRACE] [main.c:330] Entering init_database()
|
||||
[2025-01-12 14:30:15] [ERROR] [config.c:125] Database locked
|
||||
@@ -525,11 +525,11 @@ Update the existing `log_*` functions to use the new debug macros
|
||||
make clean && make
|
||||
|
||||
# Test different levels
|
||||
./build/c_relay_x86 # No output
|
||||
./build/c_relay_x86 --debug-level=1 # Errors only
|
||||
./build/c_relay_x86 --debug-level=3 # Info + warnings + errors
|
||||
./build/c_relay_x86 --debug-level=4 # All debug messages
|
||||
./build/c_relay_x86 --debug-level=5 # Everything with file:line on TRACE
|
||||
./build/c_relay_pg_x86 # No output
|
||||
./build/c_relay_pg_x86 --debug-level=1 # Errors only
|
||||
./build/c_relay_pg_x86 --debug-level=3 # Info + warnings + errors
|
||||
./build/c_relay_pg_x86 --debug-level=4 # All debug messages
|
||||
./build/c_relay_pg_x86 --debug-level=5 # Everything with file:line on TRACE
|
||||
```
|
||||
|
||||
### Step 5: Gradual Migration (Ongoing)
|
||||
|
||||
@@ -34,7 +34,7 @@ static const struct {
|
||||
// NIP-11 Relay Information (relay keys will be populated at runtime)
|
||||
{"relay_description", "High-performance C Nostr relay with SQLite storage"},
|
||||
{"relay_contact", ""},
|
||||
{"relay_software", "https://git.laantungir.net/laantungir/c-relay.git"},
|
||||
{"relay_software", "https://git.laantungir.net/laantungir/c-relay-pg.git"},
|
||||
{"relay_version", "v1.0.0"},
|
||||
|
||||
// NIP-13 Proof of Work (pow_min_difficulty = 0 means PoW disabled)
|
||||
|
||||
+51
-51
@@ -52,11 +52,11 @@ sudo apt install -y build-essential git sqlite3 libsqlite3-dev \
|
||||
#### User and Directory Setup
|
||||
```bash
|
||||
# Create dedicated system user
|
||||
sudo useradd --system --home-dir /opt/c-relay --shell /bin/false c-relay
|
||||
sudo useradd --system --home-dir /opt/c-relay-pg --shell /bin/false c-relay-pg
|
||||
|
||||
# Create application directory
|
||||
sudo mkdir -p /opt/c-relay
|
||||
sudo chown c-relay:c-relay /opt/c-relay
|
||||
sudo mkdir -p /opt/c-relay-pg
|
||||
sudo chown c-relay-pg:c-relay-pg /opt/c-relay-pg
|
||||
```
|
||||
|
||||
### Build and Installation
|
||||
@@ -64,8 +64,8 @@ sudo chown c-relay:c-relay /opt/c-relay
|
||||
#### Automated Installation (Recommended)
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
git submodule update --init --recursive
|
||||
|
||||
# Build
|
||||
@@ -81,12 +81,12 @@ sudo systemd/install-service.sh
|
||||
make clean && make
|
||||
|
||||
# Install binary
|
||||
sudo cp build/c_relay_x86 /opt/c-relay/
|
||||
sudo chown c-relay:c-relay /opt/c-relay/c_relay_x86
|
||||
sudo chmod +x /opt/c-relay/c_relay_x86
|
||||
sudo cp build/c_relay_pg_x86 /opt/c-relay-pg/
|
||||
sudo chown c-relay-pg:c-relay-pg /opt/c-relay-pg/c_relay_pg_x86
|
||||
sudo chmod +x /opt/c-relay-pg/c_relay_pg_x86
|
||||
|
||||
# Install systemd service
|
||||
sudo cp systemd/c-relay.service /etc/systemd/system/
|
||||
sudo cp systemd/c-relay-pg.service /etc/systemd/system/
|
||||
sudo systemctl daemon-reload
|
||||
```
|
||||
|
||||
@@ -95,22 +95,22 @@ sudo systemctl daemon-reload
|
||||
#### Start and Enable Service
|
||||
```bash
|
||||
# Start the service
|
||||
sudo systemctl start c-relay
|
||||
sudo systemctl start c-relay-pg
|
||||
|
||||
# Enable auto-start on boot
|
||||
sudo systemctl enable c-relay
|
||||
sudo systemctl enable c-relay-pg
|
||||
|
||||
# Check status
|
||||
sudo systemctl status c-relay
|
||||
sudo systemctl status c-relay-pg
|
||||
```
|
||||
|
||||
#### Capture Admin Keys (CRITICAL)
|
||||
```bash
|
||||
# View startup logs to get admin keys
|
||||
sudo journalctl -u c-relay --since "5 minutes ago" | grep -A 10 "IMPORTANT: SAVE THIS ADMIN PRIVATE KEY"
|
||||
sudo journalctl -u c-relay-pg --since "5 minutes ago" | grep -A 10 "IMPORTANT: SAVE THIS ADMIN PRIVATE KEY"
|
||||
|
||||
# Or check the full log
|
||||
sudo journalctl -u c-relay --no-pager | grep "Admin Private Key"
|
||||
sudo journalctl -u c-relay-pg --no-pager | grep "Admin Private Key"
|
||||
```
|
||||
|
||||
⚠️ **CRITICAL**: Save the admin private key immediately - it's only shown once and is needed for all configuration updates!
|
||||
@@ -151,8 +151,8 @@ sudo iptables-save > /etc/iptables/rules.v4
|
||||
ssh -i your-key.pem ubuntu@your-instance-ip
|
||||
|
||||
# Use the simple deployment script
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
sudo examples/deployment/simple-vps/deploy.sh
|
||||
```
|
||||
|
||||
@@ -168,10 +168,10 @@ sudo examples/deployment/simple-vps/deploy.sh
|
||||
sudo mkfs.ext4 /dev/xvdf
|
||||
sudo mkdir /data
|
||||
sudo mount /dev/xvdf /data
|
||||
sudo chown c-relay:c-relay /data
|
||||
sudo chown c-relay-pg:c-relay-pg /data
|
||||
|
||||
# Update systemd service to use /data
|
||||
sudo sed -i 's/WorkingDirectory=\/opt\/c-relay/WorkingDirectory=\/data/' /etc/systemd/system/c-relay.service
|
||||
sudo sed -i 's/WorkingDirectory=\/opt\/c-relay-pg/WorkingDirectory=\/data/' /etc/systemd/system/c-relay-pg.service
|
||||
sudo systemctl daemon-reload
|
||||
```
|
||||
|
||||
@@ -180,7 +180,7 @@ sudo systemctl daemon-reload
|
||||
#### Compute Engine Setup
|
||||
```bash
|
||||
# Create VM instance (e2-micro or larger)
|
||||
gcloud compute instances create c-relay-instance \
|
||||
gcloud compute instances create c-relay-pg-instance \
|
||||
--image-family=ubuntu-2204-lts \
|
||||
--image-project=ubuntu-os-cloud \
|
||||
--machine-type=e2-micro \
|
||||
@@ -193,9 +193,9 @@ gcloud compute firewall-rules create allow-nostr-relay \
|
||||
--target-tags nostr-relay
|
||||
|
||||
# SSH and deploy
|
||||
gcloud compute ssh c-relay-instance
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
gcloud compute ssh c-relay-pg-instance
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
sudo examples/deployment/simple-vps/deploy.sh
|
||||
```
|
||||
|
||||
@@ -203,13 +203,13 @@ sudo examples/deployment/simple-vps/deploy.sh
|
||||
```bash
|
||||
# Create and attach persistent disk
|
||||
gcloud compute disks create relay-data --size=50GB
|
||||
gcloud compute instances attach-disk c-relay-instance --disk=relay-data
|
||||
gcloud compute instances attach-disk c-relay-pg-instance --disk=relay-data
|
||||
|
||||
# Format and mount
|
||||
sudo mkfs.ext4 /dev/sdb
|
||||
sudo mkdir /data
|
||||
sudo mount /dev/sdb /data
|
||||
sudo chown c-relay:c-relay /data
|
||||
sudo chown c-relay-pg:c-relay-pg /data
|
||||
```
|
||||
|
||||
### DigitalOcean
|
||||
@@ -223,8 +223,8 @@ sudo chown c-relay:c-relay /data
|
||||
ssh root@your-droplet-ip
|
||||
|
||||
# Deploy relay
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
examples/deployment/simple-vps/deploy.sh
|
||||
```
|
||||
|
||||
@@ -245,8 +245,8 @@ The `examples/deployment/` directory contains ready-to-use scripts:
|
||||
### Simple VPS Deployment
|
||||
```bash
|
||||
# Clone repository and run automated deployment
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
sudo examples/deployment/simple-vps/deploy.sh
|
||||
```
|
||||
|
||||
@@ -417,9 +417,9 @@ LOG_FILE="/var/log/relay-monitor.log"
|
||||
DATE=$(date '+%Y-%m-%d %H:%M:%S')
|
||||
|
||||
# Check if relay is running
|
||||
if ! pgrep -f "c_relay_x86" > /dev/null; then
|
||||
if ! pgrep -f "c_relay_pg_x86" > /dev/null; then
|
||||
echo "[$DATE] ERROR: Relay process not running" >> $LOG_FILE
|
||||
systemctl restart c-relay
|
||||
systemctl restart c-relay-pg
|
||||
fi
|
||||
|
||||
# Check port availability
|
||||
@@ -428,14 +428,14 @@ if ! netstat -tln | grep -q ":8888"; then
|
||||
fi
|
||||
|
||||
# Check database file
|
||||
RELAY_DB=$(find /opt/c-relay -name "*.nrdb" | head -1)
|
||||
RELAY_DB=$(find /opt/c-relay-pg -name "*.nrdb" | head -1)
|
||||
if [[ -n "$RELAY_DB" ]]; then
|
||||
DB_SIZE=$(du -h "$RELAY_DB" | cut -f1)
|
||||
echo "[$DATE] INFO: Database size: $DB_SIZE" >> $LOG_FILE
|
||||
fi
|
||||
|
||||
# Check memory usage
|
||||
MEM_USAGE=$(ps aux | grep c_relay_x86 | grep -v grep | awk '{print $6}')
|
||||
MEM_USAGE=$(ps aux | grep c_relay_pg_x86 | grep -v grep | awk '{print $6}')
|
||||
if [[ -n "$MEM_USAGE" ]]; then
|
||||
echo "[$DATE] INFO: Memory usage: ${MEM_USAGE}KB" >> $LOG_FILE
|
||||
fi
|
||||
@@ -454,8 +454,8 @@ sudo chmod +x /usr/local/bin/relay-monitor.sh
|
||||
|
||||
#### Centralized Logging with rsyslog
|
||||
```bash
|
||||
# /etc/rsyslog.d/50-c-relay.conf
|
||||
if $programname == 'c-relay' then /var/log/c-relay.log
|
||||
# /etc/rsyslog.d/50-c-relay-pg.conf
|
||||
if $programname == 'c-relay-pg' then /var/log/c-relay-pg.log
|
||||
& stop
|
||||
```
|
||||
|
||||
@@ -465,7 +465,7 @@ if $programname == 'c-relay' then /var/log/c-relay.log
|
||||
```yaml
|
||||
# /etc/prometheus/prometheus.yml
|
||||
scrape_configs:
|
||||
- job_name: 'c-relay'
|
||||
- job_name: 'c-relay-pg'
|
||||
static_configs:
|
||||
- targets: ['localhost:8888']
|
||||
metrics_path: '/metrics' # If implemented
|
||||
@@ -479,30 +479,30 @@ scrape_configs:
|
||||
#### Service User Restrictions
|
||||
```bash
|
||||
# Restrict service user
|
||||
sudo usermod -s /bin/false c-relay
|
||||
sudo usermod -d /opt/c-relay c-relay
|
||||
sudo usermod -s /bin/false c-relay-pg
|
||||
sudo usermod -d /opt/c-relay-pg c-relay-pg
|
||||
|
||||
# Set proper permissions
|
||||
sudo chmod 700 /opt/c-relay
|
||||
sudo chown -R c-relay:c-relay /opt/c-relay
|
||||
sudo chmod 700 /opt/c-relay-pg
|
||||
sudo chown -R c-relay-pg:c-relay-pg /opt/c-relay-pg
|
||||
```
|
||||
|
||||
#### File System Restrictions
|
||||
```bash
|
||||
# Mount data directory with appropriate options
|
||||
echo "/dev/sdb /opt/c-relay ext4 defaults,noexec,nosuid,nodev 0 2" >> /etc/fstab
|
||||
echo "/dev/sdb /opt/c-relay-pg ext4 defaults,noexec,nosuid,nodev 0 2" >> /etc/fstab
|
||||
```
|
||||
|
||||
### Network Security
|
||||
|
||||
#### Fail2Ban Configuration
|
||||
```ini
|
||||
# /etc/fail2ban/jail.d/c-relay.conf
|
||||
[c-relay-dos]
|
||||
# /etc/fail2ban/jail.d/c-relay-pg.conf
|
||||
[c-relay-pg-dos]
|
||||
enabled = true
|
||||
port = 8888
|
||||
filter = c-relay-dos
|
||||
logpath = /var/log/c-relay.log
|
||||
filter = c-relay-pg-dos
|
||||
logpath = /var/log/c-relay-pg.log
|
||||
maxretry = 10
|
||||
findtime = 60
|
||||
bantime = 300
|
||||
@@ -534,9 +534,9 @@ sudo mkfs.ext4 /dev/mapper/relay-data
|
||||
#!/bin/bash
|
||||
# /usr/local/bin/backup-relay.sh
|
||||
|
||||
BACKUP_DIR="/backup/c-relay"
|
||||
BACKUP_DIR="/backup/c-relay-pg"
|
||||
DATE=$(date +%Y%m%d_%H%M%S)
|
||||
RELAY_DB=$(find /opt/c-relay -name "*.nrdb" | head -1)
|
||||
RELAY_DB=$(find /opt/c-relay-pg -name "*.nrdb" | head -1)
|
||||
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
|
||||
@@ -574,7 +574,7 @@ sudo apt install -y awscli
|
||||
aws configure
|
||||
|
||||
# Sync backups to S3
|
||||
aws s3 sync /backup/c-relay/ s3://your-backup-bucket/c-relay/ --delete
|
||||
aws s3 sync /backup/c-relay-pg/ s3://your-backup-bucket/c-relay-pg/ --delete
|
||||
```
|
||||
|
||||
### Disaster Recovery
|
||||
@@ -583,16 +583,16 @@ aws s3 sync /backup/c-relay/ s3://your-backup-bucket/c-relay/ --delete
|
||||
```bash
|
||||
# 1. Restore from backup
|
||||
gunzip backup/relay_backup_20231201_020000.nrdb.gz
|
||||
cp backup/relay_backup_20231201_020000.nrdb /opt/c-relay/
|
||||
cp backup/relay_backup_20231201_020000.nrdb /opt/c-relay-pg/
|
||||
|
||||
# 2. Fix permissions
|
||||
sudo chown c-relay:c-relay /opt/c-relay/*.nrdb
|
||||
sudo chown c-relay-pg:c-relay-pg /opt/c-relay-pg/*.nrdb
|
||||
|
||||
# 3. Restart service
|
||||
sudo systemctl restart c-relay
|
||||
sudo systemctl restart c-relay-pg
|
||||
|
||||
# 4. Verify recovery
|
||||
sudo journalctl -u c-relay --since "1 minute ago"
|
||||
sudo journalctl -u c-relay-pg --since "1 minute ago"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
+25
-25
@@ -2,7 +2,7 @@
|
||||
|
||||
## Overview
|
||||
|
||||
This guide explains how to build truly portable MUSL-based static binaries of c-relay using Alpine Linux Docker containers. These binaries have **zero runtime dependencies** and work on any Linux distribution.
|
||||
This guide explains how to build truly portable MUSL-based static binaries of c-relay-pg using Alpine Linux Docker containers. These binaries have **zero runtime dependencies** and work on any Linux distribution.
|
||||
|
||||
## Why MUSL?
|
||||
|
||||
@@ -36,8 +36,8 @@ This guide explains how to build truly portable MUSL-based static binaries of c-
|
||||
./build_static.sh
|
||||
|
||||
# The binary will be created at:
|
||||
# build/c_relay_static_musl_x86_64 (on x86_64)
|
||||
# build/c_relay_static_musl_arm64 (on ARM64)
|
||||
# build/c_relay_pg_static_musl_x86_64 (on x86_64)
|
||||
# build/c_relay_pg_static_musl_arm64 (on ARM64)
|
||||
```
|
||||
|
||||
### What Happens During Build
|
||||
@@ -56,7 +56,7 @@ This guide explains how to build truly portable MUSL-based static binaries of c-
|
||||
- Includes required NIPs: 001, 006, 013, 017, 019, 044, 059
|
||||
- Produces static library (~316KB)
|
||||
|
||||
4. **c-relay Compilation**: Links everything statically:
|
||||
4. **c-relay-pg Compilation**: Links everything statically:
|
||||
- All source files compiled with `-static` flag
|
||||
- Fortification disabled to avoid `__*_chk` symbols
|
||||
- Results in ~7.6MB stripped binary
|
||||
@@ -78,7 +78,7 @@ FROM alpine:3.19 AS builder
|
||||
- Install build tools and static libraries
|
||||
- Build dependencies from source
|
||||
- Compile nostr_core_lib with MUSL flags
|
||||
- Compile c-relay with full static linking
|
||||
- Compile c-relay-pg with full static linking
|
||||
- Strip binary to reduce size
|
||||
|
||||
# Stage 2: Output (scratch)
|
||||
@@ -93,7 +93,7 @@ FROM scratch AS output
|
||||
CFLAGS="-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -Wall -Wextra -std=c99 -fPIC -O2"
|
||||
```
|
||||
|
||||
**For c-relay:**
|
||||
**For c-relay-pg:**
|
||||
```bash
|
||||
gcc -static -O2 -Wall -Wextra -std=c99 \
|
||||
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 \
|
||||
@@ -110,7 +110,7 @@ gcc -static -O2 -Wall -Wextra -std=c99 \
|
||||
- `-U_FORTIFY_SOURCE` (undefine any existing definition)
|
||||
- `-D_FORTIFY_SOURCE=0` (set to 0)
|
||||
|
||||
This must be applied to **both** nostr_core_lib and c-relay compilation.
|
||||
This must be applied to **both** nostr_core_lib and c-relay-pg compilation.
|
||||
|
||||
### NIP Dependencies
|
||||
|
||||
@@ -129,26 +129,26 @@ The build includes these NIPs in nostr_core_lib:
|
||||
|
||||
```bash
|
||||
# Should show "statically linked"
|
||||
file build/c_relay_static_musl_x86_64
|
||||
file build/c_relay_pg_static_musl_x86_64
|
||||
|
||||
# Should show "not a dynamic executable"
|
||||
ldd build/c_relay_static_musl_x86_64
|
||||
ldd build/c_relay_pg_static_musl_x86_64
|
||||
|
||||
# Check size (should be ~7.6MB)
|
||||
ls -lh build/c_relay_static_musl_x86_64
|
||||
ls -lh build/c_relay_pg_static_musl_x86_64
|
||||
```
|
||||
|
||||
### Test Execution
|
||||
|
||||
```bash
|
||||
# Show help
|
||||
./build/c_relay_static_musl_x86_64 --help
|
||||
./build/c_relay_pg_static_musl_x86_64 --help
|
||||
|
||||
# Show version
|
||||
./build/c_relay_static_musl_x86_64 --version
|
||||
./build/c_relay_pg_static_musl_x86_64 --version
|
||||
|
||||
# Run relay
|
||||
./build/c_relay_static_musl_x86_64 --port 8888
|
||||
./build/c_relay_pg_static_musl_x86_64 --port 8888
|
||||
```
|
||||
|
||||
### Cross-Distribution Testing
|
||||
@@ -157,16 +157,16 @@ Test the binary on different distributions to verify portability:
|
||||
|
||||
```bash
|
||||
# Alpine Linux
|
||||
docker run --rm -v $(pwd)/build:/app alpine:latest /app/c_relay_static_musl_x86_64 --version
|
||||
docker run --rm -v $(pwd)/build:/app alpine:latest /app/c_relay_pg_static_musl_x86_64 --version
|
||||
|
||||
# Ubuntu
|
||||
docker run --rm -v $(pwd)/build:/app ubuntu:latest /app/c_relay_static_musl_x86_64 --version
|
||||
docker run --rm -v $(pwd)/build:/app ubuntu:latest /app/c_relay_pg_static_musl_x86_64 --version
|
||||
|
||||
# Debian
|
||||
docker run --rm -v $(pwd)/build:/app debian:latest /app/c_relay_static_musl_x86_64 --version
|
||||
docker run --rm -v $(pwd)/build:/app debian:latest /app/c_relay_pg_static_musl_x86_64 --version
|
||||
|
||||
# CentOS
|
||||
docker run --rm -v $(pwd)/build:/app centos:latest /app/c_relay_static_musl_x86_64 --version
|
||||
docker run --rm -v $(pwd)/build:/app centos:latest /app/c_relay_pg_static_musl_x86_64 --version
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
@@ -187,7 +187,7 @@ newgrp docker # Or logout and login again
|
||||
|
||||
**Solution**: Ensure fortification is disabled in both:
|
||||
1. nostr_core_lib build.sh (line 534)
|
||||
2. c-relay compilation flags in Dockerfile
|
||||
2. c-relay-pg compilation flags in Dockerfile
|
||||
|
||||
### Binary Won't Execute
|
||||
|
||||
@@ -213,26 +213,26 @@ newgrp docker # Or logout and login again
|
||||
|
||||
```bash
|
||||
# Copy binary to server
|
||||
scp build/c_relay_static_musl_x86_64 user@server:/opt/c-relay/
|
||||
scp build/c_relay_pg_static_musl_x86_64 user@server:/opt/c-relay-pg/
|
||||
|
||||
# Run on server (no dependencies needed!)
|
||||
ssh user@server
|
||||
cd /opt/c-relay
|
||||
./c_relay_static_musl_x86_64 --port 8888
|
||||
cd /opt/c-relay-pg
|
||||
./c_relay_pg_static_musl_x86_64 --port 8888
|
||||
```
|
||||
|
||||
### SystemD Service
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=C-Relay Nostr Relay (MUSL Static)
|
||||
Description=C-Relay-PG Nostr Relay (MUSL Static)
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=c-relay
|
||||
WorkingDirectory=/opt/c-relay
|
||||
ExecStart=/opt/c-relay/c_relay_static_musl_x86_64
|
||||
User=c-relay-pg
|
||||
WorkingDirectory=/opt/c-relay-pg
|
||||
ExecStart=/opt/c-relay-pg/c_relay_pg_static_musl_x86_64
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
|
||||
@@ -23,7 +23,7 @@ We chose Option B because:
|
||||
|
||||
## Detailed Implementation Steps
|
||||
|
||||
### Phase 1: Configuration Setup in c-relay
|
||||
### Phase 1: Configuration Setup in c-relay-pg
|
||||
|
||||
#### 1.1 Add Configuration Parameter
|
||||
**File:** `src/default_config_event.h`
|
||||
@@ -293,7 +293,7 @@ int nostr_nip17_send_dm(cJSON* dm_event,
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Update c-relay Call Sites
|
||||
### Phase 4: Update c-relay-pg Call Sites
|
||||
|
||||
#### 4.1 Update src/api.c
|
||||
**Location:** Line 1319
|
||||
@@ -470,7 +470,7 @@ causing compatibility issues.
|
||||
- [ ] Update `nostr_nip17_send_dm()` signature and implementation
|
||||
- [ ] Update `nip017.h` function declaration and documentation
|
||||
|
||||
### c-relay Changes
|
||||
### c-relay-pg Changes
|
||||
- [ ] Add `nip59_timestamp_max_delay_sec` to `default_config_event.h`
|
||||
- [ ] Add validation in `config.c` for new parameter
|
||||
- [ ] Update `src/api.c` call site to pass `max_delay_sec`
|
||||
@@ -494,7 +494,7 @@ causing compatibility issues.
|
||||
|
||||
If issues arise:
|
||||
1. Revert nostr_core_lib changes (git revert in submodule)
|
||||
2. Revert c-relay changes
|
||||
2. Revert c-relay-pg changes
|
||||
3. Configuration parameter will be ignored if not used
|
||||
4. Default behavior (0) provides maximum compatibility
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## Overview
|
||||
|
||||
This document describes the design for a general-purpose SQL query interface for the C-Relay admin API. This allows administrators to execute read-only SQL queries against the relay database through cryptographically signed kind 23456 events with NIP-44 encrypted command arrays.
|
||||
This document describes the design for a general-purpose SQL query interface for the C-Relay-PG admin API. This allows administrators to execute read-only SQL queries against the relay database through cryptographically signed kind 23456 events with NIP-44 encrypted command arrays.
|
||||
|
||||
## Security Model
|
||||
|
||||
@@ -271,7 +271,7 @@ const SQL_QUERY_TEMPLATES = {
|
||||
};
|
||||
|
||||
// Query history management (localStorage)
|
||||
const QUERY_HISTORY_KEY = 'c_relay_sql_history';
|
||||
const QUERY_HISTORY_KEY = 'c_relay_pg_sql_history';
|
||||
const MAX_HISTORY_ITEMS = 20;
|
||||
|
||||
// Load query history from localStorage
|
||||
|
||||
@@ -224,7 +224,7 @@ The script should:
|
||||
- `curl` or `websocat` for WebSocket communication
|
||||
- `jq` for JSON parsing
|
||||
- Nostr CLI tools (optional, for event signing)
|
||||
- Running c-relay instance
|
||||
- Running c-relay-pg instance
|
||||
|
||||
## Example Output
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# C-Relay Complete Startup Flow Documentation
|
||||
# C-Relay-PG Complete Startup Flow Documentation
|
||||
|
||||
## Overview
|
||||
|
||||
C-Relay has two distinct startup paths:
|
||||
C-Relay-PG has two distinct startup paths:
|
||||
1. **First-Time Startup**: No database exists, generates keys and initializes system
|
||||
2. **Existing Relay Startup**: Database exists, loads configuration and resumes operation
|
||||
|
||||
@@ -220,7 +220,7 @@ int populate_default_config_values(void) {
|
||||
|
||||
struct config_default defaults[] = {
|
||||
{"relay_port", "8888", "integer", "WebSocket port", "network", 1},
|
||||
{"relay_name", "C-Relay", "string", "Relay name", "info", 0},
|
||||
{"relay_name", "C-Relay-PG", "string", "Relay name", "info", 0},
|
||||
{"relay_description", "High-performance C Nostr relay", "string", "Description", "info", 0},
|
||||
{"max_subscriptions_per_client", "25", "integer", "Max subs per client", "limits", 0},
|
||||
{"pow_min_difficulty", "0", "integer", "Minimum PoW difficulty", "security", 0},
|
||||
@@ -983,7 +983,7 @@ if (current_version < LATEST_VERSION) {
|
||||
|
||||
**Solutions**:
|
||||
- Use `--port <number>` to specify different port
|
||||
- Kill existing process: `pkill -f c_relay_`
|
||||
- Kill existing process: `pkill -f c_relay_pg_`
|
||||
- Force kill port: `fuser -k 8888/tcp`
|
||||
- Use `--strict-port` to fail fast instead of trying fallback ports
|
||||
|
||||
@@ -994,7 +994,7 @@ if (current_version < LATEST_VERSION) {
|
||||
**Solutions**:
|
||||
- Kill existing relay processes
|
||||
- Remove WAL files: `rm build/*.db-wal build/*.db-shm`
|
||||
- Check for stale processes: `ps aux | grep c_relay_`
|
||||
- Check for stale processes: `ps aux | grep c_relay_pg_`
|
||||
|
||||
#### 3. Missing Admin Private Key
|
||||
|
||||
@@ -1071,7 +1071,7 @@ Typical startup times (on modern hardware):
|
||||
|
||||
## Summary
|
||||
|
||||
The c-relay startup system is designed for:
|
||||
The c-relay-pg startup system is designed for:
|
||||
|
||||
1. **Security**: Admin keys never stored, relay keys encrypted
|
||||
2. **Reliability**: Automatic port fallback, schema migrations
|
||||
|
||||
@@ -18,10 +18,10 @@ The script now attempts to build with `musl-gcc` for truly portable static binar
|
||||
|
||||
SQLite is now built once and cached for future builds:
|
||||
|
||||
- **Cache location**: `~/.cache/c-relay-sqlite/`
|
||||
- **Cache location**: `~/.cache/c-relay-pg-sqlite/`
|
||||
- **Version-specific**: Each SQLite version gets its own cache directory
|
||||
- **Significant speedup**: Subsequent builds skip the SQLite compilation step
|
||||
- **Manual cleanup**: `rm -rf ~/.cache/c-relay-sqlite` to clear cache
|
||||
- **Manual cleanup**: `rm -rf ~/.cache/c-relay-pg-sqlite` to clear cache
|
||||
|
||||
### 3. Smart Package Installation
|
||||
|
||||
@@ -51,13 +51,13 @@ The script will:
|
||||
## Binary Types
|
||||
|
||||
### MUSL Static Binary (Ideal - Currently Not Achievable)
|
||||
- **Filename**: `build/c_relay_static_musl_x86_64`
|
||||
- **Filename**: `build/c_relay_pg_static_musl_x86_64`
|
||||
- **Dependencies**: None (truly static)
|
||||
- **Portability**: Works on any Linux distribution
|
||||
- **Status**: Requires MUSL-compiled libwebsockets and other dependencies (not available by default)
|
||||
|
||||
### Glibc Static Binary (Current Output)
|
||||
- **Filename**: `build/c_relay_static_x86_64` or `build/c_relay_static_glibc_x86_64`
|
||||
- **Filename**: `build/c_relay_pg_static_x86_64` or `build/c_relay_pg_static_glibc_x86_64`
|
||||
- **Dependencies**: None - fully statically linked with glibc
|
||||
- **Portability**: Works on most Linux distributions (glibc is statically included)
|
||||
- **Note**: Despite using glibc, this is a **fully static binary** with no runtime dependencies
|
||||
@@ -68,11 +68,11 @@ The script automatically verifies binaries using `ldd` and `file`:
|
||||
|
||||
```bash
|
||||
# For MUSL binary
|
||||
ldd build/c_relay_static_musl_x86_64
|
||||
ldd build/c_relay_pg_static_musl_x86_64
|
||||
# Output: "not a dynamic executable" (good!)
|
||||
|
||||
# For glibc binary
|
||||
ldd build/c_relay_static_glibc_x86_64
|
||||
ldd build/c_relay_pg_static_glibc_x86_64
|
||||
# Output: Shows glibc dependencies
|
||||
```
|
||||
|
||||
@@ -112,7 +112,7 @@ The script attempts MUSL compilation but falls back to glibc:
|
||||
|
||||
### Clear SQLite Cache
|
||||
```bash
|
||||
rm -rf ~/.cache/c-relay-sqlite
|
||||
rm -rf ~/.cache/c-relay-pg-sqlite
|
||||
```
|
||||
|
||||
### Force Package Reinstall
|
||||
@@ -127,8 +127,8 @@ cat /tmp/musl_build.log
|
||||
|
||||
### Verify Binary Type
|
||||
```bash
|
||||
file build/c_relay_static_*
|
||||
ldd build/c_relay_static_* 2>&1
|
||||
file build/c_relay_pg_static_*
|
||||
ldd build/c_relay_pg_static_* 2>&1
|
||||
```
|
||||
|
||||
## Performance Impact
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## Problem Summary
|
||||
|
||||
The c-relay Nostr relay experienced severe performance degradation (90-100% CPU) due to subscription accumulation in the database. Investigation revealed **323,644 orphaned subscriptions** that were never properly closed when WebSocket connections dropped.
|
||||
The c-relay-pg Nostr relay experienced severe performance degradation (90-100% CPU) due to subscription accumulation in the database. Investigation revealed **323,644 orphaned subscriptions** that were never properly closed when WebSocket connections dropped.
|
||||
|
||||
## Solution: Two-Component Approach
|
||||
|
||||
|
||||
@@ -586,7 +586,7 @@ int add_pubkeys_to_config_table(void) {
|
||||
rm -f *.db
|
||||
|
||||
# Start relay with defaults
|
||||
./build/c_relay_x86
|
||||
./build/c_relay_pg_x86
|
||||
|
||||
# Verify config table complete
|
||||
sqlite3 <relay_pubkey>.db "SELECT COUNT(*) FROM config;"
|
||||
@@ -602,7 +602,7 @@ int add_pubkeys_to_config_table(void) {
|
||||
rm -f *.db
|
||||
|
||||
# Start relay with port override
|
||||
./build/c_relay_x86 --port 9999
|
||||
./build/c_relay_pg_x86 --port 9999
|
||||
|
||||
# Verify port override applied
|
||||
sqlite3 <relay_pubkey>.db "SELECT value FROM config WHERE key='relay_port';"
|
||||
@@ -612,13 +612,13 @@ int add_pubkeys_to_config_table(void) {
|
||||
3. **Restart with Existing Database**
|
||||
```bash
|
||||
# Start relay (creates database)
|
||||
./build/c_relay_x86
|
||||
./build/c_relay_pg_x86
|
||||
|
||||
# Stop relay
|
||||
pkill -f c_relay_
|
||||
pkill -f c_relay_pg_
|
||||
|
||||
# Restart relay
|
||||
./build/c_relay_x86
|
||||
./build/c_relay_pg_x86
|
||||
|
||||
# Verify config unchanged
|
||||
# Check relay.log for validation message
|
||||
@@ -627,13 +627,13 @@ int add_pubkeys_to_config_table(void) {
|
||||
4. **Restart with CLI Overrides**
|
||||
```bash
|
||||
# Start relay (creates database)
|
||||
./build/c_relay_x86
|
||||
./build/c_relay_pg_x86
|
||||
|
||||
# Stop relay
|
||||
pkill -f c_relay_
|
||||
pkill -f c_relay_pg_
|
||||
|
||||
# Restart with port override
|
||||
./build/c_relay_x86 --port 9999
|
||||
./build/c_relay_pg_x86 --port 9999
|
||||
|
||||
# Verify port override applied
|
||||
sqlite3 <relay_pubkey>.db "SELECT value FROM config WHERE key='relay_port';"
|
||||
|
||||
+32
-32
@@ -19,12 +19,12 @@ Complete guide for deploying, configuring, and managing the C Nostr Relay with e
|
||||
```bash
|
||||
# Clone and build
|
||||
git clone <repository-url>
|
||||
cd c-relay
|
||||
cd c-relay-pg
|
||||
git submodule update --init --recursive
|
||||
make
|
||||
|
||||
# Start relay (zero configuration needed)
|
||||
./build/c_relay_x86
|
||||
./build/c_relay_pg_x86
|
||||
```
|
||||
|
||||
### 2. First Startup - Save Keys
|
||||
@@ -78,7 +78,7 @@ brew install git sqlite libwebsockets openssl libsecp256k1 curl zlib
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone <repository-url>
|
||||
cd c-relay
|
||||
cd c-relay-pg
|
||||
|
||||
# Initialize submodules
|
||||
git submodule update --init --recursive
|
||||
@@ -87,7 +87,7 @@ git submodule update --init --recursive
|
||||
make clean && make
|
||||
|
||||
# Verify build
|
||||
ls -la build/c_relay_x86
|
||||
ls -la build/c_relay_pg_x86
|
||||
```
|
||||
|
||||
### Production Deployment
|
||||
@@ -98,27 +98,27 @@ ls -la build/c_relay_x86
|
||||
sudo systemd/install-service.sh
|
||||
|
||||
# Start service
|
||||
sudo systemctl start c-relay
|
||||
sudo systemctl start c-relay-pg
|
||||
|
||||
# Enable auto-start
|
||||
sudo systemctl enable c-relay
|
||||
sudo systemctl enable c-relay-pg
|
||||
|
||||
# Check status
|
||||
sudo systemctl status c-relay
|
||||
sudo systemctl status c-relay-pg
|
||||
```
|
||||
|
||||
#### Manual Deployment
|
||||
```bash
|
||||
# Create dedicated user
|
||||
sudo useradd --system --home-dir /opt/c-relay --shell /bin/false c-relay
|
||||
sudo useradd --system --home-dir /opt/c-relay-pg --shell /bin/false c-relay-pg
|
||||
|
||||
# Install binary
|
||||
sudo mkdir -p /opt/c-relay
|
||||
sudo cp build/c_relay_x86 /opt/c-relay/
|
||||
sudo chown -R c-relay:c-relay /opt/c-relay
|
||||
sudo mkdir -p /opt/c-relay-pg
|
||||
sudo cp build/c_relay_pg_x86 /opt/c-relay-pg/
|
||||
sudo chown -R c-relay-pg:c-relay-pg /opt/c-relay-pg
|
||||
|
||||
# Run as service user
|
||||
sudo -u c-relay /opt/c-relay/c_relay_x86
|
||||
sudo -u c-relay-pg /opt/c-relay-pg/c_relay_pg_x86
|
||||
```
|
||||
|
||||
## Configuration Management
|
||||
@@ -188,7 +188,7 @@ Send this to your relay via WebSocket, and changes are applied immediately.
|
||||
|-----------|-------------|---------|---------|
|
||||
| `relay_description` | Relay description for NIP-11 | "C Nostr Relay" | "My awesome relay" |
|
||||
| `relay_contact` | Admin contact information | "" | "admin@example.com" |
|
||||
| `relay_software` | Software identifier | "c-relay" | "c-relay v1.0" |
|
||||
| `relay_software` | Software identifier | "c-relay-pg" | "c-relay-pg v1.0" |
|
||||
|
||||
#### Client Limits
|
||||
| Parameter | Description | Default | Range |
|
||||
@@ -273,7 +273,7 @@ chmod 600 admin_keys_backup_*.txt
|
||||
#### Key Recovery
|
||||
If you lose your admin private key:
|
||||
|
||||
1. **Stop the relay**: `pkill c_relay` or `sudo systemctl stop c-relay`
|
||||
1. **Stop the relay**: `pkill c_relay_pg` or `sudo systemctl stop c-relay-pg`
|
||||
2. **Backup events**: `cp <relay_pubkey>.nrdb backup_$(date +%Y%m%d).nrdb`
|
||||
3. **Remove database**: `rm <relay_pubkey>.nrdb*`
|
||||
4. **Restart relay**: This creates new database with new keys
|
||||
@@ -300,7 +300,7 @@ sudo ufw allow 8888/tcp
|
||||
```bash
|
||||
# Secure database file permissions
|
||||
chmod 600 <relay_pubkey>.nrdb
|
||||
chown c-relay:c-relay <relay_pubkey>.nrdb
|
||||
chown c-relay-pg:c-relay-pg <relay_pubkey>.nrdb
|
||||
|
||||
# Regular backups
|
||||
cp <relay_pubkey>.nrdb backup/relay_backup_$(date +%Y%m%d_%H%M%S).nrdb
|
||||
@@ -311,10 +311,10 @@ cp <relay_pubkey>.nrdb backup/relay_backup_$(date +%Y%m%d_%H%M%S).nrdb
|
||||
### Service Status
|
||||
```bash
|
||||
# Check if relay is running
|
||||
ps aux | grep c_relay
|
||||
ps aux | grep c_relay_pg
|
||||
|
||||
# SystemD status
|
||||
sudo systemctl status c-relay
|
||||
sudo systemctl status c-relay-pg
|
||||
|
||||
# Network connections
|
||||
netstat -tln | grep 8888
|
||||
@@ -324,16 +324,16 @@ sudo ss -tlpn | grep 8888
|
||||
### Log Monitoring
|
||||
```bash
|
||||
# Real-time logs (systemd)
|
||||
sudo journalctl -u c-relay -f
|
||||
sudo journalctl -u c-relay-pg -f
|
||||
|
||||
# Recent logs
|
||||
sudo journalctl -u c-relay --since "1 hour ago"
|
||||
sudo journalctl -u c-relay-pg --since "1 hour ago"
|
||||
|
||||
# Error logs only
|
||||
sudo journalctl -u c-relay -p err
|
||||
sudo journalctl -u c-relay-pg -p err
|
||||
|
||||
# Configuration changes
|
||||
sudo journalctl -u c-relay | grep "Configuration updated via kind 33334"
|
||||
sudo journalctl -u c-relay-pg | grep "Configuration updated via kind 33334"
|
||||
```
|
||||
|
||||
### Database Analytics
|
||||
@@ -365,13 +365,13 @@ ORDER BY created_at DESC;
|
||||
du -sh <relay_pubkey>.nrdb*
|
||||
|
||||
# Memory usage
|
||||
ps aux | grep c_relay | awk '{print $6}' # RSS memory in KB
|
||||
ps aux | grep c_relay_pg | awk '{print $6}' # RSS memory in KB
|
||||
|
||||
# Connection count (approximate)
|
||||
netstat -an | grep :8888 | grep ESTABLISHED | wc -l
|
||||
|
||||
# System resources
|
||||
top -p $(pgrep c_relay)
|
||||
top -p $(pgrep c_relay_pg)
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
@@ -385,11 +385,11 @@ netstat -tln | grep 8888
|
||||
# If port in use, find process: sudo lsof -i :8888
|
||||
|
||||
# Check binary permissions
|
||||
ls -la build/c_relay_x86
|
||||
chmod +x build/c_relay_x86
|
||||
ls -la build/c_relay_pg_x86
|
||||
chmod +x build/c_relay_pg_x86
|
||||
|
||||
# Check dependencies
|
||||
ldd build/c_relay_x86
|
||||
ldd build/c_relay_pg_x86
|
||||
```
|
||||
|
||||
#### Configuration Not Updating
|
||||
@@ -442,7 +442,7 @@ sqlite3 recovered.nrdb < recovered.sql
|
||||
|
||||
# If repair fails, start fresh (loses all events)
|
||||
mv <relay_pubkey>.nrdb <relay_pubkey>.nrdb.corrupted
|
||||
./build/c_relay_x86 # Creates new database
|
||||
./build/c_relay_pg_x86 # Creates new database
|
||||
```
|
||||
|
||||
#### Lost Configuration Recovery
|
||||
@@ -455,12 +455,12 @@ If configuration is lost but database is intact:
|
||||
#### Emergency Restart
|
||||
```bash
|
||||
# Quick restart with clean state
|
||||
sudo systemctl stop c-relay
|
||||
sudo systemctl stop c-relay-pg
|
||||
mv <relay_pubkey>.nrdb <relay_pubkey>.nrdb.backup
|
||||
sudo systemctl start c-relay
|
||||
sudo systemctl start c-relay-pg
|
||||
|
||||
# Check logs for new admin keys
|
||||
sudo journalctl -u c-relay --since "5 minutes ago" | grep "Admin Private Key"
|
||||
sudo journalctl -u c-relay-pg --since "5 minutes ago" | grep "Admin Private Key"
|
||||
```
|
||||
|
||||
## Advanced Usage
|
||||
@@ -503,7 +503,7 @@ ws.on('open', function() {
|
||||
# backup-relay.sh
|
||||
DATE=$(date +%Y%m%d_%H%M%S)
|
||||
DB_FILE=$(ls *.nrdb | head -1)
|
||||
BACKUP_DIR="/backup/c-relay"
|
||||
BACKUP_DIR="/backup/c-relay-pg"
|
||||
|
||||
mkdir -p $BACKUP_DIR
|
||||
cp $DB_FILE $BACKUP_DIR/relay_backup_$DATE.nrdb
|
||||
@@ -533,7 +533,7 @@ tar czf relay_migration.tar.gz *.nrdb* relay.log
|
||||
|
||||
# Target server
|
||||
tar xzf relay_migration.tar.gz
|
||||
./build/c_relay_x86 # Will detect existing database and continue
|
||||
./build/c_relay_pg_x86 # Will detect existing database and continue
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Script to embed web files into C headers for the C-Relay admin interface
|
||||
# Script to embed web files into C headers for the C-Relay-PG admin interface
|
||||
# Converts HTML, CSS, and JS files from api/ directory into C byte arrays
|
||||
|
||||
set -e
|
||||
|
||||
@@ -60,7 +60,7 @@ All examples assume the event-based configuration system where:
|
||||
- **Save Admin Keys**: All deployment examples emphasize capturing the admin private key on first startup
|
||||
- **Firewall Configuration**: Examples include proper firewall rules
|
||||
- **SSL/TLS**: Production examples include HTTPS configuration
|
||||
- **User Isolation**: Service runs as dedicated `c-relay` system user
|
||||
- **User Isolation**: Service runs as dedicated `c-relay-pg` system user
|
||||
|
||||
## Support
|
||||
|
||||
|
||||
@@ -13,8 +13,8 @@ BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Default configuration
|
||||
RELAY_DIR="/opt/c-relay"
|
||||
BACKUP_DIR="/backup/c-relay"
|
||||
RELAY_DIR="/opt/c-relay-pg"
|
||||
BACKUP_DIR="/backup/c-relay-pg"
|
||||
RETENTION_DAYS="30"
|
||||
COMPRESS="true"
|
||||
REMOTE_BACKUP=""
|
||||
@@ -47,8 +47,8 @@ show_help() {
|
||||
echo "Usage: $0 [OPTIONS]"
|
||||
echo
|
||||
echo "Options:"
|
||||
echo " -d, --relay-dir DIR Relay directory (default: /opt/c-relay)"
|
||||
echo " -b, --backup-dir DIR Backup directory (default: /backup/c-relay)"
|
||||
echo " -d, --relay-dir DIR Relay directory (default: /opt/c-relay-pg)"
|
||||
echo " -b, --backup-dir DIR Backup directory (default: /backup/c-relay-pg)"
|
||||
echo " -r, --retention DAYS Retention period in days (default: 30)"
|
||||
echo " -n, --no-compress Don't compress backups"
|
||||
echo " -s, --s3-bucket BUCKET Upload to S3 bucket"
|
||||
@@ -231,7 +231,7 @@ upload_to_s3() {
|
||||
|
||||
print_step "Uploading backup to S3..."
|
||||
|
||||
local s3_path="s3://$S3_BUCKET/c-relay/$(date +%Y)/$(date +%m)/"
|
||||
local s3_path="s3://$S3_BUCKET/c-relay-pg/$(date +%Y)/$(date +%m)/"
|
||||
|
||||
if aws s3 cp "$BACKUP_FILE" "$s3_path" --storage-class STANDARD_IA; then
|
||||
print_success "Backup uploaded to S3: $s3_path"
|
||||
@@ -264,7 +264,7 @@ cleanup_old_backups() {
|
||||
print_step "Cleaning S3 backups older than $cutoff_date..."
|
||||
|
||||
# Note: This is a simplified approach. In production, use S3 lifecycle policies
|
||||
aws s3 ls "s3://$S3_BUCKET/c-relay/" --recursive | \
|
||||
aws s3 ls "s3://$S3_BUCKET/c-relay-pg/" --recursive | \
|
||||
awk '$1 < "'$cutoff_date'" {print $4}' | \
|
||||
while read -r key; do
|
||||
aws s3 rm "s3://$S3_BUCKET/$key"
|
||||
|
||||
@@ -13,8 +13,8 @@ BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Configuration
|
||||
RELAY_DIR="/opt/c-relay"
|
||||
SERVICE_NAME="c-relay"
|
||||
RELAY_DIR="/opt/c-relay-pg"
|
||||
SERVICE_NAME="c-relay-pg"
|
||||
RELAY_PORT="8888"
|
||||
LOG_FILE="/var/log/relay-monitor.log"
|
||||
ALERT_EMAIL=""
|
||||
@@ -60,7 +60,7 @@ show_help() {
|
||||
echo "Usage: $0 [OPTIONS]"
|
||||
echo
|
||||
echo "Options:"
|
||||
echo " -d, --relay-dir DIR Relay directory (default: /opt/c-relay)"
|
||||
echo " -d, --relay-dir DIR Relay directory (default: /opt/c-relay-pg)"
|
||||
echo " -p, --port PORT Relay port (default: 8888)"
|
||||
echo " -i, --interval SECONDS Check interval (default: 60)"
|
||||
echo " -e, --email EMAIL Alert email address"
|
||||
@@ -134,7 +134,7 @@ parse_args() {
|
||||
check_process_running() {
|
||||
print_step "Checking if relay process is running..."
|
||||
|
||||
if pgrep -f "c_relay_x86" > /dev/null; then
|
||||
if pgrep -f "c_relay_pg_x86" > /dev/null; then
|
||||
print_success "Relay process is running"
|
||||
return 0
|
||||
else
|
||||
@@ -172,7 +172,7 @@ check_service_status() {
|
||||
check_memory_usage() {
|
||||
print_step "Checking memory usage..."
|
||||
|
||||
local memory_kb=$(ps aux | grep "c_relay_x86" | grep -v grep | awk '{sum+=$6} END {print sum}')
|
||||
local memory_kb=$(ps aux | grep "c_relay_pg_x86" | grep -v grep | awk '{sum+=$6} END {print sum}')
|
||||
|
||||
if [[ -z "$memory_kb" ]]; then
|
||||
print_warning "Could not determine memory usage"
|
||||
|
||||
@@ -56,7 +56,7 @@ http {
|
||||
}
|
||||
|
||||
# Upstream for the relay
|
||||
upstream c_relay_backend {
|
||||
upstream c_relay_pg_backend {
|
||||
server 127.0.0.1:8888;
|
||||
keepalive 32;
|
||||
}
|
||||
@@ -108,7 +108,7 @@ http {
|
||||
# Main proxy location for WebSocket and HTTP
|
||||
location / {
|
||||
# Proxy settings
|
||||
proxy_pass http://c_relay_backend;
|
||||
proxy_pass http://c_relay_pg_backend;
|
||||
proxy_http_version 1.1;
|
||||
proxy_cache_bypass $http_upgrade;
|
||||
|
||||
@@ -144,7 +144,7 @@ http {
|
||||
|
||||
# Health check endpoint (if implemented)
|
||||
location /health {
|
||||
proxy_pass http://c_relay_backend/health;
|
||||
proxy_pass http://c_relay_pg_backend/health;
|
||||
access_log off;
|
||||
}
|
||||
|
||||
@@ -157,7 +157,7 @@ http {
|
||||
|
||||
# Optional: Metrics endpoint (if implemented)
|
||||
location /metrics {
|
||||
proxy_pass http://c_relay_backend/metrics;
|
||||
proxy_pass http://c_relay_pg_backend/metrics;
|
||||
# Restrict access to monitoring systems
|
||||
allow 10.0.0.0/8;
|
||||
allow 172.16.0.0/12;
|
||||
|
||||
@@ -96,9 +96,9 @@ check_root() {
|
||||
check_relay_running() {
|
||||
print_step "Checking if C Nostr Relay is running..."
|
||||
|
||||
if ! pgrep -f "c_relay_x86" > /dev/null; then
|
||||
if ! pgrep -f "c_relay_pg_x86" > /dev/null; then
|
||||
print_error "C Nostr Relay is not running"
|
||||
print_error "Please start the relay first with: sudo systemctl start c-relay"
|
||||
print_error "Please start the relay first with: sudo systemctl start c-relay-pg"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
@@ -13,9 +13,9 @@ BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Configuration
|
||||
RELAY_USER="c-relay"
|
||||
INSTALL_DIR="/opt/c-relay"
|
||||
SERVICE_NAME="c-relay"
|
||||
RELAY_USER="c-relay-pg"
|
||||
INSTALL_DIR="/opt/c-relay-pg"
|
||||
SERVICE_NAME="c-relay-pg"
|
||||
RELAY_PORT="8888"
|
||||
|
||||
# Functions
|
||||
@@ -99,7 +99,7 @@ build_relay() {
|
||||
|
||||
# Check if we're in the source directory
|
||||
if [[ ! -f "Makefile" ]]; then
|
||||
print_error "Makefile not found. Please run this script from the c-relay source directory."
|
||||
print_error "Makefile not found. Please run this script from the c-relay-pg source directory."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -107,7 +107,7 @@ build_relay() {
|
||||
make clean
|
||||
make
|
||||
|
||||
if [[ ! -f "build/c_relay_x86" ]]; then
|
||||
if [[ ! -f "build/c_relay_pg_x86" ]]; then
|
||||
print_error "Build failed - binary not found"
|
||||
exit 1
|
||||
fi
|
||||
@@ -118,9 +118,9 @@ build_relay() {
|
||||
install_binary() {
|
||||
print_step "Installing relay binary..."
|
||||
|
||||
cp build/c_relay_x86 "$INSTALL_DIR/"
|
||||
chown "$RELAY_USER:$RELAY_USER" "$INSTALL_DIR/c_relay_x86"
|
||||
chmod +x "$INSTALL_DIR/c_relay_x86"
|
||||
cp build/c_relay_pg_x86 "$INSTALL_DIR/"
|
||||
chown "$RELAY_USER:$RELAY_USER" "$INSTALL_DIR/c_relay_pg_x86"
|
||||
chmod +x "$INSTALL_DIR/c_relay_pg_x86"
|
||||
|
||||
print_success "Binary installed to $INSTALL_DIR"
|
||||
}
|
||||
@@ -129,14 +129,14 @@ install_service() {
|
||||
print_step "Installing systemd service..."
|
||||
|
||||
# Use the existing systemd service file
|
||||
if [[ -f "systemd/c-relay.service" ]]; then
|
||||
cp systemd/c-relay.service /etc/systemd/system/
|
||||
if [[ -f "systemd/c-relay-pg.service" ]]; then
|
||||
cp systemd/c-relay-pg.service /etc/systemd/system/
|
||||
systemctl daemon-reload
|
||||
print_success "Systemd service installed"
|
||||
else
|
||||
print_warning "Systemd service file not found, creating basic one..."
|
||||
|
||||
cat > /etc/systemd/system/c-relay.service << EOF
|
||||
cat > /etc/systemd/system/c-relay-pg.service << EOF
|
||||
[Unit]
|
||||
Description=C Nostr Relay
|
||||
After=network.target
|
||||
@@ -146,7 +146,7 @@ Type=simple
|
||||
User=$RELAY_USER
|
||||
Group=$RELAY_USER
|
||||
WorkingDirectory=$INSTALL_DIR
|
||||
ExecStart=$INSTALL_DIR/c_relay_x86
|
||||
ExecStart=$INSTALL_DIR/c_relay_pg_x86
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# MUSL-based fully static C-Relay builder
|
||||
# MUSL-based fully static C-Relay-PG builder
|
||||
# Produces portable binaries with zero runtime dependencies
|
||||
|
||||
FROM alpine:latest AS builder
|
||||
@@ -125,7 +125,7 @@ RUN cd /tmp && \
|
||||
--prefix=/usr && \
|
||||
make && make install
|
||||
|
||||
# Copy c-relay source
|
||||
# Copy c-relay-pg source
|
||||
COPY . /build/
|
||||
|
||||
# Initialize submodules
|
||||
@@ -134,7 +134,7 @@ RUN git submodule update --init --recursive
|
||||
# Build nostr_core_lib
|
||||
RUN cd nostr_core_lib && ./build.sh
|
||||
|
||||
# Build c-relay static
|
||||
# Build c-relay-pg static
|
||||
RUN make clean && \
|
||||
CC="musl-gcc -static" \
|
||||
CFLAGS="-O2 -Wall -Wextra -std=c99 -g" \
|
||||
@@ -143,8 +143,8 @@ RUN make clean && \
|
||||
make
|
||||
|
||||
# Strip binary for size
|
||||
RUN strip build/c_relay_x86
|
||||
RUN strip build/c_relay_pg_x86
|
||||
|
||||
# Multi-stage build to produce minimal output
|
||||
FROM scratch AS output
|
||||
COPY --from=builder /build/build/c_relay_x86 /c_relay_static_musl_x86_64
|
||||
COPY --from=builder /build/build/c_relay_pg_x86 /c_relay_pg_static_musl_x86_64
|
||||
@@ -19,7 +19,7 @@ RELEASE_MODE=false
|
||||
VERSION_INCREMENT_TYPE="patch" # "patch", "minor", or "major"
|
||||
|
||||
show_usage() {
|
||||
echo "C-Relay Increment and Push Script"
|
||||
echo "C-Relay-PG Increment and Push Script"
|
||||
echo ""
|
||||
echo "USAGE:"
|
||||
echo " $0 [OPTIONS] \"commit message\""
|
||||
@@ -330,7 +330,7 @@ build_release_binary() {
|
||||
create_source_tarball() {
|
||||
print_status "Creating source tarball..."
|
||||
|
||||
local tarball_name="c-relay-${NEW_VERSION#v}.tar.gz"
|
||||
local tarball_name="c-relay-pg-${NEW_VERSION#v}.tar.gz"
|
||||
|
||||
# Create tarball excluding build artifacts and git files
|
||||
if tar -czf "$tarball_name" \
|
||||
@@ -365,7 +365,7 @@ upload_release_assets() {
|
||||
fi
|
||||
|
||||
local token=$(cat "$HOME/.gitea_token" | tr -d '\n\r')
|
||||
local api_url="https://git.laantungir.net/api/v1/repos/laantungir/c-relay"
|
||||
local api_url="https://git.laantungir.net/api/v1/repos/laantungir/c-relay-pg"
|
||||
local assets_url="$api_url/releases/$release_id/assets"
|
||||
print_status "Assets URL: $assets_url"
|
||||
|
||||
@@ -427,7 +427,7 @@ create_gitea_release() {
|
||||
fi
|
||||
|
||||
local token=$(cat "$HOME/.gitea_token" | tr -d '\n\r')
|
||||
local api_url="https://git.laantungir.net/api/v1/repos/laantungir/c-relay"
|
||||
local api_url="https://git.laantungir.net/api/v1/repos/laantungir/c-relay-pg"
|
||||
|
||||
# Create release
|
||||
print_status "Creating release $NEW_VERSION..."
|
||||
@@ -473,7 +473,7 @@ create_gitea_release() {
|
||||
|
||||
# Main execution
|
||||
main() {
|
||||
print_status "C-Relay Increment and Push Script"
|
||||
print_status "C-Relay-PG Increment and Push Script"
|
||||
|
||||
# Check prerequisites
|
||||
check_git_repo
|
||||
@@ -505,13 +505,13 @@ main() {
|
||||
|
||||
# Build release binary
|
||||
if build_release_binary; then
|
||||
local binary_path="build/c_relay_static_x86_64"
|
||||
local binary_path="build/c_relay_pg_static_x86_64"
|
||||
else
|
||||
print_warning "Binary build failed, continuing with release creation"
|
||||
# Check if binary exists from previous build
|
||||
if [[ -f "build/c_relay_static_x86_64" ]]; then
|
||||
if [[ -f "build/c_relay_pg_static_x86_64" ]]; then
|
||||
print_status "Using existing binary from previous build"
|
||||
binary_path="build/c_relay_static_x86_64"
|
||||
binary_path="build/c_relay_pg_static_x86_64"
|
||||
else
|
||||
binary_path=""
|
||||
fi
|
||||
|
||||
+30
-12
@@ -1,6 +1,6 @@
|
||||
#!/bin/bash
|
||||
|
||||
# C-Relay Build and Restart Script
|
||||
# C-Relay-PG Build and Restart Script
|
||||
# Builds the project first, then stops any running relay and starts a new one in the background
|
||||
|
||||
echo "=== C Nostr Relay Build and Restart Script ==="
|
||||
@@ -70,6 +70,24 @@ while [[ $# -gt 0 ]]; do
|
||||
;;
|
||||
--test-keys|-t)
|
||||
USE_TEST_KEYS=true
|
||||
# Read keys from .test_keys file
|
||||
if [ -f ".test_keys" ]; then
|
||||
echo "Reading test keys from .test_keys file..."
|
||||
# Source the file to get the variables
|
||||
source .test_keys
|
||||
# Remove any single quotes from the values
|
||||
# Note: -a flag expects ADMIN_PUBKEY (public key), not ADMIN_PRIVKEY
|
||||
ADMIN_KEY=$(echo "$ADMIN_PUBKEY" | tr -d "'")
|
||||
RELAY_KEY=$(echo "$SERVER_PRIVKEY" | tr -d "'")
|
||||
echo "Using admin pubkey from .test_keys: ${ADMIN_KEY:0:16}..."
|
||||
echo "Using relay privkey from .test_keys: ${RELAY_KEY:0:16}..."
|
||||
else
|
||||
echo "ERROR: .test_keys file not found"
|
||||
echo "Please create a .test_keys file with the following format:"
|
||||
echo " ADMIN_PUBKEY='your_admin_public_key_hex'"
|
||||
echo " SERVER_PRIVKEY='your_relay_private_key_hex'"
|
||||
exit 1
|
||||
fi
|
||||
shift
|
||||
;;
|
||||
--debug-level=*)
|
||||
@@ -224,13 +242,13 @@ fi
|
||||
ARCH=$(uname -m)
|
||||
case "$ARCH" in
|
||||
x86_64)
|
||||
BINARY_PATH="./build/c_relay_static_x86_64"
|
||||
BINARY_PATH="./build/c_relay_pg_static_x86_64"
|
||||
;;
|
||||
aarch64|arm64)
|
||||
BINARY_PATH="./build/c_relay_static_arm64"
|
||||
BINARY_PATH="./build/c_relay_pg_static_arm64"
|
||||
;;
|
||||
*)
|
||||
BINARY_PATH="./build/c_relay_static_$ARCH"
|
||||
BINARY_PATH="./build/c_relay_pg_static_$ARCH"
|
||||
;;
|
||||
esac
|
||||
|
||||
@@ -252,7 +270,7 @@ echo "Build successful. Proceeding with relay restart..."
|
||||
echo "Stopping any existing relay servers..."
|
||||
|
||||
# Get all relay processes and kill them immediately with -9
|
||||
RELAY_PIDS=$(pgrep -f "c_relay_" || echo "")
|
||||
RELAY_PIDS=$(pgrep -f "c_relay_pg_" || echo "")
|
||||
if [ -n "$RELAY_PIDS" ]; then
|
||||
echo "Force killing relay processes immediately: $RELAY_PIDS"
|
||||
kill -9 $RELAY_PIDS 2>/dev/null
|
||||
@@ -273,7 +291,7 @@ for attempt in {1..15}; do
|
||||
fuser -k 8888/tcp 2>/dev/null || true
|
||||
|
||||
# Double-check for any remaining relay processes
|
||||
REMAINING_PIDS=$(pgrep -f "c_relay_" || echo "")
|
||||
REMAINING_PIDS=$(pgrep -f "c_relay_pg_" || echo "")
|
||||
if [ -n "$REMAINING_PIDS" ]; then
|
||||
echo "Killing remaining relay processes: $REMAINING_PIDS"
|
||||
kill -9 $REMAINING_PIDS 2>/dev/null || true
|
||||
@@ -291,7 +309,7 @@ for attempt in {1..15}; do
|
||||
done
|
||||
|
||||
# Final safety check - ensure no relay processes remain
|
||||
FINAL_PIDS=$(pgrep -f "c_relay_" || echo "")
|
||||
FINAL_PIDS=$(pgrep -f "c_relay_pg_" || echo "")
|
||||
if [ -n "$FINAL_PIDS" ]; then
|
||||
echo "Final cleanup: killing processes $FINAL_PIDS"
|
||||
kill -9 $FINAL_PIDS 2>/dev/null || true
|
||||
@@ -307,7 +325,7 @@ echo "Database will be initialized automatically on startup if needed"
|
||||
|
||||
# Start relay in background with output redirection
|
||||
echo "Starting relay server..."
|
||||
echo "Debug: Current processes: $(ps aux | grep 'c_relay_' | grep -v grep || echo 'None')"
|
||||
echo "Debug: Current processes: $(ps aux | grep 'c_relay_pg_' | grep -v grep || echo 'None')"
|
||||
|
||||
# Build command line arguments for relay binary
|
||||
RELAY_ARGS=""
|
||||
@@ -336,8 +354,8 @@ fi
|
||||
cd build
|
||||
# Start relay in background and capture its PID
|
||||
if [ "$USE_TEST_KEYS" = true ]; then
|
||||
echo "Using deterministic test keys for development..."
|
||||
./$(basename $BINARY_PATH) -a 6a04ab98d9e4774ad806e302dddeb63bea16b5cb5f223ee77478e861bb583eb3 -r 1111111111111111111111111111111111111111111111111111111111111111 --debug-level=$DEBUG_LEVEL --strict-port > ../relay.log 2>&1 &
|
||||
echo "Using test keys from .test_keys file..."
|
||||
./$(basename $BINARY_PATH) -a "$ADMIN_KEY" -r "$RELAY_KEY" --debug-level=$DEBUG_LEVEL --strict-port > ../relay.log 2>&1 &
|
||||
elif [ -n "$RELAY_ARGS" ]; then
|
||||
echo "Starting relay with custom configuration..."
|
||||
./$(basename $BINARY_PATH) $RELAY_ARGS --debug-level=$DEBUG_LEVEL --strict-port > ../relay.log 2>&1 &
|
||||
@@ -410,8 +428,8 @@ if ps -p "$RELAY_PID" >/dev/null 2>&1; then
|
||||
echo "=== Event-Based Relay Server Running ==="
|
||||
echo "Configuration: Event-based (kind 33334 Nostr events)"
|
||||
echo "Database: Automatically created with relay pubkey naming"
|
||||
echo "To kill relay: pkill -f 'c_relay_'"
|
||||
echo "To check status: ps aux | grep c_relay_"
|
||||
echo "To kill relay: pkill -f 'c_relay_pg_'"
|
||||
echo "To check status: ps aux | grep c_relay_pg_"
|
||||
echo "To view logs: tail -f relay.log"
|
||||
echo "Binary: $BINARY_PATH (zero configuration needed)"
|
||||
echo "Ready for Nostr client connections!"
|
||||
|
||||
Submodule
+1
Submodule nostr-rs-relay added at 64cfcaf44a
+1
-1
Submodule nostr_core_lib updated: a8dc2ed046...98cfd81b01
@@ -12,11 +12,11 @@ After the next crash, analyze it:
|
||||
# List all core dumps (most recent first)
|
||||
sudo coredumpctl list
|
||||
|
||||
# View info about the most recent c-relay crash
|
||||
sudo coredumpctl info c-relay
|
||||
# View info about the most recent c-relay-pg crash
|
||||
sudo coredumpctl info c-relay-pg
|
||||
|
||||
# Load the core dump in gdb for detailed analysis
|
||||
sudo coredumpctl gdb c-relay
|
||||
sudo coredumpctl gdb c-relay-pg
|
||||
|
||||
Inside gdb, run these commands:
|
||||
(gdb) bt full # Full backtrace with all variables
|
||||
@@ -36,8 +36,8 @@ DEBUGGING
|
||||
|
||||
Even simpler: Use this one-liner
|
||||
# Start relay and immediately attach gdb
|
||||
cd /usr/local/bin/c_relay
|
||||
sudo -u c-relay ./c_relay --debug-level=5 & sleep 2 && sudo gdb -p $(pgrep c_relay)
|
||||
cd /usr/local/bin/c_relay_pg
|
||||
sudo -u c-relay-pg ./c_relay_pg --debug-level=5 & sleep 2 && sudo gdb -p $(pgrep c_relay_pg)
|
||||
|
||||
Inside gdb, after attaching:
|
||||
|
||||
@@ -48,17 +48,17 @@ Or shorter:
|
||||
|
||||
How to View the Logs
|
||||
Check systemd journal:
|
||||
# View all c-relay logs
|
||||
sudo journalctl -u c-relay
|
||||
# View all c-relay-pg logs
|
||||
sudo journalctl -u c-relay-pg
|
||||
|
||||
# View recent logs (last 50 lines)
|
||||
sudo journalctl -u c-relay -n 50
|
||||
sudo journalctl -u c-relay-pg -n 50
|
||||
|
||||
# Follow logs in real-time
|
||||
sudo journalctl -u c-relay -f
|
||||
sudo journalctl -u c-relay-pg -f
|
||||
|
||||
# View logs since last boot
|
||||
sudo journalctl -u c-relay -b
|
||||
sudo journalctl -u c-relay-pg -b
|
||||
|
||||
Check if service is running:
|
||||
|
||||
@@ -78,9 +78,9 @@ sudo systemctl start rsyslog
|
||||
sudo systemctl status rsyslog
|
||||
|
||||
|
||||
sudo -u c-relay ./c_relay --debug-level=5 -r 85d0b37e2ae822966dcadd06b2dc9368cde73865f90ea4d44f8b57d47ef0820a -a 1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139
|
||||
sudo -u c-relay-pg ./c_relay_pg --debug-level=5 -r 85d0b37e2ae822966dcadd06b2dc9368cde73865f90ea4d44f8b57d47ef0820a -a 1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139
|
||||
|
||||
./c_relay_static_x86_64 -p 7889 --debug-level=5 -r 85d0b37e2ae822966dcadd06b2dc9368cde73865f90ea4d44f8b57d47ef0820a -a 1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139
|
||||
./c_relay_pg_static_x86_64 -p 7889 --debug-level=5 -r 85d0b37e2ae822966dcadd06b2dc9368cde73865f90ea4d44f8b57d47ef0820a -a 1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139
|
||||
|
||||
|
||||
sudo ufw allow 8888/tcp
|
||||
|
||||
Generated
+1
-1
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"name": "c-relay",
|
||||
"name": "c-relay-pg",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
|
||||
@@ -0,0 +1,632 @@
|
||||
# c-relay-pg-pg Plan — PostgreSQL Backend + Multi-Instance + Dashboard
|
||||
|
||||
## Overview
|
||||
|
||||
**c-relay-pg-pg** is a new project (separate repository) forked from c-relay-pg after the `db_ops` abstraction layer is complete. It replaces the SQLite backend with PostgreSQL, enabling horizontal scaling, external dashboards, and production-grade deployments.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- The `db_ops.h` / `db_ops.c` abstraction layer must be complete in c-relay-pg first — see [c-relay-pg thread pool plan](c_relay_pg_thread_pool_plan.md) Phase 1.
|
||||
- The fork happens after Phase 1 of that plan is done and tested.
|
||||
|
||||
### Why a Separate Project?
|
||||
|
||||
| | c-relay-pg | c-relay-pg-pg |
|
||||
|---|---------|-----------|
|
||||
| **Database** | SQLite (embedded) | PostgreSQL (client-server) |
|
||||
| **Deployment** | Single binary + DB file | Binary + PostgreSQL server |
|
||||
| **Scaling** | Single instance | Multiple instances + load balancer |
|
||||
| **Dashboard** | Embedded web UI | Separate web server + Grafana |
|
||||
| **Target user** | Personal relay, small community | Medium-large relay, production |
|
||||
| **Complexity** | Minimal | More infrastructure |
|
||||
|
||||
For the full analysis of why PostgreSQL was chosen over MySQL/MariaDB and other databases, see the [database architecture analysis](database_architecture_analysis.md).
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph c-relay-pg-pg - Production Deployment
|
||||
LB[nginx - TLS + load balancing] -->|WebSocket| R1[c-relay-pg-pg Instance 1]
|
||||
LB -->|WebSocket| R2[c-relay-pg-pg Instance 2]
|
||||
LB -->|HTTP| DASH[Dashboard]
|
||||
R1 -->|db_ops API| PGBACK[PostgreSQL Backend - db_ops_postgres.c]
|
||||
R2 -->|db_ops API| PGBACK
|
||||
PGBACK -->|libpq| PG[PostgreSQL Server]
|
||||
DASH -->|SQL| PG
|
||||
R1 <-->|LISTEN/NOTIFY| R2
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: PostgreSQL Backend
|
||||
|
||||
### Goal
|
||||
|
||||
Fork c-relay-pg into a new repository called **c-relay-pg-pg**. Replace the SQLite `db_ops` implementation with PostgreSQL using `libpq`. The `db_ops.h` interface stays identical — only the backend changes.
|
||||
|
||||
### New Files
|
||||
|
||||
- `src/db_ops_sqlite.c` — Renamed from `db_ops.c` (the Phase 1 SQLite implementation from c-relay-pg)
|
||||
- `src/db_ops_postgres.c` — New PostgreSQL implementation
|
||||
- `src/db_ops.c` — Thin dispatcher that calls the active backend
|
||||
|
||||
### PostgreSQL Schema
|
||||
|
||||
```sql
|
||||
-- PostgreSQL schema for c-relay-pg-pg
|
||||
-- No event_tags table needed — JSONB + GIN handles everything
|
||||
|
||||
CREATE TABLE events (
|
||||
id TEXT PRIMARY KEY,
|
||||
pubkey TEXT NOT NULL,
|
||||
created_at BIGINT NOT NULL,
|
||||
kind INTEGER NOT NULL,
|
||||
event_type TEXT NOT NULL CHECK (event_type IN ('regular', 'replaceable', 'ephemeral', 'addressable')),
|
||||
content TEXT NOT NULL,
|
||||
sig TEXT NOT NULL,
|
||||
tags JSONB NOT NULL DEFAULT '[]',
|
||||
event_json TEXT NOT NULL,
|
||||
first_seen BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
|
||||
);
|
||||
|
||||
-- Core indexes
|
||||
CREATE INDEX idx_events_pubkey ON events(pubkey);
|
||||
CREATE INDEX idx_events_kind ON events(kind);
|
||||
CREATE INDEX idx_events_created_at ON events(created_at DESC);
|
||||
CREATE INDEX idx_events_kind_created_at ON events(kind, created_at DESC);
|
||||
CREATE INDEX idx_events_pubkey_created_at ON events(pubkey, created_at DESC);
|
||||
CREATE INDEX idx_events_pubkey_kind ON events(pubkey, kind);
|
||||
|
||||
-- THE KEY INDEX: GIN on JSONB tags — replaces the entire event_tags table
|
||||
CREATE INDEX idx_events_tags ON events USING GIN (tags);
|
||||
|
||||
-- Partial index: only non-ephemeral events (what REQ queries actually filter)
|
||||
CREATE INDEX idx_events_non_ephemeral ON events(created_at DESC)
|
||||
WHERE kind < 20000 OR kind >= 30000;
|
||||
|
||||
-- Config table
|
||||
CREATE TABLE config (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL,
|
||||
data_type TEXT NOT NULL CHECK (data_type IN ('string', 'integer', 'boolean', 'json')),
|
||||
description TEXT,
|
||||
category TEXT DEFAULT 'general',
|
||||
requires_restart INTEGER DEFAULT 0,
|
||||
created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,
|
||||
updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
|
||||
);
|
||||
|
||||
-- Auth rules table
|
||||
CREATE TABLE auth_rules (
|
||||
id SERIAL PRIMARY KEY,
|
||||
rule_type TEXT NOT NULL CHECK (rule_type IN ('whitelist', 'blacklist', 'rate_limit', 'auth_required', 'wot_whitelist')),
|
||||
pattern_type TEXT NOT NULL CHECK (pattern_type IN ('pubkey', 'kind', 'ip', 'global')),
|
||||
pattern_value TEXT,
|
||||
active INTEGER NOT NULL DEFAULT 1,
|
||||
created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,
|
||||
updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
|
||||
);
|
||||
|
||||
CREATE INDEX idx_auth_rules_lookup ON auth_rules(rule_type, pattern_type, pattern_value) WHERE active = 1;
|
||||
|
||||
-- Relay private key storage
|
||||
CREATE TABLE relay_seckey (
|
||||
private_key_hex TEXT NOT NULL CHECK (length(private_key_hex) = 64),
|
||||
created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
|
||||
);
|
||||
|
||||
-- Subscription logging
|
||||
CREATE TABLE subscriptions (
|
||||
id SERIAL PRIMARY KEY,
|
||||
subscription_id TEXT NOT NULL,
|
||||
wsi_pointer TEXT NOT NULL,
|
||||
client_ip TEXT NOT NULL,
|
||||
event_type TEXT NOT NULL CHECK (event_type IN ('created', 'closed', 'expired', 'disconnected')),
|
||||
filter_json TEXT,
|
||||
events_sent INTEGER DEFAULT 0,
|
||||
created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,
|
||||
ended_at BIGINT,
|
||||
duration INTEGER,
|
||||
UNIQUE(subscription_id, wsi_pointer)
|
||||
);
|
||||
|
||||
-- IP ban persistence
|
||||
CREATE TABLE ip_bans (
|
||||
ip TEXT PRIMARY KEY,
|
||||
failure_count INTEGER NOT NULL DEFAULT 0,
|
||||
ban_count INTEGER NOT NULL DEFAULT 0,
|
||||
banned_until BIGINT NOT NULL DEFAULT 0,
|
||||
first_failure BIGINT NOT NULL DEFAULT 0,
|
||||
has_authed_successfully INTEGER NOT NULL DEFAULT 0,
|
||||
last_success_at BIGINT NOT NULL DEFAULT 0,
|
||||
total_connections INTEGER NOT NULL DEFAULT 0,
|
||||
total_failures INTEGER NOT NULL DEFAULT 0,
|
||||
total_successes INTEGER NOT NULL DEFAULT 0,
|
||||
first_seen BIGINT NOT NULL DEFAULT 0,
|
||||
idle_failure_count INTEGER NOT NULL DEFAULT 0,
|
||||
idle_ban_count INTEGER NOT NULL DEFAULT 0,
|
||||
idle_banned_until BIGINT NOT NULL DEFAULT 0,
|
||||
idle_first_failure BIGINT NOT NULL DEFAULT 0
|
||||
);
|
||||
|
||||
-- Materialized views for dashboard (refreshed periodically)
|
||||
CREATE MATERIALIZED VIEW event_kinds_mv AS
|
||||
SELECT kind, COUNT(*) as count,
|
||||
ROUND(COUNT(*) * 100.0 / NULLIF((SELECT COUNT(*) FROM events), 0), 2) as percentage
|
||||
FROM events GROUP BY kind;
|
||||
|
||||
CREATE MATERIALIZED VIEW time_stats_mv AS
|
||||
SELECT 'total' as period, COUNT(*) as total_events, COUNT(DISTINCT pubkey) as unique_pubkeys
|
||||
FROM events
|
||||
UNION ALL
|
||||
SELECT '24h', COUNT(*), COUNT(DISTINCT pubkey)
|
||||
FROM events WHERE created_at >= EXTRACT(EPOCH FROM NOW())::BIGINT - 86400
|
||||
UNION ALL
|
||||
SELECT '7d', COUNT(*), COUNT(DISTINCT pubkey)
|
||||
FROM events WHERE created_at >= EXTRACT(EPOCH FROM NOW())::BIGINT - 604800;
|
||||
|
||||
CREATE MATERIALIZED VIEW top_pubkeys_mv AS
|
||||
SELECT pubkey, COUNT(*) as event_count,
|
||||
ROUND(COUNT(*) * 100.0 / NULLIF((SELECT COUNT(*) FROM events), 0), 2) as percentage
|
||||
FROM events GROUP BY pubkey ORDER BY event_count DESC LIMIT 20;
|
||||
|
||||
-- Unique indexes required for CONCURRENTLY refresh
|
||||
CREATE UNIQUE INDEX idx_event_kinds_mv ON event_kinds_mv(kind);
|
||||
CREATE UNIQUE INDEX idx_time_stats_mv ON time_stats_mv(period);
|
||||
CREATE UNIQUE INDEX idx_top_pubkeys_mv ON top_pubkeys_mv(pubkey);
|
||||
```
|
||||
|
||||
### Key Implementation Details
|
||||
|
||||
#### Tag Queries with JSONB
|
||||
|
||||
The biggest win — replacing the `event_tags` subquery with JSONB containment:
|
||||
|
||||
```c
|
||||
// SQLite (current): subquery into event_tags table
|
||||
// AND id IN (SELECT event_id FROM event_tags WHERE tag_name = ? AND tag_value IN (?))
|
||||
|
||||
// PostgreSQL: JSONB containment operator with GIN index
|
||||
// AND tags @> '[["p", "pubkey_hex"]]'
|
||||
|
||||
// For multiple tag values:
|
||||
// AND (tags @> '[["p", "val1"]]' OR tags @> '[["p", "val2"]]')
|
||||
```
|
||||
|
||||
This eliminates the entire `event_tags` table (4 million rows, ~2 GB indexes in the current SQLite schema). The GIN index on JSONB handles everything in ~100–200 MB.
|
||||
|
||||
#### LISTEN/NOTIFY Integration
|
||||
|
||||
For cross-instance event broadcasting:
|
||||
|
||||
```c
|
||||
// In db_ops_postgres.c:
|
||||
int db_notify_new_event(const char* event_id) {
|
||||
char cmd[128];
|
||||
snprintf(cmd, sizeof(cmd), "NOTIFY new_event, '%s'", event_id);
|
||||
PGresult* res = PQexec(g_pg_conn, cmd);
|
||||
PQclear(res);
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Called from the lws event loop every iteration:
|
||||
int db_check_notifications(char* event_id_out, size_t max_len) {
|
||||
PQconsumeInput(g_pg_notify_conn);
|
||||
PGnotify* notify = PQnotifies(g_pg_notify_conn);
|
||||
if (notify) {
|
||||
strncpy(event_id_out, notify->extra, max_len);
|
||||
PQfreemem(notify);
|
||||
return 1; // Got a notification
|
||||
}
|
||||
return 0; // No notifications
|
||||
}
|
||||
```
|
||||
|
||||
#### Connection Management
|
||||
|
||||
```c
|
||||
// db_ops_postgres.c connection pool (simple version)
|
||||
#define DB_POOL_SIZE 4
|
||||
|
||||
static PGconn* g_pg_read_pool[DB_POOL_SIZE]; // For REQ queries
|
||||
static PGconn* g_pg_write_conn; // For EVENT inserts
|
||||
static PGconn* g_pg_notify_conn; // For LISTEN/NOTIFY
|
||||
static pthread_mutex_t g_pool_lock;
|
||||
|
||||
PGconn* db_get_read_connection(void) {
|
||||
pthread_mutex_lock(&g_pool_lock);
|
||||
// Round-robin or find idle connection
|
||||
// ...
|
||||
pthread_mutex_unlock(&g_pool_lock);
|
||||
}
|
||||
```
|
||||
|
||||
### Build System Changes
|
||||
|
||||
```makefile
|
||||
# Makefile additions
|
||||
DB_BACKEND ?= sqlite # Default to sqlite, override with: make DB_BACKEND=postgres
|
||||
|
||||
ifeq ($(DB_BACKEND),postgres)
|
||||
MAIN_SRC += src/db_ops.c src/db_ops_postgres.c
|
||||
LIBS += -lpq
|
||||
CFLAGS += -DDB_BACKEND_POSTGRES
|
||||
else
|
||||
MAIN_SRC += src/db_ops.c src/db_ops_sqlite.c
|
||||
CFLAGS += -DDB_BACKEND_SQLITE
|
||||
endif
|
||||
```
|
||||
|
||||
### Data Migration Tool
|
||||
|
||||
A standalone tool to migrate existing SQLite data to PostgreSQL:
|
||||
|
||||
```bash
|
||||
# migrate_to_postgres.sh
|
||||
# 1. Export events from SQLite
|
||||
sqlite3 old_relay.db ".mode csv" "SELECT id,pubkey,created_at,kind,event_type,content,sig,tags,event_json,first_seen FROM events" > events.csv
|
||||
|
||||
# 2. Import into PostgreSQL
|
||||
psql -d crelay -c "\COPY events FROM 'events.csv' WITH CSV"
|
||||
|
||||
# 3. Migrate config
|
||||
sqlite3 old_relay.db ".mode csv" "SELECT key,value,data_type,description,category,requires_restart FROM config" > config.csv
|
||||
psql -d crelay -c "\COPY config(key,value,data_type,description,category,requires_restart) FROM 'config.csv' WITH CSV"
|
||||
|
||||
# 4. Migrate auth rules
|
||||
sqlite3 old_relay.db ".mode csv" "SELECT rule_type,pattern_type,pattern_value,active FROM auth_rules" > auth_rules.csv
|
||||
psql -d crelay -c "\COPY auth_rules(rule_type,pattern_type,pattern_value,active) FROM 'auth_rules.csv' WITH CSV"
|
||||
|
||||
# 5. Refresh materialized views
|
||||
psql -d crelay -c "REFRESH MATERIALIZED VIEW event_kinds_mv; REFRESH MATERIALIZED VIEW time_stats_mv; REFRESH MATERIALIZED VIEW top_pubkeys_mv;"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Multi-Instance + Dashboard
|
||||
|
||||
### Goal
|
||||
|
||||
Run multiple c-relay-pg-pg instances behind a load balancer with a separate dashboard web server, all sharing the same PostgreSQL database.
|
||||
|
||||
### Components
|
||||
|
||||
1. **nginx config** — WebSocket load balancing with `ip_hash` stickiness
|
||||
2. **systemd template** — `c-relay-pg-pg@.service` for multiple instances
|
||||
3. **LISTEN/NOTIFY integration** — Cross-instance event broadcasting in the `lws_service()` loop
|
||||
4. **PgBouncer** — Connection pooling between c-relay-pg-pg instances and PostgreSQL
|
||||
5. **Dashboard** — Separate web server querying PostgreSQL directly
|
||||
6. **Materialized view refresh** — Background job to refresh analytics views
|
||||
|
||||
### Multi-Instance Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
INTERNET[Internet - Nostr Clients] -->|wss://relay.example.com| NGINX[nginx reverse proxy - TLS termination + load balancing]
|
||||
|
||||
NGINX -->|ws://localhost:8888| R1[c-relay-pg-pg Instance 1 - port 8888]
|
||||
NGINX -->|ws://localhost:8889| R2[c-relay-pg-pg Instance 2 - port 8889]
|
||||
NGINX -->|ws://localhost:8890| R3[c-relay-pg-pg Instance 3 - port 8890]
|
||||
NGINX -->|http://localhost:3000| DASHWEB[Dashboard Web Server]
|
||||
|
||||
R1 -->|libpq connection pool| PGPOOL[PgBouncer - connection pooler]
|
||||
R2 -->|libpq connection pool| PGPOOL
|
||||
R3 -->|libpq connection pool| PGPOOL
|
||||
|
||||
PGPOOL -->|pooled connections| PG[PostgreSQL Primary]
|
||||
DASHWEB -->|read queries| PG
|
||||
|
||||
PG -->|streaming replication| REPLICA[Read Replica - optional]
|
||||
DASHWEB -.->|heavy analytics| REPLICA
|
||||
```
|
||||
|
||||
### nginx Load Balancing Config
|
||||
|
||||
```nginx
|
||||
# nginx.conf - WebSocket load balancing for c-relay-pg-pg
|
||||
upstream relay_backends {
|
||||
# ip_hash ensures a client always hits the same instance
|
||||
# (important for WebSocket session stickiness)
|
||||
ip_hash;
|
||||
|
||||
server 127.0.0.1:8888;
|
||||
server 127.0.0.1:8889;
|
||||
server 127.0.0.1:8890;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name relay.example.com;
|
||||
|
||||
# TLS config...
|
||||
|
||||
location / {
|
||||
proxy_pass http://relay_backends;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_read_timeout 86400; # Keep WebSocket alive for 24h
|
||||
}
|
||||
|
||||
# Dashboard on separate path
|
||||
location /dashboard {
|
||||
proxy_pass http://127.0.0.1:3000;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Session stickiness** (`ip_hash`) ensures that once a client connects to Instance 2, all their subsequent WebSocket frames go to Instance 2. This is important because subscriptions are held in-memory per instance.
|
||||
|
||||
### Starting Multiple Instances
|
||||
|
||||
Each instance is the same binary, just on a different port, all pointing to the same PostgreSQL:
|
||||
|
||||
```bash
|
||||
# Instance 1
|
||||
./build/c_relay_pg_x86 --port 8888 --db-host localhost --db-name crelay &
|
||||
|
||||
# Instance 2
|
||||
./build/c_relay_pg_x86 --port 8889 --db-host localhost --db-name crelay &
|
||||
|
||||
# Instance 3
|
||||
./build/c_relay_pg_x86 --port 8890 --db-host localhost --db-name crelay &
|
||||
```
|
||||
|
||||
Or with systemd template units:
|
||||
|
||||
```ini
|
||||
# /etc/systemd/system/c-relay-pg-pg@.service
|
||||
[Unit]
|
||||
Description=C-Relay-PG-PG Nostr Instance %i
|
||||
After=postgresql.service
|
||||
|
||||
[Service]
|
||||
ExecStart=/opt/c-relay-pg-pg/c_relay_pg_x86 --port %i --db-host localhost --db-name crelay
|
||||
Restart=always
|
||||
User=c-relay-pg
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
```bash
|
||||
systemctl enable c-relay-pg-pg@8888 c-relay-pg-pg@8889 c-relay-pg-pg@8890
|
||||
systemctl start c-relay-pg-pg@8888 c-relay-pg-pg@8889 c-relay-pg-pg@8890
|
||||
```
|
||||
|
||||
### Cross-Instance Event Broadcasting
|
||||
|
||||
When Instance 1 receives a new EVENT and stores it in PostgreSQL, Instance 2 and Instance 3 need to know about it so they can broadcast to their connected subscribers.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Client_A as Client A - connected to Instance 1
|
||||
participant I1 as Instance 1
|
||||
participant PG as PostgreSQL
|
||||
participant I2 as Instance 2
|
||||
participant I3 as Instance 3
|
||||
participant Client_B as Client B - connected to Instance 2
|
||||
participant Client_C as Client C - connected to Instance 3
|
||||
|
||||
Client_A->>I1: EVENT - new kind:1 note
|
||||
I1->>PG: INSERT INTO events...
|
||||
PG-->>I1: OK
|
||||
I1->>I1: broadcast to local subscribers
|
||||
I1->>PG: NOTIFY new_event with event_id
|
||||
|
||||
Note over PG: PostgreSQL delivers notification to all listeners
|
||||
|
||||
PG-->>I2: NOTIFY: new_event event_id
|
||||
PG-->>I3: NOTIFY: new_event event_id
|
||||
|
||||
I2->>PG: SELECT event_json FROM events WHERE id = event_id
|
||||
PG-->>I2: event JSON
|
||||
I2->>I2: match against local subscriptions
|
||||
I2->>Client_B: EVENT message - if subscription matches
|
||||
|
||||
I3->>PG: SELECT event_json FROM events WHERE id = event_id
|
||||
PG-->>I3: event JSON
|
||||
I3->>I3: match against local subscriptions
|
||||
I3->>Client_C: EVENT message - if subscription matches
|
||||
```
|
||||
|
||||
#### PostgreSQL LISTEN/NOTIFY Implementation
|
||||
|
||||
Built into PostgreSQL — no additional infrastructure needed:
|
||||
|
||||
```c
|
||||
// === In the relay's event loop (modified lws_service loop) ===
|
||||
|
||||
// Setup: create a dedicated connection for LISTEN
|
||||
PGconn* notify_conn = PQconnectdb("host=localhost dbname=crelay");
|
||||
PQexec(notify_conn, "LISTEN new_event");
|
||||
int notify_fd = PQsocket(notify_conn); // Get the socket fd for poll()
|
||||
|
||||
// After storing an event:
|
||||
void on_event_stored(PGconn* write_conn, const char* event_id) {
|
||||
char notify_cmd[128];
|
||||
snprintf(notify_cmd, sizeof(notify_cmd),
|
||||
"NOTIFY new_event, '%s'", event_id);
|
||||
PQexec(write_conn, notify_cmd);
|
||||
}
|
||||
|
||||
// In the main event loop (runs every lws_service iteration):
|
||||
void check_cross_instance_events(PGconn* notify_conn) {
|
||||
// Non-blocking check for notifications
|
||||
PQconsumeInput(notify_conn);
|
||||
|
||||
PGnotify* notify;
|
||||
while ((notify = PQnotifies(notify_conn)) != NULL) {
|
||||
// Another instance stored a new event
|
||||
const char* event_id = notify->extra;
|
||||
|
||||
// Fetch the event and check against local subscriptions
|
||||
cJSON* event = db_get_event_by_id(event_id);
|
||||
if (event) {
|
||||
broadcast_event_to_subscriptions(event);
|
||||
cJSON_Delete(event);
|
||||
}
|
||||
|
||||
PQfreemem(notify);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Performance**: LISTEN/NOTIFY adds ~1–5ms latency for cross-instance delivery. For a Nostr relay, this is imperceptible — clients already expect network latency.
|
||||
|
||||
**Payload limit**: NOTIFY payloads are limited to 8000 bytes. For event IDs (64 hex chars), this is fine.
|
||||
|
||||
#### Alternative: Redis Pub/Sub
|
||||
|
||||
If you later need even lower latency or more sophisticated routing:
|
||||
|
||||
```c
|
||||
// Using hiredis (Redis C client)
|
||||
redisContext* redis = redisConnect("127.0.0.1", 6379);
|
||||
|
||||
// After storing event:
|
||||
redisCommand(redis, "PUBLISH new_event %s", event_json);
|
||||
|
||||
// Subscriber (in each instance):
|
||||
redisCommand(redis, "SUBSCRIBE new_event");
|
||||
// Then poll for messages in the event loop
|
||||
```
|
||||
|
||||
Redis adds sub-millisecond pub/sub but requires running a Redis server. For most relays, PostgreSQL LISTEN/NOTIFY is sufficient.
|
||||
|
||||
### Connection Pooling with PgBouncer
|
||||
|
||||
Each relay instance needs multiple database connections. Without pooling, 3 instances × 10 connections = 30 PostgreSQL backend processes. With PgBouncer:
|
||||
|
||||
```ini
|
||||
# /etc/pgbouncer/pgbouncer.ini
|
||||
[databases]
|
||||
crelay = host=127.0.0.1 port=5432 dbname=crelay
|
||||
|
||||
[pgbouncer]
|
||||
listen_port = 6432
|
||||
listen_addr = 127.0.0.1
|
||||
auth_type = md5
|
||||
pool_mode = transaction # Return connection to pool after each transaction
|
||||
max_client_conn = 200 # Total connections from all relay instances
|
||||
default_pool_size = 20 # Actual PostgreSQL connections
|
||||
```
|
||||
|
||||
Relay instances connect to PgBouncer (port 6432) instead of PostgreSQL directly (port 5432). PgBouncer multiplexes 200 client connections onto 20 actual PostgreSQL connections.
|
||||
|
||||
### Zero-Downtime Deployment
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant LB as nginx
|
||||
participant I1 as Instance 1 - v1.0
|
||||
participant I2 as Instance 2 - v1.0
|
||||
participant I3 as Instance 3 - v1.0
|
||||
participant I1_NEW as Instance 1 - v1.1
|
||||
|
||||
Note over LB,I3: Normal operation: 3 instances serving traffic
|
||||
|
||||
LB->>I1: Mark upstream as down
|
||||
Note over I1: Drain: wait for existing connections to close or timeout
|
||||
I1->>I1: Graceful shutdown
|
||||
|
||||
Note over LB: Traffic now goes to I2 and I3 only
|
||||
|
||||
I1_NEW->>I1_NEW: Start with new binary
|
||||
I1_NEW->>LB: Health check passes
|
||||
LB->>I1_NEW: Mark upstream as up
|
||||
|
||||
Note over LB,I1_NEW: Instance 1 now running v1.1, repeat for I2 and I3
|
||||
```
|
||||
|
||||
Rolling deploy script:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# rolling_deploy.sh - Zero-downtime deployment
|
||||
|
||||
for port in 8888 8889 8890; do
|
||||
echo "Deploying instance on port $port..."
|
||||
|
||||
# 1. Tell nginx to stop sending new connections
|
||||
sed -i "s/server 127.0.0.1:$port;/server 127.0.0.1:$port down;/" /etc/nginx/nginx.conf
|
||||
nginx -s reload
|
||||
|
||||
# 2. Wait for existing connections to drain (30 seconds)
|
||||
sleep 30
|
||||
|
||||
# 3. Stop old instance
|
||||
systemctl stop c-relay-pg-pg@$port
|
||||
|
||||
# 4. Deploy new binary
|
||||
cp ./build/c_relay_pg_x86 /opt/c-relay-pg-pg/c_relay_pg_x86
|
||||
|
||||
# 5. Start new instance
|
||||
systemctl start c-relay-pg-pg@$port
|
||||
|
||||
# 6. Wait for health check
|
||||
sleep 5
|
||||
|
||||
# 7. Re-enable in nginx
|
||||
sed -i "s/server 127.0.0.1:$port down;/server 127.0.0.1:$port;/" /etc/nginx/nginx.conf
|
||||
nginx -s reload
|
||||
|
||||
echo "Instance on port $port deployed successfully"
|
||||
done
|
||||
```
|
||||
|
||||
### Dashboard Options
|
||||
|
||||
With PostgreSQL, the dashboard can be built with any technology:
|
||||
|
||||
- **Grafana** — Point directly at PostgreSQL, zero custom code
|
||||
- **Custom web app** — React/Vue frontend + any backend (Node, Python, Go) querying PostgreSQL
|
||||
- **Embedded in c-relay-pg-pg** — Keep current approach but queries go through `db_ops` to PostgreSQL (no longer blocks event loop since PostgreSQL handles concurrency)
|
||||
|
||||
### Scaling Scenarios
|
||||
|
||||
| Scenario | Instances | Connections | Events/hour | Setup |
|
||||
|----------|-----------|-------------|-------------|-------|
|
||||
| **Current** | 1 | ~1,200 | 56 | Single process + SQLite |
|
||||
| **Small upgrade** | 2 | ~2,500 | 500 | 2 instances + PostgreSQL |
|
||||
| **Medium relay** | 4 | ~5,000 | 5,000 | 4 instances + PostgreSQL + PgBouncer |
|
||||
| **Large relay** | 8 | ~10,000 | 50,000 | 8 instances + PostgreSQL + read replica |
|
||||
| **Multi-region** | 2–4 per region | ~50,000 | 500,000 | Multiple servers + PostgreSQL replication |
|
||||
|
||||
### Cost Perspective
|
||||
|
||||
Running multiple relay instances with PostgreSQL on a single VPS:
|
||||
- **PostgreSQL**: ~200–500 MB RAM baseline
|
||||
- **PgBouncer**: ~10 MB RAM
|
||||
- **Each relay instance**: ~50–100 MB RAM
|
||||
- **Dashboard web server**: ~50–100 MB RAM
|
||||
- **4 instances + PostgreSQL + PgBouncer + dashboard**: ~1–2 GB total RAM
|
||||
- A **$20/month VPS** with 4 cores and 4 GB RAM handles this easily
|
||||
- A **$40/month VPS** with 8 cores and 8 GB RAM handles 8 instances comfortably
|
||||
|
||||
---
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|-----------|
|
||||
| PostgreSQL connection failures | `db_ops` returns error codes. Relay logs errors but doesn't crash. Reconnection logic with exponential backoff. |
|
||||
| Performance regression | Benchmark before/after. PostgreSQL should be faster for complex queries, similar for simple ones. |
|
||||
| Data loss during migration | Migration tool is read-only on SQLite. PostgreSQL import is idempotent (INSERT ON CONFLICT). |
|
||||
| SQLite fallback needed | Keep `db_ops_sqlite.c` working. Compile-time flag switches backends. Can always go back. |
|
||||
| LISTEN/NOTIFY message loss | NOTIFY is transactional — if the INSERT commits, the NOTIFY is guaranteed. Missed notifications during reconnect handled by periodic full-sync. |
|
||||
| PgBouncer adds latency | Transaction pooling mode adds <0.1ms. Negligible compared to query time. |
|
||||
|
||||
---
|
||||
|
||||
## Relationship to c-relay-pg
|
||||
|
||||
This project depends on the `db_ops.h` abstraction layer built in [c-relay-pg thread pool plan](c_relay_pg_thread_pool_plan.md) Phase 1. The interface is identical — only the backend implementation changes:
|
||||
|
||||
- **c-relay-pg**: `db_ops.c` → SQLite calls via `sqlite3_*`
|
||||
- **c-relay-pg-pg**: `db_ops_postgres.c` → PostgreSQL calls via `libpq` (`PQexec`, `PQgetvalue`, etc.)
|
||||
|
||||
The `db_ops.h` header file is shared between both projects. Changes to the interface should be coordinated.
|
||||
@@ -0,0 +1,332 @@
|
||||
# c-relay-pg Thread Pool Plan — db_ops Abstraction + SQLite Thread Pool
|
||||
|
||||
## Overview
|
||||
|
||||
This plan covers improvements to **c-relay-pg** (this repo) — the lean, single-binary, embedded-SQLite Nostr relay. Two phases:
|
||||
|
||||
1. **Phase 1: Database Abstraction Layer** — Consolidate all scattered `sqlite3_*` calls into a single `db_ops.h` / `db_ops.c` module. Pure refactor, no behavior change.
|
||||
2. **Phase 2: SQLite Thread Pool** — Add worker threads for concurrent reads, unblocking the `lws_service()` event loop.
|
||||
|
||||
This is the **final form of c-relay-pg**: an embedded-SQLite relay with clean architecture and non-blocking database access. The abstraction layer also enables a future fork to PostgreSQL — see [c-relay-pg-pg plan](c_relay_pg_pg_plan.md).
|
||||
|
||||
### Why This First?
|
||||
|
||||
The current architecture has a fundamental bottleneck documented in the [database architecture analysis](database_architecture_analysis.md): the single-threaded event loop blocks on every database call. A 672ms REQ query freezes every connected client. The thread pool fixes this without changing the database engine or deployment model.
|
||||
|
||||
| Metric | Current | After Thread Pool |
|
||||
|--------|---------|-------------------|
|
||||
| **Event loop blocking** | 0.1–672ms per query | Near-zero |
|
||||
| **Concurrent reads** | 1 | 4–8 |
|
||||
| **Deployment** | Single binary + DB file | Same |
|
||||
| **Database** | SQLite | Same |
|
||||
| **Code risk** | N/A | Low — additive change |
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph c-relay-pg - Final Form
|
||||
RELAY[c-relay-pg binary] -->|db_ops API| DBOPS[db_ops.c - abstraction layer]
|
||||
DBOPS -->|sqlite3 calls| SQLITE[SQLite WAL database]
|
||||
end
|
||||
```
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph c-relay-pg with Thread Pool
|
||||
LWS[lws_service event loop] -->|REQ arrives| Q[Thread-safe job queue]
|
||||
LWS -->|EVENT arrives| WQ[Write queue - single writer]
|
||||
Q --> T1[Reader Thread 1 - own sqlite3*]
|
||||
Q --> T2[Reader Thread 2 - own sqlite3*]
|
||||
Q --> T3[Reader Thread 3 - own sqlite3*]
|
||||
Q --> T4[Reader Thread 4 - own sqlite3*]
|
||||
WQ --> TW[Writer Thread - own sqlite3*]
|
||||
T1 -->|results| CB[Callback to lws event loop]
|
||||
T2 -->|results| CB
|
||||
T3 -->|results| CB
|
||||
T4 -->|results| CB
|
||||
TW -->|OK/error| CB
|
||||
CB -->|queue_message| LWS
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Database Abstraction Layer
|
||||
|
||||
### Goal
|
||||
|
||||
Consolidate all 258 scattered `sqlite3_*` calls across 8 files into a single `db_ops.h` / `db_ops.c` module with a backend-agnostic interface. SQLite continues to work — this is a pure refactor.
|
||||
|
||||
### Files That Currently Touch SQLite Directly
|
||||
|
||||
| File | `sqlite3_*` calls | Operations |
|
||||
|------|-------------------|------------|
|
||||
| [`src/main.c`](../src/main.c) | ~80 | Event store/retrieve, tag storage, REQ queries, COUNT queries, DB init, schema migration |
|
||||
| [`src/config.c`](../src/config.c) | ~60 | Config CRUD, auth rules CRUD, WoT sync, relay key storage, startup sequence |
|
||||
| [`src/api.c`](../src/api.c) | ~40 | Stats queries, monitoring views, admin SQL execution, config management |
|
||||
| [`src/dm_admin.c`](../src/dm_admin.c) | ~20 | Auth rule management, WoT whitelist operations |
|
||||
| [`src/websockets.c`](../src/websockets.c) | ~15 | COUNT queries, IP ban stats, connection tracking |
|
||||
| [`src/subscriptions.c`](../src/subscriptions.c) | ~15 | Subscription logging — create/close/disconnect |
|
||||
| [`src/nip009.c`](../src/nip009.c) | ~10 | Event deletion — by ID and by address |
|
||||
| [`src/request_validator.c`](../src/request_validator.c) | ~8 | Blacklist/whitelist checks |
|
||||
| [`src/ip_ban.c`](../src/ip_ban.c) | ~15 | IP ban table CRUD, persistence |
|
||||
|
||||
### Abstraction Layer Design
|
||||
|
||||
New files: `src/db_ops.h` and `src/db_ops.c`
|
||||
|
||||
```c
|
||||
// src/db_ops.h — Database operations abstraction layer
|
||||
#ifndef DB_OPS_H
|
||||
#define DB_OPS_H
|
||||
|
||||
#include "../nostr_core_lib/cjson/cJSON.h"
|
||||
|
||||
// ============================================================
|
||||
// Lifecycle
|
||||
// ============================================================
|
||||
int db_init(const char* connection_string); // SQLite: file path, PG: connection string
|
||||
void db_close(void);
|
||||
int db_is_available(void);
|
||||
|
||||
// ============================================================
|
||||
// Schema / Migration
|
||||
// ============================================================
|
||||
int db_ensure_schema(void);
|
||||
int db_get_schema_version(void);
|
||||
int db_execute_raw(const char* sql); // For schema DDL only
|
||||
|
||||
// ============================================================
|
||||
// Event Operations
|
||||
// ============================================================
|
||||
int db_store_event(cJSON* event);
|
||||
int db_event_exists(const char* event_id);
|
||||
char* db_get_event_json(const char* event_id); // Caller must free
|
||||
int db_delete_event_by_id(const char* event_id, const char* requester_pubkey);
|
||||
int db_delete_events_by_address(const char* pubkey, int kind,
|
||||
const char* d_tag, long before_timestamp);
|
||||
|
||||
// ============================================================
|
||||
// Event Queries - REQ handling
|
||||
// ============================================================
|
||||
typedef struct {
|
||||
int* kinds; int kind_count;
|
||||
char** authors; int author_count;
|
||||
char** ids; int id_count;
|
||||
char** tag_names; // Parallel arrays: tag_names[i] has tag_values[i][]
|
||||
char*** tag_values;
|
||||
int* tag_value_counts;
|
||||
int tag_filter_count;
|
||||
long since; // 0 = not set
|
||||
long until; // 0 = not set
|
||||
int limit; // 0 = default 500
|
||||
char* search; // NIP-50 search term, NULL = not set
|
||||
} db_event_filter_t;
|
||||
|
||||
typedef struct db_result db_result_t;
|
||||
|
||||
db_result_t* db_query_events(const db_event_filter_t* filter);
|
||||
const char* db_result_next_json(db_result_t* result); // Returns event_json, NULL when done
|
||||
int db_result_row_count(db_result_t* result);
|
||||
void db_result_free(db_result_t* result);
|
||||
|
||||
int db_count_events(const db_event_filter_t* filter);
|
||||
|
||||
// ============================================================
|
||||
// Config Operations
|
||||
// ============================================================
|
||||
char* db_get_config(const char* key); // Caller must free, NULL if not found
|
||||
int db_set_config(const char* key, const char* value, const char* data_type,
|
||||
const char* description, const char* category, int requires_restart);
|
||||
int db_update_config(const char* key, const char* value);
|
||||
int db_config_exists(const char* key);
|
||||
int db_get_config_count(void);
|
||||
|
||||
// ============================================================
|
||||
// Auth Rules Operations
|
||||
// ============================================================
|
||||
int db_add_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
|
||||
int db_remove_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
|
||||
int db_auth_rule_exists(const char* rule_type, const char* pattern_type, const char* pattern_value);
|
||||
int db_clear_auth_rules(const char* rule_type); // NULL = clear all
|
||||
int db_is_blacklisted(const char* pubkey);
|
||||
int db_is_whitelisted(const char* pubkey);
|
||||
int db_has_any_whitelist(void);
|
||||
|
||||
// ============================================================
|
||||
// Relay Key Storage
|
||||
// ============================================================
|
||||
int db_store_relay_private_key(const char* privkey_hex);
|
||||
char* db_get_relay_private_key(void); // Caller must free
|
||||
|
||||
// ============================================================
|
||||
// Subscription Logging
|
||||
// ============================================================
|
||||
int db_log_subscription_created(const char* sub_id, const char* wsi_ptr,
|
||||
const char* client_ip, const char* filter_json);
|
||||
int db_log_subscription_closed(const char* sub_id, const char* client_ip);
|
||||
int db_log_subscription_disconnected(const char* client_ip);
|
||||
int db_update_subscription_events_sent(const char* sub_id, int events_sent);
|
||||
int db_cleanup_orphaned_subscriptions(void);
|
||||
|
||||
// ============================================================
|
||||
// IP Ban Persistence
|
||||
// ============================================================
|
||||
int db_ensure_ip_ban_table(void);
|
||||
int db_load_ip_bans(void* ban_table, int table_size); // Populates in-memory table
|
||||
int db_save_ip_bans(const void* ban_table, int table_size);
|
||||
|
||||
// ============================================================
|
||||
// Analytics / Stats - for dashboard
|
||||
// ============================================================
|
||||
cJSON* db_get_event_kind_distribution(void);
|
||||
cJSON* db_get_time_based_stats(void);
|
||||
cJSON* db_get_top_pubkeys(int limit);
|
||||
cJSON* db_get_subscription_details(void);
|
||||
int db_get_total_event_count(void);
|
||||
|
||||
// ============================================================
|
||||
// Admin SQL Query
|
||||
// ============================================================
|
||||
cJSON* db_execute_admin_query(const char* sql, char* error_msg, size_t error_size);
|
||||
|
||||
#endif // DB_OPS_H
|
||||
```
|
||||
|
||||
### Migration Strategy for Phase 1
|
||||
|
||||
The key principle: **change the interface, not the behavior**. Each function in `db_ops.c` initially just wraps the existing SQLite calls.
|
||||
|
||||
**Step-by-step for each file:**
|
||||
|
||||
1. **Create `db_ops.h` and `db_ops.c`** with the interface above
|
||||
2. **Implement each `db_ops` function** by moving the existing SQLite code from the source files into `db_ops.c`
|
||||
3. **Replace direct `sqlite3_*` calls** in each source file with `db_ops_*` calls
|
||||
4. **Remove `extern sqlite3* g_db`** from each file — only `db_ops.c` knows about `g_db`
|
||||
5. **Remove `#include <sqlite3.h>`** from each file — only `db_ops.c` includes it
|
||||
6. **Update Makefile** to compile `db_ops.c`
|
||||
|
||||
### Order of Migration (by risk, lowest first)
|
||||
|
||||
1. **`src/ip_ban.c`** — Self-contained, simple CRUD. Good warmup.
|
||||
2. **`src/subscriptions.c`** — Logging only, no critical path.
|
||||
3. **`src/nip009.c`** — Event deletion, small file.
|
||||
4. **`src/request_validator.c`** — Auth checks, small file.
|
||||
5. **`src/api.c`** — Stats/monitoring queries. Larger but read-only.
|
||||
6. **`src/dm_admin.c`** — Auth rules + WoT. Medium complexity.
|
||||
7. **`src/config.c`** — Config CRUD. Large but well-structured.
|
||||
8. **`src/websockets.c`** — COUNT queries, minimal DB usage.
|
||||
9. **`src/main.c`** — Event store/retrieve, REQ queries. The big one — do last.
|
||||
|
||||
### Testing Phase 1
|
||||
|
||||
After Phase 1, the relay should behave **identically** to before. Run:
|
||||
- `tests/run_all_tests.sh` — Full test suite
|
||||
- `tests/performance_benchmarks.sh` — Verify no performance regression
|
||||
- Manual testing with production-like traffic
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: SQLite Thread Pool
|
||||
|
||||
### Goal
|
||||
|
||||
Add a pool of N worker threads, each with its own `sqlite3*` connection to the same database file. SQLite WAL mode (already enabled) supports **concurrent readers**. The event loop dispatches database work to the pool and remains responsive.
|
||||
|
||||
### Key Design Points
|
||||
|
||||
1. **Read path**: REQ queries dispatched to thread pool. Each worker opens its own `sqlite3*` connection. SQLite WAL allows unlimited concurrent readers.
|
||||
|
||||
2. **Write path**: EVENT inserts go through a single dedicated writer thread. SQLite only allows one writer at a time anyway — this serializes writes cleanly.
|
||||
|
||||
3. **Result delivery**: Worker threads cannot call `lws_write()` directly (libwebsockets is not thread-safe). Instead, they push results into a per-session message queue and call `lws_cancel_service()` to wake the event loop, which then drains the queue.
|
||||
|
||||
4. **Connection lifecycle**: Each thread opens its own connection with `PRAGMA journal_mode=WAL` and `PRAGMA busy_timeout=5000`.
|
||||
|
||||
### What Changes in the Codebase
|
||||
|
||||
| Component | Current | Thread Pool |
|
||||
|-----------|---------|-------------|
|
||||
| [`g_db`](../src/main.c:49) | Single global connection | One per thread + writer connection |
|
||||
| [`handle_req_message()`](../src/main.c:1101) | Synchronous SQL in callback | Package filter into job, dispatch to pool |
|
||||
| [`store_event()`](../src/main.c:787) | Synchronous INSERT in callback | Dispatch to writer thread |
|
||||
| [`handle_count_message()`](../src/websockets.c:2863) | Synchronous COUNT in callback | Dispatch to pool |
|
||||
| Result delivery | Direct `queue_message()` | Worker pushes to queue + `lws_cancel_service()` |
|
||||
| Config reads | Direct `sqlite3_prepare` on `g_db` | Can stay synchronous with own connection or cache |
|
||||
|
||||
### New Files
|
||||
|
||||
- `src/thread_pool.h` — Thread pool interface
|
||||
- `src/thread_pool.c` — Thread pool implementation (job queue, worker lifecycle, result delivery)
|
||||
|
||||
### Thread Pool Interface (sketch)
|
||||
|
||||
```c
|
||||
// src/thread_pool.h
|
||||
#ifndef THREAD_POOL_H
|
||||
#define THREAD_POOL_H
|
||||
|
||||
typedef enum {
|
||||
JOB_TYPE_REQ_QUERY,
|
||||
JOB_TYPE_COUNT_QUERY,
|
||||
JOB_TYPE_STORE_EVENT,
|
||||
JOB_TYPE_DELETE_EVENT,
|
||||
} job_type_t;
|
||||
|
||||
typedef struct {
|
||||
job_type_t type;
|
||||
void* session; // lws per-session data pointer
|
||||
db_event_filter_t filter; // For REQ/COUNT jobs
|
||||
cJSON* event; // For STORE jobs
|
||||
char event_id[65]; // For DELETE jobs
|
||||
} db_job_t;
|
||||
|
||||
int thread_pool_init(int num_readers, const char* db_path);
|
||||
void thread_pool_shutdown(void);
|
||||
int thread_pool_submit_read(db_job_t* job);
|
||||
int thread_pool_submit_write(db_job_t* job);
|
||||
|
||||
#endif // THREAD_POOL_H
|
||||
```
|
||||
|
||||
### Performance Characteristics
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| **Concurrent reads** | N readers in parallel (N = thread count, typically 4–8) |
|
||||
| **Write throughput** | Same as current — SQLite serializes writes regardless |
|
||||
| **Event loop latency** | Near-zero — REQ no longer blocks the loop |
|
||||
| **Max theoretical read throughput** | ~4–8x current (limited by disk I/O, not CPU) |
|
||||
| **Memory overhead** | ~50–100 MB per connection (page cache) |
|
||||
| **Latency per query** | Same as current per-query, but no head-of-line blocking |
|
||||
|
||||
### Limitations
|
||||
|
||||
- **Write contention**: SQLite still allows only ONE writer at a time. With WAL, readers don't block writers and writers don't block readers, but two simultaneous writes will serialize. At the current write rate (~56 events/hour), this is a non-issue.
|
||||
- **Database size**: The 2.7 GB index bloat problem remains. Thread pool doesn't fix the schema — it fixes the concurrency.
|
||||
- **Scaling ceiling**: Beyond ~8 reader threads, diminishing returns due to disk I/O contention on a single SQLite file.
|
||||
- **Complexity**: Need a proper job queue, thread lifecycle management, and careful handling of the lws ↔ worker thread boundary.
|
||||
|
||||
---
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|-----------|
|
||||
| Phase 1 introduces bugs | Each file migrated independently, tested after each. Full test suite runs after each file. |
|
||||
| Thread pool race conditions | Worker threads only touch their own `sqlite3*` connection. Result delivery uses a mutex-protected queue. `lws_cancel_service()` is documented as thread-safe. |
|
||||
| Performance regression from abstraction | Abstraction layer is thin wrappers — no extra allocations or copies. Benchmark before/after. |
|
||||
| lws thread safety issues | Workers never call `lws_write()` directly. They push to a queue and wake the event loop. This is the documented pattern for libwebsockets multi-threading. |
|
||||
| Rollback needed | Phase 1 is a pure refactor — easy to revert. Phase 2 thread pool is additive — can be disabled with a compile flag. |
|
||||
|
||||
---
|
||||
|
||||
## Relationship to c-relay-pg-pg
|
||||
|
||||
After Phase 1 (abstraction layer) is complete, the codebase is ready to be forked into a separate **c-relay-pg-pg** project that replaces the SQLite backend with PostgreSQL. See [c-relay-pg-pg plan](c_relay_pg_pg_plan.md) for details.
|
||||
|
||||
The `db_ops.h` interface is designed to work with any backend:
|
||||
- **SQLite** (this plan): `db_result_next_json()` copies from `sqlite3_column_text()`
|
||||
- **PostgreSQL** (c-relay-pg-pg): `db_result_next_json()` returns from `PQgetvalue()`
|
||||
- **LMDB** (future possibility): `db_result_next_json()` returns a zero-copy pointer into mmap'd memory
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,186 @@
|
||||
# Plan: Eliminate g_db from the Main Thread
|
||||
|
||||
## Problem Statement
|
||||
|
||||
All SQLite calls go through `db_ops.c`, but `db_active_connection()` falls back to `g_db` when `g_thread_db` is NULL. Since the main thread never sets `g_thread_db`, every `db_*` call from lws-main executes synchronous SQLite on the main thread. The thread pool workers have their own connections and work correctly — the problem is the **fallback path**.
|
||||
|
||||
Current perf data shows lws-main at **67.6% avg CPU**, dominated by SQLite symbols (`sqlite3BtreeTableMoveto`, `sqlite3VdbeExec`, `pcache1Fetch`).
|
||||
|
||||
## Goal
|
||||
|
||||
Remove `g_db` as a runtime connection used by the main thread. After this change:
|
||||
- The main thread has **no SQLite connection** and cannot execute synchronous queries
|
||||
- All DB work routes through thread pool workers (which have their own connections)
|
||||
- `g_db` is only used during startup/shutdown (before/after the event loop)
|
||||
- Any accidental `db_*` call from lws-main returns an error instead of silently blocking
|
||||
|
||||
## Architecture After Change
|
||||
|
||||
```
|
||||
lws-main thread:
|
||||
- WebSocket protocol handling
|
||||
- Message parsing/routing
|
||||
- Completion queue draining
|
||||
- NO SQLite access during event loop
|
||||
|
||||
db-read-0 thread:
|
||||
- REQ queries
|
||||
- COUNT queries
|
||||
- Config cache miss reads
|
||||
- Auth rule checks
|
||||
- Monitoring/stats queries
|
||||
|
||||
db-write thread:
|
||||
- EVENT inserts
|
||||
- Subscription logging
|
||||
- Config updates
|
||||
- Auth rule modifications
|
||||
|
||||
event-worker thread:
|
||||
- Signature validation
|
||||
- Duplicate check
|
||||
- Store via db-write
|
||||
```
|
||||
|
||||
## Categorized g_db Call Sites
|
||||
|
||||
### Category A: Startup-only — keep using g_db
|
||||
|
||||
These run before the event loop starts. They are fine.
|
||||
|
||||
| Function | File | Purpose |
|
||||
|----------|------|---------|
|
||||
| `db_init()` | db_ops.c:27 | Open database |
|
||||
| `db_exec_sql()` for PRAGMAs | main.c:854-882 | WAL, mmap, cache setup |
|
||||
| `db_table_exists()` | main.c:750 | Schema check |
|
||||
| `db_get_schema_version_dup()` | main.c:754 | Migration check |
|
||||
| `db_exec_sql()` for schema | main.c:842 | Create tables |
|
||||
| `populate_all_config_values_atomic()` | config.c:4290 | First-time config |
|
||||
| `populate_default_config_values()` | config.c:1657 | Default config |
|
||||
| `add_pubkeys_to_config_table()` | config.c:1778 | Pubkey storage |
|
||||
| `apply_cli_overrides_atomic()` | config.c:4211 | CLI overrides |
|
||||
| `db_store_relay_private_key_hex()` | config.c:495 | Key storage |
|
||||
| `db_populate_event_tags_from_existing()` | main.c:1156 | Tag migration |
|
||||
| `cleanup_all_subscriptions_on_startup()` | subscriptions.c:1092 | Orphan cleanup |
|
||||
|
||||
### Category B: Shutdown-only — keep using g_db
|
||||
|
||||
| Function | File | Purpose |
|
||||
|----------|------|---------|
|
||||
| `db_exec_sql()` WAL checkpoint | main.c:900 | Clean shutdown |
|
||||
| `db_close()` | main.c:904 | Close connection |
|
||||
|
||||
### Category C: Runtime hot path — must move off main thread
|
||||
|
||||
These are called during the event loop from lws-main context.
|
||||
|
||||
| Function | File | Called from | Fix strategy |
|
||||
|----------|------|------------|--------------|
|
||||
| `db_get_config_value_dup()` | db_ops.c:777 | config cache miss | Already cached with 5s TTL; pre-warm cache at startup so misses are rare |
|
||||
| `store_event()` for kind 14/1059 | main.c:983 | websockets.c sync path | Route through async event worker |
|
||||
| `store_event_post_actions()` | main.c | completion handler | Already runs on main; its DB calls need routing |
|
||||
| `db_log_subscription_created()` | db_ops.c:145 | subscriptions.c | Fire-and-forget via write queue |
|
||||
| `db_log_subscription_closed()` | db_ops.c:165 | subscriptions.c | Fire-and-forget via write queue |
|
||||
| `db_log_subscription_disconnected()` | db_ops.c:193 | subscriptions.c | Fire-and-forget via write queue |
|
||||
| `db_update_subscription_events_sent()` | db_ops.c:225 | subscriptions.c | Fire-and-forget via write queue |
|
||||
| `db_cleanup_orphaned_subscriptions()` | db_ops.c:244 | subscriptions.c | Move to startup only |
|
||||
| `generate_and_post_status_event()` | websockets.c:3408 | periodic timer | Submit to write worker |
|
||||
| `ip_ban_cleanup()` / `ip_ban_log_stats()` | websockets.c:3196 | periodic timer | Submit to write worker |
|
||||
| `db_get_total_event_count_ll()` | db_ops.c:589 | api.c monitoring | Submit to read worker |
|
||||
| `db_get_event_count_since()` | db_ops.c:606 | api.c monitoring | Submit to read worker |
|
||||
| `db_get_event_kind_distribution_rows()` | db_ops.c:625 | api.c monitoring | Submit to read worker |
|
||||
| `db_get_top_pubkeys_rows()` | db_ops.c:663 | api.c monitoring | Submit to read worker |
|
||||
| `db_get_subscription_details_rows()` | db_ops.c:697 | api.c monitoring | Submit to read worker |
|
||||
| `db_get_all_config_rows()` | db_ops.c:745 | api.c config query | Submit to read worker |
|
||||
| `db_execute_readonly_query_json()` | db_ops.c:442 | api.c SQL query | Submit to read worker |
|
||||
| `db_event_id_exists()` | db_ops.c:1041 | main.c event check | Already moved to event worker |
|
||||
| `db_retrieve_event_by_id()` | db_ops.c:1056 | main.c | Submit to read worker |
|
||||
| `db_get_event_pubkey()` | db_ops.c:262 | NIP-09 deletion | Submit to read worker |
|
||||
| `db_delete_event_by_id()` | db_ops.c:285 | NIP-09 deletion | Submit to write worker |
|
||||
| `db_delete_older_replaceable_events()` | db_ops.c:305 | store_event_core | Already on write worker |
|
||||
| `db_store_config_event()` | db_ops.c:888 | config.c | Submit to write worker |
|
||||
| `db_add_auth_rule()` | db_ops.c:1226 | config.c admin | Submit to write worker |
|
||||
| `db_remove_auth_rule()` | db_ops.c:1242 | config.c admin | Submit to write worker |
|
||||
| `db_delete_wot_whitelist_rules()` | db_ops.c:1258 | config.c WoT | Submit to write worker |
|
||||
| `db_count_wot_whitelist_rules()` | db_ops.c:1266 | config.c | Cache or read worker |
|
||||
| `db_store_event_tags_cjson()` | db_ops.c:1140 | main.c post-actions | Already on write worker path |
|
||||
| `db_get_config_row_count()` | db_ops.c:1121 | config.c diagnostics | Cache or read worker |
|
||||
| `db_set_config_value_full()` | db_ops.c:796 | config.c | Submit to write worker |
|
||||
| `db_update_config_value_only()` | db_ops.c:821 | config.c | Submit to write worker |
|
||||
| `db_upsert_config_value()` | db_ops.c:838 | api.c | Submit to write worker |
|
||||
| `db_exec_sql()` for transactions | config.c | admin events | Submit to write worker |
|
||||
| `db_count_with_sql()` | db_ops.c:415 | config.c admin | Submit to read worker |
|
||||
| `is_config_table_ready()` | config.c:4595 | config.c hybrid | Cache result at startup |
|
||||
| `generate_config_event_from_table()` | config.c:4933 | config.c | Cache or read worker |
|
||||
|
||||
### Category D: Auth rule checks — already use separate connections
|
||||
|
||||
These functions in `db_ops.c` open their own temporary read-only connections:
|
||||
|
||||
| Function | Line | Notes |
|
||||
|----------|------|-------|
|
||||
| `db_is_pubkey_blacklisted()` | 347 | Opens own connection |
|
||||
| `db_is_hash_blacklisted()` | 362 | Opens own connection |
|
||||
| `db_is_pubkey_whitelisted()` | 377 | Opens own connection |
|
||||
| `db_count_active_whitelist_rules()` | 392 | Opens own connection |
|
||||
|
||||
These are already safe — they don't use `g_db`. No change needed.
|
||||
|
||||
## Implementation Strategy
|
||||
|
||||
### Phase 1: Pre-warm caches and eliminate cache-miss DB reads
|
||||
|
||||
1. Pre-warm the config cache at startup by loading all config values into the in-memory cache before the event loop starts
|
||||
2. Pre-warm the NIP-11 cache at startup
|
||||
3. Pre-warm the auth rules fast cache at startup
|
||||
4. This eliminates the most frequent cache-miss `db_get_config_value_dup()` calls
|
||||
|
||||
### Phase 2: Route subscription logging through write worker
|
||||
|
||||
1. Add a `THREAD_POOL_JOB_FIRE_AND_FORGET` job type to thread_pool
|
||||
2. Create async wrappers: `db_log_subscription_created_async()`, etc.
|
||||
3. These submit SQL to the write worker queue and return immediately
|
||||
4. No completion callback needed — fire and forget
|
||||
|
||||
### Phase 3: Route special-kind EVENT store through async worker
|
||||
|
||||
1. Remove the `event_is_async_eligible()` exclusion for kinds 14, 1059, 23456
|
||||
2. For kind 23456: process admin command in completion handler on main thread after store
|
||||
3. For kind 14/1059: process NIP-17 DM in completion handler after store
|
||||
|
||||
### Phase 4: Route periodic timer DB work through workers
|
||||
|
||||
1. `generate_and_post_status_event()` — submit event creation to write worker
|
||||
2. `ip_ban_cleanup()` / `ip_ban_log_stats()` — submit to write worker
|
||||
3. Monitoring queries — submit to read worker with completion callback
|
||||
|
||||
### Phase 5: Null out g_db before event loop
|
||||
|
||||
1. After startup is complete and thread pool is initialized, set `g_db = NULL`
|
||||
2. Change `db_active_connection()` to assert/warn if both `g_thread_db` and `g_db` are NULL
|
||||
3. Any accidental main-thread DB call will now fail loudly instead of silently blocking
|
||||
4. Restore `g_db` briefly for shutdown checkpoint
|
||||
|
||||
## Implementation Order
|
||||
|
||||
1. Phase 1 — lowest risk, immediate benefit from eliminating cache misses
|
||||
2. Phase 2 — straightforward fire-and-forget pattern
|
||||
3. Phase 3 — moderate complexity, needs careful completion handler design
|
||||
4. Phase 4 — moderate complexity, periodic tasks need async patterns
|
||||
5. Phase 5 — the final enforcement step, only safe after phases 1-4
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- **Phase 1**: Very low risk — just pre-warming existing caches
|
||||
- **Phase 2**: Low risk — subscription logging is non-critical, fire-and-forget is safe
|
||||
- **Phase 3**: Medium risk — admin/DM event processing has complex state; needs careful testing
|
||||
- **Phase 4**: Medium risk — periodic tasks have side effects that need to complete
|
||||
- **Phase 5**: High risk if done prematurely — must verify ALL runtime paths are covered first
|
||||
|
||||
## Expected Impact
|
||||
|
||||
After all phases:
|
||||
- lws-main CPU should drop from ~67% to near 0% SQLite overhead
|
||||
- All SQLite work happens on db-read and db-write threads
|
||||
- Main thread only does WebSocket I/O, JSON parsing, and completion queue draining
|
||||
- Thread model becomes: lws-main = pure I/O, workers = all DB
|
||||
@@ -0,0 +1,312 @@
|
||||
# Event Tags Denormalization Plan
|
||||
|
||||
## Problem Statement
|
||||
|
||||
The relay is at 99% CPU with 1,178 WebSocket connections and ~120 new REQ subscriptions per minute. The root cause is that every tag-filtered REQ query (e.g., `#g`, `#e`, `#p`) uses `json_each(json(tags))` with `json_extract()` — a correlated subquery that parses the JSON tags column for every candidate row. With geohash-based subscriptions from a location app generating constant connection churn, this creates unsustainable CPU load.
|
||||
|
||||
### Evidence
|
||||
|
||||
- `EXPLAIN QUERY PLAN` shows: `CORRELATED SCALAR SUBQUERY → SCAN json_each VIRTUAL TABLE`
|
||||
- 8,077 subscription creates/hour but only 442 active at any time (connection churn)
|
||||
- Only 56 events stored/hour — the load is entirely from **read queries**, not writes
|
||||
- 14,819 events match the kind filter; each query parses JSON tags on candidate rows
|
||||
|
||||
## Solution: Denormalized `event_tags` Table
|
||||
|
||||
Replace `json_each()` queries with indexed lookups on a separate `event_tags` table. This is the standard approach used by strfry, nostream, and other production Nostr relays.
|
||||
|
||||
### Performance Impact
|
||||
|
||||
| Metric | Before | After |
|
||||
|--------|--------|-------|
|
||||
| Tag query type | `json_each()` correlated subquery | Indexed JOIN/subquery |
|
||||
| Complexity per query | O(rows × tags_per_event) JSON parsing | O(log n) B-tree lookup |
|
||||
| Expected CPU reduction | 99% | <10% for same traffic |
|
||||
|
||||
## Files to Modify
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `src/sql_schema.h` | Add `event_tags` table, indexes, cascade triggers; bump to schema v12 |
|
||||
| `src/main.c` | Add `store_event_tags()`, update `handle_req_message()` tag filter SQL, move config reads out of row loop, add startup tag population |
|
||||
| `src/nip009.c` | Add `DELETE FROM event_tags` alongside event deletions |
|
||||
| `src/dm_admin.c` | Downgrade NIP-17 decryption failure log level |
|
||||
| `src/websockets.c` | Downgrade NIP-17 error log to DEBUG at lines 780 and 1488 |
|
||||
|
||||
## Detailed Changes
|
||||
|
||||
### 1. Schema: `src/sql_schema.h`
|
||||
|
||||
Add the `event_tags` table after the events table definition. Bump schema version to 12.
|
||||
|
||||
```sql
|
||||
-- Denormalized event tags for fast indexed lookups
|
||||
-- Replaces json_each(json(tags)) queries which cause full JSON parsing per row
|
||||
CREATE TABLE event_tags (
|
||||
event_id TEXT NOT NULL,
|
||||
tag_name TEXT NOT NULL,
|
||||
tag_value TEXT NOT NULL,
|
||||
tag_index INTEGER NOT NULL DEFAULT 0,
|
||||
FOREIGN KEY (event_id) REFERENCES events(id) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
-- Primary lookup index: find events by tag name + value
|
||||
CREATE INDEX idx_event_tags_lookup ON event_tags(tag_name, tag_value);
|
||||
|
||||
-- Reverse lookup: find all tags for an event (for cleanup)
|
||||
CREATE INDEX idx_event_tags_event ON event_tags(event_id);
|
||||
|
||||
-- Composite index for common query pattern: tag + kind (via join)
|
||||
CREATE INDEX idx_event_tags_value_name ON event_tags(tag_value, tag_name);
|
||||
```
|
||||
|
||||
**Key design decisions:**
|
||||
- `ON DELETE CASCADE` handles cleanup when events are deleted (NIP-09, replaceable events)
|
||||
- `tag_index` stores the position within the tags array (useful for ordered tag access)
|
||||
- Only the first two elements of each tag are indexed (`tag_name` = `$[0]`, `tag_value` = `$[1]`) — this covers all standard Nostr tag filters (`#e`, `#p`, `#t`, `#g`, `#d`, etc.)
|
||||
- `PRAGMA foreign_keys = ON` is already in the schema, so CASCADE will work
|
||||
|
||||
### 2. Store Tags: `src/main.c` — New `store_event_tags()` Function
|
||||
|
||||
Add a new function called after successful event INSERT in `store_event()`:
|
||||
|
||||
```c
|
||||
// Insert denormalized tags into event_tags table for fast indexed lookups
|
||||
int store_event_tags(const char* event_id, cJSON* tags) {
|
||||
if (!g_db || !event_id || !tags || !cJSON_IsArray(tags)) {
|
||||
return 0; // Not an error if no tags
|
||||
}
|
||||
|
||||
const char* sql = "INSERT INTO event_tags (event_id, tag_name, tag_value, tag_index) VALUES (?, ?, ?, ?)";
|
||||
sqlite3_stmt* stmt;
|
||||
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
|
||||
if (rc != SQLITE_OK) {
|
||||
DEBUG_ERROR("Failed to prepare event_tags insert: %s", sqlite3_errmsg(g_db));
|
||||
return -1;
|
||||
}
|
||||
|
||||
int tag_index = 0;
|
||||
cJSON* tag = NULL;
|
||||
cJSON_ArrayForEach(tag, tags) {
|
||||
if (cJSON_IsArray(tag) && cJSON_GetArraySize(tag) >= 2) {
|
||||
cJSON* name = cJSON_GetArrayItem(tag, 0);
|
||||
cJSON* value = cJSON_GetArrayItem(tag, 1);
|
||||
|
||||
if (cJSON_IsString(name) && cJSON_IsString(value)) {
|
||||
sqlite3_reset(stmt);
|
||||
sqlite3_bind_text(stmt, 1, event_id, -1, SQLITE_STATIC);
|
||||
sqlite3_bind_text(stmt, 2, cJSON_GetStringValue(name), -1, SQLITE_STATIC);
|
||||
sqlite3_bind_text(stmt, 3, cJSON_GetStringValue(value), -1, SQLITE_STATIC);
|
||||
sqlite3_bind_int(stmt, 4, tag_index);
|
||||
|
||||
rc = sqlite3_step(stmt);
|
||||
if (rc != SQLITE_DONE) {
|
||||
DEBUG_ERROR("Failed to insert event tag: %s", sqlite3_errmsg(g_db));
|
||||
}
|
||||
}
|
||||
}
|
||||
tag_index++;
|
||||
}
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
**Call site** — in `store_event()` at line ~784, after `monitoring_on_event_stored()`:
|
||||
|
||||
```c
|
||||
// After successful event storage, insert denormalized tags
|
||||
store_event_tags(cJSON_GetStringValue(id), tags);
|
||||
```
|
||||
|
||||
### 3. Update REQ Query Builder: `src/main.c` — `handle_req_message()`
|
||||
|
||||
Replace the `json_each()` tag filter at lines 1243-1270 with an `event_tags` subquery:
|
||||
|
||||
**Before** (current code at line 1244):
|
||||
```c
|
||||
snprintf(sql_ptr, remaining,
|
||||
" AND EXISTS (SELECT 1 FROM json_each(json(tags)) "
|
||||
"WHERE json_extract(value, '$[0]') = ? "
|
||||
"AND json_extract(value, '$[1]') IN (");
|
||||
```
|
||||
|
||||
**After** (new code):
|
||||
```c
|
||||
snprintf(sql_ptr, remaining,
|
||||
" AND id IN (SELECT event_id FROM event_tags "
|
||||
"WHERE tag_name = ? AND tag_value IN (");
|
||||
```
|
||||
|
||||
The rest of the parameter binding code (lines 1248-1270) stays the same — the bind params are identical (`tag_name` then `tag_value(s)`). Only the SQL template changes.
|
||||
|
||||
The closing parenthesis changes from `"))` to `"))` — same syntax, just different semantics.
|
||||
|
||||
### 4. NIP-09 Delete Cascade: `src/nip009.c`
|
||||
|
||||
The `ON DELETE CASCADE` foreign key handles this automatically. When `DELETE FROM events WHERE id = ?` executes, SQLite will automatically delete matching rows from `event_tags`. **No code changes needed in nip009.c** as long as `PRAGMA foreign_keys = ON` is set (it already is in the schema).
|
||||
|
||||
However, verify the replaceable/addressable event triggers in the schema also cascade properly. The existing triggers at schema lines 97-119 use `DELETE FROM events WHERE ...` which will trigger the CASCADE.
|
||||
|
||||
### 5. Performance Fix: Move Config Reads Out of Row Loop
|
||||
|
||||
In `handle_req_message()` at lines 1400-1401, `get_config_bool()` is called **inside the `while (sqlite3_step())` loop** — meaning it executes a SQLite query for every row returned. Move these before the loop:
|
||||
|
||||
**Before** (inside loop):
|
||||
```c
|
||||
while (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
// ...
|
||||
int expiration_enabled = get_config_bool("expiration_enabled", 1); // SQLite query per row!
|
||||
int filter_responses = get_config_bool("expiration_filter", 1); // SQLite query per row!
|
||||
```
|
||||
|
||||
**After** (before loop):
|
||||
```c
|
||||
int expiration_enabled = get_config_bool("expiration_enabled", 1);
|
||||
int filter_responses = get_config_bool("expiration_filter", 1);
|
||||
|
||||
while (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
// ... use cached values
|
||||
```
|
||||
|
||||
### 6. Log Level Fixes
|
||||
|
||||
#### NIP-17 Decryption Failure — `src/websockets.c` lines 780 and 1488
|
||||
|
||||
Change from `DEBUG_ERROR` to `DEBUG_INFO`:
|
||||
```c
|
||||
// Before:
|
||||
DEBUG_ERROR("NIP-17 admin message processing failed");
|
||||
// After:
|
||||
DEBUG_INFO("NIP-17 admin message processing failed");
|
||||
```
|
||||
|
||||
This is expected behavior when non-admin gift wraps arrive — not an error condition.
|
||||
|
||||
#### Duplicate Event INSERT — `src/main.c` line 741
|
||||
|
||||
The `DEBUG_ERROR` at line 741 fires before the CONSTRAINT check at line 746. Suppress it for constraint violations:
|
||||
|
||||
```c
|
||||
// Before (line 738-741):
|
||||
if (rc != SQLITE_DONE) {
|
||||
const char* err_msg = sqlite3_errmsg(g_db);
|
||||
int extended_errcode = sqlite3_extended_errcode(g_db);
|
||||
DEBUG_ERROR("INSERT failed: rc=%d, extended_errcode=%d, msg=%s", rc, extended_errcode, err_msg);
|
||||
}
|
||||
|
||||
// After:
|
||||
if (rc != SQLITE_DONE) {
|
||||
const char* err_msg = sqlite3_errmsg(g_db);
|
||||
int extended_errcode = sqlite3_extended_errcode(g_db);
|
||||
if (rc != SQLITE_CONSTRAINT) {
|
||||
DEBUG_ERROR("INSERT failed: rc=%d, extended_errcode=%d, msg=%s", rc, extended_errcode, err_msg);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 7. Startup Tag Population
|
||||
|
||||
Since you said no migration code is needed (fresh deploy), the `event_tags` table will start empty and populate as new events arrive. Existing events won't have tags in the lookup table.
|
||||
|
||||
However, to avoid a period where old events don't appear in tag-filtered queries, add a one-time population function that runs at startup:
|
||||
|
||||
```c
|
||||
// Populate event_tags from existing events (run once at startup)
|
||||
int populate_event_tags_from_existing(void) {
|
||||
if (!g_db) return -1;
|
||||
|
||||
// Check if event_tags is already populated
|
||||
sqlite3_stmt* check_stmt;
|
||||
sqlite3_prepare_v2(g_db, "SELECT COUNT(*) FROM event_tags", -1, &check_stmt, NULL);
|
||||
if (sqlite3_step(check_stmt) == SQLITE_ROW && sqlite3_column_int(check_stmt, 0) > 0) {
|
||||
sqlite3_finalize(check_stmt);
|
||||
DEBUG_INFO("event_tags already populated, skipping");
|
||||
return 0;
|
||||
}
|
||||
sqlite3_finalize(check_stmt);
|
||||
|
||||
DEBUG_INFO("Populating event_tags from existing events...");
|
||||
|
||||
const char* sql = "SELECT id, tags FROM events WHERE tags != '[]'";
|
||||
sqlite3_stmt* stmt;
|
||||
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
|
||||
if (rc != SQLITE_OK) return -1;
|
||||
|
||||
// Use a transaction for bulk insert performance
|
||||
sqlite3_exec(g_db, "BEGIN TRANSACTION", NULL, NULL, NULL);
|
||||
|
||||
int event_count = 0;
|
||||
while (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
const char* event_id = (const char*)sqlite3_column_text(stmt, 0);
|
||||
const char* tags_json = (const char*)sqlite3_column_text(stmt, 1);
|
||||
|
||||
if (event_id && tags_json) {
|
||||
cJSON* tags = cJSON_Parse(tags_json);
|
||||
if (tags) {
|
||||
store_event_tags(event_id, tags);
|
||||
cJSON_Delete(tags);
|
||||
event_count++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
sqlite3_exec(g_db, "COMMIT", NULL, NULL, NULL);
|
||||
|
||||
DEBUG_INFO("Populated event_tags for %d events", event_count);
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
Call this during startup, after database initialization but before accepting connections.
|
||||
|
||||
## Architecture Diagram
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph "Current: O(n) per query"
|
||||
A1["REQ with #g filter"] --> B1["SQL: SELECT ... WHERE EXISTS"]
|
||||
B1 --> C1["json_each(json(tags))"]
|
||||
C1 --> D1["json_extract per row"]
|
||||
D1 --> E1["🔴 Full JSON parse per candidate row"]
|
||||
end
|
||||
|
||||
subgraph "New: O(log n) per query"
|
||||
A2["REQ with #g filter"] --> B2["SQL: SELECT ... WHERE id IN"]
|
||||
B2 --> C2["event_tags table"]
|
||||
C2 --> D2["idx_event_tags_lookup"]
|
||||
D2 --> E2["🟢 B-tree index lookup"]
|
||||
end
|
||||
|
||||
subgraph "Write Path (unchanged speed)"
|
||||
F["EVENT arrives"] --> G["store_event()"]
|
||||
G --> H["INSERT INTO events"]
|
||||
H --> I["store_event_tags()"]
|
||||
I --> J["INSERT INTO event_tags\n(one row per tag)"]
|
||||
end
|
||||
|
||||
subgraph "Delete Path (automatic)"
|
||||
K["NIP-09 DELETE"] --> L["DELETE FROM events"]
|
||||
L --> M["ON DELETE CASCADE"]
|
||||
M --> N["event_tags rows auto-deleted"]
|
||||
end
|
||||
```
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
1. Run existing test suite: `tests/run_all_tests.sh`
|
||||
2. Specifically run NIP tests that use tag filters: `tests/run_nip_tests.sh`
|
||||
3. Verify tag-filtered REQ queries return correct results
|
||||
4. Verify NIP-09 deletion cascades to event_tags
|
||||
5. Verify replaceable event triggers cascade to event_tags
|
||||
6. Monitor CPU usage after deployment with same traffic pattern
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- **Low risk**: The `event_tags` table is additive — it doesn't change the events table structure
|
||||
- **Low risk**: `ON DELETE CASCADE` is a well-tested SQLite feature
|
||||
- **Low risk**: The SQL change in `handle_req_message()` is a drop-in replacement (same bind params)
|
||||
- **Medium risk**: Startup population on a large database could take a few seconds — but with only 55K events, this should complete in under 1 second
|
||||
- **Write overhead**: Each event INSERT now also inserts ~5 rows into event_tags — negligible at 56 events/hour
|
||||
@@ -0,0 +1,585 @@
|
||||
# Plan: Ban Idle and Early-Disconnect Connections
|
||||
|
||||
## Problem Statement
|
||||
|
||||
The relay needs to defend against spam connections that:
|
||||
1. Connect via WebSocket and sit idle doing nothing (the original problem)
|
||||
2. Connect and immediately disconnect without subscribing or posting (the new requirement)
|
||||
|
||||
Both patterns indicate bot/scanner behavior that should result in IP bans.
|
||||
|
||||
## Goal
|
||||
|
||||
Implement a system that:
|
||||
- Bans IPs that connect but never send a REQ or EVENT (regardless of how the connection ends)
|
||||
- Works **independent of NIP-42 authentication** (no auth required)
|
||||
- Allows legitimate users to connect and use the relay normally
|
||||
- Uses existing IP ban infrastructure
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Client connects] --> B{IP banned?}
|
||||
B -->|Yes| C[Reject immediately]
|
||||
B -->|No| D[Set idle timeout timer]
|
||||
D --> E{Client sends REQ or EVENT?}
|
||||
E -->|Yes| F[Mark session ACTIVE<br>Cancel idle timer]
|
||||
E -->|No| G{Connection closes?}
|
||||
G --> H{Session was ACTIVE?}
|
||||
H -->|Yes| I[Clean close - no ban]
|
||||
H -->|No| J["ip_ban_record_failure()<br>→ may trigger ban"]
|
||||
F --> K[Normal operation]
|
||||
K --> L[Connection closes]
|
||||
L --> I
|
||||
```
|
||||
|
||||
## Key Insight
|
||||
|
||||
The distinction between "idle timeout" and "early disconnect" is minimal:
|
||||
- **Idle timeout**: Connection closed by server because client never became ACTIVE
|
||||
- **Early disconnect**: Connection closed by client because client never became ACTIVE
|
||||
|
||||
Both result in the same check: `if (!session_was_active) ban_ip()`
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### 1. Add Session Activity Tracking
|
||||
|
||||
**File: `src/websockets.h`**
|
||||
|
||||
Add to `struct per_session_data`:
|
||||
|
||||
```c
|
||||
// Session activity tracking for idle connection banning
|
||||
int session_active; // 1 if client sent REQ or EVENT, 0 otherwise
|
||||
int idle_timeout_sec; // Timeout value for this session (copied from config)
|
||||
```
|
||||
|
||||
### 2. Set Idle Timeout on All Connections
|
||||
|
||||
**File: `src/websockets.c` - `LWS_CALLBACK_ESTABLISHED`**
|
||||
|
||||
Currently (lines 561-572):
|
||||
```c
|
||||
// Only set timeout if auth is required
|
||||
if (pss->nip42_auth_required_events || pss->nip42_auth_required_subscriptions) {
|
||||
int auth_timeout = get_config_int("nip42_auth_timeout_sec", 10);
|
||||
if (auth_timeout > 0) {
|
||||
lws_set_timeout(wsi, PENDING_TIMEOUT_AWAITING_PING, auth_timeout);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Change to:
|
||||
```c
|
||||
// Initialize session activity tracking
|
||||
pss->session_active = 0;
|
||||
pss->idle_timeout_sec = get_config_int("idle_connection_timeout_sec", 30);
|
||||
|
||||
// Set idle timeout for ALL connections (not just auth-required)
|
||||
// This catches bots that connect and do nothing
|
||||
if (pss->idle_timeout_sec > 0) {
|
||||
lws_set_timeout(wsi, PENDING_TIMEOUT_AWAITING_PING, pss->idle_timeout_sec);
|
||||
DEBUG_TRACE("Idle timeout set: %d seconds for connection from %s",
|
||||
pss->idle_timeout_sec, pss->client_ip);
|
||||
}
|
||||
|
||||
// Also set auth timeout if auth is required (separate concern)
|
||||
if (pss->nip42_auth_required_events || pss->nip42_auth_required_subscriptions) {
|
||||
int auth_timeout = get_config_int("nip42_auth_timeout_sec", 10);
|
||||
// Use the shorter of the two timeouts
|
||||
int effective_timeout = (pss->idle_timeout_sec > 0 && pss->idle_timeout_sec < auth_timeout)
|
||||
? pss->idle_timeout_sec
|
||||
: auth_timeout;
|
||||
if (effective_timeout > 0) {
|
||||
lws_set_timeout(wsi, PENDING_TIMEOUT_AWAITING_PING, effective_timeout);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Mark Session as Active on Valid Activity
|
||||
|
||||
**File: `src/websockets.c` - `LWS_CALLBACK_RECEIVE`**
|
||||
|
||||
When processing REQ message (around line 1030):
|
||||
```c
|
||||
// Before handling REQ, mark session as active
|
||||
pthread_mutex_lock(&pss->session_lock);
|
||||
pss->session_active = 1;
|
||||
pthread_mutex_unlock(&pss->session_lock);
|
||||
|
||||
// Cancel idle timeout - this is legitimate usage
|
||||
lws_set_timeout(wsi, NO_PENDING_TIMEOUT, 0);
|
||||
```
|
||||
|
||||
When processing EVENT message (around line 1380):
|
||||
```c
|
||||
// Before handling EVENT, mark session as active
|
||||
pthread_mutex_lock(&pss->session_lock);
|
||||
pss->session_active = 1;
|
||||
pthread_mutex_unlock(&pss->session_lock);
|
||||
|
||||
// Cancel idle timeout - this is legitimate usage
|
||||
lws_set_timeout(wsi, NO_PENDING_TIMEOUT, 0);
|
||||
```
|
||||
|
||||
**Note**: We should also consider AUTH as "activity" since the client is engaging with the protocol:
|
||||
```c
|
||||
// In handle_nip42_auth_signed_event (nip042.c)
|
||||
// After successful auth, also mark session active
|
||||
pthread_mutex_lock(&pss->session_lock);
|
||||
pss->session_active = 1;
|
||||
pthread_mutex_unlock(&pss->session_lock);
|
||||
```
|
||||
|
||||
### 3.5. Separate Rate Limiting for Idle Connections
|
||||
|
||||
Per user feedback, idle/early-disconnect failures have **separate rate limiting** from auth failures. This requires:
|
||||
|
||||
**New Config Keys:**
|
||||
|
||||
| Config Key | Type | Default | Range | Description |
|
||||
|------------|------|---------|-------|-------------|
|
||||
| `idle_ban_threshold` | int | 3 | 1-100 | Idle failures before ban |
|
||||
| `idle_ban_window_sec` | int | 60 | 10-3600 | Window for counting idle failures |
|
||||
| `idle_ban_duration_sec` | int | 300 | 60-86400 | Initial idle ban duration |
|
||||
|
||||
**Add to `ip_ban_entry_t` in `src/ip_ban.c`:**
|
||||
|
||||
```c
|
||||
typedef struct {
|
||||
int state;
|
||||
char ip[46];
|
||||
|
||||
// Auth failure tracking (existing)
|
||||
int failure_count;
|
||||
time_t first_failure;
|
||||
time_t banned_until;
|
||||
int ban_count;
|
||||
|
||||
// NEW: Idle failure tracking (separate)
|
||||
int idle_failure_count;
|
||||
time_t idle_first_failure;
|
||||
time_t idle_banned_until;
|
||||
int idle_ban_count;
|
||||
|
||||
// Other existing fields...
|
||||
int has_authed_successfully;
|
||||
time_t last_success_at;
|
||||
int total_connections;
|
||||
int total_failures;
|
||||
int total_successes;
|
||||
time_t first_seen;
|
||||
} ip_ban_entry_t;
|
||||
```
|
||||
|
||||
**New function in `src/ip_ban.c`:**
|
||||
|
||||
```c
|
||||
// Record an idle/early-disconnect failure for an IP
|
||||
void ip_ban_record_idle_failure(const char* ip) {
|
||||
if (!ip || !g_initialized) return;
|
||||
if (!get_config_bool("idle_ban_enabled", 1)) return;
|
||||
|
||||
int threshold = get_config_int("idle_ban_threshold", 3);
|
||||
int window_sec = get_config_int("idle_ban_window_sec", 60);
|
||||
int ban_duration = get_config_int("idle_ban_duration_sec", 300);
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
ip_ban_entry_t* entry = get_or_create_entry(ip);
|
||||
if (!entry) {
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return;
|
||||
}
|
||||
|
||||
time_t now = time(NULL);
|
||||
|
||||
// Reset window if expired
|
||||
if (entry->idle_first_failure > 0 && (now - entry->idle_first_failure) > window_sec) {
|
||||
entry->idle_failure_count = 0;
|
||||
entry->idle_first_failure = now;
|
||||
}
|
||||
if (entry->idle_first_failure == 0) {
|
||||
entry->idle_first_failure = now;
|
||||
}
|
||||
|
||||
entry->idle_failure_count++;
|
||||
entry->total_failures++;
|
||||
|
||||
DEBUG_TRACE("IP %s idle failure count: %d/%d", ip, entry->idle_failure_count, threshold);
|
||||
|
||||
if (entry->idle_failure_count >= threshold) {
|
||||
int duration = ban_duration;
|
||||
for (int i = 0; i < entry->idle_ban_count && duration < 86400; i++) {
|
||||
duration *= 2;
|
||||
}
|
||||
if (duration > 86400) duration = 86400;
|
||||
|
||||
entry->idle_banned_until = now + duration;
|
||||
entry->idle_ban_count++;
|
||||
entry->idle_failure_count = 0;
|
||||
entry->idle_first_failure = 0;
|
||||
|
||||
DEBUG_WARN("IP %s banned for %d seconds (idle ban #%d) after %d idle failures",
|
||||
ip, duration, entry->idle_ban_count, threshold);
|
||||
}
|
||||
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
}
|
||||
```
|
||||
|
||||
**Update `ip_ban_is_banned()` to check BOTH ban types:**
|
||||
|
||||
```c
|
||||
int ip_ban_is_banned(const char* ip) {
|
||||
if (!ip || !g_initialized) return 0;
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
int idx = find_slot(ip);
|
||||
if (idx < 0 || g_ban_table[idx].state == IP_BAN_EMPTY) {
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return 0;
|
||||
}
|
||||
|
||||
ip_ban_entry_t* entry = &g_ban_table[idx];
|
||||
time_t now = time(NULL);
|
||||
int banned = 0;
|
||||
|
||||
// Check auth ban (if enabled)
|
||||
if (get_config_bool("auth_fail_ban_enabled", 1) &&
|
||||
entry->banned_until > 0 && now < entry->banned_until) {
|
||||
banned = 1;
|
||||
}
|
||||
|
||||
// Check idle ban (if enabled)
|
||||
if (get_config_bool("idle_ban_enabled", 1) &&
|
||||
entry->idle_banned_until > 0 && now < entry->idle_banned_until) {
|
||||
banned = 1;
|
||||
}
|
||||
|
||||
// Clear expired bans
|
||||
if (!banned) {
|
||||
if (entry->banned_until > 0 && now >= entry->banned_until) {
|
||||
entry->banned_until = 0;
|
||||
entry->failure_count = 0;
|
||||
entry->first_failure = 0;
|
||||
}
|
||||
if (entry->idle_banned_until > 0 && now >= entry->idle_banned_until) {
|
||||
entry->idle_banned_until = 0;
|
||||
entry->idle_failure_count = 0;
|
||||
entry->idle_first_failure = 0;
|
||||
}
|
||||
}
|
||||
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return banned;
|
||||
}
|
||||
```
|
||||
|
||||
**Update persistence:**
|
||||
- `ip_ban_load_from_db()`: Load idle_* fields from new columns
|
||||
- `ip_ban_save_to_db()`: Save idle_* fields to new columns
|
||||
- Add migration SQL to create new columns if they don't exist
|
||||
|
||||
**Add to `src/ip_ban.h`:**
|
||||
|
||||
```c
|
||||
// Record an idle/early-disconnect failure for an IP
|
||||
void ip_ban_record_idle_failure(const char* ip);
|
||||
```
|
||||
|
||||
### 4. Record Failure on Inactive Connection Close
|
||||
|
||||
**File: `src/websockets.c` - `LWS_CALLBACK_CLOSED`**
|
||||
|
||||
Currently (lines 2121-2128):
|
||||
```c
|
||||
// Record auth failure if connection closed while unauthenticated and auth was required
|
||||
if (!pss->authenticated &&
|
||||
(pss->nip42_auth_required_events || pss->nip42_auth_required_subscriptions) &&
|
||||
pss->auth_challenge_sent &&
|
||||
strlen(pss->client_ip) > 0) {
|
||||
ip_ban_record_failure(pss->client_ip);
|
||||
}
|
||||
```
|
||||
|
||||
Change to:
|
||||
```c
|
||||
// Record failure if connection closed without ever becoming active
|
||||
// This catches:
|
||||
// 1. Idle connections that timed out (never sent REQ/EVENT/AUTH)
|
||||
// 2. Early disconnects (client closed before sending REQ/EVENT/AUTH)
|
||||
if (!pss->session_active && strlen(pss->client_ip) > 0) {
|
||||
// Use separate idle failure recording (has its own threshold/duration)
|
||||
ip_ban_record_idle_failure(pss->client_ip);
|
||||
DEBUG_LOG("Recording idle/early-disconnect failure for IP %s (connected %ld seconds)",
|
||||
pss->client_ip,
|
||||
time(NULL) - pss->connection_established);
|
||||
}
|
||||
// Legacy: record auth failure if auth was required but not completed
|
||||
else if (!pss->authenticated &&
|
||||
(pss->nip42_auth_required_events || pss->nip42_auth_required_subscriptions) &&
|
||||
pss->auth_challenge_sent &&
|
||||
strlen(pss->client_ip) > 0) {
|
||||
ip_ban_record_failure(pss->client_ip);
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Add Configuration Keys
|
||||
|
||||
**File: `src/config.c` - Add validation for new keys**
|
||||
|
||||
In the config validation section, add:
|
||||
|
||||
```c
|
||||
// Idle connection ban settings
|
||||
if (strcmp(key, "idle_connection_timeout_sec") == 0) {
|
||||
if (!is_valid_positive_integer(value)) {
|
||||
snprintf(error_msg, error_size, "invalid idle_connection_timeout_sec '%s' (must be positive integer)", value);
|
||||
return -1;
|
||||
}
|
||||
int timeout = atoi(value);
|
||||
if (timeout < 5 || timeout > 300) {
|
||||
snprintf(error_msg, error_size, "idle_connection_timeout_sec must be between 5 and 300 seconds");
|
||||
return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (strcmp(key, "idle_ban_enabled") == 0) {
|
||||
if (!is_valid_boolean(value)) {
|
||||
snprintf(error_msg, error_size, "invalid boolean value '%s' for idle_ban_enabled", value);
|
||||
return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (strcmp(key, "idle_ban_threshold") == 0) {
|
||||
if (!is_valid_positive_integer(value)) {
|
||||
snprintf(error_msg, error_size, "invalid idle_ban_threshold '%s' (must be positive integer)", value);
|
||||
return -1;
|
||||
}
|
||||
int threshold = atoi(value);
|
||||
if (threshold < 1 || threshold > 100) {
|
||||
snprintf(error_msg, error_size, "idle_ban_threshold must be between 1 and 100");
|
||||
return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (strcmp(key, "idle_ban_window_sec") == 0) {
|
||||
if (!is_valid_positive_integer(value)) {
|
||||
snprintf(error_msg, error_size, "invalid idle_ban_window_sec '%s' (must be positive integer)", value);
|
||||
return -1;
|
||||
}
|
||||
int window = atoi(value);
|
||||
if (window < 10 || window > 3600) {
|
||||
snprintf(error_msg, error_size, "idle_ban_window_sec must be between 10 and 3600");
|
||||
return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (strcmp(key, "idle_ban_duration_sec") == 0) {
|
||||
if (!is_valid_positive_integer(value)) {
|
||||
snprintf(error_msg, error_size, "invalid idle_ban_duration_sec '%s' (must be positive integer)", value);
|
||||
return -1;
|
||||
}
|
||||
int duration = atoi(value);
|
||||
if (duration < 60 || duration > 86400) {
|
||||
snprintf(error_msg, error_size, "idle_ban_duration_sec must be between 60 and 86400");
|
||||
return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
Also update the config introspection/documentation functions to describe these keys.
|
||||
|
||||
### 6. Add API Documentation
|
||||
|
||||
**File: `src/api.c` - Add to config metadata**
|
||||
|
||||
```c
|
||||
{"idle_connection_timeout_sec", "int", 5, 300},
|
||||
{"idle_ban_enabled", "bool", 0, 1},
|
||||
{"idle_ban_threshold", "int", 1, 100},
|
||||
{"idle_ban_window_sec", "int", 10, 3600},
|
||||
{"idle_ban_duration_sec", "int", 60, 86400},
|
||||
```
|
||||
|
||||
Add descriptions in the config query handler:
|
||||
```c
|
||||
} else if (strcmp(key, "idle_connection_timeout_sec") == 0) {
|
||||
description = "Seconds before an idle connection (no REQ/EVENT) is closed and IP flagged. 0 to disable.";
|
||||
} else if (strcmp(key, "idle_ban_enabled") == 0) {
|
||||
description = "Whether to ban IPs that repeatedly connect without sending REQ or EVENT.";
|
||||
} else if (strcmp(key, "idle_ban_threshold") == 0) {
|
||||
description = "Number of idle/early-disconnect failures before banning an IP.";
|
||||
} else if (strcmp(key, "idle_ban_window_sec") == 0) {
|
||||
description = "Time window in seconds for counting idle failures.";
|
||||
} else if (strcmp(key, "idle_ban_duration_sec") == 0) {
|
||||
description = "Initial ban duration in seconds for idle failures (doubles each time).";
|
||||
}
|
||||
```
|
||||
|
||||
### 7. Update ip_ban Persistence and Cleanup
|
||||
|
||||
**File: `src/ip_ban.c` - Update `ip_ban_load_from_db()`**
|
||||
|
||||
Add migration SQL to create new columns if they don't exist, then load them:
|
||||
```sql
|
||||
ALTER TABLE ip_bans ADD COLUMN idle_failure_count INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE ip_bans ADD COLUMN idle_ban_count INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE ip_bans ADD COLUMN idle_banned_until INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE ip_bans ADD COLUMN idle_first_failure INTEGER NOT NULL DEFAULT 0;
|
||||
```
|
||||
|
||||
**File: `src/ip_ban.c` - Update `ip_ban_save_to_db()`**
|
||||
|
||||
Add the idle_* fields to the INSERT/REPLACE statement.
|
||||
|
||||
**File: `src/ip_ban.c` - Update `ip_ban_cleanup()`**
|
||||
|
||||
Add cleanup logic for idle ban entries (same pattern as auth ban cleanup).
|
||||
|
||||
**File: `src/ip_ban.c` - Update `ip_ban_log_stats()`**
|
||||
|
||||
Add idle ban count to the periodic log summary:
|
||||
```c
|
||||
DEBUG_WARN("IP BAN SUMMARY: %d auth-banned, %d idle-banned, %d tracked, %d trusted",
|
||||
auth_banned_count, idle_banned_count, tracked_count, trusted_count);
|
||||
```
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
### Idle Connection Settings
|
||||
|
||||
| Config Key | Type | Default | Range | Description |
|
||||
|------------|------|---------|-------|-------------|
|
||||
| `idle_connection_timeout_sec` | int | 30 | 5-300 | Seconds to wait for REQ/EVENT before closing connection. 0 = disable idle timeout. |
|
||||
| `idle_ban_enabled` | bool | true | true/false | Whether to ban IPs with repeated idle/early-disconnect failures. |
|
||||
| `idle_ban_threshold` | int | 3 | 1-100 | Idle failures before ban. |
|
||||
| `idle_ban_window_sec` | int | 60 | 10-3600 | Window for counting idle failures. |
|
||||
| `idle_ban_duration_sec` | int | 300 | 60-86400 | Initial idle ban duration. |
|
||||
|
||||
### Auth Failure Settings (Existing)
|
||||
|
||||
| Config Key | Type | Default | Range | Description |
|
||||
|------------|------|---------|-------|-------------|
|
||||
| `auth_fail_ban_enabled` | bool | true | true/false | Whether to ban IPs with failed auth. |
|
||||
| `auth_fail_ban_threshold` | int | 3 | 1-100 | Auth failures before ban. |
|
||||
| `auth_fail_window_sec` | int | 60 | 10-3600 | Window for counting auth failures. |
|
||||
| `auth_fail_ban_duration_sec` | int | 300 | 60-86400 | Initial auth ban duration. |
|
||||
|
||||
## Behavior Matrix
|
||||
|
||||
| Scenario | idle_timeout_sec | idle_ban_enabled | Result |
|
||||
|----------|------------------|------------------|--------|
|
||||
| Connect → do nothing → timeout | >0 | true | Connection closed, IP failure recorded, may be banned |
|
||||
| Connect → do nothing → timeout | >0 | false | Connection closed, no ban |
|
||||
| Connect → immediate disconnect | any | true | IP failure recorded, may be banned |
|
||||
| Connect → immediate disconnect | any | false | No ban |
|
||||
| Connect → send REQ | any | any | Session active, no ban |
|
||||
| Connect → send EVENT | any | any | Session active, no ban |
|
||||
| Connect → send AUTH (NIP-42) | any | any | Session active, no ban |
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
1. **Idle timeout test**: Connect via wscat, wait 30 seconds, verify disconnect and ban table entry
|
||||
2. **Early disconnect test**: Connect via wscat, immediately Ctrl-C, verify ban table entry
|
||||
3. **Active session test**: Connect, send REQ, disconnect immediately, verify NO ban
|
||||
4. **Config disable test**: Set `idle_ban_enabled=false`, repeat test 1, verify no ban
|
||||
5. **Timeout config test**: Set `idle_connection_timeout_sec=5`, verify faster disconnect
|
||||
|
||||
## Files to Modify
|
||||
|
||||
| File | Change Type |
|
||||
|------|-------------|
|
||||
| `src/websockets.h` | Add `session_active` and `idle_timeout_sec` fields to `per_session_data` |
|
||||
| `src/websockets.c` ~565 | Set idle timeout on ALL connections in `LWS_CALLBACK_ESTABLISHED` |
|
||||
| `src/websockets.c` ~1030 | Mark `session_active=1` and cancel idle timer on REQ |
|
||||
| `src/websockets.c` ~1380 | Mark `session_active=1` and cancel idle timer on EVENT |
|
||||
| `src/websockets.c` ~2121 | Call `ip_ban_record_idle_failure()` for inactive sessions in `LWS_CALLBACK_CLOSED` |
|
||||
| `src/nip042.c` ~130 | Mark `session_active=1` on successful AUTH |
|
||||
| `src/ip_ban.h` | Add `ip_ban_record_idle_failure()` declaration |
|
||||
| `src/ip_ban.c` | Add idle_* fields to `ip_ban_entry_t`, new `ip_ban_record_idle_failure()` function, update `ip_ban_is_banned()` to check both ban types, update persistence and cleanup |
|
||||
| `src/config.c` ~987+ | Add validation for 5 new idle_* config keys |
|
||||
| `src/api.c` ~664 | Add config metadata entries for idle_* keys |
|
||||
|
||||
## Deployment on Existing Server
|
||||
|
||||
### Config Keys: No Manual SQL Required
|
||||
|
||||
All config keys use [`get_config_int()`](src/config.c:300) and [`get_config_bool()`](src/config.c:313) which return **hardcoded defaults** when a key doesn't exist in the config table. The new code will work immediately with these defaults:
|
||||
|
||||
- `idle_connection_timeout_sec` → defaults to 30
|
||||
- `idle_ban_enabled` → defaults to true
|
||||
- `idle_ban_threshold` → defaults to 3
|
||||
- `idle_ban_window_sec` → defaults to 60
|
||||
- `idle_ban_duration_sec` → defaults to 300
|
||||
|
||||
If you want to **override** any defaults, you can insert them into the config table. From the same directory as the `.db` file:
|
||||
|
||||
```bash
|
||||
# Find your database file
|
||||
DB_FILE=$(ls *.db | head -1)
|
||||
|
||||
# Optional: Insert custom values (only if you want non-default settings)
|
||||
sqlite3 "$DB_FILE" "INSERT OR REPLACE INTO config (key, value) VALUES ('idle_connection_timeout_sec', '30');"
|
||||
sqlite3 "$DB_FILE" "INSERT OR REPLACE INTO config (key, value) VALUES ('idle_ban_enabled', 'true');"
|
||||
sqlite3 "$DB_FILE" "INSERT OR REPLACE INTO config (key, value) VALUES ('idle_ban_threshold', '3');"
|
||||
sqlite3 "$DB_FILE" "INSERT OR REPLACE INTO config (key, value) VALUES ('idle_ban_window_sec', '60');"
|
||||
sqlite3 "$DB_FILE" "INSERT OR REPLACE INTO config (key, value) VALUES ('idle_ban_duration_sec', '300');"
|
||||
```
|
||||
|
||||
Or change them via the admin API/web UI after deployment.
|
||||
|
||||
### ip_bans Table: Automatic Migration
|
||||
|
||||
The `ip_bans` table needs 4 new columns for idle tracking. The updated [`ip_ban_load_from_db()`](src/ip_ban.c:93) will run `ALTER TABLE` statements automatically on startup. These are safe because SQLite's `ALTER TABLE ADD COLUMN` is a no-op if the column already exists (we'll use error-tolerant execution).
|
||||
|
||||
If you prefer to run the migration manually before deploying:
|
||||
|
||||
```bash
|
||||
DB_FILE=$(ls *.db | head -1)
|
||||
|
||||
sqlite3 "$DB_FILE" <<'SQL'
|
||||
ALTER TABLE ip_bans ADD COLUMN idle_failure_count INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE ip_bans ADD COLUMN idle_ban_count INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE ip_bans ADD COLUMN idle_banned_until INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE ip_bans ADD COLUMN idle_first_failure INTEGER NOT NULL DEFAULT 0;
|
||||
SQL
|
||||
```
|
||||
|
||||
**Note:** SQLite will error on `ALTER TABLE ADD COLUMN` if the column already exists, but the code will handle this gracefully (ignore the error). Running it manually is optional — the relay will do it on startup.
|
||||
|
||||
### Deployment Steps
|
||||
|
||||
1. Build and deploy with `deploy_lt.sh` as usual
|
||||
2. The relay starts, `ip_ban_load_from_db()` auto-migrates the `ip_bans` table
|
||||
3. Config keys use defaults immediately — no manual SQL needed
|
||||
4. Optionally tune settings via admin API or direct SQL
|
||||
|
||||
## Backward Compatibility
|
||||
|
||||
- Default `idle_connection_timeout_sec=30` provides immediate protection
|
||||
- Default `idle_ban_enabled=true` maintains current spam protection behavior
|
||||
- Setting `idle_connection_timeout_sec=0` restores old behavior (no idle timeout)
|
||||
- Setting `idle_ban_enabled=false` allows connections without banning (for debugging)
|
||||
- Existing auth-based banning continues to work independently with its own thresholds
|
||||
- Database migration adds new columns with defaults — existing ban data preserved
|
||||
|
||||
## Summary
|
||||
|
||||
This plan implements a unified "session activity" tracking system with **separate rate limiting** for idle vs auth failures:
|
||||
|
||||
1. **Catches idle connections** via `lws_set_timeout()` on ALL connections (not just auth-required)
|
||||
2. **Catches early disconnects** via the same `!session_active` check on close
|
||||
3. **Respects legitimate users** who actually use the relay (REQ/EVENT/AUTH marks session active)
|
||||
4. **Separate idle ban tracking** — idle failures have their own threshold, window, and duration independent of auth failures
|
||||
5. **Extends existing ban infrastructure** — adds idle_* fields to `ip_ban_entry_t`, unified `ip_ban_is_banned()` checks both ban types
|
||||
6. **Fully configurable** — 5 new config keys for idle banning, all tunable via admin events
|
||||
|
||||
The key insight is that there's no meaningful difference between "idle timeout" and "early disconnect" — both indicate a client that connected but never actually used the relay. The same `!session_active` check handles both cases, and the separate rate limiting ensures idle bots don't interfere with auth failure tracking for legitimate clients who fail NIP-42.
|
||||
@@ -0,0 +1,194 @@
|
||||
# Main Thread CPU Offload Plan
|
||||
|
||||
## Problem Statement
|
||||
|
||||
The main `c_relay_pg` thread consumes ~56% CPU while the DB worker threads (`db-read-1..4`, `db-write`) sit near 0%. All protocol handling, JSON parsing, event validation, subscription matching, and message queueing happens synchronously inside `nostr_relay_callback()` on the main libwebsockets event-loop thread.
|
||||
|
||||
## Current Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
LWS[lws_service - main thread] --> CB[nostr_relay_callback]
|
||||
CB --> PARSE[JSON parse - cJSON_Parse]
|
||||
CB --> VALIDATE[Signature verify - nostr_validate_unified_request]
|
||||
CB --> STORE[store_event - builds payload]
|
||||
STORE --> SYNC_WRITE[thread_pool_execute_store_event_sync]
|
||||
SYNC_WRITE --> |blocks main thread| DB_WRITE[db-write thread]
|
||||
DB_WRITE --> |signal| SYNC_WRITE
|
||||
CB --> REQ[handle_req_message - builds SQL]
|
||||
REQ --> SYNC_READ[thread_pool_execute_req_sync]
|
||||
SYNC_READ --> |blocks main thread| DB_READ[db-read-N thread]
|
||||
DB_READ --> |signal| SYNC_READ
|
||||
CB --> BROADCAST[broadcast_event_to_subscriptions]
|
||||
BROADCAST --> QUEUE[queue_message per subscriber]
|
||||
CB --> WRITEABLE[LWS_CALLBACK_SERVER_WRITEABLE]
|
||||
WRITEABLE --> DRAIN[process_message_queue - lws_write]
|
||||
```
|
||||
|
||||
### Why the main thread is hot
|
||||
|
||||
| Work item | Where | Cost |
|
||||
|-----------|-------|------|
|
||||
| JSON parsing | `cJSON_Parse` in `LWS_CALLBACK_RECEIVE` | Medium - per message |
|
||||
| Signature verification | `nostr_validate_unified_request` - ed25519 crypto | **High** - per EVENT |
|
||||
| SQL query building | `handle_req_message` filter-to-SQL loop | Low-Medium |
|
||||
| Sync DB wait | `thread_pool_execute_*_sync` - pthread_cond_wait | Blocks but yields CPU |
|
||||
| Result iteration + expiration check | Row loop in `handle_req_message` with `cJSON_Parse` per row | Medium-High for large result sets |
|
||||
| Subscription matching | `broadcast_event_to_subscriptions` - filter matching | Medium - scales with subscriber count |
|
||||
| Message serialization + queueing | `snprintf` + `queue_message_take_ownership` per subscriber | Medium |
|
||||
| Config lookups | `get_config_int/bool` called repeatedly in hot paths | Low but frequent |
|
||||
|
||||
### Key constraint: libwebsockets is single-threaded
|
||||
|
||||
libwebsockets requires that **all `lws_write`, `lws_callback_on_writable`, and `lws_close_reason` calls happen from the service thread** (the thread running `lws_service`). This means we cannot directly write to WebSocket connections from worker threads. However, we **can** do computation on worker threads and post results back to the main thread for I/O.
|
||||
|
||||
## Offload Strategy
|
||||
|
||||
### Phase 1: Async EVENT Processing (Highest Impact)
|
||||
|
||||
Convert EVENT handling from synchronous to async. Currently the main thread does: parse → validate → store → broadcast, all blocking. Instead:
|
||||
|
||||
1. **Main thread**: Parse JSON (fast), extract event ID for dedup check, then submit a job to a new "event processing" worker thread
|
||||
2. **Worker thread**: Signature verification (expensive crypto), store_event (already goes to DB thread), prepare broadcast payload
|
||||
3. **Main thread callback**: Receive result via `wake_loop_cb` + `lws_cancel_service`, send OK response and broadcast to subscribers
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
LWS[lws_service - main thread] --> RECEIVE[LWS_CALLBACK_RECEIVE]
|
||||
RECEIVE --> PARSE[JSON parse + dedup check]
|
||||
PARSE --> SUBMIT[Submit to event-worker queue]
|
||||
SUBMIT --> LWS
|
||||
|
||||
WORKER[event-worker thread] --> VERIFY[Signature verification]
|
||||
VERIFY --> STORE_DB[store_event via DB pool]
|
||||
STORE_DB --> PREP[Prepare broadcast payload]
|
||||
PREP --> RESULT_Q[Push result to completion queue]
|
||||
RESULT_Q --> WAKE[lws_cancel_service]
|
||||
|
||||
LWS2[lws_service wakes] --> POLL[Poll completion queue]
|
||||
POLL --> OK[Send OK response via queue_message]
|
||||
POLL --> BCAST[broadcast_event_to_subscriptions]
|
||||
```
|
||||
|
||||
**What this offloads**: ed25519 signature verification (~the most expensive per-event operation), event classification, tag serialization, and the synchronous DB store wait.
|
||||
|
||||
**What stays on main thread**: JSON parse (needed to extract event ID for dedup), OK response writing, broadcast fan-out (requires lws access).
|
||||
|
||||
### Phase 2: Async REQ Query Execution (Medium Impact)
|
||||
|
||||
Convert REQ handling from sync to async:
|
||||
|
||||
1. **Main thread**: Parse filters, build SQL, create subscription, submit query job
|
||||
2. **DB reader thread**: Execute query (already happens, but currently blocks main thread via `_sync`)
|
||||
3. **Main thread callback**: Iterate results, queue EVENT messages, send EOSE
|
||||
|
||||
This is simpler than Phase 1 because the thread pool already supports async submission via `thread_pool_submit_read` with a `result_cb`. The `_sync` wrappers just add a condvar wait on top. We need to:
|
||||
- Use `thread_pool_submit_read` directly instead of `thread_pool_execute_req_sync`
|
||||
- Store pending REQ context (sub_id, wsi, pss) so the callback can complete the work
|
||||
- In the result callback, push results to a completion queue and call `lws_cancel_service`
|
||||
- On the main thread, drain the completion queue and send EVENT + EOSE messages
|
||||
|
||||
### Phase 3: Reduce Per-Row Overhead in REQ Results (Low-Medium Impact)
|
||||
|
||||
Currently each row from a REQ query gets `cJSON_Parse` just to check NIP-40 expiration. This is wasteful:
|
||||
|
||||
- Option A: Add an `expiration` column to the events table so expiration filtering can be done in SQL
|
||||
- Option B: Store expiration timestamp in a fast-parse format (extract during INSERT, store as integer column)
|
||||
- Option C: Use string search on the raw JSON for the expiration tag instead of full parse
|
||||
|
||||
### Phase 4: Config Value Caching (Low Impact, Easy Win)
|
||||
|
||||
`get_config_int` and `get_config_bool` are called on every message in hot paths. These do SQLite queries. Cache config values in memory with a TTL or invalidation signal, so the main loop only refreshes them periodically (already done for `debug_level` every 60s — extend to all hot-path config values).
|
||||
|
||||
## Implementation Priority
|
||||
|
||||
| Phase | Impact | Risk | Complexity |
|
||||
|-------|--------|------|------------|
|
||||
| Phase 1: Async EVENT | **High** - removes crypto from main thread | Medium - async state management | Medium-High |
|
||||
| Phase 2: Async REQ | **Medium** - unblocks main thread during queries | Low - infrastructure exists | Medium |
|
||||
| Phase 3: Expiration optimization | **Low-Medium** - reduces per-row parse cost | Low | Low |
|
||||
| Phase 4: Config caching | **Low** - reduces DB round-trips | Very Low | Low |
|
||||
|
||||
## Detailed Design: Phase 1 (Async EVENT Processing)
|
||||
|
||||
### New Components
|
||||
|
||||
#### Completion Queue (`src/completion_queue.h/.c`)
|
||||
A thread-safe FIFO queue for posting results from worker threads back to the main thread:
|
||||
|
||||
```c
|
||||
typedef struct {
|
||||
int type; // COMPLETION_TYPE_EVENT_RESULT, COMPLETION_TYPE_REQ_RESULT, etc.
|
||||
void* data; // Type-specific result data
|
||||
struct lws* wsi; // Target WebSocket connection
|
||||
void* pss; // Per-session data
|
||||
} completion_item_t;
|
||||
|
||||
int completion_queue_init(void);
|
||||
int completion_queue_push(completion_item_t* item);
|
||||
completion_item_t* completion_queue_pop(void); // Non-blocking
|
||||
void completion_queue_shutdown(void);
|
||||
```
|
||||
|
||||
#### Event Worker Thread
|
||||
A dedicated pthread that processes EVENT validation/storage:
|
||||
|
||||
```c
|
||||
typedef struct {
|
||||
cJSON* event; // Parsed event JSON - ownership transferred
|
||||
cJSON* full_message; // Full message JSON for context
|
||||
struct lws* wsi;
|
||||
void* pss;
|
||||
char sub_id[64]; // For response routing
|
||||
} event_work_item_t;
|
||||
```
|
||||
|
||||
#### Main Loop Integration
|
||||
Add a completion queue drain step to the main event loop. After `lws_service` returns (either from timeout or `lws_cancel_service` wake), check the completion queue:
|
||||
|
||||
```c
|
||||
while (g_server_running && !g_shutdown_flag) {
|
||||
int result = lws_service(ws_context, 1000);
|
||||
|
||||
// NEW: Drain completion queue
|
||||
completion_item_t* item;
|
||||
while ((item = completion_queue_pop()) != NULL) {
|
||||
process_completion(item); // Send OK, broadcast, etc.
|
||||
free(item);
|
||||
}
|
||||
|
||||
// ... existing periodic checks ...
|
||||
}
|
||||
```
|
||||
|
||||
### Changes to Existing Code
|
||||
|
||||
1. **`nostr_relay_callback` EVENT path**: After JSON parse and dedup check, instead of calling `nostr_validate_unified_request` + `store_event` + `broadcast_event_to_subscriptions` synchronously, submit an `event_work_item_t` to the event worker queue and return 0 immediately.
|
||||
|
||||
2. **`store_event`**: No changes needed — it already uses `thread_pool_execute_store_event_sync` which will run on the DB writer thread. The event worker thread will call it.
|
||||
|
||||
3. **`broadcast_event_to_subscriptions`**: No changes needed — it will be called from the main thread when processing the completion item, which is correct since it calls `queue_message_take_ownership` (requires lws thread).
|
||||
|
||||
### Thread Safety Considerations
|
||||
|
||||
- The `cJSON* event` object must be fully owned by the worker thread during processing. The main thread must not access it after submission.
|
||||
- The `wsi` and `pss` pointers could become invalid if the client disconnects while the event is being processed. The completion handler must validate that the connection is still alive before sending the OK response.
|
||||
- A generation counter or epoch on `pss` can detect stale references.
|
||||
|
||||
## What Cannot Be Offloaded
|
||||
|
||||
- **`lws_write` / `queue_message`**: Must happen on the lws service thread
|
||||
- **`lws_callback_on_writable`**: Must happen on the lws service thread
|
||||
- **`lws_close_reason`**: Must happen on the lws service thread
|
||||
- **Subscription list iteration for broadcast**: Accesses `lws_wsi_user` which is lws-internal
|
||||
|
||||
These are fundamental libwebsockets constraints. The pattern is always: do computation off-thread, post result to completion queue, wake main thread, do I/O on main thread.
|
||||
|
||||
## Expected Impact
|
||||
|
||||
With Phase 1 alone, the main thread would no longer perform:
|
||||
- ed25519 signature verification (~100-500μs per event depending on CPU)
|
||||
- Synchronous DB store wait (~50-200μs per event)
|
||||
- Event classification, tag serialization, JSON serialization for storage
|
||||
|
||||
This should reduce main-thread CPU by roughly 30-50% for EVENT-heavy workloads, shifting that work to the event worker thread and DB threads.
|
||||
@@ -0,0 +1,207 @@
|
||||
# C-Relay-PG Memory Leak Investigation & Fix Plan
|
||||
|
||||
## Problem Statement
|
||||
|
||||
c-relay-pg reached **1.56 GB** of its 1.5 GB `MemoryHigh` cgroup limit after only **1 hour 37 minutes** of runtime. The kernel is actively throttling the process with swap pressure (1.8M swap used). This strongly indicates one or more memory leaks causing unbounded growth.
|
||||
|
||||
## Root Cause Analysis
|
||||
|
||||
After thorough code review, I identified **three categories** of memory issues:
|
||||
|
||||
1. **Confirmed `get_config_value()` leaks** — the most pervasive issue
|
||||
2. **Potential structural leaks** in subscription/connection lifecycle
|
||||
3. **Stack pressure** from large stack-allocated arrays
|
||||
|
||||
---
|
||||
|
||||
## Category 1: `get_config_value()` Leaks (HIGH SEVERITY — Most Likely Root Cause)
|
||||
|
||||
### The Core Problem
|
||||
|
||||
[`get_config_value()`](src/config.c:293) calls [`get_config_value_from_table()`](src/config.c:1690) which returns `strdup(value)` — a **heap-allocated string that the caller must free**. Many call sites treat the return value as a borrowed pointer and never free it.
|
||||
|
||||
**Every unfree'd call leaks ~64-256 bytes per invocation.** On a busy relay processing hundreds of events/second, this accumulates to GB-scale leaks within hours.
|
||||
|
||||
### Confirmed Leak Sites
|
||||
|
||||
#### `websockets.c` — Called on EVERY incoming event
|
||||
|
||||
| Line | Code | Frequency | Severity |
|
||||
|------|------|-----------|----------|
|
||||
| [1527](src/websockets.c:1527) | `get_config_value("relay_pubkey")` — auth bypass check for kind 14 DMs | Every kind-14 event | **HIGH** |
|
||||
| [1549](src/websockets.c:1549) | `get_config_value("admin_pubkey")` — auth bypass check for kind 23456 | Every kind-23456 event | **MEDIUM** |
|
||||
| [2704](src/websockets.c:2704) | `get_config_value("relay_pubkey")` — DM stats command processing | Every DM to relay | **MEDIUM** |
|
||||
| [2742](src/websockets.c:2742) | `get_config_value("admin_pubkey")` — DM admin check | Every DM to relay | **MEDIUM** |
|
||||
|
||||
#### `main.c` — Called on EVERY admin event check
|
||||
|
||||
| Line | Code | Frequency | Severity |
|
||||
|------|------|-----------|----------|
|
||||
| [1718](src/main.c:1718) | `get_config_value("relay_pubkey")` — inside a loop iterating tags | Every admin event, per tag | **CRITICAL** |
|
||||
| [1742](src/main.c:1742) | `get_config_value("admin_pubkey")` — admin authorization | Every admin event | **HIGH** |
|
||||
|
||||
#### `config.c` — Called on EVERY event kind check
|
||||
|
||||
| Line | Code | Frequency | Severity |
|
||||
|------|------|-----------|----------|
|
||||
| [377](src/config.c:377) | `get_config_value("nip42_auth_required_kinds")` — NIP-42 kind check | Every incoming event | **CRITICAL** |
|
||||
|
||||
#### `ip_ban.c` — Called on EVERY ban check
|
||||
|
||||
| Line | Code | Frequency | Severity |
|
||||
|------|------|-----------|----------|
|
||||
| [255](src/ip_ban.c:255) | `get_config_value("idle_ban_whitelist")` — whitelist check | Every connection check | **HIGH** |
|
||||
|
||||
#### `nip042.c` — Called on every auth verification
|
||||
|
||||
| Line | Code | Frequency | Severity |
|
||||
|------|------|-----------|----------|
|
||||
| [99](src/nip042.c:99) | `get_config_value("relay_url")` — relay URL for auth verification | Every NIP-42 auth | **MEDIUM** |
|
||||
|
||||
#### `request_validator.c` — Called on every auth rules check
|
||||
|
||||
| Line | Code | Frequency | Severity |
|
||||
|------|------|-----------|----------|
|
||||
| [208](src/request_validator.c:208) | `get_config_value("auth_enabled")` — properly freed | N/A | OK |
|
||||
| [216](src/request_validator.c:216) | `get_config_value("auth_rules_enabled")` — properly freed | N/A | OK |
|
||||
| [304](src/request_validator.c:304) | `get_config_value("admin_pubkey")` — NOT freed | Every event validation | **HIGH** |
|
||||
| [320](src/request_validator.c:320) | `get_config_value("nip42_auth_enabled")` — NOT freed | Every event validation | **HIGH** |
|
||||
|
||||
### Files That DO Free Correctly (for reference)
|
||||
|
||||
- [`nip011.c`](src/nip011.c:127) — Properly frees all `get_config_value()` results
|
||||
- [`nip013.c`](src/nip013.c:43) — Properly frees `pow_mode`
|
||||
- [`request_validator.c`](src/request_validator.c:191) — Properly frees `nip42_timeout` and `nip42_tolerance`
|
||||
|
||||
### Estimated Impact
|
||||
|
||||
On a relay processing ~100 events/second:
|
||||
- `is_nip42_auth_required_for_kind()` leaks ~100-200 bytes × 100/sec = **~10-20 KB/sec**
|
||||
- `ip_is_whitelisted()` leaks ~100 bytes × connections/sec
|
||||
- `is_authorized_admin_event()` leaks inside tag loop — potentially **multiple leaks per event**
|
||||
- Combined: **~50-100 KB/sec → ~180-360 MB/hour** — consistent with the observed 1.56 GB in 97 minutes
|
||||
|
||||
---
|
||||
|
||||
## Category 2: Structural Lifecycle Leaks (MEDIUM SEVERITY)
|
||||
|
||||
### 2a. `LWS_CALLBACK_CLOSED` — Inactive Subscription Skip
|
||||
|
||||
In [`websockets.c:2339`](src/websockets.c:2339), the cleanup handler only collects subscription IDs where `sub->active` is true:
|
||||
|
||||
```c
|
||||
if (sub->active) { // Only process active subscriptions
|
||||
temp_sub_id_t* temp = malloc(sizeof(temp_sub_id_t));
|
||||
```
|
||||
|
||||
If a subscription was marked inactive by another thread between the time it was added and the connection close, it will be **skipped** — never removed from the global manager, never freed. The subscription object, its filters, and all cJSON objects within leak permanently.
|
||||
|
||||
### 2b. `connection_list_remove` Potential
|
||||
|
||||
The `connection_list_remove(wsi)` call at [`websockets.c:2252`](src/websockets.c:2252) happens before pss cleanup. Need to verify the connection list implementation doesn't hold references that prevent cleanup.
|
||||
|
||||
### 2c. `ip_connection_info_t` Leak on Disconnect
|
||||
|
||||
The per-IP connection tracking in [`subscriptions.h:79`](src/subscriptions.h:79) creates `ip_connection_info_t` nodes via `calloc`. These are only removed by `remove_ip_connection()` — need to verify this is called on every disconnect path.
|
||||
|
||||
---
|
||||
|
||||
## Category 3: Stack Pressure (LOW SEVERITY but notable)
|
||||
|
||||
### 3a. `candidates_to_check` Stack Array
|
||||
|
||||
In [`subscriptions.c:810`](src/subscriptions.c:810):
|
||||
|
||||
```c
|
||||
subscription_t* candidates_to_check[MAX_TOTAL_SUBSCRIPTIONS]; // 5000 × 8 bytes = 40 KB
|
||||
```
|
||||
|
||||
This allocates **40 KB on the stack** for every `broadcast_event_to_subscriptions()` call. While not a heap leak, it contributes to high memory pressure under concurrent load.
|
||||
|
||||
### 3b. `added_kinds` Bitmap
|
||||
|
||||
In [`subscriptions.c:68`](src/subscriptions.c:68):
|
||||
|
||||
```c
|
||||
unsigned char added_kinds[8192] = {0}; // 8 KB on stack
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Fix Implementation Plan
|
||||
|
||||
### Phase 1: Fix `get_config_value()` Leaks (Highest Impact)
|
||||
|
||||
**Strategy A — Preferred: Add a config cache layer**
|
||||
|
||||
Instead of fixing 20+ call sites, add a **cached config system** that reads values once and caches them in memory, invalidating on config change events. This eliminates both the leak AND the repeated SQLite queries per event.
|
||||
|
||||
```
|
||||
Config Cache Architecture:
|
||||
- Static hash table of key-value pairs
|
||||
- Populated at startup and on config change events
|
||||
- get_config_value_cached() returns const pointer to cached value (no allocation)
|
||||
- Existing get_config_value() kept for dynamic/rare lookups
|
||||
```
|
||||
|
||||
**Strategy B — Quick fix: Free at every call site**
|
||||
|
||||
Add `free((char*)result)` after every `get_config_value()` call that doesn't already free. This is more error-prone but faster to implement.
|
||||
|
||||
**Recommendation: Implement Strategy A for hot-path values (relay_pubkey, admin_pubkey, auth settings), Strategy B for cold-path values.**
|
||||
|
||||
### Phase 2: Fix Structural Leaks
|
||||
|
||||
1. **Fix inactive subscription skip in CLOSED handler** — Remove the `if (sub->active)` guard, or add a second pass that also collects inactive subscriptions for cleanup
|
||||
2. **Verify `remove_ip_connection()` is called on all disconnect paths**
|
||||
3. **Add defensive cleanup** — periodic sweep of orphaned subscriptions
|
||||
|
||||
### Phase 3: Add Memory Monitoring
|
||||
|
||||
1. **Add a `/stats` endpoint** that reports:
|
||||
- Current RSS/VSZ from `/proc/self/status`
|
||||
- Active subscription count
|
||||
- Message queue depth across all sessions
|
||||
- Config cache hit/miss ratio
|
||||
2. **Add periodic memory logging** to the service loop
|
||||
3. **Consider integrating jemalloc** for better allocation tracking in production
|
||||
|
||||
### Phase 4: Reduce Stack Pressure
|
||||
|
||||
1. Move `candidates_to_check` to heap allocation with `malloc`/`free`
|
||||
2. Consider reducing `MAX_TOTAL_SUBSCRIPTIONS` or using a dynamic array
|
||||
|
||||
---
|
||||
|
||||
## Verification Strategy
|
||||
|
||||
1. **Build with AddressSanitizer** (`-fsanitize=address`) for development testing
|
||||
2. **Run under Valgrind** with `--leak-check=full` for comprehensive leak detection
|
||||
3. **Monitor RSS over time** after fixes — should plateau rather than grow linearly
|
||||
4. **Load test** with `tests/load_tests.sh` while monitoring memory
|
||||
|
||||
---
|
||||
|
||||
## Immediate Mitigation
|
||||
|
||||
While fixes are being implemented:
|
||||
1. **Raise `MemoryHigh`** to 2 GB in the systemd unit to prevent throttling
|
||||
2. **Add a cron job** to restart the service every 12-24 hours
|
||||
3. **Monitor with**: `watch -n 5 'cat /proc/$(pgrep c_relay_pg)/status | grep -E "VmRSS|VmSwap"'`
|
||||
|
||||
---
|
||||
|
||||
## Implementation Priority Order
|
||||
|
||||
| Priority | Task | Impact |
|
||||
|----------|------|--------|
|
||||
| P0 | Fix `is_nip42_auth_required_for_kind()` leak in config.c:377 | Called on every event |
|
||||
| P0 | Fix `ip_is_whitelisted()` leak in ip_ban.c:255 | Called on every connection check |
|
||||
| P0 | Fix `is_authorized_admin_event()` leaks in main.c:1718,1742 | Called per admin event, leaks in loop |
|
||||
| P1 | Fix websockets.c:1527,1549 auth bypass leaks | Called on every event with auth |
|
||||
| P1 | Fix request_validator.c:304,320 leaks | Called on every event validation |
|
||||
| P1 | Fix nip042.c:99 relay_url leak | Called on every NIP-42 auth |
|
||||
| P2 | Implement config cache for hot-path values | Eliminates leak class entirely |
|
||||
| P2 | Fix inactive subscription cleanup in CLOSED handler | Prevents slow subscription leak |
|
||||
| P3 | Add memory monitoring endpoint | Ongoing visibility |
|
||||
| P3 | Move large stack arrays to heap | Reduces stack pressure |
|
||||
@@ -0,0 +1,257 @@
|
||||
# PostgreSQL Nostr Relay Architecture Analysis
|
||||
|
||||
This document summarizes common approaches and best practices for building Nostr relays backed by PostgreSQL, based on industry-standard designs and existing open-source relay implementations.
|
||||
|
||||
## Identified Projects
|
||||
* **Nostrss**: High-performance Rust-based relay.
|
||||
* **Nostrgres**: Focused on PostgreSQL integration with complex event filtering.
|
||||
* **Relay-rs (Postgres branch)**: General purpose high-throughput relay.
|
||||
* **Nostream** ([github.com/Cameri/nostream](https://github.com/Cameri/nostream)): Production-ready TypeScript relay backed by PostgreSQL and Redis.
|
||||
* **Nostr-Relay-NestJS** ([github.com/CodyTseng/nostr-relay-nestjs](https://github.com/CodyTseng/nostr-relay-nestjs)): High-performance relay built with NestJS and PostgreSQL, utilizing Kysely for type-safe database interactions.
|
||||
|
||||
## Detailed Analysis
|
||||
|
||||
### 1. Common Schema Patterns
|
||||
* **Event Storage (JSONB)**: Most PostgreSQL-backed relays store the raw event as a JSONB object in a central `events` table. This provides flexibility for the evolving Nostr spec while allowing efficient extraction of fields.
|
||||
* **Tag Denormalization**: While the raw event is in JSONB, performant relays extract tags (like `p`, `e`, `t`, `d`) into a separate `tags` table with foreign key relationships to the `events` table. This avoids slow JSONB traversal during complex filter queries.
|
||||
* **Author/Publisher Tracking**: A dedicated `authors` or `pubkeys` table is typically used to maintain indexing on the `pubkey` field, enabling quick lookups of user activity.
|
||||
|
||||
### 2. Performance Optimization
|
||||
* **GIN Indexes on JSONB**: Crucial for filtering on specific event tags or custom properties stored within the event object. GIN indexes with `jsonb_path_ops` are generally preferred for equality checks.
|
||||
* **Time-Series Partitioning**: Given the append-only nature of Nostr, partitioning the `events` table by time (e.g., daily or weekly chunks) is highly recommended. This significantly improves query performance for recent data and simplifies data expiration/deletion.
|
||||
* **Clustering**: Clustering the `events` table by the timestamp index can reduce disk I/O, as it keeps temporally related events physically adjacent on the disk.
|
||||
|
||||
### 3. Schema Management Approaches
|
||||
* **Migrations**: Most mature relays utilize migration tools (like `Flyway`, `Diesel migrations`, or `Golang-migrate`) to version control database schema changes. This is critical for production stability.
|
||||
* **Auto-generation**: Some lightweight prototypes auto-generate schemas at startup. This is generally discouraged for production due to the risk of destructive changes, data loss, or blocking DDL operations on large tables.
|
||||
|
||||
## Recommendations for Building a New Relay
|
||||
1. **Prioritize Denormalization**: Do not rely solely on JSONB queries for high-traffic filters. Extract indexed tags into dedicated columns or tables.
|
||||
2. **Use Partitioning from Day One**: Implementing native PostgreSQL partitioning (e.g., using `pg_partman`) is much easier before the dataset grows to millions of rows.
|
||||
3. **Connection Pooling**: PostgreSQL requires aggressive connection management. Use a high-performance pooler like `PgBouncer` to handle the large number of concurrent, short-lived connections common in WebSocket-based relay traffic.
|
||||
4. **Asynchronous Writes**: Decouple the WebSocket ingestion thread from the database writer thread to ensure that slow database writes do not impact the relay's responsiveness.
|
||||
|
||||
## Nostream ([github.com/Cameri/nostream](https://github.com/Cameri/nostream))
|
||||
|
||||
A production-ready Nostr relay written in TypeScript (Node.js), backed by PostgreSQL 14+ and Redis. Uses Knex.js for versioned migrations.
|
||||
|
||||
### PostgreSQL Schema
|
||||
|
||||
The schema is fully normalized — no JSONB blob for the whole event. Tags are stored separately and populated by a trigger.
|
||||
|
||||
**`events` table** — one row per event, tags kept as a JSONB column for the trigger to process:
|
||||
```sql
|
||||
CREATE TABLE events (
|
||||
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
|
||||
event_id BYTEA UNIQUE NOT NULL, -- 32-byte hash
|
||||
event_pubkey BYTEA NOT NULL,
|
||||
event_kind INTEGER UNSIGNED NOT NULL,
|
||||
event_created_at INTEGER UNSIGNED NOT NULL,
|
||||
event_content TEXT NOT NULL,
|
||||
event_tags JSONB, -- raw tags array, source of truth for trigger
|
||||
event_signature BYTEA NOT NULL,
|
||||
remote_address TEXT,
|
||||
expires_at INTEGER,
|
||||
deleted_at TIMESTAMP,
|
||||
event_deduplication JSONB, -- used for replaceable event conflict key
|
||||
first_seen TIMESTAMP DEFAULT now()
|
||||
);
|
||||
|
||||
-- Indexes
|
||||
CREATE INDEX ON events (event_pubkey);
|
||||
CREATE INDEX ON events (event_kind);
|
||||
CREATE INDEX ON events (event_created_at);
|
||||
CREATE INDEX ON events (expires_at);
|
||||
CREATE INDEX event_tags_idx ON events USING GIN (event_tags); -- GIN on raw JSONB
|
||||
|
||||
-- Unique index enforcing NIP-16 replaceable event semantics
|
||||
CREATE UNIQUE INDEX replaceable_events_idx ON events (event_pubkey, event_kind)
|
||||
WHERE event_kind = 0 OR event_kind = 3
|
||||
OR (event_kind >= 10000 AND event_kind < 20000);
|
||||
```
|
||||
|
||||
**`event_tags` table** — denormalized single-letter tags, populated by a PostgreSQL trigger on `events`:
|
||||
```sql
|
||||
CREATE TABLE event_tags (
|
||||
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
|
||||
event_id BYTEA NOT NULL,
|
||||
tag_name TEXT NOT NULL, -- single-letter only (e.g. 'e', 'p', 't')
|
||||
tag_value TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX ON event_tags (event_id);
|
||||
CREATE INDEX ON event_tags (tag_name, tag_value);
|
||||
|
||||
-- Trigger: on INSERT/UPDATE/DELETE to events, repopulate event_tags
|
||||
CREATE OR REPLACE FUNCTION process_event_tags() RETURNS TRIGGER AS $$
|
||||
DECLARE
|
||||
tag_element jsonb;
|
||||
BEGIN
|
||||
DELETE FROM event_tags WHERE event_id = OLD.event_id;
|
||||
IF TG_OP = 'INSERT' OR TG_OP = 'UPDATE' THEN
|
||||
FOR tag_element IN SELECT jsonb_array_elements(NEW.event_tags) LOOP
|
||||
IF length(trim('"' FROM (tag_element->0)::text)) = 1
|
||||
AND (tag_element->1)::text IS NOT NULL THEN
|
||||
INSERT INTO event_tags (event_id, tag_name, tag_value)
|
||||
VALUES (NEW.event_id,
|
||||
trim('"' FROM (tag_element->0)::text),
|
||||
trim('"' FROM (tag_element->1)::text));
|
||||
END IF;
|
||||
END LOOP;
|
||||
END IF;
|
||||
RETURN NEW;
|
||||
END;
|
||||
$$ LANGUAGE plpgsql;
|
||||
|
||||
CREATE TRIGGER insert_event_tags
|
||||
AFTER INSERT OR UPDATE OR DELETE ON events
|
||||
FOR EACH ROW EXECUTE FUNCTION process_event_tags();
|
||||
```
|
||||
|
||||
Tag queries (`#e`, `#p`, etc.) join `event_tags` rather than traversing the JSONB array.
|
||||
|
||||
**`users` and `invoices` tables** — for paid relay admission (Lightning payments):
|
||||
```sql
|
||||
CREATE TABLE users (
|
||||
pubkey BYTEA PRIMARY KEY,
|
||||
is_admitted BOOLEAN DEFAULT FALSE,
|
||||
balance BIGINT DEFAULT 0, -- in msats
|
||||
tos_accepted_at DATETIME
|
||||
);
|
||||
|
||||
CREATE TABLE invoices (
|
||||
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
|
||||
pubkey BYTEA NOT NULL,
|
||||
bolt11 TEXT NOT NULL,
|
||||
amount_requested BIGINT UNSIGNED NOT NULL,
|
||||
amount_paid BIGINT UNSIGNED,
|
||||
unit ENUM('msats','sats','btc'),
|
||||
status ENUM('pending','completed','expired'),
|
||||
description TEXT,
|
||||
confirmed_at DATETIME,
|
||||
expires_at DATETIME
|
||||
);
|
||||
```
|
||||
|
||||
### How Redis Is Used
|
||||
|
||||
Redis is **not** used for subscription fan-out. Subscription delivery is handled entirely in-process: each `WebSocketAdapter` holds a `Map<subscriptionId, filters[]>`, and a broadcast event walks all connected clients and filters locally.
|
||||
|
||||
Redis has two actual roles:
|
||||
|
||||
1. **Sliding-window rate limiting** — The `SlidingWindowRateLimiter` uses Redis sorted sets to track request timestamps per key (pubkey, IP, etc.) within a rolling time window:
|
||||
- `ZREMRANGEBYSCORE key 0 (now - period)` — evict old entries
|
||||
- `ZADD key timestamp member` — record this hit
|
||||
- `ZRANGE key 0 -1` — count hits in window
|
||||
- `EXPIRE key period` — auto-cleanup
|
||||
- Keys follow the pattern `<pubkey>:events:<period>` or `<ip>:message:<period>`
|
||||
|
||||
2. **General-purpose cache** — A `RedisAdapter` wraps the client with `get`/`set`/`hasKey` helpers, used for caching admission checks (e.g. `<pubkey>:is-admitted`). The event handler has a `TODO` comment noting that the `userRepository.findByPubkey()` call should be replaced with a cache lookup — meaning this is partially implemented.
|
||||
|
||||
---
|
||||
|
||||
## Schema Definitions
|
||||
|
||||
### Relay-rs (Postgres)
|
||||
```sql
|
||||
-- Events table
|
||||
CREATE TABLE "event" (
|
||||
id bytea NOT NULL,
|
||||
pub_key bytea NOT NULL,
|
||||
created_at timestamp with time zone NOT NULL,
|
||||
kind integer NOT NULL,
|
||||
"content" bytea NOT NULL,
|
||||
hidden bit(1) NOT NULL DEFAULT 0::bit(1),
|
||||
delegated_by bytea NULL,
|
||||
first_seen timestamp with time zone NOT NULL DEFAULT now(),
|
||||
expires_at timestamp(0) with time zone,
|
||||
CONSTRAINT event_pkey PRIMARY KEY (id)
|
||||
);
|
||||
CREATE INDEX event_created_at_idx ON "event" (created_at,kind);
|
||||
CREATE INDEX event_pub_key_idx ON "event" (pub_key);
|
||||
CREATE INDEX event_delegated_by_idx ON "event" (delegated_by);
|
||||
CREATE INDEX event_expires_at_idx ON "event" (expires_at);
|
||||
|
||||
-- Tags table
|
||||
CREATE TABLE "tag" (
|
||||
id int8 NOT NULL GENERATED BY DEFAULT AS IDENTITY,
|
||||
event_id bytea NOT NULL,
|
||||
"name" varchar NOT NULL,
|
||||
value bytea NULL,
|
||||
value_hex bytea NULL,
|
||||
CONSTRAINT tag_fk FOREIGN KEY (event_id) REFERENCES "event"(id) ON DELETE CASCADE,
|
||||
CONSTRAINT unique_constraint_name UNIQUE (event_id, "name", value, value_hex)
|
||||
);
|
||||
CREATE INDEX tag_event_id_idx ON tag USING btree (event_id, name);
|
||||
CREATE INDEX tag_value_idx ON tag USING btree (value);
|
||||
CREATE INDEX tag_value_hex_idx ON tag USING btree (value_hex);
|
||||
|
||||
-- Account table
|
||||
CREATE TABLE "account" (
|
||||
pubkey varchar NOT NULL,
|
||||
is_admitted BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
balance BIGINT NOT NULL DEFAULT 0,
|
||||
tos_accepted_at TIMESTAMP,
|
||||
CONSTRAINT account_pkey PRIMARY KEY (pubkey)
|
||||
);
|
||||
|
||||
-- Invoice table
|
||||
CREATE TYPE status AS ENUM ('Paid', 'Unpaid', 'Expired');
|
||||
CREATE TABLE "invoice" (
|
||||
payment_hash varchar NOT NULL,
|
||||
pubkey varchar NOT NULL,
|
||||
invoice varchar NOT NULL,
|
||||
amount BIGINT NOT NULL,
|
||||
status status NOT NULL DEFAULT 'Unpaid',
|
||||
description varchar,
|
||||
created_at timestamp,
|
||||
confirmed_at timestamp,
|
||||
CONSTRAINT invoice_payment_hash PRIMARY KEY (payment_hash),
|
||||
CONSTRAINT invoice_pubkey_fkey FOREIGN KEY (pubkey) REFERENCES account (pubkey) ON DELETE CASCADE
|
||||
);
|
||||
```
|
||||
|
||||
### Nostr-Relay-NestJS (PostgreSQL via Kysely)
|
||||
```sql
|
||||
-- Events table
|
||||
CREATE TABLE events (
|
||||
id CHAR(64) PRIMARY KEY,
|
||||
pubkey CHAR(64) NOT NULL,
|
||||
created_at BIGINT NOT NULL,
|
||||
kind INTEGER NOT NULL,
|
||||
tags JSONB NOT NULL DEFAULT '[]',
|
||||
generic_tags TEXT[] NOT NULL DEFAULT '{}',
|
||||
content TEXT NOT NULL DEFAULT '',
|
||||
sig CHAR(128) NOT NULL,
|
||||
expired_at BIGINT,
|
||||
d_tag_value TEXT,
|
||||
delegator CHAR(64),
|
||||
create_date TIMESTAMP NOT NULL DEFAULT now(),
|
||||
update_date TIMESTAMP NOT NULL DEFAULT now(),
|
||||
delete_date TIMESTAMP
|
||||
);
|
||||
|
||||
-- Indices
|
||||
CREATE EXTENSION IF NOT EXISTS btree_gin;
|
||||
CREATE INDEX generic_tags_kind_idx ON events USING gin (generic_tags, kind);
|
||||
CREATE INDEX pubkey_kind_created_at_idx ON events (pubkey, kind, created_at);
|
||||
CREATE INDEX delegator_kind_created_at_idx ON events (delegator, kind, created_at);
|
||||
CREATE INDEX created_at_kind_idx ON events (created_at, kind);
|
||||
|
||||
-- Generic tags table (for optimized tag queries)
|
||||
CREATE TABLE generic_tags (
|
||||
id SERIAL PRIMARY KEY,
|
||||
tag TEXT NOT NULL,
|
||||
author CHAR(64) NOT NULL,
|
||||
kind INTEGER NOT NULL,
|
||||
event_id CHAR(64) NOT NULL REFERENCES events(id),
|
||||
created_at BIGINT NOT NULL
|
||||
);
|
||||
|
||||
-- Indices for tag filtering
|
||||
CREATE UNIQUE INDEX g_tag_event_id_idx ON generic_tags (tag, event_id);
|
||||
CREATE INDEX g_tag_kind_created_at_idx ON generic_tags (tag, kind, created_at);
|
||||
CREATE INDEX g_tag_created_at_idx ON generic_tags (tag, created_at);
|
||||
```
|
||||
Nostr-Relay-NestJS features a hybrid storage model where tags are stored both in a JSONB field (for general lookup) and a normalized `generic_tags` table (for optimized tag query performance). The schema uses Kysely for type-safe interaction.
|
||||
|
||||
@@ -0,0 +1,204 @@
|
||||
# Remaining Implementation Plan — Zero SQLite on Main Thread + PG Fork Readiness
|
||||
|
||||
## Overview
|
||||
|
||||
This plan covers all remaining work to achieve two goals:
|
||||
1. **Zero direct SQLite calls on lws-main** during the event loop
|
||||
2. **Clean `db_ops` abstraction boundary** so the PG fork has no raw `sqlite3_*` calls outside `db_ops.c`
|
||||
|
||||
## Current State
|
||||
|
||||
### Completed (Fixes 1-5, 7 + Phases 1-3, 5)
|
||||
- Global config cache with 5s TTL
|
||||
- Dedup check moved to event worker
|
||||
- Async COUNT queries via thread pool
|
||||
- WoT auth rules cached per-session
|
||||
- Special-kind EVENT store through async worker
|
||||
- Subscription logging via fire-and-forget write queue
|
||||
- IP ban periodic save via write queue
|
||||
- g_db nulled before event loop
|
||||
|
||||
### Remaining
|
||||
- Fix 6: Reduce reader threads (trivial)
|
||||
- Fix 8: Move periodic tasks off main thread (partial — IP ban save done, monitoring/status not)
|
||||
- Fix 9: Cache NIP-11 response
|
||||
- Sync `store_event()` calls from admin/DM/NIP-09 paths
|
||||
- Raw `sqlite3_*` calls in `thread_pool.c` and `websockets.c`
|
||||
- COUNT query path in `websockets.c` still uses `json_each()` instead of `event_tags`
|
||||
|
||||
---
|
||||
|
||||
## Phase A: Move Remaining Periodic Tasks Off Main Thread
|
||||
|
||||
**Goal:** Eliminate the last regular SQLite calls from the lws-main event loop timer.
|
||||
|
||||
### A1: Move `generate_and_post_status_event()` to write worker
|
||||
|
||||
Currently called synchronously from the main event loop at `websockets.c:3455`.
|
||||
|
||||
**Changes:**
|
||||
- Add `THREAD_POOL_JOB_STATUS_POST` to `thread_pool.h` job type enum
|
||||
- Add executor in `thread_pool.c` that calls `generate_and_post_status_event()`
|
||||
- In `websockets.c` main loop, replace direct call with `thread_pool_submit_write()` job
|
||||
- The write worker has a DB connection, so `generate_stats_json()` → `store_event()` → `broadcast_event_to_subscriptions()` all work naturally
|
||||
- Note: `broadcast_event_to_subscriptions()` is thread-safe (uses its own mutex)
|
||||
|
||||
**Files:** `thread_pool.h`, `thread_pool.c`, `websockets.c`
|
||||
|
||||
### A2: Move monitoring queries to read/write worker
|
||||
|
||||
`monitoring_on_event_stored()` and `monitoring_on_subscription_change()` in `api.c` call `generate_event_driven_monitoring()` / `generate_subscription_driven_monitoring()` which do DB queries and then broadcast ephemeral events.
|
||||
|
||||
**Changes:**
|
||||
- Add `THREAD_POOL_JOB_MONITORING` job type
|
||||
- Submit monitoring generation as a read job (it mostly does SELECTs)
|
||||
- The completion broadcasts the ephemeral event via `lws_cancel_service()` + completion queue
|
||||
- Alternative simpler approach: since these are throttled (default 5s) and only fire when kind 24567 subscribers exist, they could run on the write worker as fire-and-forget
|
||||
|
||||
**Files:** `thread_pool.h`, `thread_pool.c`, `api.c`, `websockets.c`
|
||||
|
||||
### A3: Move `ip_ban_cleanup()` and `ip_ban_log_stats()` to write worker
|
||||
|
||||
Currently called from the 60-second periodic timer at `websockets.c:3479-3480`.
|
||||
|
||||
**Changes:**
|
||||
- Add `THREAD_POOL_JOB_IP_BAN_CLEANUP` job type (or reuse a generic callback job)
|
||||
- Submit as fire-and-forget write job from the periodic timer
|
||||
- `ip_ban_cleanup()` does in-memory cleanup + optional DB writes
|
||||
- `ip_ban_log_stats()` is purely logging, no DB
|
||||
|
||||
**Files:** `thread_pool.h`, `thread_pool.c`, `ip_ban.c`, `websockets.c`
|
||||
|
||||
---
|
||||
|
||||
## Phase B: Route Remaining Sync `store_event()` Calls Through Async Worker
|
||||
|
||||
**Goal:** Eliminate all synchronous `store_event()` calls from lws-main context.
|
||||
|
||||
### Current sync `store_event()` call sites on main thread
|
||||
|
||||
| Call site | File | Context | Frequency |
|
||||
|-----------|------|---------|-----------|
|
||||
| Kind 1059 gift wrap store | `websockets.c:1795,1803,1819` | NIP-17 DM processing | Low |
|
||||
| Kind 14 DM store | `websockets.c:1843` | NIP-17 DM processing | Low |
|
||||
| Regular event fallback | `websockets.c:1862,1876` | Non-async-eligible events | Low |
|
||||
| Kind 1059 (auth path) | `websockets.c:2594,2602,2618` | NIP-42 auth + DM | Low |
|
||||
| Kind 14 (auth path) | `websockets.c:2642` | NIP-42 auth + DM | Low |
|
||||
| Regular (auth path) | `websockets.c:2661,2675` | NIP-42 auth fallback | Low |
|
||||
| Admin response store | `config.c:2585` | Admin API response | Very low |
|
||||
| DM admin gift wrap | `dm_admin.c:389` | Admin DM response | Very low |
|
||||
| NIP-09 deletion store | `nip009.c:135` | Deletion events | Low |
|
||||
| Status event store | `api.c:595` | Periodic status post | Very low (handled by Phase A1) |
|
||||
| Admin response event | `api.c:908` | Admin API | Very low |
|
||||
| DM chat response | `api.c:1377` | Admin DM chat | Very low |
|
||||
| Relay-generated event | `api.c:2350` | Relay metadata | Very low |
|
||||
| DM stats response | `websockets.c:3648` | DM stats command | Very low |
|
||||
|
||||
### Strategy
|
||||
|
||||
Rather than converting each call site individually, create a **synchronous-to-async bridge**:
|
||||
|
||||
**Changes:**
|
||||
- Create `store_event_async(cJSON* event, store_event_callback_t cb, void* ctx)` that submits the event to the write worker
|
||||
- For call sites that need the result (e.g., to broadcast after store), use a completion callback
|
||||
- For fire-and-forget sites (admin responses, DM responses), use NULL callback
|
||||
- The existing `store_event()` wrapper becomes: submit to write worker + block on completion (for backward compat during transition)
|
||||
- Eventually, all callers migrate to the async version
|
||||
|
||||
**Alternative simpler approach:** Since these are all low-frequency paths, and the write worker already has a DB connection, we can submit them as generic write jobs with a callback that does the store + broadcast. The main thread just needs to hand off the event JSON.
|
||||
|
||||
**Files:** `main.c`, `websockets.c`, `api.c`, `config.c`, `dm_admin.c`, `nip009.c`, `thread_pool.h`, `thread_pool.c`
|
||||
|
||||
---
|
||||
|
||||
## Phase C: `db_ops` Abstraction Cleanup for PG Fork
|
||||
|
||||
**Goal:** Ensure all raw `sqlite3_*` calls are inside `db_ops.c` only. External code uses only `db_ops.h` functions.
|
||||
|
||||
### C1: Add `db_open_worker_connection()` / `db_close_worker_connection()` to `db_ops.h`
|
||||
|
||||
Currently, `thread_pool.c:208` and `websockets.c:537` call `sqlite3_open_v2()` directly.
|
||||
|
||||
**Changes:**
|
||||
- Add to `db_ops.h`:
|
||||
```c
|
||||
void* db_open_worker_connection(void); // Returns opaque connection handle
|
||||
void db_close_worker_connection(void* conn);
|
||||
```
|
||||
- Implementation in `db_ops.c`: opens a new SQLite connection with WAL + busy timeout
|
||||
- `thread_pool.c` worker init calls `db_open_worker_connection()` instead of raw `sqlite3_open_v2()`
|
||||
- `websockets.c` event-worker calls `db_open_worker_connection()` instead of raw `sqlite3_open_v2()`
|
||||
- Remove `#include <sqlite3.h>` from `thread_pool.c` and `websockets.c`
|
||||
|
||||
**Files:** `db_ops.h`, `db_ops.c`, `thread_pool.c`, `websockets.c`
|
||||
|
||||
### C2: Audit and remove remaining `sqlite3` includes outside `db_ops.c`
|
||||
|
||||
After C1, grep for any remaining `sqlite3` references outside `db_ops.c` and eliminate them.
|
||||
|
||||
**Files:** All `.c` files
|
||||
|
||||
### C3: Unify COUNT query tag path
|
||||
|
||||
The COUNT query builder in `websockets.c:3849` still uses `json_each(json(tags))` while the REQ query builder in `main.c:1604` uses the indexed `event_tags` table. Unify to use `event_tags` for consistency.
|
||||
|
||||
**Files:** `websockets.c`
|
||||
|
||||
---
|
||||
|
||||
## Phase D: Quick Wins
|
||||
|
||||
### D1: Fix 6 — Reduce reader threads from 4 to 1
|
||||
|
||||
Change default in `thread_pool_init()` or make configurable. With all reads going through the pool, a single reader is sufficient unless profiling shows otherwise.
|
||||
|
||||
**Files:** `thread_pool.c` or config default
|
||||
|
||||
### D2: Fix 9 — Cache NIP-11 response
|
||||
|
||||
Cache the serialized NIP-11 JSON string with a 60-second TTL. Invalidate on admin config change.
|
||||
|
||||
**Files:** `nip011.c`
|
||||
|
||||
---
|
||||
|
||||
## Implementation Order
|
||||
|
||||
| # | Task | Risk | Dependencies |
|
||||
|---|------|------|-------------|
|
||||
| 1 | A3: ip_ban_cleanup to write worker | Very Low | None |
|
||||
| 2 | A1: status post to write worker | Low | None |
|
||||
| 3 | A2: monitoring to worker | Low | None |
|
||||
| 4 | D1: reduce reader threads to 1 | Very Low | None |
|
||||
| 5 | D2: cache NIP-11 response | Very Low | None |
|
||||
| 6 | C1: db_open/close_worker_connection | Low | None |
|
||||
| 7 | C3: unify COUNT tag query path | Low | None |
|
||||
| 8 | C2: audit/remove sqlite3 includes | Very Low | C1 |
|
||||
| 9 | B: route sync store_event through async | Medium | A1 (status post already handled) |
|
||||
|
||||
## Expected Outcome
|
||||
|
||||
After all phases:
|
||||
- **lws-main has zero SQLite calls** during the event loop
|
||||
- **All `sqlite3_*` calls are inside `db_ops.c`** — clean abstraction boundary
|
||||
- **PG fork can replace `db_ops.c`** without touching any other file
|
||||
- Thread model is fully realized:
|
||||
- `lws-main` = WebSocket I/O + JSON parse + subscription match + completion drain
|
||||
- `event-worker` = signature verify + dedup + store submission
|
||||
- `db-read` = REQ queries + COUNT queries + monitoring reads
|
||||
- `db-write` = event INSERTs + subscription logging + periodic tasks + admin writes
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
CLIENT[Nostr Clients] --> LWS[lws-main<br/>Zero SQLite<br/>WS I/O + JSON + Sub Match]
|
||||
|
||||
LWS -->|EVENT| EW[event-worker<br/>Verify + Dedup]
|
||||
EW -->|store job| DW[db-write<br/>INSERT events<br/>Sub logging<br/>Periodic tasks<br/>Admin writes]
|
||||
EW -->|completion| LWS
|
||||
DW -->|completion| LWS
|
||||
|
||||
LWS -->|REQ/COUNT| DR[db-read<br/>SELECT queries<br/>Monitoring reads]
|
||||
DR -->|completion| LWS
|
||||
|
||||
LWS -->|OK/EVENT/EOSE/COUNT| CLIENT
|
||||
```
|
||||
@@ -0,0 +1,379 @@
|
||||
# Plan: Remove All SQLite from lws-main Thread
|
||||
|
||||
## Problem Statement
|
||||
|
||||
After implementing the 4 offload fixes (v2.1.4), `lws-main` still consumes **53.87% avg CPU** (down from 68.71%). Perf profiling shows **100% of the top symbols are SQLite** — B-tree traversal, page cache, VFS reads. The worker threads (`db-read`, `db-write`, `event-worker`) are nearly idle at 0.0–0.1%.
|
||||
|
||||
The goal is to eliminate all synchronous SQLite access from `lws-main` so it only does: JSON parsing, WebSocket I/O, subscription matching (in-memory), and message queuing.
|
||||
|
||||
## Thread Architecture Change
|
||||
|
||||
Reduce from 7 threads to 4:
|
||||
|
||||
```
|
||||
Current: lws-main, db-read-1, db-read-2, db-read-3, db-read-4, db-write, event-worker
|
||||
Proposed: lws-main, db-read, db-write, event-worker
|
||||
```
|
||||
|
||||
The 4 reader threads are overkill — they average 0.07–0.10% CPU each. A single `db-read` thread is sufficient for the current workload. This can be made configurable later if needed.
|
||||
|
||||
## Remaining SQLite Calls on lws-main
|
||||
|
||||
Every synchronous SQLite call that currently runs on `lws-main`:
|
||||
|
||||
### Hot Path — Per-Event (every incoming EVENT message)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `event_id_exists_in_db()` | websockets.c:1324, 2134 | Every EVENT | ~10-50us per B-tree lookup |
|
||||
| `get_config_value("relay_pubkey")` | websockets.c:2050 | Every kind-14 EVENT | SQLite SELECT |
|
||||
| `get_config_value("admin_pubkey")` | websockets.c:2073 | Every kind-23456 EVENT | SQLite SELECT |
|
||||
|
||||
### Hot Path — Per-Connection (every new WebSocket connection)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `get_config_bool("trust_proxy_headers")` | websockets.c:1071 | Every connection | SQLite SELECT |
|
||||
| `get_config_bool("nip42_auth_required_events")` | websockets.c:1134 | Every connection | SQLite SELECT |
|
||||
| `get_config_bool("nip42_auth_required_subscriptions")` | websockets.c:1135 | Every connection | SQLite SELECT |
|
||||
|
||||
### Hot Path — Per-REQ (every subscription request)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `get_config_bool("expiration_enabled")` | main.c:1424 | Every REQ | SQLite SELECT |
|
||||
| `get_config_bool("expiration_filter")` | main.c:1425 | Every REQ | SQLite SELECT |
|
||||
| `check_database_auth_rules()` | websockets.c:1706 | Every REQ when WoT enabled | SQLite SELECT |
|
||||
|
||||
### Medium Path — Per-COUNT (NIP-45)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `thread_pool_execute_count_sync()` | websockets.c:3689 | Every COUNT message | Blocks main thread waiting for db-read |
|
||||
|
||||
### Low Path — Periodic (every 60s or on timer)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `refresh_hot_config_if_needed()` | websockets.c:228-247 | Every 5s | 8 SQLite SELECTs |
|
||||
| `generate_and_post_status_event()` | websockets.c:3213 | Every N hours | store_event + broadcast |
|
||||
| `ip_ban_cleanup/log_stats` | websockets.c:3004-3005 | Every 60s | SQLite reads/writes |
|
||||
|
||||
### Per-EVENT — Validator Path (called from event-worker AND sync path)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `get_config_value("admin_pubkey")` | request_validator.c:308 | Every kind-23456 EVENT | SQLite SELECT |
|
||||
| `get_config_value("nip42_auth_enabled")` | request_validator.c:326 | Every EVENT | SQLite SELECT |
|
||||
| `get_config_bool("pow_enabled")` | request_validator.c:370 | Every EVENT | SQLite SELECT |
|
||||
| `get_config_int("pow_min_difficulty")` | request_validator.c:371 | Every EVENT | SQLite SELECT |
|
||||
| `get_config_int("pow_validation_flags")` | request_validator.c:372 | Every EVENT | SQLite SELECT |
|
||||
| `get_config_int("nip40_expiration_grace_period")` | request_validator.c:432 | Every EVENT with expiration | SQLite SELECT |
|
||||
| `get_config_value("auth_enabled")` | request_validator.c:212 | Every EVENT (reload_auth_config) | SQLite SELECT |
|
||||
| `get_config_value("auth_rules_enabled")` | request_validator.c:220 | Every EVENT (reload_auth_config) | SQLite SELECT |
|
||||
| `db_count_active_whitelist_rules()` | request_validator.c:536 | Every 5s (cached) | Opens separate SQLite connection |
|
||||
|
||||
**Note:** These run on the **event-worker** thread for async-eligible events, but on **lws-main** for special kinds (14, 1059, 23456) that go through the sync validation path. They also run on lws-main for the `nostr_validate_unified_request()` call in the sync fallback.
|
||||
|
||||
### Per-EVENT — NIP-13 PoW Validation
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `get_config_bool("pow_enabled")` | nip013.c:27 | Every EVENT | SQLite SELECT |
|
||||
| `get_config_int("pow_min_difficulty")` | nip013.c:28 | Every EVENT | SQLite SELECT |
|
||||
| `get_config_value("pow_mode")` | nip013.c:29 | Every EVENT | SQLite SELECT |
|
||||
|
||||
### Per-Broadcast — NIP-40 Expiration Check
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `get_config_bool("expiration_enabled")` | subscriptions.c:764 | Every broadcast | SQLite SELECT |
|
||||
| `get_config_bool("expiration_filter")` | subscriptions.c:765 | Every broadcast | SQLite SELECT |
|
||||
|
||||
### Per-NIP-11 Request (HTTP, not WebSocket)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| 15+ `get_config_value()` calls | nip011.c:96-110 | Every NIP-11 HTTP request | 15 SQLite SELECTs |
|
||||
| 10+ `get_config_int()`/`get_config_bool()` calls | nip011.c:113-123 | Every NIP-11 HTTP request | 10 SQLite SELECTs |
|
||||
|
||||
### Per-NIP-42 Auth Event
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `get_config_value("relay_url")` | nip042.c:99 | Every AUTH event | SQLite SELECT |
|
||||
| `get_config_int("relay_port")` | nip042.c:103 | Every AUTH event (fallback) | SQLite SELECT |
|
||||
|
||||
### Per-NIP-40 Expiration Check (EVENT submission)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| 5 `get_config_bool()`/`get_config_int()` calls | nip040.c:37-41 | Every EVENT with expiration tag | 5 SQLite SELECTs |
|
||||
|
||||
### Subscription Lifecycle (per sub create/close/disconnect)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `db_log_subscription_created()` | subscriptions.c:1043 | Every REQ | SQLite INSERT |
|
||||
| `db_log_subscription_closed()` | subscriptions.c:1053 | Every CLOSE | SQLite INSERT + UPDATE |
|
||||
| `db_log_subscription_disconnected()` | subscriptions.c:1060 | Every disconnect | SQLite UPDATE + INSERT |
|
||||
| `db_update_subscription_events_sent()` | subscriptions.c:1088 | Every broadcast match | SQLite UPDATE |
|
||||
| `db_cleanup_orphaned_subscriptions()` | subscriptions.c:1100 | Startup only | SQLite DELETE |
|
||||
|
||||
### IP Ban System (per-connection and periodic)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `get_config_value("idle_ban_whitelist")` | ip_ban.c:254 | Every ban check | SQLite SELECT |
|
||||
| `get_config_bool("auth_fail_ban_enabled")` | ip_ban.c:301, 343 | Every ban check + failure | SQLite SELECT |
|
||||
| `get_config_bool("idle_ban_enabled")` | ip_ban.c:307, 394 | Every ban check + idle failure | SQLite SELECT |
|
||||
| 6+ `get_config_int()` calls | ip_ban.c:345-399, 464-465 | Every failure recording | SQLite SELECTs |
|
||||
| `db_prepare()`/`db_step_stmt()` for ip_bans | ip_ban.c:141-238 | Load at startup + periodic save | SQLite reads/writes |
|
||||
|
||||
### Special Kind Sync Path (kinds 14, 1059, 23456)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `store_event()` | websockets.c:1548-1629 | Every special-kind EVENT | Full sync store |
|
||||
| `process_admin_event_in_config()` | websockets.c:1468 | Every kind-23456 | Config DB writes |
|
||||
| `process_nip17_admin_message()` | websockets.c:1523 | Every kind-1059 when enabled | Decrypt + DB |
|
||||
| `process_dm_stats_command()` | websockets.c:3240 | Admin DM stats | Decrypt + query + encrypt |
|
||||
| `generate_stats_json()` | api.c:1167 (via dm_admin.c:805) | Admin stats command | Multiple SQLite queries |
|
||||
|
||||
### DM Admin Path (called from lws-main for kind 1059/14)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `get_config_value("relay_pubkey")` | dm_admin.c:358, 451 | Every admin DM | SQLite SELECT |
|
||||
| `get_config_value("admin_pubkey")` | dm_admin.c:504 | Every admin DM | SQLite SELECT |
|
||||
| `get_config_int("nip59_timestamp_max_delay_sec")` | dm_admin.c:373 | Every admin DM | SQLite SELECT |
|
||||
| `get_config_int("wot_enabled")` | dm_admin.c:613, 640, 660 | WoT commands | SQLite SELECT |
|
||||
|
||||
### API/Monitoring (called from lws-main periodic timer)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `generate_stats_json()` | api.c:1167 | Status post + admin stats | ~10 SQLite queries |
|
||||
| `query_time_based_statistics()` | api.c:102 | Monitoring events | SQLite queries |
|
||||
| `query_subscription_details()` | api.c:197 | Monitoring events | SQLite queries |
|
||||
| `get_config_value("relay_pubkey")` | api.c:419, 821, 1291 | Every monitoring event | SQLite SELECT |
|
||||
| `get_config_int("kind_1_status_posts_hours")` | api.c:547 | Status post check | SQLite SELECT |
|
||||
| `get_config_int("kind_24567_reporting_throttle_sec")` | api.c:60 | Monitoring throttle | SQLite SELECT |
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Fix 1: Create Global Config Cache — Eliminate ALL get_config_* SQLite Calls
|
||||
|
||||
**Impact: Very High** — eliminates 50+ distinct `get_config_value/int/bool` call sites across 8 source files
|
||||
|
||||
The audit found `get_config_value/int/bool` calls in: `websockets.c`, `main.c`, `subscriptions.c`, `request_validator.c`, `nip011.c`, `nip013.c`, `nip040.c`, `nip042.c`, `ip_ban.c`, `dm_admin.c`, `api.c`, `config.c`. Every single one does a `SELECT value FROM config WHERE key=?` SQLite query.
|
||||
|
||||
**Approach:** Replace `get_config_value_from_table()` with an in-memory hash map that is loaded once at startup and refreshed periodically (every 5s) or on admin config change events.
|
||||
|
||||
**Changes:**
|
||||
|
||||
1. **New: `config_cache` in config.c** — A simple hash map or fixed array of key-value pairs loaded from the config table:
|
||||
```c
|
||||
typedef struct {
|
||||
char key[64];
|
||||
char value[512];
|
||||
} config_cache_entry_t;
|
||||
|
||||
static config_cache_entry_t g_config_cache[256];
|
||||
static int g_config_cache_count = 0;
|
||||
static time_t g_config_cache_last_refresh = 0;
|
||||
static pthread_mutex_t g_config_cache_mutex;
|
||||
#define CONFIG_CACHE_TTL_SEC 5
|
||||
```
|
||||
|
||||
2. **Modify `get_config_value_from_table()`** — Instead of querying SQLite, look up in the in-memory cache. If cache is stale, refresh from SQLite (but only once per TTL period, not per call).
|
||||
|
||||
3. **Add `config_cache_refresh()`** — Loads all rows from `SELECT key, value FROM config` into the hash map. Called:
|
||||
- Once at startup after config table is populated
|
||||
- Every 5 seconds from the existing `refresh_hot_config_if_needed()` timer
|
||||
- Immediately after `process_admin_event_in_config()` modifies config
|
||||
|
||||
4. **Thread safety:** The cache is read from lws-main and event-worker threads. Use a read-write lock or double-buffer pattern. Since config changes are rare, a simple mutex with short hold time is fine.
|
||||
|
||||
5. **Remove the existing `hot_config_cache_t`** in websockets.c — it becomes redundant since the global config cache serves the same purpose with broader coverage.
|
||||
|
||||
This single change eliminates SQLite calls from:
|
||||
- Per-connection: `trust_proxy_headers`, `nip42_auth_required_*` (websockets.c)
|
||||
- Per-REQ: `expiration_enabled`, `expiration_filter` (main.c)
|
||||
- Per-EVENT: `admin_pubkey`, `relay_pubkey`, `nip42_auth_enabled`, `pow_*` (request_validator.c, nip013.c)
|
||||
- Per-broadcast: `expiration_enabled`, `expiration_filter` (subscriptions.c)
|
||||
- Per-NIP-11: 25+ config values (nip011.c)
|
||||
- Per-NIP-42: `relay_url`, `relay_port` (nip042.c)
|
||||
- Per-NIP-40: 5 expiration config values (nip040.c)
|
||||
- Per-IP-ban: `idle_ban_*`, `auth_fail_*` config values (ip_ban.c)
|
||||
- Per-admin-DM: `relay_pubkey`, `admin_pubkey`, `wot_enabled` (dm_admin.c)
|
||||
- Per-monitoring: `relay_pubkey`, throttle values (api.c)
|
||||
|
||||
### Fix 2: Move Duplicate Check to Event Worker Thread
|
||||
|
||||
**Impact: High** — eliminates the most frequent per-EVENT SQLite call from lws-main
|
||||
|
||||
Currently `event_id_exists_in_db()` runs on lws-main before async submission. Move it into the `async_event_worker_main()` function, which already has its own SQLite connection.
|
||||
|
||||
**Changes:**
|
||||
- Remove `event_id_exists_in_db()` calls from both EVENT paths in websockets.c (lines 1324 and 2134)
|
||||
- Add the duplicate check at the start of `async_event_worker_main()` before `nostr_validate_unified_request()`
|
||||
- If duplicate found, set `completion->success = 1` and `completion->error_message = "duplicate: already have this event"` and skip crypto
|
||||
- The completion handler already sends OK responses, so this works seamlessly
|
||||
|
||||
**Trade-off:** Without the early dedup check on lws-main, duplicate events will be submitted to the event worker queue and consume a queue slot + thread wakeup before being detected. This is acceptable because:
|
||||
- The worker thread dedup check is still fast (B-tree index lookup)
|
||||
- It avoids the much larger cost of running SQLite on lws-main for every event
|
||||
- Queue depth is 4096, so even under heavy duplicate traffic this won't overflow
|
||||
|
||||
### Fix 3: Make COUNT Queries Async
|
||||
|
||||
**Impact: Medium** — eliminates blocking wait on lws-main for NIP-45 COUNT
|
||||
|
||||
Currently `handle_count_message()` calls `thread_pool_execute_count_sync()` which blocks lws-main waiting for the db-read worker. Convert to the same async pattern used for REQ queries.
|
||||
|
||||
**Changes:**
|
||||
- Create `count_async_state_t` similar to `req_async_state_t`
|
||||
- Create `count_async_completion_t` with count result
|
||||
- Add `submit_count_query_async()` that uses `thread_pool_submit_read()` with a callback
|
||||
- The callback pushes to a count completion queue
|
||||
- Add `process_count_async_completions()` to drain the queue on lws-main and send COUNT responses
|
||||
- Call it from the main event loop alongside `process_req_async_completions()`
|
||||
|
||||
### Fix 4: Cache Auth Rules Check Result
|
||||
|
||||
**Impact: Low-Medium** — eliminates per-REQ SQLite when WoT is enabled
|
||||
|
||||
`check_database_auth_rules()` at websockets.c:1706 runs a SQLite query for every REQ when `wot_enabled == 2`. Cache the result per-session.
|
||||
|
||||
**Changes:**
|
||||
- Add `wot_checked` and `wot_allowed` fields to `per_session_data`
|
||||
- On first REQ, call `check_database_auth_rules()` and cache the result
|
||||
- On subsequent REQs from the same session, use the cached value
|
||||
- Reset cache when auth state changes (e.g., after NIP-42 AUTH)
|
||||
|
||||
### Fix 5: Move Special-Kind EVENT Store to Event Worker
|
||||
|
||||
**Impact: Low** — special kinds are rare but currently do full sync store on lws-main
|
||||
|
||||
Kinds 14, 1059, and 23456 are currently excluded from async processing because they have special handling. However, the `store_event()` call within those paths still blocks lws-main.
|
||||
|
||||
**Changes:**
|
||||
- For kind 1059 and kind 14: after special processing completes, submit the store to the event worker instead of calling `store_event()` synchronously
|
||||
- For kind 23456: admin events are not stored (processed by admin API), so no change needed
|
||||
- This requires the special processing (decryption, admin check) to still happen on lws-main (it needs `pss` context), but the final DB write goes async
|
||||
|
||||
### Fix 6: Reduce Reader Threads from 4 to 1
|
||||
|
||||
**Impact: Resource savings** — 3 fewer threads, 3 fewer SQLite connections
|
||||
|
||||
**Changes:**
|
||||
- Change default `reader_threads` in `thread_pool_init()` from 4 to 1
|
||||
- Or make it configurable via config with default 1
|
||||
- The single `db-read` thread handles all REQ and COUNT queries sequentially
|
||||
- If future profiling shows the read thread becoming a bottleneck, increase back to 2+
|
||||
|
||||
### Fix 7: Move Subscription Logging Off Main Thread
|
||||
|
||||
**Impact: Medium** — eliminates per-REQ INSERT, per-CLOSE UPDATE, per-broadcast UPDATE
|
||||
|
||||
The subscription lifecycle functions in subscriptions.c do synchronous SQLite writes on lws-main:
|
||||
- `db_log_subscription_created()` — INSERT on every REQ
|
||||
- `db_log_subscription_closed()` — INSERT + UPDATE on every CLOSE
|
||||
- `db_log_subscription_disconnected()` — UPDATE + INSERT on every disconnect
|
||||
- `db_update_subscription_events_sent()` — UPDATE on every broadcast match
|
||||
|
||||
**Changes:**
|
||||
- Submit these as fire-and-forget write jobs to the db-write thread via `thread_pool_submit_write()`
|
||||
- Create a new job type `THREAD_POOL_JOB_SUBSCRIPTION_LOG` or simply use a generic callback job
|
||||
- The main thread doesn't need to wait for these to complete — they're purely observational logging
|
||||
|
||||
### Fix 8: Move Periodic Tasks Off Main Thread
|
||||
|
||||
**Impact: Low** — these run every 60s but can still cause latency spikes
|
||||
|
||||
- `generate_and_post_status_event()`: Submit as an async job to the event worker
|
||||
- `ip_ban_cleanup()`/`ip_ban_log_stats()`/`ip_ban_save()`: These do SQLite reads/writes. Move to a periodic job on the db-write thread
|
||||
- Config cache refresh (Fix 1) replaces the old `refresh_hot_config_if_needed()` SQLite queries
|
||||
|
||||
### Fix 9: Cache NIP-11 Response
|
||||
|
||||
**Impact: Low-Medium** — eliminates 25+ SQLite SELECTs per NIP-11 HTTP request
|
||||
|
||||
`generate_relay_info_json()` in nip011.c calls `get_config_value()` 15 times and `get_config_int()`/`get_config_bool()` 10 times. After Fix 1 (global config cache), these become in-memory lookups. But we can go further:
|
||||
|
||||
**Changes:**
|
||||
- Cache the serialized NIP-11 JSON response string with a TTL (e.g., 60 seconds)
|
||||
- On NIP-11 request, return the cached string directly without rebuilding
|
||||
- Invalidate on admin config change events
|
||||
|
||||
## Implementation Order
|
||||
|
||||
| # | Fix | Risk | Complexity |
|
||||
|---|-----|------|------------|
|
||||
| 1 | Global config cache (replaces all get_config_* SQLite) | Low | Medium |
|
||||
| 2 | Move dedup check to event worker | Low | Low |
|
||||
| 3 | Async COUNT queries | Low | Medium |
|
||||
| 4 | Cache WoT auth rules per-session | Very Low | Low |
|
||||
| 5 | Async store for special kinds | Medium | Medium |
|
||||
| 6 | Reduce reader threads to 1 | Very Low | Trivial |
|
||||
| 7 | Move subscription logging off main thread | Low | Low-Medium |
|
||||
| 8 | Move periodic tasks off main thread | Low | Medium |
|
||||
| 9 | Cache NIP-11 response | Very Low | Low |
|
||||
|
||||
## Expected Impact
|
||||
|
||||
After all 9 fixes, `lws-main` should have **zero direct SQLite calls**. Its work becomes:
|
||||
- JSON parse incoming messages (~fast)
|
||||
- In-memory subscription matching (~fast)
|
||||
- In-memory config lookups (~fast, hash map)
|
||||
- Message formatting and queuing (~fast)
|
||||
- `lws_service()` I/O (~fast)
|
||||
- Draining completion queues (~fast)
|
||||
|
||||
### Impact by fix
|
||||
|
||||
| Fix | SQLite calls eliminated | Frequency |
|
||||
|-----|------------------------|-----------|
|
||||
| Fix 1: Global config cache | ~50+ `get_config_*` call sites across 8 files | Per-event, per-connection, per-REQ, per-broadcast, per-NIP-11 |
|
||||
| Fix 2: Dedup to event worker | 2 `event_id_exists_in_db()` calls | Per-EVENT |
|
||||
| Fix 3: Async COUNT | 1 `thread_pool_execute_count_sync()` | Per-COUNT message |
|
||||
| Fix 4: WoT cache per-session | 1 `check_database_auth_rules()` | Per-REQ when WoT enabled |
|
||||
| Fix 5: Async special-kind store | ~6 `store_event()` calls | Per special-kind EVENT |
|
||||
| Fix 6: Reduce readers | N/A (resource savings) | N/A |
|
||||
| Fix 7: Async subscription logging | 4 `db_log_*`/`db_update_*` calls | Per-REQ, per-CLOSE, per-broadcast |
|
||||
| Fix 8: Async periodic tasks | `generate_stats_json()`, `ip_ban_*` | Every 60s |
|
||||
| Fix 9: Cache NIP-11 | 25+ config lookups per request | Per NIP-11 HTTP request |
|
||||
|
||||
Estimated main-thread CPU reduction: from **54% to ~5-10%** (mostly WebSocket I/O, subscription matching, and JSON parsing).
|
||||
|
||||
## Thread Model After Changes
|
||||
|
||||
```
|
||||
lws-main — WebSocket I/O, JSON parse, subscription match, message queue
|
||||
NO SQLite. Pure compute + I/O.
|
||||
|
||||
db-read — All SELECT queries: REQ results, COUNT results, config refresh,
|
||||
auth rule checks, event dedup checks via completion queue
|
||||
|
||||
db-write — All INSERT/UPDATE: event storage, tag inserts, IP ban persistence,
|
||||
status event generation
|
||||
|
||||
event-worker — EVENT validation: signature verification, then submits store to
|
||||
db-write via completion queue, results flow back to lws-main
|
||||
```
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
CLIENT[Nostr Clients] --> LWS[lws-main<br/>WebSocket I/O<br/>JSON parse<br/>Sub matching<br/>NO SQLite]
|
||||
|
||||
LWS -->|EVENT submit| EW[event-worker<br/>Sig verify<br/>Dedup check]
|
||||
EW -->|store job| DW[db-write<br/>INSERT events<br/>INSERT tags<br/>IP ban save]
|
||||
EW -->|completion| LWS
|
||||
DW -->|completion| LWS
|
||||
|
||||
LWS -->|REQ/COUNT submit| DR[db-read<br/>SELECT queries<br/>Config refresh<br/>Auth checks]
|
||||
DR -->|completion| LWS
|
||||
|
||||
LWS -->|OK/EVENT/EOSE| CLIENT
|
||||
```
|
||||
@@ -0,0 +1,107 @@
|
||||
# WAL Checkpoint Fix — db-write Thread CPU Plan
|
||||
|
||||
## Problem
|
||||
|
||||
Fresh profiling of the production server shows the `db-write` thread consuming **77% average CPU** (peaking at 100%) while `lws-main` sits at 0.3%.
|
||||
|
||||
The `perf` callgraph confirms 100% of the hot symbols are SQLite **B-tree read operations** running on the `db-write` thread:
|
||||
|
||||
| % CPU | Symbol | Role |
|
||||
|-------|--------|------|
|
||||
| 13.88 | `sqlite3BtreeTableMoveto` | B-tree seek |
|
||||
| 10.05 | `sqlite3VdbeExec` | VM execution |
|
||||
| 7.77 | `rep_movs_alternative` | kernel memcpy in pread |
|
||||
| 7.76 | `sqlite3GetVarint` | varint decode |
|
||||
| 6.66 | `pcache1Fetch` | page cache lookup |
|
||||
| 6.02 | `memcpy` | data copy |
|
||||
|
||||
These are **read-path** functions, not write-path. The `event-worker` thread accumulated only 3 CPU ticks over 5 minutes, meaning very few events are actually being stored.
|
||||
|
||||
## Root Cause
|
||||
|
||||
SQLite WAL auto-checkpoint. By default SQLite triggers a checkpoint every 1000 WAL pages. The checkpoint is executed by the **next writer** — which is always the `db-write` thread since all writes are funnelled through it.
|
||||
|
||||
A WAL checkpoint reads every dirty page from the WAL file and writes it back to the main database file. This involves B-tree traversals to locate pages, which explains the `BtreeTableMoveto` dominance.
|
||||
|
||||
The current `db_open_worker_connection` in `src/db_ops.c` sets `PRAGMA journal_mode=WAL` and `busy_timeout=5000` but does **not** configure `wal_autocheckpoint`. SQLite's default of 1000 pages applies.
|
||||
|
||||
In earlier profiling runs the same CPU burn appeared on `lws-main` at 55-67% — because that thread was doing the writes directly before the thread-pool refactoring. The work simply moved threads; the total cost is unchanged.
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Step 1 — Disable auto-checkpoint on the write connection
|
||||
|
||||
In `db_open_worker_connection` add:
|
||||
|
||||
```c
|
||||
sqlite3_exec(db, "PRAGMA wal_autocheckpoint=0;", NULL, NULL, NULL);
|
||||
```
|
||||
|
||||
This prevents the write thread from ever running a checkpoint inline with normal writes.
|
||||
|
||||
### Step 2 — Add a periodic checkpoint job type to the thread pool
|
||||
|
||||
Add a new job type `THREAD_POOL_JOB_WAL_CHECKPOINT` to `thread_pool.h`.
|
||||
|
||||
The handler in `thread_pool.c` calls:
|
||||
|
||||
```c
|
||||
sqlite3_wal_checkpoint_v2(db, NULL, SQLITE_CHECKPOINT_PASSIVE, NULL, NULL);
|
||||
```
|
||||
|
||||
`PASSIVE` mode checkpoints only pages that are not currently being read, so it never blocks readers.
|
||||
|
||||
### Step 3 — Submit the checkpoint job on a timer
|
||||
|
||||
In the existing 60-second periodic timer in `websockets.c`, add a call to submit a WAL checkpoint job to the write queue:
|
||||
|
||||
```c
|
||||
thread_pool_submit_wal_checkpoint();
|
||||
```
|
||||
|
||||
This runs the checkpoint once per minute on the write thread, but as a **bounded, predictable** operation rather than being triggered unpredictably by every INSERT.
|
||||
|
||||
### Step 4 — Also disable auto-checkpoint on reader connections
|
||||
|
||||
In `db_open_worker_connection`, the same `wal_autocheckpoint=0` applies to reader connections too. Readers can also trigger checkpoints in SQLite; disabling it on all connections ensures only the explicit periodic job does checkpointing.
|
||||
|
||||
### Step 5 — Run a TRUNCATE checkpoint at shutdown
|
||||
|
||||
In `thread_pool_shutdown`, before closing the write connection, run:
|
||||
|
||||
```c
|
||||
sqlite3_wal_checkpoint_v2(db, NULL, SQLITE_CHECKPOINT_TRUNCATE, NULL, NULL);
|
||||
```
|
||||
|
||||
This ensures the WAL is fully flushed and truncated on clean shutdown, keeping the database file compact.
|
||||
|
||||
## Files Changed
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `src/db_ops.c` | Add `PRAGMA wal_autocheckpoint=0` in `db_open_worker_connection` |
|
||||
| `src/thread_pool.h` | Add `THREAD_POOL_JOB_WAL_CHECKPOINT` enum value |
|
||||
| `src/thread_pool.c` | Add `execute_wal_checkpoint_job` handler; add checkpoint-at-shutdown in `thread_pool_shutdown`; add `thread_pool_submit_wal_checkpoint` helper |
|
||||
| `src/websockets.c` | Call `thread_pool_submit_wal_checkpoint()` in the 60-second periodic timer |
|
||||
|
||||
## Expected Impact
|
||||
|
||||
- `db-write` CPU should drop from ~77% to near 0% when idle (no events to store)
|
||||
- Periodic checkpoint bursts of a few seconds every 60s instead of continuous burn
|
||||
- No change to data durability — WAL still protects against crashes; checkpoint just moves data from WAL to main DB file
|
||||
- Readers are unaffected — PASSIVE checkpoint never blocks them
|
||||
|
||||
## Risk
|
||||
|
||||
- If the relay crashes between checkpoints, the WAL file may be larger than before (up to 60s of accumulated writes). This is safe — SQLite replays the WAL on next open. The WAL file size is bounded by the write rate, which is low.
|
||||
- PASSIVE checkpoint may not checkpoint all pages if readers hold them. This is fine — the next checkpoint will catch up.
|
||||
|
||||
## Verification
|
||||
|
||||
After deploying, re-run the profiler:
|
||||
|
||||
```bash
|
||||
./deploy_lt_debug.sh && DURATION=300 INTERVAL=5 ./tests/thread_cpu_profile.sh
|
||||
```
|
||||
|
||||
Expected: `db-write` avg CPU drops below 5%.
|
||||
@@ -0,0 +1,298 @@
|
||||
# Web of Trust (WoT) Implementation Plan
|
||||
|
||||
## Feature Description
|
||||
|
||||
When enabled, the relay restricts access to pubkeys that the admin follows. The admin's kind 3 (contact list) event is used as the source of truth — all `p` tags in that event become whitelisted pubkeys. The admin themselves is always whitelisted.
|
||||
|
||||
Single config variable `wot_enabled` with three levels:
|
||||
|
||||
| Value | Mode | Effect |
|
||||
|-------|------|--------|
|
||||
| `0` | Off | Open relay — anyone can read and write |
|
||||
| `1` | Write-only | Only followed pubkeys can **publish** events. Anyone can read/subscribe. |
|
||||
| `2` | Full | Only followed pubkeys can **publish AND subscribe**. Requires NIP-42 auth for subscriptions. Eliminates subscription churn from anonymous connections. |
|
||||
|
||||
## How It Works
|
||||
|
||||
### WoT Sync Flow
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["Admin publishes kind 3 event\n- contact list -"] --> B{wot_enabled > 0?}
|
||||
B -->|No| C["Store event normally"]
|
||||
B -->|Yes| D["Extract p tags from kind 3"]
|
||||
D --> E["Clear existing WoT whitelist rules"]
|
||||
E --> F["Insert new whitelist rules\nfor each followed pubkey"]
|
||||
F --> G["Enable auth_enabled"]
|
||||
G --> H{wot_enabled == 2?}
|
||||
H -->|Yes| I["Enable nip42_auth_required_subscriptions"]
|
||||
H -->|No| J["Done - write-only restriction"]
|
||||
```
|
||||
|
||||
### Event Publishing Flow — Write Restriction (wot_enabled >= 1)
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["EVENT message arrives"] --> B{auth_enabled?}
|
||||
B -->|No| C["Accept event"]
|
||||
B -->|Yes| D["check_database_auth_rules - pubkey -"]
|
||||
D --> E{Pubkey in whitelist\nor wot_whitelist?}
|
||||
E -->|Yes| C
|
||||
E -->|No| F{Any whitelist rules exist?}
|
||||
F -->|Yes| G["REJECT: not whitelisted"]
|
||||
F -->|No| C
|
||||
```
|
||||
|
||||
### Subscription Flow — Read Restriction (wot_enabled == 2)
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["REQ message arrives"] --> B{wot_enabled == 2?}
|
||||
B -->|No| C["Allow subscription"]
|
||||
B -->|Yes| D{NIP-42 authenticated?}
|
||||
D -->|No| E["Send AUTH challenge"]
|
||||
D -->|Yes| F["check_database_auth_rules\nusing authenticated_pubkey"]
|
||||
F --> G{Pubkey in whitelist\nor wot_whitelist?}
|
||||
G -->|Yes| C
|
||||
G -->|No| H["REJECT: not authorized\nfor subscriptions"]
|
||||
```
|
||||
|
||||
## Design Decisions
|
||||
|
||||
### Leveraging Existing Infrastructure
|
||||
|
||||
The relay already has everything needed:
|
||||
|
||||
1. **`auth_rules` table** — stores whitelist/blacklist rules with `rule_type`, `pattern_type`, `pattern_value`
|
||||
2. **`check_database_auth_rules()`** in [`request_validator.c:529`](src/request_validator.c:529) — already implements the logic: "if whitelist rules exist and pubkey is not whitelisted, deny"
|
||||
3. **`add_auth_rule_from_config()`** / **`remove_auth_rule_from_config()`** in [`config.c`](src/config.c:2082) — already manage auth rules
|
||||
4. **`event_tags` table** — can efficiently query `p` tags from kind 3 events
|
||||
5. **Config system** — `auth_enabled` flag already controls whether auth rules are checked
|
||||
6. **NIP-42 auth** — `nip42_auth_required_subscriptions` already gates REQ access, `pss->authenticated_pubkey` stores the authenticated pubkey
|
||||
|
||||
### WoT-Specific Whitelist Rules
|
||||
|
||||
To distinguish WoT-generated whitelist rules from manually-added ones, we use a new `rule_type` value: `wot_whitelist`. This allows:
|
||||
- Clearing all WoT rules without affecting manual whitelist/blacklist rules
|
||||
- Querying WoT status separately
|
||||
- The existing `check_database_auth_rules()` function already checks for `rule_type = 'whitelist'` — we need to also match `wot_whitelist` in the whitelist check
|
||||
|
||||
### Trigger Mechanism
|
||||
|
||||
The WoT sync happens when:
|
||||
1. **A kind 3 event from the admin is stored** — detected in `store_event()` after successful INSERT
|
||||
2. **Startup** — if `wot_enabled > 0`, sync from the most recent admin kind 3 event in the database
|
||||
3. **Admin DM command** — `wot sync` to force a manual resync
|
||||
|
||||
### What Gets Whitelisted
|
||||
|
||||
- All pubkeys in `p` tags of the admin's kind 3 event
|
||||
- The admin pubkey itself (always whitelisted)
|
||||
- The relay pubkey (always whitelisted — for admin DM responses)
|
||||
|
||||
### What Is NOT Blocked
|
||||
|
||||
Even with WoT enabled, these are always allowed:
|
||||
- Admin events (kind 23456) — already bypassed in [`request_validator.c:303`](src/request_validator.c:303)
|
||||
- NIP-42 auth events (kind 22242) — already bypassed in [`request_validator.c:318`](src/request_validator.c:318)
|
||||
- Kind 3 events from the admin — needed to update the follow list itself
|
||||
- Kind 1059 gift wraps addressed to the relay — needed for admin DMs
|
||||
|
||||
## Files to Modify
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `src/default_config_event.h` | Add `wot_enabled` config key (default: `0`) |
|
||||
| `src/config.c` | Add `wot_sync_from_admin_kind3()` function |
|
||||
| `src/main.c` | Add WoT trigger in `store_event()` when admin kind 3 is stored, add startup WoT sync |
|
||||
| `src/request_validator.c` | Update whitelist SQL to also match `wot_whitelist` rule type |
|
||||
| `src/sql_schema.h` | Update `auth_rules` CHECK constraint to include `wot_whitelist` |
|
||||
| `src/websockets.c` | Add WoT pubkey check after NIP-42 auth check in REQ handler |
|
||||
| `src/dm_admin.c` | Add `wot 0`, `wot 1`, `wot 2`, `wot sync`, `wot status` DM commands |
|
||||
|
||||
## Detailed Changes
|
||||
|
||||
### 1. Schema: `src/sql_schema.h`
|
||||
|
||||
Update the `auth_rules` table CHECK constraint to allow `wot_whitelist`:
|
||||
|
||||
```sql
|
||||
-- Before:
|
||||
rule_type TEXT NOT NULL CHECK (rule_type IN ('whitelist', 'blacklist', 'rate_limit', 'auth_required'))
|
||||
|
||||
-- After:
|
||||
rule_type TEXT NOT NULL CHECK (rule_type IN ('whitelist', 'blacklist', 'rate_limit', 'auth_required', 'wot_whitelist'))
|
||||
```
|
||||
|
||||
### 2. Config: `src/default_config_event.h`
|
||||
|
||||
Add WoT configuration:
|
||||
|
||||
```c
|
||||
// Web of Trust Settings
|
||||
// 0 = off, 1 = write-only (followed pubkeys can publish), 2 = full (followed pubkeys can publish AND subscribe)
|
||||
{"wot_enabled", "0"},
|
||||
```
|
||||
|
||||
### 3. Core WoT Sync Function: `src/config.c`
|
||||
|
||||
New function `wot_sync_from_admin_kind3()`:
|
||||
|
||||
```c
|
||||
int wot_sync_from_admin_kind3(void) {
|
||||
int wot_level = get_config_int("wot_enabled", 0);
|
||||
if (wot_level <= 0) return 0; // WoT disabled
|
||||
|
||||
// 1. Get admin pubkey from config
|
||||
// 2. Query event_tags for p tags from admin's latest kind 3 event:
|
||||
// SELECT DISTINCT et.tag_value FROM event_tags et
|
||||
// JOIN events e ON et.event_id = e.id
|
||||
// WHERE e.kind = 3 AND e.pubkey = ? AND et.tag_name = 'p'
|
||||
// ORDER BY e.created_at DESC
|
||||
// 3. BEGIN TRANSACTION
|
||||
// 4. DELETE FROM auth_rules WHERE rule_type = 'wot_whitelist'
|
||||
// 5. INSERT wot_whitelist for admin pubkey
|
||||
// 6. INSERT wot_whitelist for relay pubkey
|
||||
// 7. For each p tag: INSERT INTO auth_rules (rule_type, pattern_type, pattern_value)
|
||||
// VALUES ('wot_whitelist', 'pubkey', ?)
|
||||
// 8. COMMIT
|
||||
// 9. Set auth_enabled = true
|
||||
// 10. If wot_level == 2: set nip42_auth_required_subscriptions = true
|
||||
// 11. Log count of whitelisted pubkeys
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Trigger on Kind 3 Store: `src/main.c`
|
||||
|
||||
In `store_event()`, after successful INSERT and after `store_event_tags()`:
|
||||
|
||||
```c
|
||||
// Check if this is a kind 3 event from the admin — trigger WoT sync
|
||||
if ((int)cJSON_GetNumberValue(kind) == 3) {
|
||||
int wot_level = get_config_int("wot_enabled", 0);
|
||||
if (wot_level > 0) {
|
||||
const char* admin_pubkey = get_config_value("admin_pubkey");
|
||||
if (admin_pubkey && strcmp(cJSON_GetStringValue(pubkey), admin_pubkey) == 0) {
|
||||
DEBUG_INFO("Admin kind 3 event stored — triggering WoT sync");
|
||||
wot_sync_from_admin_kind3();
|
||||
}
|
||||
if (admin_pubkey) free((char*)admin_pubkey);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Startup WoT Sync: `src/main.c`
|
||||
|
||||
In `main()`, after `populate_event_tags_from_existing()`:
|
||||
|
||||
```c
|
||||
// Sync Web of Trust whitelist if enabled
|
||||
int wot_level = get_config_int("wot_enabled", 0);
|
||||
if (wot_level > 0) {
|
||||
wot_sync_from_admin_kind3();
|
||||
}
|
||||
```
|
||||
|
||||
### 6. Update Whitelist Check: `src/request_validator.c`
|
||||
|
||||
Update the whitelist SQL queries to also match `wot_whitelist`:
|
||||
|
||||
```c
|
||||
// Before:
|
||||
"SELECT rule_type FROM auth_rules WHERE rule_type = 'whitelist' AND pattern_type = 'pubkey' AND pattern_value = ? AND active = 1 LIMIT 1"
|
||||
|
||||
// After:
|
||||
"SELECT rule_type FROM auth_rules WHERE rule_type IN ('whitelist', 'wot_whitelist') AND pattern_type = 'pubkey' AND pattern_value = ? AND active = 1 LIMIT 1"
|
||||
```
|
||||
|
||||
And the whitelist-exists check:
|
||||
```c
|
||||
// Before:
|
||||
"SELECT COUNT(*) FROM auth_rules WHERE rule_type = 'whitelist' AND pattern_type = 'pubkey' AND active = 1 LIMIT 1"
|
||||
|
||||
// After:
|
||||
"SELECT COUNT(*) FROM auth_rules WHERE rule_type IN ('whitelist', 'wot_whitelist') AND pattern_type = 'pubkey' AND active = 1 LIMIT 1"
|
||||
```
|
||||
|
||||
### 7. REQ Handler WoT Check: `src/websockets.c`
|
||||
|
||||
When `wot_enabled == 2`, add a pubkey whitelist check **after** the existing NIP-42 auth check in the REQ handler. The existing code at line 903 already requires NIP-42 auth when `nip42_auth_required_subscriptions` is set. We add a WoT check right after:
|
||||
|
||||
```c
|
||||
// Existing NIP-42 auth check (line 903):
|
||||
if (pss && pss->nip42_auth_required_subscriptions && !pss->authenticated) {
|
||||
// ... send AUTH challenge or NOTICE ...
|
||||
return 0;
|
||||
}
|
||||
|
||||
// NEW: WoT read restriction check (wot_enabled == 2)
|
||||
if (pss && pss->authenticated && get_config_int("wot_enabled", 0) == 2) {
|
||||
// Client is authenticated — check if their pubkey is in the WoT whitelist
|
||||
int wot_result = check_database_auth_rules(pss->authenticated_pubkey, "subscription", NULL);
|
||||
if (wot_result != NOSTR_SUCCESS) {
|
||||
send_notice_message(wsi, pss, "restricted: your pubkey is not in this relay's web of trust");
|
||||
DEBUG_INFO("REQ rejected: pubkey %s not in WoT whitelist", pss->authenticated_pubkey);
|
||||
cJSON_Delete(json);
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 8. Admin DM Commands: `src/dm_admin.c`
|
||||
|
||||
Add plain text DM commands:
|
||||
|
||||
| Command | Action |
|
||||
|---------|--------|
|
||||
| `wot 0` or `wot off` | Disable WoT, delete all `wot_whitelist` rules, reset `nip42_auth_required_subscriptions` |
|
||||
| `wot 1` or `wot write` | Write-only WoT — sync follow list, set `auth_enabled=true` |
|
||||
| `wot 2` or `wot full` | Full WoT — sync follow list, set `auth_enabled=true`, set `nip42_auth_required_subscriptions=true` |
|
||||
| `wot sync` | Force resync from admin's kind 3 event |
|
||||
| `wot status` | Show WoT level (0/1/2), count of whitelisted pubkeys |
|
||||
|
||||
### 9. NIP-11 Update
|
||||
|
||||
When WoT is enabled (level 1 or 2), the relay should indicate this in NIP-11 relay info. Add to the `limitation` object:
|
||||
```json
|
||||
"auth_required": true
|
||||
```
|
||||
|
||||
## Edge Cases
|
||||
|
||||
1. **Admin has no kind 3 event in database**: WoT sync does nothing, logs a warning. No whitelist rules created = open relay.
|
||||
|
||||
2. **Admin updates follow list**: New kind 3 event triggers full resync — old WoT rules are cleared, new ones inserted. This is atomic (transaction).
|
||||
|
||||
3. **WoT disabled (set to 0)**: Clears all `wot_whitelist` rules. Manual `whitelist` rules are preserved. `nip42_auth_required_subscriptions` is reset to false.
|
||||
|
||||
4. **Admin unfollows someone**: Next kind 3 event triggers resync, the unfollowed pubkey's `wot_whitelist` rule is removed (full clear + reinsert).
|
||||
|
||||
5. **Kind 1059 gift wraps**: These need special handling — the relay needs to accept gift wraps addressed to it even from non-whitelisted pubkeys (for admin DMs). The `is_nip17_gift_wrap_for_relay()` check should bypass WoT.
|
||||
|
||||
6. **Read restriction without NIP-42 support (level 2)**: If a client doesn't support NIP-42, they cannot authenticate and therefore cannot subscribe. This is by design — it's the most effective way to reduce subscription churn from anonymous connections.
|
||||
|
||||
7. **NIP-11 discovery**: Clients can check NIP-11 to see if `auth_required` is true before connecting, avoiding wasted connections.
|
||||
|
||||
8. **Changing level from 2 to 1**: `nip42_auth_required_subscriptions` is reset to false, allowing anonymous subscriptions again. WoT whitelist rules remain for write restriction.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
1. Enable write-only WoT via DM: `wot 1`
|
||||
2. Verify admin can still publish events
|
||||
3. Verify followed pubkeys can publish events
|
||||
4. Verify non-followed pubkeys are rejected for EVENT
|
||||
5. Verify non-followed pubkeys can still subscribe (REQ)
|
||||
6. Enable full WoT via DM: `wot 2`
|
||||
7. Verify unauthenticated clients get AUTH challenge on REQ
|
||||
8. Verify authenticated non-followed pubkeys are rejected for REQ
|
||||
9. Verify authenticated followed pubkeys can subscribe
|
||||
10. Verify `wot status` shows correct level and count
|
||||
11. Publish new kind 3 event, verify auto-resync
|
||||
12. `wot 0` — verify all pubkeys can publish and subscribe again
|
||||
13. Verify admin DMs still work when WoT is enabled
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
- WoT sync is O(n) where n = number of follows (typically 100-2000)
|
||||
- Uses a transaction for bulk insert — fast even for large follow lists
|
||||
- Auth rule check is already indexed: `idx_auth_rules_pattern ON auth_rules(pattern_type, pattern_value)`
|
||||
- No additional per-event overhead — the existing `check_database_auth_rules()` already runs on every event when auth is enabled
|
||||
- **Level 2 directly addresses CPU load**: With `wot_enabled=2`, unauthenticated connections cannot create subscriptions, eliminating the ~120 REQ/minute churn from anonymous connections that was causing 99% CPU
|
||||
@@ -0,0 +1,388 @@
|
||||
# Web of Trust — Admin Web UI Plan
|
||||
|
||||
## Overview
|
||||
|
||||
Add a **Web of Trust** section to the existing Authorization page in the admin web interface. The section lets the admin see whether their kind 3 contact list is on the relay, select a WoT level, trigger a sync, and view the current whitelist count.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Data Flow
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A[Web UI] -->|config_set wot_enabled N| B[Kind 23456 Event]
|
||||
A -->|system_command wot_sync| B
|
||||
A -->|system_command wot_status| B
|
||||
B --> C[Relay Backend]
|
||||
C -->|Kind 23457 Response| D[Web UI updates display]
|
||||
```
|
||||
|
||||
### How It Works
|
||||
|
||||
1. **On page load** — the Authorization page already calls `loadAuthRules`. We add a parallel call to `loadWotStatus` which sends a `system_command` / `wot_status` event.
|
||||
2. **Backend responds** with a kind 23457 JSON containing: `wot_enabled` level, `wot_whitelist_count`, and `admin_kind3_exists` boolean.
|
||||
3. **UI renders** a card showing:
|
||||
- Whether the admin's kind 3 event exists on the relay
|
||||
- Current WoT level as a 3-option radio/button group
|
||||
- Count of whitelisted pubkeys
|
||||
- A SYNC button to force resync
|
||||
4. **Changing level** sends `config_set` / `wot_enabled` / `0|1|2` via the existing admin API, then triggers `wot_sync` if level > 0.
|
||||
5. **SYNC button** sends `system_command` / `wot_sync`.
|
||||
|
||||
## Detailed Changes
|
||||
|
||||
### 1. Backend: New system commands in `src/config.c`
|
||||
|
||||
Add two new branches to `handle_system_command_unified`:
|
||||
|
||||
#### `wot_status` command
|
||||
|
||||
Returns JSON response:
|
||||
```json
|
||||
{
|
||||
"command": "wot_status",
|
||||
"status": "success",
|
||||
"wot_enabled": 2,
|
||||
"admin_kind3_exists": true,
|
||||
"wot_whitelist_count": 347,
|
||||
"timestamp": 1234567890
|
||||
}
|
||||
```
|
||||
|
||||
Implementation:
|
||||
- Read `wot_enabled` from config table
|
||||
- Query `SELECT COUNT(*) FROM events WHERE kind = 3 AND pubkey = ?` with admin_pubkey to check kind 3 existence
|
||||
- Query `SELECT COUNT(*) FROM auth_rules WHERE rule_type = 'wot_whitelist' AND active = 1` for whitelist count
|
||||
- Return as kind 23457 signed response
|
||||
|
||||
#### `wot_sync` command
|
||||
|
||||
Calls `wot_sync_from_admin_kind3` and returns result:
|
||||
```json
|
||||
{
|
||||
"command": "wot_sync",
|
||||
"status": "success",
|
||||
"wot_whitelist_count": 347,
|
||||
"timestamp": 1234567890
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Backend: Hook `config_set` for `wot_enabled`
|
||||
|
||||
In `handle_config_set_unified`, after the config value is updated, add a check:
|
||||
```c
|
||||
// After successful update, trigger WoT sync if wot_enabled changed
|
||||
if (strcmp(config_key, "wot_enabled") == 0) {
|
||||
int new_level = atoi(config_value);
|
||||
if (new_level > 0) {
|
||||
wot_sync_from_admin_kind3();
|
||||
} else {
|
||||
// Level 0: clear wot_whitelist rules and reset auth flags
|
||||
sqlite3_exec(g_db, "DELETE FROM auth_rules WHERE rule_type = 'wot_whitelist'", NULL, NULL, NULL);
|
||||
update_config_in_table("nip42_auth_required_subscriptions", "false");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Frontend: HTML — WoT section in `api/index.html`
|
||||
|
||||
Add a new div **inside** the `authRulesSection`, before the auth rules table. This keeps WoT visually grouped with authorization:
|
||||
|
||||
```html
|
||||
<!-- Web of Trust Section -->
|
||||
<div id="wotSection" class="input-group">
|
||||
<div class="section-header" style="font-size: 14px; margin-bottom: 10px;">
|
||||
WEB OF TRUST
|
||||
</div>
|
||||
|
||||
<!-- Kind 3 Status Indicator -->
|
||||
<div id="wotKind3Status" class="wot-status-row">
|
||||
<span>Admin Contact List (kind 3):</span>
|
||||
<span id="wotKind3Indicator" class="wot-indicator wot-unknown">Checking...</span>
|
||||
</div>
|
||||
|
||||
<!-- WoT Level Selector -->
|
||||
<div class="wot-level-selector">
|
||||
<label>WoT Level:</label>
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="wotLevel0Btn" class="wot-level-btn" onclick="setWotLevel(0)">
|
||||
OFF
|
||||
</button>
|
||||
<button type="button" id="wotLevel1Btn" class="wot-level-btn" onclick="setWotLevel(1)">
|
||||
WRITE ONLY
|
||||
</button>
|
||||
<button type="button" id="wotLevel2Btn" class="wot-level-btn" onclick="setWotLevel(2)">
|
||||
FULL
|
||||
</button>
|
||||
</div>
|
||||
<div class="wot-level-description" id="wotLevelDescription">
|
||||
Level 0: Open relay — anyone can read and write
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- WoT Stats -->
|
||||
<div class="wot-stats-row">
|
||||
<span>Whitelisted Pubkeys:</span>
|
||||
<span id="wotWhitelistCount">—</span>
|
||||
</div>
|
||||
|
||||
<!-- Sync Button -->
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="wotSyncBtn" onclick="syncWot()">SYNC FROM KIND 3</button>
|
||||
<button type="button" id="wotRefreshBtn" onclick="loadWotStatus()">REFRESH STATUS</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<hr style="margin: 15px 0; border-color: var(--border-color);">
|
||||
```
|
||||
|
||||
### 4. Frontend: CSS additions in `api/index.css`
|
||||
|
||||
```css
|
||||
/* Web of Trust styles */
|
||||
.wot-status-row, .wot-stats-row {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: 8px 0;
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.wot-indicator {
|
||||
padding: 2px 10px;
|
||||
border-radius: 4px;
|
||||
font-weight: bold;
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
.wot-indicator.wot-found { background: #28a745; color: white; }
|
||||
.wot-indicator.wot-missing { background: #dc3545; color: white; }
|
||||
.wot-indicator.wot-unknown { background: #6c757d; color: white; }
|
||||
|
||||
.wot-level-selector { padding: 10px 0; }
|
||||
.wot-level-selector label { display: block; margin-bottom: 5px; font-weight: bold; }
|
||||
|
||||
.wot-level-btn { min-width: 100px; }
|
||||
.wot-level-btn.active {
|
||||
background: var(--accent-color, #007bff);
|
||||
color: white;
|
||||
border-color: var(--accent-color, #007bff);
|
||||
}
|
||||
|
||||
.wot-level-description {
|
||||
font-size: 12px;
|
||||
color: var(--text-secondary);
|
||||
margin-top: 5px;
|
||||
font-style: italic;
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Frontend: JavaScript functions in `api/index.js`
|
||||
|
||||
#### `loadWotStatus` — query current WoT state
|
||||
|
||||
```javascript
|
||||
async function loadWotStatus() {
|
||||
try {
|
||||
if (!isLoggedIn || !userPubkey || !relayPool) return;
|
||||
|
||||
const command_array = ["system_command", "wot_status"];
|
||||
const encrypted_content = await encryptForRelay(JSON.stringify(command_array));
|
||||
if (!encrypted_content) return;
|
||||
|
||||
const event = {
|
||||
kind: 23456,
|
||||
pubkey: userPubkey,
|
||||
created_at: Math.floor(Date.now() / 1000),
|
||||
tags: [["p", getRelayPubkey()]],
|
||||
content: encrypted_content
|
||||
};
|
||||
|
||||
const signedEvent = await window.nostr.signEvent(event);
|
||||
const url = relayConnectionUrl.value.trim();
|
||||
await relayPool.publish([url], signedEvent);
|
||||
|
||||
log('WoT status query sent', 'INFO');
|
||||
} catch (error) {
|
||||
log('Failed to load WoT status: ' + error.message, 'ERROR');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### `setWotLevel` — change WoT level via config_set
|
||||
|
||||
```javascript
|
||||
async function setWotLevel(level) {
|
||||
try {
|
||||
if (!isLoggedIn || !userPubkey || !relayPool) return;
|
||||
|
||||
// Send config_set for wot_enabled
|
||||
const command_array = ["config_set", "wot_enabled", String(level)];
|
||||
const encrypted_content = await encryptForRelay(JSON.stringify(command_array));
|
||||
if (!encrypted_content) return;
|
||||
|
||||
const event = {
|
||||
kind: 23456,
|
||||
pubkey: userPubkey,
|
||||
created_at: Math.floor(Date.now() / 1000),
|
||||
tags: [["p", getRelayPubkey()]],
|
||||
content: encrypted_content
|
||||
};
|
||||
|
||||
const signedEvent = await window.nostr.signEvent(event);
|
||||
const url = relayConnectionUrl.value.trim();
|
||||
await relayPool.publish([url], signedEvent);
|
||||
|
||||
log('WoT level set to ' + level, 'INFO');
|
||||
|
||||
// Refresh status after a short delay
|
||||
setTimeout(() => loadWotStatus(), 1500);
|
||||
} catch (error) {
|
||||
log('Failed to set WoT level: ' + error.message, 'ERROR');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### `syncWot` — force WoT sync
|
||||
|
||||
```javascript
|
||||
async function syncWot() {
|
||||
try {
|
||||
if (!isLoggedIn || !userPubkey || !relayPool) return;
|
||||
|
||||
const command_array = ["system_command", "wot_sync"];
|
||||
const encrypted_content = await encryptForRelay(JSON.stringify(command_array));
|
||||
if (!encrypted_content) return;
|
||||
|
||||
const event = {
|
||||
kind: 23456,
|
||||
pubkey: userPubkey,
|
||||
created_at: Math.floor(Date.now() / 1000),
|
||||
tags: [["p", getRelayPubkey()]],
|
||||
content: encrypted_content
|
||||
};
|
||||
|
||||
const signedEvent = await window.nostr.signEvent(event);
|
||||
const url = relayConnectionUrl.value.trim();
|
||||
await relayPool.publish([url], signedEvent);
|
||||
|
||||
log('WoT sync command sent', 'INFO');
|
||||
|
||||
// Refresh status after sync completes
|
||||
setTimeout(() => loadWotStatus(), 2000);
|
||||
} catch (error) {
|
||||
log('Failed to sync WoT: ' + error.message, 'ERROR');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### `handleWotStatusResponse` — process backend response
|
||||
|
||||
```javascript
|
||||
function handleWotStatusResponse(responseData) {
|
||||
const kind3Indicator = document.getElementById('wotKind3Indicator');
|
||||
const whitelistCount = document.getElementById('wotWhitelistCount');
|
||||
const levelDesc = document.getElementById('wotLevelDescription');
|
||||
|
||||
// Update kind 3 indicator
|
||||
if (kind3Indicator) {
|
||||
if (responseData.admin_kind3_exists) {
|
||||
kind3Indicator.textContent = 'Found ✓';
|
||||
kind3Indicator.className = 'wot-indicator wot-found';
|
||||
} else {
|
||||
kind3Indicator.textContent = 'Not Found ✗';
|
||||
kind3Indicator.className = 'wot-indicator wot-missing';
|
||||
}
|
||||
}
|
||||
|
||||
// Update whitelist count
|
||||
if (whitelistCount) {
|
||||
whitelistCount.textContent = responseData.wot_whitelist_count || 0;
|
||||
}
|
||||
|
||||
// Update level buttons
|
||||
const level = responseData.wot_enabled || 0;
|
||||
['wotLevel0Btn', 'wotLevel1Btn', 'wotLevel2Btn'].forEach((id, i) => {
|
||||
const btn = document.getElementById(id);
|
||||
if (btn) {
|
||||
btn.classList.toggle('active', i === level);
|
||||
}
|
||||
});
|
||||
|
||||
// Update description
|
||||
const descriptions = [
|
||||
'Level 0: Open relay — anyone can read and write',
|
||||
'Level 1: Write-only — only followed pubkeys can publish events',
|
||||
'Level 2: Full — only followed pubkeys can publish AND subscribe (NIP-42 required)'
|
||||
];
|
||||
if (levelDesc) {
|
||||
levelDesc.textContent = descriptions[level] || descriptions[0];
|
||||
}
|
||||
|
||||
// Enable/disable sync button based on kind 3 existence
|
||||
const syncBtn = document.getElementById('wotSyncBtn');
|
||||
if (syncBtn) {
|
||||
syncBtn.disabled = !responseData.admin_kind3_exists;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Wire into existing response handler
|
||||
|
||||
In the existing `handleSystemCommandResponse` function, add:
|
||||
```javascript
|
||||
if (responseData.command === 'wot_status' || responseData.command === 'wot_sync') {
|
||||
handleWotStatusResponse(responseData);
|
||||
}
|
||||
```
|
||||
|
||||
#### Wire into page navigation
|
||||
|
||||
In the `authorization` case of the page switch handler, add:
|
||||
```javascript
|
||||
case 'authorization':
|
||||
loadAuthRules().catch(...);
|
||||
loadWotStatus().catch(error => {
|
||||
console.log('Auto-load WoT status failed: ' + error.message);
|
||||
});
|
||||
break;
|
||||
```
|
||||
|
||||
## Files to Modify
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `src/config.c` | Add `wot_status` and `wot_sync` branches to `handle_system_command_unified`; add WoT sync hook to `handle_config_set_unified` |
|
||||
| `api/index.html` | Add WoT section HTML inside `authRulesSection` |
|
||||
| `api/index.css` | Add WoT-specific CSS styles |
|
||||
| `api/index.js` | Add `loadWotStatus`, `setWotLevel`, `syncWot`, `handleWotStatusResponse` functions; wire into page nav and response handler |
|
||||
|
||||
## Edge Cases
|
||||
|
||||
1. **Admin not logged in** — WoT section shows but buttons are disabled; status shows "Checking..."
|
||||
2. **No kind 3 event** — Indicator shows red "Not Found ✗"; SYNC button is disabled; level selector still works but sync will whitelist only admin + relay
|
||||
3. **Level change from 2 to 0** — Backend clears all wot_whitelist rules and resets nip42_auth_required_subscriptions
|
||||
4. **Concurrent config_set and wot_sync** — The backend handles these sequentially via SQLite transactions; no race condition
|
||||
|
||||
## UI Mockup
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ WEB OF TRUST │
|
||||
│ │
|
||||
│ Admin Contact List (kind 3): [Found ✓] │
|
||||
│ │
|
||||
│ WoT Level: │
|
||||
│ [ OFF ] [ WRITE ONLY ] [ FULL ] │
|
||||
│ Level 2: Full — only followed pubkeys can │
|
||||
│ publish AND subscribe (NIP-42 required) │
|
||||
│ │
|
||||
│ Whitelisted Pubkeys: 347 │
|
||||
│ │
|
||||
│ [ SYNC FROM KIND 3 ] [ REFRESH STATUS ] │
|
||||
├─────────────────────────────────────────────┤
|
||||
│ ─────────────────────────────────────────── │
|
||||
│ AUTH RULES MANAGEMENT │
|
||||
│ ... existing auth rules table ... │
|
||||
└─────────────────────────────────────────────┘
|
||||
```
|
||||
@@ -0,0 +1,10 @@
|
||||
https://github.com/rushmi0/Fenrir-s
|
||||
https://github.com/barkyq/gnost-relay
|
||||
https://github.com/lpicanco/knostr
|
||||
https://github.com/bezysoftware/netstr
|
||||
https://github.com/lebrunel/nex
|
||||
https://github.com/CodyTseng/nostr-relay-nestjs
|
||||
https://github.com/mattn/nostr-relay
|
||||
https://github.com/Cameri/nostream
|
||||
https://github.com/Giszmo/NostrPostr/tree/master/NostrRelay
|
||||
https://github.com/fiatjaf/relayer/tree/master/examples/basic
|
||||
+1113
-600
File diff suppressed because it is too large
Load Diff
+5
-2
@@ -1,7 +1,6 @@
|
||||
#ifndef CONFIG_H
|
||||
#define CONFIG_H
|
||||
|
||||
#include <sqlite3.h>
|
||||
#include <cjson/cJSON.h>
|
||||
#include <time.h>
|
||||
#include <pthread.h>
|
||||
@@ -78,6 +77,7 @@ char* extract_pubkey_from_filename(const char* filename);
|
||||
int store_relay_private_key(const char* relay_privkey_hex);
|
||||
char* get_relay_private_key(void);
|
||||
const char* get_temp_relay_private_key(void); // For first-time startup only
|
||||
int preload_relay_private_key_cache(void); // Preload relay private key before strict DB mode
|
||||
|
||||
// NIP-42 authentication configuration functions
|
||||
int parse_auth_required_kinds(const char* kinds_str, int* kinds_array, int max_kinds);
|
||||
@@ -116,7 +116,10 @@ cJSON* build_query_response(const char* query_type, cJSON* results_array, int to
|
||||
int add_auth_rule_from_config(const char* rule_type, const char* pattern_type,
|
||||
const char* pattern_value);
|
||||
int remove_auth_rule_from_config(const char* rule_type, const char* pattern_type,
|
||||
const char* pattern_value);
|
||||
const char* pattern_value);
|
||||
|
||||
// Web of Trust (WoT) sync function
|
||||
int wot_sync_from_admin_kind3(void);
|
||||
|
||||
// Unified configuration cache management
|
||||
void force_config_cache_refresh(void);
|
||||
|
||||
+1397
File diff suppressed because it is too large
Load Diff
+120
@@ -0,0 +1,120 @@
|
||||
#ifndef DB_OPS_H
|
||||
#define DB_OPS_H
|
||||
|
||||
#include <stddef.h>
|
||||
#include <time.h>
|
||||
#include <cjson/cJSON.h>
|
||||
|
||||
// Generic helpers
|
||||
int db_init(const char* connection_string);
|
||||
void db_close(void);
|
||||
int db_is_available(void);
|
||||
const char* db_last_error(void);
|
||||
const char* db_get_database_path(void);
|
||||
|
||||
// Per-thread connection override (used by thread pool workers)
|
||||
int db_set_thread_connection(void* connection);
|
||||
void db_clear_thread_connection(void);
|
||||
|
||||
// Worker/runtime SQLite connection lifecycle (kept internal to db_ops.c)
|
||||
int db_open_worker_connection(const char* db_path, void** out_connection);
|
||||
void db_close_worker_connection(void* connection);
|
||||
|
||||
// DB result codes (backend-agnostic)
|
||||
#define DB_OK 0
|
||||
#define DB_ERROR 1
|
||||
#define DB_CONSTRAINT 19
|
||||
#define DB_MISUSE 21
|
||||
#define DB_ROW 100
|
||||
#define DB_DONE 101
|
||||
|
||||
typedef struct db_stmt db_stmt_t;
|
||||
|
||||
// Generic statement helpers (Phase 1 migration)
|
||||
int db_prepare(const char* sql, db_stmt_t** out_stmt);
|
||||
int db_bind_text_param(db_stmt_t* stmt, int index, const char* value);
|
||||
int db_bind_int_param(db_stmt_t* stmt, int index, int value);
|
||||
int db_bind_int64_param(db_stmt_t* stmt, int index, long long value);
|
||||
int db_step_stmt(db_stmt_t* stmt);
|
||||
int db_reset_stmt(db_stmt_t* stmt);
|
||||
const char* db_column_text_value(db_stmt_t* stmt, int col);
|
||||
int db_column_int_value(db_stmt_t* stmt, int col);
|
||||
long long db_column_int64_value(db_stmt_t* stmt, int col);
|
||||
double db_column_double_value(db_stmt_t* stmt, int col);
|
||||
void db_finalize_stmt(db_stmt_t* stmt);
|
||||
|
||||
// Subscription logging
|
||||
int db_log_subscription_created(const char* sub_id, const char* wsi_ptr,
|
||||
const char* client_ip, const char* filter_json);
|
||||
int db_log_subscription_closed(const char* sub_id, const char* client_ip);
|
||||
int db_log_subscription_disconnected(const char* client_ip);
|
||||
int db_update_subscription_events_sent(const char* sub_id, int events_sent);
|
||||
int db_cleanup_orphaned_subscriptions(void);
|
||||
|
||||
// NIP-09 event deletion helpers
|
||||
int db_get_event_pubkey(const char* event_id, char* pubkey_out, size_t pubkey_out_size);
|
||||
int db_delete_event_by_id(const char* event_id, const char* requester_pubkey);
|
||||
int db_delete_events_by_address(const char* pubkey, int kind,
|
||||
const char* d_tag, long before_timestamp);
|
||||
|
||||
// Auth rule checks for request validator
|
||||
int db_is_pubkey_blacklisted(const char* pubkey);
|
||||
int db_is_hash_blacklisted(const char* resource_hash);
|
||||
int db_is_pubkey_whitelisted(const char* pubkey);
|
||||
int db_count_active_whitelist_rules(void);
|
||||
|
||||
// Generic prepared COUNT helper
|
||||
int db_count_with_sql(const char* sql, const char** bind_params, int bind_param_count, int* out_count);
|
||||
char* db_execute_readonly_query_json(const char* query, const char* request_id,
|
||||
char* error_message, size_t error_size,
|
||||
int max_rows, int timeout_ms);
|
||||
|
||||
// Monitoring/stat helpers (Phase 1 api.c migration)
|
||||
int db_get_total_event_count_ll(long long* out_count);
|
||||
int db_get_event_count_since(time_t cutoff, long long* out_count);
|
||||
cJSON* db_get_event_kind_distribution_rows(long long* out_total_events);
|
||||
cJSON* db_get_top_pubkeys_rows(int limit);
|
||||
cJSON* db_get_subscription_details_rows(void);
|
||||
|
||||
// Config helpers
|
||||
cJSON* db_get_all_config_rows(void);
|
||||
char* db_get_config_value_dup(const char* key);
|
||||
int db_set_config_value_full(const char* key, const char* value, const char* data_type,
|
||||
const char* description, const char* category, int requires_restart);
|
||||
int db_update_config_value_only(const char* key, const char* value);
|
||||
int db_upsert_config_value(const char* key, const char* value, const char* data_type);
|
||||
int db_store_relay_private_key_hex(const char* relay_privkey_hex);
|
||||
char* db_get_relay_private_key_hex_dup(void);
|
||||
int db_store_config_event(const cJSON* event);
|
||||
|
||||
// Event storage/timestamp/retrieval helpers
|
||||
int db_insert_event_with_json(const char* id, const char* pubkey, long long created_at,
|
||||
int kind, const char* event_type, const char* content,
|
||||
const char* sig, const char* tags_json, const char* event_json,
|
||||
int* out_step_rc, int* out_extended_errcode);
|
||||
int db_get_event_time_bounds(long long* out_min_created_at, long long* out_max_created_at);
|
||||
int db_event_id_exists(const char* event_id, int* out_exists);
|
||||
cJSON* db_retrieve_event_by_id(const char* event_id);
|
||||
char* db_get_latest_event_pubkey_for_kind_dup(int kind);
|
||||
|
||||
// Config table helpers
|
||||
int db_get_config_row_count(int* out_count);
|
||||
|
||||
// Event tag helpers
|
||||
int db_store_event_tags_cjson(const char* event_id, const cJSON* tags);
|
||||
int db_populate_event_tags_from_existing(void);
|
||||
|
||||
// Auth/WoT rule helpers
|
||||
int db_add_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
|
||||
int db_remove_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
|
||||
int db_delete_wot_whitelist_rules(void);
|
||||
int db_count_wot_whitelist_rules(void);
|
||||
|
||||
// Schema/DDL helpers
|
||||
int db_table_exists(const char* table_name, int* out_exists);
|
||||
char* db_get_schema_version_dup(void);
|
||||
int db_exec_sql(const char* sql);
|
||||
int db_wal_checkpoint_passive(void);
|
||||
int db_wal_checkpoint_truncate(void);
|
||||
|
||||
#endif // DB_OPS_H
|
||||
@@ -83,11 +83,52 @@ static const struct {
|
||||
// IP-based rate limiting or access control (which would require firewall protection anyway)
|
||||
{"trust_proxy_headers", "true"},
|
||||
|
||||
// Debug Level (0=none, 1=errors, 2=warnings, 3=info, 4=debug, 5=trace)
|
||||
// Can be changed at runtime without restart via config_set admin command
|
||||
{"debug_level", "3"},
|
||||
|
||||
// IP Auth Failure Ban Settings
|
||||
// Ban IPs that repeatedly fail NIP-42 authentication
|
||||
{"auth_fail_ban_enabled", "true"},
|
||||
{"auth_fail_ban_threshold", "3"}, // failures before ban
|
||||
{"auth_fail_window_sec", "60"}, // window to count failures in
|
||||
{"auth_fail_ban_duration_sec", "300"}, // initial ban duration (doubles each time, max 24h)
|
||||
|
||||
// NIP-42 Authentication Timeout
|
||||
// Seconds after connection before unauthenticated clients are disconnected (0 = disabled)
|
||||
// Prevents unauthenticated connections from accumulating under heavy load
|
||||
{"nip42_auth_timeout_sec", "10"},
|
||||
|
||||
// Idle Connection Ban Settings
|
||||
// Ban IPs that connect but never send REQ or EVENT (idle or early disconnect)
|
||||
{"idle_connection_timeout_sec", "0"}, // Seconds before idle connection is closed (0 = disabled)
|
||||
{"idle_ban_enabled", "false"}, // Whether to ban IPs with idle failures
|
||||
{"idle_ban_threshold", "1"}, // Idle failures before ban (1 = ban on first offense)
|
||||
{"idle_ban_window_sec", "30"}, // Window to count idle failures in
|
||||
{"idle_ban_duration_sec", "300"}, // Initial ban duration (doubles each time, max 24h)
|
||||
|
||||
// SQLite Performance Tuning
|
||||
// mmap_size: bytes of database file to memory-map (0 = disabled, 268435456 = 256MB recommended)
|
||||
// Eliminates pread64 syscall overhead for database reads — significant CPU savings under load
|
||||
{"sqlite_mmap_size", "268435456"},
|
||||
// cache_size_kb: SQLite page cache size in KB (negative = KB, positive = pages of 4KB each)
|
||||
// Default 2000KB is too small for a busy relay; 65536KB (64MB) keeps hot data in memory
|
||||
{"sqlite_cache_size_kb", "65536"},
|
||||
|
||||
// NIP-59 Gift Wrap Timestamp Configuration
|
||||
{"nip59_timestamp_max_delay_sec", "0"},
|
||||
|
||||
// Kind 1 Status Posts
|
||||
{"kind_1_status_posts_hours", "1"}
|
||||
{"kind_1_status_posts_hours", "1"},
|
||||
|
||||
// Web of Trust Settings
|
||||
// 0 = off, 1 = write-only (followed pubkeys can publish), 2 = full (followed pubkeys can publish AND subscribe)
|
||||
{"wot_enabled", "0"},
|
||||
|
||||
// NIP-17 Admin DM Settings
|
||||
// When false (default), Kind 1059 gift wrap events are stored normally without expensive decryption attempts.
|
||||
// Enable only if you intend to use NIP-17 DMs to send admin commands to the relay.
|
||||
{"nip17_admin_enabled", "false"}
|
||||
};
|
||||
|
||||
// Number of default configuration values
|
||||
|
||||
+153
-13
@@ -5,6 +5,7 @@
|
||||
#include "config.h"
|
||||
#include "debug.h"
|
||||
#include "api.h"
|
||||
#include "db_ops.h"
|
||||
#include "../nostr_core_lib/nostr_core/nostr_core.h"
|
||||
#include "../nostr_core_lib/nostr_core/nip017.h"
|
||||
#include "../nostr_core_lib/nostr_core/nip044.h"
|
||||
@@ -17,9 +18,6 @@
|
||||
#include <cjson/cJSON.h>
|
||||
#include <libwebsockets.h>
|
||||
|
||||
// External database connection (from main.c)
|
||||
extern sqlite3* g_db;
|
||||
|
||||
// Forward declarations for unified handlers
|
||||
extern int handle_auth_query_unified(cJSON* event, const char* query_type, char* error_message, size_t error_size, struct lws* wsi);
|
||||
extern int handle_config_query_unified(cJSON* event, const char* query_type, char* error_message, size_t error_size, struct lws* wsi);
|
||||
@@ -37,10 +35,13 @@ extern const char* get_tag_value(cJSON* event, const char* tag_name, int value_i
|
||||
extern char* get_relay_private_key(void);
|
||||
extern const char* get_config_value(const char* key);
|
||||
extern int get_config_bool(const char* key, int default_value);
|
||||
extern int get_config_int(const char* key, int default_value);
|
||||
extern int update_config_in_table(const char* key, const char* value);
|
||||
|
||||
// Forward declarations for database functions
|
||||
extern int store_event(cJSON* event);
|
||||
extern int broadcast_event_to_subscriptions(cJSON* event);
|
||||
extern int store_event_tags(const char* event_id, cJSON* tags);
|
||||
|
||||
// Forward declarations for stats generation (moved to api.c)
|
||||
extern char* generate_stats_json(void);
|
||||
@@ -138,7 +139,7 @@ int process_dm_admin_command(cJSON* command_array, cJSON* event, char* error_mes
|
||||
cJSON_AddItemToArray(synthetic_tags, command_tag);
|
||||
|
||||
// Add existing event tags
|
||||
cJSON* existing_tags = cJSON_GetObjectItem(event, "tags");
|
||||
cJSON* existing_tags = cJSON_GetObjectItemCaseSensitive(event, "tags");
|
||||
if (existing_tags && cJSON_IsArray(existing_tags)) {
|
||||
cJSON* tag = NULL;
|
||||
cJSON_ArrayForEach(tag, existing_tags) {
|
||||
@@ -301,7 +302,7 @@ cJSON* process_nip17_admin_message(cJSON* gift_wrap_event, char* error_message,
|
||||
// Only create a generic response for other command types that don't handle their own responses
|
||||
if (result == 0) {
|
||||
// Extract content to check if it's a plain text command
|
||||
cJSON* content_obj = cJSON_GetObjectItem(inner_dm, "content");
|
||||
cJSON* content_obj = cJSON_GetObjectItemCaseSensitive(inner_dm, "content");
|
||||
if (content_obj && cJSON_IsString(content_obj)) {
|
||||
const char* dm_content = cJSON_GetStringValue(content_obj);
|
||||
|
||||
@@ -344,7 +345,7 @@ cJSON* process_nip17_admin_message(cJSON* gift_wrap_event, char* error_message,
|
||||
return NULL;
|
||||
|
||||
// Get sender pubkey for response from the decrypted DM event
|
||||
cJSON* sender_pubkey_obj = cJSON_GetObjectItem(inner_dm, "pubkey");
|
||||
cJSON* sender_pubkey_obj = cJSON_GetObjectItemCaseSensitive(inner_dm, "pubkey");
|
||||
if (sender_pubkey_obj && cJSON_IsString(sender_pubkey_obj)) {
|
||||
const char* sender_pubkey = cJSON_GetStringValue(sender_pubkey_obj);
|
||||
|
||||
@@ -436,13 +437,13 @@ int is_nip17_gift_wrap_for_relay(cJSON* event) {
|
||||
}
|
||||
|
||||
// Check kind
|
||||
cJSON* kind_obj = cJSON_GetObjectItem(event, "kind");
|
||||
cJSON* kind_obj = cJSON_GetObjectItemCaseSensitive(event, "kind");
|
||||
if (!kind_obj || !cJSON_IsNumber(kind_obj) || (int)cJSON_GetNumberValue(kind_obj) != 1059) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Check tags for "p" tag with relay pubkey
|
||||
cJSON* tags = cJSON_GetObjectItem(event, "tags");
|
||||
cJSON* tags = cJSON_GetObjectItemCaseSensitive(event, "tags");
|
||||
if (!tags || !cJSON_IsArray(tags)) {
|
||||
return 0;
|
||||
}
|
||||
@@ -481,7 +482,7 @@ int process_nip17_admin_command(cJSON* dm_event, char* error_message, size_t err
|
||||
DEBUG_INFO("DM_ADMIN: Processing NIP-17 admin command from decrypted DM");
|
||||
|
||||
// Extract content from DM
|
||||
cJSON* content_obj = cJSON_GetObjectItem(dm_event, "content");
|
||||
cJSON* content_obj = cJSON_GetObjectItemCaseSensitive(dm_event, "content");
|
||||
if (!content_obj || !cJSON_IsString(content_obj)) {
|
||||
DEBUG_INFO("DM_ADMIN: DM missing content field");
|
||||
strncpy(error_message, "NIP-17: DM missing content", error_size - 1);
|
||||
@@ -492,7 +493,7 @@ int process_nip17_admin_command(cJSON* dm_event, char* error_message, size_t err
|
||||
DEBUG_INFO("DM_ADMIN: Extracted DM content: %.100s%s", dm_content, strlen(dm_content) > 100 ? "..." : "");
|
||||
|
||||
// Check if sender is admin before processing any commands
|
||||
cJSON* sender_pubkey_obj = cJSON_GetObjectItem(dm_event, "pubkey");
|
||||
cJSON* sender_pubkey_obj = cJSON_GetObjectItemCaseSensitive(dm_event, "pubkey");
|
||||
if (!sender_pubkey_obj || !cJSON_IsString(sender_pubkey_obj)) {
|
||||
DEBUG_INFO("DM_ADMIN: DM missing sender pubkey - treating as user DM");
|
||||
return 0; // Not an error, just treat as user DM
|
||||
@@ -598,6 +599,145 @@ int process_nip17_admin_command(cJSON* dm_event, char* error_message, size_t err
|
||||
DEBUG_INFO("DM_ADMIN: Status command processed successfully");
|
||||
return 0;
|
||||
}
|
||||
// Check for WoT (Web of Trust) commands
|
||||
else if (strstr(content_lower, "wot") != NULL) {
|
||||
DEBUG_INFO("DM_ADMIN: Processing WoT command");
|
||||
|
||||
// Skip "wot" prefix and find the subcommand
|
||||
char* wot_cmd = strstr(content_lower, "wot");
|
||||
if (wot_cmd) {
|
||||
wot_cmd += 3; // Skip "wot"
|
||||
while (*wot_cmd == ' ') wot_cmd++; // Skip spaces
|
||||
|
||||
char response_msg[1024];
|
||||
int wot_level = get_config_int("wot_enabled", 0);
|
||||
extern int wot_sync_from_admin_kind3(void);
|
||||
|
||||
if (strcmp(wot_cmd, "0") == 0 || strcmp(wot_cmd, "off") == 0) {
|
||||
// Disable WoT
|
||||
update_config_in_table("wot_enabled", "0");
|
||||
update_config_in_table("auth_enabled", "false");
|
||||
update_config_in_table("nip42_auth_required_subscriptions", "false");
|
||||
|
||||
// Clear wot_whitelist rules
|
||||
db_delete_wot_whitelist_rules();
|
||||
|
||||
snprintf(response_msg, sizeof(response_msg),
|
||||
"🔓 Web of Trust Disabled\n"
|
||||
"━━━━━━━━━━━━━━━━━━━━━━\n"
|
||||
"\n"
|
||||
"WoT has been turned off.\n"
|
||||
"\n"
|
||||
"The relay is now open to all pubkeys for reading and writing.\n"
|
||||
"Manual whitelist/blacklist rules are preserved.");
|
||||
send_nip17_response(sender_pubkey, response_msg, NULL, 0);
|
||||
DEBUG_INFO("DM_ADMIN: WoT disabled");
|
||||
}
|
||||
else if (strcmp(wot_cmd, "1") == 0 || strcmp(wot_cmd, "write") == 0) {
|
||||
// Write-only mode
|
||||
update_config_in_table("wot_enabled", "1");
|
||||
wot_sync_from_admin_kind3();
|
||||
|
||||
wot_level = get_config_int("wot_enabled", 0);
|
||||
snprintf(response_msg, sizeof(response_msg),
|
||||
"✍️ Web of Trust: Write-Only Mode\n"
|
||||
"━━━━━━━━━━━━━━━━━━━━━━\n"
|
||||
"\n"
|
||||
"Level: 1 (Write-only restriction)\n"
|
||||
"\n"
|
||||
"Only pubkeys you follow can publish events.\n"
|
||||
"Anyone can still subscribe and read events.\n"
|
||||
"\n"
|
||||
"The relay will now sync from your kind 3 (contact list) event.");
|
||||
send_nip17_response(sender_pubkey, response_msg, NULL, 0);
|
||||
DEBUG_INFO("DM_ADMIN: WoT set to write-only mode");
|
||||
}
|
||||
else if (strcmp(wot_cmd, "2") == 0 || strcmp(wot_cmd, "full") == 0) {
|
||||
// Full mode (write + read restriction)
|
||||
update_config_in_table("wot_enabled", "2");
|
||||
wot_sync_from_admin_kind3();
|
||||
|
||||
wot_level = get_config_int("wot_enabled", 0);
|
||||
snprintf(response_msg, sizeof(response_msg),
|
||||
"🔒 Web of Trust: Full Mode\n"
|
||||
"━━━━━━━━━━━━━━━━━━━━━━\n"
|
||||
"\n"
|
||||
"Level: 2 (Full restriction)\n"
|
||||
"\n"
|
||||
"Only pubkeys you follow can:\n"
|
||||
" • Publish events\n"
|
||||
" • Subscribe to events (requires NIP-42 auth)\n"
|
||||
"\n"
|
||||
"This eliminates anonymous subscription churn.\n"
|
||||
"\n"
|
||||
"The relay will now sync from your kind 3 (contact list) event.");
|
||||
send_nip17_response(sender_pubkey, response_msg, NULL, 0);
|
||||
DEBUG_INFO("DM_ADMIN: WoT set to full mode");
|
||||
}
|
||||
else if (strcmp(wot_cmd, "sync") == 0) {
|
||||
// Force resync
|
||||
wot_sync_from_admin_kind3();
|
||||
|
||||
snprintf(response_msg, sizeof(response_msg),
|
||||
"🔄 Web of Trust: Sync Complete\n"
|
||||
"━━━━━━━━━━━━━━━━━━━━━━\n"
|
||||
"\n"
|
||||
"WoT whitelist has been refreshed from your\n"
|
||||
"kind 3 (contact list) event.\n"
|
||||
"\n"
|
||||
"Current level: %d", wot_level);
|
||||
send_nip17_response(sender_pubkey, response_msg, NULL, 0);
|
||||
DEBUG_INFO("DM_ADMIN: WoT manual sync completed");
|
||||
}
|
||||
else if (strcmp(wot_cmd, "status") == 0 || strlen(wot_cmd) == 0) {
|
||||
// Show status
|
||||
// Count whitelisted pubkeys
|
||||
int whitelist_count = db_count_wot_whitelist_rules();
|
||||
|
||||
const char* level_str;
|
||||
if (wot_level == 0) level_str = "Off (open relay)";
|
||||
else if (wot_level == 1) level_str = "Write-only (followed pubkeys can publish)";
|
||||
else if (wot_level == 2) level_str = "Full (followed pubkeys can publish AND subscribe)";
|
||||
else level_str = "Unknown";
|
||||
|
||||
snprintf(response_msg, sizeof(response_msg),
|
||||
"📊 Web of Trust Status\n"
|
||||
"━━━━━━━━━━━━━━━━━━━━━━\n"
|
||||
"\n"
|
||||
"Level: %d\n"
|
||||
"%s\n"
|
||||
"\n"
|
||||
"Whitelisted pubkeys: %d\n"
|
||||
"\n"
|
||||
"Commands:\n"
|
||||
" wot 0/off - Disable WoT\n"
|
||||
" wot 1/write - Write-only mode\n"
|
||||
" wot 2/full - Full restriction mode\n"
|
||||
" wot sync - Force resync from kind 3\n"
|
||||
" wot status - Show this status",
|
||||
wot_level, level_str, whitelist_count);
|
||||
send_nip17_response(sender_pubkey, response_msg, NULL, 0);
|
||||
DEBUG_INFO("DM_ADMIN: WoT status displayed");
|
||||
}
|
||||
else {
|
||||
// Unknown wot subcommand
|
||||
snprintf(response_msg, sizeof(response_msg),
|
||||
"❌ Unknown WoT Command\n"
|
||||
"━━━━━━━━━━━━━━━━━━━━━━\n"
|
||||
"\n"
|
||||
"Usage: wot [command]\n"
|
||||
"\n"
|
||||
"Commands:\n"
|
||||
" 0, off - Disable WoT\n"
|
||||
" 1, write - Write-only mode\n"
|
||||
" 2, full - Full restriction mode\n"
|
||||
" sync - Force resync from kind 3\n"
|
||||
" status - Show current status");
|
||||
send_nip17_response(sender_pubkey, response_msg, NULL, 0);
|
||||
}
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
else {
|
||||
DEBUG_INFO("DM_ADMIN: Checking for confirmation or config change requests");
|
||||
// Check if it's a confirmation response (yes/no)
|
||||
@@ -652,7 +792,7 @@ int process_nip17_admin_command(cJSON* dm_event, char* error_message, size_t err
|
||||
if (cJSON_IsString(first_item) && strcmp(cJSON_GetStringValue(first_item), "stats") == 0) {
|
||||
DEBUG_INFO("DM_ADMIN: Processing JSON stats command");
|
||||
// Get sender pubkey for response
|
||||
cJSON* sender_pubkey_obj = cJSON_GetObjectItem(dm_event, "pubkey");
|
||||
cJSON* sender_pubkey_obj = cJSON_GetObjectItemCaseSensitive(dm_event, "pubkey");
|
||||
if (!sender_pubkey_obj || !cJSON_IsString(sender_pubkey_obj)) {
|
||||
cJSON_Delete(command_array);
|
||||
DEBUG_INFO("DM_ADMIN: DM missing sender pubkey for stats command");
|
||||
@@ -694,13 +834,13 @@ int process_nip17_admin_command(cJSON* dm_event, char* error_message, size_t err
|
||||
cJSON_AddStringToObject(synthetic_event, "content", dm_content);
|
||||
|
||||
// Copy pubkey from DM
|
||||
cJSON* pubkey_obj = cJSON_GetObjectItem(dm_event, "pubkey");
|
||||
cJSON* pubkey_obj = cJSON_GetObjectItemCaseSensitive(dm_event, "pubkey");
|
||||
if (pubkey_obj && cJSON_IsString(pubkey_obj)) {
|
||||
cJSON_AddStringToObject(synthetic_event, "pubkey", cJSON_GetStringValue(pubkey_obj));
|
||||
}
|
||||
|
||||
// Copy tags from DM
|
||||
cJSON* tags = cJSON_GetObjectItem(dm_event, "tags");
|
||||
cJSON* tags = cJSON_GetObjectItemCaseSensitive(dm_event, "tags");
|
||||
if (tags) {
|
||||
cJSON_AddItemToObject(synthetic_event, "tags", cJSON_Duplicate(tags, 1));
|
||||
}
|
||||
|
||||
File diff suppressed because one or more lines are too long
+603
@@ -0,0 +1,603 @@
|
||||
#define _GNU_SOURCE
|
||||
#include "ip_ban.h"
|
||||
#include "debug.h"
|
||||
#include "config.h"
|
||||
#include "db_ops.h"
|
||||
#include "thread_pool.h"
|
||||
#include <string.h>
|
||||
#include <stdlib.h>
|
||||
#include <pthread.h>
|
||||
|
||||
// ============================================================
|
||||
// IP Auth Failure Ban System
|
||||
//
|
||||
// Fixed-size open-addressing hash table. No malloc after init.
|
||||
// Thread-safe via a single mutex (low contention — only called
|
||||
// at connection open/close, not in the hot event path).
|
||||
//
|
||||
// State is persisted to the ip_bans SQLite table every 5 minutes
|
||||
// and loaded at startup so bans survive relay restarts.
|
||||
// ============================================================
|
||||
|
||||
#define IP_BAN_EMPTY 0
|
||||
#define IP_BAN_ACTIVE 1
|
||||
|
||||
typedef struct {
|
||||
int state; // IP_BAN_EMPTY or IP_BAN_ACTIVE
|
||||
char ip[46]; // IPv4 or IPv6 string
|
||||
|
||||
// Auth failure tracking (existing)
|
||||
int failure_count; // failures in current window
|
||||
time_t first_failure; // start of current failure window
|
||||
time_t banned_until; // 0 = not banned
|
||||
int ban_count; // escalation level (for exponential backoff)
|
||||
|
||||
// NEW: Idle failure tracking (separate)
|
||||
int idle_failure_count;
|
||||
time_t idle_first_failure;
|
||||
time_t idle_banned_until;
|
||||
int idle_ban_count;
|
||||
|
||||
// Other existing fields
|
||||
int has_authed_successfully; // 1 if this IP has ever authenticated
|
||||
time_t last_success_at; // timestamp of last successful auth
|
||||
int total_connections; // lifetime connection count
|
||||
int total_failures; // lifetime auth failure count
|
||||
int total_successes; // lifetime successful auth count
|
||||
time_t first_seen; // when this IP was first seen
|
||||
} ip_ban_entry_t;
|
||||
|
||||
static ip_ban_entry_t g_ban_table[IP_BAN_TABLE_SIZE];
|
||||
static pthread_mutex_t g_ban_mutex = PTHREAD_MUTEX_INITIALIZER;
|
||||
static int g_initialized = 0;
|
||||
|
||||
// Simple FNV-1a hash for IP strings
|
||||
static unsigned int ip_hash(const char* ip) {
|
||||
unsigned int hash = 2166136261u;
|
||||
while (*ip) {
|
||||
hash ^= (unsigned char)*ip++;
|
||||
hash *= 16777619u;
|
||||
}
|
||||
return hash % IP_BAN_TABLE_SIZE;
|
||||
}
|
||||
|
||||
// Find slot for IP (open addressing with linear probing)
|
||||
static int find_slot(const char* ip) {
|
||||
unsigned int start = ip_hash(ip);
|
||||
for (unsigned int i = 0; i < IP_BAN_TABLE_SIZE; i++) {
|
||||
unsigned int idx = (start + i) % IP_BAN_TABLE_SIZE;
|
||||
if (g_ban_table[idx].state == IP_BAN_EMPTY) {
|
||||
return (int)idx;
|
||||
}
|
||||
if (strcmp(g_ban_table[idx].ip, ip) == 0) {
|
||||
return (int)idx;
|
||||
}
|
||||
}
|
||||
return -1;
|
||||
}
|
||||
|
||||
// Get or create an entry for an IP. Returns NULL if table is full.
|
||||
// Caller must hold g_ban_mutex.
|
||||
static ip_ban_entry_t* get_or_create_entry(const char* ip) {
|
||||
int idx = find_slot(ip);
|
||||
if (idx < 0) {
|
||||
DEBUG_WARN("IP ban table full, cannot track %s", ip);
|
||||
return NULL;
|
||||
}
|
||||
ip_ban_entry_t* entry = &g_ban_table[idx];
|
||||
if (entry->state == IP_BAN_EMPTY) {
|
||||
entry->state = IP_BAN_ACTIVE;
|
||||
strncpy(entry->ip, ip, sizeof(entry->ip) - 1);
|
||||
entry->ip[sizeof(entry->ip) - 1] = '\0';
|
||||
entry->first_seen = time(NULL);
|
||||
}
|
||||
return entry;
|
||||
}
|
||||
|
||||
void ip_ban_init(void) {
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
memset(g_ban_table, 0, sizeof(g_ban_table));
|
||||
g_initialized = 1;
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
DEBUG_LOG("IP ban table initialized (%d slots)", IP_BAN_TABLE_SIZE);
|
||||
}
|
||||
|
||||
void ip_ban_load_from_db(void) {
|
||||
if (!db_is_available() || !g_initialized) return;
|
||||
|
||||
// Create table if it doesn't exist (handles existing databases)
|
||||
const char* create_sql =
|
||||
"CREATE TABLE IF NOT EXISTS ip_bans ("
|
||||
" ip TEXT PRIMARY KEY,"
|
||||
" failure_count INTEGER NOT NULL DEFAULT 0,"
|
||||
" ban_count INTEGER NOT NULL DEFAULT 0,"
|
||||
" banned_until INTEGER NOT NULL DEFAULT 0,"
|
||||
" first_failure INTEGER NOT NULL DEFAULT 0,"
|
||||
" has_authed_successfully INTEGER NOT NULL DEFAULT 0,"
|
||||
" last_success_at INTEGER NOT NULL DEFAULT 0,"
|
||||
" total_connections INTEGER NOT NULL DEFAULT 0,"
|
||||
" total_failures INTEGER NOT NULL DEFAULT 0,"
|
||||
" total_successes INTEGER NOT NULL DEFAULT 0,"
|
||||
" first_seen INTEGER NOT NULL DEFAULT 0,"
|
||||
" updated_at INTEGER NOT NULL DEFAULT (strftime('%s', 'now'))"
|
||||
");";
|
||||
if (db_exec_sql(create_sql) != 0) {
|
||||
DEBUG_ERROR("Failed to create ip_bans table: %s", db_last_error());
|
||||
return;
|
||||
}
|
||||
|
||||
// Migration: Add idle_* columns if they don't exist (ignore errors if already exists)
|
||||
(void)db_exec_sql("ALTER TABLE ip_bans ADD COLUMN idle_failure_count INTEGER NOT NULL DEFAULT 0");
|
||||
(void)db_exec_sql("ALTER TABLE ip_bans ADD COLUMN idle_ban_count INTEGER NOT NULL DEFAULT 0");
|
||||
(void)db_exec_sql("ALTER TABLE ip_bans ADD COLUMN idle_banned_until INTEGER NOT NULL DEFAULT 0");
|
||||
(void)db_exec_sql("ALTER TABLE ip_bans ADD COLUMN idle_first_failure INTEGER NOT NULL DEFAULT 0");
|
||||
|
||||
const char* sql =
|
||||
"SELECT ip, failure_count, ban_count, banned_until, first_failure,"
|
||||
" has_authed_successfully, last_success_at, total_connections,"
|
||||
" total_failures, total_successes, first_seen,"
|
||||
" idle_failure_count, idle_ban_count, idle_banned_until, idle_first_failure"
|
||||
" FROM ip_bans";
|
||||
|
||||
db_stmt_t* stmt;
|
||||
if (db_prepare(sql, &stmt) != DB_OK) {
|
||||
DEBUG_ERROR("Failed to prepare ip_bans load query");
|
||||
return;
|
||||
}
|
||||
|
||||
int loaded = 0;
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
while (db_step_stmt(stmt) == DB_ROW) {
|
||||
const char* ip = db_column_text_value(stmt, 0);
|
||||
if (!ip) continue;
|
||||
|
||||
int idx = find_slot(ip);
|
||||
if (idx < 0) continue;
|
||||
|
||||
ip_ban_entry_t* entry = &g_ban_table[idx];
|
||||
entry->state = IP_BAN_ACTIVE;
|
||||
strncpy(entry->ip, ip, sizeof(entry->ip) - 1);
|
||||
entry->ip[sizeof(entry->ip) - 1] = '\0';
|
||||
entry->failure_count = db_column_int_value(stmt, 1);
|
||||
entry->ban_count = db_column_int_value(stmt, 2);
|
||||
entry->banned_until = (time_t)db_column_int64_value(stmt, 3);
|
||||
entry->first_failure = (time_t)db_column_int64_value(stmt, 4);
|
||||
entry->has_authed_successfully = db_column_int_value(stmt, 5);
|
||||
entry->last_success_at = (time_t)db_column_int64_value(stmt, 6);
|
||||
entry->total_connections = db_column_int_value(stmt, 7);
|
||||
entry->total_failures = db_column_int_value(stmt, 8);
|
||||
entry->total_successes = db_column_int_value(stmt, 9);
|
||||
entry->first_seen = (time_t)db_column_int64_value(stmt, 10);
|
||||
// Load idle tracking fields (default to 0 if columns don't exist yet)
|
||||
entry->idle_failure_count = db_column_int_value(stmt, 11);
|
||||
entry->idle_ban_count = db_column_int_value(stmt, 12);
|
||||
entry->idle_banned_until = (time_t)db_column_int64_value(stmt, 13);
|
||||
entry->idle_first_failure = (time_t)db_column_int64_value(stmt, 14);
|
||||
loaded++;
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
db_finalize_stmt(stmt);
|
||||
|
||||
// Count how many are still actively banned
|
||||
time_t now = time(NULL);
|
||||
int still_banned = 0;
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
for (int i = 0; i < IP_BAN_TABLE_SIZE; i++) {
|
||||
if (g_ban_table[i].state == IP_BAN_ACTIVE &&
|
||||
g_ban_table[i].banned_until > now) {
|
||||
still_banned++;
|
||||
}
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
|
||||
DEBUG_WARN("IP ban table loaded: %d IPs restored (%d still banned)", loaded, still_banned);
|
||||
}
|
||||
|
||||
void ip_ban_save_to_db(void) {
|
||||
if (!db_is_available() || !g_initialized) return;
|
||||
|
||||
const char* upsert_sql =
|
||||
"INSERT OR REPLACE INTO ip_bans"
|
||||
" (ip, failure_count, ban_count, banned_until, first_failure,"
|
||||
" has_authed_successfully, last_success_at, total_connections,"
|
||||
" total_failures, total_successes, first_seen, updated_at,"
|
||||
" idle_failure_count, idle_ban_count, idle_banned_until, idle_first_failure)"
|
||||
" VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, strftime('%s', 'now'), ?, ?, ?, ?)";
|
||||
|
||||
db_stmt_t* stmt;
|
||||
if (db_prepare(upsert_sql, &stmt) != DB_OK) {
|
||||
DEBUG_ERROR("Failed to prepare ip_bans save query");
|
||||
return;
|
||||
}
|
||||
|
||||
(void)db_exec_sql("BEGIN TRANSACTION");
|
||||
|
||||
int saved = 0;
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
for (int i = 0; i < IP_BAN_TABLE_SIZE; i++) {
|
||||
ip_ban_entry_t* entry = &g_ban_table[i];
|
||||
if (entry->state != IP_BAN_ACTIVE) continue;
|
||||
|
||||
(void)db_reset_stmt(stmt);
|
||||
db_bind_text_param(stmt, 1, entry->ip);
|
||||
db_bind_int_param(stmt, 2, entry->failure_count);
|
||||
db_bind_int_param(stmt, 3, entry->ban_count);
|
||||
db_bind_int64_param(stmt, 4, (long long)entry->banned_until);
|
||||
db_bind_int64_param(stmt, 5, (long long)entry->first_failure);
|
||||
db_bind_int_param(stmt, 6, entry->has_authed_successfully);
|
||||
db_bind_int64_param(stmt, 7, (long long)entry->last_success_at);
|
||||
db_bind_int_param(stmt, 8, entry->total_connections);
|
||||
db_bind_int_param(stmt, 9, entry->total_failures);
|
||||
db_bind_int_param(stmt, 10, entry->total_successes);
|
||||
db_bind_int64_param(stmt, 11, (long long)entry->first_seen);
|
||||
// Idle tracking fields
|
||||
db_bind_int_param(stmt, 12, entry->idle_failure_count);
|
||||
db_bind_int_param(stmt, 13, entry->idle_ban_count);
|
||||
db_bind_int64_param(stmt, 14, (long long)entry->idle_banned_until);
|
||||
db_bind_int64_param(stmt, 15, (long long)entry->idle_first_failure);
|
||||
|
||||
if (db_step_stmt(stmt) != DB_DONE) {
|
||||
DEBUG_WARN("Failed to save ip_ban entry for %s", entry->ip);
|
||||
} else {
|
||||
saved++;
|
||||
}
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
|
||||
db_finalize_stmt(stmt);
|
||||
(void)db_exec_sql("COMMIT");
|
||||
|
||||
DEBUG_TRACE("IP ban table saved: %d entries written to DB", saved);
|
||||
}
|
||||
|
||||
// Check if an IP is in the idle_ban_whitelist config (comma-separated list)
|
||||
static int ip_is_whitelisted(const char* ip) {
|
||||
if (!ip) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Always trust loopback to avoid local test/dev self-bans.
|
||||
if (strcmp(ip, "127.0.0.1") == 0 || strcmp(ip, "::1") == 0) {
|
||||
return 1;
|
||||
}
|
||||
|
||||
const char* whitelist = get_config_value("idle_ban_whitelist");
|
||||
if (!whitelist || whitelist[0] == '\0') {
|
||||
if (whitelist) free((char*)whitelist);
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Make a mutable copy to tokenize
|
||||
char buf[1024];
|
||||
strncpy(buf, whitelist, sizeof(buf) - 1);
|
||||
buf[sizeof(buf) - 1] = '\0';
|
||||
|
||||
int is_match = 0;
|
||||
char* token = strtok(buf, ",");
|
||||
while (token) {
|
||||
// Trim leading/trailing spaces
|
||||
while (*token == ' ') token++;
|
||||
char* end = token + strlen(token) - 1;
|
||||
while (end > token && *end == ' ') { *end = '\0'; end--; }
|
||||
if (strcmp(token, ip) == 0) {
|
||||
is_match = 1;
|
||||
break;
|
||||
}
|
||||
token = strtok(NULL, ",");
|
||||
}
|
||||
|
||||
free((char*)whitelist);
|
||||
return is_match;
|
||||
}
|
||||
|
||||
int ip_ban_is_banned(const char* ip) {
|
||||
if (!ip || !g_initialized) return 0;
|
||||
|
||||
// Whitelisted IPs are never banned
|
||||
if (ip_is_whitelisted(ip)) return 0;
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
int idx = find_slot(ip);
|
||||
if (idx < 0 || g_ban_table[idx].state == IP_BAN_EMPTY) {
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return 0;
|
||||
}
|
||||
|
||||
ip_ban_entry_t* entry = &g_ban_table[idx];
|
||||
time_t now = time(NULL);
|
||||
int banned = 0;
|
||||
|
||||
// Check auth ban (if enabled)
|
||||
if (get_config_bool("auth_fail_ban_enabled", 1) &&
|
||||
entry->banned_until > 0 && now < entry->banned_until) {
|
||||
banned = 1;
|
||||
}
|
||||
|
||||
// Check idle ban (if enabled)
|
||||
if (get_config_bool("idle_ban_enabled", 1) &&
|
||||
entry->idle_banned_until > 0 && now < entry->idle_banned_until) {
|
||||
banned = 1;
|
||||
}
|
||||
|
||||
// Clear expired bans
|
||||
if (!banned) {
|
||||
if (entry->banned_until > 0 && now >= entry->banned_until) {
|
||||
entry->banned_until = 0;
|
||||
entry->failure_count = 0;
|
||||
entry->first_failure = 0;
|
||||
}
|
||||
if (entry->idle_banned_until > 0 && now >= entry->idle_banned_until) {
|
||||
entry->idle_banned_until = 0;
|
||||
entry->idle_failure_count = 0;
|
||||
entry->idle_first_failure = 0;
|
||||
}
|
||||
}
|
||||
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return banned;
|
||||
}
|
||||
|
||||
void ip_ban_record_connection(const char* ip) {
|
||||
if (!ip || !g_initialized) return;
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
ip_ban_entry_t* entry = get_or_create_entry(ip);
|
||||
if (entry) {
|
||||
entry->total_connections++;
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
}
|
||||
|
||||
void ip_ban_record_failure(const char* ip) {
|
||||
if (!ip || !g_initialized) return;
|
||||
if (!get_config_bool("auth_fail_ban_enabled", 1)) return;
|
||||
|
||||
int threshold = get_config_int("auth_fail_ban_threshold", 3);
|
||||
int window_sec = get_config_int("auth_fail_window_sec", 60);
|
||||
int ban_duration = get_config_int("auth_fail_ban_duration_sec", 300);
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
ip_ban_entry_t* entry = get_or_create_entry(ip);
|
||||
if (!entry) {
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return;
|
||||
}
|
||||
|
||||
time_t now = time(NULL);
|
||||
|
||||
// Reset window if expired
|
||||
if (entry->first_failure > 0 && (now - entry->first_failure) > window_sec) {
|
||||
entry->failure_count = 0;
|
||||
entry->first_failure = now;
|
||||
}
|
||||
if (entry->first_failure == 0) {
|
||||
entry->first_failure = now;
|
||||
}
|
||||
|
||||
entry->failure_count++;
|
||||
entry->total_failures++;
|
||||
|
||||
DEBUG_TRACE("IP %s auth failure count: %d/%d", ip, entry->failure_count, threshold);
|
||||
|
||||
if (entry->failure_count >= threshold) {
|
||||
int duration = ban_duration;
|
||||
for (int i = 0; i < entry->ban_count && duration < 86400; i++) {
|
||||
duration *= 2;
|
||||
}
|
||||
if (duration > 86400) duration = 86400;
|
||||
|
||||
entry->banned_until = now + duration;
|
||||
entry->ban_count++;
|
||||
entry->failure_count = 0;
|
||||
entry->first_failure = 0;
|
||||
|
||||
DEBUG_WARN("IP %s banned for %d seconds (ban #%d) after %d auth failures",
|
||||
ip, duration, entry->ban_count, threshold);
|
||||
}
|
||||
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
}
|
||||
|
||||
// Record an idle/early-disconnect failure for an IP
|
||||
void ip_ban_record_idle_failure(const char* ip) {
|
||||
if (!ip || !g_initialized) return;
|
||||
if (!get_config_bool("idle_ban_enabled", 1)) return;
|
||||
if (ip_is_whitelisted(ip)) return; // Never record idle failures for whitelisted IPs
|
||||
|
||||
int threshold = get_config_int("idle_ban_threshold", 1);
|
||||
int window_sec = get_config_int("idle_ban_window_sec", 30);
|
||||
int ban_duration = get_config_int("idle_ban_duration_sec", 300);
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
ip_ban_entry_t* entry = get_or_create_entry(ip);
|
||||
if (!entry) {
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return;
|
||||
}
|
||||
|
||||
time_t now = time(NULL);
|
||||
|
||||
// Reset window if expired
|
||||
if (entry->idle_first_failure > 0 && (now - entry->idle_first_failure) > window_sec) {
|
||||
entry->idle_failure_count = 0;
|
||||
entry->idle_first_failure = now;
|
||||
}
|
||||
if (entry->idle_first_failure == 0) {
|
||||
entry->idle_first_failure = now;
|
||||
}
|
||||
|
||||
entry->idle_failure_count++;
|
||||
entry->total_failures++;
|
||||
|
||||
DEBUG_TRACE("IP %s idle failure count: %d/%d", ip, entry->idle_failure_count, threshold);
|
||||
|
||||
if (entry->idle_failure_count >= threshold) {
|
||||
int duration = ban_duration;
|
||||
for (int i = 0; i < entry->idle_ban_count && duration < 86400; i++) {
|
||||
duration *= 2;
|
||||
}
|
||||
if (duration > 86400) duration = 86400;
|
||||
|
||||
entry->idle_banned_until = now + duration;
|
||||
entry->idle_ban_count++;
|
||||
entry->idle_failure_count = 0;
|
||||
entry->idle_first_failure = 0;
|
||||
|
||||
DEBUG_WARN("IP %s banned for %d seconds (idle ban #%d) after %d idle failures",
|
||||
ip, duration, entry->idle_ban_count, threshold);
|
||||
}
|
||||
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
}
|
||||
|
||||
void ip_ban_record_success(const char* ip) {
|
||||
if (!ip || !g_initialized) return;
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
ip_ban_entry_t* entry = get_or_create_entry(ip);
|
||||
if (entry) {
|
||||
entry->failure_count = 0;
|
||||
entry->first_failure = 0;
|
||||
entry->has_authed_successfully = 1;
|
||||
entry->last_success_at = time(NULL);
|
||||
entry->total_successes++;
|
||||
DEBUG_TRACE("IP %s authenticated successfully — failure count cleared", ip);
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
}
|
||||
|
||||
void ip_ban_cleanup(void) {
|
||||
if (!g_initialized) return;
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
time_t now = time(NULL);
|
||||
int window_sec = get_config_int("auth_fail_window_sec", 60);
|
||||
int idle_window_sec = get_config_int("idle_ban_window_sec", 60);
|
||||
int cleaned = 0;
|
||||
|
||||
for (int i = 0; i < IP_BAN_TABLE_SIZE; i++) {
|
||||
if (g_ban_table[i].state != IP_BAN_ACTIVE) continue;
|
||||
ip_ban_entry_t* entry = &g_ban_table[i];
|
||||
|
||||
// Check auth failure window expiration
|
||||
int auth_ban_expired = (entry->banned_until == 0 || now >= entry->banned_until);
|
||||
int auth_window_expired = (entry->first_failure == 0 || (now - entry->first_failure) > window_sec * 10);
|
||||
|
||||
// Check idle failure window expiration
|
||||
int idle_ban_expired = (entry->idle_banned_until == 0 || now >= entry->idle_banned_until);
|
||||
int idle_window_expired = (entry->idle_first_failure == 0 || (now - entry->idle_first_failure) > idle_window_sec * 10);
|
||||
|
||||
if (auth_ban_expired && auth_window_expired && entry->failure_count == 0 &&
|
||||
idle_ban_expired && idle_window_expired && entry->idle_failure_count == 0) {
|
||||
int retain_sec = 86400; // 24 hours
|
||||
int last_auth_ban_expired_long_ago = (entry->banned_until == 0 ||
|
||||
(now - entry->banned_until) > retain_sec);
|
||||
int last_idle_ban_expired_long_ago = (entry->idle_banned_until == 0 ||
|
||||
(now - entry->idle_banned_until) > retain_sec);
|
||||
|
||||
if (last_auth_ban_expired_long_ago && last_idle_ban_expired_long_ago &&
|
||||
!entry->has_authed_successfully &&
|
||||
entry->ban_count == 0 && entry->idle_ban_count == 0 &&
|
||||
entry->total_connections <= 1) {
|
||||
// Fully clean — never banned, never authenticated, only seen once
|
||||
memset(entry, 0, sizeof(ip_ban_entry_t));
|
||||
cleaned++;
|
||||
} else {
|
||||
// Keep entry permanently — preserve ban_count for escalation.
|
||||
// An IP that has been banned before will always get at least a 24-hour ban
|
||||
// if it fails auth again, regardless of how long it has been away.
|
||||
entry->failure_count = 0;
|
||||
entry->first_failure = 0;
|
||||
entry->idle_failure_count = 0;
|
||||
entry->idle_first_failure = 0;
|
||||
if (last_auth_ban_expired_long_ago) {
|
||||
entry->banned_until = 0;
|
||||
// ban_count intentionally NOT reset — permanent escalation
|
||||
}
|
||||
if (last_idle_ban_expired_long_ago) {
|
||||
entry->idle_banned_until = 0;
|
||||
// idle_ban_count intentionally NOT reset — permanent escalation
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (cleaned > 0) {
|
||||
DEBUG_TRACE("IP ban cleanup: freed %d stale entries", cleaned);
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
}
|
||||
|
||||
int ip_ban_get_banned_count(void) {
|
||||
if (!g_initialized) return 0;
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
time_t now = time(NULL);
|
||||
int count = 0;
|
||||
for (int i = 0; i < IP_BAN_TABLE_SIZE; i++) {
|
||||
if (g_ban_table[i].state != IP_BAN_ACTIVE) continue;
|
||||
// Count if either auth banned or idle banned
|
||||
if (g_ban_table[i].banned_until > now ||
|
||||
g_ban_table[i].idle_banned_until > now) {
|
||||
count++;
|
||||
}
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return count;
|
||||
}
|
||||
|
||||
int ip_ban_get_tracked_count(void) {
|
||||
if (!g_initialized) return 0;
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
int count = 0;
|
||||
for (int i = 0; i < IP_BAN_TABLE_SIZE; i++) {
|
||||
if (g_ban_table[i].state == IP_BAN_ACTIVE) count++;
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return count;
|
||||
}
|
||||
|
||||
void ip_ban_log_stats(void) {
|
||||
if (!g_initialized) return;
|
||||
|
||||
static time_t last_log = 0;
|
||||
time_t now = time(NULL);
|
||||
|
||||
if (now - last_log < 300) return;
|
||||
last_log = now;
|
||||
|
||||
// Save to DB every 5 minutes
|
||||
if (thread_pool_is_running()) {
|
||||
thread_pool_job_t job;
|
||||
memset(&job, 0, sizeof(job));
|
||||
job.type = THREAD_POOL_JOB_IP_BAN_SAVE;
|
||||
|
||||
thread_pool_status_t rc = thread_pool_submit_write(&job, NULL);
|
||||
if (rc != THREAD_POOL_STATUS_OK) {
|
||||
DEBUG_WARN("Failed to queue IP_BAN_SAVE job (%d); saving synchronously", rc);
|
||||
ip_ban_save_to_db();
|
||||
}
|
||||
} else {
|
||||
ip_ban_save_to_db();
|
||||
}
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
int auth_banned_count = 0;
|
||||
int idle_banned_count = 0;
|
||||
int tracked_count = 0;
|
||||
int trusted_count = 0;
|
||||
|
||||
for (int i = 0; i < IP_BAN_TABLE_SIZE; i++) {
|
||||
if (g_ban_table[i].state != IP_BAN_ACTIVE) continue;
|
||||
tracked_count++;
|
||||
if (g_ban_table[i].banned_until > now) auth_banned_count++;
|
||||
if (g_ban_table[i].idle_banned_until > now) idle_banned_count++;
|
||||
if (g_ban_table[i].has_authed_successfully) trusted_count++;
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
|
||||
int total_banned = auth_banned_count + idle_banned_count;
|
||||
if (total_banned > 0 || tracked_count > 0) {
|
||||
DEBUG_WARN("IP BAN SUMMARY: %d auth-banned, %d idle-banned, %d tracked, %d trusted (ever authed)",
|
||||
auth_banned_count, idle_banned_count, tracked_count, trusted_count);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
#ifndef IP_BAN_H
|
||||
#define IP_BAN_H
|
||||
|
||||
// IP Auth Failure Ban System
|
||||
//
|
||||
// Tracks auth failures per IP address and temporarily bans IPs that repeatedly
|
||||
// fail NIP-42 authentication. Uses an in-memory fixed-size hash table — no
|
||||
// database writes on the hot path. State is persisted to the ip_bans table
|
||||
// every 5 minutes and loaded at startup.
|
||||
//
|
||||
// Config keys (all read from the config table at runtime):
|
||||
// auth_fail_ban_enabled bool default: true (0 = disabled)
|
||||
// auth_fail_ban_threshold int default: 3 (failures before ban)
|
||||
// auth_fail_window_sec int default: 60 (window to count failures)
|
||||
// auth_fail_ban_duration_sec int default: 300 (initial ban duration, doubles each time)
|
||||
|
||||
#include <time.h>
|
||||
|
||||
// Maximum number of IPs tracked simultaneously (fixed-size, no malloc)
|
||||
#define IP_BAN_TABLE_SIZE 4096
|
||||
|
||||
// Initialize the IP ban table (call once at startup, before loading from DB)
|
||||
void ip_ban_init(void);
|
||||
|
||||
// Load ban state from the ip_bans database table.
|
||||
// Call after ip_ban_init() and after the database is open.
|
||||
void ip_ban_load_from_db(void);
|
||||
|
||||
// Save current ban state to the ip_bans database table.
|
||||
// Called every 5 minutes from the maintenance timer.
|
||||
void ip_ban_save_to_db(void);
|
||||
|
||||
// Check if an IP is currently banned.
|
||||
// Returns 1 if banned (connection should be rejected), 0 if allowed.
|
||||
int ip_ban_is_banned(const char* ip);
|
||||
|
||||
// Record an auth failure for an IP.
|
||||
// Called when a connection is closed due to auth timeout.
|
||||
// May trigger a ban if the threshold is exceeded.
|
||||
void ip_ban_record_failure(const char* ip);
|
||||
|
||||
// Record an idle/early-disconnect failure for an IP.
|
||||
// Called when a connection closes without ever sending REQ or EVENT.
|
||||
// May trigger a ban if the threshold is exceeded.
|
||||
void ip_ban_record_idle_failure(const char* ip);
|
||||
|
||||
// Record a successful auth for an IP.
|
||||
// Sets has_authed_successfully=1 and clears failure count.
|
||||
void ip_ban_record_success(const char* ip);
|
||||
|
||||
// Record a new connection from an IP (increment total_connections).
|
||||
void ip_ban_record_connection(const char* ip);
|
||||
|
||||
// Periodic cleanup: expire old entries (call from the connection age checker).
|
||||
void ip_ban_cleanup(void);
|
||||
|
||||
// Emit a periodic WARN-level log summary of banned IPs (every 5 minutes).
|
||||
// Also saves state to DB.
|
||||
void ip_ban_log_stats(void);
|
||||
|
||||
// Get stats for logging/monitoring
|
||||
int ip_ban_get_banned_count(void);
|
||||
int ip_ban_get_tracked_count(void);
|
||||
|
||||
#endif // IP_BAN_H
|
||||
+840
-410
File diff suppressed because it is too large
Load Diff
+25
-7
@@ -1,5 +1,5 @@
|
||||
/*
|
||||
* C-Relay Main Header - Version and Metadata Information
|
||||
* C-Relay-PG Main Header - Version and Metadata Information
|
||||
*
|
||||
* This header contains version information and relay metadata.
|
||||
* Version macros are auto-updated by the build system.
|
||||
@@ -11,16 +11,16 @@
|
||||
|
||||
// Version information (auto-updated by build system)
|
||||
// Using CRELAY_ prefix to avoid conflicts with nostr_core_lib VERSION macros
|
||||
#define CRELAY_VERSION_MAJOR 1
|
||||
#define CRELAY_VERSION_MINOR 2
|
||||
#define CRELAY_VERSION_PATCH 1
|
||||
#define CRELAY_VERSION "v1.2.1"
|
||||
#define CRELAY_VERSION_MAJOR 2
|
||||
#define CRELAY_VERSION_MINOR 1
|
||||
#define CRELAY_VERSION_PATCH 9
|
||||
#define CRELAY_VERSION "v2.1.9"
|
||||
|
||||
// Relay metadata (authoritative source for NIP-11 information)
|
||||
#define RELAY_NAME "C-Relay"
|
||||
#define RELAY_NAME "C-Relay-PG"
|
||||
#define RELAY_DESCRIPTION "High-performance C Nostr relay with SQLite storage"
|
||||
#define RELAY_CONTACT ""
|
||||
#define RELAY_SOFTWARE "https://git.laantungir.net/laantungir/c-relay.git"
|
||||
#define RELAY_SOFTWARE "https://git.laantungir.net/laantungir/c-relay-pg.git"
|
||||
#define RELAY_VERSION CRELAY_VERSION // Use the same version as the build
|
||||
#define SUPPORTED_NIPS "1,2,4,9,11,12,13,15,16,20,22,33,40,42,50,70"
|
||||
#define LANGUAGE_TAGS ""
|
||||
@@ -28,4 +28,22 @@
|
||||
#define POSTING_POLICY ""
|
||||
#define PAYMENTS_URL ""
|
||||
|
||||
// Forward declaration to avoid pulling cJSON headers into all includers.
|
||||
typedef struct cJSON cJSON;
|
||||
|
||||
// Async REQ handler status used by websocket callback to defer EOSE until worker completion
|
||||
#define HANDLE_REQ_ASYNC_PENDING (-2)
|
||||
|
||||
// Async EVENT storage split:
|
||||
// - store_event_core(): worker-safe core DB write path
|
||||
// returns 0 when inserted, 1 when handled without insert (duplicate/ephemeral), -1 on error
|
||||
// - store_event_post_actions(): main-thread-only follow-up actions (monitoring/tags/WoT sync)
|
||||
// - store_event(): legacy wrapper preserving original behavior for synchronous call sites
|
||||
int store_event_core(cJSON* event);
|
||||
void store_event_post_actions(cJSON* event);
|
||||
int store_event(cJSON* event);
|
||||
|
||||
// Drains completed async REQ jobs and queues EVENT/EOSE on the lws service thread.
|
||||
void process_req_async_completions(void);
|
||||
|
||||
#endif /* MAIN_H */
|
||||
|
||||
+39
-95
@@ -7,7 +7,7 @@
|
||||
/////////////////////////////////////////////////////////////////////////////////////////
|
||||
#include <cjson/cJSON.h>
|
||||
#include "debug.h"
|
||||
#include <sqlite3.h>
|
||||
#include "db_ops.h"
|
||||
#include <string.h>
|
||||
#include <stdlib.h>
|
||||
#include <time.h>
|
||||
@@ -21,10 +21,6 @@ int store_event(cJSON* event);
|
||||
int delete_events_by_id(const char* requester_pubkey, cJSON* event_ids);
|
||||
int delete_events_by_address(const char* requester_pubkey, cJSON* addresses, long deletion_timestamp);
|
||||
|
||||
// Global database variable
|
||||
extern sqlite3* g_db;
|
||||
|
||||
|
||||
// Handle NIP-09 deletion request event (kind 5)
|
||||
int handle_deletion_request(cJSON* event, char* error_message, size_t error_size) {
|
||||
if (!event) {
|
||||
@@ -33,12 +29,12 @@ int handle_deletion_request(cJSON* event, char* error_message, size_t error_size
|
||||
}
|
||||
|
||||
// Extract event details
|
||||
cJSON* kind_obj = cJSON_GetObjectItem(event, "kind");
|
||||
cJSON* pubkey_obj = cJSON_GetObjectItem(event, "pubkey");
|
||||
cJSON* created_at_obj = cJSON_GetObjectItem(event, "created_at");
|
||||
cJSON* tags_obj = cJSON_GetObjectItem(event, "tags");
|
||||
cJSON* content_obj = cJSON_GetObjectItem(event, "content");
|
||||
cJSON* event_id_obj = cJSON_GetObjectItem(event, "id");
|
||||
cJSON* kind_obj = cJSON_GetObjectItemCaseSensitive(event, "kind");
|
||||
cJSON* pubkey_obj = cJSON_GetObjectItemCaseSensitive(event, "pubkey");
|
||||
cJSON* created_at_obj = cJSON_GetObjectItemCaseSensitive(event, "created_at");
|
||||
cJSON* tags_obj = cJSON_GetObjectItemCaseSensitive(event, "tags");
|
||||
cJSON* content_obj = cJSON_GetObjectItemCaseSensitive(event, "content");
|
||||
cJSON* event_id_obj = cJSON_GetObjectItemCaseSensitive(event, "id");
|
||||
|
||||
if (!kind_obj || !pubkey_obj || !created_at_obj || !tags_obj || !event_id_obj) {
|
||||
snprintf(error_message, error_size, "invalid: incomplete deletion request");
|
||||
@@ -146,97 +142,72 @@ int handle_deletion_request(cJSON* event, char* error_message, size_t error_size
|
||||
|
||||
// Delete events by ID (with pubkey authorization)
|
||||
int delete_events_by_id(const char* requester_pubkey, cJSON* event_ids) {
|
||||
if (!g_db || !requester_pubkey || !event_ids || !cJSON_IsArray(event_ids)) {
|
||||
if (!db_is_available() || !requester_pubkey || !event_ids || !cJSON_IsArray(event_ids)) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
||||
int deleted_count = 0;
|
||||
|
||||
|
||||
cJSON* event_id = NULL;
|
||||
cJSON_ArrayForEach(event_id, event_ids) {
|
||||
if (!cJSON_IsString(event_id)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
|
||||
const char* id = cJSON_GetStringValue(event_id);
|
||||
|
||||
// First check if event exists and if requester is authorized
|
||||
const char* check_sql = "SELECT pubkey FROM events WHERE id = ?";
|
||||
sqlite3_stmt* check_stmt;
|
||||
|
||||
int rc = sqlite3_prepare_v2(g_db, check_sql, -1, &check_stmt, NULL);
|
||||
if (rc != SQLITE_OK) {
|
||||
char event_pubkey[65] = {0};
|
||||
int exists = db_get_event_pubkey(id, event_pubkey, sizeof(event_pubkey));
|
||||
if (exists <= 0) {
|
||||
continue;
|
||||
}
|
||||
|
||||
sqlite3_bind_text(check_stmt, 1, id, -1, SQLITE_STATIC);
|
||||
|
||||
if (sqlite3_step(check_stmt) == SQLITE_ROW) {
|
||||
const char* event_pubkey = (char*)sqlite3_column_text(check_stmt, 0);
|
||||
|
||||
// Only delete if the requester is the author
|
||||
if (event_pubkey && strcmp(event_pubkey, requester_pubkey) == 0) {
|
||||
sqlite3_finalize(check_stmt);
|
||||
|
||||
// Delete the event
|
||||
const char* delete_sql = "DELETE FROM events WHERE id = ? AND pubkey = ?";
|
||||
sqlite3_stmt* delete_stmt;
|
||||
|
||||
rc = sqlite3_prepare_v2(g_db, delete_sql, -1, &delete_stmt, NULL);
|
||||
if (rc == SQLITE_OK) {
|
||||
sqlite3_bind_text(delete_stmt, 1, id, -1, SQLITE_STATIC);
|
||||
sqlite3_bind_text(delete_stmt, 2, requester_pubkey, -1, SQLITE_STATIC);
|
||||
|
||||
if (sqlite3_step(delete_stmt) == SQLITE_DONE && sqlite3_changes(g_db) > 0) {
|
||||
deleted_count++;
|
||||
}
|
||||
sqlite3_finalize(delete_stmt);
|
||||
}
|
||||
} else {
|
||||
sqlite3_finalize(check_stmt);
|
||||
char warning_msg[128];
|
||||
snprintf(warning_msg, sizeof(warning_msg), "Unauthorized deletion attempt for event: %.16s...", id);
|
||||
DEBUG_WARN(warning_msg);
|
||||
|
||||
// Only delete if the requester is the author
|
||||
if (strcmp(event_pubkey, requester_pubkey) == 0) {
|
||||
int changes = db_delete_event_by_id(id, requester_pubkey);
|
||||
if (changes > 0) {
|
||||
deleted_count += changes;
|
||||
}
|
||||
} else {
|
||||
sqlite3_finalize(check_stmt);
|
||||
char warning_msg[128];
|
||||
snprintf(warning_msg, sizeof(warning_msg), "Unauthorized deletion attempt for event: %.16s...", id);
|
||||
DEBUG_WARN(warning_msg);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
return deleted_count;
|
||||
}
|
||||
|
||||
// Delete events by addressable reference (kind:pubkey:d-identifier)
|
||||
int delete_events_by_address(const char* requester_pubkey, cJSON* addresses, long deletion_timestamp) {
|
||||
if (!g_db || !requester_pubkey || !addresses || !cJSON_IsArray(addresses)) {
|
||||
if (!db_is_available() || !requester_pubkey || !addresses || !cJSON_IsArray(addresses)) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
||||
int deleted_count = 0;
|
||||
|
||||
|
||||
cJSON* address = NULL;
|
||||
cJSON_ArrayForEach(address, addresses) {
|
||||
if (!cJSON_IsString(address)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
|
||||
const char* addr = cJSON_GetStringValue(address);
|
||||
|
||||
|
||||
// Parse address format: kind:pubkey:d-identifier
|
||||
char* addr_copy = strdup(addr);
|
||||
if (!addr_copy) continue;
|
||||
|
||||
|
||||
char* kind_str = strtok(addr_copy, ":");
|
||||
char* pubkey_str = strtok(NULL, ":");
|
||||
char* d_identifier = strtok(NULL, ":");
|
||||
|
||||
|
||||
if (!kind_str || !pubkey_str) {
|
||||
free(addr_copy);
|
||||
continue;
|
||||
}
|
||||
|
||||
|
||||
int kind = atoi(kind_str);
|
||||
|
||||
|
||||
// Only delete if the requester is the author
|
||||
if (strcmp(pubkey_str, requester_pubkey) != 0) {
|
||||
free(addr_copy);
|
||||
@@ -245,42 +216,15 @@ int delete_events_by_address(const char* requester_pubkey, cJSON* addresses, lon
|
||||
DEBUG_WARN(warning_msg);
|
||||
continue;
|
||||
}
|
||||
|
||||
// Build deletion query based on whether we have d-identifier
|
||||
const char* delete_sql;
|
||||
sqlite3_stmt* delete_stmt;
|
||||
|
||||
if (d_identifier && strlen(d_identifier) > 0) {
|
||||
// Delete specific addressable event with d-tag
|
||||
delete_sql = "DELETE FROM events WHERE kind = ? AND pubkey = ? AND created_at <= ? "
|
||||
"AND json_extract(tags, '$[*]') LIKE '%[\"d\",\"' || ? || '\"]%'";
|
||||
} else {
|
||||
// Delete all events of this kind by this author up to deletion timestamp
|
||||
delete_sql = "DELETE FROM events WHERE kind = ? AND pubkey = ? AND created_at <= ?";
|
||||
|
||||
int changes = db_delete_events_by_address(requester_pubkey, kind, d_identifier, deletion_timestamp);
|
||||
if (changes > 0) {
|
||||
deleted_count += changes;
|
||||
}
|
||||
|
||||
int rc = sqlite3_prepare_v2(g_db, delete_sql, -1, &delete_stmt, NULL);
|
||||
if (rc == SQLITE_OK) {
|
||||
sqlite3_bind_int(delete_stmt, 1, kind);
|
||||
sqlite3_bind_text(delete_stmt, 2, requester_pubkey, -1, SQLITE_STATIC);
|
||||
sqlite3_bind_int64(delete_stmt, 3, deletion_timestamp);
|
||||
|
||||
if (d_identifier && strlen(d_identifier) > 0) {
|
||||
sqlite3_bind_text(delete_stmt, 4, d_identifier, -1, SQLITE_STATIC);
|
||||
}
|
||||
|
||||
if (sqlite3_step(delete_stmt) == SQLITE_DONE) {
|
||||
int changes = sqlite3_changes(g_db);
|
||||
if (changes > 0) {
|
||||
deleted_count += changes;
|
||||
}
|
||||
}
|
||||
sqlite3_finalize(delete_stmt);
|
||||
}
|
||||
|
||||
|
||||
free(addr_copy);
|
||||
}
|
||||
|
||||
|
||||
return deleted_count;
|
||||
}
|
||||
|
||||
|
||||
+94
-38
@@ -8,7 +8,7 @@
|
||||
#include <libwebsockets.h>
|
||||
#include "../nostr_core_lib/cjson/cJSON.h"
|
||||
#include "config.h"
|
||||
|
||||
#include "main.h"
|
||||
|
||||
// Forward declarations for configuration functions
|
||||
const char* get_config_value(const char* key);
|
||||
@@ -23,6 +23,67 @@ int get_config_bool(const char* key, int default_value);
|
||||
#define HTTP_STATUS_INTERNAL_SERVER_ERROR 500
|
||||
|
||||
|
||||
#define NIP11_CACHE_TTL_SEC 5
|
||||
|
||||
static pthread_mutex_t g_nip11_cache_mutex = PTHREAD_MUTEX_INITIALIZER;
|
||||
static char* g_nip11_cached_json = NULL;
|
||||
static size_t g_nip11_cached_json_len = 0;
|
||||
static time_t g_nip11_cache_expires_at = 0;
|
||||
|
||||
cJSON* generate_relay_info_json();
|
||||
|
||||
static void invalidate_nip11_cache_locked(void) {
|
||||
if (g_nip11_cached_json) {
|
||||
free(g_nip11_cached_json);
|
||||
g_nip11_cached_json = NULL;
|
||||
}
|
||||
g_nip11_cached_json_len = 0;
|
||||
g_nip11_cache_expires_at = 0;
|
||||
}
|
||||
|
||||
static char* get_nip11_json_cached_dup(size_t* out_len) {
|
||||
time_t now = time(NULL);
|
||||
|
||||
pthread_mutex_lock(&g_nip11_cache_mutex);
|
||||
if (g_nip11_cached_json && now < g_nip11_cache_expires_at) {
|
||||
char* copy = strdup(g_nip11_cached_json);
|
||||
if (copy && out_len) {
|
||||
*out_len = g_nip11_cached_json_len;
|
||||
}
|
||||
pthread_mutex_unlock(&g_nip11_cache_mutex);
|
||||
return copy;
|
||||
}
|
||||
pthread_mutex_unlock(&g_nip11_cache_mutex);
|
||||
|
||||
cJSON* info_json = generate_relay_info_json();
|
||||
if (!info_json) {
|
||||
return NULL;
|
||||
}
|
||||
|
||||
char* fresh_json = cJSON_Print(info_json);
|
||||
cJSON_Delete(info_json);
|
||||
if (!fresh_json) {
|
||||
return NULL;
|
||||
}
|
||||
|
||||
size_t fresh_len = strlen(fresh_json);
|
||||
|
||||
pthread_mutex_lock(&g_nip11_cache_mutex);
|
||||
invalidate_nip11_cache_locked();
|
||||
g_nip11_cached_json = strdup(fresh_json);
|
||||
if (g_nip11_cached_json) {
|
||||
g_nip11_cached_json_len = fresh_len;
|
||||
g_nip11_cache_expires_at = now + NIP11_CACHE_TTL_SEC;
|
||||
}
|
||||
pthread_mutex_unlock(&g_nip11_cache_mutex);
|
||||
|
||||
if (out_len) {
|
||||
*out_len = fresh_len;
|
||||
}
|
||||
return fresh_json;
|
||||
}
|
||||
|
||||
|
||||
/////////////////////////////////////////////////////////////////////////////////////////
|
||||
/////////////////////////////////////////////////////////////////////////////////////////
|
||||
// NIP-11 RELAY INFORMATION DOCUMENT
|
||||
@@ -80,8 +141,9 @@ void init_relay_info() {
|
||||
|
||||
// Clean up relay information JSON objects
|
||||
void cleanup_relay_info() {
|
||||
// NIP-11 relay information is now generated dynamically from config table
|
||||
// No cleanup needed - data is fetched directly from database when requested
|
||||
pthread_mutex_lock(&g_nip11_cache_mutex);
|
||||
invalidate_nip11_cache_locked();
|
||||
pthread_mutex_unlock(&g_nip11_cache_mutex);
|
||||
}
|
||||
|
||||
// Generate NIP-11 compliant JSON document
|
||||
@@ -118,6 +180,8 @@ cJSON* generate_relay_info_json() {
|
||||
int default_limit = get_config_int("default_limit", 500);
|
||||
int min_pow_difficulty = get_config_int("pow_min_difficulty", 0);
|
||||
int admin_enabled = get_config_bool("admin_enabled", 0);
|
||||
int wot_enabled = get_config_int("wot_enabled", 0);
|
||||
int auth_enabled = get_config_bool("auth_enabled", 0);
|
||||
|
||||
// Add basic relay information
|
||||
if (relay_name && strlen(relay_name) > 0) {
|
||||
@@ -182,14 +246,14 @@ cJSON* generate_relay_info_json() {
|
||||
cJSON_AddStringToObject(info, "software", relay_software);
|
||||
free((char*)relay_software);
|
||||
} else {
|
||||
cJSON_AddStringToObject(info, "software", "https://git.laantungir.net/laantungir/c-relay.git");
|
||||
cJSON_AddStringToObject(info, "software", RELAY_SOFTWARE);
|
||||
}
|
||||
|
||||
if (relay_version && strlen(relay_version) > 0) {
|
||||
cJSON_AddStringToObject(info, "version", relay_version);
|
||||
free((char*)relay_version);
|
||||
} else {
|
||||
cJSON_AddStringToObject(info, "version", "0.2.0");
|
||||
cJSON_AddStringToObject(info, "version", RELAY_VERSION);
|
||||
}
|
||||
|
||||
// Add policies
|
||||
@@ -209,6 +273,9 @@ cJSON* generate_relay_info_json() {
|
||||
}
|
||||
|
||||
// Add server limitations
|
||||
// restricted_writes is true when WoT or auth is enabled (only trusted pubkeys can write)
|
||||
int restricted_writes = (wot_enabled > 0 || auth_enabled) ? 1 : 0;
|
||||
|
||||
cJSON* limitation = cJSON_CreateObject();
|
||||
if (limitation) {
|
||||
cJSON_AddNumberToObject(limitation, "max_message_length", max_message_length);
|
||||
@@ -218,15 +285,31 @@ cJSON* generate_relay_info_json() {
|
||||
cJSON_AddNumberToObject(limitation, "max_event_tags", max_event_tags);
|
||||
cJSON_AddNumberToObject(limitation, "max_content_length", max_content_length);
|
||||
cJSON_AddNumberToObject(limitation, "min_pow_difficulty", min_pow_difficulty);
|
||||
cJSON_AddBoolToObject(limitation, "auth_required", admin_enabled ? cJSON_True : cJSON_False);
|
||||
cJSON_AddBoolToObject(limitation, "auth_required", auth_enabled ? cJSON_True : cJSON_False);
|
||||
cJSON_AddBoolToObject(limitation, "payment_required", cJSON_False);
|
||||
cJSON_AddBoolToObject(limitation, "restricted_writes", cJSON_False);
|
||||
cJSON_AddBoolToObject(limitation, "restricted_writes", restricted_writes ? cJSON_True : cJSON_False);
|
||||
cJSON_AddNumberToObject(limitation, "created_at_lower_limit", 0);
|
||||
cJSON_AddNumberToObject(limitation, "created_at_upper_limit", 2147483647);
|
||||
cJSON_AddNumberToObject(limitation, "default_limit", default_limit);
|
||||
cJSON_AddItemToObject(info, "limitation", limitation);
|
||||
}
|
||||
|
||||
// Add Web of Trust information when enabled
|
||||
// This informs clients that the relay operates on a trust network
|
||||
if (wot_enabled > 0) {
|
||||
cJSON* wot_info = cJSON_CreateObject();
|
||||
if (wot_info) {
|
||||
cJSON_AddNumberToObject(wot_info, "level", wot_enabled);
|
||||
const char* level_desc = (wot_enabled == 1)
|
||||
? "write-only: only followed pubkeys can publish events"
|
||||
: "full: only followed pubkeys can publish and subscribe";
|
||||
cJSON_AddStringToObject(wot_info, "description", level_desc);
|
||||
cJSON_AddStringToObject(wot_info, "policy",
|
||||
"Events from pubkeys not in the relay operator's Web of Trust are rejected.");
|
||||
cJSON_AddItemToObject(info, "web_of_trust", wot_info);
|
||||
}
|
||||
}
|
||||
|
||||
// Add retention policies (empty array for now)
|
||||
cJSON* retention = cJSON_CreateArray();
|
||||
if (retention) {
|
||||
@@ -327,9 +410,10 @@ int handle_nip11_http_request(struct lws* wsi, const char* accept_header) {
|
||||
return -1; // Close connection
|
||||
}
|
||||
|
||||
// Generate relay information JSON
|
||||
cJSON* info_json = generate_relay_info_json();
|
||||
if (!info_json) {
|
||||
// Generate (or fetch cached) relay information JSON
|
||||
size_t json_len = 0;
|
||||
char* json_string = get_nip11_json_cached_dup(&json_len);
|
||||
if (!json_string) {
|
||||
DEBUG_ERROR("Failed to generate relay info JSON");
|
||||
unsigned char buf[LWS_PRE + 256];
|
||||
unsigned char *p = &buf[LWS_PRE];
|
||||
@@ -352,34 +436,6 @@ int handle_nip11_http_request(struct lws* wsi, const char* accept_header) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
char* json_string = cJSON_Print(info_json);
|
||||
cJSON_Delete(info_json);
|
||||
|
||||
if (!json_string) {
|
||||
DEBUG_ERROR("Failed to serialize relay info JSON");
|
||||
unsigned char buf[LWS_PRE + 256];
|
||||
unsigned char *p = &buf[LWS_PRE];
|
||||
unsigned char *start = p;
|
||||
unsigned char *end = &buf[sizeof(buf) - 1];
|
||||
|
||||
if (lws_add_http_header_status(wsi, HTTP_STATUS_INTERNAL_SERVER_ERROR, &p, end)) {
|
||||
return -1;
|
||||
}
|
||||
if (lws_add_http_header_by_token(wsi, WSI_TOKEN_HTTP_CONTENT_TYPE, (unsigned char*)"text/plain", 10, &p, end)) {
|
||||
return -1;
|
||||
}
|
||||
if (lws_add_http_header_content_length(wsi, 0, &p, end)) {
|
||||
return -1;
|
||||
}
|
||||
if (lws_finalize_http_header(wsi, &p, end)) {
|
||||
return -1;
|
||||
}
|
||||
lws_write(wsi, start, p - start, LWS_WRITE_HTTP_HEADERS);
|
||||
return -1;
|
||||
}
|
||||
|
||||
size_t json_len = strlen(json_string);
|
||||
|
||||
// Allocate session data to manage buffer lifetime across callbacks
|
||||
struct nip11_session_data* session_data = malloc(sizeof(struct nip11_session_data));
|
||||
if (!session_data) {
|
||||
|
||||
+1
-1
@@ -55,7 +55,7 @@ int validate_event_pow(cJSON* event, char* error_message, size_t error_size) {
|
||||
// If min_pow_difficulty is 0, only validate events that have nonce tags
|
||||
// This allows events without PoW when difficulty requirement is 0
|
||||
if (min_pow_difficulty == 0) {
|
||||
cJSON* tags = cJSON_GetObjectItem(event, "tags");
|
||||
cJSON* tags = cJSON_GetObjectItemCaseSensitive(event, "tags");
|
||||
int has_nonce_tag = 0;
|
||||
|
||||
if (tags && cJSON_IsArray(tags)) {
|
||||
|
||||
+2
-2
@@ -114,7 +114,7 @@ int is_event_expired(cJSON* event, time_t current_time) {
|
||||
return 0; // Invalid event, not expired
|
||||
}
|
||||
|
||||
cJSON* tags = cJSON_GetObjectItem(event, "tags");
|
||||
cJSON* tags = cJSON_GetObjectItemCaseSensitive(event, "tags");
|
||||
long expiration_ts = extract_expiration_timestamp(tags);
|
||||
|
||||
if (expiration_ts == 0) {
|
||||
@@ -140,7 +140,7 @@ int validate_event_expiration(cJSON* event, char* error_message, size_t error_si
|
||||
time_t current_time = time(NULL);
|
||||
if (is_event_expired(event, current_time)) {
|
||||
if (g_expiration_config.strict_mode) {
|
||||
cJSON* tags = cJSON_GetObjectItem(event, "tags");
|
||||
cJSON* tags = cJSON_GetObjectItemCaseSensitive(event, "tags");
|
||||
long expiration_ts = extract_expiration_timestamp(tags);
|
||||
|
||||
snprintf(error_message, error_size,
|
||||
|
||||
+22
-3
@@ -12,7 +12,9 @@
|
||||
#include <string.h>
|
||||
#include <stdlib.h>
|
||||
#include <time.h>
|
||||
#include <stdio.h>
|
||||
#include "websockets.h"
|
||||
#include "ip_ban.h"
|
||||
|
||||
|
||||
// Forward declaration for notice message function
|
||||
@@ -93,13 +95,22 @@ void handle_nip42_auth_signed_event(struct lws* wsi, struct per_session_data* ps
|
||||
|
||||
// Verify authentication using existing request_validator function
|
||||
// Note: nostr_nip42_verify_auth_event doesn't extract pubkey, we need to do that separately
|
||||
char relay_url[RELAY_URL_MAX_LENGTH];
|
||||
const char* configured_relay_url = get_config_value("relay_url");
|
||||
if (configured_relay_url && configured_relay_url[0] != '\0') {
|
||||
snprintf(relay_url, sizeof(relay_url), "%s", configured_relay_url);
|
||||
} else {
|
||||
int relay_port = (g_relay_port > 0) ? g_relay_port : get_config_int("relay_port", DEFAULT_PORT);
|
||||
snprintf(relay_url, sizeof(relay_url), "ws://127.0.0.1:%d", relay_port);
|
||||
}
|
||||
|
||||
int result = nostr_nip42_verify_auth_event(auth_event, challenge_copy,
|
||||
"ws://localhost:8888", 600); // 10 minutes tolerance
|
||||
relay_url, 600); // 10 minutes tolerance
|
||||
|
||||
char authenticated_pubkey[65] = {0};
|
||||
if (result == 0) {
|
||||
// Extract pubkey from the auth event
|
||||
cJSON* pubkey_json = cJSON_GetObjectItem(auth_event, "pubkey");
|
||||
cJSON* pubkey_json = cJSON_GetObjectItemCaseSensitive(auth_event, "pubkey");
|
||||
if (pubkey_json && cJSON_IsString(pubkey_json)) {
|
||||
const char* pubkey_str = cJSON_GetStringValue(pubkey_json);
|
||||
if (pubkey_str && strlen(pubkey_str) == 64) {
|
||||
@@ -125,8 +136,16 @@ void handle_nip42_auth_signed_event(struct lws* wsi, struct per_session_data* ps
|
||||
memset(pss->active_challenge, 0, sizeof(pss->active_challenge));
|
||||
pss->challenge_expires = 0;
|
||||
pss->auth_challenge_sent = 0;
|
||||
// Mark session as active - client sent AUTH
|
||||
pss->session_active = 1;
|
||||
pthread_mutex_unlock(&pss->session_lock);
|
||||
|
||||
|
||||
// Cancel the auth timeout — client has authenticated, keep connection open
|
||||
lws_set_timeout(wsi, NO_PENDING_TIMEOUT, 0);
|
||||
|
||||
// Record successful auth for this IP — marks it as trusted
|
||||
ip_ban_record_success(pss->client_ip);
|
||||
|
||||
send_notice_message(wsi, pss, "NIP-42 authentication successful");
|
||||
} else {
|
||||
// Authentication failed
|
||||
|
||||
+53
-108
@@ -1,5 +1,5 @@
|
||||
/*
|
||||
* C-Relay Request Validator - Integrated Authentication System
|
||||
* C-Relay-PG Request Validator - Integrated Authentication System
|
||||
*
|
||||
* Provides complete request validation including:
|
||||
* - Protocol validation via nostr_core_lib (signatures, pubkey extraction,
|
||||
@@ -17,17 +17,13 @@
|
||||
#include "../nostr_core_lib/nostr_core/utils.h"
|
||||
#include "debug.h" // C-relay debug system
|
||||
#include "config.h" // C-relay configuration system
|
||||
#include <sqlite3.h>
|
||||
#include "db_ops.h"
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <strings.h>
|
||||
#include <time.h>
|
||||
|
||||
// External references to C-relay global state
|
||||
extern sqlite3* g_db;
|
||||
extern char g_database_path[512];
|
||||
|
||||
// Forward declaration for C-relay event storage function
|
||||
extern int store_event(cJSON* event);
|
||||
|
||||
@@ -133,6 +129,14 @@ typedef struct {
|
||||
static nip42_challenge_manager_t g_challenge_manager = {0};
|
||||
static int g_validator_initialized = 0;
|
||||
|
||||
typedef struct {
|
||||
int has_active_whitelist_rules;
|
||||
time_t last_refresh;
|
||||
} auth_rules_fast_cache_t;
|
||||
|
||||
static auth_rules_fast_cache_t g_auth_rules_fast_cache = {0};
|
||||
#define AUTH_RULES_CACHE_TTL_SEC 5
|
||||
|
||||
// Last rule violation details for status code mapping
|
||||
struct {
|
||||
char violation_type[100]; // "pubkey_blacklist", "hash_blacklist",
|
||||
@@ -252,13 +256,13 @@ int nostr_validate_unified_request(const char* json_string, size_t json_length)
|
||||
}
|
||||
|
||||
// 4. Validate basic event structure
|
||||
cJSON *id = cJSON_GetObjectItem(event, "id");
|
||||
cJSON *pubkey = cJSON_GetObjectItem(event, "pubkey");
|
||||
cJSON *created_at = cJSON_GetObjectItem(event, "created_at");
|
||||
cJSON *kind = cJSON_GetObjectItem(event, "kind");
|
||||
cJSON *tags = cJSON_GetObjectItem(event, "tags");
|
||||
cJSON *content = cJSON_GetObjectItem(event, "content");
|
||||
cJSON *sig = cJSON_GetObjectItem(event, "sig");
|
||||
cJSON *id = cJSON_GetObjectItemCaseSensitive(event, "id");
|
||||
cJSON *pubkey = cJSON_GetObjectItemCaseSensitive(event, "pubkey");
|
||||
cJSON *created_at = cJSON_GetObjectItemCaseSensitive(event, "created_at");
|
||||
cJSON *kind = cJSON_GetObjectItemCaseSensitive(event, "kind");
|
||||
cJSON *tags = cJSON_GetObjectItemCaseSensitive(event, "tags");
|
||||
cJSON *content = cJSON_GetObjectItemCaseSensitive(event, "content");
|
||||
cJSON *sig = cJSON_GetObjectItemCaseSensitive(event, "sig");
|
||||
|
||||
if (!id || !cJSON_IsString(id) ||
|
||||
!pubkey || !cJSON_IsString(pubkey) ||
|
||||
@@ -304,9 +308,11 @@ int nostr_validate_unified_request(const char* json_string, size_t json_length)
|
||||
const char* admin_pubkey = get_config_value("admin_pubkey");
|
||||
if (admin_pubkey && strcmp(event_pubkey, admin_pubkey) == 0) {
|
||||
// Valid admin event - bypass remaining validation
|
||||
free((char*)admin_pubkey);
|
||||
cJSON_Delete(event);
|
||||
return NOSTR_SUCCESS;
|
||||
}
|
||||
if (admin_pubkey) free((char*)admin_pubkey);
|
||||
// Not from admin - continue with normal validation
|
||||
}
|
||||
|
||||
@@ -380,7 +386,7 @@ int nostr_validate_unified_request(const char* json_string, size_t json_length)
|
||||
// Always check expiration tags if present (following NIP-40 specification)
|
||||
|
||||
cJSON *expiration_tag = NULL;
|
||||
cJSON *tags_array = cJSON_GetObjectItem(event, "tags");
|
||||
cJSON *tags_array = cJSON_GetObjectItemCaseSensitive(event, "tags");
|
||||
|
||||
if (tags_array && cJSON_IsArray(tags_array)) {
|
||||
cJSON *tag = NULL;
|
||||
@@ -504,10 +510,10 @@ void nostr_request_result_free_file_data(nostr_request_result_t *result) {
|
||||
|
||||
|
||||
/**
|
||||
* Force cache refresh - cache no longer exists, function kept for compatibility
|
||||
* Force cache refresh for auth rule fast cache.
|
||||
*/
|
||||
void nostr_request_validator_force_cache_refresh(void) {
|
||||
// Cache no longer exists - direct database queries are used
|
||||
g_auth_rules_fast_cache.last_refresh = 0;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -522,118 +528,57 @@ static int reload_auth_config(void) {
|
||||
|
||||
// Note: Blossom protocol validation removed - C-relay uses standard Nostr events only
|
||||
|
||||
static int has_active_whitelist_rules_cached(void) {
|
||||
time_t now = time(NULL);
|
||||
if (g_auth_rules_fast_cache.last_refresh == 0 ||
|
||||
(now - g_auth_rules_fast_cache.last_refresh) >= AUTH_RULES_CACHE_TTL_SEC) {
|
||||
g_auth_rules_fast_cache.has_active_whitelist_rules =
|
||||
(db_count_active_whitelist_rules() > 0) ? 1 : 0;
|
||||
g_auth_rules_fast_cache.last_refresh = now;
|
||||
}
|
||||
return g_auth_rules_fast_cache.has_active_whitelist_rules;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check database authentication rules for the request
|
||||
* Implements the 6-step rule evaluation engine from AUTH_API.md
|
||||
*/
|
||||
int check_database_auth_rules(const char *pubkey, const char *operation __attribute__((unused)),
|
||||
const char *resource_hash) {
|
||||
sqlite3 *db = NULL;
|
||||
sqlite3_stmt *stmt = NULL;
|
||||
int rc;
|
||||
|
||||
DEBUG_TRACE("Checking auth rules for pubkey: %s", pubkey);
|
||||
|
||||
if (!pubkey) {
|
||||
return NOSTR_ERROR_INVALID_INPUT;
|
||||
}
|
||||
|
||||
// Open database using global database path
|
||||
if (strlen(g_database_path) == 0) {
|
||||
return NOSTR_SUCCESS; // Default allow on DB error
|
||||
}
|
||||
|
||||
rc = sqlite3_open_v2(g_database_path, &db, SQLITE_OPEN_READONLY, NULL);
|
||||
if (rc != SQLITE_OK) {
|
||||
return NOSTR_SUCCESS; // Default allow on DB error
|
||||
}
|
||||
|
||||
// Step 1: Check pubkey blacklist (highest priority)
|
||||
const char *blacklist_sql =
|
||||
"SELECT rule_type FROM auth_rules WHERE rule_type = "
|
||||
"'blacklist' AND pattern_type = 'pubkey' AND pattern_value = ? AND active = 1 LIMIT 1";
|
||||
DEBUG_TRACE("Blacklist SQL: %s", blacklist_sql);
|
||||
rc = sqlite3_prepare_v2(db, blacklist_sql, -1, &stmt, NULL);
|
||||
if (rc == SQLITE_OK) {
|
||||
sqlite3_bind_text(stmt, 1, pubkey, -1, SQLITE_STATIC);
|
||||
|
||||
int step_result = sqlite3_step(stmt);
|
||||
DEBUG_TRACE("Blacklist query result: %s", step_result == SQLITE_ROW ? "FOUND" : "NOT_FOUND");
|
||||
|
||||
if (step_result == SQLITE_ROW) {
|
||||
DEBUG_TRACE("BLACKLIST HIT: Denying access for pubkey: %s", pubkey);
|
||||
// Set specific violation details for status code mapping
|
||||
strcpy(g_last_rule_violation.violation_type, "pubkey_blacklist");
|
||||
sprintf(g_last_rule_violation.reason, "Public key blacklisted");
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
sqlite3_close(db);
|
||||
return NOSTR_ERROR_AUTH_REQUIRED;
|
||||
}
|
||||
sqlite3_finalize(stmt);
|
||||
if (db_is_pubkey_blacklisted(pubkey)) {
|
||||
DEBUG_TRACE("BLACKLIST HIT: Denying access for pubkey: %s", pubkey);
|
||||
strcpy(g_last_rule_violation.violation_type, "pubkey_blacklist");
|
||||
sprintf(g_last_rule_violation.reason, "Public key blacklisted");
|
||||
return NOSTR_ERROR_AUTH_REQUIRED;
|
||||
}
|
||||
|
||||
// Step 2: Check hash blacklist
|
||||
if (resource_hash) {
|
||||
const char *hash_blacklist_sql =
|
||||
"SELECT rule_type FROM auth_rules WHERE rule_type = "
|
||||
"'blacklist' AND pattern_type = 'hash' AND pattern_value = ? AND active = 1 LIMIT 1";
|
||||
rc = sqlite3_prepare_v2(db, hash_blacklist_sql, -1, &stmt, NULL);
|
||||
if (rc == SQLITE_OK) {
|
||||
sqlite3_bind_text(stmt, 1, resource_hash, -1, SQLITE_STATIC);
|
||||
|
||||
if (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
// Set specific violation details for status code mapping
|
||||
strcpy(g_last_rule_violation.violation_type, "hash_blacklist");
|
||||
sprintf(g_last_rule_violation.reason, "File hash blacklisted");
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
sqlite3_close(db);
|
||||
return NOSTR_ERROR_AUTH_REQUIRED;
|
||||
}
|
||||
sqlite3_finalize(stmt);
|
||||
}
|
||||
if (resource_hash && db_is_hash_blacklisted(resource_hash)) {
|
||||
strcpy(g_last_rule_violation.violation_type, "hash_blacklist");
|
||||
sprintf(g_last_rule_violation.reason, "File hash blacklisted");
|
||||
return NOSTR_ERROR_AUTH_REQUIRED;
|
||||
}
|
||||
|
||||
// Step 3: Check pubkey whitelist
|
||||
const char *whitelist_sql =
|
||||
"SELECT rule_type FROM auth_rules WHERE rule_type = "
|
||||
"'whitelist' AND pattern_type = 'pubkey' AND pattern_value = ? AND active = 1 LIMIT 1";
|
||||
rc = sqlite3_prepare_v2(db, whitelist_sql, -1, &stmt, NULL);
|
||||
if (rc == SQLITE_OK) {
|
||||
sqlite3_bind_text(stmt, 1, pubkey, -1, SQLITE_STATIC);
|
||||
|
||||
if (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
sqlite3_finalize(stmt);
|
||||
sqlite3_close(db);
|
||||
return NOSTR_SUCCESS; // Allow whitelisted pubkey
|
||||
}
|
||||
sqlite3_finalize(stmt);
|
||||
// Step 3: Check pubkey whitelist (includes wot_whitelist)
|
||||
if (db_is_pubkey_whitelisted(pubkey)) {
|
||||
return NOSTR_SUCCESS; // Allow whitelisted pubkey
|
||||
}
|
||||
|
||||
// Step 4: Check if any whitelist rules exist - if yes, deny by default
|
||||
const char *whitelist_exists_sql =
|
||||
"SELECT COUNT(*) FROM auth_rules WHERE rule_type = 'whitelist' "
|
||||
"AND pattern_type = 'pubkey' AND active = 1 LIMIT 1";
|
||||
rc = sqlite3_prepare_v2(db, whitelist_exists_sql, -1, &stmt, NULL);
|
||||
if (rc == SQLITE_OK) {
|
||||
if (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
int whitelist_count = sqlite3_column_int(stmt, 0);
|
||||
if (whitelist_count > 0) {
|
||||
// Set specific violation details for status code mapping
|
||||
strcpy(g_last_rule_violation.violation_type, "whitelist_violation");
|
||||
strcpy(g_last_rule_violation.reason,
|
||||
"Public key not whitelisted for this operation");
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
sqlite3_close(db);
|
||||
return NOSTR_ERROR_AUTH_REQUIRED;
|
||||
}
|
||||
}
|
||||
sqlite3_finalize(stmt);
|
||||
// Step 4: If any whitelist rules exist, deny by default
|
||||
if (has_active_whitelist_rules_cached()) {
|
||||
strcpy(g_last_rule_violation.violation_type, "whitelist_violation");
|
||||
strcpy(g_last_rule_violation.reason,
|
||||
"Public key not whitelisted for this operation");
|
||||
return NOSTR_ERROR_AUTH_REQUIRED;
|
||||
}
|
||||
|
||||
sqlite3_close(db);
|
||||
return NOSTR_SUCCESS; // Default allow if no restrictive rules matched
|
||||
}
|
||||
|
||||
@@ -818,7 +763,7 @@ int nostr_nip42_verify_auth_event(cJSON *event, const char *challenge_id,
|
||||
}
|
||||
|
||||
// Check if event has the required tags for NIP-42
|
||||
cJSON *tags = cJSON_GetObjectItem(event, "tags");
|
||||
cJSON *tags = cJSON_GetObjectItemCaseSensitive(event, "tags");
|
||||
if (!tags || !cJSON_IsArray(tags)) {
|
||||
return NOSTR_ERROR_NIP42_AUTH_EVENT_INVALID;
|
||||
}
|
||||
@@ -896,7 +841,7 @@ int nostr_nip42_verify_auth_event(cJSON *event, const char *challenge_id,
|
||||
}
|
||||
|
||||
// Check created_at timestamp for reasonable bounds
|
||||
cJSON *created_at_json = cJSON_GetObjectItem(event, "created_at");
|
||||
cJSON *created_at_json = cJSON_GetObjectItemCaseSensitive(event, "created_at");
|
||||
if (created_at_json && cJSON_IsNumber(created_at_json)) {
|
||||
time_t created_at = (time_t)cJSON_GetNumberValue(created_at_json);
|
||||
if (abs((int)(now - created_at)) > time_tolerance_seconds) {
|
||||
|
||||
+26
-7
@@ -1,11 +1,11 @@
|
||||
/* Embedded SQL Schema for C Nostr Relay
|
||||
* Schema Version: 11
|
||||
* Schema Version: 12
|
||||
*/
|
||||
#ifndef SQL_SCHEMA_H
|
||||
#define SQL_SCHEMA_H
|
||||
|
||||
/* Schema version constant */
|
||||
#define EMBEDDED_SCHEMA_VERSION "11"
|
||||
#define EMBEDDED_SCHEMA_VERSION "12"
|
||||
|
||||
/* Embedded SQL schema as C string literal */
|
||||
static const char* const EMBEDDED_SCHEMA_SQL =
|
||||
@@ -14,7 +14,7 @@ static const char* const EMBEDDED_SCHEMA_SQL =
|
||||
-- Configuration system using config table\n\
|
||||
\n\
|
||||
-- Schema version tracking\n\
|
||||
PRAGMA user_version = 11;\n\
|
||||
PRAGMA user_version = 12;\n\
|
||||
\n\
|
||||
-- Enable foreign key support\n\
|
||||
PRAGMA foreign_keys = ON;\n\
|
||||
@@ -49,6 +49,25 @@ CREATE INDEX idx_events_kind_created_at ON events(kind, created_at DESC);\n\
|
||||
CREATE INDEX idx_events_pubkey_created_at ON events(pubkey, created_at DESC);\n\
|
||||
CREATE INDEX idx_events_pubkey_kind ON events(pubkey, kind);\n\
|
||||
\n\
|
||||
-- Denormalized event tags for fast indexed lookups\n\
|
||||
-- Replaces json_each(json(tags)) queries which cause full JSON parsing per row\n\
|
||||
CREATE TABLE event_tags (\n\
|
||||
event_id TEXT NOT NULL,\n\
|
||||
tag_name TEXT NOT NULL,\n\
|
||||
tag_value TEXT NOT NULL,\n\
|
||||
tag_index INTEGER NOT NULL DEFAULT 0,\n\
|
||||
FOREIGN KEY (event_id) REFERENCES events(id) ON DELETE CASCADE\n\
|
||||
);\n\
|
||||
\n\
|
||||
-- Primary lookup index: find events by tag name + value\n\
|
||||
CREATE INDEX idx_event_tags_lookup ON event_tags(tag_name, tag_value);\n\
|
||||
\n\
|
||||
-- Reverse lookup: find all tags for an event (for cleanup)\n\
|
||||
CREATE INDEX idx_event_tags_event ON event_tags(event_id);\n\
|
||||
\n\
|
||||
-- Composite index for common query pattern: tag + kind (via join)\n\
|
||||
CREATE INDEX idx_event_tags_value_name ON event_tags(tag_value, tag_name);\n\
|
||||
\n\
|
||||
-- Schema information table\n\
|
||||
CREATE TABLE schema_info (\n\
|
||||
key TEXT PRIMARY KEY,\n\
|
||||
@@ -58,8 +77,8 @@ CREATE TABLE schema_info (\n\
|
||||
\n\
|
||||
-- Insert schema metadata\n\
|
||||
INSERT INTO schema_info (key, value) VALUES\n\
|
||||
('version', '11'),\n\
|
||||
('description', 'Added event_json column for 2500x performance improvement in event retrieval'),\n\
|
||||
('version', '12'),\n\
|
||||
('description', 'Added event_tags table for fast indexed tag lookups, replacing json_each() correlated subqueries'),\n\
|
||||
('created_at', strftime('%s', 'now'));\n\
|
||||
\n\
|
||||
-- Helper views for common queries\n\
|
||||
@@ -129,7 +148,7 @@ CREATE TABLE relay_seckey (\n\
|
||||
-- Used by request_validator.c for unified validation\n\
|
||||
CREATE TABLE auth_rules (\n\
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,\n\
|
||||
rule_type TEXT NOT NULL CHECK (rule_type IN ('whitelist', 'blacklist', 'rate_limit', 'auth_required')),\n\
|
||||
rule_type TEXT NOT NULL CHECK (rule_type IN ('whitelist', 'blacklist', 'rate_limit', 'auth_required', 'wot_whitelist')),\n\
|
||||
pattern_type TEXT NOT NULL CHECK (pattern_type IN ('pubkey', 'kind', 'ip', 'global')),\n\
|
||||
pattern_value TEXT,\n\
|
||||
active INTEGER NOT NULL DEFAULT 1,\n\
|
||||
@@ -311,4 +330,4 @@ SELECT\n\
|
||||
FROM events\n\
|
||||
WHERE created_at >= (strftime('%s', 'now') - 2592000);";
|
||||
|
||||
#endif /* SQL_SCHEMA_H */
|
||||
#endif /* SQL_SCHEMA_H */
|
||||
|
||||
+227
-179
@@ -1,7 +1,8 @@
|
||||
#define _GNU_SOURCE
|
||||
#include <cjson/cJSON.h>
|
||||
#include "debug.h"
|
||||
#include <sqlite3.h>
|
||||
#include "db_ops.h"
|
||||
#include "thread_pool.h"
|
||||
#include <string.h>
|
||||
#include <stdlib.h>
|
||||
#include <time.h>
|
||||
@@ -10,6 +11,34 @@
|
||||
#include <libwebsockets.h>
|
||||
#include "subscriptions.h"
|
||||
|
||||
static void free_sub_log_created_payload_local(thread_pool_sub_log_created_payload_t* payload) {
|
||||
if (!payload) return;
|
||||
free(payload->sub_id);
|
||||
free(payload->wsi_ptr);
|
||||
free(payload->client_ip);
|
||||
free(payload->filter_json);
|
||||
free(payload);
|
||||
}
|
||||
|
||||
static void free_sub_log_closed_payload_local(thread_pool_sub_log_closed_payload_t* payload) {
|
||||
if (!payload) return;
|
||||
free(payload->sub_id);
|
||||
free(payload->client_ip);
|
||||
free(payload);
|
||||
}
|
||||
|
||||
static void free_sub_log_disconnected_payload_local(thread_pool_sub_log_disconnected_payload_t* payload) {
|
||||
if (!payload) return;
|
||||
free(payload->client_ip);
|
||||
free(payload);
|
||||
}
|
||||
|
||||
static void free_sub_update_events_payload_local(thread_pool_sub_update_events_payload_t* payload) {
|
||||
if (!payload) return;
|
||||
free(payload->sub_id);
|
||||
free(payload);
|
||||
}
|
||||
|
||||
// Forward declarations for logging functions
|
||||
|
||||
// Forward declarations for configuration functions
|
||||
@@ -28,9 +57,6 @@ int validate_search_term(const char* search_term, char* error_message, size_t er
|
||||
// Forward declaration for monitoring function
|
||||
void monitoring_on_subscription_change(void);
|
||||
|
||||
// Global database variable
|
||||
extern sqlite3* g_db;
|
||||
|
||||
// Configuration functions from config.c
|
||||
extern int get_config_bool(const char* key, int default_value);
|
||||
|
||||
@@ -192,32 +218,32 @@ subscription_filter_t* create_subscription_filter(cJSON* filter_json) {
|
||||
}
|
||||
|
||||
// Copy filter criteria
|
||||
cJSON* kinds = cJSON_GetObjectItem(filter_json, "kinds");
|
||||
cJSON* kinds = cJSON_GetObjectItemCaseSensitive(filter_json, "kinds");
|
||||
if (kinds && cJSON_IsArray(kinds)) {
|
||||
filter->kinds = cJSON_Duplicate(kinds, 1);
|
||||
}
|
||||
|
||||
cJSON* authors = cJSON_GetObjectItem(filter_json, "authors");
|
||||
cJSON* authors = cJSON_GetObjectItemCaseSensitive(filter_json, "authors");
|
||||
if (authors && cJSON_IsArray(authors)) {
|
||||
filter->authors = cJSON_Duplicate(authors, 1);
|
||||
}
|
||||
|
||||
cJSON* ids = cJSON_GetObjectItem(filter_json, "ids");
|
||||
cJSON* ids = cJSON_GetObjectItemCaseSensitive(filter_json, "ids");
|
||||
if (ids && cJSON_IsArray(ids)) {
|
||||
filter->ids = cJSON_Duplicate(ids, 1);
|
||||
}
|
||||
|
||||
cJSON* since = cJSON_GetObjectItem(filter_json, "since");
|
||||
cJSON* since = cJSON_GetObjectItemCaseSensitive(filter_json, "since");
|
||||
if (since && cJSON_IsNumber(since)) {
|
||||
filter->since = (long)cJSON_GetNumberValue(since);
|
||||
}
|
||||
|
||||
cJSON* until = cJSON_GetObjectItem(filter_json, "until");
|
||||
cJSON* until = cJSON_GetObjectItemCaseSensitive(filter_json, "until");
|
||||
if (until && cJSON_IsNumber(until)) {
|
||||
filter->until = (long)cJSON_GetNumberValue(until);
|
||||
}
|
||||
|
||||
cJSON* limit = cJSON_GetObjectItem(filter_json, "limit");
|
||||
cJSON* limit = cJSON_GetObjectItemCaseSensitive(filter_json, "limit");
|
||||
if (limit && cJSON_IsNumber(limit)) {
|
||||
filter->limit = (int)cJSON_GetNumberValue(limit);
|
||||
}
|
||||
@@ -265,11 +291,14 @@ int validate_subscription_id(const char* sub_id) {
|
||||
return 0; // Empty or too long
|
||||
}
|
||||
|
||||
// Check for valid characters (alphanumeric, underscore, hyphen, colon, comma)
|
||||
// Check for valid characters (alphanumeric, underscore, hyphen, colon, comma, plus)
|
||||
// Plus URL-safe special characters: ! $ % & ( ) * . = ? @ # ~
|
||||
for (size_t i = 0; i < len; i++) {
|
||||
char c = sub_id[i];
|
||||
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') ||
|
||||
(c >= '0' && c <= '9') || c == '_' || c == '-' || c == ':' || c == ',')) {
|
||||
(c >= '0' && c <= '9') || c == '_' || c == '-' || c == ':' || c == ',' || c == '+' ||
|
||||
c == '!' || c == '$' || c == '%' || c == '&' || c == '(' || c == ')' ||
|
||||
c == '*' || c == '.' || c == '=' || c == '?' || c == '@' || c == '#' || c == '~')) {
|
||||
return 0; // Invalid character
|
||||
}
|
||||
}
|
||||
@@ -529,9 +558,9 @@ int event_matches_filter(cJSON* event, subscription_filter_t* filter) {
|
||||
}
|
||||
|
||||
// Debug: Log event details being tested
|
||||
cJSON* event_kind_obj = cJSON_GetObjectItem(event, "kind");
|
||||
cJSON* event_id_obj = cJSON_GetObjectItem(event, "id");
|
||||
cJSON* event_created_at_obj = cJSON_GetObjectItem(event, "created_at");
|
||||
cJSON* event_kind_obj = cJSON_GetObjectItemCaseSensitive(event, "kind");
|
||||
cJSON* event_id_obj = cJSON_GetObjectItemCaseSensitive(event, "id");
|
||||
cJSON* event_created_at_obj = cJSON_GetObjectItemCaseSensitive(event, "created_at");
|
||||
|
||||
DEBUG_TRACE("FILTER_MATCH: Testing event kind=%d id=%.8s created_at=%ld",
|
||||
event_kind_obj ? (int)cJSON_GetNumberValue(event_kind_obj) : -1,
|
||||
@@ -542,7 +571,7 @@ int event_matches_filter(cJSON* event, subscription_filter_t* filter) {
|
||||
if (filter->kinds && cJSON_IsArray(filter->kinds)) {
|
||||
DEBUG_TRACE("FILTER_MATCH: Checking kinds filter with %d kinds", cJSON_GetArraySize(filter->kinds));
|
||||
|
||||
cJSON* event_kind = cJSON_GetObjectItem(event, "kind");
|
||||
cJSON* event_kind = cJSON_GetObjectItemCaseSensitive(event, "kind");
|
||||
if (!event_kind || !cJSON_IsNumber(event_kind)) {
|
||||
DEBUG_WARN("FILTER_MATCH: Event has no valid kind field");
|
||||
return 0;
|
||||
@@ -574,7 +603,7 @@ int event_matches_filter(cJSON* event, subscription_filter_t* filter) {
|
||||
|
||||
// Check authors filter
|
||||
if (filter->authors && cJSON_IsArray(filter->authors)) {
|
||||
cJSON* event_pubkey = cJSON_GetObjectItem(event, "pubkey");
|
||||
cJSON* event_pubkey = cJSON_GetObjectItemCaseSensitive(event, "pubkey");
|
||||
if (!event_pubkey || !cJSON_IsString(event_pubkey)) {
|
||||
return 0;
|
||||
}
|
||||
@@ -601,7 +630,7 @@ int event_matches_filter(cJSON* event, subscription_filter_t* filter) {
|
||||
|
||||
// Check IDs filter
|
||||
if (filter->ids && cJSON_IsArray(filter->ids)) {
|
||||
cJSON* event_id = cJSON_GetObjectItem(event, "id");
|
||||
cJSON* event_id = cJSON_GetObjectItemCaseSensitive(event, "id");
|
||||
if (!event_id || !cJSON_IsString(event_id)) {
|
||||
return 0;
|
||||
}
|
||||
@@ -628,7 +657,7 @@ int event_matches_filter(cJSON* event, subscription_filter_t* filter) {
|
||||
|
||||
// Check since filter
|
||||
if (filter->since > 0) {
|
||||
cJSON* event_created_at = cJSON_GetObjectItem(event, "created_at");
|
||||
cJSON* event_created_at = cJSON_GetObjectItemCaseSensitive(event, "created_at");
|
||||
if (!event_created_at || !cJSON_IsNumber(event_created_at)) {
|
||||
DEBUG_WARN("FILTER_MATCH: Event has no valid created_at field");
|
||||
return 0;
|
||||
@@ -647,7 +676,7 @@ int event_matches_filter(cJSON* event, subscription_filter_t* filter) {
|
||||
|
||||
// Check until filter
|
||||
if (filter->until > 0) {
|
||||
cJSON* event_created_at = cJSON_GetObjectItem(event, "created_at");
|
||||
cJSON* event_created_at = cJSON_GetObjectItemCaseSensitive(event, "created_at");
|
||||
if (!event_created_at || !cJSON_IsNumber(event_created_at)) {
|
||||
return 0;
|
||||
}
|
||||
@@ -660,7 +689,7 @@ int event_matches_filter(cJSON* event, subscription_filter_t* filter) {
|
||||
|
||||
// Check tag filters (e.g., #e, #p tags)
|
||||
if (filter->tag_filters && cJSON_IsObject(filter->tag_filters)) {
|
||||
cJSON* event_tags = cJSON_GetObjectItem(event, "tags");
|
||||
cJSON* event_tags = cJSON_GetObjectItemCaseSensitive(event, "tags");
|
||||
if (!event_tags || !cJSON_IsArray(event_tags)) {
|
||||
return 0; // Event has no tags but filter requires tags
|
||||
}
|
||||
@@ -774,9 +803,9 @@ int broadcast_event_to_subscriptions(cJSON* event) {
|
||||
int broadcasts = 0;
|
||||
|
||||
// Log event details
|
||||
cJSON* event_kind = cJSON_GetObjectItem(event, "kind");
|
||||
cJSON* event_id = cJSON_GetObjectItem(event, "id");
|
||||
cJSON* event_created_at = cJSON_GetObjectItem(event, "created_at");
|
||||
cJSON* event_kind = cJSON_GetObjectItemCaseSensitive(event, "kind");
|
||||
cJSON* event_id = cJSON_GetObjectItemCaseSensitive(event, "id");
|
||||
cJSON* event_created_at = cJSON_GetObjectItemCaseSensitive(event, "created_at");
|
||||
|
||||
DEBUG_TRACE("BROADCAST: Event kind=%d id=%.8s created_at=%ld",
|
||||
event_kind ? (int)cJSON_GetNumberValue(event_kind) : -1,
|
||||
@@ -870,30 +899,40 @@ int broadcast_event_to_subscriptions(cJSON* event) {
|
||||
// Second pass: send messages without holding lock
|
||||
temp_sub_t* current_temp = matching_subs;
|
||||
while (current_temp) {
|
||||
// Create EVENT message for this subscription
|
||||
cJSON* event_msg = cJSON_CreateArray();
|
||||
cJSON_AddItemToArray(event_msg, cJSON_CreateString("EVENT"));
|
||||
cJSON_AddItemToArray(event_msg, cJSON_CreateString(current_temp->id));
|
||||
cJSON_AddItemToArray(event_msg, cJSON_Duplicate(event, 1));
|
||||
|
||||
char* msg_str = cJSON_Print(event_msg);
|
||||
if (msg_str) {
|
||||
size_t msg_len = strlen(msg_str);
|
||||
unsigned char* buf = malloc(LWS_PRE + msg_len);
|
||||
// Serialize event once per subscription using pre-serialized event_json if available,
|
||||
// otherwise fall back to cJSON serialization.
|
||||
// Format: ["EVENT","<sub_id>",<event_json>]
|
||||
const char* event_json_str = NULL;
|
||||
char* event_json_allocated = NULL;
|
||||
|
||||
// Try to get pre-serialized event_json field first (fast path)
|
||||
cJSON* event_json_field = cJSON_GetObjectItemCaseSensitive(event, "event_json");
|
||||
if (event_json_field && cJSON_IsString(event_json_field)) {
|
||||
event_json_str = cJSON_GetStringValue(event_json_field);
|
||||
} else {
|
||||
// Fall back to serializing the event
|
||||
event_json_allocated = cJSON_PrintUnformatted(event);
|
||||
event_json_str = event_json_allocated;
|
||||
}
|
||||
|
||||
if (event_json_str) {
|
||||
size_t sub_id_len = strlen(current_temp->id);
|
||||
size_t event_json_len = strlen(event_json_str);
|
||||
// ["EVENT","<sub_id>",<event_json>]
|
||||
size_t msg_len = 10 + sub_id_len + 3 + event_json_len + 1;
|
||||
|
||||
// Zero-copy: allocate with LWS_PRE prefix, write directly, transfer ownership to queue
|
||||
unsigned char* buf = malloc(LWS_PRE + msg_len + 1);
|
||||
if (buf) {
|
||||
memcpy(buf + LWS_PRE, msg_str, msg_len);
|
||||
|
||||
// DEBUG: Log WebSocket frame details before sending
|
||||
DEBUG_TRACE("WS_FRAME_SEND: type=EVENT sub=%s len=%zu data=%.100s%s",
|
||||
current_temp->id,
|
||||
msg_len,
|
||||
msg_str,
|
||||
msg_len > 100 ? "..." : "");
|
||||
|
||||
// Queue message for proper libwebsockets pattern
|
||||
char* msg_ptr = (char*)(buf + LWS_PRE);
|
||||
snprintf(msg_ptr, msg_len + 1, "[\"EVENT\",\"%s\",%s]", current_temp->id, event_json_str);
|
||||
size_t actual_len = strlen(msg_ptr);
|
||||
|
||||
DEBUG_TRACE("WS_FRAME_SEND: type=EVENT sub=%s len=%zu", current_temp->id, actual_len);
|
||||
|
||||
struct per_session_data* pss = (struct per_session_data*)lws_wsi_user(current_temp->wsi);
|
||||
if (queue_message(current_temp->wsi, pss, msg_str, msg_len, LWS_WRITE_TEXT) == 0) {
|
||||
// Message queued successfully
|
||||
// queue_message_take_ownership takes buf ownership — no memcpy, no free needed here
|
||||
if (queue_message_take_ownership(current_temp->wsi, pss, buf, actual_len, LWS_WRITE_TEXT) == 0) {
|
||||
broadcasts++;
|
||||
|
||||
// Update events sent counter for this subscription
|
||||
@@ -902,30 +941,22 @@ int broadcast_event_to_subscriptions(cJSON* event) {
|
||||
while (update_sub) {
|
||||
if (update_sub->wsi == current_temp->wsi &&
|
||||
strcmp(update_sub->id, current_temp->id) == 0 &&
|
||||
update_sub->active) { // Add active check to prevent use-after-free
|
||||
update_sub->active) {
|
||||
update_sub->events_sent++;
|
||||
break;
|
||||
}
|
||||
update_sub = update_sub->next;
|
||||
}
|
||||
pthread_mutex_unlock(&g_subscription_manager.subscriptions_lock);
|
||||
|
||||
// Log event broadcast to database (optional - can be disabled for performance)
|
||||
// NOTE: event_broadcasts table removed due to FOREIGN KEY constraint issues
|
||||
// cJSON* event_id_obj = cJSON_GetObjectItem(event, "id");
|
||||
// if (event_id_obj && cJSON_IsString(event_id_obj)) {
|
||||
// log_event_broadcast(cJSON_GetStringValue(event_id_obj), current_temp->id, current_temp->client_ip);
|
||||
// }
|
||||
} else {
|
||||
DEBUG_ERROR("Failed to queue EVENT message for sub=%s", current_temp->id);
|
||||
// buf already freed by queue_message_take_ownership on failure
|
||||
}
|
||||
|
||||
free(buf);
|
||||
}
|
||||
free(msg_str);
|
||||
}
|
||||
|
||||
cJSON_Delete(event_msg);
|
||||
if (event_json_allocated) {
|
||||
free(event_json_allocated);
|
||||
}
|
||||
current_temp = current_temp->next;
|
||||
}
|
||||
|
||||
@@ -986,21 +1017,21 @@ int has_subscriptions_for_kind(int event_kind) {
|
||||
|
||||
// Log subscription creation to database
|
||||
void log_subscription_created(const subscription_t* sub) {
|
||||
if (!g_db || !sub) return;
|
||||
|
||||
if (!sub) return;
|
||||
|
||||
// Convert wsi pointer to string
|
||||
char wsi_str[32];
|
||||
snprintf(wsi_str, sizeof(wsi_str), "%p", (void*)sub->wsi);
|
||||
|
||||
|
||||
// Create filter JSON for logging
|
||||
char* filter_json = NULL;
|
||||
if (sub->filters) {
|
||||
cJSON* filters_array = cJSON_CreateArray();
|
||||
subscription_filter_t* filter = sub->filters;
|
||||
|
||||
|
||||
while (filter) {
|
||||
cJSON* filter_obj = cJSON_CreateObject();
|
||||
|
||||
|
||||
if (filter->kinds) {
|
||||
cJSON_AddItemToObject(filter_obj, "kinds", cJSON_Duplicate(filter->kinds, 1));
|
||||
}
|
||||
@@ -1029,99 +1060,119 @@ void log_subscription_created(const subscription_t* sub) {
|
||||
}
|
||||
cJSON_Delete(tags_obj);
|
||||
}
|
||||
|
||||
|
||||
cJSON_AddItemToArray(filters_array, filter_obj);
|
||||
filter = filter->next;
|
||||
}
|
||||
|
||||
|
||||
filter_json = cJSON_Print(filters_array);
|
||||
cJSON_Delete(filters_array);
|
||||
}
|
||||
|
||||
// Use INSERT OR REPLACE to handle duplicates automatically
|
||||
const char* sql =
|
||||
"INSERT OR REPLACE INTO subscriptions (subscription_id, wsi_pointer, client_ip, event_type, filter_json) "
|
||||
"VALUES (?, ?, ?, 'created', ?)";
|
||||
|
||||
sqlite3_stmt* stmt;
|
||||
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
|
||||
if (rc == SQLITE_OK) {
|
||||
sqlite3_bind_text(stmt, 1, sub->id, -1, SQLITE_STATIC);
|
||||
sqlite3_bind_text(stmt, 2, wsi_str, -1, SQLITE_TRANSIENT);
|
||||
sqlite3_bind_text(stmt, 3, sub->client_ip, -1, SQLITE_STATIC);
|
||||
sqlite3_bind_text(stmt, 4, filter_json ? filter_json : "[]", -1, SQLITE_TRANSIENT);
|
||||
|
||||
sqlite3_step(stmt);
|
||||
sqlite3_finalize(stmt);
|
||||
|
||||
if (!thread_pool_is_running()) {
|
||||
db_log_subscription_created(sub->id, wsi_str, sub->client_ip, filter_json ? filter_json : "[]");
|
||||
if (filter_json) free(filter_json);
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
thread_pool_sub_log_created_payload_t* payload = calloc(1, sizeof(*payload));
|
||||
if (!payload) {
|
||||
if (filter_json) free(filter_json);
|
||||
return;
|
||||
}
|
||||
|
||||
payload->sub_id = strdup(sub->id);
|
||||
payload->wsi_ptr = strdup(wsi_str);
|
||||
payload->client_ip = strdup(sub->client_ip);
|
||||
payload->filter_json = strdup(filter_json ? filter_json : "[]");
|
||||
|
||||
if (!payload->sub_id || !payload->wsi_ptr || !payload->client_ip || !payload->filter_json) {
|
||||
free(payload->sub_id);
|
||||
free(payload->wsi_ptr);
|
||||
free(payload->client_ip);
|
||||
free(payload->filter_json);
|
||||
free(payload);
|
||||
if (filter_json) free(filter_json);
|
||||
return;
|
||||
}
|
||||
|
||||
thread_pool_job_t job;
|
||||
memset(&job, 0, sizeof(job));
|
||||
job.type = THREAD_POOL_JOB_LOG_SUB_CREATED;
|
||||
job.payload = payload;
|
||||
job.payload_free = (thread_pool_payload_free_cb)free_sub_log_created_payload_local;
|
||||
|
||||
thread_pool_status_t submit_rc = thread_pool_submit_write(&job, NULL);
|
||||
if (submit_rc != THREAD_POOL_STATUS_OK) {
|
||||
free_sub_log_created_payload_local(payload);
|
||||
}
|
||||
|
||||
if (filter_json) free(filter_json);
|
||||
}
|
||||
|
||||
// Log subscription closure to database
|
||||
void log_subscription_closed(const char* sub_id, const char* client_ip, const char* reason) {
|
||||
(void)reason; // Mark as intentionally unused
|
||||
if (!g_db || !sub_id) return;
|
||||
|
||||
const char* sql =
|
||||
"INSERT INTO subscriptions (subscription_id, wsi_pointer, client_ip, event_type) "
|
||||
"VALUES (?, '', ?, 'closed')";
|
||||
|
||||
sqlite3_stmt* stmt;
|
||||
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
|
||||
if (rc == SQLITE_OK) {
|
||||
sqlite3_bind_text(stmt, 1, sub_id, -1, SQLITE_STATIC);
|
||||
sqlite3_bind_text(stmt, 2, client_ip ? client_ip : "unknown", -1, SQLITE_STATIC);
|
||||
|
||||
sqlite3_step(stmt);
|
||||
sqlite3_finalize(stmt);
|
||||
if (!sub_id) return;
|
||||
|
||||
if (!thread_pool_is_running()) {
|
||||
db_log_subscription_closed(sub_id, client_ip);
|
||||
return;
|
||||
}
|
||||
|
||||
// Update the corresponding 'created' entry with end time and events sent
|
||||
const char* update_sql =
|
||||
"UPDATE subscriptions "
|
||||
"SET ended_at = strftime('%s', 'now') "
|
||||
"WHERE subscription_id = ? AND event_type = 'created' AND ended_at IS NULL";
|
||||
|
||||
rc = sqlite3_prepare_v2(g_db, update_sql, -1, &stmt, NULL);
|
||||
if (rc == SQLITE_OK) {
|
||||
sqlite3_bind_text(stmt, 1, sub_id, -1, SQLITE_STATIC);
|
||||
sqlite3_step(stmt);
|
||||
sqlite3_finalize(stmt);
|
||||
|
||||
thread_pool_sub_log_closed_payload_t* payload = calloc(1, sizeof(*payload));
|
||||
if (!payload) {
|
||||
return;
|
||||
}
|
||||
|
||||
payload->sub_id = strdup(sub_id);
|
||||
payload->client_ip = strdup(client_ip ? client_ip : "unknown");
|
||||
if (!payload->sub_id || !payload->client_ip) {
|
||||
free_sub_log_closed_payload_local(payload);
|
||||
return;
|
||||
}
|
||||
|
||||
thread_pool_job_t job;
|
||||
memset(&job, 0, sizeof(job));
|
||||
job.type = THREAD_POOL_JOB_LOG_SUB_CLOSED;
|
||||
job.payload = payload;
|
||||
job.payload_free = (thread_pool_payload_free_cb)free_sub_log_closed_payload_local;
|
||||
|
||||
thread_pool_status_t submit_rc = thread_pool_submit_write(&job, NULL);
|
||||
if (submit_rc != THREAD_POOL_STATUS_OK) {
|
||||
free_sub_log_closed_payload_local(payload);
|
||||
}
|
||||
}
|
||||
|
||||
// Log subscription disconnection to database
|
||||
void log_subscription_disconnected(const char* client_ip) {
|
||||
if (!g_db || !client_ip) return;
|
||||
|
||||
// Mark all active subscriptions for this client as disconnected
|
||||
const char* sql =
|
||||
"UPDATE subscriptions "
|
||||
"SET ended_at = strftime('%s', 'now') "
|
||||
"WHERE client_ip = ? AND event_type = 'created' AND ended_at IS NULL";
|
||||
|
||||
sqlite3_stmt* stmt;
|
||||
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
|
||||
if (rc == SQLITE_OK) {
|
||||
sqlite3_bind_text(stmt, 1, client_ip, -1, SQLITE_STATIC);
|
||||
int changes = sqlite3_changes(g_db);
|
||||
sqlite3_step(stmt);
|
||||
sqlite3_finalize(stmt);
|
||||
|
||||
if (changes > 0) {
|
||||
// Log a disconnection event
|
||||
const char* insert_sql =
|
||||
"INSERT INTO subscriptions (subscription_id, wsi_pointer, client_ip, event_type) "
|
||||
"VALUES ('disconnect', '', ?, 'disconnected')";
|
||||
|
||||
rc = sqlite3_prepare_v2(g_db, insert_sql, -1, &stmt, NULL);
|
||||
if (rc == SQLITE_OK) {
|
||||
sqlite3_bind_text(stmt, 1, client_ip, -1, SQLITE_STATIC);
|
||||
sqlite3_step(stmt);
|
||||
sqlite3_finalize(stmt);
|
||||
}
|
||||
}
|
||||
if (!client_ip) return;
|
||||
|
||||
if (!thread_pool_is_running()) {
|
||||
db_log_subscription_disconnected(client_ip);
|
||||
return;
|
||||
}
|
||||
|
||||
thread_pool_sub_log_disconnected_payload_t* payload = calloc(1, sizeof(*payload));
|
||||
if (!payload) {
|
||||
return;
|
||||
}
|
||||
|
||||
payload->client_ip = strdup(client_ip);
|
||||
if (!payload->client_ip) {
|
||||
free_sub_log_disconnected_payload_local(payload);
|
||||
return;
|
||||
}
|
||||
|
||||
thread_pool_job_t job;
|
||||
memset(&job, 0, sizeof(job));
|
||||
job.type = THREAD_POOL_JOB_LOG_SUB_DISCONNECTED;
|
||||
job.payload = payload;
|
||||
job.payload_free = (thread_pool_payload_free_cb)free_sub_log_disconnected_payload_local;
|
||||
|
||||
thread_pool_status_t submit_rc = thread_pool_submit_write(&job, NULL);
|
||||
if (submit_rc != THREAD_POOL_STATUS_OK) {
|
||||
free_sub_log_disconnected_payload_local(payload);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1148,55 +1199,52 @@ void log_subscription_disconnected(const char* client_ip) {
|
||||
|
||||
// Update events sent counter for a subscription
|
||||
void update_subscription_events_sent(const char* sub_id, int events_sent) {
|
||||
if (!g_db || !sub_id) return;
|
||||
if (!sub_id) return;
|
||||
|
||||
const char* sql =
|
||||
"UPDATE subscriptions "
|
||||
"SET events_sent = ? "
|
||||
"WHERE subscription_id = ? AND event_type = 'created'";
|
||||
if (!thread_pool_is_running()) {
|
||||
db_update_subscription_events_sent(sub_id, events_sent);
|
||||
return;
|
||||
}
|
||||
|
||||
sqlite3_stmt* stmt;
|
||||
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
|
||||
if (rc == SQLITE_OK) {
|
||||
sqlite3_bind_int(stmt, 1, events_sent);
|
||||
sqlite3_bind_text(stmt, 2, sub_id, -1, SQLITE_STATIC);
|
||||
thread_pool_sub_update_events_payload_t* payload = calloc(1, sizeof(*payload));
|
||||
if (!payload) {
|
||||
return;
|
||||
}
|
||||
|
||||
sqlite3_step(stmt);
|
||||
sqlite3_finalize(stmt);
|
||||
payload->sub_id = strdup(sub_id);
|
||||
payload->events_sent = events_sent;
|
||||
if (!payload->sub_id) {
|
||||
free_sub_update_events_payload_local(payload);
|
||||
return;
|
||||
}
|
||||
|
||||
thread_pool_job_t job;
|
||||
memset(&job, 0, sizeof(job));
|
||||
job.type = THREAD_POOL_JOB_UPDATE_SUB_EVENTS_SENT;
|
||||
job.payload = payload;
|
||||
job.payload_free = (thread_pool_payload_free_cb)free_sub_update_events_payload_local;
|
||||
|
||||
thread_pool_status_t submit_rc = thread_pool_submit_write(&job, NULL);
|
||||
if (submit_rc != THREAD_POOL_STATUS_OK) {
|
||||
free_sub_update_events_payload_local(payload);
|
||||
}
|
||||
}
|
||||
|
||||
// Cleanup all subscriptions on startup
|
||||
void cleanup_all_subscriptions_on_startup(void) {
|
||||
if (!g_db) {
|
||||
if (!db_is_available()) {
|
||||
DEBUG_ERROR("Database not available for startup cleanup");
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
DEBUG_LOG("Performing startup subscription cleanup");
|
||||
|
||||
// Mark all active subscriptions as disconnected
|
||||
const char* sql =
|
||||
"UPDATE subscriptions "
|
||||
"SET ended_at = strftime('%s', 'now') "
|
||||
"WHERE event_type = 'created' AND ended_at IS NULL";
|
||||
|
||||
sqlite3_stmt* stmt;
|
||||
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
|
||||
if (rc != SQLITE_OK) {
|
||||
DEBUG_ERROR("Failed to prepare startup cleanup query");
|
||||
return;
|
||||
}
|
||||
|
||||
rc = sqlite3_step(stmt);
|
||||
int changes = sqlite3_changes(g_db);
|
||||
sqlite3_finalize(stmt);
|
||||
|
||||
if (rc != SQLITE_DONE) {
|
||||
|
||||
int changes = db_cleanup_orphaned_subscriptions();
|
||||
if (changes < 0) {
|
||||
DEBUG_ERROR("Failed to execute startup cleanup");
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
if (changes > 0) {
|
||||
DEBUG_LOG("Startup cleanup: marked %d orphaned subscriptions as disconnected", changes);
|
||||
} else {
|
||||
@@ -1470,7 +1518,7 @@ int validate_filter_values(cJSON* filter_json, char* error_message, size_t error
|
||||
int has_kind_99999 = 0; // Track if we encounter kind 99999 (NDK ping)
|
||||
|
||||
// Validate kinds array
|
||||
cJSON* kinds = cJSON_GetObjectItem(filter_json, "kinds");
|
||||
cJSON* kinds = cJSON_GetObjectItemCaseSensitive(filter_json, "kinds");
|
||||
if (kinds) {
|
||||
if (!cJSON_IsArray(kinds)) {
|
||||
snprintf(error_message, error_size, "kinds must be an array");
|
||||
@@ -1513,7 +1561,7 @@ int validate_filter_values(cJSON* filter_json, char* error_message, size_t error
|
||||
}
|
||||
|
||||
// Validate authors array
|
||||
cJSON* authors = cJSON_GetObjectItem(filter_json, "authors");
|
||||
cJSON* authors = cJSON_GetObjectItemCaseSensitive(filter_json, "authors");
|
||||
if (authors) {
|
||||
if (!cJSON_IsArray(authors)) {
|
||||
snprintf(error_message, error_size, "authors must be an array");
|
||||
@@ -1553,7 +1601,7 @@ int validate_filter_values(cJSON* filter_json, char* error_message, size_t error
|
||||
}
|
||||
|
||||
// Validate ids array
|
||||
cJSON* ids = cJSON_GetObjectItem(filter_json, "ids");
|
||||
cJSON* ids = cJSON_GetObjectItemCaseSensitive(filter_json, "ids");
|
||||
if (ids) {
|
||||
if (!cJSON_IsArray(ids)) {
|
||||
snprintf(error_message, error_size, "ids must be an array");
|
||||
@@ -1595,7 +1643,7 @@ int validate_filter_values(cJSON* filter_json, char* error_message, size_t error
|
||||
// Validate since/until timestamps
|
||||
long since_val = 0, until_val = 0;
|
||||
|
||||
cJSON* since = cJSON_GetObjectItem(filter_json, "since");
|
||||
cJSON* since = cJSON_GetObjectItemCaseSensitive(filter_json, "since");
|
||||
if (since) {
|
||||
if (!cJSON_IsNumber(since)) {
|
||||
snprintf(error_message, error_size, "since must be a number");
|
||||
@@ -1604,7 +1652,7 @@ int validate_filter_values(cJSON* filter_json, char* error_message, size_t error
|
||||
since_val = (long)cJSON_GetNumberValue(since);
|
||||
}
|
||||
|
||||
cJSON* until = cJSON_GetObjectItem(filter_json, "until");
|
||||
cJSON* until = cJSON_GetObjectItemCaseSensitive(filter_json, "until");
|
||||
if (until) {
|
||||
if (!cJSON_IsNumber(until)) {
|
||||
snprintf(error_message, error_size, "until must be a number");
|
||||
@@ -1618,7 +1666,7 @@ int validate_filter_values(cJSON* filter_json, char* error_message, size_t error
|
||||
}
|
||||
|
||||
// Validate limit
|
||||
cJSON* limit = cJSON_GetObjectItem(filter_json, "limit");
|
||||
cJSON* limit = cJSON_GetObjectItemCaseSensitive(filter_json, "limit");
|
||||
if (limit) {
|
||||
if (!cJSON_IsNumber(limit)) {
|
||||
snprintf(error_message, error_size, "limit must be a number");
|
||||
@@ -1632,7 +1680,7 @@ int validate_filter_values(cJSON* filter_json, char* error_message, size_t error
|
||||
}
|
||||
|
||||
// Validate search term
|
||||
cJSON* search = cJSON_GetObjectItem(filter_json, "search");
|
||||
cJSON* search = cJSON_GetObjectItemCaseSensitive(filter_json, "search");
|
||||
if (search) {
|
||||
if (!cJSON_IsString(search)) {
|
||||
snprintf(error_message, error_size, "search must be a string");
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
// Subscription system structures and functions for C-Relay
|
||||
// Subscription system structures and functions for C-Relay-PG
|
||||
// This header defines subscription management functionality
|
||||
|
||||
#ifndef SUBSCRIPTIONS_H
|
||||
|
||||
@@ -0,0 +1,953 @@
|
||||
#define _GNU_SOURCE
|
||||
|
||||
#include "thread_pool.h"
|
||||
#include "debug.h"
|
||||
#include "db_ops.h"
|
||||
#include "ip_ban.h"
|
||||
|
||||
#include <pthread.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
typedef struct thread_pool_job_node {
|
||||
uint64_t job_id;
|
||||
thread_pool_job_t job;
|
||||
struct thread_pool_job_node* next;
|
||||
} thread_pool_job_node_t;
|
||||
|
||||
typedef struct {
|
||||
thread_pool_job_node_t* head;
|
||||
thread_pool_job_node_t* tail;
|
||||
int size;
|
||||
int max_size;
|
||||
pthread_mutex_t mutex;
|
||||
pthread_cond_t cond;
|
||||
} thread_pool_queue_t;
|
||||
|
||||
typedef struct {
|
||||
int running;
|
||||
uint64_t next_job_id;
|
||||
|
||||
int reader_count;
|
||||
pthread_t* readers;
|
||||
pthread_t writer;
|
||||
|
||||
thread_pool_queue_t read_q;
|
||||
thread_pool_queue_t write_q;
|
||||
|
||||
thread_pool_wake_loop_cb wake_loop_cb;
|
||||
void* wake_loop_ctx;
|
||||
|
||||
char db_path[512];
|
||||
|
||||
pthread_mutex_t state_mutex;
|
||||
} thread_pool_state_t;
|
||||
|
||||
static thread_pool_state_t g_pool = {0};
|
||||
|
||||
typedef struct {
|
||||
pthread_mutex_t mutex;
|
||||
pthread_cond_t cond;
|
||||
int done;
|
||||
thread_pool_status_t status;
|
||||
void* result_data;
|
||||
size_t result_size;
|
||||
} thread_pool_wait_ctx_t;
|
||||
|
||||
static void wait_ctx_init(thread_pool_wait_ctx_t* ctx) {
|
||||
memset(ctx, 0, sizeof(*ctx));
|
||||
pthread_mutex_init(&ctx->mutex, NULL);
|
||||
pthread_cond_init(&ctx->cond, NULL);
|
||||
}
|
||||
|
||||
static void wait_ctx_destroy(thread_pool_wait_ctx_t* ctx) {
|
||||
pthread_mutex_destroy(&ctx->mutex);
|
||||
pthread_cond_destroy(&ctx->cond);
|
||||
}
|
||||
|
||||
static void wait_ctx_result_cb(const thread_pool_result_t* result, void* user_ctx) {
|
||||
thread_pool_wait_ctx_t* ctx = (thread_pool_wait_ctx_t*)user_ctx;
|
||||
if (!ctx || !result) return;
|
||||
|
||||
pthread_mutex_lock(&ctx->mutex);
|
||||
ctx->status = result->status;
|
||||
ctx->result_data = result->result_data;
|
||||
ctx->result_size = result->result_size;
|
||||
ctx->done = 1;
|
||||
pthread_cond_signal(&ctx->cond);
|
||||
pthread_mutex_unlock(&ctx->mutex);
|
||||
}
|
||||
|
||||
static void wait_ctx_wait(thread_pool_wait_ctx_t* ctx) {
|
||||
pthread_mutex_lock(&ctx->mutex);
|
||||
while (!ctx->done) {
|
||||
pthread_cond_wait(&ctx->cond, &ctx->mutex);
|
||||
}
|
||||
pthread_mutex_unlock(&ctx->mutex);
|
||||
}
|
||||
|
||||
static void free_count_payload(void* p) {
|
||||
thread_pool_count_payload_t* payload = (thread_pool_count_payload_t*)p;
|
||||
if (!payload) return;
|
||||
free(payload->sql);
|
||||
if (payload->bind_params) {
|
||||
for (int i = 0; i < payload->bind_param_count; i++) {
|
||||
free(payload->bind_params[i]);
|
||||
}
|
||||
free(payload->bind_params);
|
||||
}
|
||||
free(payload);
|
||||
}
|
||||
|
||||
static void free_req_payload(void* p) {
|
||||
thread_pool_req_payload_t* payload = (thread_pool_req_payload_t*)p;
|
||||
if (!payload) return;
|
||||
free(payload->sql);
|
||||
if (payload->bind_params) {
|
||||
for (int i = 0; i < payload->bind_param_count; i++) {
|
||||
free(payload->bind_params[i]);
|
||||
}
|
||||
free(payload->bind_params);
|
||||
}
|
||||
free(payload);
|
||||
}
|
||||
|
||||
static void free_store_event_payload(void* p) {
|
||||
thread_pool_store_event_payload_t* payload = (thread_pool_store_event_payload_t*)p;
|
||||
if (!payload) return;
|
||||
free(payload->id);
|
||||
free(payload->pubkey);
|
||||
free(payload->event_type);
|
||||
free(payload->content);
|
||||
free(payload->sig);
|
||||
free(payload->tags_json);
|
||||
free(payload->event_json);
|
||||
free(payload);
|
||||
}
|
||||
|
||||
static void queue_init(thread_pool_queue_t* q, int max_size) {
|
||||
memset(q, 0, sizeof(*q));
|
||||
q->max_size = (max_size > 0) ? max_size : 4096;
|
||||
pthread_mutex_init(&q->mutex, NULL);
|
||||
pthread_cond_init(&q->cond, NULL);
|
||||
}
|
||||
|
||||
static void queue_destroy(thread_pool_queue_t* q) {
|
||||
pthread_mutex_lock(&q->mutex);
|
||||
thread_pool_job_node_t* cur = q->head;
|
||||
while (cur) {
|
||||
thread_pool_job_node_t* next = cur->next;
|
||||
if (cur->job.payload_free && cur->job.payload) {
|
||||
cur->job.payload_free(cur->job.payload);
|
||||
}
|
||||
free(cur);
|
||||
cur = next;
|
||||
}
|
||||
q->head = q->tail = NULL;
|
||||
q->size = 0;
|
||||
pthread_mutex_unlock(&q->mutex);
|
||||
|
||||
pthread_mutex_destroy(&q->mutex);
|
||||
pthread_cond_destroy(&q->cond);
|
||||
}
|
||||
|
||||
static int queue_push(thread_pool_queue_t* q, thread_pool_job_node_t* node) {
|
||||
pthread_mutex_lock(&q->mutex);
|
||||
if (q->size >= q->max_size) {
|
||||
pthread_mutex_unlock(&q->mutex);
|
||||
return 0;
|
||||
}
|
||||
|
||||
node->next = NULL;
|
||||
if (!q->tail) {
|
||||
q->head = q->tail = node;
|
||||
} else {
|
||||
q->tail->next = node;
|
||||
q->tail = node;
|
||||
}
|
||||
q->size++;
|
||||
pthread_cond_signal(&q->cond);
|
||||
pthread_mutex_unlock(&q->mutex);
|
||||
return 1;
|
||||
}
|
||||
|
||||
static thread_pool_job_node_t* queue_pop(thread_pool_queue_t* q) {
|
||||
pthread_mutex_lock(&q->mutex);
|
||||
while (g_pool.running && q->size == 0) {
|
||||
pthread_cond_wait(&q->cond, &q->mutex);
|
||||
}
|
||||
|
||||
if (!g_pool.running && q->size == 0) {
|
||||
pthread_mutex_unlock(&q->mutex);
|
||||
return NULL;
|
||||
}
|
||||
|
||||
thread_pool_job_node_t* node = q->head;
|
||||
q->head = node->next;
|
||||
if (!q->head) q->tail = NULL;
|
||||
q->size--;
|
||||
pthread_mutex_unlock(&q->mutex);
|
||||
return node;
|
||||
}
|
||||
|
||||
static void wake_event_loop(void) {
|
||||
if (g_pool.wake_loop_cb) {
|
||||
g_pool.wake_loop_cb(g_pool.wake_loop_ctx);
|
||||
}
|
||||
}
|
||||
|
||||
static void* open_worker_connection(void) {
|
||||
void* db_conn = NULL;
|
||||
if (db_open_worker_connection(g_pool.db_path, &db_conn) != 0) {
|
||||
return NULL;
|
||||
}
|
||||
return db_conn;
|
||||
}
|
||||
|
||||
static void complete_job_with_result(thread_pool_job_node_t* node,
|
||||
thread_pool_status_t status,
|
||||
const char* message,
|
||||
void* result_data,
|
||||
size_t result_size) {
|
||||
thread_pool_result_t result;
|
||||
memset(&result, 0, sizeof(result));
|
||||
result.job_id = node->job_id;
|
||||
result.type = node->job.type;
|
||||
result.status = status;
|
||||
result.session = node->job.session;
|
||||
result.result_data = result_data;
|
||||
result.result_size = result_size;
|
||||
result.message = message;
|
||||
|
||||
if (node->job.result_cb) {
|
||||
node->job.result_cb(&result, node->job.result_cb_ctx);
|
||||
}
|
||||
wake_event_loop();
|
||||
|
||||
if (node->job.payload_free && node->job.payload) {
|
||||
node->job.payload_free(node->job.payload);
|
||||
}
|
||||
free(node);
|
||||
}
|
||||
|
||||
static void execute_count_job(thread_pool_job_node_t* node) {
|
||||
thread_pool_count_payload_t* payload = (thread_pool_count_payload_t*)node->job.payload;
|
||||
if (!payload || !payload->sql) {
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"COUNT payload missing", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
int count = 0;
|
||||
if (db_count_with_sql(payload->sql, (const char**)payload->bind_params,
|
||||
payload->bind_param_count, &count) != 0) {
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"COUNT query failed", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
thread_pool_count_result_t* out = calloc(1, sizeof(*out));
|
||||
if (!out) {
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"COUNT result allocation failed", NULL, 0);
|
||||
return;
|
||||
}
|
||||
out->count = count;
|
||||
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
|
||||
"COUNT query completed", out, sizeof(*out));
|
||||
}
|
||||
|
||||
static void execute_req_job(thread_pool_job_node_t* node) {
|
||||
thread_pool_req_payload_t* payload = (thread_pool_req_payload_t*)node->job.payload;
|
||||
if (!payload || !payload->sql) {
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"REQ payload missing", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
db_stmt_t* stmt = NULL;
|
||||
if (db_prepare(payload->sql, &stmt) != DB_OK) {
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"REQ prepare failed", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
for (int i = 0; i < payload->bind_param_count; i++) {
|
||||
const char* v = (payload->bind_params && payload->bind_params[i]) ? payload->bind_params[i] : "";
|
||||
(void)db_bind_text_param(stmt, i + 1, v);
|
||||
}
|
||||
|
||||
int cap = (payload->row_limit > 0) ? payload->row_limit : 500;
|
||||
if (cap < 1) cap = 1;
|
||||
|
||||
thread_pool_req_result_t* out = calloc(1, sizeof(*out));
|
||||
if (!out) {
|
||||
db_finalize_stmt(stmt);
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"REQ result allocation failed", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
out->event_json_rows = calloc((size_t)cap, sizeof(char*));
|
||||
if (!out->event_json_rows) {
|
||||
db_finalize_stmt(stmt);
|
||||
free(out);
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"REQ row array allocation failed", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
int rc = DB_OK;
|
||||
while ((rc = db_step_stmt(stmt)) == DB_ROW) {
|
||||
if (out->row_count >= cap) break;
|
||||
const char* row = db_column_text_value(stmt, 0);
|
||||
if (!row) continue;
|
||||
|
||||
out->event_json_rows[out->row_count] = strdup(row);
|
||||
if (!out->event_json_rows[out->row_count]) {
|
||||
break;
|
||||
}
|
||||
out->row_count++;
|
||||
}
|
||||
|
||||
db_finalize_stmt(stmt);
|
||||
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
|
||||
"REQ query completed", out, sizeof(*out));
|
||||
}
|
||||
|
||||
static void execute_store_event_job(thread_pool_job_node_t* node) {
|
||||
thread_pool_store_event_payload_t* payload = (thread_pool_store_event_payload_t*)node->job.payload;
|
||||
if (!payload || !payload->id || !payload->pubkey || !payload->content || !payload->sig) {
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"STORE_EVENT payload missing required fields", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
thread_pool_store_event_result_t* out = calloc(1, sizeof(*out));
|
||||
if (!out) {
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"STORE_EVENT result allocation failed", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
int step_rc = DB_ERROR;
|
||||
int extended_errcode = 0;
|
||||
if (db_insert_event_with_json(payload->id,
|
||||
payload->pubkey,
|
||||
payload->created_at,
|
||||
payload->kind,
|
||||
payload->event_type ? payload->event_type : "regular",
|
||||
payload->content,
|
||||
payload->sig,
|
||||
payload->tags_json ? payload->tags_json : "[]",
|
||||
payload->event_json ? payload->event_json : "{}",
|
||||
&step_rc,
|
||||
&extended_errcode) != 0) {
|
||||
free(out);
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"STORE_EVENT execution failed", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
out->step_rc = step_rc;
|
||||
out->extended_errcode = extended_errcode;
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
|
||||
"STORE_EVENT completed", out, sizeof(*out));
|
||||
}
|
||||
|
||||
static void execute_sub_log_created_job(thread_pool_job_node_t* node) {
|
||||
thread_pool_sub_log_created_payload_t* payload = (thread_pool_sub_log_created_payload_t*)node->job.payload;
|
||||
if (!payload || !payload->sub_id || !payload->wsi_ptr || !payload->client_ip) {
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"SUB_LOG_CREATED payload missing", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
if (db_log_subscription_created(payload->sub_id,
|
||||
payload->wsi_ptr,
|
||||
payload->client_ip,
|
||||
payload->filter_json ? payload->filter_json : "[]") != 0) {
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"SUB_LOG_CREATED failed", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
|
||||
"SUB_LOG_CREATED completed", NULL, 0);
|
||||
}
|
||||
|
||||
static void execute_sub_log_closed_job(thread_pool_job_node_t* node) {
|
||||
thread_pool_sub_log_closed_payload_t* payload = (thread_pool_sub_log_closed_payload_t*)node->job.payload;
|
||||
if (!payload || !payload->sub_id) {
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"SUB_LOG_CLOSED payload missing", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
if (db_log_subscription_closed(payload->sub_id, payload->client_ip) != 0) {
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"SUB_LOG_CLOSED failed", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
|
||||
"SUB_LOG_CLOSED completed", NULL, 0);
|
||||
}
|
||||
|
||||
static void execute_sub_log_disconnected_job(thread_pool_job_node_t* node) {
|
||||
thread_pool_sub_log_disconnected_payload_t* payload = (thread_pool_sub_log_disconnected_payload_t*)node->job.payload;
|
||||
if (!payload || !payload->client_ip) {
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"SUB_LOG_DISCONNECTED payload missing", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
if (db_log_subscription_disconnected(payload->client_ip) < 0) {
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"SUB_LOG_DISCONNECTED failed", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
|
||||
"SUB_LOG_DISCONNECTED completed", NULL, 0);
|
||||
}
|
||||
|
||||
static void execute_sub_update_events_job(thread_pool_job_node_t* node) {
|
||||
thread_pool_sub_update_events_payload_t* payload = (thread_pool_sub_update_events_payload_t*)node->job.payload;
|
||||
if (!payload || !payload->sub_id) {
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"SUB_UPDATE_EVENTS payload missing", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
if (db_update_subscription_events_sent(payload->sub_id, payload->events_sent) != 0) {
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"SUB_UPDATE_EVENTS failed", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
|
||||
"SUB_UPDATE_EVENTS completed", NULL, 0);
|
||||
}
|
||||
|
||||
static void execute_ip_ban_save_job(thread_pool_job_node_t* node) {
|
||||
ip_ban_save_to_db();
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
|
||||
"IP_BAN_SAVE completed", NULL, 0);
|
||||
}
|
||||
|
||||
static void execute_wal_checkpoint_job(thread_pool_job_node_t* node) {
|
||||
if (db_wal_checkpoint_passive() != 0) {
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_INTERNAL_ERROR,
|
||||
"WAL_CHECKPOINT failed", NULL, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_OK,
|
||||
"WAL_CHECKPOINT completed", NULL, 0);
|
||||
}
|
||||
|
||||
static void execute_read_job(thread_pool_job_node_t* node) {
|
||||
switch (node->job.type) {
|
||||
case THREAD_POOL_JOB_REQ_QUERY:
|
||||
execute_req_job(node);
|
||||
break;
|
||||
case THREAD_POOL_JOB_COUNT_QUERY:
|
||||
execute_count_job(node);
|
||||
break;
|
||||
default:
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_NOT_IMPLEMENTED,
|
||||
"Read job type not implemented", NULL, 0);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
static void execute_write_job(thread_pool_job_node_t* node) {
|
||||
switch (node->job.type) {
|
||||
case THREAD_POOL_JOB_STORE_EVENT:
|
||||
execute_store_event_job(node);
|
||||
break;
|
||||
case THREAD_POOL_JOB_LOG_SUB_CREATED:
|
||||
execute_sub_log_created_job(node);
|
||||
break;
|
||||
case THREAD_POOL_JOB_LOG_SUB_CLOSED:
|
||||
execute_sub_log_closed_job(node);
|
||||
break;
|
||||
case THREAD_POOL_JOB_LOG_SUB_DISCONNECTED:
|
||||
execute_sub_log_disconnected_job(node);
|
||||
break;
|
||||
case THREAD_POOL_JOB_UPDATE_SUB_EVENTS_SENT:
|
||||
execute_sub_update_events_job(node);
|
||||
break;
|
||||
case THREAD_POOL_JOB_IP_BAN_SAVE:
|
||||
execute_ip_ban_save_job(node);
|
||||
break;
|
||||
case THREAD_POOL_JOB_WAL_CHECKPOINT:
|
||||
execute_wal_checkpoint_job(node);
|
||||
break;
|
||||
default:
|
||||
complete_job_with_result(node, THREAD_POOL_STATUS_NOT_IMPLEMENTED,
|
||||
"Write job type not implemented", NULL, 0);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
static void* reader_worker_main(void* arg) {
|
||||
int reader_index = (int)(intptr_t)arg;
|
||||
|
||||
char thread_name[16];
|
||||
snprintf(thread_name, sizeof(thread_name), "db-read-%d", reader_index);
|
||||
pthread_setname_np(pthread_self(), thread_name);
|
||||
|
||||
void* worker_db = open_worker_connection();
|
||||
if (!worker_db) {
|
||||
DEBUG_ERROR("Reader worker failed to open SQLite connection");
|
||||
return NULL;
|
||||
}
|
||||
|
||||
while (g_pool.running) {
|
||||
thread_pool_job_node_t* node = queue_pop(&g_pool.read_q);
|
||||
if (!node) break;
|
||||
|
||||
execute_read_job(node);
|
||||
}
|
||||
|
||||
db_close_worker_connection(worker_db);
|
||||
return NULL;
|
||||
}
|
||||
|
||||
static void* writer_worker_main(void* arg) {
|
||||
(void)arg;
|
||||
|
||||
pthread_setname_np(pthread_self(), "db-write");
|
||||
|
||||
void* worker_db = open_worker_connection();
|
||||
if (!worker_db) {
|
||||
DEBUG_ERROR("Writer worker failed to open SQLite connection");
|
||||
return NULL;
|
||||
}
|
||||
|
||||
while (g_pool.running) {
|
||||
thread_pool_job_node_t* node = queue_pop(&g_pool.write_q);
|
||||
if (!node) break;
|
||||
|
||||
execute_write_job(node);
|
||||
}
|
||||
|
||||
if (db_wal_checkpoint_truncate() != 0) {
|
||||
DEBUG_WARN("Writer shutdown TRUNCATE checkpoint failed");
|
||||
}
|
||||
|
||||
db_close_worker_connection(worker_db);
|
||||
return NULL;
|
||||
}
|
||||
|
||||
int thread_pool_init(const thread_pool_config_t* config) {
|
||||
if (!config) return -1;
|
||||
|
||||
pthread_mutex_lock(&g_pool.state_mutex);
|
||||
if (g_pool.running) {
|
||||
pthread_mutex_unlock(&g_pool.state_mutex);
|
||||
return 0;
|
||||
}
|
||||
|
||||
g_pool.reader_count = (config->reader_threads > 0) ? config->reader_threads : 4;
|
||||
g_pool.readers = calloc((size_t)g_pool.reader_count, sizeof(pthread_t));
|
||||
if (!g_pool.readers) {
|
||||
pthread_mutex_unlock(&g_pool.state_mutex);
|
||||
return -1;
|
||||
}
|
||||
|
||||
queue_init(&g_pool.read_q, config->max_queue_depth);
|
||||
queue_init(&g_pool.write_q, config->max_queue_depth);
|
||||
|
||||
g_pool.next_job_id = 1;
|
||||
g_pool.wake_loop_cb = config->wake_loop_cb;
|
||||
g_pool.wake_loop_ctx = config->wake_loop_ctx;
|
||||
|
||||
const char* db_path = (config->db_path && config->db_path[0] != '\0') ? config->db_path : db_get_database_path();
|
||||
if (!db_path || db_path[0] == '\0') {
|
||||
free(g_pool.readers);
|
||||
g_pool.readers = NULL;
|
||||
pthread_mutex_unlock(&g_pool.state_mutex);
|
||||
return -1;
|
||||
}
|
||||
strncpy(g_pool.db_path, db_path, sizeof(g_pool.db_path) - 1);
|
||||
g_pool.db_path[sizeof(g_pool.db_path) - 1] = '\0';
|
||||
|
||||
g_pool.running = 1;
|
||||
|
||||
for (int i = 0; i < g_pool.reader_count; i++) {
|
||||
if (pthread_create(&g_pool.readers[i], NULL, reader_worker_main, (void*)(intptr_t)(i + 1)) != 0) {
|
||||
g_pool.running = 0;
|
||||
pthread_cond_broadcast(&g_pool.read_q.cond);
|
||||
pthread_cond_broadcast(&g_pool.write_q.cond);
|
||||
pthread_mutex_unlock(&g_pool.state_mutex);
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
|
||||
if (pthread_create(&g_pool.writer, NULL, writer_worker_main, NULL) != 0) {
|
||||
g_pool.running = 0;
|
||||
pthread_cond_broadcast(&g_pool.read_q.cond);
|
||||
pthread_cond_broadcast(&g_pool.write_q.cond);
|
||||
pthread_mutex_unlock(&g_pool.state_mutex);
|
||||
return -1;
|
||||
}
|
||||
|
||||
pthread_mutex_unlock(&g_pool.state_mutex);
|
||||
DEBUG_LOG("Thread pool initialized: %d readers + 1 writer", g_pool.reader_count);
|
||||
return 0;
|
||||
}
|
||||
|
||||
void thread_pool_shutdown(void) {
|
||||
pthread_mutex_lock(&g_pool.state_mutex);
|
||||
if (!g_pool.running) {
|
||||
pthread_mutex_unlock(&g_pool.state_mutex);
|
||||
return;
|
||||
}
|
||||
|
||||
g_pool.running = 0;
|
||||
pthread_cond_broadcast(&g_pool.read_q.cond);
|
||||
pthread_cond_broadcast(&g_pool.write_q.cond);
|
||||
pthread_mutex_unlock(&g_pool.state_mutex);
|
||||
|
||||
for (int i = 0; i < g_pool.reader_count; i++) {
|
||||
pthread_join(g_pool.readers[i], NULL);
|
||||
}
|
||||
pthread_join(g_pool.writer, NULL);
|
||||
|
||||
free(g_pool.readers);
|
||||
g_pool.readers = NULL;
|
||||
g_pool.reader_count = 0;
|
||||
|
||||
queue_destroy(&g_pool.read_q);
|
||||
queue_destroy(&g_pool.write_q);
|
||||
|
||||
DEBUG_LOG("Thread pool shutdown complete");
|
||||
}
|
||||
|
||||
int thread_pool_is_running(void) {
|
||||
return g_pool.running;
|
||||
}
|
||||
|
||||
static thread_pool_status_t submit_to_queue(thread_pool_queue_t* q, const thread_pool_job_t* job, uint64_t* out_job_id) {
|
||||
if (!g_pool.running) return THREAD_POOL_STATUS_SHUTTING_DOWN;
|
||||
if (!job) return THREAD_POOL_STATUS_INTERNAL_ERROR;
|
||||
|
||||
thread_pool_job_node_t* node = calloc(1, sizeof(*node));
|
||||
if (!node) return THREAD_POOL_STATUS_INTERNAL_ERROR;
|
||||
|
||||
node->job = *job;
|
||||
|
||||
pthread_mutex_lock(&g_pool.state_mutex);
|
||||
node->job_id = g_pool.next_job_id++;
|
||||
pthread_mutex_unlock(&g_pool.state_mutex);
|
||||
|
||||
if (out_job_id) *out_job_id = node->job_id;
|
||||
|
||||
if (!queue_push(q, node)) {
|
||||
if (job->payload_free && job->payload) {
|
||||
job->payload_free(job->payload);
|
||||
}
|
||||
free(node);
|
||||
return THREAD_POOL_STATUS_QUEUE_FULL;
|
||||
}
|
||||
|
||||
return THREAD_POOL_STATUS_OK;
|
||||
}
|
||||
|
||||
thread_pool_status_t thread_pool_submit_read(const thread_pool_job_t* job, uint64_t* out_job_id) {
|
||||
return submit_to_queue(&g_pool.read_q, job, out_job_id);
|
||||
}
|
||||
|
||||
thread_pool_status_t thread_pool_submit_write(const thread_pool_job_t* job, uint64_t* out_job_id) {
|
||||
return submit_to_queue(&g_pool.write_q, job, out_job_id);
|
||||
}
|
||||
|
||||
int thread_pool_submit_wal_checkpoint(void) {
|
||||
if (!thread_pool_is_running()) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
thread_pool_job_t job;
|
||||
memset(&job, 0, sizeof(job));
|
||||
job.type = THREAD_POOL_JOB_WAL_CHECKPOINT;
|
||||
|
||||
thread_pool_status_t rc = thread_pool_submit_write(&job, NULL);
|
||||
return (rc == THREAD_POOL_STATUS_OK) ? 0 : -1;
|
||||
}
|
||||
|
||||
int thread_pool_execute_count_sync(const char* sql, const char** bind_params, int bind_param_count, int* out_count) {
|
||||
if (!sql || !out_count) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
if (!thread_pool_is_running()) {
|
||||
return db_count_with_sql(sql, bind_params, bind_param_count, out_count);
|
||||
}
|
||||
|
||||
thread_pool_count_payload_t* payload = calloc(1, sizeof(*payload));
|
||||
if (!payload) return -1;
|
||||
|
||||
payload->sql = strdup(sql);
|
||||
payload->bind_param_count = bind_param_count;
|
||||
if (!payload->sql) {
|
||||
free_count_payload(payload);
|
||||
return -1;
|
||||
}
|
||||
|
||||
if (bind_param_count > 0) {
|
||||
payload->bind_params = calloc((size_t)bind_param_count, sizeof(char*));
|
||||
if (!payload->bind_params) {
|
||||
free_count_payload(payload);
|
||||
return -1;
|
||||
}
|
||||
for (int i = 0; i < bind_param_count; i++) {
|
||||
const char* v = (bind_params && bind_params[i]) ? bind_params[i] : "";
|
||||
payload->bind_params[i] = strdup(v);
|
||||
if (!payload->bind_params[i]) {
|
||||
free_count_payload(payload);
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
thread_pool_wait_ctx_t wait_ctx;
|
||||
wait_ctx_init(&wait_ctx);
|
||||
|
||||
thread_pool_job_t job;
|
||||
memset(&job, 0, sizeof(job));
|
||||
job.type = THREAD_POOL_JOB_COUNT_QUERY;
|
||||
job.payload = payload;
|
||||
job.payload_free = free_count_payload;
|
||||
job.result_cb = wait_ctx_result_cb;
|
||||
job.result_cb_ctx = &wait_ctx;
|
||||
|
||||
thread_pool_status_t submit_rc = thread_pool_submit_read(&job, NULL);
|
||||
if (submit_rc != THREAD_POOL_STATUS_OK) {
|
||||
free_count_payload(payload);
|
||||
wait_ctx_destroy(&wait_ctx);
|
||||
return -1;
|
||||
}
|
||||
|
||||
wait_ctx_wait(&wait_ctx);
|
||||
|
||||
int rc = -1;
|
||||
if (wait_ctx.status == THREAD_POOL_STATUS_OK && wait_ctx.result_data) {
|
||||
thread_pool_count_result_t* result = (thread_pool_count_result_t*)wait_ctx.result_data;
|
||||
*out_count = result->count;
|
||||
free(result);
|
||||
rc = 0;
|
||||
}
|
||||
|
||||
wait_ctx_destroy(&wait_ctx);
|
||||
return rc;
|
||||
}
|
||||
|
||||
int thread_pool_execute_req_sync(const char* sql, const char** bind_params, int bind_param_count,
|
||||
int row_limit, thread_pool_req_result_t** out_result) {
|
||||
if (!sql || !out_result) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
if (!thread_pool_is_running()) {
|
||||
db_stmt_t* stmt = NULL;
|
||||
if (db_prepare(sql, &stmt) != DB_OK) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
for (int i = 0; i < bind_param_count; i++) {
|
||||
const char* v = (bind_params && bind_params[i]) ? bind_params[i] : "";
|
||||
(void)db_bind_text_param(stmt, i + 1, v);
|
||||
}
|
||||
|
||||
int cap = (row_limit > 0) ? row_limit : 500;
|
||||
if (cap < 1) cap = 1;
|
||||
|
||||
thread_pool_req_result_t* result = calloc(1, sizeof(*result));
|
||||
if (!result) {
|
||||
db_finalize_stmt(stmt);
|
||||
return -1;
|
||||
}
|
||||
|
||||
result->event_json_rows = calloc((size_t)cap, sizeof(char*));
|
||||
if (!result->event_json_rows) {
|
||||
free(result);
|
||||
db_finalize_stmt(stmt);
|
||||
return -1;
|
||||
}
|
||||
|
||||
int step_rc = DB_OK;
|
||||
while ((step_rc = db_step_stmt(stmt)) == DB_ROW) {
|
||||
if (result->row_count >= cap) break;
|
||||
const char* row = db_column_text_value(stmt, 0);
|
||||
if (!row) continue;
|
||||
|
||||
result->event_json_rows[result->row_count] = strdup(row);
|
||||
if (!result->event_json_rows[result->row_count]) {
|
||||
break;
|
||||
}
|
||||
result->row_count++;
|
||||
}
|
||||
|
||||
db_finalize_stmt(stmt);
|
||||
*out_result = result;
|
||||
return 0;
|
||||
}
|
||||
|
||||
thread_pool_req_payload_t* payload = calloc(1, sizeof(*payload));
|
||||
if (!payload) return -1;
|
||||
|
||||
payload->sql = strdup(sql);
|
||||
payload->bind_param_count = bind_param_count;
|
||||
payload->row_limit = row_limit;
|
||||
if (!payload->sql) {
|
||||
free_req_payload(payload);
|
||||
return -1;
|
||||
}
|
||||
|
||||
if (bind_param_count > 0) {
|
||||
payload->bind_params = calloc((size_t)bind_param_count, sizeof(char*));
|
||||
if (!payload->bind_params) {
|
||||
free_req_payload(payload);
|
||||
return -1;
|
||||
}
|
||||
for (int i = 0; i < bind_param_count; i++) {
|
||||
const char* v = (bind_params && bind_params[i]) ? bind_params[i] : "";
|
||||
payload->bind_params[i] = strdup(v);
|
||||
if (!payload->bind_params[i]) {
|
||||
free_req_payload(payload);
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
thread_pool_wait_ctx_t wait_ctx;
|
||||
wait_ctx_init(&wait_ctx);
|
||||
|
||||
thread_pool_job_t job;
|
||||
memset(&job, 0, sizeof(job));
|
||||
job.type = THREAD_POOL_JOB_REQ_QUERY;
|
||||
job.payload = payload;
|
||||
job.payload_free = free_req_payload;
|
||||
job.result_cb = wait_ctx_result_cb;
|
||||
job.result_cb_ctx = &wait_ctx;
|
||||
|
||||
thread_pool_status_t submit_rc = thread_pool_submit_read(&job, NULL);
|
||||
if (submit_rc != THREAD_POOL_STATUS_OK) {
|
||||
free_req_payload(payload);
|
||||
wait_ctx_destroy(&wait_ctx);
|
||||
return -1;
|
||||
}
|
||||
|
||||
wait_ctx_wait(&wait_ctx);
|
||||
|
||||
int rc = -1;
|
||||
if (wait_ctx.status == THREAD_POOL_STATUS_OK && wait_ctx.result_data) {
|
||||
*out_result = (thread_pool_req_result_t*)wait_ctx.result_data;
|
||||
rc = 0;
|
||||
}
|
||||
|
||||
wait_ctx_destroy(&wait_ctx);
|
||||
return rc;
|
||||
}
|
||||
|
||||
void thread_pool_free_req_result(thread_pool_req_result_t* result) {
|
||||
if (!result) return;
|
||||
if (result->event_json_rows) {
|
||||
for (int i = 0; i < result->row_count; i++) {
|
||||
free(result->event_json_rows[i]);
|
||||
}
|
||||
free(result->event_json_rows);
|
||||
}
|
||||
free(result);
|
||||
}
|
||||
|
||||
int thread_pool_execute_store_event_sync(const thread_pool_store_event_payload_t* payload,
|
||||
thread_pool_store_event_result_t* out_result) {
|
||||
if (!payload || !out_result || !payload->id || !payload->pubkey || !payload->content || !payload->sig) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
if (!thread_pool_is_running()) {
|
||||
int step_rc = DB_ERROR;
|
||||
int extended_errcode = 0;
|
||||
if (db_insert_event_with_json(payload->id,
|
||||
payload->pubkey,
|
||||
payload->created_at,
|
||||
payload->kind,
|
||||
payload->event_type ? payload->event_type : "regular",
|
||||
payload->content,
|
||||
payload->sig,
|
||||
payload->tags_json ? payload->tags_json : "[]",
|
||||
payload->event_json ? payload->event_json : "{}",
|
||||
&step_rc,
|
||||
&extended_errcode) != 0) {
|
||||
return -1;
|
||||
}
|
||||
out_result->step_rc = step_rc;
|
||||
out_result->extended_errcode = extended_errcode;
|
||||
return 0;
|
||||
}
|
||||
|
||||
thread_pool_store_event_payload_t* payload_copy = calloc(1, sizeof(*payload_copy));
|
||||
if (!payload_copy) return -1;
|
||||
|
||||
payload_copy->id = payload->id ? strdup(payload->id) : NULL;
|
||||
payload_copy->pubkey = payload->pubkey ? strdup(payload->pubkey) : NULL;
|
||||
payload_copy->created_at = payload->created_at;
|
||||
payload_copy->kind = payload->kind;
|
||||
payload_copy->event_type = payload->event_type ? strdup(payload->event_type) : strdup("regular");
|
||||
payload_copy->content = payload->content ? strdup(payload->content) : NULL;
|
||||
payload_copy->sig = payload->sig ? strdup(payload->sig) : NULL;
|
||||
payload_copy->tags_json = payload->tags_json ? strdup(payload->tags_json) : strdup("[]");
|
||||
payload_copy->event_json = payload->event_json ? strdup(payload->event_json) : strdup("{}");
|
||||
|
||||
if (!payload_copy->id || !payload_copy->pubkey || !payload_copy->event_type ||
|
||||
!payload_copy->content || !payload_copy->sig || !payload_copy->tags_json || !payload_copy->event_json) {
|
||||
free_store_event_payload(payload_copy);
|
||||
return -1;
|
||||
}
|
||||
|
||||
thread_pool_wait_ctx_t wait_ctx;
|
||||
wait_ctx_init(&wait_ctx);
|
||||
|
||||
thread_pool_job_t job;
|
||||
memset(&job, 0, sizeof(job));
|
||||
job.type = THREAD_POOL_JOB_STORE_EVENT;
|
||||
job.payload = payload_copy;
|
||||
job.payload_free = free_store_event_payload;
|
||||
job.result_cb = wait_ctx_result_cb;
|
||||
job.result_cb_ctx = &wait_ctx;
|
||||
|
||||
thread_pool_status_t submit_rc = thread_pool_submit_write(&job, NULL);
|
||||
if (submit_rc != THREAD_POOL_STATUS_OK) {
|
||||
free_store_event_payload(payload_copy);
|
||||
wait_ctx_destroy(&wait_ctx);
|
||||
return -1;
|
||||
}
|
||||
|
||||
wait_ctx_wait(&wait_ctx);
|
||||
|
||||
int rc = -1;
|
||||
if (wait_ctx.status == THREAD_POOL_STATUS_OK && wait_ctx.result_data) {
|
||||
thread_pool_store_event_result_t* result = (thread_pool_store_event_result_t*)wait_ctx.result_data;
|
||||
out_result->step_rc = result->step_rc;
|
||||
out_result->extended_errcode = result->extended_errcode;
|
||||
free(result);
|
||||
rc = 0;
|
||||
}
|
||||
|
||||
wait_ctx_destroy(&wait_ctx);
|
||||
return rc;
|
||||
}
|
||||
|
||||
__attribute__((constructor))
|
||||
static void thread_pool_state_init_once(void) {
|
||||
pthread_mutex_init(&g_pool.state_mutex, NULL);
|
||||
}
|
||||
@@ -0,0 +1,145 @@
|
||||
#ifndef THREAD_POOL_H
|
||||
#define THREAD_POOL_H
|
||||
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
typedef enum {
|
||||
THREAD_POOL_JOB_REQ_QUERY = 0,
|
||||
THREAD_POOL_JOB_COUNT_QUERY,
|
||||
THREAD_POOL_JOB_STORE_EVENT,
|
||||
THREAD_POOL_JOB_DELETE_EVENT,
|
||||
THREAD_POOL_JOB_LOG_SUB_CREATED,
|
||||
THREAD_POOL_JOB_LOG_SUB_CLOSED,
|
||||
THREAD_POOL_JOB_LOG_SUB_DISCONNECTED,
|
||||
THREAD_POOL_JOB_UPDATE_SUB_EVENTS_SENT,
|
||||
THREAD_POOL_JOB_IP_BAN_SAVE,
|
||||
THREAD_POOL_JOB_WAL_CHECKPOINT,
|
||||
THREAD_POOL_JOB_CUSTOM
|
||||
} thread_pool_job_type_t;
|
||||
|
||||
typedef enum {
|
||||
THREAD_POOL_STATUS_OK = 0,
|
||||
THREAD_POOL_STATUS_NOT_IMPLEMENTED,
|
||||
THREAD_POOL_STATUS_QUEUE_FULL,
|
||||
THREAD_POOL_STATUS_SHUTTING_DOWN,
|
||||
THREAD_POOL_STATUS_INTERNAL_ERROR
|
||||
} thread_pool_status_t;
|
||||
|
||||
typedef struct {
|
||||
uint64_t job_id;
|
||||
thread_pool_job_type_t type;
|
||||
thread_pool_status_t status;
|
||||
void* session;
|
||||
void* result_data;
|
||||
size_t result_size;
|
||||
const char* message;
|
||||
} thread_pool_result_t;
|
||||
|
||||
typedef void (*thread_pool_result_cb)(const thread_pool_result_t* result, void* user_ctx);
|
||||
typedef void (*thread_pool_payload_free_cb)(void* payload);
|
||||
typedef void (*thread_pool_wake_loop_cb)(void* wake_ctx);
|
||||
|
||||
typedef struct {
|
||||
thread_pool_job_type_t type;
|
||||
void* session;
|
||||
void* payload;
|
||||
size_t payload_size;
|
||||
thread_pool_payload_free_cb payload_free;
|
||||
thread_pool_result_cb result_cb;
|
||||
void* result_cb_ctx;
|
||||
} thread_pool_job_t;
|
||||
|
||||
typedef struct {
|
||||
int reader_threads;
|
||||
int max_queue_depth;
|
||||
const char* db_path;
|
||||
thread_pool_wake_loop_cb wake_loop_cb;
|
||||
void* wake_loop_ctx;
|
||||
} thread_pool_config_t;
|
||||
|
||||
typedef struct {
|
||||
char* sql;
|
||||
char** bind_params;
|
||||
int bind_param_count;
|
||||
} thread_pool_count_payload_t;
|
||||
|
||||
typedef struct {
|
||||
char* sql;
|
||||
char** bind_params;
|
||||
int bind_param_count;
|
||||
int row_limit;
|
||||
} thread_pool_req_payload_t;
|
||||
|
||||
typedef struct {
|
||||
char* id;
|
||||
char* pubkey;
|
||||
long long created_at;
|
||||
int kind;
|
||||
char* event_type;
|
||||
char* content;
|
||||
char* sig;
|
||||
char* tags_json;
|
||||
char* event_json;
|
||||
} thread_pool_store_event_payload_t;
|
||||
|
||||
typedef struct {
|
||||
int count;
|
||||
} thread_pool_count_result_t;
|
||||
|
||||
typedef struct {
|
||||
char** event_json_rows;
|
||||
int row_count;
|
||||
} thread_pool_req_result_t;
|
||||
|
||||
typedef struct {
|
||||
int step_rc;
|
||||
int extended_errcode;
|
||||
} thread_pool_store_event_result_t;
|
||||
|
||||
typedef struct {
|
||||
char* sub_id;
|
||||
char* wsi_ptr;
|
||||
char* client_ip;
|
||||
char* filter_json;
|
||||
} thread_pool_sub_log_created_payload_t;
|
||||
|
||||
typedef struct {
|
||||
char* sub_id;
|
||||
char* client_ip;
|
||||
} thread_pool_sub_log_closed_payload_t;
|
||||
|
||||
typedef struct {
|
||||
char* client_ip;
|
||||
} thread_pool_sub_log_disconnected_payload_t;
|
||||
|
||||
typedef struct {
|
||||
char* sub_id;
|
||||
int events_sent;
|
||||
} thread_pool_sub_update_events_payload_t;
|
||||
|
||||
int thread_pool_init(const thread_pool_config_t* config);
|
||||
void thread_pool_shutdown(void);
|
||||
int thread_pool_is_running(void);
|
||||
|
||||
thread_pool_status_t thread_pool_submit_read(const thread_pool_job_t* job, uint64_t* out_job_id);
|
||||
thread_pool_status_t thread_pool_submit_write(const thread_pool_job_t* job, uint64_t* out_job_id);
|
||||
|
||||
int thread_pool_submit_wal_checkpoint(void);
|
||||
|
||||
int thread_pool_execute_count_sync(const char* sql, const char** bind_params, int bind_param_count, int* out_count);
|
||||
int thread_pool_execute_req_sync(const char* sql, const char** bind_params, int bind_param_count,
|
||||
int row_limit, thread_pool_req_result_t** out_result);
|
||||
void thread_pool_free_req_result(thread_pool_req_result_t* result);
|
||||
int thread_pool_execute_store_event_sync(const thread_pool_store_event_payload_t* payload,
|
||||
thread_pool_store_event_result_t* out_result);
|
||||
|
||||
#ifdef __cplusplus
|
||||
}
|
||||
#endif
|
||||
|
||||
#endif // THREAD_POOL_H
|
||||
+1399
-244
File diff suppressed because it is too large
Load Diff
+25
-1
@@ -1,4 +1,4 @@
|
||||
// WebSocket protocol structures and constants for C-Relay
|
||||
// WebSocket protocol structures and constants for C-Relay-PG
|
||||
// This header defines structures shared between main.c and websockets.c
|
||||
|
||||
#ifndef WEBSOCKETS_H
|
||||
@@ -19,6 +19,11 @@
|
||||
#define MALFORMED_REQUEST_BLOCK_DURATION 3600 // 1 hour in seconds
|
||||
#define RATE_LIMIT_CLEANUP_INTERVAL 300 // 5 minutes
|
||||
|
||||
// Maximum number of messages allowed in a per-client write queue.
|
||||
// When exceeded, new messages are dropped to prevent unbounded memory growth
|
||||
// under high-traffic or slow-client conditions.
|
||||
#define MAX_MESSAGE_QUEUE_SIZE 500
|
||||
|
||||
// Filter validation constants
|
||||
#define MAX_FILTERS_PER_REQUEST 10
|
||||
#define MAX_AUTHORS_PER_FILTER 1000
|
||||
@@ -84,8 +89,23 @@ struct per_session_data {
|
||||
int db_queries_executed; // Total SELECT queries executed by this connection
|
||||
int db_rows_returned; // Total rows returned across all queries
|
||||
time_t query_tracking_start; // When connection was established (for rate calculation)
|
||||
|
||||
// Session activity tracking for idle connection banning
|
||||
int session_active; // 1 if client sent REQ or EVENT, 0 otherwise
|
||||
int idle_timeout_sec; // Timeout value for this session (copied from config)
|
||||
int is_websocket; // 1 if this is a WebSocket connection, 0 for HTTP
|
||||
};
|
||||
|
||||
// Get current active WebSocket connection count
|
||||
int get_active_connection_count(void);
|
||||
|
||||
// Safely resolve per-session data only for currently tracked live WebSocket sessions.
|
||||
// Returns 1 and sets *out_pss on success, otherwise returns 0.
|
||||
int websocket_get_live_pss(struct lws* wsi, struct per_session_data** out_pss);
|
||||
|
||||
// Actual relay port bound by libwebsockets at runtime (after fallback/retry logic)
|
||||
extern int g_relay_port;
|
||||
|
||||
// NIP-11 HTTP session data structure for managing buffer lifetime
|
||||
struct nip11_session_data {
|
||||
int type; // 0 for NIP-11
|
||||
@@ -102,6 +122,10 @@ int start_websocket_relay(int port_override, int strict_port);
|
||||
int queue_message(struct lws* wsi, struct per_session_data* pss, const char* message, size_t length, enum lws_write_protocol type);
|
||||
int process_message_queue(struct lws* wsi, struct per_session_data* pss);
|
||||
|
||||
// Zero-copy variant: caller allocates (LWS_PRE + length) bytes, writes message at buf+LWS_PRE,
|
||||
// then passes ownership to the queue. The queue will free buf when done. No memcpy performed.
|
||||
int queue_message_take_ownership(struct lws* wsi, struct per_session_data* pss, unsigned char* buf, size_t length, enum lws_write_protocol type);
|
||||
|
||||
// Auth rules checking function from request_validator.c
|
||||
int check_database_auth_rules(const char *pubkey, const char *operation, const char *resource_hash);
|
||||
|
||||
|
||||
+34
-34
@@ -8,7 +8,7 @@ The C Nostr Relay now uses a revolutionary **zero-configuration** approach where
|
||||
|
||||
## Files
|
||||
|
||||
- **`c-relay.service`** - SystemD service unit file
|
||||
- **`c-relay-pg.service`** - SystemD service unit file
|
||||
- **`install-service.sh`** - Automated installation script
|
||||
- **`uninstall-service.sh`** - Automated uninstall script
|
||||
- **`README.md`** - This documentation
|
||||
@@ -27,12 +27,12 @@ The C Nostr Relay now uses a revolutionary **zero-configuration** approach where
|
||||
|
||||
3. **Start the service:**
|
||||
```bash
|
||||
sudo systemctl start c-relay
|
||||
sudo systemctl start c-relay-pg
|
||||
```
|
||||
|
||||
4. **Check admin keys (IMPORTANT!):**
|
||||
```bash
|
||||
sudo journalctl -u c-relay --since="1 hour ago" | grep "Admin Private Key"
|
||||
sudo journalctl -u c-relay-pg --since="1 hour ago" | grep "Admin Private Key"
|
||||
```
|
||||
|
||||
## Event-Based Configuration System
|
||||
@@ -61,7 +61,7 @@ On first startup, the relay will:
|
||||
|
||||
```bash
|
||||
# View first startup logs to get admin private key
|
||||
sudo journalctl -u c-relay --since="1 hour ago" | grep -A 5 "IMPORTANT: SAVE THIS ADMIN PRIVATE KEY"
|
||||
sudo journalctl -u c-relay-pg --since="1 hour ago" | grep -A 5 "IMPORTANT: SAVE THIS ADMIN PRIVATE KEY"
|
||||
```
|
||||
|
||||
The admin private key is needed to update relay configuration by sending signed kind 33334 events.
|
||||
@@ -72,10 +72,10 @@ The admin private key is needed to update relay configuration by sending signed
|
||||
|
||||
```bash
|
||||
# Find the database file
|
||||
ls /opt/c-relay/*.nrdb
|
||||
ls /opt/c-relay-pg/*.nrdb
|
||||
|
||||
# View configuration event
|
||||
sqlite3 /opt/c-relay/<relay_pubkey>.nrdb "SELECT content, tags FROM events WHERE kind = 33334;"
|
||||
sqlite3 /opt/c-relay-pg/<relay_pubkey>.nrdb "SELECT content, tags FROM events WHERE kind = 33334;"
|
||||
```
|
||||
|
||||
### Updating Configuration
|
||||
@@ -93,57 +93,57 @@ Send a new kind 33334 event to the relay via WebSocket:
|
||||
|
||||
```bash
|
||||
# Start service
|
||||
sudo systemctl start c-relay
|
||||
sudo systemctl start c-relay-pg
|
||||
|
||||
# Stop service
|
||||
sudo systemctl stop c-relay
|
||||
sudo systemctl stop c-relay-pg
|
||||
|
||||
# Restart service
|
||||
sudo systemctl restart c-relay
|
||||
sudo systemctl restart c-relay-pg
|
||||
|
||||
# Enable auto-start on boot
|
||||
sudo systemctl enable c-relay
|
||||
sudo systemctl enable c-relay-pg
|
||||
|
||||
# Check status
|
||||
sudo systemctl status c-relay
|
||||
sudo systemctl status c-relay-pg
|
||||
|
||||
# View logs (live)
|
||||
sudo journalctl -u c-relay -f
|
||||
sudo journalctl -u c-relay-pg -f
|
||||
|
||||
# View recent logs
|
||||
sudo journalctl -u c-relay --since="1 hour ago"
|
||||
sudo journalctl -u c-relay-pg --since="1 hour ago"
|
||||
```
|
||||
|
||||
### Log Analysis
|
||||
|
||||
```bash
|
||||
# Check for successful startup
|
||||
sudo journalctl -u c-relay | grep "First-time startup sequence completed"
|
||||
sudo journalctl -u c-relay-pg | grep "First-time startup sequence completed"
|
||||
|
||||
# Find admin keys
|
||||
sudo journalctl -u c-relay | grep "Admin Private Key"
|
||||
sudo journalctl -u c-relay-pg | grep "Admin Private Key"
|
||||
|
||||
# Check configuration updates
|
||||
sudo journalctl -u c-relay | grep "Configuration updated via kind 33334"
|
||||
sudo journalctl -u c-relay-pg | grep "Configuration updated via kind 33334"
|
||||
|
||||
# Monitor real-time activity
|
||||
sudo journalctl -u c-relay -f | grep -E "(INFO|SUCCESS|ERROR)"
|
||||
sudo journalctl -u c-relay-pg -f | grep -E "(INFO|SUCCESS|ERROR)"
|
||||
```
|
||||
|
||||
## File Locations
|
||||
|
||||
After installation:
|
||||
|
||||
- **Binary:** `/opt/c-relay/c_relay_x86`
|
||||
- **Database:** `/opt/c-relay/<relay_pubkey>.nrdb` (created automatically)
|
||||
- **Service File:** `/etc/systemd/system/c-relay.service`
|
||||
- **User:** `c-relay` (system user created automatically)
|
||||
- **Binary:** `/opt/c-relay-pg/c_relay_pg_x86`
|
||||
- **Database:** `/opt/c-relay-pg/<relay_pubkey>.nrdb` (created automatically)
|
||||
- **Service File:** `/etc/systemd/system/c-relay-pg.service`
|
||||
- **User:** `c-relay-pg` (system user created automatically)
|
||||
|
||||
## Security Features
|
||||
|
||||
The systemd service includes security hardening:
|
||||
|
||||
- Runs as dedicated system user `c-relay`
|
||||
- Runs as dedicated system user `c-relay-pg`
|
||||
- `NoNewPrivileges=true`
|
||||
- `ProtectSystem=strict`
|
||||
- `ProtectHome=true`
|
||||
@@ -165,7 +165,7 @@ The database file contains everything:
|
||||
|
||||
```bash
|
||||
# Backup database file
|
||||
sudo cp /opt/c-relay/*.nrdb /backup/location/
|
||||
sudo cp /opt/c-relay-pg/*.nrdb /backup/location/
|
||||
|
||||
# The .nrdb file contains:
|
||||
# - All Nostr events
|
||||
@@ -177,7 +177,7 @@ sudo cp /opt/c-relay/*.nrdb /backup/location/
|
||||
|
||||
To migrate to new server:
|
||||
|
||||
1. Copy `.nrdb` file to new server's `/opt/c-relay/` directory
|
||||
1. Copy `.nrdb` file to new server's `/opt/c-relay-pg/` directory
|
||||
2. Install service with `install-service.sh`
|
||||
3. Start service - it will automatically detect existing configuration
|
||||
|
||||
@@ -187,39 +187,39 @@ To migrate to new server:
|
||||
|
||||
```bash
|
||||
# Check service status
|
||||
sudo systemctl status c-relay
|
||||
sudo systemctl status c-relay-pg
|
||||
|
||||
# Check logs for errors
|
||||
sudo journalctl -u c-relay --no-pager
|
||||
sudo journalctl -u c-relay-pg --no-pager
|
||||
|
||||
# Check if binary exists and is executable
|
||||
ls -la /opt/c-relay/c_relay_x86
|
||||
ls -la /opt/c-relay-pg/c_relay_pg_x86
|
||||
|
||||
# Check permissions
|
||||
sudo -u c-relay ls -la /opt/c-relay/
|
||||
sudo -u c-relay-pg ls -la /opt/c-relay-pg/
|
||||
```
|
||||
|
||||
### Database Issues
|
||||
|
||||
```bash
|
||||
# Check if database file exists
|
||||
ls -la /opt/c-relay/*.nrdb*
|
||||
ls -la /opt/c-relay-pg/*.nrdb*
|
||||
|
||||
# Check database integrity
|
||||
sqlite3 /opt/c-relay/*.nrdb "PRAGMA integrity_check;"
|
||||
sqlite3 /opt/c-relay-pg/*.nrdb "PRAGMA integrity_check;"
|
||||
|
||||
# View database schema
|
||||
sqlite3 /opt/c-relay/*.nrdb ".schema"
|
||||
sqlite3 /opt/c-relay-pg/*.nrdb ".schema"
|
||||
```
|
||||
|
||||
### Configuration Issues
|
||||
|
||||
```bash
|
||||
# Check if configuration event exists
|
||||
sqlite3 /opt/c-relay/*.nrdb "SELECT COUNT(*) FROM events WHERE kind = 33334;"
|
||||
sqlite3 /opt/c-relay-pg/*.nrdb "SELECT COUNT(*) FROM events WHERE kind = 33334;"
|
||||
|
||||
# View configuration event
|
||||
sqlite3 /opt/c-relay/*.nrdb "SELECT id, created_at, LENGTH(tags) FROM events WHERE kind = 33334;"
|
||||
sqlite3 /opt/c-relay-pg/*.nrdb "SELECT id, created_at, LENGTH(tags) FROM events WHERE kind = 33334;"
|
||||
```
|
||||
|
||||
## Uninstallation
|
||||
@@ -238,7 +238,7 @@ The uninstall script will:
|
||||
|
||||
For issues with the event-based configuration system:
|
||||
|
||||
1. Check service logs: `sudo journalctl -u c-relay -f`
|
||||
1. Check service logs: `sudo journalctl -u c-relay-pg -f`
|
||||
2. Verify database integrity
|
||||
3. Ensure admin private key is saved securely
|
||||
4. Check WebSocket connectivity on port 8888
|
||||
|
||||
@@ -1,27 +1,27 @@
|
||||
[Unit]
|
||||
Description=C Nostr Relay Server (Event-Based Configuration)
|
||||
Documentation=https://github.com/your-repo/c-relay
|
||||
Documentation=https://github.com/your-repo/c-relay-pg
|
||||
After=network.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=c-relay
|
||||
Group=c-relay
|
||||
WorkingDirectory=/opt/c-relay
|
||||
User=c-relay-pg
|
||||
Group=c-relay-pg
|
||||
WorkingDirectory=/opt/c-relay-pg
|
||||
Environment=DEBUG_LEVEL=0
|
||||
ExecStart=/opt/c-relay/c_relay_x86 --debug-level=$DEBUG_LEVEL
|
||||
ExecStart=/opt/c-relay-pg/c_relay_pg_x86 --debug-level=$DEBUG_LEVEL
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
SyslogIdentifier=c-relay
|
||||
SyslogIdentifier=c-relay-pg
|
||||
|
||||
# Security settings
|
||||
NoNewPrivileges=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ReadWritePaths=/opt/c-relay
|
||||
ReadWritePaths=/opt/c-relay-pg
|
||||
PrivateTmp=true
|
||||
ProtectKernelTunables=true
|
||||
ProtectKernelModules=true
|
||||
|
||||
@@ -6,10 +6,10 @@
|
||||
set -e
|
||||
|
||||
# Configuration
|
||||
SERVICE_NAME="c-relay"
|
||||
SERVICE_USER="c-relay"
|
||||
INSTALL_DIR="/opt/c-relay"
|
||||
BINARY_NAME="c_relay_x86"
|
||||
SERVICE_NAME="c-relay-pg"
|
||||
SERVICE_USER="c-relay-pg"
|
||||
INSTALL_DIR="/opt/c-relay-pg"
|
||||
BINARY_NAME="c_relay_pg_x86"
|
||||
|
||||
# Colors for output
|
||||
RED='\033[0;31m'
|
||||
|
||||
+13
-13
@@ -1,15 +1,15 @@
|
||||
#!/bin/bash
|
||||
|
||||
# C-Relay Systemd Service Installation Script
|
||||
# This script installs the C-Relay as a systemd service
|
||||
# C-Relay-PG Systemd Service Installation Script
|
||||
# This script installs the C-Relay-PG as a systemd service
|
||||
|
||||
set -e
|
||||
|
||||
# Configuration
|
||||
INSTALL_DIR="/opt/c-relay"
|
||||
SERVICE_NAME="c-relay"
|
||||
SERVICE_FILE="c-relay.service"
|
||||
BINARY_NAME="c_relay_x86"
|
||||
INSTALL_DIR="/opt/c-relay-pg"
|
||||
SERVICE_NAME="c-relay-pg"
|
||||
SERVICE_FILE="c-relay-pg.service"
|
||||
BINARY_NAME="c_relay_pg_x86"
|
||||
|
||||
# Colors for output
|
||||
RED='\033[0;31m'
|
||||
@@ -17,7 +17,7 @@ GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
echo -e "${GREEN}=== C-Relay Systemd Service Installation ===${NC}"
|
||||
echo -e "${GREEN}=== C-Relay-PG Systemd Service Installation ===${NC}"
|
||||
|
||||
# Check if running as root
|
||||
if [[ $EUID -ne 0 ]]; then
|
||||
@@ -39,12 +39,12 @@ if [ ! -f "$SERVICE_FILE" ]; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Create c-relay user if it doesn't exist
|
||||
if ! id "c-relay" &>/dev/null; then
|
||||
echo -e "${YELLOW}Creating c-relay user...${NC}"
|
||||
useradd --system --shell /bin/false --home-dir $INSTALL_DIR --create-home c-relay
|
||||
# Create c-relay-pg user if it doesn't exist
|
||||
if ! id "c-relay-pg" &>/dev/null; then
|
||||
echo -e "${YELLOW}Creating c-relay-pg user...${NC}"
|
||||
useradd --system --shell /bin/false --home-dir $INSTALL_DIR --create-home c-relay-pg
|
||||
else
|
||||
echo -e "${GREEN}User c-relay already exists${NC}"
|
||||
echo -e "${GREEN}User c-relay-pg already exists${NC}"
|
||||
fi
|
||||
|
||||
# Create installation directory
|
||||
@@ -59,7 +59,7 @@ chmod +x $INSTALL_DIR/$BINARY_NAME
|
||||
|
||||
# Set permissions
|
||||
echo -e "${YELLOW}Setting permissions...${NC}"
|
||||
chown -R c-relay:c-relay $INSTALL_DIR
|
||||
chown -R c-relay-pg:c-relay-pg $INSTALL_DIR
|
||||
|
||||
# Install systemd service
|
||||
echo -e "${YELLOW}Installing systemd service...${NC}"
|
||||
|
||||
@@ -6,9 +6,9 @@
|
||||
set -e
|
||||
|
||||
# Configuration
|
||||
SERVICE_NAME="c-relay"
|
||||
SERVICE_USER="c-relay"
|
||||
INSTALL_DIR="/opt/c-relay"
|
||||
SERVICE_NAME="c-relay-pg"
|
||||
SERVICE_USER="c-relay-pg"
|
||||
INSTALL_DIR="/opt/c-relay-pg"
|
||||
|
||||
# Colors for output
|
||||
RED='\033[0;31m'
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user