Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c02ae2be5b | ||
|
|
fd3efdd01f | ||
|
|
232f93c16f | ||
|
|
9a9365dcfa | ||
|
|
5577b57149 | ||
|
|
22c943d495 | ||
|
|
7e3d9b6825 | ||
|
|
547d22d09f | ||
|
|
ce9ae5a488 | ||
|
|
9b248b740f | ||
|
|
0f77aeb72b | ||
|
|
bb64651a0c | ||
|
|
17be9c2b03 | ||
|
|
f396c622b6 | ||
|
|
cbd260c1ae | ||
|
|
bc9ac290ab | ||
|
|
192eeb248d | ||
|
|
da6c505420 | ||
|
|
c077c209e5 | ||
|
|
9bf02702c2 | ||
|
|
0920cc092d | ||
|
|
a89460be5d | ||
|
|
ff2a3aa335 | ||
|
|
94b61e8a7c | ||
|
|
b0c0754e83 | ||
|
|
3265e3d114 | ||
|
|
927659ece1 | ||
|
|
b6ff4150b4 | ||
|
|
63bc526163 | ||
|
|
a1f712236a | ||
|
|
06e6c17b7b | ||
|
|
0751a7c55c | ||
|
|
f1728932a9 | ||
|
|
ef8bdef2a8 | ||
|
|
c11a8ba292 | ||
|
|
0f124fe575 | ||
|
|
6b20452fab | ||
|
|
10c19bc243 | ||
|
|
3d7aa2196f | ||
|
|
a416c3f275 | ||
|
|
bba9baabc3 | ||
|
|
31187c4c4f | ||
|
|
55f862b879 | ||
|
|
76c9b3fcf0 | ||
|
|
929bd09164 | ||
|
|
d17f1dd8d5 | ||
|
|
207a949835 | ||
|
|
d08f4e4221 | ||
|
|
c96736fa6a | ||
|
|
f8ec4ae924 | ||
|
|
fe7304ac7f | ||
|
|
e5d39c984b | ||
|
|
5e45f21e35 | ||
|
|
5321a238b8 | ||
|
|
083bc14972 | ||
|
|
11aaccba9b | ||
|
|
bd1bbd763d | ||
|
|
2bd7aa5a10 | ||
|
|
361912ec85 | ||
|
|
0de491382e | ||
|
|
3148bbbee7 | ||
|
|
3965ba04d8 | ||
|
|
b96af938bd | ||
|
|
89c8248013 | ||
|
|
d8f477c6cf | ||
|
|
040eeadb13 | ||
|
|
83f8b0ab88 | ||
|
|
b82f7eaaf3 | ||
|
|
0dc5b75d7c | ||
|
|
e94b1a81e3 | ||
|
|
1adabdbc4e | ||
|
|
81d44c3d8c | ||
|
|
7acc0bdd90 | ||
|
|
c4ef71d673 | ||
|
|
086d2af56c | ||
|
|
18a7deec54 | ||
|
|
c794370a3f | ||
|
|
2ed4b96058 | ||
|
|
c0051b22be | ||
|
|
4cc2d2376e | ||
|
|
30dc4bf67d | ||
|
|
a1928cc5d7 | ||
|
|
7bf0757b1f | ||
|
|
11b0a88cdd | ||
|
|
e8f8e3b0cf | ||
|
|
35b1461ff6 | ||
|
|
87c6aa5e16 | ||
|
|
f0462929ea | ||
|
|
0ccf1959c2 | ||
|
|
ef172c8356 | ||
|
|
ae0370b47f | ||
|
|
137df6253d | ||
|
|
79f06bf1ed | ||
|
|
9e5a42fd3c | ||
|
|
553d398ef5 | ||
|
|
ac0bf17c4f | ||
|
|
4efd7140d7 | ||
|
|
43b9027d4a | ||
|
|
9a0b2714f5 | ||
|
|
10fc80aa07 | ||
|
|
9fa4e1dcf1 | ||
|
|
11c9f8c2a4 | ||
|
|
17ddf3f2b8 | ||
|
|
f70d5077e8 | ||
|
|
19cc1fa804 | ||
|
|
ed1de7d32d | ||
|
|
7b02c6a2cb | ||
|
|
dba229040d | ||
|
|
d8b09e0dbb | ||
|
|
73fb681118 | ||
|
|
cf0f606dd9 | ||
|
|
cb165ddab1 | ||
|
|
58eb19e4e7 | ||
|
|
aeaecddb5a |
+1
-1
@@ -11,4 +11,4 @@ copy_executable_local.sh
|
||||
nostr_login_lite/
|
||||
style_guide/
|
||||
nostr-tools
|
||||
|
||||
.test_keys
|
||||
+11
-11
@@ -1,12 +1,12 @@
|
||||
|
||||
# AGENTS.md - AI Agent Integration Guide for Architect Mode
|
||||
|
||||
**Project-Specific Information for AI Agents Working with C-Relay in Architect Mode**
|
||||
**Project-Specific Information for AI Agents Working with C-Relay-PG in Architect Mode**
|
||||
|
||||
## Critical Architecture Understanding
|
||||
|
||||
### System Architecture Overview
|
||||
C-Relay implements a **unique event-based configuration architecture** that fundamentally differs from traditional Nostr relays:
|
||||
C-Relay-PG implements a **unique event-based configuration architecture** that fundamentally differs from traditional Nostr relays:
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
|
||||
@@ -51,9 +51,9 @@ C-Relay implements a **unique event-based configuration architecture** that fund
|
||||
## Architectural Decision Analysis
|
||||
|
||||
### Configuration System Design
|
||||
**Traditional Approach vs C-Relay:**
|
||||
**Traditional Approach vs C-Relay-PG:**
|
||||
```
|
||||
Traditional: C-Relay:
|
||||
Traditional: C-Relay-PG:
|
||||
config.json → kind 33334 events
|
||||
ENV variables → cryptographically signed tags
|
||||
File watching → database polling/restart
|
||||
@@ -119,18 +119,18 @@ File watching → database polling/restart
|
||||
```
|
||||
Developer Machine:
|
||||
├── ./make_and_restart_relay.sh
|
||||
├── build/c_relay_x86
|
||||
├── build/c_relay_pg_x86
|
||||
├── build/<relay_pubkey>.db
|
||||
└── relay.log
|
||||
```
|
||||
|
||||
### Production SystemD Deployment
|
||||
```
|
||||
/opt/c-relay/:
|
||||
├── c_relay_x86
|
||||
/opt/c-relay-pg/:
|
||||
├── c_relay_pg_x86
|
||||
├── <relay_pubkey>.db
|
||||
├── systemd service (c-relay.service)
|
||||
└── c-relay user isolation
|
||||
├── systemd service (c-relay-pg.service)
|
||||
└── c-relay-pg user isolation
|
||||
```
|
||||
|
||||
### Container Deployment Architecture
|
||||
@@ -144,7 +144,7 @@ Container:
|
||||
|
||||
### Reverse Proxy Architecture
|
||||
```
|
||||
Internet → Nginx/HAProxy → C-Relay
|
||||
Internet → Nginx/HAProxy → C-Relay-PG
|
||||
├── WebSocket upgrade handling
|
||||
├── SSL termination
|
||||
└── Rate limiting
|
||||
@@ -292,7 +292,7 @@ Admin Signs Event → WebSocket Submit → Validate → Store → Restart Requir
|
||||
**Decision**: Same port serves both protocols
|
||||
**Consequences**: Simplified deployment, protocol detection overhead, libwebsockets dependency
|
||||
|
||||
These architectural decisions form the foundation of C-Relay's unique approach to Nostr relay implementation and should be carefully considered when planning extensions or modifications.
|
||||
These architectural decisions form the foundation of C-Relay-PG's unique approach to Nostr relay implementation and should be carefully considered when planning extensions or modifications.
|
||||
**
|
||||
|
||||
[Response interrupted by a tool use result. Only one tool may be used at a time and should be placed at the end of the message.]
|
||||
Executable
BIN
Binary file not shown.
@@ -1,6 +1,6 @@
|
||||
# AGENTS.md - AI Agent Integration Guide
|
||||
|
||||
**Project-Specific Information for AI Agents Working with C-Relay**
|
||||
**Project-Specific Information for AI Agents Working with C-Relay-PG**
|
||||
|
||||
## Critical Build Commands
|
||||
|
||||
@@ -15,9 +15,9 @@
|
||||
- Starts relay in background with proper logging
|
||||
|
||||
### Architecture-Specific Binary Outputs
|
||||
- **x86_64**: `./build/c_relay_x86`
|
||||
- **ARM64**: `./build/c_relay_arm64`
|
||||
- **Other**: `./build/c_relay_$(ARCH)`
|
||||
- **x86_64**: `./build/c_relay_pg_x86`
|
||||
- **ARM64**: `./build/c_relay_pg_arm64`
|
||||
- **Other**: `./build/c_relay_pg_$(ARCH)`
|
||||
|
||||
### Database File Naming Convention
|
||||
- **Format**: `<relay_pubkey>.db` (NOT `.nrdb` as shown in docs)
|
||||
@@ -75,10 +75,10 @@
|
||||
### Process Management
|
||||
```bash
|
||||
# Kill existing relay processes
|
||||
pkill -f "c_relay_"
|
||||
pkill -f "c_relay_pg_"
|
||||
|
||||
# Check running processes
|
||||
ps aux | grep c_relay_
|
||||
ps aux | grep c_relay_pg_
|
||||
|
||||
# Force kill port binding
|
||||
fuser -k 8888/tcp
|
||||
@@ -95,7 +95,7 @@ fuser -k 8888/tcp
|
||||
- Event configuration tests: `tests/event_config_tests.sh`
|
||||
|
||||
### SystemD Integration Considerations
|
||||
- Service runs as `c-relay` user in `/opt/c-relay`
|
||||
- Service runs as `c-relay-pg` user in `/opt/c-relay-pg`
|
||||
- Database files created in WorkingDirectory automatically
|
||||
- No environment variables needed (event-based config)
|
||||
- Resource limits: 65536 file descriptors, 4096 processes
|
||||
@@ -137,7 +137,7 @@ fuser -k 8888/tcp
|
||||
## Quick Debugging Commands
|
||||
```bash
|
||||
# Check relay status
|
||||
ps aux | grep c_relay_ && netstat -tln | grep 8888
|
||||
ps aux | grep c_relay_pg_ && netstat -tln | grep 8888
|
||||
|
||||
# View logs
|
||||
tail -f relay.log
|
||||
|
||||
@@ -0,0 +1,995 @@
|
||||
# C-Relay-PG API Documentation
|
||||
|
||||
Complete API reference for the C-Relay-PG event-based administration system and advanced features.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Overview](#overview)
|
||||
- [Authentication](#authentication)
|
||||
- [Admin API](#admin-api)
|
||||
- [Event Structure](#event-structure)
|
||||
- [Configuration Management](#configuration-management)
|
||||
- [Auth Rules Management](#auth-rules-management)
|
||||
- [System Commands](#system-commands)
|
||||
- [Database Queries](#database-queries)
|
||||
- [Configuration Reference](#configuration-reference)
|
||||
- [Real-time Monitoring](#real-time-monitoring)
|
||||
- [Direct Message Admin](#direct-message-admin)
|
||||
- [Response Formats](#response-formats)
|
||||
- [Error Handling](#error-handling)
|
||||
- [Examples](#examples)
|
||||
|
||||
## Overview
|
||||
|
||||
C-Relay-PG uses an innovative **event-based administration system** where all configuration and management commands are sent as cryptographically signed Nostr events. This provides:
|
||||
|
||||
- **Cryptographic security**: All commands must be signed with the admin private key
|
||||
- **Audit trail**: Complete history of all administrative actions
|
||||
- **Real-time updates**: Configuration changes applied instantly
|
||||
- **Standard protocol**: Uses Nostr events, no custom protocols
|
||||
- **NIP-44 encryption**: All admin commands and responses are encrypted
|
||||
|
||||
### Key Concepts
|
||||
|
||||
1. **Admin Keypair**: Generated on first startup, used to sign all admin commands
|
||||
2. **Relay Keypair**: The relay's identity on the Nostr network
|
||||
3. **Admin Events**: Kind 23456 events with NIP-44 encrypted commands
|
||||
4. **Response Events**: Kind 23457 events with NIP-44 encrypted responses
|
||||
5. **Event-Based Config**: All settings stored as events in the database
|
||||
|
||||
## Authentication
|
||||
|
||||
### Admin Private Key
|
||||
|
||||
The admin private key is displayed **only once** during first startup:
|
||||
|
||||
```
|
||||
========================================
|
||||
IMPORTANT: SAVE THIS ADMIN PRIVATE KEY
|
||||
========================================
|
||||
Admin Private Key: nsec1abc123...
|
||||
Admin Public Key: npub1def456...
|
||||
========================================
|
||||
```
|
||||
|
||||
**Critical**: Save this key immediately. It cannot be recovered and is required for all administrative operations.
|
||||
|
||||
### Secure Storage
|
||||
|
||||
Store the admin private key securely:
|
||||
|
||||
```bash
|
||||
# Environment variable
|
||||
export C_RELAY_ADMIN_KEY="nsec1abc123..."
|
||||
|
||||
# Secure file
|
||||
echo "nsec1abc123..." > ~/.c-relay-pg-admin
|
||||
chmod 600 ~/.c-relay-pg-admin
|
||||
|
||||
# Password manager (recommended)
|
||||
# Store in 1Password, Bitwarden, etc.
|
||||
```
|
||||
|
||||
### Key Loss Recovery
|
||||
|
||||
If you lose the admin private key:
|
||||
|
||||
1. Stop the relay
|
||||
2. Delete the database file (`*.db`)
|
||||
3. Restart the relay (generates new keys)
|
||||
4. **Note**: This deletes all events and configuration
|
||||
|
||||
## Admin API
|
||||
|
||||
### Event Structure
|
||||
|
||||
All admin commands use the same event structure with NIP-44 encrypted content.
|
||||
|
||||
#### Admin Command Event (Kind 23456)
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "computed_event_id",
|
||||
"pubkey": "admin_public_key_hex",
|
||||
"created_at": 1234567890,
|
||||
"kind": 23456,
|
||||
"content": "AqHBUgcM7dXFYLQuDVzGwMST1G8jtWYyVvYxXhVGEu4nAb4LVw...",
|
||||
"tags": [
|
||||
["p", "relay_public_key_hex"]
|
||||
],
|
||||
"sig": "event_signature"
|
||||
}
|
||||
```
|
||||
|
||||
**Fields**:
|
||||
- `kind`: Must be `23456` for admin commands
|
||||
- `pubkey`: Admin public key (hex format)
|
||||
- `content`: NIP-44 encrypted JSON array containing the command
|
||||
- `tags`: Must include `["p", "relay_pubkey"]` tag
|
||||
- `sig`: Valid signature from admin private key
|
||||
|
||||
#### Encrypted Content Format
|
||||
|
||||
The `content` field contains a NIP-44 encrypted JSON array:
|
||||
|
||||
```json
|
||||
["command_name", "param1", "param2", ...]
|
||||
```
|
||||
|
||||
Examples:
|
||||
```json
|
||||
["config_query", "all"]
|
||||
["config_update", [{"key": "relay_name", "value": "My Relay", ...}]]
|
||||
["blacklist", "pubkey", "abc123..."]
|
||||
["auth_query", "all"]
|
||||
["sql_query", "SELECT * FROM events LIMIT 10"]
|
||||
```
|
||||
|
||||
#### Admin Response Event (Kind 23457)
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "response_event_id",
|
||||
"pubkey": "relay_public_key_hex",
|
||||
"created_at": 1234567890,
|
||||
"kind": 23457,
|
||||
"content": "BpKCVhfN8eYtRmPqSvWxZnMkL2gHjUiOp3rTyEwQaS5dFg...",
|
||||
"tags": [
|
||||
["p", "admin_public_key_hex"],
|
||||
["e", "request_event_id"]
|
||||
],
|
||||
"sig": "response_signature"
|
||||
}
|
||||
```
|
||||
|
||||
**Fields**:
|
||||
- `kind`: Always `23457` for admin responses
|
||||
- `pubkey`: Relay public key (hex format)
|
||||
- `content`: NIP-44 encrypted JSON response object
|
||||
- `tags`: Includes `["p", "admin_pubkey"]` and optionally `["e", "request_id"]`
|
||||
- `sig`: Valid signature from relay private key
|
||||
|
||||
### Configuration Management
|
||||
|
||||
#### Query All Configuration
|
||||
|
||||
**Command**:
|
||||
```json
|
||||
["config_query", "all"]
|
||||
```
|
||||
|
||||
**Response** (decrypted):
|
||||
```json
|
||||
{
|
||||
"query_type": "config_all",
|
||||
"total_results": 27,
|
||||
"timestamp": 1234567890,
|
||||
"data": [
|
||||
{
|
||||
"key": "relay_name",
|
||||
"value": "C-Relay-PG",
|
||||
"data_type": "string",
|
||||
"category": "relay",
|
||||
"description": "Relay name displayed in NIP-11"
|
||||
},
|
||||
{
|
||||
"key": "auth_enabled",
|
||||
"value": "false",
|
||||
"data_type": "boolean",
|
||||
"category": "auth",
|
||||
"description": "Enable whitelist/blacklist authentication"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### Update Configuration
|
||||
|
||||
**Command**:
|
||||
```json
|
||||
["config_update", [
|
||||
{
|
||||
"key": "relay_name",
|
||||
"value": "My Awesome Relay",
|
||||
"data_type": "string",
|
||||
"category": "relay"
|
||||
},
|
||||
{
|
||||
"key": "max_subscriptions_per_client",
|
||||
"value": "50",
|
||||
"data_type": "integer",
|
||||
"category": "limits"
|
||||
}
|
||||
]]
|
||||
```
|
||||
|
||||
**Response** (decrypted):
|
||||
```json
|
||||
{
|
||||
"query_type": "config_update",
|
||||
"status": "success",
|
||||
"total_results": 2,
|
||||
"timestamp": 1234567890,
|
||||
"data": [
|
||||
{
|
||||
"key": "relay_name",
|
||||
"value": "My Awesome Relay",
|
||||
"status": "updated"
|
||||
},
|
||||
{
|
||||
"key": "max_subscriptions_per_client",
|
||||
"value": "50",
|
||||
"status": "updated"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Auth Rules Management
|
||||
|
||||
#### Add Blacklist Rule
|
||||
|
||||
**Command**:
|
||||
```json
|
||||
["blacklist", "pubkey", "abc123def456..."]
|
||||
```
|
||||
|
||||
**Response** (decrypted):
|
||||
```json
|
||||
{
|
||||
"query_type": "auth_add",
|
||||
"status": "success",
|
||||
"message": "Blacklist rule added successfully",
|
||||
"timestamp": 1234567890
|
||||
}
|
||||
```
|
||||
|
||||
#### Add Whitelist Rule
|
||||
|
||||
**Command**:
|
||||
```json
|
||||
["whitelist", "pubkey", "def456abc123..."]
|
||||
```
|
||||
|
||||
#### Delete Auth Rule
|
||||
|
||||
**Command**:
|
||||
```json
|
||||
["delete_auth_rule", "blacklist", "pubkey", "abc123def456..."]
|
||||
```
|
||||
|
||||
**Response** (decrypted):
|
||||
```json
|
||||
{
|
||||
"query_type": "auth_delete",
|
||||
"status": "success",
|
||||
"message": "Auth rule deleted successfully",
|
||||
"timestamp": 1234567890
|
||||
}
|
||||
```
|
||||
|
||||
#### Query All Auth Rules
|
||||
|
||||
**Command**:
|
||||
```json
|
||||
["auth_query", "all"]
|
||||
```
|
||||
|
||||
**Response** (decrypted):
|
||||
```json
|
||||
{
|
||||
"query_type": "auth_rules_all",
|
||||
"total_results": 5,
|
||||
"timestamp": 1234567890,
|
||||
"data": [
|
||||
{
|
||||
"rule_type": "blacklist",
|
||||
"pattern_type": "pubkey",
|
||||
"pattern_value": "abc123...",
|
||||
"action": "deny"
|
||||
},
|
||||
{
|
||||
"rule_type": "whitelist",
|
||||
"pattern_type": "pubkey",
|
||||
"pattern_value": "def456...",
|
||||
"action": "allow"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### Query Specific Rule Type
|
||||
|
||||
**Command**:
|
||||
```json
|
||||
["auth_query", "whitelist"]
|
||||
```
|
||||
|
||||
#### Query Specific Pattern
|
||||
|
||||
**Command**:
|
||||
```json
|
||||
["auth_query", "pattern", "abc123..."]
|
||||
```
|
||||
|
||||
### System Commands
|
||||
|
||||
#### System Status
|
||||
|
||||
**Command**:
|
||||
```json
|
||||
["system_command", "system_status"]
|
||||
```
|
||||
|
||||
**Response** (decrypted):
|
||||
```json
|
||||
{
|
||||
"query_type": "system_status",
|
||||
"timestamp": 1234567890,
|
||||
"status": "running",
|
||||
"uptime_seconds": 86400,
|
||||
"version": "0.6.0",
|
||||
"relay_pubkey": "relay_public_key_hex",
|
||||
"database_size_bytes": 10485760,
|
||||
"total_events": 15432,
|
||||
"active_connections": 42,
|
||||
"active_subscriptions": 156
|
||||
}
|
||||
```
|
||||
|
||||
#### Clear All Auth Rules
|
||||
|
||||
**Command**:
|
||||
```json
|
||||
["system_command", "clear_all_auth_rules"]
|
||||
```
|
||||
|
||||
**Response** (decrypted):
|
||||
```json
|
||||
{
|
||||
"query_type": "system_command",
|
||||
"status": "success",
|
||||
"message": "All auth rules cleared",
|
||||
"timestamp": 1234567890
|
||||
}
|
||||
```
|
||||
|
||||
#### Database Statistics
|
||||
|
||||
**Command**:
|
||||
```json
|
||||
["stats_query"]
|
||||
```
|
||||
|
||||
**Response** (decrypted):
|
||||
```json
|
||||
{
|
||||
"query_type": "stats_query",
|
||||
"timestamp": 1234567890,
|
||||
"database_size_bytes": 10485760,
|
||||
"total_events": 15432,
|
||||
"database_created_at": 1234567800,
|
||||
"latest_event_at": 1234567890,
|
||||
"event_kinds": [
|
||||
{
|
||||
"kind": 1,
|
||||
"count": 12000,
|
||||
"percentage": 77.8
|
||||
},
|
||||
{
|
||||
"kind": 0,
|
||||
"count": 2500,
|
||||
"percentage": 16.2
|
||||
}
|
||||
],
|
||||
"time_stats": {
|
||||
"total": 15432,
|
||||
"last_24h": 234,
|
||||
"last_7d": 1456,
|
||||
"last_30d": 5432
|
||||
},
|
||||
"top_pubkeys": [
|
||||
{
|
||||
"pubkey": "abc123...",
|
||||
"event_count": 1234,
|
||||
"percentage": 8.0
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Database Queries
|
||||
|
||||
#### SQL Query Command
|
||||
|
||||
Execute read-only SQL queries against the relay database.
|
||||
|
||||
**Command**:
|
||||
```json
|
||||
["sql_query", "SELECT * FROM events ORDER BY created_at DESC LIMIT 10"]
|
||||
```
|
||||
|
||||
**Response** (decrypted):
|
||||
```json
|
||||
{
|
||||
"query_type": "sql_query",
|
||||
"request_id": "request_event_id",
|
||||
"timestamp": 1234567890,
|
||||
"query": "SELECT * FROM events ORDER BY created_at DESC LIMIT 10",
|
||||
"execution_time_ms": 45,
|
||||
"row_count": 10,
|
||||
"columns": ["id", "pubkey", "created_at", "kind", "content", "tags", "sig"],
|
||||
"rows": [
|
||||
["abc123...", "def456...", 1234567890, 1, "Hello world", "[]", "sig123..."],
|
||||
["ghi789...", "jkl012...", 1234567880, 0, "{\"name\":\"Alice\"}", "[]", "sig456..."]
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### Security Features
|
||||
|
||||
- **Read-only**: Only SELECT statements allowed
|
||||
- **Query timeout**: 5 seconds maximum
|
||||
- **Result limit**: 1000 rows maximum
|
||||
- **Logging**: All queries logged with execution time
|
||||
- **Validation**: SQL injection protection
|
||||
|
||||
#### Available Tables and Views
|
||||
|
||||
**Core Tables**:
|
||||
- `events` - All Nostr events
|
||||
- `config` - Configuration parameters
|
||||
- `auth_rules` - Authentication rules
|
||||
- `subscription_events` - Subscription lifecycle log
|
||||
- `event_broadcasts` - Event broadcast log
|
||||
|
||||
**Views**:
|
||||
- `recent_events` - Last 1000 events
|
||||
- `event_stats` - Event statistics by type
|
||||
- `subscription_analytics` - Subscription metrics
|
||||
- `active_subscriptions_log` - Currently active subscriptions
|
||||
- `event_kinds_view` - Event distribution by kind
|
||||
- `top_pubkeys_view` - Top 10 pubkeys by event count
|
||||
- `time_stats_view` - Time-based statistics
|
||||
|
||||
#### Example Queries
|
||||
|
||||
**Recent events**:
|
||||
```sql
|
||||
SELECT id, pubkey, created_at, kind
|
||||
FROM events
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 20
|
||||
```
|
||||
|
||||
**Event distribution**:
|
||||
```sql
|
||||
SELECT * FROM event_kinds_view
|
||||
ORDER BY count DESC
|
||||
```
|
||||
|
||||
**Active subscriptions**:
|
||||
```sql
|
||||
SELECT * FROM active_subscriptions_log
|
||||
ORDER BY created_at DESC
|
||||
```
|
||||
|
||||
**Database statistics**:
|
||||
```sql
|
||||
SELECT
|
||||
(SELECT COUNT(*) FROM events) as total_events,
|
||||
(SELECT COUNT(*) FROM subscription_events) as total_subscriptions,
|
||||
(SELECT COUNT(DISTINCT pubkey) FROM events) as unique_pubkeys
|
||||
```
|
||||
|
||||
**Events by specific pubkey**:
|
||||
```sql
|
||||
SELECT id, created_at, kind, content
|
||||
FROM events
|
||||
WHERE pubkey = 'abc123...'
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 50
|
||||
```
|
||||
|
||||
**Events in time range**:
|
||||
```sql
|
||||
SELECT COUNT(*) as count, kind
|
||||
FROM events
|
||||
WHERE created_at BETWEEN 1234567000 AND 1234567890
|
||||
GROUP BY kind
|
||||
ORDER BY count DESC
|
||||
```
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
### Basic Relay Settings
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `relay_name` | string | "C-Relay-PG" | Relay name (NIP-11) |
|
||||
| `relay_description` | string | "C Nostr Relay" | Relay description |
|
||||
| `relay_contact` | string | "" | Admin contact info |
|
||||
| `relay_software` | string | "c-relay-pg" | Software identifier |
|
||||
| `relay_version` | string | auto | Software version |
|
||||
| `supported_nips` | string | "1,9,11,13,15,20,33,40,42,45,50,70" | Supported NIPs |
|
||||
| `language_tags` | string | "*" | Supported languages |
|
||||
| `relay_countries` | string | "*" | Supported countries |
|
||||
| `posting_policy` | string | "" | Posting policy URL |
|
||||
| `payments_url` | string | "" | Payment URL |
|
||||
|
||||
### Connection & Limits
|
||||
|
||||
| Key | Type | Default | Range | Restart Required |
|
||||
|-----|------|---------|-------|------------------|
|
||||
| `max_connections` | integer | 1000 | 1-10000 | Yes |
|
||||
| `max_subscriptions_per_client` | integer | 25 | 1-100 | No |
|
||||
| `max_total_subscriptions` | integer | 5000 | 100-50000 | No |
|
||||
| `max_message_length` | integer | 65536 | 1024-1048576 | No |
|
||||
| `max_event_tags` | integer | 2000 | 10-10000 | No |
|
||||
| `max_content_length` | integer | 65536 | 1-1048576 | No |
|
||||
|
||||
### Authentication & Access Control
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `auth_enabled` | boolean | false | Enable whitelist/blacklist |
|
||||
| `nip42_auth_required` | boolean | false | Require NIP-42 auth |
|
||||
| `nip42_auth_required_kinds` | string | "" | Kinds requiring NIP-42 |
|
||||
| `nip42_challenge_timeout` | integer | 300 | Challenge timeout (seconds) |
|
||||
|
||||
### Proof of Work (NIP-13)
|
||||
|
||||
| Key | Type | Default | Values | Description |
|
||||
|-----|------|---------|--------|-------------|
|
||||
| `pow_min_difficulty` | integer | 0 | 0-40 | Minimum PoW difficulty |
|
||||
| `pow_mode` | string | "optional" | disabled/optional/required | PoW enforcement mode |
|
||||
|
||||
### Event Expiration (NIP-40)
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `nip40_expiration_enabled` | boolean | true | Enable expiration support |
|
||||
| `nip40_expiration_strict` | boolean | false | Reject expired events |
|
||||
| `nip40_expiration_filter` | boolean | true | Filter expired from results |
|
||||
| `nip40_expiration_grace_period` | integer | 300 | Grace period (seconds) |
|
||||
|
||||
### Monitoring
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `kind_24567_reporting_throttle_sec` | integer | 5 | Monitoring event throttle |
|
||||
|
||||
### Dynamic vs Restart-Required
|
||||
|
||||
**Dynamic (No Restart)**:
|
||||
- All NIP-11 relay information
|
||||
- Authentication settings
|
||||
- Subscription limits
|
||||
- Event validation limits
|
||||
- Proof of Work settings
|
||||
- Expiration settings
|
||||
|
||||
**Restart Required**:
|
||||
- `max_connections`
|
||||
- `relay_port`
|
||||
- Database settings
|
||||
|
||||
## Real-time Monitoring
|
||||
|
||||
C-Relay-PG provides subscription-based real-time monitoring using ephemeral events (kind 24567).
|
||||
|
||||
### Activation
|
||||
|
||||
Subscribe to kind 24567 events to activate monitoring:
|
||||
|
||||
```json
|
||||
["REQ", "monitoring-sub", {"kinds": [24567]}]
|
||||
```
|
||||
|
||||
### Monitoring Event Types
|
||||
|
||||
Subscribe to specific monitoring types using d-tag filters:
|
||||
|
||||
```json
|
||||
["REQ", "event-kinds", {"kinds": [24567], "#d": ["event_kinds"]}]
|
||||
["REQ", "time-stats", {"kinds": [24567], "#d": ["time_stats"]}]
|
||||
["REQ", "top-pubkeys", {"kinds": [24567], "#d": ["top_pubkeys"]}]
|
||||
["REQ", "cpu-metrics", {"kinds": [24567], "#d": ["cpu_metrics"]}]
|
||||
```
|
||||
|
||||
### Event Structure
|
||||
|
||||
```json
|
||||
{
|
||||
"kind": 24567,
|
||||
"pubkey": "relay_pubkey",
|
||||
"created_at": 1234567890,
|
||||
"content": "{\"data_type\":\"event_kinds\",\"timestamp\":1234567890,...}",
|
||||
"tags": [
|
||||
["d", "event_kinds"]
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Monitoring Types
|
||||
|
||||
#### Event Distribution (`event_kinds`)
|
||||
|
||||
```json
|
||||
{
|
||||
"data_type": "event_kinds",
|
||||
"timestamp": 1234567890,
|
||||
"total_events": 15432,
|
||||
"kinds": [
|
||||
{"kind": 1, "count": 12000, "percentage": 77.8},
|
||||
{"kind": 0, "count": 2500, "percentage": 16.2}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### Time Statistics (`time_stats`)
|
||||
|
||||
```json
|
||||
{
|
||||
"data_type": "time_stats",
|
||||
"timestamp": 1234567890,
|
||||
"total_events": 15432,
|
||||
"last_24h": 234,
|
||||
"last_7d": 1456,
|
||||
"last_30d": 5432
|
||||
}
|
||||
```
|
||||
|
||||
#### Top Publishers (`top_pubkeys`)
|
||||
|
||||
```json
|
||||
{
|
||||
"data_type": "top_pubkeys",
|
||||
"timestamp": 1234567890,
|
||||
"top_pubkeys": [
|
||||
{"pubkey": "abc123...", "count": 1234, "percentage": 8.0},
|
||||
{"pubkey": "def456...", "count": 987, "percentage": 6.4}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### CPU Metrics (`cpu_metrics`)
|
||||
|
||||
```json
|
||||
{
|
||||
"data_type": "cpu_metrics",
|
||||
"timestamp": 1234567890,
|
||||
"cpu_percent": 12.5,
|
||||
"memory_mb": 45.2,
|
||||
"uptime_seconds": 86400
|
||||
}
|
||||
```
|
||||
|
||||
#### Active Subscriptions (`active_subscriptions`) - Admin Only
|
||||
|
||||
```json
|
||||
{
|
||||
"data_type": "active_subscriptions",
|
||||
"timestamp": 1234567890,
|
||||
"total_subscriptions": 156,
|
||||
"subscriptions_by_client": [
|
||||
{"client_id": "client1", "count": 12},
|
||||
{"client_id": "client2", "count": 8}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
Control monitoring frequency:
|
||||
|
||||
```json
|
||||
["config_update", [{
|
||||
"key": "kind_24567_reporting_throttle_sec",
|
||||
"value": "10",
|
||||
"data_type": "integer",
|
||||
"category": "monitoring"
|
||||
}]]
|
||||
```
|
||||
|
||||
### Performance
|
||||
|
||||
- Events are ephemeral (not stored)
|
||||
- Automatic activation/deactivation based on subscriptions
|
||||
- Throttling prevents excessive event generation
|
||||
- Minimal overhead when no clients monitoring
|
||||
|
||||
## Direct Message Admin
|
||||
|
||||
Control your relay by sending direct messages from any Nostr client.
|
||||
|
||||
### Setup
|
||||
|
||||
1. The relay has its own keypair (shown on startup)
|
||||
2. The relay knows the admin public key
|
||||
3. Send NIP-17 direct messages to the relay
|
||||
|
||||
### Available Commands
|
||||
|
||||
Send a DM containing any of these keywords:
|
||||
|
||||
| Command | Aliases | Response |
|
||||
|---------|---------|----------|
|
||||
| Statistics | stats, statistics | Database statistics |
|
||||
| Configuration | config, configuration | Current configuration |
|
||||
|
||||
### Example
|
||||
|
||||
Using any Nostr client that supports NIP-17:
|
||||
|
||||
1. Find the relay's public key (shown on startup)
|
||||
2. Send a DM: "stats"
|
||||
3. Receive a DM with current relay statistics
|
||||
|
||||
### Response Format
|
||||
|
||||
The relay responds with a NIP-17 DM containing:
|
||||
|
||||
**Stats Response**:
|
||||
```
|
||||
Relay Statistics
|
||||
================
|
||||
Total Events: 15,432
|
||||
Database Size: 10.5 MB
|
||||
Active Connections: 42
|
||||
Active Subscriptions: 156
|
||||
Uptime: 1 day, 2 hours
|
||||
```
|
||||
|
||||
**Config Response**:
|
||||
```
|
||||
Relay Configuration
|
||||
===================
|
||||
Name: My Awesome Relay
|
||||
Description: Community relay
|
||||
Max Subscriptions: 25
|
||||
Auth Enabled: false
|
||||
PoW Difficulty: 0
|
||||
```
|
||||
|
||||
## Response Formats
|
||||
|
||||
### Success Response
|
||||
|
||||
```json
|
||||
{
|
||||
"query_type": "command_name",
|
||||
"status": "success",
|
||||
"message": "Operation completed successfully",
|
||||
"timestamp": 1234567890,
|
||||
"data": [...]
|
||||
}
|
||||
```
|
||||
|
||||
### Error Response
|
||||
|
||||
```json
|
||||
{
|
||||
"query_type": "command_name",
|
||||
"status": "error",
|
||||
"error": "Error description",
|
||||
"timestamp": 1234567890
|
||||
}
|
||||
```
|
||||
|
||||
### Query Response
|
||||
|
||||
```json
|
||||
{
|
||||
"query_type": "query_name",
|
||||
"total_results": 10,
|
||||
"timestamp": 1234567890,
|
||||
"data": [...]
|
||||
}
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Common Errors
|
||||
|
||||
| Error | Cause | Solution |
|
||||
|-------|-------|----------|
|
||||
| `invalid_signature` | Event signature invalid | Check admin private key |
|
||||
| `unauthorized` | Wrong admin pubkey | Use correct admin key |
|
||||
| `invalid_command` | Unknown command | Check command format |
|
||||
| `validation_failed` | Invalid parameter value | Check parameter ranges |
|
||||
| `database_error` | Database operation failed | Check database integrity |
|
||||
| `timeout` | Query took too long | Simplify query or increase timeout |
|
||||
|
||||
### Error Response Example
|
||||
|
||||
```json
|
||||
{
|
||||
"query_type": "config_update",
|
||||
"status": "error",
|
||||
"error": "field validation failed: invalid port number '99999' (must be 1-65535)",
|
||||
"timestamp": 1234567890
|
||||
}
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
### JavaScript/TypeScript Example
|
||||
|
||||
```javascript
|
||||
import { SimplePool, nip44, getPublicKey, finalizeEvent } from 'nostr-tools';
|
||||
|
||||
const adminPrivkey = 'your_admin_privkey_hex';
|
||||
const adminPubkey = getPublicKey(adminPrivkey);
|
||||
const relayPubkey = 'relay_pubkey_hex';
|
||||
const relayUrl = 'ws://localhost:8888';
|
||||
|
||||
// Create admin command
|
||||
async function sendAdminCommand(command) {
|
||||
const pool = new SimplePool();
|
||||
|
||||
// Encrypt command with NIP-44
|
||||
const encryptedContent = await nip44.encrypt(
|
||||
adminPrivkey,
|
||||
relayPubkey,
|
||||
JSON.stringify(command)
|
||||
);
|
||||
|
||||
// Create event
|
||||
const event = finalizeEvent({
|
||||
kind: 23456,
|
||||
created_at: Math.floor(Date.now() / 1000),
|
||||
tags: [['p', relayPubkey]],
|
||||
content: encryptedContent
|
||||
}, adminPrivkey);
|
||||
|
||||
// Publish event
|
||||
await pool.publish([relayUrl], event);
|
||||
|
||||
// Subscribe to response
|
||||
const sub = pool.sub([relayUrl], [{
|
||||
kinds: [23457],
|
||||
'#p': [adminPubkey],
|
||||
since: Math.floor(Date.now() / 1000)
|
||||
}]);
|
||||
|
||||
return new Promise((resolve) => {
|
||||
sub.on('event', async (event) => {
|
||||
// Decrypt response
|
||||
const decrypted = await nip44.decrypt(
|
||||
adminPrivkey,
|
||||
relayPubkey,
|
||||
event.content
|
||||
);
|
||||
resolve(JSON.parse(decrypted));
|
||||
sub.unsub();
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// Query configuration
|
||||
const config = await sendAdminCommand(['config_query', 'all']);
|
||||
console.log(config);
|
||||
|
||||
// Update configuration
|
||||
const result = await sendAdminCommand(['config_update', [
|
||||
{
|
||||
key: 'relay_name',
|
||||
value: 'My Relay',
|
||||
data_type: 'string',
|
||||
category: 'relay'
|
||||
}
|
||||
]]);
|
||||
console.log(result);
|
||||
|
||||
// Add blacklist rule
|
||||
const blacklist = await sendAdminCommand([
|
||||
'blacklist',
|
||||
'pubkey',
|
||||
'abc123...'
|
||||
]);
|
||||
console.log(blacklist);
|
||||
|
||||
// Execute SQL query
|
||||
const query = await sendAdminCommand([
|
||||
'sql_query',
|
||||
'SELECT * FROM events ORDER BY created_at DESC LIMIT 10'
|
||||
]);
|
||||
console.log(query);
|
||||
```
|
||||
|
||||
### Python Example
|
||||
|
||||
```python
|
||||
from nostr_sdk import Keys, Client, EventBuilder, Filter, nip44
|
||||
|
||||
admin_privkey = "your_admin_privkey_hex"
|
||||
relay_pubkey = "relay_pubkey_hex"
|
||||
relay_url = "ws://localhost:8888"
|
||||
|
||||
# Initialize
|
||||
keys = Keys.parse(admin_privkey)
|
||||
client = Client(keys)
|
||||
client.add_relay(relay_url)
|
||||
client.connect()
|
||||
|
||||
# Send admin command
|
||||
async def send_admin_command(command):
|
||||
# Encrypt command
|
||||
encrypted = nip44.encrypt(
|
||||
keys.secret_key(),
|
||||
relay_pubkey,
|
||||
json.dumps(command)
|
||||
)
|
||||
|
||||
# Create event
|
||||
event = EventBuilder.new(
|
||||
kind=23456,
|
||||
content=encrypted,
|
||||
tags=[["p", relay_pubkey]]
|
||||
).to_event(keys)
|
||||
|
||||
# Publish
|
||||
await client.send_event(event)
|
||||
|
||||
# Wait for response
|
||||
filter = Filter().kind(23457).pubkey(relay_pubkey).since(int(time.time()))
|
||||
events = await client.get_events_of([filter], timeout=5)
|
||||
|
||||
if events:
|
||||
# Decrypt response
|
||||
decrypted = nip44.decrypt(
|
||||
keys.secret_key(),
|
||||
relay_pubkey,
|
||||
events[0].content()
|
||||
)
|
||||
return json.loads(decrypted)
|
||||
|
||||
# Query configuration
|
||||
config = await send_admin_command(["config_query", "all"])
|
||||
print(config)
|
||||
```
|
||||
|
||||
### Bash/curl Example
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
# Note: This is a simplified example. Real implementation requires:
|
||||
# - NIP-44 encryption
|
||||
# - Event signing
|
||||
# - WebSocket connection
|
||||
|
||||
RELAY_URL="ws://localhost:8888"
|
||||
ADMIN_PRIVKEY="your_admin_privkey"
|
||||
RELAY_PUBKEY="relay_pubkey"
|
||||
|
||||
# Use nostrtool or similar for proper event creation
|
||||
nostrtool event \
|
||||
--kind 23456 \
|
||||
--content "$(echo '["config_query","all"]' | nip44-encrypt)" \
|
||||
--tag p "$RELAY_PUBKEY" \
|
||||
--private-key "$ADMIN_PRIVKEY" \
|
||||
| nostrtool send "$RELAY_URL"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- **[Configuration Guide](docs/configuration_guide.md)** - Detailed configuration options
|
||||
- **[Deployment Guide](docs/deployment_guide.md)** - Production deployment
|
||||
- **[NIP-42 Authentication](docs/NIP-42_Authentication.md)** - Authentication setup
|
||||
- **[User Guide](docs/user_guide.md)** - End-user documentation
|
||||
|
||||
## Support
|
||||
|
||||
For API questions or issues:
|
||||
- Open an issue on GitHub
|
||||
- Check existing documentation
|
||||
- Join the Nostr community
|
||||
|
||||
---
|
||||
|
||||
**API Version**: 0.6.0
|
||||
**Last Updated**: 2026-01-23
|
||||
+42
-33
@@ -1,12 +1,14 @@
|
||||
# Alpine-based MUSL static binary builder for C-Relay
|
||||
# Alpine-based MUSL static binary builder for C-Relay-PG
|
||||
# Produces truly portable binaries with zero runtime dependencies
|
||||
|
||||
ARG DEBUG_BUILD=false
|
||||
ARG DB_BACKEND=sqlite
|
||||
|
||||
FROM alpine:3.19 AS builder
|
||||
|
||||
# Re-declare build argument in this stage
|
||||
# Re-declare build arguments in this stage
|
||||
ARG DEBUG_BUILD=false
|
||||
ARG DB_BACKEND=sqlite
|
||||
|
||||
# Install build dependencies
|
||||
RUN apk add --no-cache \
|
||||
@@ -26,9 +28,11 @@ RUN apk add --no-cache \
|
||||
curl-static \
|
||||
sqlite-dev \
|
||||
sqlite-static \
|
||||
postgresql-dev \
|
||||
linux-headers \
|
||||
wget \
|
||||
bash
|
||||
bash \
|
||||
openssh-client
|
||||
|
||||
# Set working directory
|
||||
WORKDIR /build
|
||||
@@ -68,15 +72,8 @@ RUN cd /tmp && \
|
||||
make install && \
|
||||
rm -rf /tmp/libwebsockets
|
||||
|
||||
# Copy only submodule configuration and git directory
|
||||
COPY .gitmodules /build/.gitmodules
|
||||
COPY .git /build/.git
|
||||
|
||||
# Clean up any stale submodule references (nips directory is not a submodule)
|
||||
RUN git rm --cached nips 2>/dev/null || true
|
||||
|
||||
# Initialize submodules (cached unless .gitmodules changes)
|
||||
RUN git submodule update --init --recursive
|
||||
# Submodules are provided in the local build context and copied explicitly below.
|
||||
# Avoid network/SSH dependency inside Docker image build.
|
||||
|
||||
# Copy nostr_core_lib source files (cached unless nostr_core_lib changes)
|
||||
COPY nostr_core_lib /build/nostr_core_lib/
|
||||
@@ -84,11 +81,13 @@ COPY nostr_core_lib /build/nostr_core_lib/
|
||||
# Copy c_utils_lib source files (cached unless c_utils_lib changes)
|
||||
COPY c_utils_lib /build/c_utils_lib/
|
||||
|
||||
# Build c_utils_lib with MUSL-compatible flags (cached unless c_utils_lib changes)
|
||||
# Build c_utils_lib static library for relay logging utilities
|
||||
# (build only debug.c to avoid broken/missing version header surface in submodule revision)
|
||||
RUN cd c_utils_lib && \
|
||||
sed -i 's/CFLAGS = -Wall -Wextra -std=c99 -O2 -g/CFLAGS = -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -Wall -Wextra -std=c99 -O2 -g/' Makefile && \
|
||||
make clean && \
|
||||
make
|
||||
mkdir -p build && \
|
||||
gcc -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -Wall -Wextra -std=c99 -O2 -g \
|
||||
-Isrc -Iinclude -c src/debug.c -o build/debug.o && \
|
||||
ar rcs libc_utils.a build/debug.o
|
||||
|
||||
# Build nostr_core_lib with required NIPs (cached unless nostr_core_lib changes)
|
||||
# Disable fortification in build.sh to prevent __*_chk symbol issues
|
||||
@@ -99,44 +98,54 @@ RUN cd nostr_core_lib && \
|
||||
rm -f *.o *.a 2>/dev/null || true && \
|
||||
./build.sh --nips=1,6,13,17,19,44,59
|
||||
|
||||
# Copy c-relay source files LAST (only this layer rebuilds on source changes)
|
||||
# Copy c-relay-pg source files LAST (only this layer rebuilds on source changes)
|
||||
COPY src/ /build/src/
|
||||
COPY Makefile /build/Makefile
|
||||
|
||||
# Build c-relay with full static linking (only rebuilds when src/ changes)
|
||||
# Build c-relay-pg with full static linking (only rebuilds when src/ changes)
|
||||
# Disable fortification to avoid __*_chk symbols that don't exist in MUSL
|
||||
# Use conditional compilation flags based on DEBUG_BUILD argument
|
||||
# Use conditional compilation flags based on DEBUG_BUILD and DB_BACKEND build args
|
||||
RUN if [ "$DEBUG_BUILD" = "true" ]; then \
|
||||
CFLAGS="-g -O0 -DDEBUG"; \
|
||||
STRIP_CMD=""; \
|
||||
echo "Building with DEBUG symbols enabled"; \
|
||||
CFLAGS="-g -O2 -DDEBUG"; \
|
||||
STRIP_CMD="echo 'Keeping debug symbols'"; \
|
||||
echo "Building with DEBUG symbols enabled (optimized with -O2)"; \
|
||||
else \
|
||||
CFLAGS="-O2"; \
|
||||
STRIP_CMD="strip /build/c_relay_static"; \
|
||||
echo "Building optimized production binary"; \
|
||||
STRIP_CMD="strip /build/c_relay_pg_static"; \
|
||||
echo "Building optimized production binary (symbols stripped)"; \
|
||||
fi && \
|
||||
gcc -static $CFLAGS -Wall -Wextra -std=c99 \
|
||||
if [ "$DB_BACKEND" = "postgres" ]; then \
|
||||
DB_FLAGS="-DDB_BACKEND_POSTGRES -DHAVE_LIBPQ"; \
|
||||
DB_LIBS="-lpq -lpgcommon -lpgport"; \
|
||||
echo "Compiling with PostgreSQL backend"; \
|
||||
else \
|
||||
DB_FLAGS=""; \
|
||||
DB_LIBS=""; \
|
||||
echo "Compiling with SQLite backend"; \
|
||||
fi && \
|
||||
gcc -static $CFLAGS $DB_FLAGS -Wall -Wextra -std=c99 \
|
||||
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 \
|
||||
-I. -Ic_utils_lib/src -Inostr_core_lib -Inostr_core_lib/nostr_core \
|
||||
-Inostr_core_lib/cjson -Inostr_core_lib/nostr_websocket \
|
||||
-Inostr_core_lib/cjson -Inostr_core_lib/nostr_websocket -I/usr/include/postgresql \
|
||||
src/main.c src/config.c src/dm_admin.c src/request_validator.c \
|
||||
src/nip009.c src/nip011.c src/nip013.c src/nip040.c src/nip042.c \
|
||||
src/websockets.c src/subscriptions.c src/api.c src/embedded_web_content.c \
|
||||
-o /build/c_relay_static \
|
||||
src/websockets.c src/subscriptions.c src/api.c src/embedded_web_content.c src/ip_ban.c \
|
||||
src/db_ops.c src/db_ops_sqlite.c src/db_ops_postgres.c src/thread_pool.c \
|
||||
-o /build/c_relay_pg_static \
|
||||
c_utils_lib/libc_utils.a \
|
||||
nostr_core_lib/libnostr_core_x64.a \
|
||||
-lwebsockets -lssl -lcrypto -lsqlite3 -lsecp256k1 \
|
||||
-lcurl -lz -lpthread -lm -ldl && \
|
||||
-lcurl -lz -lpthread -lm -ldl $DB_LIBS && \
|
||||
eval "$STRIP_CMD"
|
||||
|
||||
# Verify it's truly static
|
||||
RUN echo "=== Binary Information ===" && \
|
||||
file /build/c_relay_static && \
|
||||
ls -lh /build/c_relay_static && \
|
||||
file /build/c_relay_pg_static && \
|
||||
ls -lh /build/c_relay_pg_static && \
|
||||
echo "=== Checking for dynamic dependencies ===" && \
|
||||
(ldd /build/c_relay_static 2>&1 || echo "Binary is static") && \
|
||||
(ldd /build/c_relay_pg_static 2>&1 || echo "Binary is static") && \
|
||||
echo "=== Build complete ==="
|
||||
|
||||
# Output stage - just the binary
|
||||
FROM scratch AS output
|
||||
COPY --from=builder /build/c_relay_static /c_relay_static
|
||||
COPY --from=builder /build/c_relay_pg_static /c_relay_pg_static
|
||||
@@ -1,28 +1,40 @@
|
||||
# C-Relay Makefile
|
||||
# C-Relay-PG Makefile
|
||||
|
||||
CC = gcc
|
||||
CFLAGS = -Wall -Wextra -std=c99 -g -O2
|
||||
INCLUDES = -I. -Ic_utils_lib/src -Inostr_core_lib -Inostr_core_lib/nostr_core -Inostr_core_lib/cjson -Inostr_core_lib/nostr_websocket
|
||||
LIBS = -lsqlite3 -lwebsockets -lz -ldl -lpthread -lm -L/usr/local/lib -lsecp256k1 -lssl -lcrypto -L/usr/local/lib -lcurl -Lc_utils_lib -lc_utils
|
||||
|
||||
DB_BACKEND ?= sqlite
|
||||
|
||||
# Build directory
|
||||
BUILD_DIR = build
|
||||
|
||||
# Source files
|
||||
MAIN_SRC = src/main.c src/config.c src/dm_admin.c src/request_validator.c src/nip009.c src/nip011.c src/nip013.c src/nip040.c src/nip042.c src/websockets.c src/subscriptions.c src/api.c src/embedded_web_content.c
|
||||
MAIN_SRC = src/main.c src/config.c src/dm_admin.c src/request_validator.c src/nip009.c src/nip011.c src/nip013.c src/nip040.c src/nip042.c src/websockets.c src/subscriptions.c src/api.c src/embedded_web_content.c src/ip_ban.c src/thread_pool.c
|
||||
DB_OPS_SRC = src/db_ops.c
|
||||
|
||||
ifeq ($(DB_BACKEND),postgres)
|
||||
CFLAGS += -DDB_BACKEND_POSTGRES -DHAVE_LIBPQ
|
||||
DB_OPS_SRC += src/db_ops_postgres.c
|
||||
LIBS += -lpq
|
||||
else
|
||||
DB_OPS_SRC += src/db_ops_sqlite.c
|
||||
endif
|
||||
|
||||
NOSTR_CORE_LIB = nostr_core_lib/libnostr_core_x64.a
|
||||
C_UTILS_LIB = c_utils_lib/libc_utils.a
|
||||
|
||||
# Architecture detection
|
||||
ARCH = $(shell uname -m)
|
||||
ifeq ($(ARCH),x86_64)
|
||||
TARGET = $(BUILD_DIR)/c_relay_x86
|
||||
TARGET = $(BUILD_DIR)/c_relay_pg_x86
|
||||
else ifeq ($(ARCH),aarch64)
|
||||
TARGET = $(BUILD_DIR)/c_relay_arm64
|
||||
TARGET = $(BUILD_DIR)/c_relay_pg_arm64
|
||||
else ifeq ($(ARCH),arm64)
|
||||
TARGET = $(BUILD_DIR)/c_relay_arm64
|
||||
TARGET = $(BUILD_DIR)/c_relay_pg_arm64
|
||||
else
|
||||
TARGET = $(BUILD_DIR)/c_relay_$(ARCH)
|
||||
TARGET = $(BUILD_DIR)/c_relay_pg_$(ARCH)
|
||||
endif
|
||||
|
||||
# Default target
|
||||
@@ -81,19 +93,19 @@ force-version:
|
||||
@$(MAKE) src/main.h
|
||||
|
||||
# Build the relay
|
||||
$(TARGET): $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
|
||||
@echo "Compiling C-Relay for architecture: $(ARCH)"
|
||||
$(CC) $(CFLAGS) $(INCLUDES) $(MAIN_SRC) -o $(TARGET) $(NOSTR_CORE_LIB) $(C_UTILS_LIB) $(LIBS)
|
||||
$(TARGET): $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(DB_OPS_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
|
||||
@echo "Compiling C-Relay-PG for architecture: $(ARCH) (backend: $(DB_BACKEND))"
|
||||
$(CC) $(CFLAGS) $(INCLUDES) $(MAIN_SRC) $(DB_OPS_SRC) -o $(TARGET) $(NOSTR_CORE_LIB) $(C_UTILS_LIB) $(LIBS)
|
||||
@echo "Build complete: $(TARGET)"
|
||||
|
||||
# Build for specific architectures
|
||||
x86: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
|
||||
@echo "Building C-Relay for x86_64..."
|
||||
$(CC) $(CFLAGS) $(INCLUDES) $(MAIN_SRC) -o $(BUILD_DIR)/c_relay_x86 $(NOSTR_CORE_LIB) $(C_UTILS_LIB) $(LIBS)
|
||||
@echo "Build complete: $(BUILD_DIR)/c_relay_x86"
|
||||
x86: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(DB_OPS_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
|
||||
@echo "Building C-Relay-PG for x86_64 (backend: $(DB_BACKEND))..."
|
||||
$(CC) $(CFLAGS) $(INCLUDES) $(MAIN_SRC) $(DB_OPS_SRC) -o $(BUILD_DIR)/c_relay_pg_x86 $(NOSTR_CORE_LIB) $(C_UTILS_LIB) $(LIBS)
|
||||
@echo "Build complete: $(BUILD_DIR)/c_relay_pg_x86"
|
||||
|
||||
arm64: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
|
||||
@echo "Cross-compiling C-Relay for ARM64..."
|
||||
arm64: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(DB_OPS_SRC) $(NOSTR_CORE_LIB) $(C_UTILS_LIB)
|
||||
@echo "Cross-compiling C-Relay-PG for ARM64 (backend: $(DB_BACKEND))..."
|
||||
@if ! command -v aarch64-linux-gnu-gcc >/dev/null 2>&1; then \
|
||||
echo "ERROR: ARM64 cross-compiler not found."; \
|
||||
echo "Install with: make install-cross-tools"; \
|
||||
@@ -116,9 +128,9 @@ arm64: $(BUILD_DIR) src/main.h src/sql_schema.h $(MAIN_SRC) $(NOSTR_CORE_LIB) $(
|
||||
fi
|
||||
@echo "Using aarch64-linux-gnu-gcc with ARM64 libraries..."
|
||||
PKG_CONFIG_PATH=/usr/lib/aarch64-linux-gnu/pkgconfig:/usr/share/pkgconfig \
|
||||
aarch64-linux-gnu-gcc $(CFLAGS) $(INCLUDES) $(MAIN_SRC) -o $(BUILD_DIR)/c_relay_arm64 $(NOSTR_CORE_LIB) $(C_UTILS_LIB) \
|
||||
aarch64-linux-gnu-gcc $(CFLAGS) $(INCLUDES) $(MAIN_SRC) $(DB_OPS_SRC) -o $(BUILD_DIR)/c_relay_pg_arm64 $(NOSTR_CORE_LIB) $(C_UTILS_LIB) \
|
||||
-L/usr/lib/aarch64-linux-gnu $(LIBS)
|
||||
@echo "Build complete: $(BUILD_DIR)/c_relay_arm64"
|
||||
@echo "Build complete: $(BUILD_DIR)/c_relay_pg_arm64"
|
||||
|
||||
# Install ARM64 cross-compilation dependencies
|
||||
install-arm64-deps:
|
||||
@@ -160,7 +172,7 @@ test: $(TARGET)
|
||||
init-db:
|
||||
@echo "Database initialization is now handled automatically when the server starts."
|
||||
@echo "The schema is embedded in the binary - no external files needed."
|
||||
@echo "To manually recreate database: rm -f db/c_nostr_relay.db && ./build/c_relay_x86"
|
||||
@echo "To manually recreate database: rm -f db/c_nostr_relay.db && ./build/c_relay_pg_x86"
|
||||
|
||||
# Clean build artifacts
|
||||
clean:
|
||||
@@ -180,7 +192,7 @@ install-deps:
|
||||
|
||||
# Help
|
||||
help:
|
||||
@echo "C-Relay Build System"
|
||||
@echo "C-Relay-PG Build System"
|
||||
@echo ""
|
||||
@echo "Targets:"
|
||||
@echo " all Build the relay for current architecture (default)"
|
||||
@@ -209,15 +221,15 @@ help:
|
||||
# Build fully static MUSL binaries using Docker
|
||||
static-musl-x86_64:
|
||||
@echo "Building fully static MUSL binary for x86_64..."
|
||||
docker buildx build --platform linux/amd64 -f examples/deployment/static-builder.Dockerfile -t c-relay-static-builder-x86_64 --load .
|
||||
docker run --rm -v $(PWD)/build:/output c-relay-static-builder-x86_64 sh -c "cp /c_relay_static_musl_x86_64 /output/"
|
||||
@echo "Static binary created: build/c_relay_static_musl_x86_64"
|
||||
docker buildx build --platform linux/amd64 -f examples/deployment/static-builder.Dockerfile -t c-relay-pg-static-builder-x86_64 --load .
|
||||
docker run --rm -v $(PWD)/build:/output c-relay-pg-static-builder-x86_64 sh -c "cp /c_relay_pg_static_musl_x86_64 /output/"
|
||||
@echo "Static binary created: build/c_relay_pg_static_musl_x86_64"
|
||||
|
||||
static-musl-arm64:
|
||||
@echo "Building fully static MUSL binary for ARM64..."
|
||||
docker buildx build --platform linux/arm64 -f examples/deployment/static-builder.Dockerfile -t c-relay-static-builder-arm64 --load .
|
||||
docker run --rm -v $(PWD)/build:/output c-relay-static-builder-arm64 sh -c "cp /c_relay_static_musl_x86_64 /output/c_relay_static_musl_arm64"
|
||||
@echo "Static binary created: build/c_relay_static_musl_arm64"
|
||||
docker buildx build --platform linux/arm64 -f examples/deployment/static-builder.Dockerfile -t c-relay-pg-static-builder-arm64 --load .
|
||||
docker run --rm -v $(PWD)/build:/output c-relay-pg-static-builder-arm64 sh -c "cp /c_relay_pg_static_musl_x86_64 /output/c_relay_pg_static_musl_arm64"
|
||||
@echo "Static binary created: build/c_relay_pg_static_musl_arm64"
|
||||
|
||||
static-musl: static-musl-x86_64 static-musl-arm64
|
||||
@echo "Built static MUSL binaries for both architectures"
|
||||
|
||||
+37
-14
@@ -10,18 +10,18 @@ This relay is written in C99 with a sqlite database.
|
||||
|
||||
It implements the following NIPs.
|
||||
|
||||
- [X] NIP-01: Basic protocol flow implementation
|
||||
- [X] NIP-09: Event deletion
|
||||
- [X] NIP-11: Relay information document
|
||||
- [X] NIP-13: Proof of Work
|
||||
- [X] NIP-15: End of Stored Events Notice
|
||||
- [X] NIP-20: Command Results
|
||||
- [X] NIP-33: Parameterized Replaceable Events
|
||||
- [X] NIP-40: Expiration Timestamp
|
||||
- [X] NIP-42: Authentication of clients to relays
|
||||
- [X] NIP-45: Counting results
|
||||
- [X] NIP-50: Keywords filter
|
||||
- [X] NIP-70: Protected Events
|
||||
- [x] NIP-01: Basic protocol flow implementation
|
||||
- [x] NIP-09: Event deletion
|
||||
- [x] NIP-11: Relay information document
|
||||
- [x] NIP-13: Proof of Work
|
||||
- [x] NIP-15: End of Stored Events Notice
|
||||
- [x] NIP-20: Command Results
|
||||
- [x] NIP-33: Parameterized Replaceable Events
|
||||
- [x] NIP-40: Expiration Timestamp
|
||||
- [x] NIP-42: Authentication of clients to relays
|
||||
- [x] NIP-45: Counting results
|
||||
- [x] NIP-50: Keywords filter
|
||||
- [x] NIP-70: Protected Events
|
||||
|
||||
## The unconventional
|
||||
|
||||
@@ -33,10 +33,33 @@ I'm not bothering with other operating systems.
|
||||
|
||||
### The relay is a full nostr citizen with it's own public and private keys.
|
||||
|
||||
For example, you can see my relay (wss://relay.laantungir.net) running here:
|
||||
For example, you can see my relay's profile (wss://relay.laantungir.net) running here:
|
||||
|
||||
[https://primal.net/p/nprofile1qqswn2jsmm8lq8evas0v9vhqkdpn9nuujt90mtz60nqgsxndy66es4qjjnhr7](https://)
|
||||
[Primal link](https://primal.net/p/nprofile1qqswn2jsmm8lq8evas0v9vhqkdpn9nuujt90mtz60nqgsxndy66es4qjjnhr7)
|
||||
|
||||
What this means in practice is that when you start the relay, it generates keys for itself, and for it's administrator (You can specify these if you wish)
|
||||
|
||||
Now the program and the administrator can have verifed communication between the two, simply by using nostr events. For example, the administrator can send DMs to the relay, asking it's status, and changing it's configuration through any client that can handle nip17 DMs. The relay can also send notifications to the administrator about it's current status, or it can publish it's status on a regular schedule directly to NOSTR as kind-1 notes.
|
||||
|
||||
Also included is a more standard administrative web front end. This front end communicates to the relay using an extensive api, which again is nostr events signed by the administrator. You can completely control the relay through signed nostr events.
|
||||
|
||||
## Screenshots
|
||||
|
||||

|
||||
Main page with real time updates.
|
||||
|
||||

|
||||
Set your configuration preferences.
|
||||
|
||||

|
||||
View current subscriptions
|
||||
|
||||

|
||||
Add npubs to white or black lists.
|
||||
|
||||

|
||||
Run sql queries on the database.
|
||||
|
||||

|
||||
Light mode.
|
||||
|
||||
|
||||
@@ -1,474 +1,285 @@
|
||||
# C-Nostr Relay
|
||||
# C-Relay-PG: High-Performance Nostr Relay
|
||||
|
||||
A high-performance Nostr relay implemented in C with SQLite backend, featuring nostr event-based management.
|
||||
A blazingly fast, production-ready Nostr relay implemented in C with an innovative event-based configuration system. Built for performance, security, and ease of deployment.
|
||||
|
||||
## Supported NIPs
|
||||
## 🚀 Why C-Relay-PG?
|
||||
|
||||
<!--
|
||||
NOTE FOR ASSISTANTS: When updating the NIPs checklist below, ONLY change [ ] to [x] to mark as complete.
|
||||
Do NOT modify the formatting, add emojis, or change the text. Keep the simple format consistent.
|
||||
-->
|
||||
### Event-Based Configuration
|
||||
Unlike traditional relays that require config files, C-Relay-PG uses **cryptographically signed Nostr events** for all configuration. This means:
|
||||
- **Zero config files** - Everything stored in the database
|
||||
- **Real-time updates** - Changes applied instantly without restart
|
||||
- **Cryptographic security** - All changes must be signed by admin
|
||||
- **Complete audit trail** - Every configuration change is timestamped and signed
|
||||
- **Version control built-in** - Configuration history is part of the event stream
|
||||
|
||||
- [x] NIP-01: Basic protocol flow implementation
|
||||
- [x] NIP-09: Event deletion
|
||||
- [x] NIP-11: Relay information document
|
||||
- [x] NIP-13: Proof of Work
|
||||
- [x] NIP-15: End of Stored Events Notice
|
||||
- [x] NIP-20: Command Results
|
||||
- [x] NIP-33: Parameterized Replaceable Events
|
||||
- [x] NIP-40: Expiration Timestamp
|
||||
- [x] NIP-42: Authentication of clients to relays
|
||||
- [x] NIP-45: Counting results
|
||||
- [x] NIP-50: Keywords filter
|
||||
- [x] NIP-70: Protected Events
|
||||
### Built-in Web Admin Interface
|
||||
Access a full-featured web dashboard at `http://localhost:8888/api/` with:
|
||||
- Real-time configuration management
|
||||
- Database statistics and analytics
|
||||
- Auth rules management (whitelist/blacklist)
|
||||
- NIP-42 authentication for secure access
|
||||
- No external dependencies - all files embedded in the binary
|
||||
|
||||
## Quick Start
|
||||
### Direct Message Admin System
|
||||
Control your relay by sending direct messages from any Nostr client:
|
||||
- Send "stats" to get relay statistics
|
||||
- Send "config" to view current configuration
|
||||
- Full Nostr citizen with its own keypair
|
||||
- Works with any NIP-17 compatible client
|
||||
|
||||
Get your C-Relay up and running in minutes with a static binary (no dependencies required):
|
||||
### Performance & Efficiency
|
||||
- **Written in C** - Maximum performance and minimal resource usage
|
||||
- **SQLite backend** - Reliable, fast, and self-contained
|
||||
- **Static binary available** - Single file deployment with zero dependencies
|
||||
- **Efficient memory management** - Optimized for long-running operation
|
||||
- **WebSocket native** - Built on libwebsockets for optimal protocol support
|
||||
|
||||
### 1. Download Static Binary
|
||||
## 📋 Supported NIPs
|
||||
|
||||
Download the latest static release from the [releases page](https://git.laantungir.net/laantungir/c-relay/releases):
|
||||
C-Relay-PG implements a comprehensive set of Nostr Improvement Proposals:
|
||||
|
||||
- ✅ **NIP-01**: Basic protocol flow implementation
|
||||
- ✅ **NIP-09**: Event deletion
|
||||
- ✅ **NIP-11**: Relay information document
|
||||
- ✅ **NIP-13**: Proof of Work
|
||||
- ✅ **NIP-15**: End of Stored Events Notice
|
||||
- ✅ **NIP-20**: Command Results
|
||||
- ✅ **NIP-33**: Parameterized Replaceable Events
|
||||
- ✅ **NIP-40**: Expiration Timestamp
|
||||
- ✅ **NIP-42**: Authentication of clients to relays
|
||||
- ✅ **NIP-45**: Counting results
|
||||
- ✅ **NIP-50**: Keywords filter
|
||||
- ✅ **NIP-70**: Protected Events
|
||||
|
||||
## 🎯 Key Features
|
||||
|
||||
### Security
|
||||
- **NIP-42 Authentication** - Cryptographic client authentication
|
||||
- **Proof of Work** - Configurable PoW requirements (NIP-13)
|
||||
- **Protected Events** - Support for encrypted/protected content (NIP-70)
|
||||
- **Whitelist/Blacklist** - Flexible access control by pubkey
|
||||
- **Admin key security** - Private key shown only once, never stored
|
||||
|
||||
### Flexibility
|
||||
- **Dynamic configuration** - Most settings update without restart
|
||||
- **Subscription management** - Configurable limits per client and globally
|
||||
- **Event expiration** - Automatic cleanup of expired events (NIP-40)
|
||||
- **Parameterized events** - Full support for replaceable events (NIP-33)
|
||||
- **Keyword search** - Built-in full-text search (NIP-50)
|
||||
|
||||
### Monitoring
|
||||
- **Real-time statistics** - Live event distribution and metrics
|
||||
- **Subscription-based monitoring** - Ephemeral events (kind 24567) for dashboards
|
||||
- **SQL query API** - Direct database queries for advanced analytics
|
||||
- **Resource tracking** - CPU, memory, and database size monitoring
|
||||
- **Event broadcast logging** - Complete audit trail of all operations
|
||||
|
||||
### Developer-Friendly
|
||||
- **Comprehensive test suite** - Automated tests for all NIPs
|
||||
- **Clear documentation** - Detailed guides for deployment and configuration
|
||||
- **SystemD integration** - Production-ready service files included
|
||||
- **Docker support** - Container deployment with Alpine Linux
|
||||
- **Cross-platform** - Builds on Linux, macOS, and Windows (WSL)
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Option 1: Static Binary (Recommended)
|
||||
|
||||
Download and run - no dependencies required:
|
||||
|
||||
```bash
|
||||
# Static binary - works on all Linux distributions (no dependencies)
|
||||
wget https://git.laantungir.net/laantungir/c-relay/releases/download/v0.6.0/c-relay-v0.6.0-linux-x86_64-static
|
||||
chmod +x c-relay-v0.6.0-linux-x86_64-static
|
||||
mv c-relay-v0.6.0-linux-x86_64-static c-relay
|
||||
# Download the latest static release
|
||||
wget https://git.laantungir.net/laantungir/c-relay-pg/releases/download/v0.6.0/c-relay-pg-v0.6.0-linux-x86_64-static
|
||||
chmod +x c-relay-pg-v0.6.0-linux-x86_64-static
|
||||
mv c-relay-pg-v0.6.0-linux-x86_64-static c-relay-pg
|
||||
|
||||
# Run the relay
|
||||
./c-relay-pg
|
||||
```
|
||||
|
||||
### 2. Start the Relay
|
||||
**Important**: On first startup, save the **Admin Private Key** displayed in the console. You'll need it for all administrative operations.
|
||||
|
||||
Simply run the binary - no configuration files needed:
|
||||
### Option 2: Build from Source
|
||||
|
||||
```bash
|
||||
./c-relay
|
||||
# Install dependencies (Ubuntu/Debian)
|
||||
sudo apt install -y build-essential git sqlite3 libsqlite3-dev \
|
||||
libwebsockets-dev libssl-dev libsecp256k1-dev libcurl4-openssl-dev zlib1g-dev
|
||||
|
||||
# Clone and build
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
git submodule update --init --recursive
|
||||
./make_and_restart_relay.sh
|
||||
```
|
||||
|
||||
On first startup, you'll see:
|
||||
- **Admin Private Key**: Save this securely! You'll need it for administration
|
||||
- **Relay Public Key**: Your relay's identity on the Nostr network
|
||||
- **Port Information**: Default is 8888, or the next available port
|
||||
The relay will start on port 8888 (or the next available port).
|
||||
|
||||
### 3. Access the Web Interface
|
||||
## 🌐 Access the Web Interface
|
||||
|
||||
Once running, open your browser to:
|
||||
|
||||
Open your browser and navigate to:
|
||||
```
|
||||
http://localhost:8888/api/
|
||||
```
|
||||
|
||||
The web interface provides:
|
||||
- Real-time configuration management
|
||||
- Database statistics dashboard
|
||||
- Auth rules management
|
||||
- Secure admin authentication with your Nostr identity
|
||||
- Configuration management with live updates
|
||||
- Database statistics and event distribution charts
|
||||
- Auth rules management (whitelist/blacklist)
|
||||
- SQL query interface for advanced analytics
|
||||
- Real-time monitoring dashboard
|
||||
|
||||
### 4. Test Your Relay
|
||||
## 📦 Installation Options
|
||||
|
||||
### Production Deployment (SystemD)
|
||||
|
||||
Test basic connectivity:
|
||||
```bash
|
||||
# Test WebSocket connection
|
||||
curl -H "Accept: application/nostr+json" http://localhost:8888
|
||||
# Clone repository
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
git submodule update --init --recursive
|
||||
|
||||
# Test with a Nostr client
|
||||
# Add ws://localhost:8888 to your client's relay list
|
||||
# Build
|
||||
make clean && make
|
||||
|
||||
# Install as system service
|
||||
sudo systemd/install-service.sh
|
||||
|
||||
# Start and enable
|
||||
sudo systemctl start c-relay-pg
|
||||
sudo systemctl enable c-relay-pg
|
||||
|
||||
# Capture admin keys from logs
|
||||
sudo journalctl -u c-relay-pg | grep "Admin Private Key"
|
||||
```
|
||||
|
||||
### 5. Configure Your Relay (Optional)
|
||||
### Docker Deployment
|
||||
|
||||
Use the web interface or send admin commands to customize:
|
||||
- Relay name and description
|
||||
- Authentication rules (whitelist/blacklist)
|
||||
- Connection limits
|
||||
- Proof-of-work requirements
|
||||
```bash
|
||||
# Build Docker image
|
||||
docker build -f Dockerfile.alpine-musl -t c-relay-pg .
|
||||
|
||||
**That's it!** Your relay is now running with zero configuration required. The event-based configuration system means you can adjust all settings through the web interface or admin API without editing config files.
|
||||
|
||||
|
||||
## Web Admin Interface
|
||||
|
||||
C-Relay includes a **built-in web-based administration interface** accessible at `http://localhost:8888/api/`. The interface provides:
|
||||
|
||||
- **Real-time Configuration Management**: View and edit all relay settings through a web UI
|
||||
- **Database Statistics Dashboard**: Monitor event counts, storage usage, and performance metrics
|
||||
- **Auth Rules Management**: Configure whitelist/blacklist rules for pubkeys
|
||||
- **NIP-42 Authentication**: Secure access using your Nostr identity
|
||||
- **Event-Based Updates**: All changes are applied as cryptographically signed Nostr events
|
||||
|
||||
The web interface serves embedded static files with no external dependencies and includes proper CORS headers for browser compatibility.
|
||||
|
||||
|
||||
## Administrator API
|
||||
|
||||
C-Relay uses an innovative **event-based administration system** where all configuration and management commands are sent as signed Nostr events using the admin private key generated during first startup. All admin commands use **NIP-44 encrypted command arrays** for security and compatibility.
|
||||
|
||||
### Authentication
|
||||
|
||||
All admin commands require signing with the admin private key displayed during first-time startup. **Save this key securely** - it cannot be recovered and is needed for all administrative operations.
|
||||
|
||||
### Event Structure
|
||||
|
||||
All admin commands use the same unified event structure with NIP-44 encrypted content:
|
||||
|
||||
**Admin Command Event:**
|
||||
```json
|
||||
{
|
||||
"id": "event_id",
|
||||
"pubkey": "admin_public_key",
|
||||
"created_at": 1234567890,
|
||||
"kind": 23456,
|
||||
"content": "AqHBUgcM7dXFYLQuDVzGwMST1G8jtWYyVvYxXhVGEu4nAb4LVw...",
|
||||
"tags": [
|
||||
["p", "relay_public_key"]
|
||||
],
|
||||
"sig": "event_signature"
|
||||
}
|
||||
# Run container
|
||||
docker run -d \
|
||||
--name c-relay-pg \
|
||||
-p 8888:8888 \
|
||||
-v /path/to/data:/data \
|
||||
c-relay-pg
|
||||
```
|
||||
|
||||
The `content` field contains a NIP-44 encrypted JSON array representing the command.
|
||||
### Cloud Deployment
|
||||
|
||||
**Admin Response Event:**
|
||||
```json
|
||||
["EVENT", "temp_sub_id", {
|
||||
"id": "response_event_id",
|
||||
"pubkey": "relay_public_key",
|
||||
"created_at": 1234567890,
|
||||
"kind": 23457,
|
||||
"content": "BpKCVhfN8eYtRmPqSvWxZnMkL2gHjUiOp3rTyEwQaS5dFg...",
|
||||
"tags": [
|
||||
["p", "admin_public_key"]
|
||||
],
|
||||
"sig": "response_event_signature"
|
||||
}]
|
||||
Quick deployment scripts for popular cloud providers:
|
||||
|
||||
```bash
|
||||
# AWS, GCP, DigitalOcean, etc.
|
||||
sudo examples/deployment/simple-vps/deploy.sh
|
||||
```
|
||||
|
||||
The `content` field contains a NIP-44 encrypted JSON response object.
|
||||
See [`docs/deployment_guide.md`](docs/deployment_guide.md) for detailed deployment instructions.
|
||||
|
||||
### Admin Commands
|
||||
## 🔧 Configuration
|
||||
|
||||
All commands are sent as NIP-44 encrypted JSON arrays in the event content. The following table lists all available commands:
|
||||
C-Relay-PG uses an innovative event-based configuration system. All settings are managed through signed Nostr events.
|
||||
|
||||
| Command Type | Command Format | Description |
|
||||
|--------------|----------------|-------------|
|
||||
| **Configuration Management** |
|
||||
| `config_update` | `["config_update", [{"key": "auth_enabled", "value": "true", "data_type": "boolean", "category": "auth"}, {"key": "relay_description", "value": "My Relay", "data_type": "string", "category": "relay"}, ...]]` | Update relay configuration parameters (supports multiple updates) |
|
||||
| `config_query` | `["config_query", "all"]` | Query all configuration parameters |
|
||||
| **Auth Rules Management** |
|
||||
| `auth_add_blacklist` | `["blacklist", "pubkey", "abc123..."]` | Add pubkey to blacklist |
|
||||
| `auth_add_whitelist` | `["whitelist", "pubkey", "def456..."]` | Add pubkey to whitelist |
|
||||
| `auth_delete_rule` | `["delete_auth_rule", "blacklist", "pubkey", "abc123..."]` | Delete specific auth rule |
|
||||
| `auth_query_all` | `["auth_query", "all"]` | Query all auth rules |
|
||||
| `auth_query_type` | `["auth_query", "whitelist"]` | Query specific rule type |
|
||||
| `auth_query_pattern` | `["auth_query", "pattern", "abc123..."]` | Query specific pattern |
|
||||
| **System Commands** |
|
||||
| `system_clear_auth` | `["system_command", "clear_all_auth_rules"]` | Clear all auth rules |
|
||||
| `system_status` | `["system_command", "system_status"]` | Get system status |
|
||||
| `stats_query` | `["stats_query"]` | Get comprehensive database statistics |
|
||||
| **Database Queries** |
|
||||
| `sql_query` | `["sql_query", "SELECT * FROM events LIMIT 10"]` | Execute read-only SQL query against relay database |
|
||||
### Basic Configuration
|
||||
|
||||
### Available Configuration Keys
|
||||
Use the web interface at `http://localhost:8888/api/` or send admin commands via the API.
|
||||
|
||||
**Basic Relay Settings:**
|
||||
- `relay_name`: Relay name (displayed in NIP-11)
|
||||
- `relay_description`: Relay description text
|
||||
- `relay_contact`: Contact information
|
||||
- `relay_software`: Software URL
|
||||
- `relay_version`: Software version
|
||||
- `supported_nips`: Comma-separated list of supported NIP numbers (e.g., "1,2,4,9,11,12,13,15,16,20,22,33,40,42")
|
||||
- `language_tags`: Comma-separated list of supported language tags (e.g., "en,es,fr" or "*" for all)
|
||||
- `relay_countries`: Comma-separated list of supported country codes (e.g., "US,CA,MX" or "*" for all)
|
||||
- `posting_policy`: Posting policy URL or text
|
||||
- `payments_url`: Payment URL for premium features
|
||||
- `max_connections`: Maximum concurrent connections
|
||||
- `max_subscriptions_per_client`: Max subscriptions per client
|
||||
- `max_event_tags`: Maximum tags per event
|
||||
- `max_content_length`: Maximum event content length
|
||||
### Common Settings
|
||||
|
||||
**Authentication & Access Control:**
|
||||
- `auth_enabled`: Enable whitelist/blacklist auth rules (`true`/`false`)
|
||||
- `nip42_auth_required`: Enable NIP-42 cryptographic authentication (`true`/`false`)
|
||||
- `nip42_auth_required_kinds`: Event kinds requiring NIP-42 auth (comma-separated)
|
||||
- `nip42_challenge_timeout`: NIP-42 challenge expiration seconds
|
||||
- **Relay Information**: Name, description, contact info
|
||||
- **Connection Limits**: Max subscriptions per client, total subscriptions
|
||||
- **Authentication**: Enable/disable NIP-42, whitelist/blacklist rules
|
||||
- **Proof of Work**: Minimum difficulty, enforcement mode
|
||||
- **Event Validation**: Max tags, content length, message size
|
||||
- **Expiration**: Enable/disable NIP-40 event expiration
|
||||
|
||||
**Proof of Work & Validation:**
|
||||
- `pow_min_difficulty`: Minimum proof-of-work difficulty
|
||||
- `nip40_expiration_enabled`: Enable event expiration (`true`/`false`)
|
||||
### Dynamic Updates
|
||||
|
||||
**Monitoring Settings:**
|
||||
- `kind_24567_reporting_throttle_sec`: Minimum seconds between monitoring events (default: 5)
|
||||
Most configuration changes take effect immediately without restart:
|
||||
- Relay information (NIP-11)
|
||||
- Authentication settings
|
||||
- Subscription limits
|
||||
- Event validation rules
|
||||
- Proof of Work settings
|
||||
|
||||
### Dynamic Configuration Updates
|
||||
See [`docs/configuration_guide.md`](docs/configuration_guide.md) for complete configuration reference.
|
||||
|
||||
C-Relay supports **dynamic configuration updates** without requiring a restart for most settings. Configuration parameters are categorized as either **dynamic** (can be updated immediately) or **restart-required** (require relay restart to take effect).
|
||||
## 📚 Documentation
|
||||
|
||||
**Dynamic Configuration Parameters (No Restart Required):**
|
||||
- All relay information (NIP-11) settings: `relay_name`, `relay_description`, `relay_contact`, `relay_software`, `relay_version`, `supported_nips`, `language_tags`, `relay_countries`, `posting_policy`, `payments_url`
|
||||
- Authentication settings: `auth_enabled`, `nip42_auth_required`, `nip42_auth_required_kinds`, `nip42_challenge_timeout`
|
||||
- Subscription limits: `max_subscriptions_per_client`, `max_total_subscriptions`
|
||||
- Event validation limits: `max_event_tags`, `max_content_length`, `max_message_length`
|
||||
- Proof of Work settings: `pow_min_difficulty`, `pow_mode`
|
||||
- Event expiration settings: `nip40_expiration_enabled`, `nip40_expiration_strict`, `nip40_expiration_filter`, `nip40_expiration_grace_period`
|
||||
- **[API Documentation](API.md)** - Complete API reference and advanced features
|
||||
- **[Configuration Guide](docs/configuration_guide.md)** - Detailed configuration options
|
||||
- **[Deployment Guide](docs/deployment_guide.md)** - Production deployment instructions
|
||||
- **[User Guide](docs/user_guide.md)** - End-user documentation
|
||||
- **[NIP-42 Authentication](docs/NIP-42_Authentication.md)** - Authentication setup guide
|
||||
|
||||
**Restart-Required Configuration Parameters:**
|
||||
- Connection settings: `max_connections`, `relay_port`
|
||||
- Database and core system settings
|
||||
## 🧪 Testing
|
||||
|
||||
When updating configuration, the admin API response will indicate whether a restart is required for each parameter. Dynamic updates take effect immediately and are reflected in NIP-11 relay information documents without restart.
|
||||
Run the comprehensive test suite:
|
||||
|
||||
### Response Format
|
||||
```bash
|
||||
# Run all tests
|
||||
./tests/run_all_tests.sh
|
||||
|
||||
All admin commands return **signed EVENT responses** via WebSocket following standard Nostr protocol. Responses use JSON content with structured data.
|
||||
# Run NIP-specific tests
|
||||
./tests/run_nip_tests.sh
|
||||
|
||||
#### Response Examples
|
||||
|
||||
**Success Response:**
|
||||
```json
|
||||
["EVENT", "temp_sub_id", {
|
||||
"id": "response_event_id",
|
||||
"pubkey": "relay_public_key",
|
||||
"created_at": 1234567890,
|
||||
"kind": 23457,
|
||||
"content": "nip44 encrypted:{\"query_type\": \"config_update\", \"status\": \"success\", \"message\": \"Operation completed successfully\", \"timestamp\": 1234567890}",
|
||||
"tags": [
|
||||
["p", "admin_public_key"]
|
||||
],
|
||||
"sig": "response_event_signature"
|
||||
}]
|
||||
# Test specific NIPs
|
||||
./tests/42_nip_test.sh # NIP-42 authentication
|
||||
./tests/13_nip_test.sh # NIP-13 proof of work
|
||||
```
|
||||
|
||||
**Error Response:**
|
||||
```json
|
||||
["EVENT", "temp_sub_id", {
|
||||
"id": "response_event_id",
|
||||
"pubkey": "relay_public_key",
|
||||
"created_at": 1234567890,
|
||||
"kind": 23457,
|
||||
"content": "nip44 encrypted:{\"query_type\": \"config_update\", \"status\": \"error\", \"error\": \"invalid configuration value\", \"timestamp\": 1234567890}",
|
||||
"tags": [
|
||||
["p", "admin_public_key"]
|
||||
],
|
||||
"sig": "response_event_signature"
|
||||
}]
|
||||
## 🔒 Security
|
||||
|
||||
### Admin Key Management
|
||||
|
||||
The admin private key is displayed **only once** during first startup. Store it securely:
|
||||
|
||||
```bash
|
||||
# Save to secure location
|
||||
echo "ADMIN_PRIVKEY=your_admin_private_key" > ~/.c-relay-pg-admin
|
||||
chmod 600 ~/.c-relay-pg-admin
|
||||
```
|
||||
|
||||
**Auth Rules Query Response:**
|
||||
```json
|
||||
["EVENT", "temp_sub_id", {
|
||||
"id": "response_event_id",
|
||||
"pubkey": "relay_public_key",
|
||||
"created_at": 1234567890,
|
||||
"kind": 23457,
|
||||
"content": "nip44 encrypted:{\"query_type\": \"auth_rules_all\", \"total_results\": 2, \"timestamp\": 1234567890, \"data\": [{\"rule_type\": \"blacklist\", \"pattern_type\": \"pubkey\", \"pattern_value\": \"abc123...\", \"action\": \"allow\"}]}",
|
||||
"tags": [
|
||||
["p", "admin_public_key"]
|
||||
],
|
||||
"sig": "response_event_signature"
|
||||
}]
|
||||
```
|
||||
### Production Security
|
||||
|
||||
**Configuration Query Response:**
|
||||
```json
|
||||
["EVENT", "temp_sub_id", {
|
||||
"id": "response_event_id",
|
||||
"pubkey": "relay_public_key",
|
||||
"created_at": 1234567890,
|
||||
"kind": 23457,
|
||||
"content": "nip44 encrypted:{\"query_type\": \"config_all\", \"total_results\": 27, \"timestamp\": 1234567890, \"data\": [{\"key\": \"auth_enabled\", \"value\": \"false\", \"data_type\": \"boolean\", \"category\": \"auth\", \"description\": \"Enable NIP-42 authentication\"}, {\"key\": \"relay_description\", \"value\": \"My Relay\", \"data_type\": \"string\", \"category\": \"relay\", \"description\": \"Relay description text\"}]}",
|
||||
"tags": [
|
||||
["p", "admin_public_key"]
|
||||
],
|
||||
"sig": "response_event_signature"
|
||||
}]
|
||||
```
|
||||
- Use HTTPS/WSS with reverse proxy (nginx/Apache)
|
||||
- Enable NIP-42 authentication for client verification
|
||||
- Configure whitelist/blacklist for access control
|
||||
- Set up firewall rules to restrict access
|
||||
- Enable Proof of Work to prevent spam
|
||||
- Regular database backups
|
||||
|
||||
**Configuration Update Success Response:**
|
||||
```json
|
||||
["EVENT", "temp_sub_id", {
|
||||
"id": "response_event_id",
|
||||
"pubkey": "relay_public_key",
|
||||
"created_at": 1234567890,
|
||||
"kind": 23457,
|
||||
"content": "nip44 encrypted:{\"query_type\": \"config_update\", \"total_results\": 2, \"timestamp\": 1234567890, \"status\": \"success\", \"data\": [{\"key\": \"auth_enabled\", \"value\": \"true\", \"status\": \"updated\"}, {\"key\": \"relay_description\", \"value\": \"My Updated Relay\", \"status\": \"updated\"}]}",
|
||||
"tags": [
|
||||
["p", "admin_public_key"]
|
||||
],
|
||||
"sig": "response_event_signature"
|
||||
}]
|
||||
```
|
||||
See [`docs/deployment_guide.md`](docs/deployment_guide.md#security-hardening) for security hardening guide.
|
||||
|
||||
**Configuration Update Error Response:**
|
||||
```json
|
||||
["EVENT", "temp_sub_id", {
|
||||
"id": "response_event_id",
|
||||
"pubkey": "relay_public_key",
|
||||
"created_at": 1234567890,
|
||||
"kind": 23457,
|
||||
"content": "nip44 encrypted:{\"query_type\": \"config_update\", \"status\": \"error\", \"error\": \"field validation failed: invalid port number '99999' (must be 1-65535)\", \"timestamp\": 1234567890}",
|
||||
"tags": [
|
||||
["p", "admin_public_key"]
|
||||
],
|
||||
"sig": "response_event_signature"
|
||||
}]
|
||||
```
|
||||
## 🤝 Contributing
|
||||
|
||||
**Database Statistics Query Response:**
|
||||
```json
|
||||
["EVENT", "temp_sub_id", {
|
||||
"id": "response_event_id",
|
||||
"pubkey": "relay_public_key",
|
||||
"created_at": 1234567890,
|
||||
"kind": 23457,
|
||||
"content": "nip44 encrypted:{\"query_type\": \"stats_query\", \"timestamp\": 1234567890, \"database_size_bytes\": 1048576, \"total_events\": 15432, \"database_created_at\": 1234567800, \"latest_event_at\": 1234567890, \"event_kinds\": [{\"kind\": 1, \"count\": 12000, \"percentage\": 77.8}, {\"kind\": 0, \"count\": 2500, \"percentage\": 16.2}], \"time_stats\": {\"total\": 15432, \"last_24h\": 234, \"last_7d\": 1456, \"last_30d\": 5432}, \"top_pubkeys\": [{\"pubkey\": \"abc123...\", \"event_count\": 1234, \"percentage\": 8.0}, {\"pubkey\": \"def456...\", \"event_count\": 987, \"percentage\": 6.4}]}",
|
||||
"tags": [
|
||||
["p", "admin_public_key"]
|
||||
],
|
||||
"sig": "response_event_signature"
|
||||
}]
|
||||
```
|
||||
Contributions are welcome! Please:
|
||||
|
||||
**SQL Query Response:**
|
||||
```json
|
||||
["EVENT", "temp_sub_id", {
|
||||
"id": "response_event_id",
|
||||
"pubkey": "relay_public_key",
|
||||
"created_at": 1234567890,
|
||||
"kind": 23457,
|
||||
"content": "nip44 encrypted:{\"query_type\": \"sql_query\", \"request_id\": \"request_event_id\", \"timestamp\": 1234567890, \"query\": \"SELECT * FROM events LIMIT 10\", \"execution_time_ms\": 45, \"row_count\": 10, \"columns\": [\"id\", \"pubkey\", \"created_at\", \"kind\", \"content\"], \"rows\": [[\"abc123...\", \"def456...\", 1234567890, 1, \"Hello world\"], ...]}",
|
||||
"tags": [
|
||||
["p", "admin_public_key"],
|
||||
["e", "request_event_id"]
|
||||
],
|
||||
"sig": "response_event_signature"
|
||||
}]
|
||||
```
|
||||
1. Fork the repository
|
||||
2. Create a feature branch
|
||||
3. Make your changes with tests
|
||||
4. Submit a pull request
|
||||
|
||||
### SQL Query Command
|
||||
## 📄 License
|
||||
|
||||
The `sql_query` command allows administrators to execute read-only SQL queries against the relay database. This provides powerful analytics and debugging capabilities through the admin API.
|
||||
[Add your license here]
|
||||
|
||||
**Request/Response Correlation:**
|
||||
- Each response includes the request event ID in both the `tags` array (`["e", "request_event_id"]`) and the decrypted content (`"request_id": "request_event_id"`)
|
||||
- This allows proper correlation when multiple queries are submitted concurrently
|
||||
- Frontend can track pending queries and match responses to requests
|
||||
## 🔗 Links
|
||||
|
||||
**Security Features:**
|
||||
- Only SELECT statements allowed (INSERT, UPDATE, DELETE, DROP, etc. are blocked)
|
||||
- Query timeout: 5 seconds (configurable)
|
||||
- Result row limit: 1000 rows (configurable)
|
||||
- All queries logged with execution time
|
||||
- **Repository**: [https://github.com/your-org/c-relay-pg](https://github.com/your-org/c-relay-pg)
|
||||
- **Releases**: [https://git.laantungir.net/laantungir/c-relay-pg/releases](https://git.laantungir.net/laantungir/c-relay-pg/releases)
|
||||
- **Issues**: [https://github.com/your-org/c-relay-pg/issues](https://github.com/your-org/c-relay-pg/issues)
|
||||
- **Nostr Protocol**: [https://github.com/nostr-protocol/nostr](https://github.com/nostr-protocol/nostr)
|
||||
|
||||
**Available Tables and Views:**
|
||||
- `events` - All Nostr events
|
||||
- `config` - Configuration parameters
|
||||
- `auth_rules` - Authentication rules
|
||||
- `subscription_events` - Subscription lifecycle log
|
||||
- `event_broadcasts` - Event broadcast log
|
||||
- `recent_events` - Last 1000 events (view)
|
||||
- `event_stats` - Event statistics by type (view)
|
||||
- `subscription_analytics` - Subscription metrics (view)
|
||||
- `active_subscriptions_log` - Currently active subscriptions (view)
|
||||
- `event_kinds_view` - Event distribution by kind (view)
|
||||
- `top_pubkeys_view` - Top 10 pubkeys by event count (view)
|
||||
- `time_stats_view` - Time-based statistics (view)
|
||||
## 💬 Support
|
||||
|
||||
**Example Queries:**
|
||||
```sql
|
||||
-- Recent events
|
||||
SELECT id, pubkey, created_at, kind FROM events ORDER BY created_at DESC LIMIT 20
|
||||
|
||||
-- Event distribution by kind
|
||||
SELECT * FROM event_kinds_view ORDER BY count DESC
|
||||
|
||||
-- Active subscriptions
|
||||
SELECT * FROM active_subscriptions_log ORDER BY created_at DESC
|
||||
|
||||
-- Database statistics
|
||||
SELECT
|
||||
(SELECT COUNT(*) FROM events) as total_events,
|
||||
(SELECT COUNT(*) FROM subscription_events) as total_subscriptions
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Real-time Monitoring System
|
||||
|
||||
C-Relay includes a subscription-based monitoring system that broadcasts real-time relay statistics using ephemeral events (kind 24567).
|
||||
|
||||
### Activation
|
||||
|
||||
The monitoring system activates automatically when clients subscribe to kind 24567 events:
|
||||
|
||||
```json
|
||||
["REQ", "monitoring-sub", {"kinds": [24567]}]
|
||||
```
|
||||
|
||||
For specific monitoring types, use d-tag filters:
|
||||
|
||||
```json
|
||||
["REQ", "event-kinds-sub", {"kinds": [24567], "#d": ["event_kinds"]}]
|
||||
["REQ", "time-stats-sub", {"kinds": [24567], "#d": ["time_stats"]}]
|
||||
["REQ", "top-pubkeys-sub", {"kinds": [24567], "#d": ["top_pubkeys"]}]
|
||||
```
|
||||
|
||||
When no subscriptions exist, monitoring is dormant to conserve resources.
|
||||
|
||||
### Monitoring Event Types
|
||||
|
||||
| Type | d Tag | Description |
|
||||
|------|-------|-------------|
|
||||
| Event Distribution | `event_kinds` | Event count by kind with percentages |
|
||||
| Time Statistics | `time_stats` | Events in last 24h, 7d, 30d |
|
||||
| Top Publishers | `top_pubkeys` | Top 10 pubkeys by event count |
|
||||
| Active Subscriptions | `active_subscriptions` | Current subscription details (admin only) |
|
||||
| Subscription Details | `subscription_details` | Detailed subscription info (admin only) |
|
||||
| CPU Metrics | `cpu_metrics` | Process CPU and memory usage |
|
||||
|
||||
### Event Structure
|
||||
|
||||
```json
|
||||
{
|
||||
"kind": 24567,
|
||||
"pubkey": "<relay_pubkey>",
|
||||
"created_at": <timestamp>,
|
||||
"content": "{\"data_type\":\"event_kinds\",\"timestamp\":1234567890,...}",
|
||||
"tags": [
|
||||
["d", "event_kinds"]
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
- `kind_24567_reporting_throttle_sec`: Minimum seconds between monitoring events (default: 5)
|
||||
|
||||
### Web Dashboard Integration
|
||||
|
||||
The built-in web dashboard (`/api/`) automatically subscribes to monitoring events and displays real-time statistics.
|
||||
|
||||
### Performance Considerations
|
||||
|
||||
- Monitoring events are ephemeral (not stored in database)
|
||||
- Throttling prevents excessive event generation
|
||||
- Automatic activation/deactivation based on subscriptions
|
||||
- Minimal overhead when no clients are monitoring
|
||||
|
||||
## Direct Messaging Admin System
|
||||
|
||||
In addition to the above admin API, c-relay allows the administrator to direct message the relay to get information or control some settings. As long as the administrator is signed in with any nostr client that allows sending nip-17 direct messages (DMs), they can control the relay.
|
||||
|
||||
The is possible because the relay is a full nostr citizen with it's own private and public key, and it knows the administrator's public key.
|
||||
|
||||
|
||||
|
||||
**Available DM commands**
|
||||
|
||||
The intent is not to be strict in the formatting of the DM. So for example if the relay receives any DM from the administrator with the words "stats" or "statistics" in it, it will respond to the administrator with a reply DM with the current relay statistics.
|
||||
|
||||
- `stats`|`statistics`: Relay statistics
|
||||
- `config`|`configuration`: Relay configuration
|
||||
- Open an issue on GitHub
|
||||
- Join the Nostr community
|
||||
- Contact via Nostr DM (relay pubkey shown on startup)
|
||||
|
||||
---
|
||||
|
||||
**Built with ❤️ for the Nostr protocol**
|
||||
|
||||
@@ -0,0 +1,174 @@
|
||||
# Real-Time Traffic Monitoring Commands (Direct Server Use)
|
||||
|
||||
Copy and paste these commands directly on your server.
|
||||
|
||||
## Quick Status Checks
|
||||
|
||||
### See IPs visiting in the last few minutes:
|
||||
```bash
|
||||
sudo tail -500 /var/log/nginx/access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head -20
|
||||
```
|
||||
|
||||
### See what status codes they're getting:
|
||||
```bash
|
||||
sudo tail -500 /var/log/nginx/access.log | awk '{print $1, $9}' | grep '216.73.216.38'
|
||||
```
|
||||
|
||||
### Count status codes (200 vs 403):
|
||||
```bash
|
||||
sudo tail -500 /var/log/nginx/access.log | awk '{print $9}' | sort | uniq -c
|
||||
```
|
||||
|
||||
## Real-Time Monitoring
|
||||
|
||||
### Watch live traffic (updates every 2 seconds):
|
||||
```bash
|
||||
watch -n 2 'sudo tail -200 /var/log/nginx/access.log | awk "{print \$1}" | sort | uniq -c | sort -rn | head -15'
|
||||
```
|
||||
|
||||
### See live log entries as they happen:
|
||||
```bash
|
||||
sudo tail -f /var/log/nginx/access.log
|
||||
```
|
||||
|
||||
### Live GoAccess dashboard:
|
||||
```bash
|
||||
sudo tail -f /var/log/nginx/access.log | goaccess -
|
||||
```
|
||||
|
||||
## Active Connections
|
||||
|
||||
### See who's connected RIGHT NOW:
|
||||
```bash
|
||||
sudo netstat -tn | grep ':443' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -rn
|
||||
```
|
||||
|
||||
### Alternative (using ss command):
|
||||
```bash
|
||||
sudo ss -tn | grep ':443' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -rn
|
||||
```
|
||||
|
||||
## Detailed Analysis
|
||||
|
||||
### Last 100 requests with timestamps:
|
||||
```bash
|
||||
sudo tail -100 /var/log/nginx/access.log | awk '{print $4, $1}' | sed 's/\[//'
|
||||
```
|
||||
|
||||
### See what blocked IPs are trying to access:
|
||||
```bash
|
||||
sudo tail -500 /var/log/nginx/access.log | grep '216.73.216.38' | awk '{print $7}' | head -10
|
||||
```
|
||||
|
||||
### Show all 403 (blocked) requests:
|
||||
```bash
|
||||
sudo tail -500 /var/log/nginx/access.log | awk '$9==403 {print $1}' | sort | uniq -c | sort -rn
|
||||
```
|
||||
|
||||
### Show all successful (200) requests:
|
||||
```bash
|
||||
sudo tail -500 /var/log/nginx/access.log | awk '$9==200 {print $1}' | sort | uniq -c | sort -rn | head -10
|
||||
```
|
||||
|
||||
## Comprehensive Monitoring Script
|
||||
|
||||
### Create a monitoring script:
|
||||
```bash
|
||||
cat > /tmp/monitor-traffic.sh << 'EOF'
|
||||
#!/bin/bash
|
||||
echo "=== Traffic in last 5 minutes ==="
|
||||
echo "Time: $(date)"
|
||||
echo ""
|
||||
echo "Top IPs:"
|
||||
sudo tail -1000 /var/log/nginx/access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head -10
|
||||
echo ""
|
||||
echo "Blocked IPs (403 errors):"
|
||||
sudo tail -1000 /var/log/nginx/access.log | awk '$9==403 {print $1}' | sort | uniq -c | sort -rn
|
||||
echo ""
|
||||
echo "Successful requests (200):"
|
||||
sudo tail -1000 /var/log/nginx/access.log | awk '$9==200 {print $1}' | sort | uniq -c | sort -rn | head -5
|
||||
echo ""
|
||||
echo "Status Code Summary:"
|
||||
sudo tail -1000 /var/log/nginx/access.log | awk '{print $9}' | sort | uniq -c | sort -rn
|
||||
EOF
|
||||
chmod +x /tmp/monitor-traffic.sh
|
||||
```
|
||||
|
||||
### Run the monitoring script:
|
||||
```bash
|
||||
/tmp/monitor-traffic.sh
|
||||
```
|
||||
|
||||
## Auto-Refreshing Dashboard
|
||||
|
||||
### Live dashboard (refreshes every 5 seconds):
|
||||
```bash
|
||||
watch -n 5 'echo "=== Last 5 minutes ==="
|
||||
date
|
||||
echo ""
|
||||
echo "Top IPs:"
|
||||
sudo tail -1000 /var/log/nginx/access.log | awk "{print \$1}" | sort | uniq -c | sort -rn | head -10
|
||||
echo ""
|
||||
echo "Status Codes:"
|
||||
sudo tail -1000 /var/log/nginx/access.log | awk "{print \$9}" | sort | uniq -c | sort -rn'
|
||||
```
|
||||
|
||||
Press `Ctrl+C` to exit.
|
||||
|
||||
## GoAccess HTML Report (Live Updating)
|
||||
|
||||
### Generate live HTML report:
|
||||
```bash
|
||||
sudo goaccess /var/log/nginx/access.log -o /var/www/html/live-stats.html --real-time-html --daemonize
|
||||
```
|
||||
|
||||
Then visit: https://git.laantungir.net/live-stats.html
|
||||
|
||||
### Stop the live report:
|
||||
```bash
|
||||
sudo pkill -f "goaccess.*live-stats"
|
||||
```
|
||||
|
||||
## Filter by Time
|
||||
|
||||
### Get timestamp from 5 minutes ago:
|
||||
```bash
|
||||
date -d '5 minutes ago' '+%d/%b/%Y:%H:%M'
|
||||
```
|
||||
|
||||
### Analyze only recent logs (replace timestamp):
|
||||
```bash
|
||||
sudo awk '/01\/Feb\/2026:19:09/,0' /var/log/nginx/access.log | goaccess -
|
||||
```
|
||||
|
||||
## Check Gitea CPU
|
||||
|
||||
### Current CPU usage:
|
||||
```bash
|
||||
ps aux | grep gitea | grep -v grep
|
||||
```
|
||||
|
||||
### Watch CPU in real-time:
|
||||
```bash
|
||||
watch -n 2 'ps aux | grep gitea | grep -v grep'
|
||||
```
|
||||
|
||||
## Most Useful Command for Quick Check
|
||||
|
||||
This one-liner shows everything you need:
|
||||
```bash
|
||||
|
||||
echo "=== Quick Status ===" && \
|
||||
echo "Time: $(date)" && \
|
||||
echo "" && \
|
||||
echo "Top 10 IPs (last 1000 requests):" && \
|
||||
sudo tail -1000 /var/log/nginx/access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head -10 && \
|
||||
echo "" && \
|
||||
echo "Status Codes:" && \
|
||||
sudo tail -1000 /var/log/nginx/access.log | awk '{print $9}' | sort | uniq -c && \
|
||||
echo "" && \
|
||||
echo "Gitea CPU:" && \
|
||||
ps aux | grep gitea | grep -v grep
|
||||
```
|
||||
|
||||
Copy any of these commands and run them directly on your server!
|
||||
@@ -0,0 +1,612 @@
|
||||
# Static MUSL Build Guide for C Programs
|
||||
|
||||
## Overview
|
||||
|
||||
This guide explains how to build truly portable static binaries using Alpine Linux and MUSL libc. These binaries have **zero runtime dependencies** and work on any Linux distribution without modification.
|
||||
|
||||
This guide is specifically tailored for C programs that use:
|
||||
- **nostr_core_lib** - Nostr protocol implementation
|
||||
- **nostr_login_lite** - Nostr authentication library
|
||||
- Common dependencies: libwebsockets, OpenSSL, SQLite, curl, secp256k1
|
||||
|
||||
## Why MUSL Static Binaries?
|
||||
|
||||
### Advantages Over glibc
|
||||
|
||||
| Feature | MUSL Static | glibc Static | glibc Dynamic |
|
||||
|---------|-------------|--------------|---------------|
|
||||
| **Portability** | ✓ Any Linux | ⚠ glibc only | ✗ Requires matching libs |
|
||||
| **Binary Size** | ~7-10 MB | ~12-15 MB | ~2-3 MB |
|
||||
| **Dependencies** | None | NSS libs | Many system libs |
|
||||
| **Deployment** | Single file | Single file + NSS | Binary + libraries |
|
||||
| **Compatibility** | Universal | glibc version issues | Library version hell |
|
||||
|
||||
### Key Benefits
|
||||
|
||||
1. **True Portability**: Works on Alpine, Ubuntu, Debian, CentOS, Arch, etc.
|
||||
2. **No Library Hell**: No `GLIBC_2.XX not found` errors
|
||||
3. **Simple Deployment**: Just copy one file
|
||||
4. **Reproducible Builds**: Same Docker image = same binary
|
||||
5. **Security**: No dependency on system libraries with vulnerabilities
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Docker installed and running
|
||||
- Your C project with source code
|
||||
- Internet connection for downloading dependencies
|
||||
|
||||
### Basic Build Process
|
||||
|
||||
```bash
|
||||
# 1. Copy the Dockerfile template (see below)
|
||||
cp /path/to/c-relay-pg/Dockerfile.alpine-musl ./Dockerfile.static
|
||||
|
||||
# 2. Customize for your project (see Customization section)
|
||||
vim Dockerfile.static
|
||||
|
||||
# 3. Build the static binary
|
||||
docker build --platform linux/amd64 -f Dockerfile.static -t my-app-builder .
|
||||
|
||||
# 4. Extract the binary
|
||||
docker create --name temp-container my-app-builder
|
||||
docker cp temp-container:/build/my_app_static ./my_app_static
|
||||
docker rm temp-container
|
||||
|
||||
# 5. Verify it's static
|
||||
ldd ./my_app_static # Should show "not a dynamic executable"
|
||||
```
|
||||
|
||||
## Dockerfile Template
|
||||
|
||||
Here's a complete Dockerfile template you can customize for your project:
|
||||
|
||||
```dockerfile
|
||||
# Alpine-based MUSL static binary builder
|
||||
# Produces truly portable binaries with zero runtime dependencies
|
||||
|
||||
FROM alpine:3.19 AS builder
|
||||
|
||||
# Install build dependencies
|
||||
RUN apk add --no-cache \
|
||||
build-base \
|
||||
musl-dev \
|
||||
git \
|
||||
cmake \
|
||||
pkgconfig \
|
||||
autoconf \
|
||||
automake \
|
||||
libtool \
|
||||
openssl-dev \
|
||||
openssl-libs-static \
|
||||
zlib-dev \
|
||||
zlib-static \
|
||||
curl-dev \
|
||||
curl-static \
|
||||
sqlite-dev \
|
||||
sqlite-static \
|
||||
linux-headers \
|
||||
wget \
|
||||
bash
|
||||
|
||||
WORKDIR /build
|
||||
|
||||
# Build libsecp256k1 static (required for Nostr)
|
||||
RUN cd /tmp && \
|
||||
git clone https://github.com/bitcoin-core/secp256k1.git && \
|
||||
cd secp256k1 && \
|
||||
./autogen.sh && \
|
||||
./configure --enable-static --disable-shared --prefix=/usr \
|
||||
CFLAGS="-fPIC" && \
|
||||
make -j$(nproc) && \
|
||||
make install && \
|
||||
rm -rf /tmp/secp256k1
|
||||
|
||||
# Build libwebsockets static (if needed for WebSocket support)
|
||||
RUN cd /tmp && \
|
||||
git clone --depth 1 --branch v4.3.3 https://github.com/warmcat/libwebsockets.git && \
|
||||
cd libwebsockets && \
|
||||
mkdir build && cd build && \
|
||||
cmake .. \
|
||||
-DLWS_WITH_STATIC=ON \
|
||||
-DLWS_WITH_SHARED=OFF \
|
||||
-DLWS_WITH_SSL=ON \
|
||||
-DLWS_WITHOUT_TESTAPPS=ON \
|
||||
-DLWS_WITHOUT_TEST_SERVER=ON \
|
||||
-DLWS_WITHOUT_TEST_CLIENT=ON \
|
||||
-DLWS_WITHOUT_TEST_PING=ON \
|
||||
-DLWS_WITH_HTTP2=OFF \
|
||||
-DLWS_WITH_LIBUV=OFF \
|
||||
-DLWS_WITH_LIBEVENT=OFF \
|
||||
-DLWS_IPV6=ON \
|
||||
-DCMAKE_BUILD_TYPE=Release \
|
||||
-DCMAKE_INSTALL_PREFIX=/usr \
|
||||
-DCMAKE_C_FLAGS="-fPIC" && \
|
||||
make -j$(nproc) && \
|
||||
make install && \
|
||||
rm -rf /tmp/libwebsockets
|
||||
|
||||
# Copy git configuration for submodules
|
||||
COPY .gitmodules /build/.gitmodules
|
||||
COPY .git /build/.git
|
||||
|
||||
# Initialize submodules
|
||||
RUN git submodule update --init --recursive
|
||||
|
||||
# Copy and build nostr_core_lib
|
||||
COPY nostr_core_lib /build/nostr_core_lib/
|
||||
RUN cd nostr_core_lib && \
|
||||
chmod +x build.sh && \
|
||||
sed -i 's/CFLAGS="-Wall -Wextra -std=c99 -fPIC -O2"/CFLAGS="-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -Wall -Wextra -std=c99 -fPIC -O2"/' build.sh && \
|
||||
rm -f *.o *.a 2>/dev/null || true && \
|
||||
./build.sh --nips=1,6,13,17,19,44,59
|
||||
|
||||
# Copy and build nostr_login_lite (if used)
|
||||
# COPY nostr_login_lite /build/nostr_login_lite/
|
||||
# RUN cd nostr_login_lite && make static
|
||||
|
||||
# Copy your application source
|
||||
COPY src/ /build/src/
|
||||
COPY Makefile /build/Makefile
|
||||
|
||||
# Build your application with full static linking
|
||||
RUN gcc -static -O2 -Wall -Wextra -std=c99 \
|
||||
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 \
|
||||
-I. -Inostr_core_lib -Inostr_core_lib/nostr_core \
|
||||
-Inostr_core_lib/cjson -Inostr_core_lib/nostr_websocket \
|
||||
src/*.c \
|
||||
-o /build/my_app_static \
|
||||
nostr_core_lib/libnostr_core_x64.a \
|
||||
-lwebsockets -lssl -lcrypto -lsqlite3 -lsecp256k1 \
|
||||
-lcurl -lz -lpthread -lm -ldl && \
|
||||
strip /build/my_app_static
|
||||
|
||||
# Verify it's truly static
|
||||
RUN echo "=== Binary Information ===" && \
|
||||
file /build/my_app_static && \
|
||||
ls -lh /build/my_app_static && \
|
||||
echo "=== Checking for dynamic dependencies ===" && \
|
||||
(ldd /build/my_app_static 2>&1 || echo "Binary is static")
|
||||
|
||||
# Output stage - just the binary
|
||||
FROM scratch AS output
|
||||
COPY --from=builder /build/my_app_static /my_app_static
|
||||
```
|
||||
|
||||
## Customization Guide
|
||||
|
||||
### 1. Adjust Dependencies
|
||||
|
||||
**Add dependencies** by modifying the `apk add` section:
|
||||
|
||||
```dockerfile
|
||||
RUN apk add --no-cache \
|
||||
build-base \
|
||||
musl-dev \
|
||||
# Add your dependencies here:
|
||||
libpng-dev \
|
||||
libpng-static \
|
||||
libjpeg-turbo-dev \
|
||||
libjpeg-turbo-static
|
||||
```
|
||||
|
||||
**Remove unused dependencies** to speed up builds:
|
||||
- Remove `libwebsockets` section if you don't need WebSocket support
|
||||
- Remove `sqlite` if you don't use databases
|
||||
- Remove `curl` if you don't make HTTP requests
|
||||
|
||||
### 2. Configure nostr_core_lib NIPs
|
||||
|
||||
Specify which NIPs your application needs:
|
||||
|
||||
```bash
|
||||
./build.sh --nips=1,6,19 # Minimal: Basic protocol, keys, bech32
|
||||
./build.sh --nips=1,6,13,17,19,44,59 # Full: All common NIPs
|
||||
./build.sh --nips=all # Everything available
|
||||
```
|
||||
|
||||
**Common NIP combinations:**
|
||||
- **Basic client**: `1,6,19` (events, keys, bech32)
|
||||
- **With encryption**: `1,6,19,44` (add modern encryption)
|
||||
- **With DMs**: `1,6,17,19,44,59` (add private messages)
|
||||
- **Relay/server**: `1,6,13,17,19,42,44,59` (add PoW, auth)
|
||||
|
||||
### 3. Modify Compilation Flags
|
||||
|
||||
**For your application:**
|
||||
|
||||
```dockerfile
|
||||
RUN gcc -static -O2 -Wall -Wextra -std=c99 \
|
||||
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 \ # REQUIRED for MUSL
|
||||
-I. -Inostr_core_lib \ # Include paths
|
||||
src/*.c \ # Your source files
|
||||
-o /build/my_app_static \ # Output binary
|
||||
nostr_core_lib/libnostr_core_x64.a \ # Nostr library
|
||||
-lwebsockets -lssl -lcrypto \ # Link libraries
|
||||
-lsqlite3 -lsecp256k1 -lcurl \
|
||||
-lz -lpthread -lm -ldl
|
||||
```
|
||||
|
||||
**Debug build** (with symbols, no optimization):
|
||||
|
||||
```dockerfile
|
||||
RUN gcc -static -g -O0 -DDEBUG \
|
||||
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 \
|
||||
# ... rest of flags
|
||||
```
|
||||
|
||||
### 4. Multi-Architecture Support
|
||||
|
||||
Build for different architectures:
|
||||
|
||||
```bash
|
||||
# x86_64 (Intel/AMD)
|
||||
docker build --platform linux/amd64 -f Dockerfile.static -t my-app-x86 .
|
||||
|
||||
# ARM64 (Apple Silicon, Raspberry Pi 4+)
|
||||
docker build --platform linux/arm64 -f Dockerfile.static -t my-app-arm64 .
|
||||
```
|
||||
|
||||
## Build Script Template
|
||||
|
||||
Create a `build_static.sh` script for convenience:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
BUILD_DIR="$SCRIPT_DIR/build"
|
||||
DOCKERFILE="$SCRIPT_DIR/Dockerfile.static"
|
||||
|
||||
# Detect architecture
|
||||
ARCH=$(uname -m)
|
||||
case "$ARCH" in
|
||||
x86_64)
|
||||
PLATFORM="linux/amd64"
|
||||
OUTPUT_NAME="my_app_static_x86_64"
|
||||
;;
|
||||
aarch64|arm64)
|
||||
PLATFORM="linux/arm64"
|
||||
OUTPUT_NAME="my_app_static_arm64"
|
||||
;;
|
||||
*)
|
||||
echo "Unknown architecture: $ARCH"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
echo "Building for platform: $PLATFORM"
|
||||
mkdir -p "$BUILD_DIR"
|
||||
|
||||
# Build Docker image
|
||||
docker build \
|
||||
--platform "$PLATFORM" \
|
||||
-f "$DOCKERFILE" \
|
||||
-t my-app-builder:latest \
|
||||
--progress=plain \
|
||||
.
|
||||
|
||||
# Extract binary
|
||||
CONTAINER_ID=$(docker create my-app-builder:latest)
|
||||
docker cp "$CONTAINER_ID:/build/my_app_static" "$BUILD_DIR/$OUTPUT_NAME"
|
||||
docker rm "$CONTAINER_ID"
|
||||
|
||||
chmod +x "$BUILD_DIR/$OUTPUT_NAME"
|
||||
|
||||
echo "✓ Build complete: $BUILD_DIR/$OUTPUT_NAME"
|
||||
echo "✓ Size: $(du -h "$BUILD_DIR/$OUTPUT_NAME" | cut -f1)"
|
||||
|
||||
# Verify
|
||||
if ldd "$BUILD_DIR/$OUTPUT_NAME" 2>&1 | grep -q "not a dynamic executable"; then
|
||||
echo "✓ Binary is fully static"
|
||||
else
|
||||
echo "⚠ Warning: Binary may have dynamic dependencies"
|
||||
fi
|
||||
```
|
||||
|
||||
Make it executable:
|
||||
|
||||
```bash
|
||||
chmod +x build_static.sh
|
||||
./build_static.sh
|
||||
```
|
||||
|
||||
## Common Issues and Solutions
|
||||
|
||||
### Issue 1: Fortification Errors
|
||||
|
||||
**Error:**
|
||||
```
|
||||
undefined reference to '__snprintf_chk'
|
||||
undefined reference to '__fprintf_chk'
|
||||
```
|
||||
|
||||
**Cause**: GCC's `-O2` enables fortification by default, which uses glibc-specific functions.
|
||||
|
||||
**Solution**: Add these flags to **all** compilation commands:
|
||||
```bash
|
||||
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0
|
||||
```
|
||||
|
||||
This must be applied to:
|
||||
1. nostr_core_lib build.sh
|
||||
2. Your application compilation
|
||||
3. Any other libraries you build
|
||||
|
||||
### Issue 2: Missing Symbols from nostr_core_lib
|
||||
|
||||
**Error:**
|
||||
```
|
||||
undefined reference to 'nostr_create_event'
|
||||
undefined reference to 'nostr_sign_event'
|
||||
```
|
||||
|
||||
**Cause**: Required NIPs not included in nostr_core_lib build.
|
||||
|
||||
**Solution**: Add missing NIPs:
|
||||
```bash
|
||||
./build.sh --nips=1,6,19 # Add the NIPs you need
|
||||
```
|
||||
|
||||
### Issue 3: Docker Permission Denied
|
||||
|
||||
**Error:**
|
||||
```
|
||||
permission denied while trying to connect to the Docker daemon socket
|
||||
```
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
sudo usermod -aG docker $USER
|
||||
newgrp docker # Or logout and login
|
||||
```
|
||||
|
||||
### Issue 4: Binary Won't Run on Target System
|
||||
|
||||
**Checks**:
|
||||
```bash
|
||||
# 1. Verify it's static
|
||||
ldd my_app_static # Should show "not a dynamic executable"
|
||||
|
||||
# 2. Check architecture
|
||||
file my_app_static # Should match target system
|
||||
|
||||
# 3. Test on different distributions
|
||||
docker run --rm -v $(pwd):/app alpine:latest /app/my_app_static --version
|
||||
docker run --rm -v $(pwd):/app ubuntu:latest /app/my_app_static --version
|
||||
```
|
||||
|
||||
## Project Structure Example
|
||||
|
||||
Organize your project for easy static builds:
|
||||
|
||||
```
|
||||
my-nostr-app/
|
||||
├── src/
|
||||
│ ├── main.c
|
||||
│ ├── handlers.c
|
||||
│ └── utils.c
|
||||
├── nostr_core_lib/ # Git submodule
|
||||
├── nostr_login_lite/ # Git submodule (if used)
|
||||
├── Dockerfile.static # Static build Dockerfile
|
||||
├── build_static.sh # Build script
|
||||
├── Makefile # Regular build
|
||||
└── README.md
|
||||
```
|
||||
|
||||
### Makefile Integration
|
||||
|
||||
Add static build targets to your Makefile:
|
||||
|
||||
```makefile
|
||||
# Regular dynamic build
|
||||
all: my_app
|
||||
|
||||
my_app: src/*.c
|
||||
gcc -O2 src/*.c -o my_app \
|
||||
nostr_core_lib/libnostr_core_x64.a \
|
||||
-lssl -lcrypto -lsecp256k1 -lz -lpthread -lm
|
||||
|
||||
# Static MUSL build via Docker
|
||||
static:
|
||||
./build_static.sh
|
||||
|
||||
# Clean
|
||||
clean:
|
||||
rm -f my_app build/my_app_static_*
|
||||
|
||||
.PHONY: all static clean
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
### Single Binary Deployment
|
||||
|
||||
```bash
|
||||
# Copy to server
|
||||
scp build/my_app_static_x86_64 user@server:/opt/my-app/
|
||||
|
||||
# Run (no dependencies needed!)
|
||||
ssh user@server
|
||||
/opt/my-app/my_app_static_x86_64
|
||||
```
|
||||
|
||||
### SystemD Service
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=My Nostr Application
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=myapp
|
||||
WorkingDirectory=/opt/my-app
|
||||
ExecStart=/opt/my-app/my_app_static_x86_64
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
### Docker Container (Minimal)
|
||||
|
||||
```dockerfile
|
||||
FROM scratch
|
||||
COPY my_app_static_x86_64 /app
|
||||
ENTRYPOINT ["/app"]
|
||||
```
|
||||
|
||||
Build and run:
|
||||
```bash
|
||||
docker build -t my-app:latest .
|
||||
docker run --rm my-app:latest --help
|
||||
```
|
||||
|
||||
## Reusing c-relay-pg Files
|
||||
|
||||
You can directly copy these files from c-relay-pg:
|
||||
|
||||
### 1. Dockerfile.alpine-musl
|
||||
```bash
|
||||
cp /path/to/c-relay-pg/Dockerfile.alpine-musl ./Dockerfile.static
|
||||
```
|
||||
|
||||
Then customize:
|
||||
- Change binary name (line 125)
|
||||
- Adjust source files (line 122-124)
|
||||
- Modify include paths (line 120-121)
|
||||
|
||||
### 2. build_static.sh
|
||||
```bash
|
||||
cp /path/to/c-relay-pg/build_static.sh ./
|
||||
```
|
||||
|
||||
Then customize:
|
||||
- Change `OUTPUT_NAME` variable (lines 66, 70)
|
||||
- Update Docker image name (line 98)
|
||||
- Modify verification commands (lines 180-184)
|
||||
|
||||
### 3. .dockerignore (Optional)
|
||||
```bash
|
||||
cp /path/to/c-relay-pg/.dockerignore ./
|
||||
```
|
||||
|
||||
Helps speed up Docker builds by excluding unnecessary files.
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Version Control**: Commit your Dockerfile and build script
|
||||
2. **Tag Builds**: Include git commit hash in binary version
|
||||
3. **Test Thoroughly**: Verify on multiple distributions
|
||||
4. **Document Dependencies**: List required NIPs and libraries
|
||||
5. **Automate**: Use CI/CD to build on every commit
|
||||
6. **Archive Binaries**: Keep old versions for rollback
|
||||
|
||||
## Performance Comparison
|
||||
|
||||
| Metric | MUSL Static | glibc Dynamic |
|
||||
|--------|-------------|---------------|
|
||||
| Binary Size | 7-10 MB | 2-3 MB + libs |
|
||||
| Startup Time | ~50ms | ~40ms |
|
||||
| Memory Usage | Similar | Similar |
|
||||
| Portability | ✓ Universal | ✗ System-dependent |
|
||||
| Deployment | Single file | Binary + libraries |
|
||||
|
||||
## References
|
||||
|
||||
- [MUSL libc](https://musl.libc.org/)
|
||||
- [Alpine Linux](https://alpinelinux.org/)
|
||||
- [nostr_core_lib](https://github.com/chebizarro/nostr_core_lib)
|
||||
- [Static Linking Best Practices](https://www.musl-libc.org/faq.html)
|
||||
- [c-relay-pg Implementation](./docs/musl_static_build.md)
|
||||
|
||||
## Example: Minimal Nostr Client
|
||||
|
||||
Here's a complete example of building a minimal Nostr client:
|
||||
|
||||
```c
|
||||
// minimal_client.c
|
||||
#include "nostr_core/nostr_core.h"
|
||||
#include <stdio.h>
|
||||
|
||||
int main() {
|
||||
// Generate keypair
|
||||
char nsec[64], npub[64];
|
||||
nostr_generate_keypair(nsec, npub);
|
||||
|
||||
printf("Generated keypair:\n");
|
||||
printf("Private key (nsec): %s\n", nsec);
|
||||
printf("Public key (npub): %s\n", npub);
|
||||
|
||||
// Create event
|
||||
cJSON *event = nostr_create_event(1, "Hello, Nostr!", NULL);
|
||||
nostr_sign_event(event, nsec);
|
||||
|
||||
char *json = cJSON_Print(event);
|
||||
printf("\nSigned event:\n%s\n", json);
|
||||
|
||||
free(json);
|
||||
cJSON_Delete(event);
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
**Dockerfile.static:**
|
||||
```dockerfile
|
||||
FROM alpine:3.19 AS builder
|
||||
RUN apk add --no-cache build-base musl-dev git autoconf automake libtool \
|
||||
openssl-dev openssl-libs-static zlib-dev zlib-static
|
||||
|
||||
WORKDIR /build
|
||||
|
||||
# Build secp256k1
|
||||
RUN cd /tmp && git clone https://github.com/bitcoin-core/secp256k1.git && \
|
||||
cd secp256k1 && ./autogen.sh && \
|
||||
./configure --enable-static --disable-shared --prefix=/usr CFLAGS="-fPIC" && \
|
||||
make -j$(nproc) && make install
|
||||
|
||||
# Copy and build nostr_core_lib
|
||||
COPY nostr_core_lib /build/nostr_core_lib/
|
||||
RUN cd nostr_core_lib && \
|
||||
sed -i 's/CFLAGS="-Wall/CFLAGS="-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -Wall/' build.sh && \
|
||||
./build.sh --nips=1,6,19
|
||||
|
||||
# Build application
|
||||
COPY minimal_client.c /build/
|
||||
RUN gcc -static -O2 -Wall -std=c99 \
|
||||
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 \
|
||||
-Inostr_core_lib -Inostr_core_lib/nostr_core -Inostr_core_lib/cjson \
|
||||
minimal_client.c -o /build/minimal_client_static \
|
||||
nostr_core_lib/libnostr_core_x64.a \
|
||||
-lssl -lcrypto -lsecp256k1 -lz -lpthread -lm -ldl && \
|
||||
strip /build/minimal_client_static
|
||||
|
||||
FROM scratch
|
||||
COPY --from=builder /build/minimal_client_static /minimal_client_static
|
||||
```
|
||||
|
||||
**Build and run:**
|
||||
```bash
|
||||
docker build -f Dockerfile.static -t minimal-client .
|
||||
docker create --name temp minimal-client
|
||||
docker cp temp:/minimal_client_static ./
|
||||
docker rm temp
|
||||
|
||||
./minimal_client_static
|
||||
```
|
||||
|
||||
## Conclusion
|
||||
|
||||
Static MUSL binaries provide the best portability for C applications. While they're slightly larger than dynamic binaries, the benefits of zero dependencies and universal compatibility make them ideal for:
|
||||
|
||||
- Server deployments across different Linux distributions
|
||||
- Embedded systems and IoT devices
|
||||
- Docker containers (FROM scratch)
|
||||
- Distribution to users without dependency management
|
||||
- Long-term archival and reproducibility
|
||||
|
||||
Follow this guide to create portable, self-contained binaries for your Nostr applications!
|
||||
+238
@@ -1261,3 +1261,241 @@ body.dark-mode .sql-results-table tbody tr:nth-child(even) {
|
||||
.header-title.clickable:hover {
|
||||
opacity: 0.8;
|
||||
}
|
||||
|
||||
/* ================================
|
||||
SUBSCRIPTION TABLE COLLAPSIBLE GROUPS
|
||||
================================ */
|
||||
|
||||
/* Subscription group header styles */
|
||||
.subscription-group-header {
|
||||
|
||||
font-weight: 500;
|
||||
|
||||
cursor: pointer;
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
.subscription-group-header:hover {
|
||||
background-color: var(--secondary-color);
|
||||
}
|
||||
|
||||
.expand-icon {
|
||||
display: inline-block;
|
||||
width: 20px;
|
||||
transition: transform 0.2s ease;
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
/* Detail row styles */
|
||||
.subscription-detail-row {
|
||||
/* background-color: var(--secondary-color); */
|
||||
}
|
||||
|
||||
/* ================================
|
||||
WEB OF TRUST (WoT) STYLES
|
||||
================================ */
|
||||
|
||||
.wot-status-row, .wot-stats-row {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: 8px 0;
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.wot-indicator {
|
||||
padding: 2px 10px;
|
||||
border-radius: 4px;
|
||||
font-weight: bold;
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
.wot-indicator.wot-found { background: #28a745; color: white; }
|
||||
.wot-indicator.wot-missing { background: #dc3545; color: white; }
|
||||
.wot-indicator.wot-unknown { background: #6c757d; color: white; }
|
||||
|
||||
.wot-level-selector { padding: 10px 0; }
|
||||
.wot-level-selector label { display: block; margin-bottom: 5px; font-weight: bold; }
|
||||
|
||||
.wot-level-btn { min-width: 100px; }
|
||||
.wot-level-btn.active {
|
||||
background: var(--accent-color, #ff0000);
|
||||
color: white;
|
||||
border-color: var(--accent-color, #ff0000);
|
||||
}
|
||||
|
||||
.wot-level-description {
|
||||
font-size: 12px;
|
||||
color: var(--primary-color);
|
||||
margin-top: 5px;
|
||||
font-style: italic;
|
||||
opacity: 0.8;
|
||||
}
|
||||
|
||||
/* Dark mode adjustments for WoT */
|
||||
body.dark-mode .wot-indicator.wot-found { background: #28a745; }
|
||||
body.dark-mode .wot-indicator.wot-missing { background: #dc3545; }
|
||||
body.dark-mode .wot-indicator.wot-unknown { background: #6c757d; }
|
||||
|
||||
/* ================================
|
||||
ADMIN ACCESS GATE STYLES
|
||||
================================ */
|
||||
|
||||
/* Access Denied Overlay */
|
||||
.access-denied-overlay {
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: 0;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
background: rgba(0, 0, 0, 0.85);
|
||||
z-index: 9999;
|
||||
display: none;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.access-denied-content {
|
||||
background: var(--card-bg);
|
||||
border: 2px solid #dc3545;
|
||||
border-radius: 12px;
|
||||
padding: 40px 60px;
|
||||
text-align: center;
|
||||
max-width: 500px;
|
||||
box-shadow: 0 10px 40px rgba(220, 53, 69, 0.3);
|
||||
}
|
||||
|
||||
.access-denied-icon {
|
||||
font-size: 64px;
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
|
||||
.access-denied-content h2 {
|
||||
color: #dc3545;
|
||||
font-size: 32px;
|
||||
margin-bottom: 20px;
|
||||
letter-spacing: 2px;
|
||||
}
|
||||
|
||||
.access-denied-message {
|
||||
font-size: 16px;
|
||||
color: var(--primary-color);
|
||||
margin-bottom: 10px;
|
||||
line-height: 1.5;
|
||||
}
|
||||
|
||||
.access-denied-submessage {
|
||||
font-size: 14px;
|
||||
color: var(--muted-color);
|
||||
margin-bottom: 30px;
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
.access-denied-logout-btn {
|
||||
background: #dc3545;
|
||||
color: white;
|
||||
border: none;
|
||||
padding: 12px 40px;
|
||||
font-size: 16px;
|
||||
font-weight: bold;
|
||||
border-radius: 6px;
|
||||
cursor: pointer;
|
||||
transition: all 0.3s ease;
|
||||
}
|
||||
|
||||
.access-denied-logout-btn:hover {
|
||||
background: #c82333;
|
||||
transform: translateY(-2px);
|
||||
box-shadow: 0 4px 12px rgba(220, 53, 69, 0.4);
|
||||
}
|
||||
|
||||
/* Admin Verification Loading Overlay */
|
||||
.admin-verification-overlay {
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: 0;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
background: rgba(0, 0, 0, 0.8);
|
||||
z-index: 9998;
|
||||
display: none;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.admin-verification-content {
|
||||
background: var(--card-bg);
|
||||
border: 1px solid var(--border-color);
|
||||
border-radius: 12px;
|
||||
padding: 40px 60px;
|
||||
text-align: center;
|
||||
max-width: 450px;
|
||||
}
|
||||
|
||||
.admin-verification-content h3 {
|
||||
color: var(--accent-color);
|
||||
font-size: 20px;
|
||||
margin-bottom: 15px;
|
||||
}
|
||||
|
||||
.admin-verification-content p {
|
||||
color: var(--muted-color);
|
||||
font-size: 14px;
|
||||
margin-top: 15px;
|
||||
}
|
||||
|
||||
/* Spinner Animation */
|
||||
.spinner {
|
||||
width: 50px;
|
||||
height: 50px;
|
||||
border: 4px solid var(--border-color);
|
||||
border-top: 4px solid var(--accent-color);
|
||||
border-radius: 50%;
|
||||
animation: spin 1s linear infinite;
|
||||
margin: 0 auto 20px;
|
||||
}
|
||||
|
||||
@keyframes spin {
|
||||
0% { transform: rotate(0deg); }
|
||||
100% { transform: rotate(360deg); }
|
||||
}
|
||||
|
||||
/* Dark mode adjustments */
|
||||
body.dark-mode .access-denied-content {
|
||||
background: var(--card-bg);
|
||||
}
|
||||
|
||||
body.dark-mode .admin-verification-content {
|
||||
background: var(--card-bg);
|
||||
}
|
||||
|
||||
.subscription-detail-row:hover {
|
||||
background-color: var(--muted-color);
|
||||
}
|
||||
|
||||
/* Detail row cell styles */
|
||||
.subscription-detail-prefix {
|
||||
padding-left: 30px;
|
||||
font-family: 'Courier New', monospace;
|
||||
font-size: 11px;
|
||||
color: var(--muted-color);
|
||||
}
|
||||
|
||||
.subscription-detail-id {
|
||||
font-family: 'Courier New', monospace;
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
|
||||
/* ================================
|
||||
IP BANS TABLE - compact rows
|
||||
================================ */
|
||||
#ip-bans-table td, #ip-bans-table th {
|
||||
padding: 4px 8px;
|
||||
line-height: 1.3;
|
||||
font-size: 13px;
|
||||
}
|
||||
#ip-bans-table button {
|
||||
padding: 2px 8px;
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
+199
-25
@@ -4,7 +4,7 @@
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>C-Relay Admin</title>
|
||||
<title>C-Relay-PG Admin</title>
|
||||
<link rel="stylesheet" href="/api/index.css">
|
||||
</head>
|
||||
|
||||
@@ -16,6 +16,7 @@
|
||||
<li><button class="nav-item" data-page="subscriptions">Subscriptions</button></li>
|
||||
<li><button class="nav-item" data-page="configuration">Configuration</button></li>
|
||||
<li><button class="nav-item" data-page="authorization">Authorization</button></li>
|
||||
<li><button class="nav-item" data-page="ip-bans">IP BANS</button></li>
|
||||
<li><button class="nav-item" data-page="relay-events">Relay Events</button></li>
|
||||
<li><button class="nav-item" data-page="dm">DM</button></li>
|
||||
<li><button class="nav-item" data-page="database">Database Query</button></li>
|
||||
@@ -41,7 +42,7 @@
|
||||
<span class="relay-letter" data-letter="Y">Y</span>
|
||||
</div>
|
||||
<div class="relay-info">
|
||||
<div id="relay-name" class="relay-name">C-Relay</div>
|
||||
<div id="relay-name" class="relay-name">C-Relay-PG</div>
|
||||
<div id="relay-description" class="relay-description">Loading...</div>
|
||||
<div id="relay-pubkey-container" class="relay-pubkey-container">
|
||||
<div id="relay-pubkey" class="relay-pubkey">Loading...</div>
|
||||
@@ -67,6 +68,17 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Access Denied Overlay (shown when non-admin user logs in) -->
|
||||
<div id="access-denied-overlay" class="access-denied-overlay" style="display: none;">
|
||||
<div class="access-denied-content">
|
||||
<div class="access-denied-icon">⛔</div>
|
||||
<h2>ACCESS DENIED</h2>
|
||||
<p class="access-denied-message">This interface is restricted to the relay administrator.</p>
|
||||
<p class="access-denied-submessage">The logged-in account does not have admin privileges.</p>
|
||||
<button type="button" class="access-denied-logout-btn" onclick="logout()">LOGOUT</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- DATABASE STATISTICS Section -->
|
||||
<!-- Subscribe to kind 24567 events to receive real-time monitoring data -->
|
||||
<div class="section flex-section" id="databaseStatisticsSection" style="display: none;">
|
||||
@@ -100,6 +112,10 @@
|
||||
<td>Process ID</td>
|
||||
<td id="process-id">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>WebSocket Connections</td>
|
||||
<td id="websocket-connections">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Active Subscriptions</td>
|
||||
<td id="active-subscriptions">-</td>
|
||||
@@ -108,14 +124,14 @@
|
||||
<td>Memory Usage</td>
|
||||
<td id="memory-usage">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>CPU Core</td>
|
||||
<td id="cpu-core">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>CPU Usage</td>
|
||||
<td id="cpu-usage">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>CPU Core</td>
|
||||
<td id="cpu-core">-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Oldest Event</td>
|
||||
<td id="oldest-event">-</td>
|
||||
@@ -212,18 +228,9 @@
|
||||
<div class="input-group">
|
||||
<div class="config-table-container">
|
||||
<table class="config-table" id="subscription-details-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Subscription ID</th>
|
||||
<th>Client IP</th>
|
||||
<th>WSI Pointer</th>
|
||||
<th>Duration</th>
|
||||
<th>Filters</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody id="subscription-details-table-body">
|
||||
<tr>
|
||||
<td colspan="5" style="text-align: center; font-style: italic;">No subscriptions active</td>
|
||||
<td colspan="4" style="text-align: center; font-style: italic;">No subscriptions active</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
@@ -260,6 +267,7 @@
|
||||
</div>
|
||||
|
||||
<!-- Auth Rules Management - Moved after configuration -->
|
||||
<!-- AUTH RULES MANAGEMENT SECTION -->
|
||||
<div class="section flex-section" id="authRulesSection" style="display: none;">
|
||||
<div class="section-header">
|
||||
AUTH RULES MANAGEMENT
|
||||
@@ -282,16 +290,11 @@ AUTH RULES MANAGEMENT
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<!-- Simplified Auth Rule Input Section -->
|
||||
<!-- Auth Rule Input Section -->
|
||||
<div id="authRuleInputSections" style="display: block;">
|
||||
|
||||
<!-- Combined Pubkey Auth Rule Section -->
|
||||
|
||||
|
||||
<div class="input-group">
|
||||
<label for="authRulePubkey">Pubkey (nsec or hex):</label>
|
||||
<input type="text" id="authRulePubkey" placeholder="nsec1... or 64-character hex pubkey">
|
||||
|
||||
<label for="authRulePubkey">Public Key (npub or hex):</label>
|
||||
<input type="text" id="authRulePubkey" placeholder="npub1... or 64-character hex pubkey">
|
||||
</div>
|
||||
<div id="whitelistWarning" class="warning-box" style="display: none;">
|
||||
<strong>⚠️ WARNING:</strong> Adding whitelist rules changes relay behavior to whitelist-only
|
||||
@@ -305,10 +308,51 @@ AUTH RULES MANAGEMENT
|
||||
BLACKLIST</button>
|
||||
<button type="button" id="refreshAuthRulesBtn">REFRESH</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<!-- WEB OF TRUST SECTION -->
|
||||
<div class="section flex-section" id="wotSection" style="display: none;">
|
||||
<div class="section-header">
|
||||
WEB OF TRUST
|
||||
</div>
|
||||
|
||||
<!-- Kind 3 Status Indicator -->
|
||||
<div id="wotKind3Status" class="wot-status-row">
|
||||
<span>Admin Contact List (kind 3):</span>
|
||||
<span id="wotKind3Indicator" class="wot-indicator wot-unknown">Checking...</span>
|
||||
</div>
|
||||
|
||||
<!-- WoT Level Selector -->
|
||||
<div class="wot-level-selector">
|
||||
<label>WoT Level:</label>
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="wotLevel0Btn" class="wot-level-btn" onclick="setWotLevel(0)">
|
||||
OFF
|
||||
</button>
|
||||
<button type="button" id="wotLevel1Btn" class="wot-level-btn" onclick="setWotLevel(1)">
|
||||
WRITE ONLY
|
||||
</button>
|
||||
<button type="button" id="wotLevel2Btn" class="wot-level-btn" onclick="setWotLevel(2)">
|
||||
FULL
|
||||
</button>
|
||||
</div>
|
||||
<div class="wot-level-description" id="wotLevelDescription">
|
||||
Level 0: Open relay — anyone can read and write
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- WoT Stats -->
|
||||
<div class="wot-stats-row">
|
||||
<span>Whitelisted Pubkeys:</span>
|
||||
<span id="wotWhitelistCount">—</span>
|
||||
</div>
|
||||
|
||||
<!-- Sync Button -->
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="wotSyncBtn" onclick="syncWot()">SYNC FROM KIND 3</button>
|
||||
<button type="button" id="wotRefreshBtn" onclick="loadWotStatus()">REFRESH STATUS</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@@ -343,12 +387,138 @@ AUTH RULES MANAGEMENT
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- IP BANS Section -->
|
||||
<div class="section" id="ipBansSection" style="display: none;">
|
||||
<div class="section-header">
|
||||
IP BAN MANAGEMENT
|
||||
</div>
|
||||
|
||||
<!-- Statistics Cards -->
|
||||
<div class="input-group">
|
||||
<div class="config-table-container">
|
||||
<table class="config-table" id="ip-bans-stats-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Total IPs Tracked</th>
|
||||
<th>Currently Banned</th>
|
||||
<th>Total Bans Issued</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td id="ip-bans-total">-</td>
|
||||
<td id="ip-bans-active">-</td>
|
||||
<td id="ip-bans-issued">-</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Add Ban Form -->
|
||||
<div class="input-group">
|
||||
<h3>Manually Ban IP Address</h3>
|
||||
<div class="form-group">
|
||||
<label for="ban-ip-input">IP Address:</label>
|
||||
<input type="text" id="ban-ip-input" placeholder="192.168.1.100">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="ban-duration-select">Ban Duration:</label>
|
||||
<select id="ban-duration-select">
|
||||
<option value="3600">1 Hour</option>
|
||||
<option value="86400" selected>24 Hours</option>
|
||||
<option value="604800">7 Days</option>
|
||||
<option value="2592000">30 Days</option>
|
||||
<option value="31536000">1 Year</option>
|
||||
<option value="999999999">Permanent</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="add-ban-btn">BAN IP</button>
|
||||
</div>
|
||||
<div id="add-ban-status" class="status-message"></div>
|
||||
</div>
|
||||
|
||||
<!-- Whitelist Management -->
|
||||
<div class="input-group">
|
||||
<h3>IP Whitelist (Never Banned)</h3>
|
||||
<p style="font-size:13px;opacity:0.8;">IPs in this list are never idle-banned. Comma-separated.</p>
|
||||
<div class="form-group">
|
||||
<label for="whitelist-ip-input">Add IP to Whitelist:</label>
|
||||
<input type="text" id="whitelist-ip-input" placeholder="103.81.231.220">
|
||||
</div>
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="add-whitelist-btn">ADD TO WHITELIST</button>
|
||||
</div>
|
||||
<div id="whitelist-status" class="status-message"></div>
|
||||
<div id="whitelist-current" style="margin-top:8px;font-size:13px;"></div>
|
||||
</div>
|
||||
|
||||
<!-- Filter Controls -->
|
||||
<div class="input-group">
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="ip-ban-filter-all" class="active">All IPs</button>
|
||||
<button type="button" id="ip-ban-filter-banned">Currently Banned</button>
|
||||
<button type="button" id="ip-ban-filter-expired">Expired</button>
|
||||
<button type="button" id="refresh-ip-bans-btn">REFRESH</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- IP Bans List -->
|
||||
<div class="input-group">
|
||||
<label>Banned IP Addresses:</label>
|
||||
<div class="config-table-container">
|
||||
<table class="config-table" id="ip-bans-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>IP Address</th>
|
||||
<th>Status</th>
|
||||
<th>Banned Until</th>
|
||||
<th>Failures</th>
|
||||
<th>Authed Successfully</th>
|
||||
<th>Connection Attempts</th>
|
||||
<th>Actions</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody id="ip-bans-tbody">
|
||||
<tr>
|
||||
<td colspan="7" style="text-align: center;">Click REFRESH to load IP bans</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- RELAY EVENTS Section -->
|
||||
<div class="section" id="relayEventsSection" style="display: none;">
|
||||
<div class="section-header">
|
||||
RELAY EVENTS MANAGEMENT
|
||||
</div>
|
||||
|
||||
<!-- Live Relay Event Feed -->
|
||||
<div class="input-group">
|
||||
<h3>Live Relay Event Feed (Normal Nostr Subscription)</h3>
|
||||
<div class="config-table-container">
|
||||
<table class="config-table" id="live-relay-events-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Time</th>
|
||||
<th>Kind</th>
|
||||
<th>Pubkey</th>
|
||||
<th>ID</th>
|
||||
<th>Content Preview</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody id="live-relay-events-table-body">
|
||||
<tr>
|
||||
<td colspan="5" style="text-align: center;">Waiting for live events...</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Kind 0: User Metadata -->
|
||||
<div class="input-group">
|
||||
<h3>Kind 0: User Metadata</h3>
|
||||
@@ -428,6 +598,10 @@ AUTH RULES MANAGEMENT
|
||||
<option value="event_kinds">Event Kinds Distribution</option>
|
||||
<option value="time_stats">Time-based Statistics</option>
|
||||
</optgroup>
|
||||
<optgroup label="IP Ban Queries">
|
||||
<option value="banned_ips_summary">Banned IPs Summary</option>
|
||||
<option value="banned_ips_list">All Banned IP Addresses</option>
|
||||
</optgroup>
|
||||
<optgroup label="Query History" id="history-group">
|
||||
<!-- Dynamically populated from localStorage -->
|
||||
</optgroup>
|
||||
|
||||
+1292
-187
File diff suppressed because it is too large
Load Diff
+95
-1
@@ -3297,8 +3297,18 @@ var NostrTools = (() => {
|
||||
this.enablePing = opts.enablePing;
|
||||
}
|
||||
async ensureRelay(url, params) {
|
||||
const rawUrl = url;
|
||||
const debugEnabled = typeof window !== "undefined" && !!window.__SIMPLE_POOL_DEBUG__;
|
||||
url = normalizeURL(url);
|
||||
let relay = this.relays.get(url);
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL ensureRelay START", {
|
||||
rawUrl,
|
||||
normalizedUrl: url,
|
||||
hadRelay: !!relay,
|
||||
relayMapKeysBefore: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
if (!relay) {
|
||||
relay = new AbstractRelay(url, {
|
||||
verifyEvent: this.trustedRelayURLs.has(url) ? alwaysTrue : this.verifyEvent,
|
||||
@@ -3306,13 +3316,38 @@ var NostrTools = (() => {
|
||||
enablePing: this.enablePing
|
||||
});
|
||||
relay.onclose = () => {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL ensureRelay onclose", {
|
||||
normalizedUrl: url,
|
||||
relayMapKeysBeforeDelete: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
this.relays.delete(url);
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL ensureRelay onclose complete", {
|
||||
normalizedUrl: url,
|
||||
relayMapKeysAfterDelete: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
};
|
||||
if (params?.connectionTimeout)
|
||||
relay.connectionTimeout = params.connectionTimeout;
|
||||
this.relays.set(url, relay);
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL ensureRelay created relay", {
|
||||
normalizedUrl: url,
|
||||
relayMapKeysAfterCreate: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
}
|
||||
await relay.connect();
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL ensureRelay connected", {
|
||||
normalizedUrl: url,
|
||||
relayConnected: relay.connected,
|
||||
relayMapKeysAfterConnect: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
return relay;
|
||||
}
|
||||
close(relays) {
|
||||
@@ -3334,16 +3369,26 @@ var NostrTools = (() => {
|
||||
}
|
||||
subscribeMany(relays, filters, params) {
|
||||
params.onauth = params.onauth || params.doauth;
|
||||
const debugEnabled = typeof window !== "undefined" && !!window.__SIMPLE_POOL_DEBUG__;
|
||||
const request = [];
|
||||
const uniqUrls = [];
|
||||
for (let i2 = 0; i2 < relays.length; i2++) {
|
||||
const url = normalizeURL(relays[i2]);
|
||||
if (uniqUrls.indexOf(url) === -1) {
|
||||
uniqUrls.push(url);
|
||||
for (let f2 = 0; f2 < filters.length; f2++) {
|
||||
request.push({ url, filter: filters[f2] });
|
||||
}
|
||||
}
|
||||
}
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL subscribeMany", {
|
||||
relaysInput: relays,
|
||||
uniqUrls,
|
||||
filtersCount: filters.length,
|
||||
requestsCount: request.length
|
||||
});
|
||||
}
|
||||
return this.subscribeMap(request, params);
|
||||
}
|
||||
subscribeMap(requests, params) {
|
||||
@@ -3391,14 +3436,31 @@ var NostrTools = (() => {
|
||||
_knownIds.add(id);
|
||||
return have;
|
||||
};
|
||||
const debugEnabled = typeof window !== "undefined" && !!window.__SIMPLE_POOL_DEBUG__;
|
||||
const allOpened = Promise.all(
|
||||
requests.map(async ({ url, filter }, i2) => {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL subscribeMap request", {
|
||||
index: i2,
|
||||
url,
|
||||
filterKinds: filter?.kinds,
|
||||
hasAuthorFilter: !!filter?.authors,
|
||||
hasPTagFilter: !!filter?.["#p"]
|
||||
});
|
||||
}
|
||||
let relay;
|
||||
try {
|
||||
relay = await this.ensureRelay(url, {
|
||||
connectionTimeout: params.maxWait ? Math.max(params.maxWait * 0.8, params.maxWait - 1e3) : void 0
|
||||
});
|
||||
} catch (err) {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL subscribeMap ensureRelay FAILED", {
|
||||
index: i2,
|
||||
url,
|
||||
error: err?.message || String(err)
|
||||
});
|
||||
}
|
||||
handleClose(i2, err?.message || String(err));
|
||||
return;
|
||||
}
|
||||
@@ -3480,18 +3542,50 @@ var NostrTools = (() => {
|
||||
return events[0] || null;
|
||||
}
|
||||
publish(relays, event, options) {
|
||||
return relays.map(normalizeURL).map(async (url, i2, arr) => {
|
||||
const debugEnabled = typeof window !== "undefined" && !!window.__SIMPLE_POOL_DEBUG__;
|
||||
const normalizedRelays = relays.map(normalizeURL);
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL publish START", {
|
||||
relaysInput: relays,
|
||||
normalizedRelays,
|
||||
eventKind: event?.kind,
|
||||
eventId: event?.id
|
||||
});
|
||||
}
|
||||
return normalizedRelays.map(async (url, i2, arr) => {
|
||||
if (arr.indexOf(url) !== i2) {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL publish duplicate URL", { url, index: i2 });
|
||||
}
|
||||
return Promise.reject("duplicate url");
|
||||
}
|
||||
let r = await this.ensureRelay(url);
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL publish ensured relay", {
|
||||
url,
|
||||
relayConnected: r?.connected,
|
||||
relayMapKeys: Array.from(this.relays.keys())
|
||||
});
|
||||
}
|
||||
return r.publish(event).catch(async (err) => {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL publish ERROR", {
|
||||
url,
|
||||
error: err?.message || String(err)
|
||||
});
|
||||
}
|
||||
if (err instanceof Error && err.message.startsWith("auth-required: ") && options?.onauth) {
|
||||
await r.auth(options.onauth);
|
||||
return r.publish(event);
|
||||
}
|
||||
throw err;
|
||||
}).then((reason) => {
|
||||
if (debugEnabled) {
|
||||
console.log("🔎 SIMPLE_POOL publish SUCCESS", {
|
||||
url,
|
||||
reason
|
||||
});
|
||||
}
|
||||
if (this.trackRelays) {
|
||||
let set = this.seenOn.get(event.id);
|
||||
if (!set) {
|
||||
|
||||
@@ -0,0 +1,813 @@
|
||||
<!DOCTYPE html>
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<html lang="en" dir="ltr">
|
||||
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<title>TEMPLATE</title>
|
||||
|
||||
<link rel="stylesheet" href="./css/client.css" />
|
||||
|
||||
<!-- Initialize theme BEFORE any components load -->
|
||||
<script>
|
||||
(function () {
|
||||
const savedTheme = localStorage.getItem('theme');
|
||||
if (savedTheme === 'dark') {
|
||||
document.documentElement.classList.add('dark-mode');
|
||||
if (document.body) {
|
||||
document.body.classList.add('dark-mode');
|
||||
}
|
||||
}
|
||||
})();
|
||||
</script>
|
||||
<link rel="shortcut icon" type="image/x-icon" href="./favicon/favicon-dots2.ico" />
|
||||
|
||||
<!-- SVG.js library (required by HamburgerMorphing) -->
|
||||
<script src="https://cdn.jsdelivr.net/npm/@svgdotjs/svg.js@3.0/dist/svg.min.js"></script>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<!-- ================================================================
|
||||
HAMBURGER BUTTON (Fixed, separate from header)
|
||||
================================================================
|
||||
The hamburger button is a fixed element outside the header
|
||||
to ensure it stays visible above the sidenav (z-index: 10 > 3).
|
||||
================================================================ -->
|
||||
<div id="divSvgHam" class="divHeaderButtons">
|
||||
<!-- HamburgerMorphing will be injected here -->
|
||||
</div>
|
||||
|
||||
<!-- ================================================================
|
||||
HEADER
|
||||
================================================================
|
||||
Standard header with title (center).
|
||||
================================================================ -->
|
||||
<div id="divHeader">
|
||||
<div id="divHeaderFlexLeft">
|
||||
<!-- Hamburger is now separate fixed element -->
|
||||
</div>
|
||||
|
||||
<div id="divHeaderFlexCenter">
|
||||
<div class="divHeaderText"></div>
|
||||
</div>
|
||||
|
||||
<div id="divHeaderFlexRight">
|
||||
<!-- No button in header right - logout is in sidenav footer -->
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- ================================================================
|
||||
BODY
|
||||
================================================================
|
||||
Main content area. Add your page-specific content here.
|
||||
================================================================ -->
|
||||
<div id="divBody">
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
<!-- ================================================================
|
||||
FOOTER
|
||||
================================================================
|
||||
Three-section footer layout:
|
||||
- Left: Relay status animations (HamburgerMorphing instances)
|
||||
- Center: General status information
|
||||
- Right: Additional information
|
||||
================================================================ -->
|
||||
<div id="divFooter">
|
||||
<div id="divFooterLeft" class="divFooterBox"></div>
|
||||
<div id="divFooterCenter" class="divFooterBox"></div>
|
||||
<div id="divFooterRight" class="divFooterBox"></div>
|
||||
<div id="divFooterBalance" class="divFooterBox">0 sats</div>
|
||||
</div>
|
||||
|
||||
<!-- ================================================================
|
||||
SIDENAV
|
||||
================================================================
|
||||
Slide-out navigation panel. Opens from left when hamburger clicked.
|
||||
Uses flexbox layout to pin version bar to bottom.
|
||||
Includes a version bar footer with theme toggle and logout buttons.
|
||||
================================================================ -->
|
||||
<div id="divSideNav">
|
||||
<div id="divSideNavHeader">
|
||||
<!-- No close button - use main hamburger to close -->
|
||||
</div>
|
||||
|
||||
<div id="divSideNavBody">
|
||||
<div id="divFiles"></div>
|
||||
</div>
|
||||
|
||||
<div id="divAiSection" class="sidenavSection">
|
||||
<div id="divAiSectionTitle" class="sidenavSectionTitle">AI</div>
|
||||
<div id="divAiList" class="sidenavSectionList">
|
||||
<div id="divAiProvidersList">No saved providers yet.</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<div id="divRelaySection">
|
||||
<div id="divRelaySectionTitle">
|
||||
リレー
|
||||
</div>
|
||||
<div id="divRelayList">
|
||||
Loading relays...
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="divBlossomSection">
|
||||
|
||||
<div id="divBlossomSectionTitle">ブロッサム</div>
|
||||
|
||||
<div id="divBlossomList">Loading blossom servers...</div>
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
<div id="divVersionBar">
|
||||
<span id="versionDisplay">v0.0.1</span>
|
||||
<div id="divVersionBarButtons">
|
||||
<button id="themeToggleButton" title="Toggle Dark/Light Mode">
|
||||
<div id="themeToggleHamburgerContainer"></div>
|
||||
</button>
|
||||
<button id="logoutButton" title="Logout">
|
||||
<div id="logoutHamburgerContainer"></div>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- ================================================================
|
||||
REQUIRED SCRIPTS
|
||||
================================================================
|
||||
These scripts must be loaded in this order:
|
||||
1. nostr.bundle.js - Nostr tools library
|
||||
2. nostr-lite.js - Authentication modal (nostr-login-lite)
|
||||
================================================================ -->
|
||||
<script src="./nostr.bundle.js"></script>
|
||||
<script src="./nostr-lite.js"></script>
|
||||
|
||||
<script type="module">
|
||||
/* ================================================================
|
||||
IMPORTS
|
||||
================================================================
|
||||
Import shared NDK functionality from init-ndk.mjs:
|
||||
- initNDKPage() - Initialize authentication and worker
|
||||
- getPubkey() - Get current user's pubkey
|
||||
- subscribe() - Create NDK subscriptions
|
||||
- publishEvent() - Publish events via NDK
|
||||
- disconnect() - Disconnect from worker
|
||||
- getRelayData() - Get relay connection data
|
||||
- getRelayStats() - Get relay activity statistics
|
||||
|
||||
Import HamburgerMorphing for animated icons
|
||||
================================================================ */
|
||||
import {
|
||||
initNDKPage,
|
||||
getPubkey, injectHeaderAvatar,
|
||||
subscribe,
|
||||
publishEvent,
|
||||
disconnect,
|
||||
getVersion,
|
||||
updateVersionDisplay,
|
||||
getUserSettings,
|
||||
patchUserSettings,
|
||||
onUserSettings
|
||||
} from './js/init-ndk.mjs';
|
||||
import { HamburgerMorphing } from "./hamburger_morphing/hamburger.mjs";
|
||||
import { initFooterRelayStatus, updateFooterRelayStatus, initSidenavRelaySection, updateSidenavRelaySection, setRelayActivityState } from './js/relay-ui.mjs';
|
||||
|
||||
import { initBlossomSection, updateBlossomSection } from './js/blossom-ui.mjs';
|
||||
|
||||
import { initAiSectionWithLocalConfig } from './js/ai-ui.mjs';
|
||||
// Version will be loaded asynchronously
|
||||
const versionInfo = await getVersion();
|
||||
const VERSION = versionInfo.VERSION;
|
||||
console.log(`[template.html ${VERSION}] Loading...`);
|
||||
|
||||
/* ================================================================
|
||||
GLOBAL VARIABLES
|
||||
================================================================
|
||||
Track state for hamburger menu, relay status, and theme.
|
||||
================================================================ */
|
||||
let updateIntervalId = null;
|
||||
let currentPubkey = null;
|
||||
|
||||
/*
|
||||
AUTH STATE MODEL (Template reference)
|
||||
------------------------------------------------------------------
|
||||
This template now demonstrates three auth modes for standalone pages:
|
||||
|
||||
- required (default):
|
||||
Behaves like existing pages: login is required immediately.
|
||||
|
||||
- optional:
|
||||
Page can render public/read-only data without login, but can still
|
||||
prompt login later for user actions (publish, settings, etc).
|
||||
|
||||
- none:
|
||||
Never auto-login on load (pure public page).
|
||||
|
||||
URL behavior in this template:
|
||||
- If ?auth=required|optional|none is present, it wins.
|
||||
- Otherwise, if URL includes ?npub=... or ?pubkey=..., mode defaults
|
||||
to optional because pages with explicit profile targets are commonly
|
||||
public-readable.
|
||||
- Otherwise, mode defaults to required.
|
||||
*/
|
||||
let isAuthenticated = false;
|
||||
let authMode = 'required';
|
||||
let authedPageInitialized = false;
|
||||
let relayActivityListenersBound = false;
|
||||
|
||||
// Hamburger menu
|
||||
let hamburgerInstance = null;
|
||||
let isNavOpen = false;
|
||||
|
||||
// Version bar buttons
|
||||
let logoutHamburger = null;
|
||||
let themeToggleHamburger = null;
|
||||
let isDarkMode = false;
|
||||
|
||||
// App-wide user settings (NIP-78 kind 30078, d:user-settings)
|
||||
let pageSettings = {};
|
||||
let unsubscribeUserSettings = null;
|
||||
|
||||
/* ================================================================
|
||||
DOM VARIABLES
|
||||
================================================================
|
||||
Cache DOM element references for better performance.
|
||||
================================================================ */
|
||||
const divBody = document.getElementById("divBody");
|
||||
const divSideNav = document.getElementById("divSideNav");
|
||||
const divSideNavBody = document.getElementById("divSideNavBody");
|
||||
const divFooterCenter = document.getElementById("divFooterCenter");
|
||||
const divFooterRight = document.getElementById("divFooterRight");
|
||||
|
||||
/* ================================================================
|
||||
HAMBURGER MENU
|
||||
================================================================
|
||||
Initialize and control the animated hamburger menu.
|
||||
================================================================ */
|
||||
function initHamburgerMenu() {
|
||||
hamburgerInstance = new HamburgerMorphing('#divSvgHam', {
|
||||
foreground: 'var(--primary-color)',
|
||||
background: 'var(--secondary-color)',
|
||||
hover: 'var(--accent-color)'
|
||||
});
|
||||
hamburgerInstance.animateTo('burger');
|
||||
}
|
||||
|
||||
/* ================================================================
|
||||
SIDENAV FUNCTIONS
|
||||
================================================================
|
||||
Open/close sidenav with hamburger morphing animation.
|
||||
================================================================ */
|
||||
function openNav() {
|
||||
divSideNav.style.zIndex = 3;
|
||||
divSideNav.style.width = "clamp(400px, 50vw, 600px)";
|
||||
isNavOpen = true;
|
||||
if (hamburgerInstance) {
|
||||
hamburgerInstance.animateTo('arrow_left');
|
||||
}
|
||||
|
||||
|
||||
// Initialize version bar buttons when sidenav opens (lazy load)
|
||||
if (!logoutHamburger) {
|
||||
logoutHamburger = new HamburgerMorphing('#logoutHamburgerContainer', {
|
||||
size: 24,
|
||||
foreground: 'var(--primary-color)',
|
||||
background: 'var(--secondary-color)',
|
||||
hover: 'var(--accent-color)'
|
||||
});
|
||||
logoutHamburger.animateTo('x');
|
||||
}
|
||||
|
||||
if (!themeToggleHamburger) {
|
||||
themeToggleHamburger = new HamburgerMorphing('#themeToggleHamburgerContainer', {
|
||||
size: 24,
|
||||
foreground: 'var(--primary-color)',
|
||||
background: 'var(--secondary-color)',
|
||||
hover: 'var(--accent-color)'
|
||||
});
|
||||
|
||||
// Determine current theme
|
||||
const savedTheme = localStorage.getItem('theme');
|
||||
isDarkMode = savedTheme === 'dark' || document.body.classList.contains('dark-mode');
|
||||
const initialShape = isDarkMode ? 'moon' : 'circle';
|
||||
themeToggleHamburger.animateTo(initialShape);
|
||||
}
|
||||
}
|
||||
|
||||
function closeNav() {
|
||||
divSideNav.style.width = "0vw";
|
||||
divSideNav.style.zIndex = -1;
|
||||
isNavOpen = false;
|
||||
if (hamburgerInstance) {
|
||||
hamburgerInstance.animateTo('burger');
|
||||
}
|
||||
}
|
||||
|
||||
function toggleNav() {
|
||||
if (isNavOpen) {
|
||||
closeNav();
|
||||
} else {
|
||||
openNav();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/* ================================================================
|
||||
AUTH MODE HELPERS
|
||||
================================================================
|
||||
These functions are meant as reusable guidance for future pages.
|
||||
================================================================ */
|
||||
function hasTargetPubkeyInUrl() {
|
||||
const params = new URLSearchParams(window.location.search || '');
|
||||
const npub = String(params.get('npub') || '').trim();
|
||||
const pubkey = String(params.get('pubkey') || '').trim();
|
||||
return Boolean(npub || pubkey);
|
||||
}
|
||||
|
||||
function resolveAuthModeFromUrl() {
|
||||
const params = new URLSearchParams(window.location.search || '');
|
||||
const explicitAuth = String(params.get('auth') || '').trim().toLowerCase();
|
||||
if (explicitAuth === 'required' || explicitAuth === 'optional' || explicitAuth === 'none') {
|
||||
return explicitAuth;
|
||||
}
|
||||
|
||||
// Convention: explicit target profiles are public-readable by default.
|
||||
if (hasTargetPubkeyInUrl()) {
|
||||
return 'optional';
|
||||
}
|
||||
|
||||
return 'required';
|
||||
}
|
||||
|
||||
function isAuthRequiredError(error) {
|
||||
const message = String(error?.message || error || '').toLowerCase();
|
||||
return message.includes('authentication required');
|
||||
}
|
||||
|
||||
async function initializeAuthentication(mode) {
|
||||
// required: existing behavior, throw if auth fails.
|
||||
if (mode === 'required') {
|
||||
await initNDKPage();
|
||||
currentPubkey = await getPubkey();
|
||||
isAuthenticated = true;
|
||||
return;
|
||||
}
|
||||
|
||||
// none: public page, no login attempt on load.
|
||||
if (mode === 'none') {
|
||||
isAuthenticated = false;
|
||||
currentPubkey = null;
|
||||
return;
|
||||
}
|
||||
|
||||
// optional: try silent/normal init; if auth required, continue public.
|
||||
try {
|
||||
await initNDKPage();
|
||||
currentPubkey = await getPubkey();
|
||||
isAuthenticated = true;
|
||||
} catch (error) {
|
||||
if (isAuthRequiredError(error)) {
|
||||
console.log('[template.html] Optional auth mode: continuing unauthenticated');
|
||||
isAuthenticated = false;
|
||||
currentPubkey = null;
|
||||
return;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async function initializeAuthenticatedPageFeatures() {
|
||||
if (!isAuthenticated || authedPageInitialized) return;
|
||||
|
||||
await injectHeaderAvatar(currentPubkey);
|
||||
console.log('[template.html] Authenticated as:', currentPubkey);
|
||||
|
||||
// Hydrate app-wide user settings for this page
|
||||
try {
|
||||
pageSettings = await getUserSettings();
|
||||
} catch (error) {
|
||||
console.warn('[template.html] getUserSettings failed:', error);
|
||||
pageSettings = {};
|
||||
}
|
||||
|
||||
// Subscribe to live user settings updates (cross-tab + publish echoes)
|
||||
if (!unsubscribeUserSettings) {
|
||||
unsubscribeUserSettings = onUserSettings((settings) => {
|
||||
pageSettings = settings || {};
|
||||
// TODO: Re-render page-specific UI from pageSettings here.
|
||||
});
|
||||
}
|
||||
|
||||
// Initialize relay-dependent UI only once authenticated.
|
||||
initFooterRelayStatus();
|
||||
initSidenavRelaySection();
|
||||
await initBlossomSection();
|
||||
initAiSectionWithLocalConfig();
|
||||
await UpdateFooter();
|
||||
|
||||
if (!updateIntervalId) {
|
||||
updateIntervalId = setInterval(UpdateFooter, 1000);
|
||||
}
|
||||
|
||||
// Relay activity listeners only matter after worker init/auth.
|
||||
if (!relayActivityListenersBound) {
|
||||
window.addEventListener('ndkRelayActivity', (event) => {
|
||||
const { relayUrl, activity, stats } = event.detail;
|
||||
console.log(`[template.html] Relay activity: ${relayUrl} - ${activity}`, stats);
|
||||
setRelayActivityState(relayUrl, activity);
|
||||
});
|
||||
|
||||
window.addEventListener('message', (event) => {
|
||||
if (event.data && event.data.type === 'relayActivity') {
|
||||
const { relayUrl, activity } = event.data;
|
||||
console.log(`[template.html] Relay activity: ${relayUrl} - ${activity}`);
|
||||
setRelayActivityState(relayUrl, activity);
|
||||
}
|
||||
});
|
||||
|
||||
relayActivityListenersBound = true;
|
||||
}
|
||||
|
||||
authedPageInitialized = true;
|
||||
}
|
||||
|
||||
async function promptLoginIfNeeded() {
|
||||
if (isAuthenticated) return true;
|
||||
|
||||
await initNDKPage();
|
||||
currentPubkey = await getPubkey();
|
||||
isAuthenticated = true;
|
||||
await initializeAuthenticatedPageFeatures();
|
||||
return true;
|
||||
}
|
||||
|
||||
/* ================================================================
|
||||
UPDATE FOOTER
|
||||
================================================================
|
||||
Update footer sections with relay status, pubkey, and other info.
|
||||
Called periodically by update loop.
|
||||
================================================================ */
|
||||
const UpdateFooter = async () => {
|
||||
|
||||
try {
|
||||
// Update relay status visuals in footer and sidenav
|
||||
await updateFooterRelayStatus();
|
||||
await updateSidenavRelaySection();
|
||||
|
||||
await updateBlossomSection();
|
||||
// Clear center and right sections
|
||||
divFooterCenter.innerHTML = '';
|
||||
divFooterRight.innerHTML = '';
|
||||
} catch (error) {
|
||||
console.error('[template.html] Error updating footer:', error);
|
||||
}
|
||||
};
|
||||
|
||||
/* ================================================================
|
||||
LOGOUT
|
||||
================================================================
|
||||
Complete logout process:
|
||||
1. Stop update loop
|
||||
2. Disconnect from NDK worker
|
||||
3. Logout from nostr-login-lite
|
||||
4. Clear all storage (localStorage, sessionStorage, IndexedDB)
|
||||
5. Reload page
|
||||
================================================================ */
|
||||
const Logout = async () => {
|
||||
console.log("[template.html] Starting logout process...");
|
||||
|
||||
// Stop the update loop
|
||||
if (updateIntervalId) {
|
||||
clearInterval(updateIntervalId);
|
||||
updateIntervalId = null;
|
||||
}
|
||||
|
||||
// Disconnect from worker
|
||||
disconnect();
|
||||
|
||||
// Logout from nostr-login-lite
|
||||
if (window.NOSTR_LOGIN_LITE && window.NOSTR_LOGIN_LITE.logout) {
|
||||
await window.NOSTR_LOGIN_LITE.logout();
|
||||
}
|
||||
|
||||
// Clear all storage
|
||||
localStorage.clear();
|
||||
sessionStorage.clear();
|
||||
|
||||
// Clear IndexedDB
|
||||
if (window.indexedDB) {
|
||||
const databases = await window.indexedDB.databases();
|
||||
for (const db of databases) {
|
||||
if (db.name) {
|
||||
window.indexedDB.deleteDatabase(db.name);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
console.log("[template.html] Logged out, reloading page");
|
||||
location.reload(true);
|
||||
};
|
||||
|
||||
/* ================================================================
|
||||
EVENT LISTENERS
|
||||
================================================================
|
||||
Wire up UI interactions.
|
||||
Main hamburger button click handler is set up in main() after initialization.
|
||||
================================================================ */
|
||||
|
||||
/* ================================================================
|
||||
SUBSCRIPTION EXAMPLE
|
||||
================================================================
|
||||
Example of how to subscribe to Nostr events:
|
||||
|
||||
const sub = subscribe(
|
||||
{ kinds: [1], authors: [pubkey], limit: 10 },
|
||||
{ closeOnEose: false, cacheUsage: 'CACHE_FIRST' }
|
||||
);
|
||||
|
||||
Cache usage options:
|
||||
- 'CACHE_FIRST' - Check cache first, then relays
|
||||
- 'ONLY_RELAY' - Only query relays
|
||||
- 'ONLY_CACHE' - Only query cache
|
||||
- 'PARALLEL' - Query cache and relays simultaneously
|
||||
|
||||
Listen for events via window events:
|
||||
window.addEventListener('ndkEvent', (event) => {
|
||||
const evt = event.detail;
|
||||
console.log('Received event:', evt);
|
||||
});
|
||||
================================================================ */
|
||||
|
||||
/* ================================================================
|
||||
PUBLISH EXAMPLE
|
||||
================================================================
|
||||
Example of how to publish a Nostr event:
|
||||
|
||||
const event = {
|
||||
created_at: Math.floor(Date.now() / 1000),
|
||||
kind: 1,
|
||||
tags: [],
|
||||
content: "Hello, Nostr!"
|
||||
};
|
||||
|
||||
try {
|
||||
const result = await publishEvent(event);
|
||||
console.log("✅ Published to:", result.relayResults.successful);
|
||||
console.log("❌ Failed:", result.relayResults.failed);
|
||||
console.log("Total relays:", result.totalRelays);
|
||||
} catch (error) {
|
||||
console.error("Publish error:", error);
|
||||
}
|
||||
|
||||
Note: Events are automatically signed by the NDK worker using
|
||||
the message-based signer (which calls window.nostr.signEvent).
|
||||
================================================================ */
|
||||
|
||||
/* ================================================================
|
||||
USER SETTINGS EXAMPLE (NIP-78)
|
||||
================================================================
|
||||
Read/subscribe/write helper pattern for all pages:
|
||||
|
||||
// Read latest merged settings (cache + relay hydrated by worker)
|
||||
const settings = await getUserSettings();
|
||||
|
||||
// Subscribe to cross-tab updates
|
||||
const unsubscribe = onUserSettings((nextSettings) => {
|
||||
// Re-render page from nextSettings
|
||||
});
|
||||
|
||||
// Patch only your feature namespace
|
||||
await patchUserSettings({
|
||||
myFeature: {
|
||||
someFlag: true
|
||||
}
|
||||
});
|
||||
|
||||
// On page teardown (if applicable)
|
||||
// unsubscribe();
|
||||
================================================================ */
|
||||
|
||||
/* ================================================================
|
||||
INITIALIZATION
|
||||
================================================================
|
||||
Main initialization sequence:
|
||||
1. Initialize hamburger menu
|
||||
2. Set up hamburger click handler
|
||||
3. Resolve auth mode from URL/query policy
|
||||
4. Initialize authentication based on mode
|
||||
5. Initialize authenticated-only features (if signed in)
|
||||
6. Set up version bar button listeners
|
||||
7. Restore sidenav state
|
||||
8. Update version display
|
||||
|
||||
Notes:
|
||||
- required mode = existing behavior (prompt login on load)
|
||||
- optional mode = allow public load, login later on demand
|
||||
- none mode = no auto-login on load
|
||||
================================================================ */
|
||||
(async function main() {
|
||||
console.log("[template.html] Starting initialization...");
|
||||
|
||||
try {
|
||||
// Initialize hamburger menu first
|
||||
initHamburgerMenu();
|
||||
|
||||
// Add click handler to hamburger
|
||||
const divSvgHam = document.getElementById('divSvgHam');
|
||||
if (divSvgHam) {
|
||||
divSvgHam.addEventListener('click', toggleNav);
|
||||
}
|
||||
|
||||
// Initialize version bar buttons
|
||||
const themeToggleButton = document.getElementById('themeToggleButton');
|
||||
const logoutButton = document.getElementById('logoutButton');
|
||||
|
||||
if (themeToggleButton) {
|
||||
themeToggleButton.addEventListener('click', () => {
|
||||
isDarkMode = !isDarkMode;
|
||||
if (isDarkMode) {
|
||||
localStorage.setItem('theme', 'dark');
|
||||
} else {
|
||||
localStorage.setItem('theme', 'light');
|
||||
}
|
||||
// Save sidenav state before reload
|
||||
localStorage.setItem('sidenavWasOpen', isNavOpen ? 'true' : 'false');
|
||||
window.location.reload();
|
||||
});
|
||||
}
|
||||
|
||||
if (logoutButton) {
|
||||
logoutButton.addEventListener('click', async () => {
|
||||
try {
|
||||
// In optional/none modes this doubles as a "Sign in" entry point.
|
||||
if (!isAuthenticated) {
|
||||
await promptLoginIfNeeded();
|
||||
return;
|
||||
}
|
||||
await Logout();
|
||||
} catch (error) {
|
||||
console.error('Logout/login action failed:', error);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Resolve and initialize page auth policy.
|
||||
authMode = resolveAuthModeFromUrl();
|
||||
console.log('[template.html] Resolved auth mode:', authMode);
|
||||
await initializeAuthentication(authMode);
|
||||
|
||||
// Initialize authenticated features only when signed in.
|
||||
await initializeAuthenticatedPageFeatures();
|
||||
|
||||
/* ============================================================
|
||||
EXAMPLE: Subscribe to events
|
||||
============================================================
|
||||
Uncomment to subscribe to user's notes:
|
||||
|
||||
const notesSub = subscribe(
|
||||
{ kinds: [1], authors: [currentPubkey], limit: 10 },
|
||||
{ closeOnEose: false, cacheUsage: 'CACHE_FIRST' }
|
||||
);
|
||||
console.log("[template.html] Subscribed to kind 1");
|
||||
============================================================ */
|
||||
|
||||
/* ============================================================
|
||||
EXAMPLE: Listen for events from worker
|
||||
============================================================
|
||||
Uncomment to handle incoming events:
|
||||
|
||||
window.addEventListener('ndkEvent', (event) => {
|
||||
const evt = event.detail;
|
||||
console.log("[template.html] Received event:", evt.kind, evt.pubkey);
|
||||
|
||||
if (evt.pubkey === currentPubkey) {
|
||||
if (evt.kind === 1) {
|
||||
// Handle note event
|
||||
console.log("Note:", evt.content);
|
||||
}
|
||||
}
|
||||
});
|
||||
============================================================ */
|
||||
|
||||
/* ============================================================
|
||||
EXAMPLE: Listen for cached profile
|
||||
============================================================
|
||||
Uncomment to handle cached profile data:
|
||||
|
||||
window.addEventListener('ndkProfile', (event) => {
|
||||
console.log("[template.html] Cached profile:", event.detail);
|
||||
// event.detail contains profile object (name, about, etc.)
|
||||
});
|
||||
============================================================ */
|
||||
|
||||
// Restore sidenav state if it was open before theme toggle
|
||||
const sidenavWasOpen = localStorage.getItem('sidenavWasOpen');
|
||||
if (sidenavWasOpen === 'true') {
|
||||
localStorage.removeItem('sidenavWasOpen');
|
||||
openNav();
|
||||
}
|
||||
|
||||
// Optional UX note for public mode pages.
|
||||
if (!isAuthenticated && (authMode === 'optional' || authMode === 'none')) {
|
||||
divFooterCenter.textContent = 'Public mode';
|
||||
divFooterRight.textContent = 'Sign in from side menu for private features';
|
||||
}
|
||||
|
||||
// Update version display
|
||||
await updateVersionDisplay();
|
||||
|
||||
console.log('[template.html] Initialization complete');
|
||||
} catch (error) {
|
||||
console.error('[template.html] Initialization failed:', error);
|
||||
divBody.innerHTML = `<div style="text-align: center; padding: 50px;">
|
||||
<div style="font-size: 24px; margin-bottom: 20px; color: red;">❌ Authentication Error</div>
|
||||
<div style="font-size: 16px; color: #666;">${error.message}</div>
|
||||
<div style="margin-top: 20px;">
|
||||
<button onclick="location.reload()" style="padding: 10px 20px; font-size: 16px;">Retry</button>
|
||||
</div>
|
||||
</div>`;
|
||||
}
|
||||
})();
|
||||
|
||||
/* ================================================================
|
||||
WORKER MESSAGE TYPES
|
||||
================================================================
|
||||
The NDK worker can send these message types:
|
||||
|
||||
1. 'response' - Response to init/subscribe/publish requests
|
||||
- data.profile - User profile (from init)
|
||||
- data.relays - User relays (from init)
|
||||
- data.success - Publish success status
|
||||
- data.relayResults - Relay publish results
|
||||
|
||||
2. 'event' - Nostr event from subscription
|
||||
- Dispatched as 'ndkEvent' window event
|
||||
- event.detail contains the Nostr event
|
||||
|
||||
3. 'eose' - End of stored events for subscription
|
||||
- Dispatched as 'ndkEose' window event
|
||||
- event.detail.subId contains subscription ID
|
||||
|
||||
4. 'signRequest' - Request to sign event/encrypt/decrypt
|
||||
- Handled automatically by init-ndk.mjs
|
||||
- Calls window.nostr methods and sends response
|
||||
|
||||
5. 'error' - Error from worker
|
||||
- Logged to console automatically
|
||||
|
||||
6. 'relayActivity' - Relay read/write activity notification
|
||||
- Dispatched as 'ndkRelayActivity' window event
|
||||
- Used to animate relay status icons in footer
|
||||
================================================================ */
|
||||
|
||||
/* ================================================================
|
||||
DISTRIBUTED ARCHITECTURE NOTES
|
||||
================================================================
|
||||
Each page is independently accessible and self-contained:
|
||||
|
||||
1. Authentication persists via nostr-login-lite localStorage
|
||||
- Login once on any page
|
||||
- All other pages automatically authenticated
|
||||
|
||||
2. NDK SharedWorker is shared across all tabs/pages
|
||||
- Single NDK instance manages all connections
|
||||
- Subscriptions from all pages handled by one worker
|
||||
- Events broadcast to all connected pages
|
||||
|
||||
3. Dexie cache is shared across all pages
|
||||
- IndexedDB persists across sessions
|
||||
- Cache-first queries are fast
|
||||
- Reduces relay load
|
||||
|
||||
4. User settings are centralized and shared
|
||||
- Worker hydrates kind 30078 (`d:user-settings`) on init
|
||||
- Pages read via getUserSettings()
|
||||
- Pages patch via patchUserSettings({ featureNamespace: ... })
|
||||
- Pages subscribe via onUserSettings() for live updates
|
||||
|
||||
5. Each page can be distributed independently
|
||||
- Copy template.html and customize
|
||||
- No dependencies on other pages
|
||||
- Works standalone or as part of suite
|
||||
|
||||
6. Message-based signer bridges worker and page
|
||||
- Worker's NDK uses MessageBasedSigner
|
||||
- Signer sends sign requests to page
|
||||
- Page calls window.nostr.signEvent()
|
||||
- Response sent back to worker
|
||||
- NDK completes signing and publishing
|
||||
|
||||
7. Relay status visualization
|
||||
- Footer left section shows connected relays
|
||||
- Each relay has animated icon (HamburgerMorphing)
|
||||
- Icons morph based on activity (read/write)
|
||||
- Temporary animations show real-time activity
|
||||
================================================================ */
|
||||
</script>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
+77
-13
@@ -1,6 +1,6 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Build fully static MUSL binaries for C-Relay using Alpine Docker
|
||||
# Build fully static MUSL binaries for C-Relay-PG using Alpine Docker
|
||||
# Produces truly portable binaries with zero runtime dependencies
|
||||
|
||||
set -e
|
||||
@@ -11,19 +11,52 @@ DOCKERFILE="$SCRIPT_DIR/Dockerfile.alpine-musl"
|
||||
|
||||
# Parse command line arguments
|
||||
DEBUG_BUILD=false
|
||||
if [[ "$1" == "--debug" ]]; then
|
||||
DEBUG_BUILD=true
|
||||
DB_BACKEND="${DB_BACKEND:-postgres}"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--debug)
|
||||
DEBUG_BUILD=true
|
||||
shift
|
||||
;;
|
||||
--db-backend)
|
||||
if [[ -z "$2" ]]; then
|
||||
echo "ERROR: --db-backend requires a value (sqlite|postgres)"
|
||||
exit 1
|
||||
fi
|
||||
DB_BACKEND="$2"
|
||||
shift 2
|
||||
;;
|
||||
--db-backend=*)
|
||||
DB_BACKEND="${1#*=}"
|
||||
shift
|
||||
;;
|
||||
*)
|
||||
echo "ERROR: Unknown argument: $1"
|
||||
echo "Usage: $0 [--debug] [--db-backend postgres|sqlite]"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ "$DB_BACKEND" != "sqlite" && "$DB_BACKEND" != "postgres" ]]; then
|
||||
echo "ERROR: Invalid DB backend '$DB_BACKEND'. Use sqlite or postgres."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ "$DEBUG_BUILD" == "true" ]]; then
|
||||
echo "=========================================="
|
||||
echo "C-Relay MUSL Static Binary Builder (DEBUG MODE)"
|
||||
echo "C-Relay-PG MUSL Static Binary Builder (DEBUG MODE)"
|
||||
echo "=========================================="
|
||||
else
|
||||
echo "=========================================="
|
||||
echo "C-Relay MUSL Static Binary Builder (PRODUCTION MODE)"
|
||||
echo "C-Relay-PG MUSL Static Binary Builder (PRODUCTION MODE)"
|
||||
echo "=========================================="
|
||||
fi
|
||||
echo "Project directory: $SCRIPT_DIR"
|
||||
echo "Build directory: $BUILD_DIR"
|
||||
echo "Debug build: $DEBUG_BUILD"
|
||||
echo "DB backend: $DB_BACKEND"
|
||||
echo ""
|
||||
|
||||
# Create build directory
|
||||
@@ -63,24 +96,52 @@ ARCH=$(uname -m)
|
||||
case "$ARCH" in
|
||||
x86_64)
|
||||
PLATFORM="linux/amd64"
|
||||
OUTPUT_NAME="c_relay_static_x86_64"
|
||||
OUTPUT_NAME="c_relay_pg_static_x86_64"
|
||||
;;
|
||||
aarch64|arm64)
|
||||
PLATFORM="linux/arm64"
|
||||
OUTPUT_NAME="c_relay_static_arm64"
|
||||
OUTPUT_NAME="c_relay_pg_static_arm64"
|
||||
;;
|
||||
*)
|
||||
echo "WARNING: Unknown architecture: $ARCH"
|
||||
echo "Defaulting to linux/amd64"
|
||||
PLATFORM="linux/amd64"
|
||||
OUTPUT_NAME="c_relay_static_${ARCH}"
|
||||
OUTPUT_NAME="c_relay_pg_static_${ARCH}"
|
||||
;;
|
||||
esac
|
||||
|
||||
# Append _debug suffix to output name for debug builds so production binary is never overwritten
|
||||
if [ "$DEBUG_BUILD" = true ]; then
|
||||
OUTPUT_NAME="${OUTPUT_NAME}_debug"
|
||||
fi
|
||||
|
||||
echo "Building for platform: $PLATFORM"
|
||||
echo "Output binary: $OUTPUT_NAME"
|
||||
echo ""
|
||||
|
||||
# Check if Alpine base image is cached
|
||||
echo "Checking for cached Alpine Docker image..."
|
||||
if ! docker images alpine:3.19 --format "{{.Repository}}:{{.Tag}}" | grep -q "alpine:3.19"; then
|
||||
echo "⚠ Alpine 3.19 image not found in cache"
|
||||
echo "Attempting to pull Alpine 3.19 image..."
|
||||
if ! docker pull alpine:3.19; then
|
||||
echo ""
|
||||
echo "ERROR: Failed to pull Alpine 3.19 image"
|
||||
echo "This is required for the static build."
|
||||
echo ""
|
||||
echo "Possible solutions:"
|
||||
echo " 1. Check your internet connection"
|
||||
echo " 2. Try again later (Docker Hub may be temporarily unavailable)"
|
||||
echo " 3. If you have IPv6 issues, disable IPv6 for Docker"
|
||||
echo ""
|
||||
exit 1
|
||||
fi
|
||||
echo "✓ Alpine 3.19 image pulled successfully"
|
||||
else
|
||||
echo "✓ Alpine 3.19 image found in cache"
|
||||
fi
|
||||
echo ""
|
||||
|
||||
# Build the Docker image
|
||||
echo "=========================================="
|
||||
echo "Step 1: Building Alpine Docker image"
|
||||
@@ -88,14 +149,15 @@ echo "=========================================="
|
||||
echo "This will:"
|
||||
echo " - Use Alpine Linux (native MUSL)"
|
||||
echo " - Build all dependencies statically"
|
||||
echo " - Compile c-relay with full static linking"
|
||||
echo " - Compile c-relay-pg with full static linking"
|
||||
echo ""
|
||||
|
||||
$DOCKER_CMD build \
|
||||
--platform "$PLATFORM" \
|
||||
--build-arg DEBUG_BUILD=$DEBUG_BUILD \
|
||||
--build-arg DB_BACKEND=$DB_BACKEND \
|
||||
-f "$DOCKERFILE" \
|
||||
-t c-relay-musl-builder:latest \
|
||||
-t c-relay-pg-musl-builder:latest \
|
||||
--progress=plain \
|
||||
. || {
|
||||
echo ""
|
||||
@@ -117,16 +179,17 @@ echo "=========================================="
|
||||
$DOCKER_CMD build \
|
||||
--platform "$PLATFORM" \
|
||||
--build-arg DEBUG_BUILD=$DEBUG_BUILD \
|
||||
--build-arg DB_BACKEND=$DB_BACKEND \
|
||||
--target builder \
|
||||
-f "$DOCKERFILE" \
|
||||
-t c-relay-static-builder-stage:latest \
|
||||
-t c-relay-pg-static-builder-stage:latest \
|
||||
. > /dev/null 2>&1
|
||||
|
||||
# Create a temporary container to copy the binary
|
||||
CONTAINER_ID=$($DOCKER_CMD create c-relay-static-builder-stage:latest)
|
||||
CONTAINER_ID=$($DOCKER_CMD create c-relay-pg-static-builder-stage:latest)
|
||||
|
||||
# Copy binary from container
|
||||
$DOCKER_CMD cp "$CONTAINER_ID:/build/c_relay_static" "$BUILD_DIR/$OUTPUT_NAME" || {
|
||||
$DOCKER_CMD cp "$CONTAINER_ID:/build/c_relay_pg_static" "$BUILD_DIR/$OUTPUT_NAME" || {
|
||||
echo "ERROR: Failed to extract binary from container"
|
||||
$DOCKER_CMD rm "$CONTAINER_ID" 2>/dev/null
|
||||
exit 1
|
||||
@@ -196,6 +259,7 @@ if [ "$DEBUG_BUILD" = true ]; then
|
||||
else
|
||||
echo "Build Type: PRODUCTION (optimized, stripped)"
|
||||
fi
|
||||
echo "DB Backend: $DB_BACKEND"
|
||||
if [ "$TRULY_STATIC" = true ]; then
|
||||
echo "Linkage: Fully static binary (Alpine MUSL-based)"
|
||||
echo "Portability: Works on ANY Linux distribution"
|
||||
|
||||
+1
-1
Submodule c_utils_lib updated: 442facd7e3...f46747b243
@@ -0,0 +1 @@
|
||||
key,value,data_type,description,category,requires_restart,created_at,updated_at
|
||||
|
+9970
File diff suppressed because it is too large
Load Diff
@@ -1,72 +0,0 @@
|
||||
|
||||
=== NOSTR WebSocket Debug Log Started ===
|
||||
[14:16:28.243] SEND localhost:8888: ["EVENT", {
|
||||
"pubkey": "193279d1459ba1399aadb954422bf8595aa77367dccf482c682f5f208e435844",
|
||||
"created_at": 1761499411,
|
||||
"kind": 1059,
|
||||
"tags": [["p", "4f355bdcb7cc0af728ef3cceb9615d90684bb5b2ca5f859ab0f0b704075871aa"]],
|
||||
"content": "AmWNi4P5J126kk69XH2o5mYvGj+69+Fjfr/nZx892I2z8edkwtp2IH7XAnPUqdGPu7x1xiZF1sNfr21VKThOhE54K/uQHLFydZN3acgUfX13sCeWhrvnQD0EvjvZC6QzW9DfFayYoYl+rEPYcra1/N68a+N1R7XnNcf1K/ZFh5Grcnln0H5YdXKRBhQI9aai4iFp1VGy2V0IR+6gDJGbJ7TbAbD3wgGWv1i77C03skH3RgzH+f2b7VBtm+vjKX6q7v6v8j3w1lRFE5Qh0Tqgedh3+UsnwqQta7OCzF9OyAVPK7EqKQBss5LzYSRUpcCE1vw5b7I7yeBFwU9WfnGLUW+uZxMJ2C3P4NBBrVO8UFIkBrPL2cqkoD5c8DgMLJjXGmc4EWfB4ZWb3KjbfLbgi6DVQ++cDjBbnCOPhX+/4qOnWq+gI28e/xk3cBvQtgUOkvWX3oGl3/Q33u4UGtxkFEXGfzdKHVDkR86kqf7RMZjIwTjLGpx4uov0cNmzj07hYEdoG/lJ4yA1v/GyF7viJdnnz3tE0hCZaViqSCev0rfUHWRDDMXJzJ9SS+OwpVswSG4NKvYsDhDM89BjhFs08HshTFdIh2AY45jR/16CsZM9JudH5BwqcX23wToYdZ+lrerOA0EkYb0DJUzGVe4lMpdJZoB8qXLHxMAKwKu0UEWEkeBnnZbvTGwCRbfGorxwPrnyqUCy9tzJx0GOLhRIzBmt6lki607VLDYjK97VIz0dff3fyWPAfy/yBlO2nHhVubUgpPaAjcaYNkO/iZwuP8oJkClWWmKwAQoNoxt+Ly2llrkz+Ne8oXMQdJSq416x6MLHo2JbKH8uwjx0yKG0oldLyWaz3A8OHYkJuxOi7HPVTlOOJrsjG4kMn2g97rVUXLs5v9F/StOjzxtiQWmCBtCsvK2LEEK/DzfavcJstEMxQztJjhiYRO3MJanL7lN2zu1ZHO149FJrgqGV6RQ8DDXf55yuabqHilBuUSDKpI0gl0+Efuor1my+L9J7MjJQ83aSwGizX7uXedMsGQRcvU3++Uvbw7sd2l67fb7IoYU04TPGZkIm120qwf7GAUpnDL7Lhulu/9LFMFs3UnGl9cLzY6EAJtDANHjMAoXbGbYclnoSiNW4yr3X9PBHO5o2YhIxfpTyEgLebJLOkzoziuCTpX8/MdhOhFtlIyo5B8Mbt5GDOHh4x1ZMKOl02J00Vvgui0hLw4Vri8Lz/ErPIRSlrEOB+8K5zPzJy/bD8XrOKlOwSbF5j9dsqs+8uCTC/v9YNQ0cC9wP7gVAxErQ3suJVeV7pzY+eGR051AcW7ppTs1gShhxDDaSaKdMlrkBdFDZcCJ+tomSgW56bOi45erpmk8Lcv4RrBzjBtq1hz+XSaTBAtEnGtHNH2uOn7KP/NNaD38dYkpb3N1VR3zuV67RcuPZeB+5WR9jhnLoSMGox2s=",
|
||||
"id": "c6c18d902744fc0aaa4ca9172b3bcd0dde3fd7d943b41b2a39a16927ede67804",
|
||||
"sig": "d67e0e914aa361c528510efd216548b6734a5fa68c46426571fbc87626bf19a9ec46e16883e7fad700f4fee5cfffd9bba03c3c08e57938fbca77a28b30a32bb7"
|
||||
}]
|
||||
[14:16:28.256] RECV localhost:8888: ["OK", "c6c18d902744fc0aaa4ca9172b3bcd0dde3fd7d943b41b2a39a16927ede67804", true, ""]
|
||||
|
||||
=== NOSTR WebSocket Debug Log Started ===
|
||||
[15:01:18.592] SEND localhost:8888: ["EVENT", {
|
||||
"pubkey": "ec9578ade9e74358ed35d8091d41bfa277e86d649614a8865e3725e38ebe5bc9",
|
||||
"created_at": 1761502101,
|
||||
"kind": 1059,
|
||||
"tags": [["p", "4f355bdcb7cc0af728ef3cceb9615d90684bb5b2ca5f859ab0f0b704075871aa"]],
|
||||
"content": "AlgLnVwti8Dk2nu0e4bMrXeZiR/u+RPnA85kpts2svaFGfMByS4iap7xqdiSrXpSQPjQsix6jP9Qiy1a6rrvC6MutqTi3JfsMexLR61/ZKTK41sWTXNDTT3keH543vx3fVQH1mq+LgG4mjNzkPe0RqkYFvC8R0nxyAcCecHDxZUlmXQmAGiB5JB2GvstA4eoZLP1OI3fcLA3qaITLNRJOwRUoTYKqUkENHwz74CW0TnYDrKRZVe9zKNWQBLmtsgVoGd5CXNAVgXwmm2h0eCNIcRGnFqDHzpegpEGO+A7tvB0KJwlj4j/GmRgmnWO4pkrM2fmsTdlb5KNqe7NPuTVgYfdvld70zWpenp7jF/0psaEQEl8R7FbG2rNCv8fXtH+womvJQj4S0eUBxfvsUU1wWYmhusEzvyTfpV/nw0Er+pmAUZ2eGk7LEB2GMsJrkT+G5oohm0n+5c72iWJqW9A1eAzjR6Z21FkH4kAEJOl70fw9Xeig+s9rYk3GcKlMvj42zf7DepMXHPy62TbqUeclcm5W/semyasGP521GBuw152IN+dS67OVVmEvEJ89xhwiTeIty78enR4Gq7d1eNK+rqStdtJ7FN6kD/8gv4sFojUXyi0sIxzaSPrwI3ohOqbpEK1dTs6fmTUiyT/Buq++IhD9UwsZgz/kYpZfm1NVnWx+yTEv4I1H80FDxmMzbYnTHuIdRJFeh/NJRy9h+gXoZlZnteHkbwm1w2AejTkVnGs2Pz7aUZgC+1Za8fhtXq3Z9N3R8f8gtVFnnjRzApg5U89QrXUBS0R6F9dTqINk4qti94JWO4dYcPuudCutME5hfCYBoo+LHuRmdPKry7vSK1WgLYQsHuG+r313Ak8DhZYNbL+0d5UJ9kDFlFKaP3xLahSbEc/7u+AyuN68IyM1NwEehllxqVUsX8dsD4bZ2yW5rVjAQm9tT8Ypm73kJEb+DYVqT0WjFx88ee+HX89a9NgszWf1HE1KNQ9gWjn4eH6xbwrOkS4/v2O2tQoAd00vyPKAWly3Zlrz2cRnrSnxTZ5Lt7HtwAt6Err8MhD/w5rMLXHTBCMrroG1VfMo1OgL1YPafKDZmwVcHWacqtZiB0heRx742WipmTonqMjCOTNufdwxQcRPLLio0mtqiIrzgJqqIQenBXSa1jaG6Lvb5PCUKThbg4sSFfgssoUNKM7ytmBAe+PPmOVe11/gGaFWoQeUordbvmiCtzIPUYiKsuhfeK4I3jKvEofQU37hOam8ZxUczXvX6dgOOto002EWyCVfAzFgyey6wI+FGEbhXqlw7nB+azpqLQMJnHg7pfb1stXk3d8rjgVrRsVRJe/5KrXyZ5cd7ftJuJLxpTYmfFu6CKoUE0L5eRxXiwa16Pi0BehxOLaZteiTzttyfj+ClMKs2J/2/T1BVya1oGUW2Wg6ri/qS8oXv8bqiXBZ1/BwfI=",
|
||||
"id": "ea5bb419a8efea8ee86bb8696406a70a0387a7d0ac6e60760026d1aea28b427f",
|
||||
"sig": "0ffde3fd0d83c80693aa656668f2553807f8d474738ff3d9676090a5b8748a8e8e0c75a1d64963e4604046e18a806c4371a9cf2af2fd72f9db50f15bc78a4e25"
|
||||
}]
|
||||
[15:01:18.604] RECV localhost:8888: ["OK", "ea5bb419a8efea8ee86bb8696406a70a0387a7d0ac6e60760026d1aea28b427f", true, ""]
|
||||
|
||||
=== NOSTR WebSocket Debug Log Started ===
|
||||
[07:46:36.863] SEND localhost:8888: ["EVENT", {
|
||||
"pubkey": "99e37bc774d260b464e936ad8945deec62e8f5f8af53e9db662038a717d39bd5",
|
||||
"created_at": 1761562419,
|
||||
"kind": 1059,
|
||||
"tags": [["p", "4f355bdcb7cc0af728ef3cceb9615d90684bb5b2ca5f859ab0f0b704075871aa"]],
|
||||
"content": "AjYCV8Esqa1L9LQE2G8cDVn+hSXjAJlFVp5nAaC6nuag/1AphKpsAJFGrWZvJ+rte5+4dbmdk+osvlxxfRQHtaZqjaTbVDKJA2b2KTYLgICe7O9rqTqR8oC4sYOVQViEjo30ox3IfDgdR5ONlaprvQ8r71E+oplWOjahUNvf5Yb9OOHbFphOqqWtbYRYqAqvO0bj3rB+tmyUJ8v3mU8NsKJtOTOIvN+jTIwU8cbN6AM6/A87WSi7J7X9/wLpFigBNxrx82MJ025ryApNWyt5PuBia3krPDa51F/A+jFVp1QicVwSt6tP01ktYJn3uyR3qizIiZiXzmxV9+TXopq+mOTlAiwcZBm1ZkS/PgfoUMDUOOVcCAW6ppZlg3oK5jScuDl1d4cZgTmmAPneaHhgB9A/DbWWr03W1vJAXCmDQRUoACfwsvLQf5esXkPcJV+ANgLl8sKd4EPmDAzr946uDcs2BUDftr4++jbdTkg9yIHb4SHnI10osbsqP7BqTrF1TbZHnxev4l5XyaIqhGm6WdQ90uGn1VDSUXXuou3IbqwhZReifYQAL9/5PAeSH6RrrL4neEzBjosSNkcMmtAxqBfd8dOHfqT6r15osKXc1eSmO9qDjZxUHUb5zIJjrkDW0jY8vAfiMqZhKd5Vl7stYf2iJJxdm04r4zpxGBjlYQva2LyrPclBIGxax6sTQTxoRyUBwyis7OnxUC0HO7bqr464RzCX1/OHMOYhFQu3BY8Rvytl9E1hS+rJpWgNkHsbjr5zs1l/B+qwt+zqnYlAtjZZ2T56Pjd+jZcRTt/NKDtyQnLPxreBWgNy8IykNI0q2fgFiWJwh7GnlbYrx5zco04Ory/P+nW2/Xosp0232I48e2KhxtH6L1e6dOWqbZXFQXBqzKNlVTRkPyS9ykSSs7NAVknRz/vF86+jJVJa2z32Y4oQtJna8vK5J5HA3rRSlSEINwmcSiFUmuxeFAcFjYjyVGlBhmH3B/98CtT2+JHgUYpMiG51+HR+OI9qBGgsF5SI9JKai7CFC3OqfaW1rZHN96uta4VVGQ1mJetz/xB3W+QThsZ0IJ6/wBnbUpPBoab4rfnYeeVwOMxiK5B2UIZ1+ihRrSMsjMC8DAEbUAn9XNABJHhDo0KJcYqtpHBIkQgbqfuSKTLmc4mZNJCp8wmry9Tc9ZQo2jT0dZa/NZO+qtqWXWqZWbMngXFer4AtR+Vethhg6BdhYOYI/j8gOW1m8qodBlj9BHiKEU3Ig9z6WawsOD95VosxhqrQDuyO07igXNWMNK5exRfvp2QiHgILuC9diZZGBXPRLIDlKERTotPc5IdutkTG6qVh6+r6wbwtVhiWJVmfVy/D0hvDvlaqzVk3FRVuRuMZI+LmF3OdNGIf0+lfMUeMAABhDNTWyyS8gG21JJZQOBxGc12x49xWvMLbXaPCKBKrqw4FLF4PTCc=",
|
||||
"id": "9899324517c0e1796ea513cfc9fa0a2592cf5532774abc7e2a1bac7bb16c4fbb",
|
||||
"sig": "0d73ac599d0d6d99dd9afa0c92d741e459bc53102557acba5d868089776bb36a521ae800303ce5ceceabc8d643116a74560744243b3a1c7749d6a52117343637"
|
||||
}]
|
||||
[07:46:36.876] RECV localhost:8888: ["OK", "9899324517c0e1796ea513cfc9fa0a2592cf5532774abc7e2a1bac7bb16c4fbb", true, ""]
|
||||
|
||||
=== NOSTR WebSocket Debug Log Started ===
|
||||
[07:46:57.426] SEND localhost:8888: ["EVENT", {
|
||||
"pubkey": "a1efe929139f3f195159389a6eb7199c127c88e32a0264cd826e95806a7c7db3",
|
||||
"created_at": 1761562440,
|
||||
"kind": 1059,
|
||||
"tags": [["p", "4f355bdcb7cc0af728ef3cceb9615d90684bb5b2ca5f859ab0f0b704075871aa"]],
|
||||
"content": "ArP2HEobkU/QYXy2R94zSkKM1OfQT5SabPeebj9dQVGbUKaKDwuN2RUYTRJ5rD8euyiXat8YzYO7PJ0CHzxclXxO8AWpdN4P76srm3zJ5z6kpQpcCFgInV7k4v6LZmmCrrtdTWqLjuLTPJJd6W9J7HqbTA3Dt4200BSpA4Por1TQAncplwH7O4vBfbPYtdv9w1RL1uSInMWcGwxttTXlyTtAJ0G0hQNofowFCMWuQCKjV5LxPfoXdOCrsp/We6x+hYKDxphsDjQ1tbdtYFmj/YRy6MJm1id4mr+i8fEnyimshE/fAhavOXUg6239MmYj8nR2RT9LMuhVckX+V5MZnVyC3mZlfzkPJiTHxiDkEREjljNOX9I+9yChg6MvyU41s7GjBlPyWiyeXedPcU2Q8ypGsFLhBl5i+IGSn6wCcBH8+h3euG8jtBgKxIP1qBYsXPTlYpSXQcisIZlW2Rubcawf/RF7HYbIRu884mpdVnURcHqfN9yquoVyfvIgQR5Fs6IbKatJ64LfMLkLNs4UIlumtQRdW3NFjglgb/rF9btKYVFHRG9dWwOpZBd0zvXtWKbts2AKQFU30/WegaPh2LT5rN9HfMsA1tI9YwZm//T2NiLaPwJCuOFWBOUiB7jIObQKtOHrICI/jXIGOAfgox5+fcAE6CaysHHzluVcwiw7GioShidaIDsZ10rWJOv1HeRpuiAJJTWk2FOBJzpOxli6s6jGj2S481Xa99I13TihgL0wAPhjsnQhz0kh40g89mipzVO5hbki101zIJCEBrDeT4Ptabc9GminXedq9k1G98usM0JSHsgtdZdztme/UyvYyAKMdez1yNgOp7YgOU15Rpz/KGL6W5Wk3MbUwpuVRzUWEMoBcyMzssn5Sa3mkh1RQqpTcoQaktTNwkhR1R5bgedka61JmcK4Uq3Hi/HfKYHbeUeta6Olu+U19PEwZia1iq+y0ZQm5gMwCK8BsoV44OLsjeDKlyRGCtIjkTc/L2LyuAZFhw560vKflkigQVcajaQVtEDgaT5odgFwvYEMOjbBDloDs589hAn8ZLyRJo3tIXNwqhctKTSqbit5qs85pOHkXSC3gsRQvDfq4qVh8iWXFotmOHlBEh4OZk89xwAnP0wiv5kd8N2c2CTB84SB224GinMhs0gkaCIXPPYv8IfVcow9+3sjnNov4dRRIB80fRXP9X3IyR7tXYCuq1uQO2iWiWKhNaqJRoTM1BUhLv0ebKYjfPevSVHUuV51CcsoFakNT8S0UnW7QHfmsESvCJLLT8ttrJqpRX2tf6SpzofHmzQHVrHFn8C7WKMVelndptmaOt/9Lek2UrZiKmzRP0CtBL+HoPRmZHF9t7y0qEhoApkrB9FPukH/IGV6jx891rH4nC1fLKc6zgkdjnYB7HDB+lWp2JKpV8Z3CbZXtR28kwIvZZIABZ23/U5cFds=",
|
||||
"id": "c8cdf8992fbc17a0ccb74f6dcb7b851f3fdd53317f5a5ea4e202a91b22e15ac6",
|
||||
"sig": "b9efba3448d67de8855838044427396af1958269642a975129fe877e48e5c0e0818d638264f8aa80404886559a7d29464339f63704044dbf11ff09eb0bdeda2b"
|
||||
}]
|
||||
[07:46:57.439] RECV localhost:8888: ["OK", "c8cdf8992fbc17a0ccb74f6dcb7b851f3fdd53317f5a5ea4e202a91b22e15ac6", true, ""]
|
||||
|
||||
=== NOSTR WebSocket Debug Log Started ===
|
||||
[07:48:51.631] SEND localhost:8888: ["EVENT", {
|
||||
"pubkey": "52feea8d0da247ed1537c88e12b2f6bc88697b69abe33bf4f059f9f10c0f2b43",
|
||||
"created_at": 1761562554,
|
||||
"kind": 1059,
|
||||
"tags": [["p", "4f355bdcb7cc0af728ef3cceb9615d90684bb5b2ca5f859ab0f0b704075871aa"]],
|
||||
"content": "AoU024VIO6IgceC43yYPvKxOb5PuuZQRAUQLC6Crdn5dtIVuHE8M/UUmmNXmWq3jB6kFbFWxNgFWuxCEG9sQHDEngk+tDOGlt+r0vx3jUZG09lzNzcghl/4l/Do48rcy0cfSm+mSHrJsy7N+MSAXQ1heKahKF2fSyfYFM+6EOSEl0sJSq09iWGFft0lWfeZ3AFpji0gp0Z5QY1hQPH3Te/TRuDCoR8GXG3NguLD22Ed8byOQzf/b7nWr70z2Sqg15zhMwyqkl//dOp9iIXE69FONqDfvFF0xttQ1L9PbzQizMt65CqbMuxGMiMA1zZsgQ8iN+xbIN4xv4DrzCtBZnYt3aJSt7cv+8Co5OmGNXu1RNUvxpZZTO8Dq/m08Y5JDYiRCvdh8kTASMVt0MfGsvWgmcHiiCINQUe7n5ynieayFpbl9j1Vtml0lmrfIOYnDQYmuqDDyG7PxhRt3G/SpiabWBwsqmTqCvrclXfTm4t5YYSr/5lKbwHzPk4qtdEs+LqH2/zd4egZnT4Xt/vIP/c55NrEWPmr50G37DpsSVbDxMQXs4dpldDntjEDFuL8VTkAzqibmiZSQnb6l+DpKYNdQCyg1S1ttnzYp2cRTAPzAVbRMqk1R6jagWnZyOs/JIK4on51JHczaUTCMypDLJFoOaFPTAedHfR/Hn2Nm05W8oQ/m/RGmxLgok6WgH3KJ8wvN+8X+XYpTgyxej/hYPqJnq3uaNqMbcD5katWRBmZtyZa3Cn2nZDqmFFJSABWacXNCyHL41Z+MhyYalYzvUev1ozUgx2NEWwxnSTvMkpOfvSDrs5Ncosle0itL9j0QVBrKFjHgq2BJ4FApv/Iq0af+8JEqhVEsMNpGwRJst1kn7kO+Q7O68PQF6PPlNqh7DNea0Bz1sN1QDt2yZTApi2b3IJTsbekye/WsOB0J+bLvxNQ/UoULcyq3SLRSqQEQMLBPz7JrijMzBdPglWpeZ58UDrbmd1KhnHzx7o1NvHyjPRuKj4M094+2/mTEFGOOIF+Ogjqj+wCSDnT5C0d/l2llQkIXCKcLONWT4bKkmTOjvNs6lX+VpBynaegGjzGOvJw1beRZIkRegTpV4pnMZH9833s175rcMcDjPnfT9FD+pDv1DkmXfww1k6MgfHTbwjSgj0K9862xFNwL3mC2g8XFNlcflC0Rd8PzXRg7TBn7855r+urqujCUqZrzdUMHvp08rEEzSJKljk4XN4DqZeWn2evv7UCbYjq46sVf2lEHCvHdqKoPf5ENU72y5Fb9tQJAyUoBTdPdb2SZ2Y2jSF+6+H2wVXrOlm8EwBquaREl25fs7Yqwjru7qz1rO9EA1jlNybFvALHEFQzHEpi8JeNi5T/mI+VleoUDrk/og2mucQVFqAQzRjASsaeDq9fZNqhv3Q3DBIpftmI4g6ZXqJhPRK8wF7Ym3mmC7eFHwalUprA=",
|
||||
"id": "9bc4b5ad293085272bf52ff17abb585f7e63bc155a5a39cfe1a5c046f141e571",
|
||||
"sig": "ee6b917761031a06bc50da0173aef881a61213473d4f533a8a4a96247edcdbd17dbf87919c4d92f8ea8719d5311d51a8028fbf62e3f40f9b8004ccbe9f3adabd"
|
||||
}]
|
||||
[07:49:01.659] RECV localhost:8888: ["OK", "9bc4b5ad293085272bf52ff17abb585f7e63bc155a5a39cfe1a5c046f141e571", false, "error: failed to store gift wrap event"]
|
||||
|
||||
=== NOSTR WebSocket Debug Log Started ===
|
||||
[07:50:47.319] SEND localhost:8888: ["EVENT", {
|
||||
"pubkey": "f206ef335cc3b360cf739680cd4540b852fb9d75aac552b58014a41cfc4c6c65",
|
||||
"created_at": 1761562670,
|
||||
"kind": 1059,
|
||||
"tags": [["p", "4f355bdcb7cc0af728ef3cceb9615d90684bb5b2ca5f859ab0f0b704075871aa"]],
|
||||
"content": "AuzXkvYpH0IX/T0BKtOAjO5QtglT3nXGXF20awgDX6T1qoWV8qykYY01qPlGLSDkOuOvhG5NZuFPs/hPMnctmnskvHTHqKJdUeT10Qe0JmZiP5y6fZSlrtLMKfyoLpNYFDXOfwooSD2Q0UN8ePfTkkB61ri7avsq8w1WjoVTUSG5kouJfQAgvh75uXNkcvTNWSX5gbCXxoL1D7twPSUXvuZBJTN0iVdh3jg8X1uZWhvpZJIIXceZIUdxaIp2EmrYVW0VEZZSbAFGAldtKasHrP3cJKwTk1IenMFXaPkJnsbvyzUZWKBwTCeBLBhMNzbBWOp5A7SgFW1vf9gm00MSQE/JjwOzDZIaIQ3vRCMbO29XLlcOCevs8FwusZ86LuXZ8EQac9vnc/7gEul/SkOQaSq7v4oGzwfY4iQ8c8VdX/+syE+ClUEjiKdN3/lcRGWhdMGYgK/uajLd9jpfY8CBzP0BZM0Oq6ZgSJt4ydntPfiUq4PtDww+56+bUUQb5V3eZ9SUnQgebO1doRgtvb6LiLGN3D9XolEiHBE3KDt/InfWPUeuf1HEf5IbDc2w7zhgzWbvXK9G4NnsAmOHQNIGbPHXLWEOOhRtcEnrHILKYgs4wfuvSnfSUzfWxHVlhXkuXj4pq/EmJKmQg3zB1C2QzKMx7O/oHplnQFGUfAMDQY9GpBWLCOhQH3ZiHWQ02AjXSze7PGB9ac7KoPmyKafUAgXbASp/G9t7n3pxarzwBNm4zQ6wPRpR5OF+mFYQ9ClJ+3MlUDNq1T8fGuJVIduxFgyWMAgKoJBQe+xP5qHEuCxhG1B2KtoKHbzAtXpV+sduXQOlm0Jq7req59VgtgrIVVoLNyn7ulUFmJvWbaPWyuMrC1z+MdjNw/oJ1mE4+zYSB4Yho7DCwVdIxWrkFSx1yH9s1WPPyERCy4/UQfHpPmAD/JyAnnkKsc7v3MpAWAYCsiil1/PgFFPRRrO7jS+Ez+veQl5tx376ac9MwaN+ZbFADqYaf8CCcWXhMlAYl/zcMWLXKqL/wKb6orpTTHiWU/iJIvbuT0MIN68LIX+G/S5QCIcAQez+G35n5pDUkKikVQguKcJG51iDZqRAc+fnjSa7ifu8HBJ8HIKZjHEEyp6oGU0LCEWH60iIa7toKAwRx8rLPP2tWo+5u41nUrhpUXhUquQu8Dr+LrNdB30qYlH123R0NBBtXG7ngW8WDv2GQcul33ftiI/14QofOthA8SiExW5B7OsWJQON8sS1ZTc5l/M6f5B17CwqmAGd4NdKPQy1SZWGD61jkefwzKW/w4fZFXfploGwuYvFI/G8/YnaJ60p/k+2Aftcst9ikAHZF4xBtuJr4IrT6/f+snv12G4EdowmaSMjXRZv30d4yKwFmwiuoDHWLyYVwBkO+UO3r0WEe1DId0Z1FZnXfgdnM+zAZwITtCVQjZMcsOSNskKd1eE=",
|
||||
"id": "ce28dc9c653a4f5451266bc215942be9a54e4777a27862fddce351a59cc2dbf3",
|
||||
"sig": "539f314c0f0fd685647da358c4153272baf671f1a1bc42b8ff61231c4b5f1f03cb8d15a36fb78437dbf094c546e9ffe8e03de7ddb3b62a981c135a714ec57f93"
|
||||
}]
|
||||
[07:50:47.325] RECV localhost:8888: ["OK", "ce28dc9c653a4f5451266bc215942be9a54e4777a27862fddce351a59cc2dbf3", true, ""]
|
||||
+5
-12
@@ -1,19 +1,12 @@
|
||||
#!/bin/bash
|
||||
# Restart the service
|
||||
sudo systemctl stop c-relay-pg.service
|
||||
|
||||
# Copy the binary to the deployment location
|
||||
cp build/c_relay_x86 ~/Storage/c_relay/crelay
|
||||
|
||||
# Copy the local service file to systemd
|
||||
sudo cp systemd/c-relay-local.service /etc/systemd/system/
|
||||
|
||||
# Reload systemd daemon to pick up the new service
|
||||
sudo systemctl daemon-reload
|
||||
|
||||
# Enable the service (if not already enabled)
|
||||
sudo systemctl enable c-relay-local.service
|
||||
cp build/c_relay_pg_static_x86_64 ~/Storage/c_relay_pg/crelay
|
||||
|
||||
# Restart the service
|
||||
sudo systemctl restart c-relay-local.service
|
||||
sudo systemctl restart c-relay-pg.service
|
||||
|
||||
# Show service status
|
||||
sudo systemctl status c-relay-local.service --no-pager -l
|
||||
sudo systemctl status c-relay-pg.service --no-pager -l
|
||||
|
||||
+88
-18
@@ -1,28 +1,98 @@
|
||||
#!/bin/bash
|
||||
|
||||
# C-Relay Static Binary Deployment Script
|
||||
# Deploys build/c_relay_static_x86_64 to server via ssh
|
||||
# C-Relay-PG PostgreSQL Deployment Script
|
||||
# Deploys static relay binary and configures PostgreSQL 18 + systemd service on remote server.
|
||||
|
||||
set -e
|
||||
set -euo pipefail
|
||||
|
||||
# Configuration
|
||||
LOCAL_BINARY="build/c_relay_static_x86_64"
|
||||
REMOTE_BINARY_PATH="/usr/local/bin/c_relay/c_relay"
|
||||
SERVICE_NAME="c-relay"
|
||||
REMOTE_HOST="ubuntu@laantungir.net"
|
||||
LOCAL_BINARY="build/c_relay_pg_static_x86_64"
|
||||
REMOTE_BINARY_DIR="/usr/local/bin/c_relay_pg"
|
||||
REMOTE_BINARY_PATH="/usr/local/bin/c_relay_pg/c_relay_pg"
|
||||
SERVICE_NAME="c-relay-pg"
|
||||
|
||||
# Create backup
|
||||
ssh ubuntu@laantungir.com "sudo cp '$REMOTE_BINARY_PATH' '${REMOTE_BINARY_PATH}.backup.$(date +%Y%m%d_%H%M%S)'" 2>/dev/null || true
|
||||
LOCAL_SERVICE_FILE="systemd/c-relay.service"
|
||||
LOCAL_PG_SETUP_SCRIPT="systemd/setup_postgres_18.sh"
|
||||
|
||||
# Upload binary to temp location
|
||||
scp "$LOCAL_BINARY" "ubuntu@laantungir.com:/tmp/c_relay.tmp"
|
||||
if [ ! -f "$LOCAL_BINARY" ]; then
|
||||
echo "ERROR: Binary not found: $LOCAL_BINARY"
|
||||
echo "Build it first (e.g. ./make_and_restart_relay.sh)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Install binary
|
||||
ssh ubuntu@laantungir.com "sudo mv '/tmp/c_relay.tmp' '$REMOTE_BINARY_PATH'"
|
||||
ssh ubuntu@laantungir.com "sudo chown c-relay:c-relay '$REMOTE_BINARY_PATH'"
|
||||
ssh ubuntu@laantungir.com "sudo chmod +x '$REMOTE_BINARY_PATH'"
|
||||
if [ ! -f "$LOCAL_SERVICE_FILE" ]; then
|
||||
echo "ERROR: Service file not found: $LOCAL_SERVICE_FILE"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Reload systemd and restart service
|
||||
ssh ubuntu@laantungir.com "sudo systemctl daemon-reload"
|
||||
ssh ubuntu@laantungir.com "sudo systemctl restart '$SERVICE_NAME'"
|
||||
if [ ! -f "$LOCAL_PG_SETUP_SCRIPT" ]; then
|
||||
echo "ERROR: PostgreSQL setup script not found: $LOCAL_PG_SETUP_SCRIPT"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Deployment complete!"
|
||||
echo "==> Uploading artifacts to $REMOTE_HOST"
|
||||
scp "$LOCAL_BINARY" "$REMOTE_HOST:/tmp/c_relay_pg.tmp"
|
||||
scp "$LOCAL_SERVICE_FILE" "$REMOTE_HOST:/tmp/c-relay-pg.service"
|
||||
scp "$LOCAL_PG_SETUP_SCRIPT" "$REMOTE_HOST:/tmp/setup_postgres_18.sh"
|
||||
|
||||
echo "==> Running remote install/configuration"
|
||||
ssh "$REMOTE_HOST" 'bash -s' <<'EOF'
|
||||
set -euo pipefail
|
||||
|
||||
SERVICE_NAME="c-relay-pg"
|
||||
RELAY_USER="c-relay-pg"
|
||||
REMOTE_BINARY_DIR="/usr/local/bin/c_relay_pg"
|
||||
REMOTE_BINARY_PATH="/usr/local/bin/c_relay_pg/c_relay_pg"
|
||||
|
||||
echo "[remote] Ensuring service user exists"
|
||||
if ! id "$RELAY_USER" >/dev/null 2>&1; then
|
||||
sudo useradd --system --home-dir /opt/c-relay-pg --shell /usr/sbin/nologin "$RELAY_USER"
|
||||
fi
|
||||
|
||||
echo "[remote] Ensuring required directories exist"
|
||||
sudo mkdir -p "$REMOTE_BINARY_DIR" /opt/c-relay-pg /etc/c-relay-pg
|
||||
sudo chown "$RELAY_USER:$RELAY_USER" /opt/c-relay-pg
|
||||
|
||||
echo "[remote] Installing PostgreSQL 18 (PGDG) if missing"
|
||||
if ! dpkg -s postgresql-18 >/dev/null 2>&1; then
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y curl ca-certificates lsb-release gnupg
|
||||
sudo install -d /usr/share/postgresql-common/pgdg
|
||||
sudo curl -fsSL https://www.postgresql.org/media/keys/ACCC4CF8.asc -o /usr/share/postgresql-common/pgdg/apt.postgresql.org.asc
|
||||
echo "deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.asc] https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" | sudo tee /etc/apt/sources.list.d/pgdg.list >/dev/null
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y postgresql-18
|
||||
fi
|
||||
|
||||
sudo systemctl enable postgresql
|
||||
sudo systemctl restart postgresql
|
||||
|
||||
echo "[remote] Configuring PostgreSQL role/database"
|
||||
sudo chmod +x /tmp/setup_postgres_18.sh
|
||||
sudo /tmp/setup_postgres_18.sh
|
||||
|
||||
echo "[remote] Installing relay binary"
|
||||
if [ -f "$REMOTE_BINARY_PATH" ]; then
|
||||
sudo cp "$REMOTE_BINARY_PATH" "${REMOTE_BINARY_PATH}.backup.$(date +%Y%m%d_%H%M%S)"
|
||||
fi
|
||||
sudo mv /tmp/c_relay_pg.tmp "$REMOTE_BINARY_PATH"
|
||||
sudo chown "$RELAY_USER:$RELAY_USER" "$REMOTE_BINARY_PATH"
|
||||
sudo chmod +x "$REMOTE_BINARY_PATH"
|
||||
|
||||
echo "[remote] Installing systemd unit"
|
||||
sudo mv /tmp/c-relay-pg.service /etc/systemd/system/c-relay-pg.service
|
||||
sudo chown root:root /etc/systemd/system/c-relay-pg.service
|
||||
sudo chmod 644 /etc/systemd/system/c-relay-pg.service
|
||||
|
||||
echo "[remote] Reloading and restarting service"
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable "$SERVICE_NAME"
|
||||
sudo systemctl restart "$SERVICE_NAME"
|
||||
|
||||
echo "[remote] Health checks"
|
||||
sudo systemctl --no-pager --full status "$SERVICE_NAME" | sed -n '1,25p'
|
||||
sudo -u "$RELAY_USER" psql -d crelay -c "SELECT current_user, current_database();"
|
||||
EOF
|
||||
|
||||
echo "Deployment complete: $REMOTE_HOST"
|
||||
|
||||
Executable
+122
@@ -0,0 +1,122 @@
|
||||
#!/bin/bash
|
||||
|
||||
# C-Relay-PG Debug Binary Deployment Script
|
||||
# Deploys build/c_relay_pg_static_x86_64_debug to server for CPU profiling
|
||||
#
|
||||
# Usage:
|
||||
# ./deploy_lt_debug.sh -- deploy debug binary and restart
|
||||
# ./deploy_lt_debug.sh --profile -- deploy, then run perf and fetch results
|
||||
#
|
||||
# After deploying, profile with:
|
||||
# sudo perf record -g -p $(pgrep c_relay_pg) -- sleep 30
|
||||
# sudo perf report --stdio --sort=symbol --no-children -n 2>/dev/null | head -80
|
||||
#
|
||||
# Restore production binary:
|
||||
# ./deploy_lt.sh
|
||||
|
||||
set -e
|
||||
|
||||
LOCAL_DEBUG_BINARY="build/c_relay_pg_static_x86_64_debug"
|
||||
REMOTE_BINARY_PATH="/usr/local/bin/c_relay_pg/c_relay_pg"
|
||||
REMOTE_PROD_BACKUP="/usr/local/bin/c_relay_pg/c_relay_pg.production"
|
||||
SERVICE_NAME="c-relay-pg"
|
||||
REMOTE_HOST="ubuntu@laantungir.com"
|
||||
|
||||
# Check debug binary exists
|
||||
if [ ! -f "$LOCAL_DEBUG_BINARY" ]; then
|
||||
echo "ERROR: Debug binary not found: $LOCAL_DEBUG_BINARY"
|
||||
echo ""
|
||||
echo "Build it first with:"
|
||||
echo " ./build_static.sh --debug"
|
||||
echo ""
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "=========================================="
|
||||
echo "C-Relay-PG Debug Deployment"
|
||||
echo "=========================================="
|
||||
echo "Binary: $LOCAL_DEBUG_BINARY ($(du -h "$LOCAL_DEBUG_BINARY" | cut -f1))"
|
||||
echo "Target: $REMOTE_HOST:$REMOTE_BINARY_PATH"
|
||||
echo ""
|
||||
echo "WARNING: Debug binary has symbols and is larger than production."
|
||||
echo " It is safe to run but should not be left deployed long-term."
|
||||
echo ""
|
||||
|
||||
# Backup production binary (only if not already backed up)
|
||||
echo "Backing up production binary..."
|
||||
ssh "$REMOTE_HOST" "
|
||||
if [ ! -f '$REMOTE_PROD_BACKUP' ]; then
|
||||
sudo cp '$REMOTE_BINARY_PATH' '$REMOTE_PROD_BACKUP'
|
||||
echo 'Production binary backed up to $REMOTE_PROD_BACKUP'
|
||||
else
|
||||
echo 'Production backup already exists, skipping'
|
||||
fi
|
||||
"
|
||||
|
||||
# Upload debug binary
|
||||
echo "Uploading debug binary..."
|
||||
scp "$LOCAL_DEBUG_BINARY" "$REMOTE_HOST:/tmp/c_relay_pg_debug.tmp"
|
||||
|
||||
# Install debug binary
|
||||
echo "Installing debug binary..."
|
||||
ssh "$REMOTE_HOST" "
|
||||
sudo mv '/tmp/c_relay_pg_debug.tmp' '$REMOTE_BINARY_PATH'
|
||||
sudo chown c-relay-pg:c-relay-pg '$REMOTE_BINARY_PATH'
|
||||
sudo chmod +x '$REMOTE_BINARY_PATH'
|
||||
"
|
||||
|
||||
# Restart service
|
||||
echo "Restarting c-relay-pg service..."
|
||||
ssh "$REMOTE_HOST" "sudo systemctl daemon-reload && sudo systemctl restart '$SERVICE_NAME'"
|
||||
|
||||
echo ""
|
||||
echo "✓ Debug binary deployed and service restarted"
|
||||
echo ""
|
||||
|
||||
# If --profile flag, run perf automatically
|
||||
if [ "$1" = "--profile" ]; then
|
||||
echo "=========================================="
|
||||
echo "Running perf profile (30 seconds)..."
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
|
||||
# Wait for relay to start
|
||||
sleep 3
|
||||
|
||||
ssh "$REMOTE_HOST" "
|
||||
PID=\$(pgrep c_relay_pg)
|
||||
if [ -z \"\$PID\" ]; then
|
||||
echo 'ERROR: c_relay_pg not running'
|
||||
exit 1
|
||||
fi
|
||||
echo \"Profiling PID \$PID for 30 seconds...\"
|
||||
sudo perf record -g -p \$PID -- sleep 30
|
||||
echo ''
|
||||
echo '=== TOP FUNCTIONS BY CPU ==='
|
||||
sudo perf report --stdio --sort=symbol --no-children -n 2>/dev/null | head -60
|
||||
"
|
||||
else
|
||||
echo "=========================================="
|
||||
echo "Next Steps: Profile the relay"
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
echo "SSH to server and run:"
|
||||
echo " sudo perf record -g -p \$(pgrep c_relay_pg) -- sleep 30"
|
||||
echo " sudo perf report --stdio --sort=symbol --no-children -n 2>/dev/null | head -60"
|
||||
echo ""
|
||||
echo "Or run with --profile to do it automatically:"
|
||||
echo " ./deploy_lt_debug.sh --profile"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "=========================================="
|
||||
echo "Restore Production Binary When Done"
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
echo "Run this to restore the production binary:"
|
||||
echo " ./deploy_lt.sh"
|
||||
echo ""
|
||||
echo "Or manually on the server:"
|
||||
echo " sudo cp '$REMOTE_PROD_BACKUP' '$REMOTE_BINARY_PATH'"
|
||||
echo " sudo systemctl restart $SERVICE_NAME"
|
||||
echo ""
|
||||
@@ -1,4 +1,4 @@
|
||||
# C-Relay Administrator API Implementation Plan
|
||||
# C-Relay-PG Administrator API Implementation Plan
|
||||
|
||||
## Problem Analysis
|
||||
|
||||
@@ -161,7 +161,7 @@ Would require changing schema, migration scripts, and storage logic.
|
||||
#### README.md Documentation Format:
|
||||
|
||||
```markdown
|
||||
# C-Relay Administrator API
|
||||
# C-Relay-PG Administrator API
|
||||
|
||||
## Authentication
|
||||
All admin commands require signing with the admin private key generated during first startup.
|
||||
|
||||
@@ -0,0 +1,190 @@
|
||||
# Agent Browser Testing Guide for C-Relay-PG
|
||||
|
||||
This document explains how to use the `agent-browser` CLI tool to test the c-relay-pg admin web UI from an AI agent context (no physical display required).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **agent-browser** installed globally: `npm install -g agent-browser`
|
||||
- Binary location: `/home/user/.nvm/versions/node/v24.14.1/bin/agent-browser`
|
||||
- The relay must be running locally (default port 8888)
|
||||
- Test keys configured in `.test_keys`
|
||||
|
||||
## Starting the Relay for Local Testing
|
||||
|
||||
```bash
|
||||
./make_and_restart_relay.sh -t
|
||||
```
|
||||
|
||||
This reads `.test_keys` and starts the relay with:
|
||||
- `ADMIN_PUBKEY` — the hex public key of the admin account
|
||||
- `ADMIN_PRIVKEY` — the hex secret key (used for browser login, not by the relay itself)
|
||||
- `SERVER_PRIVKEY` — the relay's own private key
|
||||
|
||||
## Key URLs
|
||||
|
||||
| URL | Purpose |
|
||||
|-----|---------|
|
||||
| `http://127.0.0.1:8888/api/index.html` | Admin web UI (the correct entry point) |
|
||||
| `http://127.0.0.1:8888/` | Returns 406 without NIP-11 Accept header — **do not use for browser testing** |
|
||||
| `http://127.0.0.1:8888/` with `Accept: application/nostr+json` | NIP-11 relay info JSON |
|
||||
|
||||
**Important:** The root URL `/` is a WebSocket/NIP-11 endpoint, not an HTML page. Always use `/api/index.html` for browser testing.
|
||||
|
||||
## Admin Login Credentials
|
||||
|
||||
The admin nsec for the current `.test_keys` configuration:
|
||||
|
||||
```
|
||||
nsec: nsec1zkn0hlt4jvcvn9yt5p7ea4m7f82pf3ph0gj3d4rlcz4s64x5z86satv9qm
|
||||
hex: 15a6fbfd759330c9948ba07d9ed77e49d414c4377a2516d47fc0ab0d54d411f5
|
||||
npub: npub1tlyzk6slzt8d989f4pn3synk98fea64ea3jmrdc7dtd0kw8uxlas9a9fwr
|
||||
hex pubkey: 5fc82b6a1f12ced29ca9a86718127629d39eeab9ec65b1b71e6adafb38fc37fb
|
||||
```
|
||||
|
||||
## Complete Login Flow with agent-browser
|
||||
|
||||
### Step 1: Open the admin UI
|
||||
|
||||
```bash
|
||||
agent-browser open http://127.0.0.1:8888/api/index.html
|
||||
agent-browser wait --load networkidle
|
||||
```
|
||||
|
||||
### Step 2: Take an interactive snapshot to see what is on screen
|
||||
|
||||
```bash
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
You will see a login modal with buttons like:
|
||||
- `"Browser Extension"` — skip this
|
||||
- `"Local Key"` — **use this one**
|
||||
- `"Seed Phrase"` — skip
|
||||
- `"Nostr Connect"` — skip
|
||||
- `"Read Only"` — skip
|
||||
|
||||
### Step 3: Click "Local Key"
|
||||
|
||||
```bash
|
||||
agent-browser click @e15
|
||||
```
|
||||
|
||||
(The ref number may vary — use the ref from the snapshot output for the "Local Key" button.)
|
||||
|
||||
After clicking, a text input appears asking for the secret key.
|
||||
|
||||
### Step 4: Enter the admin nsec
|
||||
|
||||
```bash
|
||||
agent-browser fill @e14 "nsec1zkn0hlt4jvcvn9yt5p7ea4m7f82pf3ph0gj3d4rlcz4s64x5z86satv9qm"
|
||||
```
|
||||
|
||||
(Use the ref from the snapshot for the textbox element.)
|
||||
|
||||
### Step 5: Click "Import Key"
|
||||
|
||||
```bash
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
Check the snapshot — the "Import Key" button should now be enabled. Click it:
|
||||
|
||||
```bash
|
||||
agent-browser click @e15
|
||||
```
|
||||
|
||||
### Step 6: Click "Continue" on the success screen
|
||||
|
||||
After import, a success screen appears with "Continue" button:
|
||||
|
||||
```bash
|
||||
agent-browser wait 800
|
||||
agent-browser snapshot -i
|
||||
agent-browser click @e19
|
||||
```
|
||||
|
||||
(Use the ref from the snapshot for the "Continue" button.)
|
||||
|
||||
### Step 7: Wait for admin UI to load
|
||||
|
||||
```bash
|
||||
agent-browser wait 5000
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
You should now see the admin dashboard with:
|
||||
- Statistics table (Database Size, Total Events, PID, etc.)
|
||||
- Navigation buttons (Statistics, Subscriptions, Configuration, Authorization, etc.)
|
||||
- Admin profile area showing "admin" label
|
||||
|
||||
## Navigating Admin Sections
|
||||
|
||||
After login, use the sidebar navigation buttons:
|
||||
|
||||
```bash
|
||||
# View configuration
|
||||
agent-browser click @e8 # Configuration button ref
|
||||
|
||||
# View authorization rules
|
||||
agent-browser click @e9 # Authorization button ref
|
||||
|
||||
# View statistics
|
||||
agent-browser click @e6 # Statistics button ref
|
||||
|
||||
# Always snapshot after navigation to see results
|
||||
agent-browser wait 2500
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
## Checking for Errors
|
||||
|
||||
```bash
|
||||
# View browser console logs
|
||||
agent-browser console
|
||||
|
||||
# View JavaScript errors
|
||||
agent-browser errors
|
||||
|
||||
# View relay server logs
|
||||
tail -n 100 relay.log
|
||||
```
|
||||
|
||||
## Chained Command Example (Full Login in One Shot)
|
||||
|
||||
```bash
|
||||
agent-browser open http://127.0.0.1:8888/api/index.html && \
|
||||
agent-browser wait --load networkidle && \
|
||||
agent-browser snapshot -i
|
||||
```
|
||||
|
||||
Then use refs from the snapshot to complete login steps.
|
||||
|
||||
## Tips for AI Agents
|
||||
|
||||
1. **Always use `/api/index.html`** — never the root URL
|
||||
2. **Use `snapshot -i`** after every action to see the current interactive elements and their refs
|
||||
3. **Refs change** between snapshots — always re-snapshot before clicking
|
||||
4. **Wait after clicks** — use `agent-browser wait 2000` (milliseconds) between actions that trigger async operations
|
||||
5. **The login flow has 3 screens**: method selection → key input → success confirmation
|
||||
6. **Console logs are cumulative** — they show all logs since page load, which is useful for debugging admin API responses
|
||||
7. **The relay log** at `relay.log` shows server-side processing of admin commands
|
||||
8. **Command chaining** with `&&` works — the browser daemon persists between commands
|
||||
|
||||
## Verifying Admin API is Working
|
||||
|
||||
After login, the statistics page should show populated data:
|
||||
- Database Size (e.g., "4 KB")
|
||||
- Process ID (the relay PID)
|
||||
- WebSocket Connections count
|
||||
- Memory Usage
|
||||
|
||||
If these show "-" or "Loading...", check:
|
||||
1. `relay.log` for errors
|
||||
2. `agent-browser console` for JavaScript errors
|
||||
3. `agent-browser errors` for page-level errors
|
||||
|
||||
## Closing the Browser
|
||||
|
||||
```bash
|
||||
agent-browser close --all
|
||||
```
|
||||
@@ -324,7 +324,7 @@ int main() {
|
||||
}
|
||||
```
|
||||
|
||||
## Migration Plan for c-relay
|
||||
## Migration Plan for c-relay-pg
|
||||
|
||||
### Phase 1: Extract Debug System
|
||||
1. Create `c_utils_lib` repository
|
||||
@@ -333,16 +333,16 @@ int main() {
|
||||
4. Add basic tests
|
||||
|
||||
### Phase 2: Add Versioning System
|
||||
1. Extract version generation logic from c-relay
|
||||
1. Extract version generation logic from c-relay-pg
|
||||
2. Create reusable version utilities
|
||||
3. Update c-relay to use new system
|
||||
3. Update c-relay-pg to use new system
|
||||
4. Update nostr_core_lib to use new system
|
||||
|
||||
### Phase 3: Add as Submodule
|
||||
1. Add `c_utils_lib` as submodule to c-relay
|
||||
2. Update c-relay Makefile
|
||||
3. Update includes in c-relay source files
|
||||
4. Remove old debug files from c-relay
|
||||
1. Add `c_utils_lib` as submodule to c-relay-pg
|
||||
2. Update c-relay-pg Makefile
|
||||
3. Update includes in c-relay-pg source files
|
||||
4. Remove old debug files from c-relay-pg
|
||||
|
||||
### Phase 4: Documentation & Examples
|
||||
1. Create comprehensive README
|
||||
@@ -352,7 +352,7 @@ int main() {
|
||||
|
||||
## Benefits
|
||||
|
||||
### For c-relay
|
||||
### For c-relay-pg
|
||||
- Cleaner separation of concerns
|
||||
- Reusable utilities across projects
|
||||
- Easier to maintain and test
|
||||
@@ -379,7 +379,7 @@ int main() {
|
||||
- Memory leak detection (valgrind)
|
||||
|
||||
### Integration Tests
|
||||
- Test with real projects (c-relay, nostr_core_lib)
|
||||
- Test with real projects (c-relay-pg, nostr_core_lib)
|
||||
- Cross-platform testing
|
||||
- Performance benchmarks
|
||||
|
||||
@@ -433,7 +433,7 @@ MIT License - permissive and suitable for learning and commercial use.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. ✅ Successfully integrated into c-relay
|
||||
1. ✅ Successfully integrated into c-relay-pg
|
||||
2. ✅ Successfully integrated into nostr_core_lib
|
||||
3. ✅ All tests passing
|
||||
4. ✅ Documentation complete
|
||||
@@ -449,7 +449,7 @@ MIT License - permissive and suitable for learning and commercial use.
|
||||
4. Create build system
|
||||
5. Write tests
|
||||
6. Create documentation
|
||||
7. Integrate into c-relay
|
||||
7. Integrate into c-relay-pg
|
||||
8. Publish to GitHub
|
||||
|
||||
---
|
||||
|
||||
@@ -2,13 +2,13 @@
|
||||
|
||||
## Overview
|
||||
|
||||
This document provides a step-by-step implementation plan for creating the `c_utils_lib` library and integrating it into the c-relay project.
|
||||
This document provides a step-by-step implementation plan for creating the `c_utils_lib` library and integrating it into the c-relay-pg project.
|
||||
|
||||
## Phase 1: Repository Setup & Structure
|
||||
|
||||
### Step 1.1: Create Repository Structure
|
||||
|
||||
**Location**: Create outside c-relay project (sibling directory)
|
||||
**Location**: Create outside c-relay-pg project (sibling directory)
|
||||
|
||||
```bash
|
||||
# Create directory structure
|
||||
@@ -42,7 +42,7 @@ git branch -M main
|
||||
|
||||
### Step 2.1: Move Debug Files
|
||||
|
||||
**Source files** (from c-relay):
|
||||
**Source files** (from c-relay-pg):
|
||||
- `src/debug.c` → `c_utils_lib/src/debug.c`
|
||||
- `src/debug.h` → `c_utils_lib/include/c_utils/debug.h`
|
||||
|
||||
@@ -484,20 +484,20 @@ How to integrate into projects:
|
||||
3. Code examples
|
||||
4. Migration from standalone utilities
|
||||
|
||||
## Phase 7: Integration with c-relay
|
||||
## Phase 7: Integration with c-relay-pg
|
||||
|
||||
### Step 7.1: Add as Submodule
|
||||
|
||||
```bash
|
||||
cd /path/to/c-relay
|
||||
cd /path/to/c-relay-pg
|
||||
git submodule add <repo-url> c_utils_lib
|
||||
git submodule update --init --recursive
|
||||
```
|
||||
|
||||
### Step 7.2: Update c-relay Makefile
|
||||
### Step 7.2: Update c-relay-pg Makefile
|
||||
|
||||
```makefile
|
||||
# Add to c-relay Makefile
|
||||
# Add to c-relay-pg Makefile
|
||||
C_UTILS_LIB = c_utils_lib/libc_utils.a
|
||||
|
||||
# Update includes
|
||||
@@ -514,7 +514,7 @@ $(C_UTILS_LIB):
|
||||
$(TARGET): $(C_UTILS_LIB) ...
|
||||
```
|
||||
|
||||
### Step 7.3: Update c-relay Source Files
|
||||
### Step 7.3: Update c-relay-pg Source Files
|
||||
|
||||
**Changes needed**:
|
||||
|
||||
@@ -543,7 +543,7 @@ $(TARGET): $(C_UTILS_LIB) ...
|
||||
### Step 7.4: Test Integration
|
||||
|
||||
```bash
|
||||
cd c-relay
|
||||
cd c-relay-pg
|
||||
make clean
|
||||
make
|
||||
./make_and_restart_relay.sh
|
||||
@@ -556,7 +556,7 @@ Verify:
|
||||
|
||||
## Phase 8: Version System Integration
|
||||
|
||||
### Step 8.1: Update c-relay Makefile for Versioning
|
||||
### Step 8.1: Update c-relay-pg Makefile for Versioning
|
||||
|
||||
```makefile
|
||||
# Add version generation
|
||||
@@ -567,7 +567,7 @@ src/version.h: .git/refs/tags/*
|
||||
$(TARGET): src/version.h ...
|
||||
```
|
||||
|
||||
### Step 8.2: Update c-relay to Use Generated Version
|
||||
### Step 8.2: Update c-relay-pg to Use Generated Version
|
||||
|
||||
Replace hardcoded version in `src/main.h` with:
|
||||
```c
|
||||
@@ -583,7 +583,7 @@ Replace hardcoded version in `src/main.h` with:
|
||||
- **Phase 4**: Build System - 2 hours
|
||||
- **Phase 5**: Examples & Tests - 3 hours
|
||||
- **Phase 6**: Documentation - 3 hours
|
||||
- **Phase 7**: c-relay Integration - 2 hours
|
||||
- **Phase 7**: c-relay-pg Integration - 2 hours
|
||||
- **Phase 8**: Version Integration - 2 hours
|
||||
|
||||
**Total**: ~19 hours
|
||||
@@ -593,11 +593,11 @@ Replace hardcoded version in `src/main.h` with:
|
||||
- [ ] c_utils_lib builds successfully
|
||||
- [ ] All tests pass
|
||||
- [ ] Examples compile and run
|
||||
- [ ] c-relay integrates successfully
|
||||
- [ ] Debug output works in c-relay
|
||||
- [ ] c-relay-pg integrates successfully
|
||||
- [ ] Debug output works in c-relay-pg
|
||||
- [ ] Version generation works
|
||||
- [ ] Documentation complete
|
||||
- [ ] No regressions in c-relay functionality
|
||||
- [ ] No regressions in c-relay-pg functionality
|
||||
|
||||
## Next Steps
|
||||
|
||||
@@ -608,7 +608,7 @@ Replace hardcoded version in `src/main.h` with:
|
||||
5. Create build system
|
||||
6. Write tests and examples
|
||||
7. Create documentation
|
||||
8. Integrate into c-relay
|
||||
8. Integrate into c-relay-pg
|
||||
9. Test thoroughly
|
||||
10. Publish to GitHub
|
||||
|
||||
|
||||
@@ -78,9 +78,9 @@ Configuration events follow the standard Nostr event format with kind 33334:
|
||||
|
||||
#### `relay_software`
|
||||
- **Description**: Software identifier for NIP-11
|
||||
- **Default**: `"c-relay"`
|
||||
- **Default**: `"c-relay-pg"`
|
||||
- **Format**: String, max 64 characters
|
||||
- **Example**: `"c-relay v1.0.0"`
|
||||
- **Example**: `"c-relay-pg v1.0.0"`
|
||||
|
||||
#### `relay_version`
|
||||
- **Description**: Software version string
|
||||
@@ -366,7 +366,7 @@ sqlite3 relay.nrdb "SELECT json_pretty(json_object(
|
||||
#### Invalid Parameter Values
|
||||
```bash
|
||||
# Check relay logs for validation errors
|
||||
journalctl -u c-relay | grep "Configuration.*invalid\|Invalid.*configuration"
|
||||
journalctl -u c-relay-pg | grep "Configuration.*invalid\|Invalid.*configuration"
|
||||
|
||||
# Common issues:
|
||||
# - Numeric values outside valid ranges
|
||||
@@ -395,10 +395,10 @@ ORDER BY date DESC;"
|
||||
#### Resource Usage After Changes
|
||||
```bash
|
||||
# Monitor system resources after configuration updates
|
||||
top -p $(pgrep c_relay)
|
||||
top -p $(pgrep c_relay_pg)
|
||||
|
||||
# Check for memory leaks
|
||||
ps aux | grep c_relay | awk '{print $6}' # RSS memory
|
||||
ps aux | grep c_relay_pg | awk '{print $6}' # RSS memory
|
||||
```
|
||||
|
||||
### Emergency Recovery
|
||||
@@ -425,7 +425,7 @@ nostrtool event \
|
||||
# If database is corrupted, backup and recreate
|
||||
cp relay.nrdb relay.nrdb.backup
|
||||
rm relay.nrdb*
|
||||
./build/c_relay_x86 # Creates fresh database with new keys
|
||||
./build/c_relay_pg_x86 # Creates fresh database with new keys
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
+14
-14
@@ -21,22 +21,22 @@ typedef enum {
|
||||
|
||||
```bash
|
||||
# Production (default - no debug output)
|
||||
./c_relay_x86
|
||||
./c_relay_pg_x86
|
||||
|
||||
# Show errors only
|
||||
./c_relay_x86 --debug-level=1
|
||||
./c_relay_pg_x86 --debug-level=1
|
||||
|
||||
# Show errors and warnings
|
||||
./c_relay_x86 --debug-level=2
|
||||
./c_relay_pg_x86 --debug-level=2
|
||||
|
||||
# Show errors, warnings, and info (recommended for development)
|
||||
./c_relay_x86 --debug-level=3
|
||||
./c_relay_pg_x86 --debug-level=3
|
||||
|
||||
# Show all debug messages
|
||||
./c_relay_x86 --debug-level=4
|
||||
./c_relay_pg_x86 --debug-level=4
|
||||
|
||||
# Show everything including trace with file:line (very verbose)
|
||||
./c_relay_x86 --debug-level=5
|
||||
./c_relay_pg_x86 --debug-level=5
|
||||
```
|
||||
|
||||
## Implementation
|
||||
@@ -475,7 +475,7 @@ When `g_debug_level = 0` (constant), you'll see the compiler has removed all deb
|
||||
|
||||
### Level 3 (Errors + Warnings + Info)
|
||||
```
|
||||
[2025-01-12 14:30:15] [INFO ] Initializing C-Relay v0.4.6
|
||||
[2025-01-12 14:30:15] [INFO ] Initializing C-Relay-PG v0.4.6
|
||||
[2025-01-12 14:30:15] [INFO ] Loading configuration from database
|
||||
[2025-01-12 14:30:15] [ERROR] Failed to open database: permission denied
|
||||
[2025-01-12 14:30:16] [WARN ] Port 8888 unavailable, trying 8889
|
||||
@@ -484,7 +484,7 @@ When `g_debug_level = 0` (constant), you'll see the compiler has removed all deb
|
||||
|
||||
### Level 4 (All Debug Messages)
|
||||
```
|
||||
[2025-01-12 14:30:15] [INFO ] Initializing C-Relay v0.4.6
|
||||
[2025-01-12 14:30:15] [INFO ] Initializing C-Relay-PG v0.4.6
|
||||
[2025-01-12 14:30:15] [DEBUG] Opening database: build/abc123...def.db
|
||||
[2025-01-12 14:30:15] [DEBUG] Executing schema initialization
|
||||
[2025-01-12 14:30:15] [INFO ] SQLite WAL mode enabled
|
||||
@@ -496,7 +496,7 @@ When `g_debug_level = 0` (constant), you'll see the compiler has removed all deb
|
||||
|
||||
### Level 5 (Everything Including file:line for ALL messages)
|
||||
```
|
||||
[2025-01-12 14:30:15] [INFO ] [main.c:1607] Initializing C-Relay v0.4.6
|
||||
[2025-01-12 14:30:15] [INFO ] [main.c:1607] Initializing C-Relay-PG v0.4.6
|
||||
[2025-01-12 14:30:15] [DEBUG] [main.c:348] Opening database: build/abc123...def.db
|
||||
[2025-01-12 14:30:15] [TRACE] [main.c:330] Entering init_database()
|
||||
[2025-01-12 14:30:15] [ERROR] [config.c:125] Database locked
|
||||
@@ -525,11 +525,11 @@ Update the existing `log_*` functions to use the new debug macros
|
||||
make clean && make
|
||||
|
||||
# Test different levels
|
||||
./build/c_relay_x86 # No output
|
||||
./build/c_relay_x86 --debug-level=1 # Errors only
|
||||
./build/c_relay_x86 --debug-level=3 # Info + warnings + errors
|
||||
./build/c_relay_x86 --debug-level=4 # All debug messages
|
||||
./build/c_relay_x86 --debug-level=5 # Everything with file:line on TRACE
|
||||
./build/c_relay_pg_x86 # No output
|
||||
./build/c_relay_pg_x86 --debug-level=1 # Errors only
|
||||
./build/c_relay_pg_x86 --debug-level=3 # Info + warnings + errors
|
||||
./build/c_relay_pg_x86 --debug-level=4 # All debug messages
|
||||
./build/c_relay_pg_x86 --debug-level=5 # Everything with file:line on TRACE
|
||||
```
|
||||
|
||||
### Step 5: Gradual Migration (Ongoing)
|
||||
|
||||
@@ -34,7 +34,7 @@ static const struct {
|
||||
// NIP-11 Relay Information (relay keys will be populated at runtime)
|
||||
{"relay_description", "High-performance C Nostr relay with SQLite storage"},
|
||||
{"relay_contact", ""},
|
||||
{"relay_software", "https://git.laantungir.net/laantungir/c-relay.git"},
|
||||
{"relay_software", "https://git.laantungir.net/laantungir/c-relay-pg.git"},
|
||||
{"relay_version", "v1.0.0"},
|
||||
|
||||
// NIP-13 Proof of Work (pow_min_difficulty = 0 means PoW disabled)
|
||||
|
||||
+51
-51
@@ -52,11 +52,11 @@ sudo apt install -y build-essential git sqlite3 libsqlite3-dev \
|
||||
#### User and Directory Setup
|
||||
```bash
|
||||
# Create dedicated system user
|
||||
sudo useradd --system --home-dir /opt/c-relay --shell /bin/false c-relay
|
||||
sudo useradd --system --home-dir /opt/c-relay-pg --shell /bin/false c-relay-pg
|
||||
|
||||
# Create application directory
|
||||
sudo mkdir -p /opt/c-relay
|
||||
sudo chown c-relay:c-relay /opt/c-relay
|
||||
sudo mkdir -p /opt/c-relay-pg
|
||||
sudo chown c-relay-pg:c-relay-pg /opt/c-relay-pg
|
||||
```
|
||||
|
||||
### Build and Installation
|
||||
@@ -64,8 +64,8 @@ sudo chown c-relay:c-relay /opt/c-relay
|
||||
#### Automated Installation (Recommended)
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
git submodule update --init --recursive
|
||||
|
||||
# Build
|
||||
@@ -81,12 +81,12 @@ sudo systemd/install-service.sh
|
||||
make clean && make
|
||||
|
||||
# Install binary
|
||||
sudo cp build/c_relay_x86 /opt/c-relay/
|
||||
sudo chown c-relay:c-relay /opt/c-relay/c_relay_x86
|
||||
sudo chmod +x /opt/c-relay/c_relay_x86
|
||||
sudo cp build/c_relay_pg_x86 /opt/c-relay-pg/
|
||||
sudo chown c-relay-pg:c-relay-pg /opt/c-relay-pg/c_relay_pg_x86
|
||||
sudo chmod +x /opt/c-relay-pg/c_relay_pg_x86
|
||||
|
||||
# Install systemd service
|
||||
sudo cp systemd/c-relay.service /etc/systemd/system/
|
||||
sudo cp systemd/c-relay-pg.service /etc/systemd/system/
|
||||
sudo systemctl daemon-reload
|
||||
```
|
||||
|
||||
@@ -95,22 +95,22 @@ sudo systemctl daemon-reload
|
||||
#### Start and Enable Service
|
||||
```bash
|
||||
# Start the service
|
||||
sudo systemctl start c-relay
|
||||
sudo systemctl start c-relay-pg
|
||||
|
||||
# Enable auto-start on boot
|
||||
sudo systemctl enable c-relay
|
||||
sudo systemctl enable c-relay-pg
|
||||
|
||||
# Check status
|
||||
sudo systemctl status c-relay
|
||||
sudo systemctl status c-relay-pg
|
||||
```
|
||||
|
||||
#### Capture Admin Keys (CRITICAL)
|
||||
```bash
|
||||
# View startup logs to get admin keys
|
||||
sudo journalctl -u c-relay --since "5 minutes ago" | grep -A 10 "IMPORTANT: SAVE THIS ADMIN PRIVATE KEY"
|
||||
sudo journalctl -u c-relay-pg --since "5 minutes ago" | grep -A 10 "IMPORTANT: SAVE THIS ADMIN PRIVATE KEY"
|
||||
|
||||
# Or check the full log
|
||||
sudo journalctl -u c-relay --no-pager | grep "Admin Private Key"
|
||||
sudo journalctl -u c-relay-pg --no-pager | grep "Admin Private Key"
|
||||
```
|
||||
|
||||
⚠️ **CRITICAL**: Save the admin private key immediately - it's only shown once and is needed for all configuration updates!
|
||||
@@ -151,8 +151,8 @@ sudo iptables-save > /etc/iptables/rules.v4
|
||||
ssh -i your-key.pem ubuntu@your-instance-ip
|
||||
|
||||
# Use the simple deployment script
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
sudo examples/deployment/simple-vps/deploy.sh
|
||||
```
|
||||
|
||||
@@ -168,10 +168,10 @@ sudo examples/deployment/simple-vps/deploy.sh
|
||||
sudo mkfs.ext4 /dev/xvdf
|
||||
sudo mkdir /data
|
||||
sudo mount /dev/xvdf /data
|
||||
sudo chown c-relay:c-relay /data
|
||||
sudo chown c-relay-pg:c-relay-pg /data
|
||||
|
||||
# Update systemd service to use /data
|
||||
sudo sed -i 's/WorkingDirectory=\/opt\/c-relay/WorkingDirectory=\/data/' /etc/systemd/system/c-relay.service
|
||||
sudo sed -i 's/WorkingDirectory=\/opt\/c-relay-pg/WorkingDirectory=\/data/' /etc/systemd/system/c-relay-pg.service
|
||||
sudo systemctl daemon-reload
|
||||
```
|
||||
|
||||
@@ -180,7 +180,7 @@ sudo systemctl daemon-reload
|
||||
#### Compute Engine Setup
|
||||
```bash
|
||||
# Create VM instance (e2-micro or larger)
|
||||
gcloud compute instances create c-relay-instance \
|
||||
gcloud compute instances create c-relay-pg-instance \
|
||||
--image-family=ubuntu-2204-lts \
|
||||
--image-project=ubuntu-os-cloud \
|
||||
--machine-type=e2-micro \
|
||||
@@ -193,9 +193,9 @@ gcloud compute firewall-rules create allow-nostr-relay \
|
||||
--target-tags nostr-relay
|
||||
|
||||
# SSH and deploy
|
||||
gcloud compute ssh c-relay-instance
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
gcloud compute ssh c-relay-pg-instance
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
sudo examples/deployment/simple-vps/deploy.sh
|
||||
```
|
||||
|
||||
@@ -203,13 +203,13 @@ sudo examples/deployment/simple-vps/deploy.sh
|
||||
```bash
|
||||
# Create and attach persistent disk
|
||||
gcloud compute disks create relay-data --size=50GB
|
||||
gcloud compute instances attach-disk c-relay-instance --disk=relay-data
|
||||
gcloud compute instances attach-disk c-relay-pg-instance --disk=relay-data
|
||||
|
||||
# Format and mount
|
||||
sudo mkfs.ext4 /dev/sdb
|
||||
sudo mkdir /data
|
||||
sudo mount /dev/sdb /data
|
||||
sudo chown c-relay:c-relay /data
|
||||
sudo chown c-relay-pg:c-relay-pg /data
|
||||
```
|
||||
|
||||
### DigitalOcean
|
||||
@@ -223,8 +223,8 @@ sudo chown c-relay:c-relay /data
|
||||
ssh root@your-droplet-ip
|
||||
|
||||
# Deploy relay
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
examples/deployment/simple-vps/deploy.sh
|
||||
```
|
||||
|
||||
@@ -245,8 +245,8 @@ The `examples/deployment/` directory contains ready-to-use scripts:
|
||||
### Simple VPS Deployment
|
||||
```bash
|
||||
# Clone repository and run automated deployment
|
||||
git clone https://github.com/your-org/c-relay.git
|
||||
cd c-relay
|
||||
git clone https://github.com/your-org/c-relay-pg.git
|
||||
cd c-relay-pg
|
||||
sudo examples/deployment/simple-vps/deploy.sh
|
||||
```
|
||||
|
||||
@@ -417,9 +417,9 @@ LOG_FILE="/var/log/relay-monitor.log"
|
||||
DATE=$(date '+%Y-%m-%d %H:%M:%S')
|
||||
|
||||
# Check if relay is running
|
||||
if ! pgrep -f "c_relay_x86" > /dev/null; then
|
||||
if ! pgrep -f "c_relay_pg_x86" > /dev/null; then
|
||||
echo "[$DATE] ERROR: Relay process not running" >> $LOG_FILE
|
||||
systemctl restart c-relay
|
||||
systemctl restart c-relay-pg
|
||||
fi
|
||||
|
||||
# Check port availability
|
||||
@@ -428,14 +428,14 @@ if ! netstat -tln | grep -q ":8888"; then
|
||||
fi
|
||||
|
||||
# Check database file
|
||||
RELAY_DB=$(find /opt/c-relay -name "*.nrdb" | head -1)
|
||||
RELAY_DB=$(find /opt/c-relay-pg -name "*.nrdb" | head -1)
|
||||
if [[ -n "$RELAY_DB" ]]; then
|
||||
DB_SIZE=$(du -h "$RELAY_DB" | cut -f1)
|
||||
echo "[$DATE] INFO: Database size: $DB_SIZE" >> $LOG_FILE
|
||||
fi
|
||||
|
||||
# Check memory usage
|
||||
MEM_USAGE=$(ps aux | grep c_relay_x86 | grep -v grep | awk '{print $6}')
|
||||
MEM_USAGE=$(ps aux | grep c_relay_pg_x86 | grep -v grep | awk '{print $6}')
|
||||
if [[ -n "$MEM_USAGE" ]]; then
|
||||
echo "[$DATE] INFO: Memory usage: ${MEM_USAGE}KB" >> $LOG_FILE
|
||||
fi
|
||||
@@ -454,8 +454,8 @@ sudo chmod +x /usr/local/bin/relay-monitor.sh
|
||||
|
||||
#### Centralized Logging with rsyslog
|
||||
```bash
|
||||
# /etc/rsyslog.d/50-c-relay.conf
|
||||
if $programname == 'c-relay' then /var/log/c-relay.log
|
||||
# /etc/rsyslog.d/50-c-relay-pg.conf
|
||||
if $programname == 'c-relay-pg' then /var/log/c-relay-pg.log
|
||||
& stop
|
||||
```
|
||||
|
||||
@@ -465,7 +465,7 @@ if $programname == 'c-relay' then /var/log/c-relay.log
|
||||
```yaml
|
||||
# /etc/prometheus/prometheus.yml
|
||||
scrape_configs:
|
||||
- job_name: 'c-relay'
|
||||
- job_name: 'c-relay-pg'
|
||||
static_configs:
|
||||
- targets: ['localhost:8888']
|
||||
metrics_path: '/metrics' # If implemented
|
||||
@@ -479,30 +479,30 @@ scrape_configs:
|
||||
#### Service User Restrictions
|
||||
```bash
|
||||
# Restrict service user
|
||||
sudo usermod -s /bin/false c-relay
|
||||
sudo usermod -d /opt/c-relay c-relay
|
||||
sudo usermod -s /bin/false c-relay-pg
|
||||
sudo usermod -d /opt/c-relay-pg c-relay-pg
|
||||
|
||||
# Set proper permissions
|
||||
sudo chmod 700 /opt/c-relay
|
||||
sudo chown -R c-relay:c-relay /opt/c-relay
|
||||
sudo chmod 700 /opt/c-relay-pg
|
||||
sudo chown -R c-relay-pg:c-relay-pg /opt/c-relay-pg
|
||||
```
|
||||
|
||||
#### File System Restrictions
|
||||
```bash
|
||||
# Mount data directory with appropriate options
|
||||
echo "/dev/sdb /opt/c-relay ext4 defaults,noexec,nosuid,nodev 0 2" >> /etc/fstab
|
||||
echo "/dev/sdb /opt/c-relay-pg ext4 defaults,noexec,nosuid,nodev 0 2" >> /etc/fstab
|
||||
```
|
||||
|
||||
### Network Security
|
||||
|
||||
#### Fail2Ban Configuration
|
||||
```ini
|
||||
# /etc/fail2ban/jail.d/c-relay.conf
|
||||
[c-relay-dos]
|
||||
# /etc/fail2ban/jail.d/c-relay-pg.conf
|
||||
[c-relay-pg-dos]
|
||||
enabled = true
|
||||
port = 8888
|
||||
filter = c-relay-dos
|
||||
logpath = /var/log/c-relay.log
|
||||
filter = c-relay-pg-dos
|
||||
logpath = /var/log/c-relay-pg.log
|
||||
maxretry = 10
|
||||
findtime = 60
|
||||
bantime = 300
|
||||
@@ -534,9 +534,9 @@ sudo mkfs.ext4 /dev/mapper/relay-data
|
||||
#!/bin/bash
|
||||
# /usr/local/bin/backup-relay.sh
|
||||
|
||||
BACKUP_DIR="/backup/c-relay"
|
||||
BACKUP_DIR="/backup/c-relay-pg"
|
||||
DATE=$(date +%Y%m%d_%H%M%S)
|
||||
RELAY_DB=$(find /opt/c-relay -name "*.nrdb" | head -1)
|
||||
RELAY_DB=$(find /opt/c-relay-pg -name "*.nrdb" | head -1)
|
||||
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
|
||||
@@ -574,7 +574,7 @@ sudo apt install -y awscli
|
||||
aws configure
|
||||
|
||||
# Sync backups to S3
|
||||
aws s3 sync /backup/c-relay/ s3://your-backup-bucket/c-relay/ --delete
|
||||
aws s3 sync /backup/c-relay-pg/ s3://your-backup-bucket/c-relay-pg/ --delete
|
||||
```
|
||||
|
||||
### Disaster Recovery
|
||||
@@ -583,16 +583,16 @@ aws s3 sync /backup/c-relay/ s3://your-backup-bucket/c-relay/ --delete
|
||||
```bash
|
||||
# 1. Restore from backup
|
||||
gunzip backup/relay_backup_20231201_020000.nrdb.gz
|
||||
cp backup/relay_backup_20231201_020000.nrdb /opt/c-relay/
|
||||
cp backup/relay_backup_20231201_020000.nrdb /opt/c-relay-pg/
|
||||
|
||||
# 2. Fix permissions
|
||||
sudo chown c-relay:c-relay /opt/c-relay/*.nrdb
|
||||
sudo chown c-relay-pg:c-relay-pg /opt/c-relay-pg/*.nrdb
|
||||
|
||||
# 3. Restart service
|
||||
sudo systemctl restart c-relay
|
||||
sudo systemctl restart c-relay-pg
|
||||
|
||||
# 4. Verify recovery
|
||||
sudo journalctl -u c-relay --since "1 minute ago"
|
||||
sudo journalctl -u c-relay-pg --since "1 minute ago"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
+25
-25
@@ -2,7 +2,7 @@
|
||||
|
||||
## Overview
|
||||
|
||||
This guide explains how to build truly portable MUSL-based static binaries of c-relay using Alpine Linux Docker containers. These binaries have **zero runtime dependencies** and work on any Linux distribution.
|
||||
This guide explains how to build truly portable MUSL-based static binaries of c-relay-pg using Alpine Linux Docker containers. These binaries have **zero runtime dependencies** and work on any Linux distribution.
|
||||
|
||||
## Why MUSL?
|
||||
|
||||
@@ -36,8 +36,8 @@ This guide explains how to build truly portable MUSL-based static binaries of c-
|
||||
./build_static.sh
|
||||
|
||||
# The binary will be created at:
|
||||
# build/c_relay_static_musl_x86_64 (on x86_64)
|
||||
# build/c_relay_static_musl_arm64 (on ARM64)
|
||||
# build/c_relay_pg_static_musl_x86_64 (on x86_64)
|
||||
# build/c_relay_pg_static_musl_arm64 (on ARM64)
|
||||
```
|
||||
|
||||
### What Happens During Build
|
||||
@@ -56,7 +56,7 @@ This guide explains how to build truly portable MUSL-based static binaries of c-
|
||||
- Includes required NIPs: 001, 006, 013, 017, 019, 044, 059
|
||||
- Produces static library (~316KB)
|
||||
|
||||
4. **c-relay Compilation**: Links everything statically:
|
||||
4. **c-relay-pg Compilation**: Links everything statically:
|
||||
- All source files compiled with `-static` flag
|
||||
- Fortification disabled to avoid `__*_chk` symbols
|
||||
- Results in ~7.6MB stripped binary
|
||||
@@ -78,7 +78,7 @@ FROM alpine:3.19 AS builder
|
||||
- Install build tools and static libraries
|
||||
- Build dependencies from source
|
||||
- Compile nostr_core_lib with MUSL flags
|
||||
- Compile c-relay with full static linking
|
||||
- Compile c-relay-pg with full static linking
|
||||
- Strip binary to reduce size
|
||||
|
||||
# Stage 2: Output (scratch)
|
||||
@@ -93,7 +93,7 @@ FROM scratch AS output
|
||||
CFLAGS="-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -Wall -Wextra -std=c99 -fPIC -O2"
|
||||
```
|
||||
|
||||
**For c-relay:**
|
||||
**For c-relay-pg:**
|
||||
```bash
|
||||
gcc -static -O2 -Wall -Wextra -std=c99 \
|
||||
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 \
|
||||
@@ -110,7 +110,7 @@ gcc -static -O2 -Wall -Wextra -std=c99 \
|
||||
- `-U_FORTIFY_SOURCE` (undefine any existing definition)
|
||||
- `-D_FORTIFY_SOURCE=0` (set to 0)
|
||||
|
||||
This must be applied to **both** nostr_core_lib and c-relay compilation.
|
||||
This must be applied to **both** nostr_core_lib and c-relay-pg compilation.
|
||||
|
||||
### NIP Dependencies
|
||||
|
||||
@@ -129,26 +129,26 @@ The build includes these NIPs in nostr_core_lib:
|
||||
|
||||
```bash
|
||||
# Should show "statically linked"
|
||||
file build/c_relay_static_musl_x86_64
|
||||
file build/c_relay_pg_static_musl_x86_64
|
||||
|
||||
# Should show "not a dynamic executable"
|
||||
ldd build/c_relay_static_musl_x86_64
|
||||
ldd build/c_relay_pg_static_musl_x86_64
|
||||
|
||||
# Check size (should be ~7.6MB)
|
||||
ls -lh build/c_relay_static_musl_x86_64
|
||||
ls -lh build/c_relay_pg_static_musl_x86_64
|
||||
```
|
||||
|
||||
### Test Execution
|
||||
|
||||
```bash
|
||||
# Show help
|
||||
./build/c_relay_static_musl_x86_64 --help
|
||||
./build/c_relay_pg_static_musl_x86_64 --help
|
||||
|
||||
# Show version
|
||||
./build/c_relay_static_musl_x86_64 --version
|
||||
./build/c_relay_pg_static_musl_x86_64 --version
|
||||
|
||||
# Run relay
|
||||
./build/c_relay_static_musl_x86_64 --port 8888
|
||||
./build/c_relay_pg_static_musl_x86_64 --port 8888
|
||||
```
|
||||
|
||||
### Cross-Distribution Testing
|
||||
@@ -157,16 +157,16 @@ Test the binary on different distributions to verify portability:
|
||||
|
||||
```bash
|
||||
# Alpine Linux
|
||||
docker run --rm -v $(pwd)/build:/app alpine:latest /app/c_relay_static_musl_x86_64 --version
|
||||
docker run --rm -v $(pwd)/build:/app alpine:latest /app/c_relay_pg_static_musl_x86_64 --version
|
||||
|
||||
# Ubuntu
|
||||
docker run --rm -v $(pwd)/build:/app ubuntu:latest /app/c_relay_static_musl_x86_64 --version
|
||||
docker run --rm -v $(pwd)/build:/app ubuntu:latest /app/c_relay_pg_static_musl_x86_64 --version
|
||||
|
||||
# Debian
|
||||
docker run --rm -v $(pwd)/build:/app debian:latest /app/c_relay_static_musl_x86_64 --version
|
||||
docker run --rm -v $(pwd)/build:/app debian:latest /app/c_relay_pg_static_musl_x86_64 --version
|
||||
|
||||
# CentOS
|
||||
docker run --rm -v $(pwd)/build:/app centos:latest /app/c_relay_static_musl_x86_64 --version
|
||||
docker run --rm -v $(pwd)/build:/app centos:latest /app/c_relay_pg_static_musl_x86_64 --version
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
@@ -187,7 +187,7 @@ newgrp docker # Or logout and login again
|
||||
|
||||
**Solution**: Ensure fortification is disabled in both:
|
||||
1. nostr_core_lib build.sh (line 534)
|
||||
2. c-relay compilation flags in Dockerfile
|
||||
2. c-relay-pg compilation flags in Dockerfile
|
||||
|
||||
### Binary Won't Execute
|
||||
|
||||
@@ -213,26 +213,26 @@ newgrp docker # Or logout and login again
|
||||
|
||||
```bash
|
||||
# Copy binary to server
|
||||
scp build/c_relay_static_musl_x86_64 user@server:/opt/c-relay/
|
||||
scp build/c_relay_pg_static_musl_x86_64 user@server:/opt/c-relay-pg/
|
||||
|
||||
# Run on server (no dependencies needed!)
|
||||
ssh user@server
|
||||
cd /opt/c-relay
|
||||
./c_relay_static_musl_x86_64 --port 8888
|
||||
cd /opt/c-relay-pg
|
||||
./c_relay_pg_static_musl_x86_64 --port 8888
|
||||
```
|
||||
|
||||
### SystemD Service
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=C-Relay Nostr Relay (MUSL Static)
|
||||
Description=C-Relay-PG Nostr Relay (MUSL Static)
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=c-relay
|
||||
WorkingDirectory=/opt/c-relay
|
||||
ExecStart=/opt/c-relay/c_relay_static_musl_x86_64
|
||||
User=c-relay-pg
|
||||
WorkingDirectory=/opt/c-relay-pg
|
||||
ExecStart=/opt/c-relay-pg/c_relay_pg_static_musl_x86_64
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
|
||||
@@ -23,7 +23,7 @@ We chose Option B because:
|
||||
|
||||
## Detailed Implementation Steps
|
||||
|
||||
### Phase 1: Configuration Setup in c-relay
|
||||
### Phase 1: Configuration Setup in c-relay-pg
|
||||
|
||||
#### 1.1 Add Configuration Parameter
|
||||
**File:** `src/default_config_event.h`
|
||||
@@ -293,7 +293,7 @@ int nostr_nip17_send_dm(cJSON* dm_event,
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Update c-relay Call Sites
|
||||
### Phase 4: Update c-relay-pg Call Sites
|
||||
|
||||
#### 4.1 Update src/api.c
|
||||
**Location:** Line 1319
|
||||
@@ -470,7 +470,7 @@ causing compatibility issues.
|
||||
- [ ] Update `nostr_nip17_send_dm()` signature and implementation
|
||||
- [ ] Update `nip017.h` function declaration and documentation
|
||||
|
||||
### c-relay Changes
|
||||
### c-relay-pg Changes
|
||||
- [ ] Add `nip59_timestamp_max_delay_sec` to `default_config_event.h`
|
||||
- [ ] Add validation in `config.c` for new parameter
|
||||
- [ ] Update `src/api.c` call site to pass `max_delay_sec`
|
||||
@@ -494,7 +494,7 @@ causing compatibility issues.
|
||||
|
||||
If issues arise:
|
||||
1. Revert nostr_core_lib changes (git revert in submodule)
|
||||
2. Revert c-relay changes
|
||||
2. Revert c-relay-pg changes
|
||||
3. Configuration parameter will be ignored if not used
|
||||
4. Default behavior (0) provides maximum compatibility
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## Overview
|
||||
|
||||
This document describes the design for a general-purpose SQL query interface for the C-Relay admin API. This allows administrators to execute read-only SQL queries against the relay database through cryptographically signed kind 23456 events with NIP-44 encrypted command arrays.
|
||||
This document describes the design for a general-purpose SQL query interface for the C-Relay-PG admin API. This allows administrators to execute read-only SQL queries against the relay database through cryptographically signed kind 23456 events with NIP-44 encrypted command arrays.
|
||||
|
||||
## Security Model
|
||||
|
||||
@@ -271,7 +271,7 @@ const SQL_QUERY_TEMPLATES = {
|
||||
};
|
||||
|
||||
// Query history management (localStorage)
|
||||
const QUERY_HISTORY_KEY = 'c_relay_sql_history';
|
||||
const QUERY_HISTORY_KEY = 'c_relay_pg_sql_history';
|
||||
const MAX_HISTORY_ITEMS = 20;
|
||||
|
||||
// Load query history from localStorage
|
||||
|
||||
@@ -224,7 +224,7 @@ The script should:
|
||||
- `curl` or `websocat` for WebSocket communication
|
||||
- `jq` for JSON parsing
|
||||
- Nostr CLI tools (optional, for event signing)
|
||||
- Running c-relay instance
|
||||
- Running c-relay-pg instance
|
||||
|
||||
## Example Output
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# C-Relay Complete Startup Flow Documentation
|
||||
# C-Relay-PG Complete Startup Flow Documentation
|
||||
|
||||
## Overview
|
||||
|
||||
C-Relay has two distinct startup paths:
|
||||
C-Relay-PG has two distinct startup paths:
|
||||
1. **First-Time Startup**: No database exists, generates keys and initializes system
|
||||
2. **Existing Relay Startup**: Database exists, loads configuration and resumes operation
|
||||
|
||||
@@ -220,7 +220,7 @@ int populate_default_config_values(void) {
|
||||
|
||||
struct config_default defaults[] = {
|
||||
{"relay_port", "8888", "integer", "WebSocket port", "network", 1},
|
||||
{"relay_name", "C-Relay", "string", "Relay name", "info", 0},
|
||||
{"relay_name", "C-Relay-PG", "string", "Relay name", "info", 0},
|
||||
{"relay_description", "High-performance C Nostr relay", "string", "Description", "info", 0},
|
||||
{"max_subscriptions_per_client", "25", "integer", "Max subs per client", "limits", 0},
|
||||
{"pow_min_difficulty", "0", "integer", "Minimum PoW difficulty", "security", 0},
|
||||
@@ -983,7 +983,7 @@ if (current_version < LATEST_VERSION) {
|
||||
|
||||
**Solutions**:
|
||||
- Use `--port <number>` to specify different port
|
||||
- Kill existing process: `pkill -f c_relay_`
|
||||
- Kill existing process: `pkill -f c_relay_pg_`
|
||||
- Force kill port: `fuser -k 8888/tcp`
|
||||
- Use `--strict-port` to fail fast instead of trying fallback ports
|
||||
|
||||
@@ -994,7 +994,7 @@ if (current_version < LATEST_VERSION) {
|
||||
**Solutions**:
|
||||
- Kill existing relay processes
|
||||
- Remove WAL files: `rm build/*.db-wal build/*.db-shm`
|
||||
- Check for stale processes: `ps aux | grep c_relay_`
|
||||
- Check for stale processes: `ps aux | grep c_relay_pg_`
|
||||
|
||||
#### 3. Missing Admin Private Key
|
||||
|
||||
@@ -1071,7 +1071,7 @@ Typical startup times (on modern hardware):
|
||||
|
||||
## Summary
|
||||
|
||||
The c-relay startup system is designed for:
|
||||
The c-relay-pg startup system is designed for:
|
||||
|
||||
1. **Security**: Admin keys never stored, relay keys encrypted
|
||||
2. **Reliability**: Automatic port fallback, schema migrations
|
||||
|
||||
@@ -18,10 +18,10 @@ The script now attempts to build with `musl-gcc` for truly portable static binar
|
||||
|
||||
SQLite is now built once and cached for future builds:
|
||||
|
||||
- **Cache location**: `~/.cache/c-relay-sqlite/`
|
||||
- **Cache location**: `~/.cache/c-relay-pg-sqlite/`
|
||||
- **Version-specific**: Each SQLite version gets its own cache directory
|
||||
- **Significant speedup**: Subsequent builds skip the SQLite compilation step
|
||||
- **Manual cleanup**: `rm -rf ~/.cache/c-relay-sqlite` to clear cache
|
||||
- **Manual cleanup**: `rm -rf ~/.cache/c-relay-pg-sqlite` to clear cache
|
||||
|
||||
### 3. Smart Package Installation
|
||||
|
||||
@@ -51,13 +51,13 @@ The script will:
|
||||
## Binary Types
|
||||
|
||||
### MUSL Static Binary (Ideal - Currently Not Achievable)
|
||||
- **Filename**: `build/c_relay_static_musl_x86_64`
|
||||
- **Filename**: `build/c_relay_pg_static_musl_x86_64`
|
||||
- **Dependencies**: None (truly static)
|
||||
- **Portability**: Works on any Linux distribution
|
||||
- **Status**: Requires MUSL-compiled libwebsockets and other dependencies (not available by default)
|
||||
|
||||
### Glibc Static Binary (Current Output)
|
||||
- **Filename**: `build/c_relay_static_x86_64` or `build/c_relay_static_glibc_x86_64`
|
||||
- **Filename**: `build/c_relay_pg_static_x86_64` or `build/c_relay_pg_static_glibc_x86_64`
|
||||
- **Dependencies**: None - fully statically linked with glibc
|
||||
- **Portability**: Works on most Linux distributions (glibc is statically included)
|
||||
- **Note**: Despite using glibc, this is a **fully static binary** with no runtime dependencies
|
||||
@@ -68,11 +68,11 @@ The script automatically verifies binaries using `ldd` and `file`:
|
||||
|
||||
```bash
|
||||
# For MUSL binary
|
||||
ldd build/c_relay_static_musl_x86_64
|
||||
ldd build/c_relay_pg_static_musl_x86_64
|
||||
# Output: "not a dynamic executable" (good!)
|
||||
|
||||
# For glibc binary
|
||||
ldd build/c_relay_static_glibc_x86_64
|
||||
ldd build/c_relay_pg_static_glibc_x86_64
|
||||
# Output: Shows glibc dependencies
|
||||
```
|
||||
|
||||
@@ -112,7 +112,7 @@ The script attempts MUSL compilation but falls back to glibc:
|
||||
|
||||
### Clear SQLite Cache
|
||||
```bash
|
||||
rm -rf ~/.cache/c-relay-sqlite
|
||||
rm -rf ~/.cache/c-relay-pg-sqlite
|
||||
```
|
||||
|
||||
### Force Package Reinstall
|
||||
@@ -127,8 +127,8 @@ cat /tmp/musl_build.log
|
||||
|
||||
### Verify Binary Type
|
||||
```bash
|
||||
file build/c_relay_static_*
|
||||
ldd build/c_relay_static_* 2>&1
|
||||
file build/c_relay_pg_static_*
|
||||
ldd build/c_relay_pg_static_* 2>&1
|
||||
```
|
||||
|
||||
## Performance Impact
|
||||
|
||||
@@ -0,0 +1,532 @@
|
||||
# Subscription Cleanup - Simplified Design
|
||||
|
||||
## Problem Summary
|
||||
|
||||
The c-relay-pg Nostr relay experienced severe performance degradation (90-100% CPU) due to subscription accumulation in the database. Investigation revealed **323,644 orphaned subscriptions** that were never properly closed when WebSocket connections dropped.
|
||||
|
||||
## Solution: Two-Component Approach
|
||||
|
||||
This simplified design focuses on two pragmatic solutions that align with Nostr's stateless design:
|
||||
|
||||
1. **Startup Cleanup**: Close all subscriptions on relay restart
|
||||
2. **Connection Age Limit**: Disconnect clients after a configurable time period
|
||||
|
||||
Both solutions force clients to reconnect and re-establish subscriptions, which is standard Nostr behavior.
|
||||
|
||||
---
|
||||
|
||||
## Component 1: Startup Cleanup
|
||||
|
||||
### Purpose
|
||||
Ensure clean state on every relay restart by closing all subscriptions in the database.
|
||||
|
||||
### Implementation
|
||||
|
||||
**File:** [`src/subscriptions.c`](src/subscriptions.c)
|
||||
|
||||
**New Function:**
|
||||
```c
|
||||
void cleanup_all_subscriptions_on_startup(void) {
|
||||
if (!g_db) {
|
||||
DEBUG_ERROR("Database not initialized for startup cleanup");
|
||||
return;
|
||||
}
|
||||
|
||||
DEBUG_LOG("Startup cleanup: Marking all active subscriptions as disconnected");
|
||||
|
||||
// Mark all 'created' subscriptions as disconnected
|
||||
const char* update_sql =
|
||||
"UPDATE subscriptions "
|
||||
"SET ended_at = strftime('%s', 'now') "
|
||||
"WHERE event_type = 'created' AND ended_at IS NULL";
|
||||
|
||||
sqlite3_stmt* stmt;
|
||||
int rc = sqlite3_prepare_v2(g_db, update_sql, -1, &stmt, NULL);
|
||||
if (rc != SQLITE_OK) {
|
||||
DEBUG_ERROR("Failed to prepare startup cleanup query: %s", sqlite3_errmsg(g_db));
|
||||
return;
|
||||
}
|
||||
|
||||
rc = sqlite3_step(stmt);
|
||||
int updated_count = sqlite3_changes(g_db);
|
||||
sqlite3_finalize(stmt);
|
||||
|
||||
if (updated_count > 0) {
|
||||
// Log a single 'disconnected' event for the startup cleanup
|
||||
const char* insert_sql =
|
||||
"INSERT INTO subscriptions (subscription_id, wsi_pointer, client_ip, event_type) "
|
||||
"VALUES ('startup_cleanup', '', 'system', 'disconnected')";
|
||||
|
||||
rc = sqlite3_prepare_v2(g_db, insert_sql, -1, &stmt, NULL);
|
||||
if (rc == SQLITE_OK) {
|
||||
sqlite3_step(stmt);
|
||||
sqlite3_finalize(stmt);
|
||||
}
|
||||
|
||||
DEBUG_LOG("Startup cleanup: Marked %d subscriptions as disconnected", updated_count);
|
||||
} else {
|
||||
DEBUG_LOG("Startup cleanup: No active subscriptions found");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Integration Point:** [`src/main.c:1810`](src/main.c:1810)
|
||||
|
||||
```c
|
||||
// Initialize subscription manager mutexes
|
||||
if (pthread_mutex_init(&g_subscription_manager.subscriptions_lock, NULL) != 0) {
|
||||
DEBUG_ERROR("Failed to initialize subscriptions mutex");
|
||||
sqlite3_close(g_db);
|
||||
return 1;
|
||||
}
|
||||
|
||||
if (pthread_mutex_init(&g_subscription_manager.ip_tracking_lock, NULL) != 0) {
|
||||
DEBUG_ERROR("Failed to initialize IP tracking mutex");
|
||||
pthread_mutex_destroy(&g_subscription_manager.subscriptions_lock);
|
||||
sqlite3_close(g_db);
|
||||
return 1;
|
||||
}
|
||||
|
||||
// **NEW: Startup cleanup - close all subscriptions**
|
||||
cleanup_all_subscriptions_on_startup();
|
||||
|
||||
// Start WebSocket relay server
|
||||
DEBUG_LOG("Starting WebSocket relay server...");
|
||||
if (start_websocket_relay(port_override, strict_port) != 0) {
|
||||
DEBUG_ERROR("Failed to start WebSocket relay");
|
||||
// ... cleanup code
|
||||
}
|
||||
```
|
||||
|
||||
### Benefits
|
||||
- **Immediate relief**: Restart relay to fix subscription issues
|
||||
- **Clean slate**: Every restart starts with zero active subscriptions
|
||||
- **Simple**: Single SQL UPDATE statement
|
||||
- **Nostr-aligned**: Clients are designed to reconnect after relay restart
|
||||
- **No configuration needed**: Always runs on startup
|
||||
|
||||
---
|
||||
|
||||
## Component 2: Connection Age Limit
|
||||
|
||||
### Purpose
|
||||
Automatically disconnect clients after a configurable time period, forcing them to reconnect and re-establish subscriptions.
|
||||
|
||||
### Why Disconnect Instead of Just Closing Subscriptions?
|
||||
|
||||
**Option 1: Send CLOSED message (keep connection)**
|
||||
- ❌ Not all clients handle `CLOSED` messages properly
|
||||
- ❌ Silent failure - client thinks it's subscribed but isn't
|
||||
- ❌ Partial cleanup - connection still consumes resources
|
||||
- ❌ More complex to implement
|
||||
|
||||
**Option 2: Disconnect client entirely (force reconnection)** ✅
|
||||
- ✅ Universal compatibility - all clients handle WebSocket reconnection
|
||||
- ✅ Complete resource cleanup (memory, file descriptors, etc.)
|
||||
- ✅ Simple implementation - single operation
|
||||
- ✅ Well-tested code path (same as network interruptions)
|
||||
- ✅ Forces re-authentication if needed
|
||||
|
||||
### Implementation
|
||||
|
||||
**File:** [`src/websockets.c`](src/websockets.c)
|
||||
|
||||
**New Function:**
|
||||
```c
|
||||
/**
|
||||
* Check connection age and disconnect clients that have been connected too long.
|
||||
* This forces clients to reconnect and re-establish subscriptions.
|
||||
*
|
||||
* Uses libwebsockets' lws_vhost_foreach_wsi() to iterate through all active
|
||||
* connections and checks their connection_established timestamp from per_session_data.
|
||||
*/
|
||||
void check_connection_age(int max_connection_seconds) {
|
||||
if (max_connection_seconds <= 0 || !ws_context) {
|
||||
return;
|
||||
}
|
||||
|
||||
time_t now = time(NULL);
|
||||
time_t cutoff = now - max_connection_seconds;
|
||||
|
||||
// Get the default vhost
|
||||
struct lws_vhost *vhost = lws_get_vhost_by_name(ws_context, "default");
|
||||
if (!vhost) {
|
||||
DEBUG_ERROR("Failed to get vhost for connection age check");
|
||||
return;
|
||||
}
|
||||
|
||||
// Iterate through all active WebSocket connections
|
||||
// Note: lws_vhost_foreach_wsi() calls our callback for each connection
|
||||
struct lws *wsi = NULL;
|
||||
while ((wsi = lws_vhost_foreach_wsi(vhost, wsi)) != NULL) {
|
||||
// Get per-session data which contains connection_established timestamp
|
||||
struct per_session_data *pss = (struct per_session_data *)lws_wsi_user(wsi);
|
||||
|
||||
if (pss && pss->connection_established > 0) {
|
||||
// Check if connection is older than cutoff
|
||||
if (pss->connection_established < cutoff) {
|
||||
// Connection is too old - close it
|
||||
long age_seconds = now - pss->connection_established;
|
||||
|
||||
DEBUG_LOG("Closing connection from %s (age: %lds, limit: %ds)",
|
||||
pss->client_ip, age_seconds, max_connection_seconds);
|
||||
|
||||
// Close with normal status and reason message
|
||||
lws_close_reason(wsi, LWS_CLOSE_STATUS_NORMAL,
|
||||
(unsigned char *)"connection age limit", 21);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Key Implementation Details:**
|
||||
|
||||
1. **No database needed**: Active connections are tracked by libwebsockets itself
|
||||
2. **Uses existing timestamp**: `pss->connection_established` is already set on line 456 of websockets.c
|
||||
3. **Built-in iterator**: `lws_vhost_foreach_wsi()` safely iterates through all active connections
|
||||
4. **Per-session data**: Each connection's `per_session_data` is accessible via `lws_wsi_user()`
|
||||
5. **Safe closure**: `lws_close_reason()` properly closes the WebSocket with a status code and message
|
||||
|
||||
**Integration Point:** [`src/websockets.c:2176`](src/websockets.c:2176) - in existing event loop
|
||||
|
||||
```c
|
||||
// Main event loop with proper signal handling
|
||||
while (g_server_running && !g_shutdown_flag) {
|
||||
int result = lws_service(ws_context, 1000);
|
||||
|
||||
if (result < 0) {
|
||||
DEBUG_ERROR("libwebsockets service error");
|
||||
break;
|
||||
}
|
||||
|
||||
// Check if it's time to post status update
|
||||
time_t current_time = time(NULL);
|
||||
int status_post_hours = get_config_int("kind_1_status_posts_hours", 0);
|
||||
|
||||
if (status_post_hours > 0) {
|
||||
int seconds_interval = status_post_hours * 3600;
|
||||
if (current_time - last_status_post_time >= seconds_interval) {
|
||||
last_status_post_time = current_time;
|
||||
generate_and_post_status_event();
|
||||
}
|
||||
}
|
||||
|
||||
// **NEW: Check for connection age limit**
|
||||
int max_connection_seconds = get_config_int("max_connection_seconds", 86400); // Default 24 hours
|
||||
if (max_connection_seconds > 0) {
|
||||
check_connection_age(max_connection_seconds);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
**Parameter:** `max_connection_seconds`
|
||||
- **Default:** `86400` (24 hours)
|
||||
- **Range:** `0` = disabled, `>0` = disconnect after X seconds
|
||||
- **Units:** Seconds (for consistency with other time-based configs)
|
||||
|
||||
**Example configurations:**
|
||||
```json
|
||||
{
|
||||
"max_connection_seconds": 86400 // 86400 seconds = 24 hours (default)
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"max_connection_seconds": 43200 // 43200 seconds = 12 hours
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"max_connection_seconds": 3600 // 3600 seconds = 1 hour
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"max_connection_seconds": 0 // Disabled
|
||||
}
|
||||
```
|
||||
|
||||
### Client Behavior
|
||||
|
||||
When disconnected due to age limit, clients will:
|
||||
1. Detect WebSocket closure
|
||||
2. Wait briefly (exponential backoff)
|
||||
3. Reconnect to relay
|
||||
4. Re-authenticate if needed (NIP-42)
|
||||
5. Re-establish all subscriptions
|
||||
6. Resume normal operation
|
||||
|
||||
This is **exactly what happens** during network interruptions, so it's a well-tested code path in all Nostr clients.
|
||||
|
||||
### Benefits
|
||||
- **No new threads**: Uses existing event loop
|
||||
- **Minimal overhead**: Check runs once per second (same as `lws_service`)
|
||||
- **Simple implementation**: Iterate through active connections
|
||||
- **Consistent pattern**: Matches existing status post checking
|
||||
- **Universal compatibility**: All clients handle reconnection
|
||||
- **Complete cleanup**: Frees all resources associated with connection
|
||||
- **Configurable**: Can be adjusted per relay needs or disabled entirely
|
||||
|
||||
---
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: Startup Cleanup (1-2 hours)
|
||||
|
||||
1. **Add `cleanup_all_subscriptions_on_startup()` function**
|
||||
- File: [`src/subscriptions.c`](src/subscriptions.c)
|
||||
- SQL UPDATE to mark all active subscriptions as disconnected
|
||||
- Add logging for cleanup count
|
||||
|
||||
2. **Integrate in main()**
|
||||
- File: [`src/main.c:1810`](src/main.c:1810)
|
||||
- Call after mutex initialization, before WebSocket server start
|
||||
|
||||
3. **Test**
|
||||
- Create subscriptions in database
|
||||
- Restart relay
|
||||
- Verify all subscriptions marked as disconnected
|
||||
- Verify `active_subscriptions_log` shows 0 subscriptions
|
||||
|
||||
**Estimated Time:** 1-2 hours
|
||||
|
||||
### Phase 2: Connection Age Limit (2-3 hours)
|
||||
|
||||
1. **Add `check_connection_age()` function**
|
||||
- File: [`src/websockets.c`](src/websockets.c)
|
||||
- Iterate through active connections
|
||||
- Close connections older than limit
|
||||
|
||||
2. **Integrate in event loop**
|
||||
- File: [`src/websockets.c:2176`](src/websockets.c:2176)
|
||||
- Add check after status post check
|
||||
- Use same pattern as status posts
|
||||
|
||||
3. **Add configuration parameter**
|
||||
- Add `max_connection_seconds` to default config
|
||||
- Default: 86400 (24 hours)
|
||||
|
||||
4. **Test**
|
||||
- Connect client
|
||||
- Wait for timeout (or reduce timeout for testing)
|
||||
- Verify client disconnected
|
||||
- Verify client reconnects automatically
|
||||
- Verify subscriptions re-established
|
||||
|
||||
**Estimated Time:** 2-3 hours
|
||||
|
||||
---
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Startup Cleanup Tests
|
||||
|
||||
```bash
|
||||
# Test 1: Clean startup with existing subscriptions
|
||||
- Create 100 active subscriptions in database
|
||||
- Restart relay
|
||||
- Verify all subscriptions marked as disconnected
|
||||
- Verify active_subscriptions_log shows 0 subscriptions
|
||||
|
||||
# Test 2: Clean startup with no subscriptions
|
||||
- Start relay with empty database
|
||||
- Verify no errors
|
||||
- Verify startup cleanup logs "No active subscriptions found"
|
||||
|
||||
# Test 3: Clients reconnect after restart
|
||||
- Create subscriptions before restart
|
||||
- Restart relay
|
||||
- Connect clients and create new subscriptions
|
||||
- Verify new subscriptions tracked correctly
|
||||
```
|
||||
|
||||
### Connection Age Limit Tests
|
||||
|
||||
```bash
|
||||
# Test 1: Connection disconnected after timeout
|
||||
- Set max_connection_seconds to 60 (for testing)
|
||||
- Connect client
|
||||
- Wait 61 seconds
|
||||
- Verify client disconnected
|
||||
- Verify client reconnects automatically
|
||||
|
||||
# Test 2: Subscriptions re-established after reconnection
|
||||
- Connect client with subscriptions
|
||||
- Wait for timeout
|
||||
- Verify client reconnects
|
||||
- Verify subscriptions re-established
|
||||
- Verify events still delivered
|
||||
|
||||
# Test 3: Disabled when set to 0
|
||||
- Set max_connection_seconds to 0
|
||||
- Connect client
|
||||
- Wait extended period
|
||||
- Verify client NOT disconnected
|
||||
```
|
||||
|
||||
### Integration Tests
|
||||
|
||||
```bash
|
||||
# Test 1: Combined behavior
|
||||
- Start relay (startup cleanup runs)
|
||||
- Connect multiple clients
|
||||
- Create subscriptions
|
||||
- Wait for connection timeout
|
||||
- Verify clients reconnect
|
||||
- Restart relay
|
||||
- Verify clean state
|
||||
|
||||
# Test 2: Load test
|
||||
- Connect 100 clients
|
||||
- Each creates 5 subscriptions
|
||||
- Wait for connection timeout
|
||||
- Verify all clients reconnect
|
||||
- Verify all subscriptions re-established
|
||||
- Monitor CPU usage (should remain low)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
### Component 1: Startup Cleanup
|
||||
- ✅ Relay starts with zero active subscriptions
|
||||
- ✅ All previous subscriptions marked as disconnected on startup
|
||||
- ✅ Clients successfully reconnect and re-establish subscriptions
|
||||
- ✅ Relay restart can be used as emergency fix for subscription issues
|
||||
- ✅ No errors during startup cleanup process
|
||||
|
||||
### Component 2: Connection Age Limit
|
||||
- ✅ Clients disconnected after configured time period
|
||||
- ✅ Clients automatically reconnect
|
||||
- ✅ Subscriptions re-established after reconnection
|
||||
- ✅ No impact on relay performance
|
||||
- ✅ Configuration parameter works correctly (including disabled state)
|
||||
|
||||
### Overall Success
|
||||
- ✅ CPU usage remains low (<10%)
|
||||
- ✅ No orphaned subscriptions accumulate
|
||||
- ✅ Database size remains stable
|
||||
- ✅ No manual intervention required
|
||||
|
||||
---
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
**New Configuration Parameters:**
|
||||
|
||||
```json
|
||||
{
|
||||
"max_connection_seconds": 86400
|
||||
}
|
||||
```
|
||||
|
||||
**Recommended Settings:**
|
||||
|
||||
- **Production:**
|
||||
- `max_connection_seconds: 86400` (24 hours)
|
||||
|
||||
- **Development:**
|
||||
- `max_connection_seconds: 3600` (1 hour for faster testing)
|
||||
|
||||
- **High-traffic:**
|
||||
- `max_connection_seconds: 43200` (12 hours)
|
||||
|
||||
- **Disabled:**
|
||||
- `max_connection_seconds: 0`
|
||||
|
||||
---
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
If issues arise after deployment:
|
||||
|
||||
1. **Disable connection age limit:**
|
||||
- Set `max_connection_seconds: 0` in config
|
||||
- Restart relay
|
||||
- Monitor for stability
|
||||
|
||||
2. **Revert code changes:**
|
||||
- Remove `check_connection_age()` call from event loop
|
||||
- Remove `cleanup_all_subscriptions_on_startup()` call from main
|
||||
- Restart relay
|
||||
|
||||
3. **Database cleanup (if needed):**
|
||||
- Manually clean up orphaned subscriptions using SQL:
|
||||
```sql
|
||||
UPDATE subscriptions
|
||||
SET ended_at = strftime('%s', 'now')
|
||||
WHERE event_type = 'created' AND ended_at IS NULL;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Comparison with Original Design
|
||||
|
||||
### Original Design (5 Components)
|
||||
1. Startup cleanup
|
||||
2. Fix WebSocket disconnection logging
|
||||
3. Enhance subscription removal with reason parameter
|
||||
4. Periodic cleanup task (background thread)
|
||||
5. Optimize database VIEW
|
||||
6. Subscription expiration (optional)
|
||||
|
||||
### Simplified Design (2 Components)
|
||||
1. Startup cleanup
|
||||
2. Connection age limit
|
||||
|
||||
### Why Simplified is Better
|
||||
|
||||
**Advantages:**
|
||||
- **Simpler**: 2 components vs 5-6 components
|
||||
- **Faster to implement**: 3-5 hours vs 11-17 hours
|
||||
- **Easier to maintain**: Less code, fewer moving parts
|
||||
- **More reliable**: Fewer potential failure points
|
||||
- **Nostr-aligned**: Leverages client reconnection behavior
|
||||
- **No new threads**: Uses existing event loop
|
||||
- **Universal compatibility**: All clients handle reconnection
|
||||
|
||||
**What We're Not Losing:**
|
||||
- Startup cleanup is identical in both designs
|
||||
- Connection age limit achieves the same goal as periodic cleanup + expiration
|
||||
- Disconnection forces complete cleanup (better than just logging)
|
||||
- Database VIEW optimization not needed if subscriptions don't accumulate
|
||||
|
||||
**Trade-offs:**
|
||||
- Less granular logging (but simpler)
|
||||
- No historical subscription analytics (but cleaner database)
|
||||
- Clients must reconnect periodically (but this is standard Nostr behavior)
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
This simplified design solves the subscription accumulation problem with two pragmatic solutions:
|
||||
|
||||
1. **Startup cleanup** ensures every relay restart starts with a clean slate
|
||||
2. **Connection age limit** prevents long-term accumulation by forcing periodic reconnection
|
||||
|
||||
Both solutions align with Nostr's stateless design where clients are expected to handle reconnection. The implementation is simple, maintainable, and leverages existing code patterns.
|
||||
|
||||
**Key Benefits:**
|
||||
- ✅ Solves the root problem (subscription accumulation)
|
||||
- ✅ Simple to implement (3-5 hours total)
|
||||
- ✅ Easy to maintain (minimal code)
|
||||
- ✅ Universal compatibility (all clients handle reconnection)
|
||||
- ✅ No new threads or background tasks
|
||||
- ✅ Configurable and can be disabled if needed
|
||||
- ✅ Relay restart as emergency fix
|
||||
|
||||
**Next Steps:**
|
||||
1. Implement Component 1 (Startup Cleanup)
|
||||
2. Test thoroughly
|
||||
3. Implement Component 2 (Connection Age Limit)
|
||||
4. Test thoroughly
|
||||
5. Deploy to production
|
||||
6. Monitor CPU usage and subscription counts
|
||||
@@ -0,0 +1,358 @@
|
||||
# Subscription Table Collapsible Groups Implementation Plan
|
||||
|
||||
## Objective
|
||||
Add collapsible/expandable functionality to subscription groups, where:
|
||||
1. Each WSI Pointer group starts collapsed, showing only a summary row
|
||||
2. Clicking the summary row expands to show all subscription details for that WSI Pointer
|
||||
3. Clicking again collapses the group back to the summary row
|
||||
|
||||
## Current Behavior
|
||||
- All subscriptions are displayed in a flat list
|
||||
- WSI Pointer value shown only on first row of each group
|
||||
- No interaction or collapse functionality
|
||||
|
||||
## Desired Behavior
|
||||
- **Collapsed state**: One row per WSI Pointer showing summary information
|
||||
- **Expanded state**: Header row + detail rows for each subscription in that group
|
||||
- **Toggle**: Click on header row to expand/collapse
|
||||
- **Visual indicator**: Arrow or icon showing expand/collapse state
|
||||
|
||||
## Design Approach
|
||||
|
||||
### Option 1: Summary Row + Detail Rows (Recommended)
|
||||
```
|
||||
[▶] WSI Pointer: 0x12345678 | Subscriptions: 3 | Total Duration: 15m
|
||||
(detail rows hidden)
|
||||
|
||||
When clicked:
|
||||
[▼] WSI Pointer: 0x12345678 | Subscriptions: 3 | Total Duration: 15m
|
||||
| sub-001 | 5m 30s | kinds:[1]
|
||||
| sub-002 | 3m 15s | kinds:[3]
|
||||
| sub-003 | 6m 15s | kinds:[1,3]
|
||||
```
|
||||
|
||||
### Option 2: First Row as Header (Alternative)
|
||||
```
|
||||
[▶] 0x12345678 | sub-001 | 5m 30s | kinds:[1] (+ 2 more)
|
||||
|
||||
When clicked:
|
||||
[▼] 0x12345678 | sub-001 | 5m 30s | kinds:[1]
|
||||
| sub-002 | 3m 15s | kinds:[3]
|
||||
| sub-003 | 6m 15s | kinds:[1,3]
|
||||
```
|
||||
|
||||
**Recommendation**: Option 1 provides clearer UX and better summary information.
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### Files to Modify
|
||||
1. `api/index.js` - Function `populateSubscriptionDetailsTable()` (lines 4277-4384)
|
||||
2. `api/index.css` - Add styles for collapsible rows
|
||||
|
||||
### Data Structure Changes
|
||||
|
||||
Need to group subscriptions by WSI Pointer first:
|
||||
```javascript
|
||||
// Group subscriptions by wsi_pointer
|
||||
const groupedSubscriptions = {};
|
||||
subscriptionsData.forEach(sub => {
|
||||
const wsiKey = sub.wsi_pointer || 'N/A';
|
||||
if (!groupedSubscriptions[wsiKey]) {
|
||||
groupedSubscriptions[wsiKey] = [];
|
||||
}
|
||||
groupedSubscriptions[wsiKey].push(sub);
|
||||
});
|
||||
```
|
||||
|
||||
### HTML Structure
|
||||
|
||||
#### Summary Row (Header)
|
||||
```html
|
||||
<tr class="subscription-group-header" data-wsi-pointer="0x12345678" data-expanded="false">
|
||||
<td colspan="4" style="cursor: pointer; user-select: none;">
|
||||
<span class="expand-icon">▶</span>
|
||||
<strong>WSI Pointer:</strong> 0x12345678
|
||||
<span class="group-summary">
|
||||
| Subscriptions: 3 | Oldest: 15m 30s
|
||||
</span>
|
||||
</td>
|
||||
</tr>
|
||||
```
|
||||
|
||||
#### Detail Rows (Initially Hidden)
|
||||
```html
|
||||
<tr class="subscription-detail-row" data-wsi-group="0x12345678" style="display: none;">
|
||||
<td style="padding-left: 30px;"><!-- Empty for WSI Pointer --></td>
|
||||
<td>sub-001</td>
|
||||
<td>5m 30s</td>
|
||||
<td>kinds:[1]</td>
|
||||
</tr>
|
||||
```
|
||||
|
||||
### JavaScript Implementation
|
||||
|
||||
#### Modified populateSubscriptionDetailsTable Function
|
||||
|
||||
```javascript
|
||||
function populateSubscriptionDetailsTable(subscriptionsData) {
|
||||
const tableBody = document.getElementById('subscription-details-table-body');
|
||||
if (!tableBody || !subscriptionsData || !Array.isArray(subscriptionsData)) return;
|
||||
|
||||
tableBody.innerHTML = '';
|
||||
|
||||
if (subscriptionsData.length === 0) {
|
||||
const row = document.createElement('tr');
|
||||
row.innerHTML = '<td colspan="4" style="text-align: center; font-style: italic;">No active subscriptions</td>';
|
||||
tableBody.appendChild(row);
|
||||
return;
|
||||
}
|
||||
|
||||
// Sort subscriptions by wsi_pointer to group them together
|
||||
subscriptionsData.sort((a, b) => {
|
||||
const wsiA = a.wsi_pointer || '';
|
||||
const wsiB = b.wsi_pointer || '';
|
||||
return wsiA.localeCompare(wsiB);
|
||||
});
|
||||
|
||||
// Group subscriptions by wsi_pointer
|
||||
const groupedSubscriptions = {};
|
||||
subscriptionsData.forEach(sub => {
|
||||
const wsiKey = sub.wsi_pointer || 'N/A';
|
||||
if (!groupedSubscriptions[wsiKey]) {
|
||||
groupedSubscriptions[wsiKey] = [];
|
||||
}
|
||||
groupedSubscriptions[wsiKey].push(sub);
|
||||
});
|
||||
|
||||
// Create rows for each group
|
||||
Object.entries(groupedSubscriptions).forEach(([wsiPointer, subscriptions]) => {
|
||||
// Calculate group summary
|
||||
const subCount = subscriptions.length;
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
const oldestDuration = Math.max(...subscriptions.map(s => now - s.created_at));
|
||||
const oldestDurationStr = formatDuration(oldestDuration);
|
||||
|
||||
// Create header row (summary)
|
||||
const headerRow = document.createElement('tr');
|
||||
headerRow.className = 'subscription-group-header';
|
||||
headerRow.setAttribute('data-wsi-pointer', wsiPointer);
|
||||
headerRow.setAttribute('data-expanded', 'false');
|
||||
headerRow.style.cursor = 'pointer';
|
||||
headerRow.style.userSelect = 'none';
|
||||
headerRow.style.backgroundColor = 'var(--hover-color, #f5f5f5)';
|
||||
|
||||
headerRow.innerHTML = `
|
||||
<td colspan="4" style="padding: 8px;">
|
||||
<span class="expand-icon" style="display: inline-block; width: 20px; transition: transform 0.2s;">▶</span>
|
||||
<strong style="font-family: 'Courier New', monospace; font-size: 12px;">WSI: ${wsiPointer}</strong>
|
||||
<span style="color: #666; margin-left: 15px;">
|
||||
Subscriptions: ${subCount} | Oldest: ${oldestDurationStr}
|
||||
</span>
|
||||
</td>
|
||||
`;
|
||||
|
||||
// Add click handler to toggle expansion
|
||||
headerRow.addEventListener('click', () => toggleSubscriptionGroup(wsiPointer));
|
||||
|
||||
tableBody.appendChild(headerRow);
|
||||
|
||||
// Create detail rows (initially hidden)
|
||||
subscriptions.forEach((subscription, index) => {
|
||||
const detailRow = document.createElement('tr');
|
||||
detailRow.className = 'subscription-detail-row';
|
||||
detailRow.setAttribute('data-wsi-group', wsiPointer);
|
||||
detailRow.style.display = 'none';
|
||||
|
||||
// Calculate duration
|
||||
const duration = now - subscription.created_at;
|
||||
const durationStr = formatDuration(duration);
|
||||
|
||||
// Format filters
|
||||
let filtersDisplay = 'None';
|
||||
if (subscription.filters && subscription.filters.length > 0) {
|
||||
const filterDetails = [];
|
||||
subscription.filters.forEach((filter) => {
|
||||
const parts = [];
|
||||
|
||||
if (filter.kinds && Array.isArray(filter.kinds) && filter.kinds.length > 0) {
|
||||
parts.push(`kinds:[${filter.kinds.join(',')}]`);
|
||||
}
|
||||
|
||||
if (filter.authors && Array.isArray(filter.authors) && filter.authors.length > 0) {
|
||||
const authorCount = filter.authors.length;
|
||||
if (authorCount === 1) {
|
||||
const shortPubkey = filter.authors[0].substring(0, 8) + '...';
|
||||
parts.push(`authors:[${shortPubkey}]`);
|
||||
} else {
|
||||
parts.push(`authors:[${authorCount} pubkeys]`);
|
||||
}
|
||||
}
|
||||
|
||||
if (filter.ids && Array.isArray(filter.ids) && filter.ids.length > 0) {
|
||||
const idCount = filter.ids.length;
|
||||
parts.push(`ids:[${idCount} event${idCount > 1 ? 's' : ''}]`);
|
||||
}
|
||||
|
||||
const timeParts = [];
|
||||
if (filter.since && filter.since > 0) {
|
||||
const sinceDate = new Date(filter.since * 1000).toLocaleString();
|
||||
timeParts.push(`since:${sinceDate}`);
|
||||
}
|
||||
if (filter.until && filter.until > 0) {
|
||||
const untilDate = new Date(filter.until * 1000).toLocaleString();
|
||||
timeParts.push(`until:${untilDate}`);
|
||||
}
|
||||
if (timeParts.length > 0) {
|
||||
parts.push(timeParts.join(', '));
|
||||
}
|
||||
|
||||
if (filter.limit && filter.limit > 0) {
|
||||
parts.push(`limit:${filter.limit}`);
|
||||
}
|
||||
|
||||
if (filter.tag_filters && Array.isArray(filter.tag_filters) && filter.tag_filters.length > 0) {
|
||||
parts.push(`tags:[${filter.tag_filters.length} filter${filter.tag_filters.length > 1 ? 's' : ''}]`);
|
||||
}
|
||||
|
||||
if (parts.length > 0) {
|
||||
filterDetails.push(parts.join(', '));
|
||||
} else {
|
||||
filterDetails.push('empty filter');
|
||||
}
|
||||
});
|
||||
|
||||
filtersDisplay = filterDetails.join(' | ');
|
||||
}
|
||||
|
||||
detailRow.innerHTML = `
|
||||
<td style="padding-left: 30px; font-family: 'Courier New', monospace; font-size: 11px; color: #999;">└─</td>
|
||||
<td style="font-family: 'Courier New', monospace; font-size: 12px;">${subscription.id || 'N/A'}</td>
|
||||
<td>${durationStr}</td>
|
||||
<td>${filtersDisplay}</td>
|
||||
`;
|
||||
|
||||
tableBody.appendChild(detailRow);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// Toggle function for expanding/collapsing groups
|
||||
function toggleSubscriptionGroup(wsiPointer) {
|
||||
const headerRow = document.querySelector(`.subscription-group-header[data-wsi-pointer="${wsiPointer}"]`);
|
||||
const detailRows = document.querySelectorAll(`.subscription-detail-row[data-wsi-group="${wsiPointer}"]`);
|
||||
const expandIcon = headerRow.querySelector('.expand-icon');
|
||||
|
||||
const isExpanded = headerRow.getAttribute('data-expanded') === 'true';
|
||||
|
||||
if (isExpanded) {
|
||||
// Collapse
|
||||
detailRows.forEach(row => row.style.display = 'none');
|
||||
expandIcon.textContent = '▶';
|
||||
expandIcon.style.transform = 'rotate(0deg)';
|
||||
headerRow.setAttribute('data-expanded', 'false');
|
||||
} else {
|
||||
// Expand
|
||||
detailRows.forEach(row => row.style.display = 'table-row');
|
||||
expandIcon.textContent = '▼';
|
||||
expandIcon.style.transform = 'rotate(90deg)';
|
||||
headerRow.setAttribute('data-expanded', 'true');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### CSS Additions (api/index.css)
|
||||
|
||||
```css
|
||||
/* Subscription group header styles */
|
||||
.subscription-group-header {
|
||||
background-color: var(--hover-color, #f5f5f5);
|
||||
font-weight: 500;
|
||||
transition: background-color 0.2s;
|
||||
}
|
||||
|
||||
.subscription-group-header:hover {
|
||||
background-color: var(--primary-color-light, #e8e8e8);
|
||||
}
|
||||
|
||||
.expand-icon {
|
||||
display: inline-block;
|
||||
width: 20px;
|
||||
transition: transform 0.2s ease;
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
/* Detail row styles */
|
||||
.subscription-detail-row {
|
||||
background-color: var(--background-color, #ffffff);
|
||||
}
|
||||
|
||||
.subscription-detail-row:hover {
|
||||
background-color: var(--hover-color-light, #fafafa);
|
||||
}
|
||||
|
||||
/* Dark mode support */
|
||||
.dark-mode .subscription-group-header {
|
||||
background-color: var(--hover-color-dark, #2a2a2a);
|
||||
}
|
||||
|
||||
.dark-mode .subscription-group-header:hover {
|
||||
background-color: var(--primary-color-dark, #333333);
|
||||
}
|
||||
|
||||
.dark-mode .subscription-detail-row {
|
||||
background-color: var(--background-color-dark, #1a1a1a);
|
||||
}
|
||||
|
||||
.dark-mode .subscription-detail-row:hover {
|
||||
background-color: var(--hover-color-dark, #252525);
|
||||
}
|
||||
```
|
||||
|
||||
## Features Included
|
||||
|
||||
1. ✅ **Collapsible groups**: Each WSI Pointer group can be collapsed/expanded
|
||||
2. ✅ **Visual indicator**: Arrow icon (▶/▼) shows current state
|
||||
3. ✅ **Summary information**: Shows subscription count and oldest duration
|
||||
4. ✅ **Smooth transitions**: Icon rotation animation
|
||||
5. ✅ **Hover effects**: Visual feedback on header rows
|
||||
6. ✅ **Tree structure**: Detail rows indented with └─ character
|
||||
7. ✅ **Dark mode support**: Proper styling for both themes
|
||||
8. ✅ **Keyboard accessible**: Can be enhanced with keyboard navigation
|
||||
|
||||
## User Experience Flow
|
||||
|
||||
1. **Initial load**: All groups collapsed, showing summary rows only
|
||||
2. **Click header**: Group expands, showing all subscription details
|
||||
3. **Click again**: Group collapses back to summary
|
||||
4. **Multiple groups**: Each group can be independently expanded/collapsed
|
||||
5. **Visual feedback**: Hover effects and smooth animations
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
1. ✅ Verify groups start collapsed by default
|
||||
2. ✅ Verify clicking header expands group
|
||||
3. ✅ Verify clicking again collapses group
|
||||
4. ✅ Verify multiple groups can be expanded simultaneously
|
||||
5. ✅ Verify summary information is accurate
|
||||
6. ✅ Verify detail rows display correctly when expanded
|
||||
7. ✅ Verify styling works in both light and dark modes
|
||||
8. ✅ Verify no console errors
|
||||
9. ✅ Verify performance with many subscriptions
|
||||
|
||||
## Optional Enhancements
|
||||
|
||||
1. **Expand/Collapse All**: Add buttons to expand or collapse all groups at once
|
||||
2. **Remember State**: Store expansion state in localStorage
|
||||
3. **Keyboard Navigation**: Add keyboard shortcuts (Space/Enter to toggle)
|
||||
4. **Animation**: Add slide-down/up animation for detail rows
|
||||
5. **Search/Filter**: Add ability to search within subscriptions
|
||||
6. **Export**: Add ability to export subscription data
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Review this plan
|
||||
2. Switch to Code mode
|
||||
3. Implement the changes in `api/index.js`
|
||||
4. Add CSS styles in `api/index.css`
|
||||
5. Test the collapsible functionality
|
||||
6. Verify all edge cases work correctly
|
||||
@@ -0,0 +1,155 @@
|
||||
# Subscription Table WSI Pointer Grouping Implementation Plan
|
||||
|
||||
## Objective
|
||||
Modify the subscription details table to show the WSI Pointer value only once per group - on the first row of each WSI Pointer group, leaving it blank for subsequent rows with the same WSI Pointer.
|
||||
|
||||
## Current Behavior
|
||||
- All rows show the WSI Pointer value
|
||||
- Rows are sorted by WSI Pointer (grouping is working)
|
||||
- Visual grouping is not clear
|
||||
|
||||
## Desired Behavior
|
||||
- First row of each WSI Pointer group shows the full WSI Pointer value
|
||||
- Subsequent rows in the same group have an empty cell for WSI Pointer
|
||||
- This creates a clear visual grouping effect
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### File to Modify
|
||||
`api/index.js` - Function `populateSubscriptionDetailsTable()` (lines 4277-4384)
|
||||
|
||||
### Code Changes Required
|
||||
|
||||
#### Current Code (lines 4291-4383):
|
||||
```javascript
|
||||
// Sort subscriptions by wsi_pointer to group them together
|
||||
subscriptionsData.sort((a, b) => {
|
||||
const wsiA = a.wsi_pointer || '';
|
||||
const wsiB = b.wsi_pointer || '';
|
||||
return wsiA.localeCompare(wsiB);
|
||||
});
|
||||
|
||||
subscriptionsData.forEach((subscription, index) => {
|
||||
const row = document.createElement('tr');
|
||||
|
||||
// Calculate duration
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
const duration = now - subscription.created_at;
|
||||
const durationStr = formatDuration(duration);
|
||||
|
||||
// Format client IP (show full IP for admin view)
|
||||
const clientIP = subscription.client_ip || 'unknown';
|
||||
|
||||
// Format wsi_pointer (show full pointer)
|
||||
const wsiPointer = subscription.wsi_pointer || 'N/A';
|
||||
|
||||
// Format filters (show actual filter details)
|
||||
let filtersDisplay = 'None';
|
||||
// ... filter formatting code ...
|
||||
|
||||
row.innerHTML = `
|
||||
<td style="font-family: 'Courier New', monospace; font-size: 12px;">${wsiPointer}</td>
|
||||
<td style="font-family: 'Courier New', monospace; font-size: 12px;">${subscription.id || 'N/A'}</td>
|
||||
<!-- <td style="font-family: 'Courier New', monospace; font-size: 12px;">${clientIP}</td> -->
|
||||
<td>${durationStr}</td>
|
||||
<td>${filtersDisplay}</td>
|
||||
`;
|
||||
tableBody.appendChild(row);
|
||||
});
|
||||
```
|
||||
|
||||
#### Modified Code:
|
||||
```javascript
|
||||
// Sort subscriptions by wsi_pointer to group them together
|
||||
subscriptionsData.sort((a, b) => {
|
||||
const wsiA = a.wsi_pointer || '';
|
||||
const wsiB = b.wsi_pointer || '';
|
||||
return wsiA.localeCompare(wsiB);
|
||||
});
|
||||
|
||||
// Track previous WSI Pointer to detect group changes
|
||||
let previousWsiPointer = null;
|
||||
|
||||
subscriptionsData.forEach((subscription, index) => {
|
||||
const row = document.createElement('tr');
|
||||
|
||||
// Calculate duration
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
const duration = now - subscription.created_at;
|
||||
const durationStr = formatDuration(duration);
|
||||
|
||||
// Format client IP (show full IP for admin view)
|
||||
const clientIP = subscription.client_ip || 'unknown';
|
||||
|
||||
// Format wsi_pointer - only show if it's different from previous row
|
||||
const currentWsiPointer = subscription.wsi_pointer || 'N/A';
|
||||
let wsiPointerDisplay = '';
|
||||
|
||||
if (currentWsiPointer !== previousWsiPointer) {
|
||||
// This is the first row of a new group - show the WSI Pointer
|
||||
wsiPointerDisplay = currentWsiPointer;
|
||||
previousWsiPointer = currentWsiPointer;
|
||||
} else {
|
||||
// This is a continuation of the same group - leave blank
|
||||
wsiPointerDisplay = '';
|
||||
}
|
||||
|
||||
// Format filters (show actual filter details)
|
||||
let filtersDisplay = 'None';
|
||||
// ... filter formatting code remains the same ...
|
||||
|
||||
row.innerHTML = `
|
||||
<td style="font-family: 'Courier New', monospace; font-size: 12px;">${wsiPointerDisplay}</td>
|
||||
<td style="font-family: 'Courier New', monospace; font-size: 12px;">${subscription.id || 'N/A'}</td>
|
||||
<!-- <td style="font-family: 'Courier New', monospace; font-size: 12px;">${clientIP}</td> -->
|
||||
<td>${durationStr}</td>
|
||||
<td>${filtersDisplay}</td>
|
||||
`;
|
||||
tableBody.appendChild(row);
|
||||
});
|
||||
```
|
||||
|
||||
### Key Changes Explained
|
||||
|
||||
1. **Add tracking variable**: `let previousWsiPointer = null;` before the forEach loop
|
||||
2. **Store current WSI Pointer**: `const currentWsiPointer = subscription.wsi_pointer || 'N/A';`
|
||||
3. **Compare with previous**: Check if `currentWsiPointer !== previousWsiPointer`
|
||||
4. **Conditional display**:
|
||||
- If different: Show the WSI Pointer value and update `previousWsiPointer`
|
||||
- If same: Show empty string (blank cell)
|
||||
5. **Use display variable**: Replace `${wsiPointer}` with `${wsiPointerDisplay}` in the row HTML
|
||||
|
||||
### Visual Result
|
||||
|
||||
**Before:**
|
||||
```
|
||||
WSI Pointer | Subscription ID | Duration | Filters
|
||||
0x12345678 | sub-001 | 5m 30s | kinds:[1]
|
||||
0x12345678 | sub-002 | 3m 15s | kinds:[3]
|
||||
0x87654321 | sub-003 | 1m 45s | kinds:[1,3]
|
||||
```
|
||||
|
||||
**After:**
|
||||
```
|
||||
WSI Pointer | Subscription ID | Duration | Filters
|
||||
0x12345678 | sub-001 | 5m 30s | kinds:[1]
|
||||
| sub-002 | 3m 15s | kinds:[3]
|
||||
0x87654321 | sub-003 | 1m 45s | kinds:[1,3]
|
||||
```
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
1. ✅ Verify first row of each group shows WSI Pointer
|
||||
2. ✅ Verify subsequent rows in same group are blank
|
||||
3. ✅ Verify grouping works with multiple subscriptions per WSI Pointer
|
||||
4. ✅ Verify single subscription per WSI Pointer still shows the value
|
||||
5. ✅ Verify empty/null WSI Pointers are handled correctly
|
||||
6. ✅ Verify table still displays correctly when no subscriptions exist
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Review this plan
|
||||
2. Switch to Code mode
|
||||
3. Implement the changes in `api/index.js`
|
||||
4. Test the implementation
|
||||
5. Verify the visual grouping effect
|
||||
@@ -586,7 +586,7 @@ int add_pubkeys_to_config_table(void) {
|
||||
rm -f *.db
|
||||
|
||||
# Start relay with defaults
|
||||
./build/c_relay_x86
|
||||
./build/c_relay_pg_x86
|
||||
|
||||
# Verify config table complete
|
||||
sqlite3 <relay_pubkey>.db "SELECT COUNT(*) FROM config;"
|
||||
@@ -602,7 +602,7 @@ int add_pubkeys_to_config_table(void) {
|
||||
rm -f *.db
|
||||
|
||||
# Start relay with port override
|
||||
./build/c_relay_x86 --port 9999
|
||||
./build/c_relay_pg_x86 --port 9999
|
||||
|
||||
# Verify port override applied
|
||||
sqlite3 <relay_pubkey>.db "SELECT value FROM config WHERE key='relay_port';"
|
||||
@@ -612,13 +612,13 @@ int add_pubkeys_to_config_table(void) {
|
||||
3. **Restart with Existing Database**
|
||||
```bash
|
||||
# Start relay (creates database)
|
||||
./build/c_relay_x86
|
||||
./build/c_relay_pg_x86
|
||||
|
||||
# Stop relay
|
||||
pkill -f c_relay_
|
||||
pkill -f c_relay_pg_
|
||||
|
||||
# Restart relay
|
||||
./build/c_relay_x86
|
||||
./build/c_relay_pg_x86
|
||||
|
||||
# Verify config unchanged
|
||||
# Check relay.log for validation message
|
||||
@@ -627,13 +627,13 @@ int add_pubkeys_to_config_table(void) {
|
||||
4. **Restart with CLI Overrides**
|
||||
```bash
|
||||
# Start relay (creates database)
|
||||
./build/c_relay_x86
|
||||
./build/c_relay_pg_x86
|
||||
|
||||
# Stop relay
|
||||
pkill -f c_relay_
|
||||
pkill -f c_relay_pg_
|
||||
|
||||
# Restart with port override
|
||||
./build/c_relay_x86 --port 9999
|
||||
./build/c_relay_pg_x86 --port 9999
|
||||
|
||||
# Verify port override applied
|
||||
sqlite3 <relay_pubkey>.db "SELECT value FROM config WHERE key='relay_port';"
|
||||
|
||||
+32
-32
@@ -19,12 +19,12 @@ Complete guide for deploying, configuring, and managing the C Nostr Relay with e
|
||||
```bash
|
||||
# Clone and build
|
||||
git clone <repository-url>
|
||||
cd c-relay
|
||||
cd c-relay-pg
|
||||
git submodule update --init --recursive
|
||||
make
|
||||
|
||||
# Start relay (zero configuration needed)
|
||||
./build/c_relay_x86
|
||||
./build/c_relay_pg_x86
|
||||
```
|
||||
|
||||
### 2. First Startup - Save Keys
|
||||
@@ -78,7 +78,7 @@ brew install git sqlite libwebsockets openssl libsecp256k1 curl zlib
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone <repository-url>
|
||||
cd c-relay
|
||||
cd c-relay-pg
|
||||
|
||||
# Initialize submodules
|
||||
git submodule update --init --recursive
|
||||
@@ -87,7 +87,7 @@ git submodule update --init --recursive
|
||||
make clean && make
|
||||
|
||||
# Verify build
|
||||
ls -la build/c_relay_x86
|
||||
ls -la build/c_relay_pg_x86
|
||||
```
|
||||
|
||||
### Production Deployment
|
||||
@@ -98,27 +98,27 @@ ls -la build/c_relay_x86
|
||||
sudo systemd/install-service.sh
|
||||
|
||||
# Start service
|
||||
sudo systemctl start c-relay
|
||||
sudo systemctl start c-relay-pg
|
||||
|
||||
# Enable auto-start
|
||||
sudo systemctl enable c-relay
|
||||
sudo systemctl enable c-relay-pg
|
||||
|
||||
# Check status
|
||||
sudo systemctl status c-relay
|
||||
sudo systemctl status c-relay-pg
|
||||
```
|
||||
|
||||
#### Manual Deployment
|
||||
```bash
|
||||
# Create dedicated user
|
||||
sudo useradd --system --home-dir /opt/c-relay --shell /bin/false c-relay
|
||||
sudo useradd --system --home-dir /opt/c-relay-pg --shell /bin/false c-relay-pg
|
||||
|
||||
# Install binary
|
||||
sudo mkdir -p /opt/c-relay
|
||||
sudo cp build/c_relay_x86 /opt/c-relay/
|
||||
sudo chown -R c-relay:c-relay /opt/c-relay
|
||||
sudo mkdir -p /opt/c-relay-pg
|
||||
sudo cp build/c_relay_pg_x86 /opt/c-relay-pg/
|
||||
sudo chown -R c-relay-pg:c-relay-pg /opt/c-relay-pg
|
||||
|
||||
# Run as service user
|
||||
sudo -u c-relay /opt/c-relay/c_relay_x86
|
||||
sudo -u c-relay-pg /opt/c-relay-pg/c_relay_pg_x86
|
||||
```
|
||||
|
||||
## Configuration Management
|
||||
@@ -188,7 +188,7 @@ Send this to your relay via WebSocket, and changes are applied immediately.
|
||||
|-----------|-------------|---------|---------|
|
||||
| `relay_description` | Relay description for NIP-11 | "C Nostr Relay" | "My awesome relay" |
|
||||
| `relay_contact` | Admin contact information | "" | "admin@example.com" |
|
||||
| `relay_software` | Software identifier | "c-relay" | "c-relay v1.0" |
|
||||
| `relay_software` | Software identifier | "c-relay-pg" | "c-relay-pg v1.0" |
|
||||
|
||||
#### Client Limits
|
||||
| Parameter | Description | Default | Range |
|
||||
@@ -273,7 +273,7 @@ chmod 600 admin_keys_backup_*.txt
|
||||
#### Key Recovery
|
||||
If you lose your admin private key:
|
||||
|
||||
1. **Stop the relay**: `pkill c_relay` or `sudo systemctl stop c-relay`
|
||||
1. **Stop the relay**: `pkill c_relay_pg` or `sudo systemctl stop c-relay-pg`
|
||||
2. **Backup events**: `cp <relay_pubkey>.nrdb backup_$(date +%Y%m%d).nrdb`
|
||||
3. **Remove database**: `rm <relay_pubkey>.nrdb*`
|
||||
4. **Restart relay**: This creates new database with new keys
|
||||
@@ -300,7 +300,7 @@ sudo ufw allow 8888/tcp
|
||||
```bash
|
||||
# Secure database file permissions
|
||||
chmod 600 <relay_pubkey>.nrdb
|
||||
chown c-relay:c-relay <relay_pubkey>.nrdb
|
||||
chown c-relay-pg:c-relay-pg <relay_pubkey>.nrdb
|
||||
|
||||
# Regular backups
|
||||
cp <relay_pubkey>.nrdb backup/relay_backup_$(date +%Y%m%d_%H%M%S).nrdb
|
||||
@@ -311,10 +311,10 @@ cp <relay_pubkey>.nrdb backup/relay_backup_$(date +%Y%m%d_%H%M%S).nrdb
|
||||
### Service Status
|
||||
```bash
|
||||
# Check if relay is running
|
||||
ps aux | grep c_relay
|
||||
ps aux | grep c_relay_pg
|
||||
|
||||
# SystemD status
|
||||
sudo systemctl status c-relay
|
||||
sudo systemctl status c-relay-pg
|
||||
|
||||
# Network connections
|
||||
netstat -tln | grep 8888
|
||||
@@ -324,16 +324,16 @@ sudo ss -tlpn | grep 8888
|
||||
### Log Monitoring
|
||||
```bash
|
||||
# Real-time logs (systemd)
|
||||
sudo journalctl -u c-relay -f
|
||||
sudo journalctl -u c-relay-pg -f
|
||||
|
||||
# Recent logs
|
||||
sudo journalctl -u c-relay --since "1 hour ago"
|
||||
sudo journalctl -u c-relay-pg --since "1 hour ago"
|
||||
|
||||
# Error logs only
|
||||
sudo journalctl -u c-relay -p err
|
||||
sudo journalctl -u c-relay-pg -p err
|
||||
|
||||
# Configuration changes
|
||||
sudo journalctl -u c-relay | grep "Configuration updated via kind 33334"
|
||||
sudo journalctl -u c-relay-pg | grep "Configuration updated via kind 33334"
|
||||
```
|
||||
|
||||
### Database Analytics
|
||||
@@ -365,13 +365,13 @@ ORDER BY created_at DESC;
|
||||
du -sh <relay_pubkey>.nrdb*
|
||||
|
||||
# Memory usage
|
||||
ps aux | grep c_relay | awk '{print $6}' # RSS memory in KB
|
||||
ps aux | grep c_relay_pg | awk '{print $6}' # RSS memory in KB
|
||||
|
||||
# Connection count (approximate)
|
||||
netstat -an | grep :8888 | grep ESTABLISHED | wc -l
|
||||
|
||||
# System resources
|
||||
top -p $(pgrep c_relay)
|
||||
top -p $(pgrep c_relay_pg)
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
@@ -385,11 +385,11 @@ netstat -tln | grep 8888
|
||||
# If port in use, find process: sudo lsof -i :8888
|
||||
|
||||
# Check binary permissions
|
||||
ls -la build/c_relay_x86
|
||||
chmod +x build/c_relay_x86
|
||||
ls -la build/c_relay_pg_x86
|
||||
chmod +x build/c_relay_pg_x86
|
||||
|
||||
# Check dependencies
|
||||
ldd build/c_relay_x86
|
||||
ldd build/c_relay_pg_x86
|
||||
```
|
||||
|
||||
#### Configuration Not Updating
|
||||
@@ -442,7 +442,7 @@ sqlite3 recovered.nrdb < recovered.sql
|
||||
|
||||
# If repair fails, start fresh (loses all events)
|
||||
mv <relay_pubkey>.nrdb <relay_pubkey>.nrdb.corrupted
|
||||
./build/c_relay_x86 # Creates new database
|
||||
./build/c_relay_pg_x86 # Creates new database
|
||||
```
|
||||
|
||||
#### Lost Configuration Recovery
|
||||
@@ -455,12 +455,12 @@ If configuration is lost but database is intact:
|
||||
#### Emergency Restart
|
||||
```bash
|
||||
# Quick restart with clean state
|
||||
sudo systemctl stop c-relay
|
||||
sudo systemctl stop c-relay-pg
|
||||
mv <relay_pubkey>.nrdb <relay_pubkey>.nrdb.backup
|
||||
sudo systemctl start c-relay
|
||||
sudo systemctl start c-relay-pg
|
||||
|
||||
# Check logs for new admin keys
|
||||
sudo journalctl -u c-relay --since "5 minutes ago" | grep "Admin Private Key"
|
||||
sudo journalctl -u c-relay-pg --since "5 minutes ago" | grep "Admin Private Key"
|
||||
```
|
||||
|
||||
## Advanced Usage
|
||||
@@ -503,7 +503,7 @@ ws.on('open', function() {
|
||||
# backup-relay.sh
|
||||
DATE=$(date +%Y%m%d_%H%M%S)
|
||||
DB_FILE=$(ls *.nrdb | head -1)
|
||||
BACKUP_DIR="/backup/c-relay"
|
||||
BACKUP_DIR="/backup/c-relay-pg"
|
||||
|
||||
mkdir -p $BACKUP_DIR
|
||||
cp $DB_FILE $BACKUP_DIR/relay_backup_$DATE.nrdb
|
||||
@@ -533,7 +533,7 @@ tar czf relay_migration.tar.gz *.nrdb* relay.log
|
||||
|
||||
# Target server
|
||||
tar xzf relay_migration.tar.gz
|
||||
./build/c_relay_x86 # Will detect existing database and continue
|
||||
./build/c_relay_pg_x86 # Will detect existing database and continue
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Script to embed web files into C headers for the C-Relay admin interface
|
||||
# Script to embed web files into C headers for the C-Relay-PG admin interface
|
||||
# Converts HTML, CSS, and JS files from api/ directory into C byte arrays
|
||||
|
||||
set -e
|
||||
|
||||
@@ -60,7 +60,7 @@ All examples assume the event-based configuration system where:
|
||||
- **Save Admin Keys**: All deployment examples emphasize capturing the admin private key on first startup
|
||||
- **Firewall Configuration**: Examples include proper firewall rules
|
||||
- **SSL/TLS**: Production examples include HTTPS configuration
|
||||
- **User Isolation**: Service runs as dedicated `c-relay` system user
|
||||
- **User Isolation**: Service runs as dedicated `c-relay-pg` system user
|
||||
|
||||
## Support
|
||||
|
||||
|
||||
@@ -13,8 +13,8 @@ BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Default configuration
|
||||
RELAY_DIR="/opt/c-relay"
|
||||
BACKUP_DIR="/backup/c-relay"
|
||||
RELAY_DIR="/opt/c-relay-pg"
|
||||
BACKUP_DIR="/backup/c-relay-pg"
|
||||
RETENTION_DAYS="30"
|
||||
COMPRESS="true"
|
||||
REMOTE_BACKUP=""
|
||||
@@ -47,8 +47,8 @@ show_help() {
|
||||
echo "Usage: $0 [OPTIONS]"
|
||||
echo
|
||||
echo "Options:"
|
||||
echo " -d, --relay-dir DIR Relay directory (default: /opt/c-relay)"
|
||||
echo " -b, --backup-dir DIR Backup directory (default: /backup/c-relay)"
|
||||
echo " -d, --relay-dir DIR Relay directory (default: /opt/c-relay-pg)"
|
||||
echo " -b, --backup-dir DIR Backup directory (default: /backup/c-relay-pg)"
|
||||
echo " -r, --retention DAYS Retention period in days (default: 30)"
|
||||
echo " -n, --no-compress Don't compress backups"
|
||||
echo " -s, --s3-bucket BUCKET Upload to S3 bucket"
|
||||
@@ -231,7 +231,7 @@ upload_to_s3() {
|
||||
|
||||
print_step "Uploading backup to S3..."
|
||||
|
||||
local s3_path="s3://$S3_BUCKET/c-relay/$(date +%Y)/$(date +%m)/"
|
||||
local s3_path="s3://$S3_BUCKET/c-relay-pg/$(date +%Y)/$(date +%m)/"
|
||||
|
||||
if aws s3 cp "$BACKUP_FILE" "$s3_path" --storage-class STANDARD_IA; then
|
||||
print_success "Backup uploaded to S3: $s3_path"
|
||||
@@ -264,7 +264,7 @@ cleanup_old_backups() {
|
||||
print_step "Cleaning S3 backups older than $cutoff_date..."
|
||||
|
||||
# Note: This is a simplified approach. In production, use S3 lifecycle policies
|
||||
aws s3 ls "s3://$S3_BUCKET/c-relay/" --recursive | \
|
||||
aws s3 ls "s3://$S3_BUCKET/c-relay-pg/" --recursive | \
|
||||
awk '$1 < "'$cutoff_date'" {print $4}' | \
|
||||
while read -r key; do
|
||||
aws s3 rm "s3://$S3_BUCKET/$key"
|
||||
|
||||
@@ -13,8 +13,8 @@ BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Configuration
|
||||
RELAY_DIR="/opt/c-relay"
|
||||
SERVICE_NAME="c-relay"
|
||||
RELAY_DIR="/opt/c-relay-pg"
|
||||
SERVICE_NAME="c-relay-pg"
|
||||
RELAY_PORT="8888"
|
||||
LOG_FILE="/var/log/relay-monitor.log"
|
||||
ALERT_EMAIL=""
|
||||
@@ -60,7 +60,7 @@ show_help() {
|
||||
echo "Usage: $0 [OPTIONS]"
|
||||
echo
|
||||
echo "Options:"
|
||||
echo " -d, --relay-dir DIR Relay directory (default: /opt/c-relay)"
|
||||
echo " -d, --relay-dir DIR Relay directory (default: /opt/c-relay-pg)"
|
||||
echo " -p, --port PORT Relay port (default: 8888)"
|
||||
echo " -i, --interval SECONDS Check interval (default: 60)"
|
||||
echo " -e, --email EMAIL Alert email address"
|
||||
@@ -134,7 +134,7 @@ parse_args() {
|
||||
check_process_running() {
|
||||
print_step "Checking if relay process is running..."
|
||||
|
||||
if pgrep -f "c_relay_x86" > /dev/null; then
|
||||
if pgrep -f "c_relay_pg_x86" > /dev/null; then
|
||||
print_success "Relay process is running"
|
||||
return 0
|
||||
else
|
||||
@@ -172,7 +172,7 @@ check_service_status() {
|
||||
check_memory_usage() {
|
||||
print_step "Checking memory usage..."
|
||||
|
||||
local memory_kb=$(ps aux | grep "c_relay_x86" | grep -v grep | awk '{sum+=$6} END {print sum}')
|
||||
local memory_kb=$(ps aux | grep "c_relay_pg_x86" | grep -v grep | awk '{sum+=$6} END {print sum}')
|
||||
|
||||
if [[ -z "$memory_kb" ]]; then
|
||||
print_warning "Could not determine memory usage"
|
||||
|
||||
@@ -56,7 +56,7 @@ http {
|
||||
}
|
||||
|
||||
# Upstream for the relay
|
||||
upstream c_relay_backend {
|
||||
upstream c_relay_pg_backend {
|
||||
server 127.0.0.1:8888;
|
||||
keepalive 32;
|
||||
}
|
||||
@@ -108,7 +108,7 @@ http {
|
||||
# Main proxy location for WebSocket and HTTP
|
||||
location / {
|
||||
# Proxy settings
|
||||
proxy_pass http://c_relay_backend;
|
||||
proxy_pass http://c_relay_pg_backend;
|
||||
proxy_http_version 1.1;
|
||||
proxy_cache_bypass $http_upgrade;
|
||||
|
||||
@@ -144,7 +144,7 @@ http {
|
||||
|
||||
# Health check endpoint (if implemented)
|
||||
location /health {
|
||||
proxy_pass http://c_relay_backend/health;
|
||||
proxy_pass http://c_relay_pg_backend/health;
|
||||
access_log off;
|
||||
}
|
||||
|
||||
@@ -157,7 +157,7 @@ http {
|
||||
|
||||
# Optional: Metrics endpoint (if implemented)
|
||||
location /metrics {
|
||||
proxy_pass http://c_relay_backend/metrics;
|
||||
proxy_pass http://c_relay_pg_backend/metrics;
|
||||
# Restrict access to monitoring systems
|
||||
allow 10.0.0.0/8;
|
||||
allow 172.16.0.0/12;
|
||||
|
||||
@@ -96,9 +96,9 @@ check_root() {
|
||||
check_relay_running() {
|
||||
print_step "Checking if C Nostr Relay is running..."
|
||||
|
||||
if ! pgrep -f "c_relay_x86" > /dev/null; then
|
||||
if ! pgrep -f "c_relay_pg_x86" > /dev/null; then
|
||||
print_error "C Nostr Relay is not running"
|
||||
print_error "Please start the relay first with: sudo systemctl start c-relay"
|
||||
print_error "Please start the relay first with: sudo systemctl start c-relay-pg"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
@@ -13,9 +13,9 @@ BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Configuration
|
||||
RELAY_USER="c-relay"
|
||||
INSTALL_DIR="/opt/c-relay"
|
||||
SERVICE_NAME="c-relay"
|
||||
RELAY_USER="c-relay-pg"
|
||||
INSTALL_DIR="/opt/c-relay-pg"
|
||||
SERVICE_NAME="c-relay-pg"
|
||||
RELAY_PORT="8888"
|
||||
|
||||
# Functions
|
||||
@@ -99,7 +99,7 @@ build_relay() {
|
||||
|
||||
# Check if we're in the source directory
|
||||
if [[ ! -f "Makefile" ]]; then
|
||||
print_error "Makefile not found. Please run this script from the c-relay source directory."
|
||||
print_error "Makefile not found. Please run this script from the c-relay-pg source directory."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -107,7 +107,7 @@ build_relay() {
|
||||
make clean
|
||||
make
|
||||
|
||||
if [[ ! -f "build/c_relay_x86" ]]; then
|
||||
if [[ ! -f "build/c_relay_pg_x86" ]]; then
|
||||
print_error "Build failed - binary not found"
|
||||
exit 1
|
||||
fi
|
||||
@@ -118,9 +118,9 @@ build_relay() {
|
||||
install_binary() {
|
||||
print_step "Installing relay binary..."
|
||||
|
||||
cp build/c_relay_x86 "$INSTALL_DIR/"
|
||||
chown "$RELAY_USER:$RELAY_USER" "$INSTALL_DIR/c_relay_x86"
|
||||
chmod +x "$INSTALL_DIR/c_relay_x86"
|
||||
cp build/c_relay_pg_x86 "$INSTALL_DIR/"
|
||||
chown "$RELAY_USER:$RELAY_USER" "$INSTALL_DIR/c_relay_pg_x86"
|
||||
chmod +x "$INSTALL_DIR/c_relay_pg_x86"
|
||||
|
||||
print_success "Binary installed to $INSTALL_DIR"
|
||||
}
|
||||
@@ -129,14 +129,14 @@ install_service() {
|
||||
print_step "Installing systemd service..."
|
||||
|
||||
# Use the existing systemd service file
|
||||
if [[ -f "systemd/c-relay.service" ]]; then
|
||||
cp systemd/c-relay.service /etc/systemd/system/
|
||||
if [[ -f "systemd/c-relay-pg.service" ]]; then
|
||||
cp systemd/c-relay-pg.service /etc/systemd/system/
|
||||
systemctl daemon-reload
|
||||
print_success "Systemd service installed"
|
||||
else
|
||||
print_warning "Systemd service file not found, creating basic one..."
|
||||
|
||||
cat > /etc/systemd/system/c-relay.service << EOF
|
||||
cat > /etc/systemd/system/c-relay-pg.service << EOF
|
||||
[Unit]
|
||||
Description=C Nostr Relay
|
||||
After=network.target
|
||||
@@ -146,7 +146,7 @@ Type=simple
|
||||
User=$RELAY_USER
|
||||
Group=$RELAY_USER
|
||||
WorkingDirectory=$INSTALL_DIR
|
||||
ExecStart=$INSTALL_DIR/c_relay_x86
|
||||
ExecStart=$INSTALL_DIR/c_relay_pg_x86
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# MUSL-based fully static C-Relay builder
|
||||
# MUSL-based fully static C-Relay-PG builder
|
||||
# Produces portable binaries with zero runtime dependencies
|
||||
|
||||
FROM alpine:latest AS builder
|
||||
@@ -125,7 +125,7 @@ RUN cd /tmp && \
|
||||
--prefix=/usr && \
|
||||
make && make install
|
||||
|
||||
# Copy c-relay source
|
||||
# Copy c-relay-pg source
|
||||
COPY . /build/
|
||||
|
||||
# Initialize submodules
|
||||
@@ -134,7 +134,7 @@ RUN git submodule update --init --recursive
|
||||
# Build nostr_core_lib
|
||||
RUN cd nostr_core_lib && ./build.sh
|
||||
|
||||
# Build c-relay static
|
||||
# Build c-relay-pg static
|
||||
RUN make clean && \
|
||||
CC="musl-gcc -static" \
|
||||
CFLAGS="-O2 -Wall -Wextra -std=c99 -g" \
|
||||
@@ -143,8 +143,8 @@ RUN make clean && \
|
||||
make
|
||||
|
||||
# Strip binary for size
|
||||
RUN strip build/c_relay_x86
|
||||
RUN strip build/c_relay_pg_x86
|
||||
|
||||
# Multi-stage build to produce minimal output
|
||||
FROM scratch AS output
|
||||
COPY --from=builder /build/build/c_relay_x86 /c_relay_static_musl_x86_64
|
||||
COPY --from=builder /build/build/c_relay_pg_x86 /c_relay_pg_static_musl_x86_64
|
||||
+252
-46
@@ -8,36 +8,59 @@ YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m'
|
||||
|
||||
print_status() { echo -e "${BLUE}[INFO]${NC} $1"; }
|
||||
print_success() { echo -e "${GREEN}[SUCCESS]${NC} $1"; }
|
||||
print_warning() { echo -e "${YELLOW}[WARNING]${NC} $1"; }
|
||||
print_error() { echo -e "${RED}[ERROR]${NC} $1"; }
|
||||
print_status() { echo -e "${BLUE}[INFO]${NC} $1" >&2; }
|
||||
print_success() { echo -e "${GREEN}[SUCCESS]${NC} $1" >&2; }
|
||||
print_warning() { echo -e "${YELLOW}[WARNING]${NC} $1" >&2; }
|
||||
print_error() { echo -e "${RED}[ERROR]${NC} $1" >&2; }
|
||||
|
||||
# Global variables
|
||||
COMMIT_MESSAGE=""
|
||||
RELEASE_MODE=false
|
||||
VERSION_INCREMENT_TYPE="patch" # "patch", "minor", or "major"
|
||||
|
||||
show_usage() {
|
||||
echo "C-Relay Increment and Push Script"
|
||||
echo "C-Relay-PG Increment and Push Script"
|
||||
echo ""
|
||||
echo "Usage:"
|
||||
echo " $0 \"commit message\" - Default: increment patch, commit & push"
|
||||
echo " $0 -r \"commit message\" - Release: increment minor, create release"
|
||||
echo "USAGE:"
|
||||
echo " $0 [OPTIONS] \"commit message\""
|
||||
echo ""
|
||||
echo "Examples:"
|
||||
echo "COMMANDS:"
|
||||
echo " $0 \"commit message\" Default: increment patch, commit & push"
|
||||
echo " $0 -p \"commit message\" Increment patch version"
|
||||
echo " $0 -m \"commit message\" Increment minor version"
|
||||
echo " $0 -M \"commit message\" Increment major version"
|
||||
echo " $0 -r \"commit message\" Create release with assets (no version increment)"
|
||||
echo " $0 -r -m \"commit message\" Create release with minor version increment"
|
||||
echo " $0 -h Show this help message"
|
||||
echo ""
|
||||
echo "OPTIONS:"
|
||||
echo " -p, --patch Increment patch version (default)"
|
||||
echo " -m, --minor Increment minor version"
|
||||
echo " -M, --major Increment major version"
|
||||
echo " -r, --release Create release with assets"
|
||||
echo " -h, --help Show this help message"
|
||||
echo ""
|
||||
echo "EXAMPLES:"
|
||||
echo " $0 \"Fixed event validation bug\""
|
||||
echo " $0 --release \"Major release with new features\""
|
||||
echo " $0 -m \"Added new features\""
|
||||
echo " $0 -M \"Breaking API changes\""
|
||||
echo " $0 -r \"Release current version\""
|
||||
echo " $0 -r -m \"Release with minor increment\""
|
||||
echo ""
|
||||
echo "Default Mode (patch increment):"
|
||||
echo " - Increment patch version (v1.2.3 → v1.2.4)"
|
||||
echo " - Git add, commit with message, and push"
|
||||
echo "VERSION INCREMENT MODES:"
|
||||
echo " -p, --patch (default): Increment patch version (v1.2.3 → v1.2.4)"
|
||||
echo " -m, --minor: Increment minor version, zero patch (v1.2.3 → v1.3.0)"
|
||||
echo " -M, --major: Increment major version, zero minor+patch (v1.2.3 → v2.0.0)"
|
||||
echo ""
|
||||
echo "Release Mode (-r flag):"
|
||||
echo " - Increment minor version, zero patch (v1.2.3 → v1.3.0)"
|
||||
echo " - Git add, commit, push, and create Gitea release"
|
||||
echo "RELEASE MODE (-r flag):"
|
||||
echo " - Build static binary using build_static.sh"
|
||||
echo " - Create source tarball"
|
||||
echo " - Git add, commit, push, and create Gitea release with assets"
|
||||
echo " - Can be combined with version increment flags"
|
||||
echo ""
|
||||
echo "Requirements for Release Mode:"
|
||||
echo "REQUIREMENTS FOR RELEASE MODE:"
|
||||
echo " - Gitea token in ~/.gitea_token for release uploads"
|
||||
echo " - Docker installed for static binary builds"
|
||||
}
|
||||
|
||||
# Parse command line arguments
|
||||
@@ -47,6 +70,18 @@ while [[ $# -gt 0 ]]; do
|
||||
RELEASE_MODE=true
|
||||
shift
|
||||
;;
|
||||
-p|--patch)
|
||||
VERSION_INCREMENT_TYPE="patch"
|
||||
shift
|
||||
;;
|
||||
-m|--minor)
|
||||
VERSION_INCREMENT_TYPE="minor"
|
||||
shift
|
||||
;;
|
||||
-M|--major)
|
||||
VERSION_INCREMENT_TYPE="major"
|
||||
shift
|
||||
;;
|
||||
-h|--help)
|
||||
show_usage
|
||||
exit 0
|
||||
@@ -79,7 +114,7 @@ check_git_repo() {
|
||||
|
||||
# Function to get current version and increment appropriately
|
||||
increment_version() {
|
||||
local increment_type="$1" # "patch" or "minor"
|
||||
local increment_type="$1" # "patch", "minor", or "major"
|
||||
|
||||
print_status "Getting current version..."
|
||||
|
||||
@@ -105,24 +140,34 @@ increment_version() {
|
||||
fi
|
||||
|
||||
# Increment version based on type
|
||||
if [[ "$increment_type" == "minor" ]]; then
|
||||
if [[ "$increment_type" == "major" ]]; then
|
||||
# Major release: increment major, zero minor and patch
|
||||
NEW_MAJOR=$((MAJOR + 1))
|
||||
NEW_MINOR=0
|
||||
NEW_PATCH=0
|
||||
NEW_VERSION="v${NEW_MAJOR}.${NEW_MINOR}.${NEW_PATCH}"
|
||||
print_status "Major version increment: incrementing major version"
|
||||
elif [[ "$increment_type" == "minor" ]]; then
|
||||
# Minor release: increment minor, zero patch
|
||||
NEW_MAJOR=$MAJOR
|
||||
NEW_MINOR=$((MINOR + 1))
|
||||
NEW_PATCH=0
|
||||
NEW_VERSION="v${MAJOR}.${NEW_MINOR}.${NEW_PATCH}"
|
||||
print_status "Release mode: incrementing minor version"
|
||||
NEW_VERSION="v${NEW_MAJOR}.${NEW_MINOR}.${NEW_PATCH}"
|
||||
print_status "Minor version increment: incrementing minor version"
|
||||
else
|
||||
# Default: increment patch
|
||||
NEW_MAJOR=$MAJOR
|
||||
NEW_MINOR=$MINOR
|
||||
NEW_PATCH=$((PATCH + 1))
|
||||
NEW_VERSION="v${MAJOR}.${MINOR}.${NEW_PATCH}"
|
||||
print_status "Default mode: incrementing patch version"
|
||||
NEW_VERSION="v${NEW_MAJOR}.${NEW_MINOR}.${NEW_PATCH}"
|
||||
print_status "Patch version increment: incrementing patch version"
|
||||
fi
|
||||
|
||||
print_status "Current version: $LATEST_TAG"
|
||||
print_status "New version: $NEW_VERSION"
|
||||
|
||||
# Update version in src/main.h
|
||||
update_version_in_header "$NEW_VERSION" "$MAJOR" "${NEW_MINOR:-$MINOR}" "${NEW_PATCH:-$PATCH}"
|
||||
update_version_in_header "$NEW_VERSION" "$NEW_MAJOR" "$NEW_MINOR" "$NEW_PATCH"
|
||||
|
||||
# Export for use in other functions
|
||||
export NEW_VERSION
|
||||
@@ -143,17 +188,17 @@ update_version_in_header() {
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Update VERSION macro
|
||||
sed -i "s/#define VERSION \".*\"/#define VERSION \"$new_version\"/" src/main.h
|
||||
# Update CRELAY_VERSION macro
|
||||
sed -i "s/#define CRELAY_VERSION \".*\"/#define CRELAY_VERSION \"$new_version\"/" src/main.h
|
||||
|
||||
# Update VERSION_MAJOR macro
|
||||
sed -i "s/#define VERSION_MAJOR [0-9]\+/#define VERSION_MAJOR $major/" src/main.h
|
||||
# Update CRELAY_VERSION_MAJOR macro
|
||||
sed -i "s/#define CRELAY_VERSION_MAJOR [0-9]\+/#define CRELAY_VERSION_MAJOR $major/" src/main.h
|
||||
|
||||
# Update VERSION_MINOR macro
|
||||
sed -i "s/#define VERSION_MINOR .*/#define VERSION_MINOR $minor/" src/main.h
|
||||
# Update CRELAY_VERSION_MINOR macro
|
||||
sed -i "s/#define CRELAY_VERSION_MINOR .*/#define CRELAY_VERSION_MINOR $minor/" src/main.h
|
||||
|
||||
# Update VERSION_PATCH macro
|
||||
sed -i "s/#define VERSION_PATCH [0-9]\+/#define VERSION_PATCH $patch/" src/main.h
|
||||
# Update CRELAY_VERSION_PATCH macro
|
||||
sed -i "s/#define CRELAY_VERSION_PATCH [0-9]\+/#define CRELAY_VERSION_PATCH $patch/" src/main.h
|
||||
|
||||
print_success "Updated version in src/main.h to $new_version"
|
||||
}
|
||||
@@ -261,6 +306,115 @@ git_commit_and_push_no_tag() {
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to build release binary
|
||||
build_release_binary() {
|
||||
print_status "Building release binary..."
|
||||
|
||||
# Check if build_static.sh exists
|
||||
if [[ ! -f "build_static.sh" ]]; then
|
||||
print_error "build_static.sh not found"
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Run the static build script
|
||||
if ./build_static.sh > /dev/null 2>&1; then
|
||||
print_success "Built static binary successfully"
|
||||
return 0
|
||||
else
|
||||
print_error "Failed to build static binary"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to create source tarball
|
||||
create_source_tarball() {
|
||||
print_status "Creating source tarball..."
|
||||
|
||||
local tarball_name="c-relay-pg-${NEW_VERSION#v}.tar.gz"
|
||||
|
||||
# Create tarball excluding build artifacts and git files
|
||||
if tar -czf "$tarball_name" \
|
||||
--exclude='build/*' \
|
||||
--exclude='.git*' \
|
||||
--exclude='*.db' \
|
||||
--exclude='*.db-*' \
|
||||
--exclude='*.log' \
|
||||
--exclude='*.tar.gz' \
|
||||
. > /dev/null 2>&1; then
|
||||
print_success "Created source tarball: $tarball_name"
|
||||
echo "$tarball_name"
|
||||
return 0
|
||||
else
|
||||
print_error "Failed to create source tarball"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to upload release assets to Gitea
|
||||
upload_release_assets() {
|
||||
local release_id="$1"
|
||||
local binary_path="$2"
|
||||
local tarball_path="$3"
|
||||
|
||||
print_status "Uploading release assets..."
|
||||
|
||||
# Check for Gitea token
|
||||
if [[ ! -f "$HOME/.gitea_token" ]]; then
|
||||
print_warning "No ~/.gitea_token found. Skipping asset uploads."
|
||||
return 0
|
||||
fi
|
||||
|
||||
local token=$(cat "$HOME/.gitea_token" | tr -d '\n\r')
|
||||
local api_url="https://git.laantungir.net/api/v1/repos/laantungir/c-relay-pg"
|
||||
local assets_url="$api_url/releases/$release_id/assets"
|
||||
print_status "Assets URL: $assets_url"
|
||||
|
||||
# Upload binary
|
||||
if [[ -f "$binary_path" ]]; then
|
||||
print_status "Uploading binary: $(basename "$binary_path")"
|
||||
|
||||
# Retry loop for eventual consistency
|
||||
local max_attempts=3
|
||||
local attempt=1
|
||||
while [[ $attempt -le $max_attempts ]]; do
|
||||
print_status "Upload attempt $attempt/$max_attempts"
|
||||
local binary_response=$(curl -fS -X POST "$assets_url" \
|
||||
-H "Authorization: token $token" \
|
||||
-F "attachment=@$binary_path;filename=$(basename "$binary_path")" \
|
||||
-F "name=$(basename "$binary_path")")
|
||||
|
||||
if echo "$binary_response" | grep -q '"id"'; then
|
||||
print_success "Uploaded binary successfully"
|
||||
break
|
||||
else
|
||||
print_warning "Upload attempt $attempt failed"
|
||||
if [[ $attempt -lt $max_attempts ]]; then
|
||||
print_status "Retrying in 2 seconds..."
|
||||
sleep 2
|
||||
else
|
||||
print_error "Failed to upload binary after $max_attempts attempts"
|
||||
print_error "Response: $binary_response"
|
||||
fi
|
||||
fi
|
||||
((attempt++))
|
||||
done
|
||||
fi
|
||||
|
||||
# Upload source tarball
|
||||
if [[ -f "$tarball_path" ]]; then
|
||||
print_status "Uploading source tarball: $(basename "$tarball_path")"
|
||||
local tarball_response=$(curl -s -X POST "$api_url/releases/$release_id/assets" \
|
||||
-H "Authorization: token $token" \
|
||||
-F "attachment=@$tarball_path;filename=$(basename "$tarball_path")")
|
||||
|
||||
if echo "$tarball_response" | grep -q '"id"'; then
|
||||
print_success "Uploaded source tarball successfully"
|
||||
else
|
||||
print_warning "Failed to upload source tarball: $tarball_response"
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to create Gitea release
|
||||
create_gitea_release() {
|
||||
print_status "Creating Gitea release..."
|
||||
@@ -273,21 +427,32 @@ create_gitea_release() {
|
||||
fi
|
||||
|
||||
local token=$(cat "$HOME/.gitea_token" | tr -d '\n\r')
|
||||
local api_url="https://git.laantungir.net/api/v1/repos/laantungir/c-relay"
|
||||
local api_url="https://git.laantungir.net/api/v1/repos/laantungir/c-relay-pg"
|
||||
|
||||
# Create release
|
||||
print_status "Creating release $NEW_VERSION..."
|
||||
local response=$(curl -s -X POST "$api_url/releases" \
|
||||
-H "Authorization: token $token" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{\"tag_name\": \"$NEW_VERSION\", \"name\": \"$NEW_VERSION\", \"body\": \"$COMMIT_MESSAGE\"}")
|
||||
-H "Authorization: token $token" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{\"tag_name\": \"$NEW_VERSION\", \"name\": \"$NEW_VERSION\", \"body\": \"$COMMIT_MESSAGE\"}")
|
||||
|
||||
if echo "$response" | grep -q '"id"'; then
|
||||
print_success "Created release $NEW_VERSION"
|
||||
return 0
|
||||
# Extract release ID for asset uploads
|
||||
local release_id=$(echo "$response" | grep -o '"id":[0-9]*' | head -1 | cut -d':' -f2)
|
||||
echo $release_id
|
||||
elif echo "$response" | grep -q "already exists"; then
|
||||
print_warning "Release $NEW_VERSION already exists"
|
||||
return 0
|
||||
# Try to get existing release ID
|
||||
local check_response=$(curl -s -H "Authorization: token $token" "$api_url/releases/tags/$NEW_VERSION")
|
||||
if echo "$check_response" | grep -q '"id"'; then
|
||||
local release_id=$(echo "$check_response" | grep -o '"id":[0-9]*' | head -1 | cut -d':' -f2)
|
||||
print_status "Using existing release ID: $release_id"
|
||||
echo $release_id
|
||||
else
|
||||
print_error "Could not find existing release ID"
|
||||
return 1
|
||||
fi
|
||||
else
|
||||
print_error "Failed to create release $NEW_VERSION"
|
||||
print_error "Response: $response"
|
||||
@@ -297,7 +462,8 @@ create_gitea_release() {
|
||||
local check_response=$(curl -s -H "Authorization: token $token" "$api_url/releases/tags/$NEW_VERSION")
|
||||
if echo "$check_response" | grep -q '"id"'; then
|
||||
print_warning "Release exists but creation response was unexpected"
|
||||
return 0
|
||||
local release_id=$(echo "$check_response" | grep -o '"id":[0-9]*' | head -1 | cut -d':' -f2)
|
||||
echo $release_id
|
||||
else
|
||||
print_error "Release does not exist and creation failed"
|
||||
return 1
|
||||
@@ -307,7 +473,7 @@ create_gitea_release() {
|
||||
|
||||
# Main execution
|
||||
main() {
|
||||
print_status "C-Relay Increment and Push Script"
|
||||
print_status "C-Relay-PG Increment and Push Script"
|
||||
|
||||
# Check prerequisites
|
||||
check_git_repo
|
||||
@@ -315,8 +481,15 @@ main() {
|
||||
if [[ "$RELEASE_MODE" == true ]]; then
|
||||
print_status "=== RELEASE MODE ==="
|
||||
|
||||
# Increment minor version for releases
|
||||
increment_version "minor"
|
||||
# Only increment version if explicitly requested (not just because of -r flag)
|
||||
if [[ "$VERSION_INCREMENT_TYPE" != "patch" ]]; then
|
||||
increment_version "$VERSION_INCREMENT_TYPE"
|
||||
else
|
||||
# In release mode without version increment, get current version
|
||||
LATEST_TAG=$(git tag -l 'v*.*.*' | sort -V | tail -n 1 || echo "v0.0.0")
|
||||
NEW_VERSION="$LATEST_TAG"
|
||||
export NEW_VERSION
|
||||
fi
|
||||
|
||||
# Create new git tag BEFORE compilation so version.h picks it up
|
||||
if git tag "$NEW_VERSION" > /dev/null 2>&1; then
|
||||
@@ -330,9 +503,42 @@ main() {
|
||||
# Commit and push (but skip tag creation since we already did it)
|
||||
git_commit_and_push_no_tag
|
||||
|
||||
# Build release binary
|
||||
if build_release_binary; then
|
||||
local binary_path="build/c_relay_pg_static_x86_64"
|
||||
else
|
||||
print_warning "Binary build failed, continuing with release creation"
|
||||
# Check if binary exists from previous build
|
||||
if [[ -f "build/c_relay_pg_static_x86_64" ]]; then
|
||||
print_status "Using existing binary from previous build"
|
||||
binary_path="build/c_relay_pg_static_x86_64"
|
||||
else
|
||||
binary_path=""
|
||||
fi
|
||||
fi
|
||||
|
||||
# Create source tarball
|
||||
local tarball_path=""
|
||||
if tarball_path=$(create_source_tarball); then
|
||||
: # tarball_path is set by the function
|
||||
else
|
||||
print_warning "Source tarball creation failed, continuing with release creation"
|
||||
fi
|
||||
|
||||
# Create Gitea release
|
||||
if create_gitea_release; then
|
||||
print_success "Release $NEW_VERSION completed successfully!"
|
||||
local release_id=""
|
||||
if release_id=$(create_gitea_release); then
|
||||
# Validate release_id is numeric
|
||||
if [[ "$release_id" =~ ^[0-9]+$ ]]; then
|
||||
# Upload assets if we have a release ID and assets
|
||||
if [[ -n "$release_id" && (-n "$binary_path" || -n "$tarball_path") ]]; then
|
||||
upload_release_assets "$release_id" "$binary_path" "$tarball_path"
|
||||
fi
|
||||
print_success "Release $NEW_VERSION completed successfully!"
|
||||
else
|
||||
print_error "Invalid release_id: $release_id"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
print_error "Release creation failed"
|
||||
fi
|
||||
@@ -340,8 +546,8 @@ main() {
|
||||
else
|
||||
print_status "=== DEFAULT MODE ==="
|
||||
|
||||
# Increment patch version for regular commits
|
||||
increment_version "patch"
|
||||
# Increment version based on type (default to patch)
|
||||
increment_version "$VERSION_INCREMENT_TYPE"
|
||||
|
||||
# Create new git tag BEFORE compilation so version.h picks it up
|
||||
if git tag "$NEW_VERSION" > /dev/null 2>&1; then
|
||||
|
||||
+251
-15
@@ -1,6 +1,6 @@
|
||||
#!/bin/bash
|
||||
|
||||
# C-Relay Build and Restart Script
|
||||
# C-Relay-PG Build and Restart Script
|
||||
# Builds the project first, then stops any running relay and starts a new one in the background
|
||||
|
||||
echo "=== C Nostr Relay Build and Restart Script ==="
|
||||
@@ -13,6 +13,13 @@ ADMIN_KEY=""
|
||||
RELAY_KEY=""
|
||||
PORT_OVERRIDE=""
|
||||
DEBUG_LEVEL="5"
|
||||
DB_BACKEND="postgres"
|
||||
DB_CONNSTRING=""
|
||||
DB_HOST=""
|
||||
DB_PORT=""
|
||||
DB_NAME=""
|
||||
DB_USER=""
|
||||
DB_PASSWORD=""
|
||||
|
||||
# Key validation function
|
||||
validate_hex_key() {
|
||||
@@ -70,6 +77,24 @@ while [[ $# -gt 0 ]]; do
|
||||
;;
|
||||
--test-keys|-t)
|
||||
USE_TEST_KEYS=true
|
||||
# Read keys from .test_keys file
|
||||
if [ -f ".test_keys" ]; then
|
||||
echo "Reading test keys from .test_keys file..."
|
||||
# Source the file to get the variables
|
||||
source .test_keys
|
||||
# Remove any single quotes from the values
|
||||
# Note: -a flag expects ADMIN_PUBKEY (public key), not ADMIN_PRIVKEY
|
||||
ADMIN_KEY=$(echo "$ADMIN_PUBKEY" | tr -d "'")
|
||||
RELAY_KEY=$(echo "$SERVER_PRIVKEY" | tr -d "'")
|
||||
echo "Using admin pubkey from .test_keys: ${ADMIN_KEY:0:16}..."
|
||||
echo "Using relay privkey from .test_keys: ${RELAY_KEY:0:16}..."
|
||||
else
|
||||
echo "ERROR: .test_keys file not found"
|
||||
echo "Please create a .test_keys file with the following format:"
|
||||
echo " ADMIN_PUBKEY='your_admin_public_key_hex'"
|
||||
echo " SERVER_PRIVKEY='your_relay_private_key_hex'"
|
||||
exit 1
|
||||
fi
|
||||
shift
|
||||
;;
|
||||
--debug-level=*)
|
||||
@@ -100,6 +125,84 @@ while [[ $# -gt 0 ]]; do
|
||||
shift 2
|
||||
fi
|
||||
;;
|
||||
--db-backend)
|
||||
if [ -z "$2" ]; then
|
||||
echo "ERROR: --db-backend requires a value (sqlite|postgres)"
|
||||
HELP=true
|
||||
shift
|
||||
else
|
||||
DB_BACKEND="$2"
|
||||
shift 2
|
||||
fi
|
||||
;;
|
||||
--db-backend=*)
|
||||
DB_BACKEND="${1#*=}"
|
||||
shift
|
||||
;;
|
||||
--db-connstring)
|
||||
if [ -z "$2" ]; then
|
||||
echo "ERROR: --db-connstring requires a value"
|
||||
HELP=true
|
||||
shift
|
||||
else
|
||||
DB_CONNSTRING="$2"
|
||||
shift 2
|
||||
fi
|
||||
;;
|
||||
--db-connstring=*)
|
||||
DB_CONNSTRING="${1#*=}"
|
||||
shift
|
||||
;;
|
||||
--db-host)
|
||||
if [ -z "$2" ]; then
|
||||
echo "ERROR: --db-host requires a value"
|
||||
HELP=true
|
||||
shift
|
||||
else
|
||||
DB_HOST="$2"
|
||||
shift 2
|
||||
fi
|
||||
;;
|
||||
--db-port)
|
||||
if [ -z "$2" ]; then
|
||||
echo "ERROR: --db-port requires a value"
|
||||
HELP=true
|
||||
shift
|
||||
else
|
||||
DB_PORT="$2"
|
||||
shift 2
|
||||
fi
|
||||
;;
|
||||
--db-name)
|
||||
if [ -z "$2" ]; then
|
||||
echo "ERROR: --db-name requires a value"
|
||||
HELP=true
|
||||
shift
|
||||
else
|
||||
DB_NAME="$2"
|
||||
shift 2
|
||||
fi
|
||||
;;
|
||||
--db-user)
|
||||
if [ -z "$2" ]; then
|
||||
echo "ERROR: --db-user requires a value"
|
||||
HELP=true
|
||||
shift
|
||||
else
|
||||
DB_USER="$2"
|
||||
shift 2
|
||||
fi
|
||||
;;
|
||||
--db-password)
|
||||
if [ -z "$2" ]; then
|
||||
echo "ERROR: --db-password requires a value"
|
||||
HELP=true
|
||||
shift
|
||||
else
|
||||
DB_PASSWORD="$2"
|
||||
shift 2
|
||||
fi
|
||||
;;
|
||||
--help|-h)
|
||||
HELP=true
|
||||
shift
|
||||
@@ -146,6 +249,98 @@ if [ -n "$DEBUG_LEVEL" ]; then
|
||||
fi
|
||||
fi
|
||||
|
||||
# Validate DB backend
|
||||
if [ "$DB_BACKEND" != "sqlite" ] && [ "$DB_BACKEND" != "postgres" ]; then
|
||||
echo "ERROR: Invalid --db-backend value '$DB_BACKEND'. Use sqlite or postgres."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Validate DB port if provided
|
||||
if [ -n "$DB_PORT" ]; then
|
||||
if ! [[ "$DB_PORT" =~ ^[0-9]+$ ]] || [ "$DB_PORT" -lt 1 ] || [ "$DB_PORT" -gt 65535 ]; then
|
||||
echo "ERROR: --db-port must be a number between 1 and 65535"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$DB_BACKEND" = "postgres" ] && [ -z "$DB_CONNSTRING" ]; then
|
||||
[ -z "$DB_HOST" ] && DB_HOST="localhost"
|
||||
[ -z "$DB_PORT" ] && DB_PORT="5432"
|
||||
[ -z "$DB_NAME" ] && DB_NAME="crelay"
|
||||
[ -z "$DB_USER" ] && DB_USER="crelay"
|
||||
[ -z "$DB_PASSWORD" ] && DB_PASSWORD="crelay"
|
||||
fi
|
||||
|
||||
ensure_postgres_database() {
|
||||
if ! command -v psql >/dev/null 2>&1; then
|
||||
echo "ERROR: psql not found. Install PostgreSQL client tools to continue."
|
||||
return 1
|
||||
fi
|
||||
|
||||
echo "Ensuring PostgreSQL database exists: host=$DB_HOST port=$DB_PORT dbname=$DB_NAME user=$DB_USER"
|
||||
|
||||
if [ -n "$DB_PASSWORD" ]; then
|
||||
export PGPASSWORD="$DB_PASSWORD"
|
||||
fi
|
||||
|
||||
# Prefer local postgres superuser for provisioning when available.
|
||||
if sudo -n -u postgres psql -d postgres -tAc "SELECT 1" >/dev/null 2>&1; then
|
||||
sudo -u postgres psql -d postgres -tAc "SELECT 1 FROM pg_roles WHERE rolname = '$DB_USER'" | grep -q 1 || \
|
||||
sudo -u postgres psql -d postgres -v ON_ERROR_STOP=1 -c "CREATE ROLE \"$DB_USER\" LOGIN PASSWORD '$DB_PASSWORD';" || return 1
|
||||
|
||||
sudo -u postgres psql -d postgres -v ON_ERROR_STOP=1 -c "ALTER ROLE \"$DB_USER\" WITH LOGIN PASSWORD '$DB_PASSWORD';" >/dev/null || return 1
|
||||
|
||||
if [ "$PRESERVE_DATABASE" = false ]; then
|
||||
echo "Resetting PostgreSQL database '$DB_NAME' for fresh start..."
|
||||
|
||||
# Ensure no active sessions block DROP DATABASE (common when relay is still running).
|
||||
sudo -u postgres psql -d postgres -v ON_ERROR_STOP=1 -c \
|
||||
"SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = '$DB_NAME' AND pid <> pg_backend_pid();" >/dev/null || return 1
|
||||
|
||||
# Prefer force drop when supported; fallback to regular dropdb.
|
||||
if ! sudo -u postgres dropdb --if-exists --force "$DB_NAME" >/dev/null 2>&1; then
|
||||
sudo -u postgres dropdb --if-exists "$DB_NAME" >/dev/null 2>&1 || return 1
|
||||
fi
|
||||
|
||||
sudo -u postgres createdb -O "$DB_USER" "$DB_NAME" >/dev/null 2>&1 || return 1
|
||||
echo "✓ Recreated PostgreSQL database '$DB_NAME' owned by '$DB_USER'"
|
||||
else
|
||||
DB_EXISTS=$(sudo -u postgres psql -d postgres -tAc "SELECT 1 FROM pg_database WHERE datname = '$DB_NAME'" 2>/dev/null | tr -d '[:space:]')
|
||||
if [ "$DB_EXISTS" != "1" ]; then
|
||||
sudo -u postgres createdb -O "$DB_USER" "$DB_NAME" >/dev/null 2>&1 || return 1
|
||||
echo "✓ Created PostgreSQL database '$DB_NAME'"
|
||||
else
|
||||
echo "✓ PostgreSQL database '$DB_NAME' already exists"
|
||||
fi
|
||||
fi
|
||||
|
||||
sudo -u postgres psql -d postgres -v ON_ERROR_STOP=1 -c "GRANT ALL PRIVILEGES ON DATABASE \"$DB_NAME\" TO \"$DB_USER\";" >/dev/null || return 1
|
||||
sudo -u postgres psql -d "$DB_NAME" -v ON_ERROR_STOP=1 -c "ALTER SCHEMA public OWNER TO \"$DB_USER\"; GRANT ALL ON SCHEMA public TO \"$DB_USER\";" >/dev/null || return 1
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Fallback: use configured DB user directly.
|
||||
if ! psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -v ON_ERROR_STOP=1 -tAc "SELECT 1" >/dev/null 2>&1; then
|
||||
echo "ERROR: Cannot connect to PostgreSQL server/database 'postgres' with current settings"
|
||||
echo " host=$DB_HOST port=$DB_PORT user=$DB_USER"
|
||||
return 1
|
||||
fi
|
||||
|
||||
DB_EXISTS=$(psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -tAc "SELECT 1 FROM pg_database WHERE datname = '$DB_NAME'" 2>/dev/null | tr -d '[:space:]')
|
||||
if [ "$DB_EXISTS" != "1" ]; then
|
||||
echo "Database '$DB_NAME' not found. Creating..."
|
||||
if ! createdb -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" "$DB_NAME" >/dev/null 2>&1; then
|
||||
echo "ERROR: Failed to create PostgreSQL database '$DB_NAME'"
|
||||
return 1
|
||||
fi
|
||||
echo "✓ Created PostgreSQL database '$DB_NAME'"
|
||||
else
|
||||
echo "✓ PostgreSQL database '$DB_NAME' already exists"
|
||||
fi
|
||||
|
||||
return 0
|
||||
}
|
||||
|
||||
# Show help
|
||||
if [ "$HELP" = true ]; then
|
||||
echo "Usage: $0 [OPTIONS]"
|
||||
@@ -157,6 +352,13 @@ if [ "$HELP" = true ]; then
|
||||
echo " -d, --debug-level <0-5> Set debug level: 0=none, 1=errors, 2=warnings, 3=info, 4=debug, 5=trace"
|
||||
echo " --preserve-database Keep existing database files (don't delete for fresh start)"
|
||||
echo " --test-keys, -t Use deterministic test keys for development (admin: all 'a's, relay: all '1's)"
|
||||
echo " --db-backend <name> Database backend: postgres (default) or sqlite"
|
||||
echo " --db-connstring <str> PostgreSQL libpq connection string"
|
||||
echo " --db-host <host> PostgreSQL host"
|
||||
echo " --db-port <port> PostgreSQL port"
|
||||
echo " --db-name <name> PostgreSQL database name"
|
||||
echo " --db-user <user> PostgreSQL database user"
|
||||
echo " --db-password <pass> PostgreSQL database password"
|
||||
echo " --help, -h Show this help message"
|
||||
echo ""
|
||||
echo "Event-Based Configuration:"
|
||||
@@ -176,6 +378,9 @@ if [ "$HELP" = true ]; then
|
||||
echo " $0 --test-keys # Use test keys for consistent development"
|
||||
echo " $0 -t --preserve-database # Use test keys and preserve database"
|
||||
echo ""
|
||||
echo "Default PostgreSQL connection (when no DB flags provided):"
|
||||
echo " host=localhost port=5432 dbname=crelay user=crelay password=crelay"
|
||||
echo ""
|
||||
echo "Key Format: Keys must be exactly 64 hexadecimal characters (0-9, a-f, A-F)"
|
||||
echo "Default behavior: Deletes existing database files to start fresh with new keys"
|
||||
echo " for development purposes"
|
||||
@@ -199,13 +404,17 @@ fi
|
||||
rm -rf dev-config/ 2>/dev/null
|
||||
rm -f db/c_nostr_relay.db* 2>/dev/null
|
||||
|
||||
if [ "$DB_BACKEND" = "postgres" ] && [ -z "$DB_CONNSTRING" ]; then
|
||||
ensure_postgres_database || exit 1
|
||||
fi
|
||||
|
||||
# Embed web files into C headers before building
|
||||
echo "Embedding web files..."
|
||||
./embed_web_files.sh
|
||||
|
||||
# Build the project - ONLY static build
|
||||
echo "Building project (static binary with SQLite JSON1 extension)..."
|
||||
./build_static.sh
|
||||
echo "Building project (static binary, backend: $DB_BACKEND)..."
|
||||
./build_static.sh --db-backend "$DB_BACKEND"
|
||||
|
||||
# Exit if static build fails - no fallback
|
||||
if [ $? -ne 0 ]; then
|
||||
@@ -224,13 +433,13 @@ fi
|
||||
ARCH=$(uname -m)
|
||||
case "$ARCH" in
|
||||
x86_64)
|
||||
BINARY_PATH="./build/c_relay_static_x86_64"
|
||||
BINARY_PATH="./build/c_relay_pg_static_x86_64"
|
||||
;;
|
||||
aarch64|arm64)
|
||||
BINARY_PATH="./build/c_relay_static_arm64"
|
||||
BINARY_PATH="./build/c_relay_pg_static_arm64"
|
||||
;;
|
||||
*)
|
||||
BINARY_PATH="./build/c_relay_static_$ARCH"
|
||||
BINARY_PATH="./build/c_relay_pg_static_$ARCH"
|
||||
;;
|
||||
esac
|
||||
|
||||
@@ -252,7 +461,7 @@ echo "Build successful. Proceeding with relay restart..."
|
||||
echo "Stopping any existing relay servers..."
|
||||
|
||||
# Get all relay processes and kill them immediately with -9
|
||||
RELAY_PIDS=$(pgrep -f "c_relay_" || echo "")
|
||||
RELAY_PIDS=$(pgrep -f "c_relay_pg_" || echo "")
|
||||
if [ -n "$RELAY_PIDS" ]; then
|
||||
echo "Force killing relay processes immediately: $RELAY_PIDS"
|
||||
kill -9 $RELAY_PIDS 2>/dev/null
|
||||
@@ -273,7 +482,7 @@ for attempt in {1..15}; do
|
||||
fuser -k 8888/tcp 2>/dev/null || true
|
||||
|
||||
# Double-check for any remaining relay processes
|
||||
REMAINING_PIDS=$(pgrep -f "c_relay_" || echo "")
|
||||
REMAINING_PIDS=$(pgrep -f "c_relay_pg_" || echo "")
|
||||
if [ -n "$REMAINING_PIDS" ]; then
|
||||
echo "Killing remaining relay processes: $REMAINING_PIDS"
|
||||
kill -9 $REMAINING_PIDS 2>/dev/null || true
|
||||
@@ -291,7 +500,7 @@ for attempt in {1..15}; do
|
||||
done
|
||||
|
||||
# Final safety check - ensure no relay processes remain
|
||||
FINAL_PIDS=$(pgrep -f "c_relay_" || echo "")
|
||||
FINAL_PIDS=$(pgrep -f "c_relay_pg_" || echo "")
|
||||
if [ -n "$FINAL_PIDS" ]; then
|
||||
echo "Final cleanup: killing processes $FINAL_PIDS"
|
||||
kill -9 $FINAL_PIDS 2>/dev/null || true
|
||||
@@ -307,7 +516,7 @@ echo "Database will be initialized automatically on startup if needed"
|
||||
|
||||
# Start relay in background with output redirection
|
||||
echo "Starting relay server..."
|
||||
echo "Debug: Current processes: $(ps aux | grep 'c_relay_' | grep -v grep || echo 'None')"
|
||||
echo "Debug: Current processes: $(ps aux | grep 'c_relay_pg_' | grep -v grep || echo 'None')"
|
||||
|
||||
# Build command line arguments for relay binary
|
||||
RELAY_ARGS=""
|
||||
@@ -332,15 +541,42 @@ if [ -n "$DEBUG_LEVEL" ]; then
|
||||
echo "Using debug level: $DEBUG_LEVEL"
|
||||
fi
|
||||
|
||||
if [ -n "$DB_CONNSTRING" ]; then
|
||||
RELAY_ARGS="$RELAY_ARGS --db-connstring '$DB_CONNSTRING'"
|
||||
echo "Using PostgreSQL connection string override"
|
||||
fi
|
||||
|
||||
if [ -n "$DB_HOST" ]; then
|
||||
RELAY_ARGS="$RELAY_ARGS --db-host '$DB_HOST'"
|
||||
fi
|
||||
|
||||
if [ -n "$DB_PORT" ]; then
|
||||
RELAY_ARGS="$RELAY_ARGS --db-port '$DB_PORT'"
|
||||
fi
|
||||
|
||||
if [ -n "$DB_NAME" ]; then
|
||||
RELAY_ARGS="$RELAY_ARGS --db-name '$DB_NAME'"
|
||||
fi
|
||||
|
||||
if [ -n "$DB_USER" ]; then
|
||||
RELAY_ARGS="$RELAY_ARGS --db-user '$DB_USER'"
|
||||
fi
|
||||
|
||||
if [ -n "$DB_PASSWORD" ]; then
|
||||
RELAY_ARGS="$RELAY_ARGS --db-password '$DB_PASSWORD'"
|
||||
fi
|
||||
|
||||
# Change to build directory before starting relay so database files are created there
|
||||
cd build
|
||||
# Start relay in background and capture its PID
|
||||
if [ "$USE_TEST_KEYS" = true ]; then
|
||||
echo "Using deterministic test keys for development..."
|
||||
./$(basename $BINARY_PATH) -a 6a04ab98d9e4774ad806e302dddeb63bea16b5cb5f223ee77478e861bb583eb3 -r 1111111111111111111111111111111111111111111111111111111111111111 --debug-level=$DEBUG_LEVEL --strict-port > ../relay.log 2>&1 &
|
||||
echo "Using test keys from .test_keys file..."
|
||||
# shellcheck disable=SC2086
|
||||
eval ./$(basename $BINARY_PATH) -a "$ADMIN_KEY" -r "$RELAY_KEY" $RELAY_ARGS --strict-port > ../relay.log 2>&1 &
|
||||
elif [ -n "$RELAY_ARGS" ]; then
|
||||
echo "Starting relay with custom configuration..."
|
||||
./$(basename $BINARY_PATH) $RELAY_ARGS --debug-level=$DEBUG_LEVEL --strict-port > ../relay.log 2>&1 &
|
||||
# shellcheck disable=SC2086
|
||||
eval ./$(basename $BINARY_PATH) $RELAY_ARGS --strict-port > ../relay.log 2>&1 &
|
||||
else
|
||||
# No command line arguments needed for random key generation
|
||||
echo "Starting relay with random key generation..."
|
||||
@@ -410,8 +646,8 @@ if ps -p "$RELAY_PID" >/dev/null 2>&1; then
|
||||
echo "=== Event-Based Relay Server Running ==="
|
||||
echo "Configuration: Event-based (kind 33334 Nostr events)"
|
||||
echo "Database: Automatically created with relay pubkey naming"
|
||||
echo "To kill relay: pkill -f 'c_relay_'"
|
||||
echo "To check status: ps aux | grep c_relay_"
|
||||
echo "To kill relay: pkill -f 'c_relay_pg_'"
|
||||
echo "To check status: ps aux | grep c_relay_pg_"
|
||||
echo "To view logs: tail -f relay.log"
|
||||
echo "Binary: $BINARY_PATH (zero configuration needed)"
|
||||
echo "Ready for Nostr client connections!"
|
||||
|
||||
+1
-1
@@ -1,3 +1,3 @@
|
||||
#!/bin/bash
|
||||
|
||||
curl -H "Accept: application/nostr+json" http://localhost:8888/
|
||||
curl -H "Accept: application/nostr+json" https://relay.laantungir.net
|
||||
|
||||
-1
Submodule nips deleted from 8c45ff5d96
+1
-1
Submodule nostr_core_lib updated: a8dc2ed046...98cfd81b01
@@ -12,11 +12,11 @@ After the next crash, analyze it:
|
||||
# List all core dumps (most recent first)
|
||||
sudo coredumpctl list
|
||||
|
||||
# View info about the most recent c-relay crash
|
||||
sudo coredumpctl info c-relay
|
||||
# View info about the most recent c-relay-pg crash
|
||||
sudo coredumpctl info c-relay-pg
|
||||
|
||||
# Load the core dump in gdb for detailed analysis
|
||||
sudo coredumpctl gdb c-relay
|
||||
sudo coredumpctl gdb c-relay-pg
|
||||
|
||||
Inside gdb, run these commands:
|
||||
(gdb) bt full # Full backtrace with all variables
|
||||
@@ -32,12 +32,12 @@ You're all set up now - just wait for the next crash and then run the coredumpct
|
||||
|
||||
|
||||
--------------------------
|
||||
|
||||
DEBUGGING
|
||||
|
||||
Even simpler: Use this one-liner
|
||||
# Start relay and immediately attach gdb
|
||||
cd /usr/local/bin/c_relay
|
||||
sudo -u c-relay ./c_relay --debug-level=5 & sleep 2 && sudo gdb -p $(pgrep c_relay)
|
||||
cd /usr/local/bin/c_relay_pg
|
||||
sudo -u c-relay-pg ./c_relay_pg --debug-level=5 & sleep 2 && sudo gdb -p $(pgrep c_relay_pg)
|
||||
|
||||
Inside gdb, after attaching:
|
||||
|
||||
@@ -48,17 +48,17 @@ Or shorter:
|
||||
|
||||
How to View the Logs
|
||||
Check systemd journal:
|
||||
# View all c-relay logs
|
||||
sudo journalctl -u c-relay
|
||||
# View all c-relay-pg logs
|
||||
sudo journalctl -u c-relay-pg
|
||||
|
||||
# View recent logs (last 50 lines)
|
||||
sudo journalctl -u c-relay -n 50
|
||||
sudo journalctl -u c-relay-pg -n 50
|
||||
|
||||
# Follow logs in real-time
|
||||
sudo journalctl -u c-relay -f
|
||||
sudo journalctl -u c-relay-pg -f
|
||||
|
||||
# View logs since last boot
|
||||
sudo journalctl -u c-relay -b
|
||||
sudo journalctl -u c-relay-pg -b
|
||||
|
||||
Check if service is running:
|
||||
|
||||
@@ -78,9 +78,9 @@ sudo systemctl start rsyslog
|
||||
sudo systemctl status rsyslog
|
||||
|
||||
|
||||
sudo -u c-relay ./c_relay --debug-level=5 -r 85d0b37e2ae822966dcadd06b2dc9368cde73865f90ea4d44f8b57d47ef0820a -a 1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139
|
||||
sudo -u c-relay-pg ./c_relay_pg --debug-level=5 -r 85d0b37e2ae822966dcadd06b2dc9368cde73865f90ea4d44f8b57d47ef0820a -a 1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139
|
||||
|
||||
./c_relay_static_x86_64 -p 7889 --debug-level=5 -r 85d0b37e2ae822966dcadd06b2dc9368cde73865f90ea4d44f8b57d47ef0820a -a 1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139
|
||||
./c_relay_pg_static_x86_64 -p 7889 --debug-level=5 -r 85d0b37e2ae822966dcadd06b2dc9368cde73865f90ea4d44f8b57d47ef0820a -a 1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139
|
||||
|
||||
|
||||
sudo ufw allow 8888/tcp
|
||||
@@ -88,4 +88,5 @@ sudo ufw delete allow 8888/tcp
|
||||
|
||||
lsof -i :7777
|
||||
kill $(lsof -t -i :7777)
|
||||
kill -9 $(lsof -t -i :7777)
|
||||
kill -9 $(lsof -t -i :7777)
|
||||
|
||||
|
||||
Generated
+1
-1
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"name": "c-relay",
|
||||
"name": "c-relay-pg",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
|
||||
@@ -0,0 +1,224 @@
|
||||
# API Upgrade Plan: Monitoring Thread + PostgreSQL Push
|
||||
|
||||
## Current API Architecture
|
||||
|
||||
### How Monitoring Works Today
|
||||
|
||||
The relay publishes monitoring data as **kind 24567 ephemeral Nostr events**. The admin dashboard is a standard Nostr client that subscribes to these events.
|
||||
|
||||
```
|
||||
Dashboard connects → subscribes to kind 24567 with d-tags
|
||||
Relay detects subscription → starts generating monitoring events
|
||||
Relay queries DB → builds JSON → signs event → broadcasts to subscribers
|
||||
Dashboard receives events → renders stats/charts
|
||||
```
|
||||
|
||||
### Current Monitoring Event Types
|
||||
|
||||
| d-tag | Query Function | What It Contains | Triggered By |
|
||||
|-------|---------------|-----------------|-------------|
|
||||
| `event_kinds` | `query_event_kind_distribution()` | Event count per kind, total events | Event storage |
|
||||
| `time_stats` | `query_time_based_statistics()` | Events in last 24h/7d/30d | Event storage |
|
||||
| `top_pubkeys` | `query_top_pubkeys()` | Top 10 pubkeys by event count | Event storage |
|
||||
| `subscription_details` | `query_subscription_details()` | Active subscriptions list | Subscription changes |
|
||||
| `cpu_metrics` | `query_cpu_metrics()` | PID, memory, CPU, connections | Both triggers |
|
||||
|
||||
### Current Admin Command System
|
||||
|
||||
Two parallel admin interfaces exist:
|
||||
|
||||
**1. Kind 23456 Admin Events (WebSocket)**
|
||||
- Admin sends NIP-44 encrypted commands as kind 23456 events
|
||||
- Relay decrypts, processes, responds with kind 23457 events
|
||||
- Commands: config get/set, auth rules, system commands, SQL queries
|
||||
|
||||
**2. NIP-17 DM Admin (Direct Messages)**
|
||||
- Admin sends gift-wrapped DMs (kind 1059) to relay
|
||||
- Relay unwraps, processes commands, responds via DM
|
||||
- Commands: stats, config, help, SQL queries
|
||||
|
||||
### Current Problems
|
||||
|
||||
1. **All monitoring runs on lws-main thread** — DB queries block the event loop
|
||||
2. **Monitoring is triggered by event storage** — `generate_event_driven_monitoring()` runs synchronously after each event insert
|
||||
3. **No dedicated monitoring thread** — monitoring piggybacks on the relay's main processing
|
||||
4. **Throttle is time-based only** — `kind_24567_reporting_throttle_sec` (default 5s) but queries still run on main thread
|
||||
5. **Dashboard polls via subscription** — no way to get data without the relay actively generating events
|
||||
|
||||
---
|
||||
|
||||
## Upgrade Plan
|
||||
|
||||
### Phase 1: Dedicated API Thread (`api-worker`)
|
||||
|
||||
**What:** Create a new thread that owns all monitoring event generation. It runs on a timer, queries PostgreSQL with its own connection, builds and signs monitoring events, and pushes them into the broadcast system.
|
||||
|
||||
**Thread behavior:**
|
||||
```
|
||||
api-worker thread:
|
||||
1. Open own PG connection
|
||||
2. Loop:
|
||||
a. Sleep for throttle_sec interval
|
||||
b. Check if anyone is subscribed to kind 24567
|
||||
c. If yes: run all monitoring queries
|
||||
d. Build kind 24567 events with results
|
||||
e. Sign events with relay key
|
||||
f. Push completed events to lws-main for broadcast
|
||||
g. Repeat
|
||||
```
|
||||
|
||||
**Key design decisions:**
|
||||
- The api-worker thread does ALL the heavy work (queries + signing)
|
||||
- It only hands off pre-built, signed events to lws-main for broadcast
|
||||
- lws-main never blocks on monitoring queries
|
||||
- The thread sleeps when no one is subscribed (zero overhead)
|
||||
|
||||
**Files to change:**
|
||||
- `src/api.c`: Refactor `generate_monitoring_event_for_type()` to be callable from any thread
|
||||
- `src/websockets.c`: Add monitor thread lifecycle (start/stop), completion queue for broadcast
|
||||
- `src/thread_pool.h`: Add api-worker thread config options
|
||||
|
||||
**New thread in the process:**
|
||||
```
|
||||
c_relay_pg process
|
||||
├── lws-main — WebSocket event loop
|
||||
├── db-read-1..N — Async REQ/COUNT queries
|
||||
├── event-worker — Async EVENT ingestion
|
||||
├── db-write-1..N — Async sub logging, misc writes
|
||||
└── api-worker — NEW: periodic monitoring event generation
|
||||
```
|
||||
|
||||
### Phase 2: Remove Monitoring from Main Thread Path
|
||||
|
||||
**What:** Remove `generate_event_driven_monitoring()` and `generate_subscription_driven_monitoring()` calls from the synchronous event processing path.
|
||||
|
||||
**Currently these are called from:**
|
||||
- After event storage (event-driven monitoring)
|
||||
- After subscription create/close (subscription-driven monitoring)
|
||||
|
||||
**After change:**
|
||||
- The api-worker thread handles all monitoring on its own timer
|
||||
- Event storage and subscription changes no longer trigger monitoring directly
|
||||
- The throttle interval controls how fresh the data is (default 5 seconds)
|
||||
|
||||
**Benefit:** Event processing becomes faster — no monitoring overhead per event.
|
||||
|
||||
### Phase 3: Dashboard Uses Normal Subscriptions for Real-Time Events
|
||||
|
||||
**What:** Instead of the relay generating special monitoring events for "recent events," the dashboard simply subscribes to all events using standard Nostr REQ filters.
|
||||
|
||||
**Current approach:** Dashboard subscribes to kind 24567 monitoring events that contain aggregated stats.
|
||||
|
||||
**New approach for real-time event feed:**
|
||||
```javascript
|
||||
// Dashboard subscribes to ALL events on the relay
|
||||
relayPool.subscribeMany([url], [
|
||||
{ kinds: [1, 3, 5, 7, ...], limit: 50 } // Recent events
|
||||
], {
|
||||
onevent(event) {
|
||||
// Render in real-time event feed
|
||||
addEventToFeed(event);
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
**What this replaces:** The "Relay Events" page in the dashboard currently shows events from monitoring. Instead, it would show live events as they arrive via normal Nostr subscription.
|
||||
|
||||
**What stays as monitoring events:** Aggregated stats (event_kinds, time_stats, top_pubkeys, cpu_metrics, subscription_details) still need to be generated by the relay because they require SQL aggregation queries that a client can't do.
|
||||
|
||||
### Phase 4: Evaluate Which Monitoring Events Can Be Replaced
|
||||
|
||||
For each current monitoring event type, here's whether it should stay as a relay-generated event or be replaced by dashboard-side logic:
|
||||
|
||||
| d-tag | Current: Relay generates | Could dashboard do it? | Recommendation |
|
||||
|-------|------------------------|----------------------|----------------|
|
||||
| `event_kinds` | SQL: `SELECT kind, COUNT(*) FROM events GROUP BY kind` | No — requires DB aggregation | **Keep as monitoring event** |
|
||||
| `time_stats` | SQL: `SELECT COUNT(*) FROM events WHERE created_at > ?` for 24h/7d/30d | No — requires DB aggregation | **Keep as monitoring event** |
|
||||
| `top_pubkeys` | SQL: `SELECT pubkey, COUNT(*) FROM events GROUP BY pubkey ORDER BY count DESC LIMIT 10` | No — requires DB aggregation | **Keep as monitoring event** |
|
||||
| `subscription_details` | SQL: `SELECT * FROM active_subscriptions_log` | No — requires DB access | **Keep as monitoring event** |
|
||||
| `cpu_metrics` | Reads `/proc/self/stat`, memory info | No — requires server-side access | **Keep as monitoring event** |
|
||||
| Real-time event feed | Currently via monitoring events | **Yes — normal Nostr subscription** | **Replace with REQ subscription** |
|
||||
| Auth rules list | Currently via admin command | No — requires DB access | **Keep as admin command** |
|
||||
| Config values | Currently via admin command | No — requires DB access | **Keep as admin command** |
|
||||
|
||||
**Conclusion:** All 5 monitoring event types should stay as relay-generated events (they require SQL aggregation or server-side data). The only thing that changes is the real-time event feed, which becomes a normal subscription.
|
||||
|
||||
### Phase 5: PostgreSQL LISTEN/NOTIFY for Monitoring Triggers (Future)
|
||||
|
||||
**What:** Instead of the monitor thread polling on a timer, PostgreSQL can push notifications when data changes.
|
||||
|
||||
**Example triggers:**
|
||||
```sql
|
||||
-- Notify when a new event is stored
|
||||
CREATE OR REPLACE FUNCTION notify_event_stored() RETURNS trigger AS $$
|
||||
BEGIN
|
||||
PERFORM pg_notify('event_stored', json_build_object(
|
||||
'kind', NEW.kind,
|
||||
'pubkey', substring(NEW.pubkey, 1, 8)
|
||||
)::text);
|
||||
RETURN NEW;
|
||||
END;
|
||||
$$ LANGUAGE plpgsql;
|
||||
|
||||
CREATE TRIGGER trg_notify_event_stored
|
||||
AFTER INSERT ON events
|
||||
FOR EACH ROW EXECUTE FUNCTION notify_event_stored();
|
||||
```
|
||||
|
||||
**How the monitor thread would use it:**
|
||||
```
|
||||
api-worker thread:
|
||||
1. LISTEN event_stored
|
||||
2. Wait for notification (blocks efficiently)
|
||||
3. On notification: batch up for throttle_sec, then generate monitoring events
|
||||
4. Repeat
|
||||
```
|
||||
|
||||
**Benefit:** Zero-polling. The monitor thread only wakes up when data actually changes.
|
||||
|
||||
**This is optional and can be added later.** The timer-based approach in Phase 1 works well and is simpler.
|
||||
|
||||
---
|
||||
|
||||
## Updated Thread Layout (After All Phases)
|
||||
|
||||
```
|
||||
c_relay_pg process
|
||||
├── lws-main — WebSocket event loop (never blocks on DB for monitoring)
|
||||
├── db-read-1..4 — Async REQ/COUNT queries
|
||||
├── event-worker — Async EVENT ingestion (validate + store)
|
||||
├── db-write-1..2 — Async sub logging, IP bans, misc writes
|
||||
└── api-worker — Periodic monitoring event generation (own PG conn)
|
||||
```
|
||||
|
||||
**Total: 9 threads, 9 PG connections** (with 4 readers + 2 writers)
|
||||
|
||||
---
|
||||
|
||||
## Implementation Priority
|
||||
|
||||
| Phase | Description | Complexity | Impact |
|
||||
|-------|-------------|-----------|--------|
|
||||
| 1 | Dedicated api-worker thread | Medium | High — unblocks main thread |
|
||||
| 2 | Remove monitoring from event processing path | Low | Medium — faster event processing |
|
||||
| 3 | Dashboard uses normal subscriptions for event feed | Low (frontend only) | Medium — simpler, more reliable |
|
||||
| 4 | Evaluate monitoring event types | Analysis only | Confirms architecture |
|
||||
| 5 | PostgreSQL LISTEN/NOTIFY triggers | Medium | Low (optimization) — add later |
|
||||
|
||||
**Recommended start:** Phase 1 + 2 together (they're coupled), then Phase 3 (frontend change), then Phase 5 when needed.
|
||||
|
||||
---
|
||||
|
||||
## Relationship to Thread Pool Plan
|
||||
|
||||
This plan complements `thread_pool_pg_optimization_plan.md`:
|
||||
|
||||
- Thread pool plan handles **reader/writer scaling** for client-facing operations
|
||||
- This plan handles **monitoring/API operations** on a dedicated thread
|
||||
- Both plans share the same PG connection model (one connection per thread)
|
||||
- The api-worker thread is independent from the thread pool — it has its own lifecycle
|
||||
|
||||
The combined target is:
|
||||
```
|
||||
lws-main (1) + readers (4) + event-worker (1) + writers (2) + api-worker (1) = 9 threads
|
||||
```
|
||||
@@ -0,0 +1,240 @@
|
||||
# C-Relay-PG Admin Page Migration Plan
|
||||
|
||||
## Overview
|
||||
|
||||
Migrate the relay admin dashboard from an embedded HTTP-served page (`api/index.html` + `api/index.js`) to a full client-ndk page (`c-relay-pg.html`) built on the client-ndk template. The page will live in the client-ndk project (`~/lt/client-ndk/www/`) alongside other pages and communicate with the relay purely through Nostr protocol subscriptions.
|
||||
|
||||
## Why This Migration
|
||||
|
||||
The current `/api` page is served directly by the relay's `lws-main` thread, which causes:
|
||||
- **Slow page delivery** — HTTP file serving competes with WebSocket event processing on a single thread
|
||||
- **Incomplete page loads** — 10-second `timeout_secs` kills connections when the main thread is busy
|
||||
- **6+ sequential HTTP round-trips** — each asset requires a two-phase header/body delivery cycle
|
||||
- **Unnecessary complexity** — embedded file generation, `lws_set_wsi_user()` overwriting, session data type dispatch
|
||||
|
||||
The admin page already communicates entirely via Nostr protocol:
|
||||
- **Kind 23456** → Admin commands (encrypted, sent by admin)
|
||||
- **Kind 23457** → Admin responses (encrypted, sent by relay)
|
||||
- **Kind 24567** → Monitoring/stats events (ephemeral, broadcast by relay)
|
||||
- **NIP-11** → Relay info (fetched via HTTP, independent of file serving)
|
||||
- **NIP-17** → DM admin commands (gift-wrapped)
|
||||
|
||||
Moving to client-ndk means the relay serves zero HTML — it only speaks Nostr protocol.
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph client-ndk project
|
||||
A[c-relay-pg.html] --> B[init-ndk.mjs]
|
||||
B --> C[NDK SharedWorker]
|
||||
end
|
||||
|
||||
subgraph c-relay-pg
|
||||
D[WebSocket Server]
|
||||
E[api-worker thread]
|
||||
end
|
||||
|
||||
C -->|kind 23456 admin cmds| D
|
||||
D -->|kind 23457 responses| C
|
||||
E -->|kind 24567 monitoring| D
|
||||
D -->|kind 24567 events| C
|
||||
C -->|kind 1059 NIP-17 DMs| D
|
||||
```
|
||||
|
||||
## Current Admin Page Sections
|
||||
|
||||
| Section | Data Source | Communication |
|
||||
|---------|-----------|---------------|
|
||||
| Statistics | kind 24567 monitoring events with d-tags: event_kinds, time_stats, top_pubkeys, cpu_metrics | Subscribe to kind 24567 |
|
||||
| Subscriptions | kind 24567 monitoring events with d-tag: subscription_details | Subscribe to kind 24567 |
|
||||
| Configuration | kind 23457 admin responses | Send kind 23456 config_query, receive kind 23457 |
|
||||
| Authorization | kind 23457 admin responses | Send kind 23456 auth rule commands, receive kind 23457 |
|
||||
| IP Bans | kind 23457 admin responses | Send kind 23456 ip_ban_* commands, receive kind 23457 |
|
||||
| Relay Events | Live subscription to all events + kind 0/10050/10002 management | Standard Nostr subscriptions |
|
||||
| DM | NIP-17 gift-wrapped DMs kind 1059 | NIP-17 encrypt/wrap/publish |
|
||||
| SQL Query | kind 23457 admin responses | Send kind 23456 sql_query, receive kind 23457 |
|
||||
|
||||
## Design Decisions
|
||||
|
||||
1. **Full client-ndk page** — Uses `client.css`, `init-ndk.mjs`, `HamburgerMorphing`, `relay-ui.mjs`, `blossom-ui.mjs`, `ai-ui.mjs` — all standard template infrastructure stays intact
|
||||
2. **Sidenav preserved** — All existing sidenav sections (Relay, Blossom, AI) remain. Admin page navigation links are added to `divSideNavBody` as clickable items
|
||||
3. **Page-specific CSS** — Follows client-ndk convention: page styles in inline `<style>` block, using only CSS variables from `client.css`
|
||||
4. **Hardcoded relay URL** — `wss://relay.laantungir.net` for now; future enhancement to support relay selection
|
||||
5. **Auth mode: required** — Admin page requires authentication
|
||||
|
||||
## Migration Steps
|
||||
|
||||
### Step 1: Create `c-relay-pg.html` from Template
|
||||
|
||||
Start from `template.html` in client-ndk. Customize:
|
||||
|
||||
- **Title**: `C-Relay-PG`
|
||||
- **Header text**: `C-Relay-PG Admin`
|
||||
- **Auth mode**: `required` (default)
|
||||
- **`divBody` content**: All 8 admin sections, each in its own container div, all `display:none` except Statistics (default)
|
||||
- **`divSideNavBody`**: Add navigation links for each section:
|
||||
```html
|
||||
<div id="divAdminNav">
|
||||
<div class="adminNavItem active" data-section="statistics">Statistics</div>
|
||||
<div class="adminNavItem" data-section="subscriptions">Subscriptions</div>
|
||||
<div class="adminNavItem" data-section="configuration">Configuration</div>
|
||||
<div class="adminNavItem" data-section="authorization">Authorization</div>
|
||||
<div class="adminNavItem" data-section="ip-bans">IP Bans</div>
|
||||
<div class="adminNavItem" data-section="relay-events">Relay Events</div>
|
||||
<div class="adminNavItem" data-section="dm">DM</div>
|
||||
<div class="adminNavItem" data-section="database">Database Query</div>
|
||||
</div>
|
||||
```
|
||||
|
||||
### Step 2: Port Admin Communication Layer
|
||||
|
||||
Replace `SimplePool` + `nostr-lite.js` with NDK equivalents:
|
||||
|
||||
| Current index.js | New c-relay-pg.html |
|
||||
|-----------------|---------------------|
|
||||
| `new SimplePool()` | `subscribe()` from init-ndk.mjs |
|
||||
| `relayPool.subscribeMany()` | `subscribe(filter, opts)` |
|
||||
| `relayPool.publish()` | `publishEvent(event)` |
|
||||
| `window.nostr.signEvent()` | Handled by NDK MessageBasedSigner |
|
||||
| `window.NostrTools.nip42` | Handled by NDK NIP-42 support |
|
||||
| `window.NostrTools.nip44` | `window.NostrTools.nip44` from nostr.bundle.js — same as before |
|
||||
| `nlLite` nostr-login-lite | `initNDKPage()` + `getPubkey()` |
|
||||
|
||||
Key function — `sendAdminCommand(commandArray)`:
|
||||
```javascript
|
||||
async function sendAdminCommand(commandArray) {
|
||||
const conversationKey = window.NostrTools.nip44.v2.utils.getConversationKey(
|
||||
adminPrivateKey, relayPubkey
|
||||
);
|
||||
const encrypted = window.NostrTools.nip44.v2.encrypt(
|
||||
JSON.stringify(commandArray), conversationKey
|
||||
);
|
||||
|
||||
const event = {
|
||||
created_at: Math.floor(Date.now() / 1000),
|
||||
kind: 23456,
|
||||
tags: [['p', relayPubkey]],
|
||||
content: encrypted
|
||||
};
|
||||
|
||||
return await publishEvent(event);
|
||||
}
|
||||
```
|
||||
|
||||
Note: NIP-44 encryption requires the admin's private key. The current page uses `window.nostr.nip44.encrypt()` which delegates to the browser extension. NDK's `publishEvent` handles signing, but encryption must be done before publishing. We'll use `window.nostr.nip44.encrypt()` if available (NIP-07 extension), same as the current page does.
|
||||
|
||||
### Step 3: Port Subscription Setup
|
||||
|
||||
Using NDK's `subscribe()` and `window.addEventListener('ndkEvent', ...)`:
|
||||
|
||||
```javascript
|
||||
// Subscribe to admin responses + monitoring events from relay
|
||||
const adminSub = subscribe(
|
||||
{ kinds: [23457, 24567], authors: [RELAY_PUBKEY] },
|
||||
{ closeOnEose: false, cacheUsage: 'ONLY_RELAY' }
|
||||
);
|
||||
|
||||
// Live event feed
|
||||
const liveSub = subscribe(
|
||||
{ limit: 50 },
|
||||
{ closeOnEose: false, cacheUsage: 'ONLY_RELAY' }
|
||||
);
|
||||
|
||||
// Route events to handlers
|
||||
window.addEventListener('ndkEvent', (e) => {
|
||||
const evt = e.detail;
|
||||
if (evt.kind === 23457) processAdminResponse(evt);
|
||||
else if (evt.kind === 24567) processMonitoringEvent(evt);
|
||||
else addEventToLiveFeed(evt);
|
||||
});
|
||||
```
|
||||
|
||||
### Step 4: Port NIP-44 Encryption
|
||||
|
||||
Admin commands require NIP-44 encryption. The template already loads `nostr.bundle.js` which includes `window.NostrTools.nip44`. Additionally, `window.nostr.nip44.encrypt()` is available from NIP-07 extensions. Use the same approach as the current page.
|
||||
|
||||
### Step 5: Port UI Sections into divBody
|
||||
|
||||
Each section's HTML goes into `divBody` as a container div. Page-specific styles go in the inline `<style>` block following client-ndk conventions:
|
||||
- Use CSS variables only (no hardcoded colors)
|
||||
- Tables use `var(--primary-color)`, `var(--muted-color)`, `var(--accent-color)`
|
||||
- Buttons follow the `var(--button-*)` pattern
|
||||
- Font is always `var(--font-family)`
|
||||
|
||||
Sections to port:
|
||||
1. **Statistics** — DB overview table, event kinds table, time stats table, top pubkeys table, event rate chart area
|
||||
2. **Subscriptions** — Active subscription details table
|
||||
3. **Configuration** — Config key/value table with inline edit, refresh button
|
||||
4. **Authorization** — Auth rules table, whitelist/blacklist inputs, WoT level selector
|
||||
5. **IP Bans** — Ban stats, manual ban form, IP whitelist, ban list table with filters
|
||||
6. **Relay Events** — Live event feed table, kind 0/10050/10002 management forms
|
||||
7. **DM** — NIP-17 message textarea + send button, inbox display
|
||||
8. **SQL Query** — Query dropdown, SQL textarea, execute button, results table
|
||||
|
||||
### Step 6: Page-Specific CSS
|
||||
|
||||
Following client-ndk rules from `client.css`:
|
||||
- Override `#divBody` layout: `flex-direction: column; overflow-y: auto; padding: 20px;`
|
||||
- Admin section containers: hidden by default, shown when nav item clicked
|
||||
- Table styles: use `var(--primary-color)` for headers, `var(--muted-color)` for borders
|
||||
- Button styles: border + transparent background, hover with `var(--accent-color)`
|
||||
- All in inline `<style media="screen">` block in `<head>`
|
||||
|
||||
### Step 7: Relay URL — Hardcoded for Now
|
||||
|
||||
```javascript
|
||||
const RELAY_WS_URL = 'wss://relay.laantungir.net';
|
||||
const RELAY_HTTP_URL = 'https://relay.laantungir.net';
|
||||
```
|
||||
|
||||
NIP-11 fetch to get relay pubkey:
|
||||
```javascript
|
||||
const nip11 = await fetch(RELAY_HTTP_URL, {
|
||||
headers: { 'Accept': 'application/nostr+json' }
|
||||
}).then(r => r.json());
|
||||
const RELAY_PUBKEY = nip11.pubkey;
|
||||
```
|
||||
|
||||
### Step 8: Admin Verification
|
||||
|
||||
Same flow as current page:
|
||||
1. Get relay pubkey from NIP-11
|
||||
2. Send kind 23456 `system_status` command
|
||||
3. If relay responds with kind 23457, user is verified as admin
|
||||
4. Show admin sections; otherwise show access denied
|
||||
|
||||
### Step 9 (Optional, Later): Remove Embedded HTTP Serving from Relay
|
||||
|
||||
After migration is complete and tested:
|
||||
- Remove `handle_embedded_file_request()` from `src/api.c`
|
||||
- Remove `handle_embedded_file_writeable()` from `src/api.c`
|
||||
- Remove embedded file dispatch from `LWS_CALLBACK_HTTP` in `src/websockets.c`
|
||||
- Remove `embed_web_files.sh` and `src/embedded_web_content.c`
|
||||
- Keep NIP-11 serving (lightweight and required by protocol)
|
||||
|
||||
## File Changes Summary
|
||||
|
||||
### New File (in client-ndk project: ~/lt/client-ndk/www/)
|
||||
- `c-relay-pg.html` — Complete admin page, self-contained with inline `<style>` and `<script type="module">`
|
||||
|
||||
### Files Unchanged
|
||||
- All relay C code for kind 23456/23457/24567 processing — stays exactly the same
|
||||
- `src/dm_admin.c` — NIP-17 DM handling stays the same
|
||||
- `src/nip011.c` — NIP-11 stays the same
|
||||
- All client-ndk shared files (client.css, init-ndk.mjs, relay-ui.mjs, etc.) — no modifications needed
|
||||
|
||||
### Files to Modify Later (optional relay cleanup)
|
||||
- `src/api.c` — Remove embedded file serving functions
|
||||
- `src/websockets.c` — Remove embedded file dispatch from HTTP callback
|
||||
- `Makefile` — Remove embedded web content compilation step
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|-----------|
|
||||
| NIP-44 encryption via window.nostr.nip44 | Same mechanism as current page — no change in encryption approach |
|
||||
| NDK SharedWorker relay connection | Use ONLY_RELAY cache usage to ensure direct relay communication |
|
||||
| Large page size with all sections inline | Sections are display:none by default — no performance impact |
|
||||
| Event rate chart from text_graph.js | Port chart logic inline or skip initially |
|
||||
| Relay pubkey needed before subscriptions | Fetch NIP-11 first in initialization sequence |
|
||||
@@ -0,0 +1,632 @@
|
||||
# c-relay-pg-pg Plan — PostgreSQL Backend + Multi-Instance + Dashboard
|
||||
|
||||
## Overview
|
||||
|
||||
**c-relay-pg-pg** is a new project (separate repository) forked from c-relay-pg after the `db_ops` abstraction layer is complete. It replaces the SQLite backend with PostgreSQL, enabling horizontal scaling, external dashboards, and production-grade deployments.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- The `db_ops.h` / `db_ops.c` abstraction layer must be complete in c-relay-pg first — see [c-relay-pg thread pool plan](c_relay_pg_thread_pool_plan.md) Phase 1.
|
||||
- The fork happens after Phase 1 of that plan is done and tested.
|
||||
|
||||
### Why a Separate Project?
|
||||
|
||||
| | c-relay-pg | c-relay-pg-pg |
|
||||
|---|---------|-----------|
|
||||
| **Database** | SQLite (embedded) | PostgreSQL (client-server) |
|
||||
| **Deployment** | Single binary + DB file | Binary + PostgreSQL server |
|
||||
| **Scaling** | Single instance | Multiple instances + load balancer |
|
||||
| **Dashboard** | Embedded web UI | Separate web server + Grafana |
|
||||
| **Target user** | Personal relay, small community | Medium-large relay, production |
|
||||
| **Complexity** | Minimal | More infrastructure |
|
||||
|
||||
For the full analysis of why PostgreSQL was chosen over MySQL/MariaDB and other databases, see the [database architecture analysis](database_architecture_analysis.md).
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph c-relay-pg-pg - Production Deployment
|
||||
LB[nginx - TLS + load balancing] -->|WebSocket| R1[c-relay-pg-pg Instance 1]
|
||||
LB -->|WebSocket| R2[c-relay-pg-pg Instance 2]
|
||||
LB -->|HTTP| DASH[Dashboard]
|
||||
R1 -->|db_ops API| PGBACK[PostgreSQL Backend - db_ops_postgres.c]
|
||||
R2 -->|db_ops API| PGBACK
|
||||
PGBACK -->|libpq| PG[PostgreSQL Server]
|
||||
DASH -->|SQL| PG
|
||||
R1 <-->|LISTEN/NOTIFY| R2
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: PostgreSQL Backend
|
||||
|
||||
### Goal
|
||||
|
||||
Fork c-relay-pg into a new repository called **c-relay-pg-pg**. Replace the SQLite `db_ops` implementation with PostgreSQL using `libpq`. The `db_ops.h` interface stays identical — only the backend changes.
|
||||
|
||||
### New Files
|
||||
|
||||
- `src/db_ops_sqlite.c` — Renamed from `db_ops.c` (the Phase 1 SQLite implementation from c-relay-pg)
|
||||
- `src/db_ops_postgres.c` — New PostgreSQL implementation
|
||||
- `src/db_ops.c` — Thin dispatcher that calls the active backend
|
||||
|
||||
### PostgreSQL Schema
|
||||
|
||||
```sql
|
||||
-- PostgreSQL schema for c-relay-pg-pg
|
||||
-- No event_tags table needed — JSONB + GIN handles everything
|
||||
|
||||
CREATE TABLE events (
|
||||
id TEXT PRIMARY KEY,
|
||||
pubkey TEXT NOT NULL,
|
||||
created_at BIGINT NOT NULL,
|
||||
kind INTEGER NOT NULL,
|
||||
event_type TEXT NOT NULL CHECK (event_type IN ('regular', 'replaceable', 'ephemeral', 'addressable')),
|
||||
content TEXT NOT NULL,
|
||||
sig TEXT NOT NULL,
|
||||
tags JSONB NOT NULL DEFAULT '[]',
|
||||
event_json TEXT NOT NULL,
|
||||
first_seen BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
|
||||
);
|
||||
|
||||
-- Core indexes
|
||||
CREATE INDEX idx_events_pubkey ON events(pubkey);
|
||||
CREATE INDEX idx_events_kind ON events(kind);
|
||||
CREATE INDEX idx_events_created_at ON events(created_at DESC);
|
||||
CREATE INDEX idx_events_kind_created_at ON events(kind, created_at DESC);
|
||||
CREATE INDEX idx_events_pubkey_created_at ON events(pubkey, created_at DESC);
|
||||
CREATE INDEX idx_events_pubkey_kind ON events(pubkey, kind);
|
||||
|
||||
-- THE KEY INDEX: GIN on JSONB tags — replaces the entire event_tags table
|
||||
CREATE INDEX idx_events_tags ON events USING GIN (tags);
|
||||
|
||||
-- Partial index: only non-ephemeral events (what REQ queries actually filter)
|
||||
CREATE INDEX idx_events_non_ephemeral ON events(created_at DESC)
|
||||
WHERE kind < 20000 OR kind >= 30000;
|
||||
|
||||
-- Config table
|
||||
CREATE TABLE config (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL,
|
||||
data_type TEXT NOT NULL CHECK (data_type IN ('string', 'integer', 'boolean', 'json')),
|
||||
description TEXT,
|
||||
category TEXT DEFAULT 'general',
|
||||
requires_restart INTEGER DEFAULT 0,
|
||||
created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,
|
||||
updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
|
||||
);
|
||||
|
||||
-- Auth rules table
|
||||
CREATE TABLE auth_rules (
|
||||
id SERIAL PRIMARY KEY,
|
||||
rule_type TEXT NOT NULL CHECK (rule_type IN ('whitelist', 'blacklist', 'rate_limit', 'auth_required', 'wot_whitelist')),
|
||||
pattern_type TEXT NOT NULL CHECK (pattern_type IN ('pubkey', 'kind', 'ip', 'global')),
|
||||
pattern_value TEXT,
|
||||
active INTEGER NOT NULL DEFAULT 1,
|
||||
created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,
|
||||
updated_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
|
||||
);
|
||||
|
||||
CREATE INDEX idx_auth_rules_lookup ON auth_rules(rule_type, pattern_type, pattern_value) WHERE active = 1;
|
||||
|
||||
-- Relay private key storage
|
||||
CREATE TABLE relay_seckey (
|
||||
private_key_hex TEXT NOT NULL CHECK (length(private_key_hex) = 64),
|
||||
created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT
|
||||
);
|
||||
|
||||
-- Subscription logging
|
||||
CREATE TABLE subscriptions (
|
||||
id SERIAL PRIMARY KEY,
|
||||
subscription_id TEXT NOT NULL,
|
||||
wsi_pointer TEXT NOT NULL,
|
||||
client_ip TEXT NOT NULL,
|
||||
event_type TEXT NOT NULL CHECK (event_type IN ('created', 'closed', 'expired', 'disconnected')),
|
||||
filter_json TEXT,
|
||||
events_sent INTEGER DEFAULT 0,
|
||||
created_at BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())::BIGINT,
|
||||
ended_at BIGINT,
|
||||
duration INTEGER,
|
||||
UNIQUE(subscription_id, wsi_pointer)
|
||||
);
|
||||
|
||||
-- IP ban persistence
|
||||
CREATE TABLE ip_bans (
|
||||
ip TEXT PRIMARY KEY,
|
||||
failure_count INTEGER NOT NULL DEFAULT 0,
|
||||
ban_count INTEGER NOT NULL DEFAULT 0,
|
||||
banned_until BIGINT NOT NULL DEFAULT 0,
|
||||
first_failure BIGINT NOT NULL DEFAULT 0,
|
||||
has_authed_successfully INTEGER NOT NULL DEFAULT 0,
|
||||
last_success_at BIGINT NOT NULL DEFAULT 0,
|
||||
total_connections INTEGER NOT NULL DEFAULT 0,
|
||||
total_failures INTEGER NOT NULL DEFAULT 0,
|
||||
total_successes INTEGER NOT NULL DEFAULT 0,
|
||||
first_seen BIGINT NOT NULL DEFAULT 0,
|
||||
idle_failure_count INTEGER NOT NULL DEFAULT 0,
|
||||
idle_ban_count INTEGER NOT NULL DEFAULT 0,
|
||||
idle_banned_until BIGINT NOT NULL DEFAULT 0,
|
||||
idle_first_failure BIGINT NOT NULL DEFAULT 0
|
||||
);
|
||||
|
||||
-- Materialized views for dashboard (refreshed periodically)
|
||||
CREATE MATERIALIZED VIEW event_kinds_mv AS
|
||||
SELECT kind, COUNT(*) as count,
|
||||
ROUND(COUNT(*) * 100.0 / NULLIF((SELECT COUNT(*) FROM events), 0), 2) as percentage
|
||||
FROM events GROUP BY kind;
|
||||
|
||||
CREATE MATERIALIZED VIEW time_stats_mv AS
|
||||
SELECT 'total' as period, COUNT(*) as total_events, COUNT(DISTINCT pubkey) as unique_pubkeys
|
||||
FROM events
|
||||
UNION ALL
|
||||
SELECT '24h', COUNT(*), COUNT(DISTINCT pubkey)
|
||||
FROM events WHERE created_at >= EXTRACT(EPOCH FROM NOW())::BIGINT - 86400
|
||||
UNION ALL
|
||||
SELECT '7d', COUNT(*), COUNT(DISTINCT pubkey)
|
||||
FROM events WHERE created_at >= EXTRACT(EPOCH FROM NOW())::BIGINT - 604800;
|
||||
|
||||
CREATE MATERIALIZED VIEW top_pubkeys_mv AS
|
||||
SELECT pubkey, COUNT(*) as event_count,
|
||||
ROUND(COUNT(*) * 100.0 / NULLIF((SELECT COUNT(*) FROM events), 0), 2) as percentage
|
||||
FROM events GROUP BY pubkey ORDER BY event_count DESC LIMIT 20;
|
||||
|
||||
-- Unique indexes required for CONCURRENTLY refresh
|
||||
CREATE UNIQUE INDEX idx_event_kinds_mv ON event_kinds_mv(kind);
|
||||
CREATE UNIQUE INDEX idx_time_stats_mv ON time_stats_mv(period);
|
||||
CREATE UNIQUE INDEX idx_top_pubkeys_mv ON top_pubkeys_mv(pubkey);
|
||||
```
|
||||
|
||||
### Key Implementation Details
|
||||
|
||||
#### Tag Queries with JSONB
|
||||
|
||||
The biggest win — replacing the `event_tags` subquery with JSONB containment:
|
||||
|
||||
```c
|
||||
// SQLite (current): subquery into event_tags table
|
||||
// AND id IN (SELECT event_id FROM event_tags WHERE tag_name = ? AND tag_value IN (?))
|
||||
|
||||
// PostgreSQL: JSONB containment operator with GIN index
|
||||
// AND tags @> '[["p", "pubkey_hex"]]'
|
||||
|
||||
// For multiple tag values:
|
||||
// AND (tags @> '[["p", "val1"]]' OR tags @> '[["p", "val2"]]')
|
||||
```
|
||||
|
||||
This eliminates the entire `event_tags` table (4 million rows, ~2 GB indexes in the current SQLite schema). The GIN index on JSONB handles everything in ~100–200 MB.
|
||||
|
||||
#### LISTEN/NOTIFY Integration
|
||||
|
||||
For cross-instance event broadcasting:
|
||||
|
||||
```c
|
||||
// In db_ops_postgres.c:
|
||||
int db_notify_new_event(const char* event_id) {
|
||||
char cmd[128];
|
||||
snprintf(cmd, sizeof(cmd), "NOTIFY new_event, '%s'", event_id);
|
||||
PGresult* res = PQexec(g_pg_conn, cmd);
|
||||
PQclear(res);
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Called from the lws event loop every iteration:
|
||||
int db_check_notifications(char* event_id_out, size_t max_len) {
|
||||
PQconsumeInput(g_pg_notify_conn);
|
||||
PGnotify* notify = PQnotifies(g_pg_notify_conn);
|
||||
if (notify) {
|
||||
strncpy(event_id_out, notify->extra, max_len);
|
||||
PQfreemem(notify);
|
||||
return 1; // Got a notification
|
||||
}
|
||||
return 0; // No notifications
|
||||
}
|
||||
```
|
||||
|
||||
#### Connection Management
|
||||
|
||||
```c
|
||||
// db_ops_postgres.c connection pool (simple version)
|
||||
#define DB_POOL_SIZE 4
|
||||
|
||||
static PGconn* g_pg_read_pool[DB_POOL_SIZE]; // For REQ queries
|
||||
static PGconn* g_pg_write_conn; // For EVENT inserts
|
||||
static PGconn* g_pg_notify_conn; // For LISTEN/NOTIFY
|
||||
static pthread_mutex_t g_pool_lock;
|
||||
|
||||
PGconn* db_get_read_connection(void) {
|
||||
pthread_mutex_lock(&g_pool_lock);
|
||||
// Round-robin or find idle connection
|
||||
// ...
|
||||
pthread_mutex_unlock(&g_pool_lock);
|
||||
}
|
||||
```
|
||||
|
||||
### Build System Changes
|
||||
|
||||
```makefile
|
||||
# Makefile additions
|
||||
DB_BACKEND ?= sqlite # Default to sqlite, override with: make DB_BACKEND=postgres
|
||||
|
||||
ifeq ($(DB_BACKEND),postgres)
|
||||
MAIN_SRC += src/db_ops.c src/db_ops_postgres.c
|
||||
LIBS += -lpq
|
||||
CFLAGS += -DDB_BACKEND_POSTGRES
|
||||
else
|
||||
MAIN_SRC += src/db_ops.c src/db_ops_sqlite.c
|
||||
CFLAGS += -DDB_BACKEND_SQLITE
|
||||
endif
|
||||
```
|
||||
|
||||
### Data Migration Tool
|
||||
|
||||
A standalone tool to migrate existing SQLite data to PostgreSQL:
|
||||
|
||||
```bash
|
||||
# migrate_to_postgres.sh
|
||||
# 1. Export events from SQLite
|
||||
sqlite3 old_relay.db ".mode csv" "SELECT id,pubkey,created_at,kind,event_type,content,sig,tags,event_json,first_seen FROM events" > events.csv
|
||||
|
||||
# 2. Import into PostgreSQL
|
||||
psql -d crelay -c "\COPY events FROM 'events.csv' WITH CSV"
|
||||
|
||||
# 3. Migrate config
|
||||
sqlite3 old_relay.db ".mode csv" "SELECT key,value,data_type,description,category,requires_restart FROM config" > config.csv
|
||||
psql -d crelay -c "\COPY config(key,value,data_type,description,category,requires_restart) FROM 'config.csv' WITH CSV"
|
||||
|
||||
# 4. Migrate auth rules
|
||||
sqlite3 old_relay.db ".mode csv" "SELECT rule_type,pattern_type,pattern_value,active FROM auth_rules" > auth_rules.csv
|
||||
psql -d crelay -c "\COPY auth_rules(rule_type,pattern_type,pattern_value,active) FROM 'auth_rules.csv' WITH CSV"
|
||||
|
||||
# 5. Refresh materialized views
|
||||
psql -d crelay -c "REFRESH MATERIALIZED VIEW event_kinds_mv; REFRESH MATERIALIZED VIEW time_stats_mv; REFRESH MATERIALIZED VIEW top_pubkeys_mv;"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Multi-Instance + Dashboard
|
||||
|
||||
### Goal
|
||||
|
||||
Run multiple c-relay-pg-pg instances behind a load balancer with a separate dashboard web server, all sharing the same PostgreSQL database.
|
||||
|
||||
### Components
|
||||
|
||||
1. **nginx config** — WebSocket load balancing with `ip_hash` stickiness
|
||||
2. **systemd template** — `c-relay-pg-pg@.service` for multiple instances
|
||||
3. **LISTEN/NOTIFY integration** — Cross-instance event broadcasting in the `lws_service()` loop
|
||||
4. **PgBouncer** — Connection pooling between c-relay-pg-pg instances and PostgreSQL
|
||||
5. **Dashboard** — Separate web server querying PostgreSQL directly
|
||||
6. **Materialized view refresh** — Background job to refresh analytics views
|
||||
|
||||
### Multi-Instance Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
INTERNET[Internet - Nostr Clients] -->|wss://relay.example.com| NGINX[nginx reverse proxy - TLS termination + load balancing]
|
||||
|
||||
NGINX -->|ws://localhost:8888| R1[c-relay-pg-pg Instance 1 - port 8888]
|
||||
NGINX -->|ws://localhost:8889| R2[c-relay-pg-pg Instance 2 - port 8889]
|
||||
NGINX -->|ws://localhost:8890| R3[c-relay-pg-pg Instance 3 - port 8890]
|
||||
NGINX -->|http://localhost:3000| DASHWEB[Dashboard Web Server]
|
||||
|
||||
R1 -->|libpq connection pool| PGPOOL[PgBouncer - connection pooler]
|
||||
R2 -->|libpq connection pool| PGPOOL
|
||||
R3 -->|libpq connection pool| PGPOOL
|
||||
|
||||
PGPOOL -->|pooled connections| PG[PostgreSQL Primary]
|
||||
DASHWEB -->|read queries| PG
|
||||
|
||||
PG -->|streaming replication| REPLICA[Read Replica - optional]
|
||||
DASHWEB -.->|heavy analytics| REPLICA
|
||||
```
|
||||
|
||||
### nginx Load Balancing Config
|
||||
|
||||
```nginx
|
||||
# nginx.conf - WebSocket load balancing for c-relay-pg-pg
|
||||
upstream relay_backends {
|
||||
# ip_hash ensures a client always hits the same instance
|
||||
# (important for WebSocket session stickiness)
|
||||
ip_hash;
|
||||
|
||||
server 127.0.0.1:8888;
|
||||
server 127.0.0.1:8889;
|
||||
server 127.0.0.1:8890;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name relay.example.com;
|
||||
|
||||
# TLS config...
|
||||
|
||||
location / {
|
||||
proxy_pass http://relay_backends;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_read_timeout 86400; # Keep WebSocket alive for 24h
|
||||
}
|
||||
|
||||
# Dashboard on separate path
|
||||
location /dashboard {
|
||||
proxy_pass http://127.0.0.1:3000;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Session stickiness** (`ip_hash`) ensures that once a client connects to Instance 2, all their subsequent WebSocket frames go to Instance 2. This is important because subscriptions are held in-memory per instance.
|
||||
|
||||
### Starting Multiple Instances
|
||||
|
||||
Each instance is the same binary, just on a different port, all pointing to the same PostgreSQL:
|
||||
|
||||
```bash
|
||||
# Instance 1
|
||||
./build/c_relay_pg_x86 --port 8888 --db-host localhost --db-name crelay &
|
||||
|
||||
# Instance 2
|
||||
./build/c_relay_pg_x86 --port 8889 --db-host localhost --db-name crelay &
|
||||
|
||||
# Instance 3
|
||||
./build/c_relay_pg_x86 --port 8890 --db-host localhost --db-name crelay &
|
||||
```
|
||||
|
||||
Or with systemd template units:
|
||||
|
||||
```ini
|
||||
# /etc/systemd/system/c-relay-pg-pg@.service
|
||||
[Unit]
|
||||
Description=C-Relay-PG-PG Nostr Instance %i
|
||||
After=postgresql.service
|
||||
|
||||
[Service]
|
||||
ExecStart=/opt/c-relay-pg-pg/c_relay_pg_x86 --port %i --db-host localhost --db-name crelay
|
||||
Restart=always
|
||||
User=c-relay-pg
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
```bash
|
||||
systemctl enable c-relay-pg-pg@8888 c-relay-pg-pg@8889 c-relay-pg-pg@8890
|
||||
systemctl start c-relay-pg-pg@8888 c-relay-pg-pg@8889 c-relay-pg-pg@8890
|
||||
```
|
||||
|
||||
### Cross-Instance Event Broadcasting
|
||||
|
||||
When Instance 1 receives a new EVENT and stores it in PostgreSQL, Instance 2 and Instance 3 need to know about it so they can broadcast to their connected subscribers.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Client_A as Client A - connected to Instance 1
|
||||
participant I1 as Instance 1
|
||||
participant PG as PostgreSQL
|
||||
participant I2 as Instance 2
|
||||
participant I3 as Instance 3
|
||||
participant Client_B as Client B - connected to Instance 2
|
||||
participant Client_C as Client C - connected to Instance 3
|
||||
|
||||
Client_A->>I1: EVENT - new kind:1 note
|
||||
I1->>PG: INSERT INTO events...
|
||||
PG-->>I1: OK
|
||||
I1->>I1: broadcast to local subscribers
|
||||
I1->>PG: NOTIFY new_event with event_id
|
||||
|
||||
Note over PG: PostgreSQL delivers notification to all listeners
|
||||
|
||||
PG-->>I2: NOTIFY: new_event event_id
|
||||
PG-->>I3: NOTIFY: new_event event_id
|
||||
|
||||
I2->>PG: SELECT event_json FROM events WHERE id = event_id
|
||||
PG-->>I2: event JSON
|
||||
I2->>I2: match against local subscriptions
|
||||
I2->>Client_B: EVENT message - if subscription matches
|
||||
|
||||
I3->>PG: SELECT event_json FROM events WHERE id = event_id
|
||||
PG-->>I3: event JSON
|
||||
I3->>I3: match against local subscriptions
|
||||
I3->>Client_C: EVENT message - if subscription matches
|
||||
```
|
||||
|
||||
#### PostgreSQL LISTEN/NOTIFY Implementation
|
||||
|
||||
Built into PostgreSQL — no additional infrastructure needed:
|
||||
|
||||
```c
|
||||
// === In the relay's event loop (modified lws_service loop) ===
|
||||
|
||||
// Setup: create a dedicated connection for LISTEN
|
||||
PGconn* notify_conn = PQconnectdb("host=localhost dbname=crelay");
|
||||
PQexec(notify_conn, "LISTEN new_event");
|
||||
int notify_fd = PQsocket(notify_conn); // Get the socket fd for poll()
|
||||
|
||||
// After storing an event:
|
||||
void on_event_stored(PGconn* write_conn, const char* event_id) {
|
||||
char notify_cmd[128];
|
||||
snprintf(notify_cmd, sizeof(notify_cmd),
|
||||
"NOTIFY new_event, '%s'", event_id);
|
||||
PQexec(write_conn, notify_cmd);
|
||||
}
|
||||
|
||||
// In the main event loop (runs every lws_service iteration):
|
||||
void check_cross_instance_events(PGconn* notify_conn) {
|
||||
// Non-blocking check for notifications
|
||||
PQconsumeInput(notify_conn);
|
||||
|
||||
PGnotify* notify;
|
||||
while ((notify = PQnotifies(notify_conn)) != NULL) {
|
||||
// Another instance stored a new event
|
||||
const char* event_id = notify->extra;
|
||||
|
||||
// Fetch the event and check against local subscriptions
|
||||
cJSON* event = db_get_event_by_id(event_id);
|
||||
if (event) {
|
||||
broadcast_event_to_subscriptions(event);
|
||||
cJSON_Delete(event);
|
||||
}
|
||||
|
||||
PQfreemem(notify);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Performance**: LISTEN/NOTIFY adds ~1–5ms latency for cross-instance delivery. For a Nostr relay, this is imperceptible — clients already expect network latency.
|
||||
|
||||
**Payload limit**: NOTIFY payloads are limited to 8000 bytes. For event IDs (64 hex chars), this is fine.
|
||||
|
||||
#### Alternative: Redis Pub/Sub
|
||||
|
||||
If you later need even lower latency or more sophisticated routing:
|
||||
|
||||
```c
|
||||
// Using hiredis (Redis C client)
|
||||
redisContext* redis = redisConnect("127.0.0.1", 6379);
|
||||
|
||||
// After storing event:
|
||||
redisCommand(redis, "PUBLISH new_event %s", event_json);
|
||||
|
||||
// Subscriber (in each instance):
|
||||
redisCommand(redis, "SUBSCRIBE new_event");
|
||||
// Then poll for messages in the event loop
|
||||
```
|
||||
|
||||
Redis adds sub-millisecond pub/sub but requires running a Redis server. For most relays, PostgreSQL LISTEN/NOTIFY is sufficient.
|
||||
|
||||
### Connection Pooling with PgBouncer
|
||||
|
||||
Each relay instance needs multiple database connections. Without pooling, 3 instances × 10 connections = 30 PostgreSQL backend processes. With PgBouncer:
|
||||
|
||||
```ini
|
||||
# /etc/pgbouncer/pgbouncer.ini
|
||||
[databases]
|
||||
crelay = host=127.0.0.1 port=5432 dbname=crelay
|
||||
|
||||
[pgbouncer]
|
||||
listen_port = 6432
|
||||
listen_addr = 127.0.0.1
|
||||
auth_type = md5
|
||||
pool_mode = transaction # Return connection to pool after each transaction
|
||||
max_client_conn = 200 # Total connections from all relay instances
|
||||
default_pool_size = 20 # Actual PostgreSQL connections
|
||||
```
|
||||
|
||||
Relay instances connect to PgBouncer (port 6432) instead of PostgreSQL directly (port 5432). PgBouncer multiplexes 200 client connections onto 20 actual PostgreSQL connections.
|
||||
|
||||
### Zero-Downtime Deployment
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant LB as nginx
|
||||
participant I1 as Instance 1 - v1.0
|
||||
participant I2 as Instance 2 - v1.0
|
||||
participant I3 as Instance 3 - v1.0
|
||||
participant I1_NEW as Instance 1 - v1.1
|
||||
|
||||
Note over LB,I3: Normal operation: 3 instances serving traffic
|
||||
|
||||
LB->>I1: Mark upstream as down
|
||||
Note over I1: Drain: wait for existing connections to close or timeout
|
||||
I1->>I1: Graceful shutdown
|
||||
|
||||
Note over LB: Traffic now goes to I2 and I3 only
|
||||
|
||||
I1_NEW->>I1_NEW: Start with new binary
|
||||
I1_NEW->>LB: Health check passes
|
||||
LB->>I1_NEW: Mark upstream as up
|
||||
|
||||
Note over LB,I1_NEW: Instance 1 now running v1.1, repeat for I2 and I3
|
||||
```
|
||||
|
||||
Rolling deploy script:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# rolling_deploy.sh - Zero-downtime deployment
|
||||
|
||||
for port in 8888 8889 8890; do
|
||||
echo "Deploying instance on port $port..."
|
||||
|
||||
# 1. Tell nginx to stop sending new connections
|
||||
sed -i "s/server 127.0.0.1:$port;/server 127.0.0.1:$port down;/" /etc/nginx/nginx.conf
|
||||
nginx -s reload
|
||||
|
||||
# 2. Wait for existing connections to drain (30 seconds)
|
||||
sleep 30
|
||||
|
||||
# 3. Stop old instance
|
||||
systemctl stop c-relay-pg-pg@$port
|
||||
|
||||
# 4. Deploy new binary
|
||||
cp ./build/c_relay_pg_x86 /opt/c-relay-pg-pg/c_relay_pg_x86
|
||||
|
||||
# 5. Start new instance
|
||||
systemctl start c-relay-pg-pg@$port
|
||||
|
||||
# 6. Wait for health check
|
||||
sleep 5
|
||||
|
||||
# 7. Re-enable in nginx
|
||||
sed -i "s/server 127.0.0.1:$port down;/server 127.0.0.1:$port;/" /etc/nginx/nginx.conf
|
||||
nginx -s reload
|
||||
|
||||
echo "Instance on port $port deployed successfully"
|
||||
done
|
||||
```
|
||||
|
||||
### Dashboard Options
|
||||
|
||||
With PostgreSQL, the dashboard can be built with any technology:
|
||||
|
||||
- **Grafana** — Point directly at PostgreSQL, zero custom code
|
||||
- **Custom web app** — React/Vue frontend + any backend (Node, Python, Go) querying PostgreSQL
|
||||
- **Embedded in c-relay-pg-pg** — Keep current approach but queries go through `db_ops` to PostgreSQL (no longer blocks event loop since PostgreSQL handles concurrency)
|
||||
|
||||
### Scaling Scenarios
|
||||
|
||||
| Scenario | Instances | Connections | Events/hour | Setup |
|
||||
|----------|-----------|-------------|-------------|-------|
|
||||
| **Current** | 1 | ~1,200 | 56 | Single process + SQLite |
|
||||
| **Small upgrade** | 2 | ~2,500 | 500 | 2 instances + PostgreSQL |
|
||||
| **Medium relay** | 4 | ~5,000 | 5,000 | 4 instances + PostgreSQL + PgBouncer |
|
||||
| **Large relay** | 8 | ~10,000 | 50,000 | 8 instances + PostgreSQL + read replica |
|
||||
| **Multi-region** | 2–4 per region | ~50,000 | 500,000 | Multiple servers + PostgreSQL replication |
|
||||
|
||||
### Cost Perspective
|
||||
|
||||
Running multiple relay instances with PostgreSQL on a single VPS:
|
||||
- **PostgreSQL**: ~200–500 MB RAM baseline
|
||||
- **PgBouncer**: ~10 MB RAM
|
||||
- **Each relay instance**: ~50–100 MB RAM
|
||||
- **Dashboard web server**: ~50–100 MB RAM
|
||||
- **4 instances + PostgreSQL + PgBouncer + dashboard**: ~1–2 GB total RAM
|
||||
- A **$20/month VPS** with 4 cores and 4 GB RAM handles this easily
|
||||
- A **$40/month VPS** with 8 cores and 8 GB RAM handles 8 instances comfortably
|
||||
|
||||
---
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|-----------|
|
||||
| PostgreSQL connection failures | `db_ops` returns error codes. Relay logs errors but doesn't crash. Reconnection logic with exponential backoff. |
|
||||
| Performance regression | Benchmark before/after. PostgreSQL should be faster for complex queries, similar for simple ones. |
|
||||
| Data loss during migration | Migration tool is read-only on SQLite. PostgreSQL import is idempotent (INSERT ON CONFLICT). |
|
||||
| SQLite fallback needed | Keep `db_ops_sqlite.c` working. Compile-time flag switches backends. Can always go back. |
|
||||
| LISTEN/NOTIFY message loss | NOTIFY is transactional — if the INSERT commits, the NOTIFY is guaranteed. Missed notifications during reconnect handled by periodic full-sync. |
|
||||
| PgBouncer adds latency | Transaction pooling mode adds <0.1ms. Negligible compared to query time. |
|
||||
|
||||
---
|
||||
|
||||
## Relationship to c-relay-pg
|
||||
|
||||
This project depends on the `db_ops.h` abstraction layer built in [c-relay-pg thread pool plan](c_relay_pg_thread_pool_plan.md) Phase 1. The interface is identical — only the backend implementation changes:
|
||||
|
||||
- **c-relay-pg**: `db_ops.c` → SQLite calls via `sqlite3_*`
|
||||
- **c-relay-pg-pg**: `db_ops_postgres.c` → PostgreSQL calls via `libpq` (`PQexec`, `PQgetvalue`, etc.)
|
||||
|
||||
The `db_ops.h` header file is shared between both projects. Changes to the interface should be coordinated.
|
||||
@@ -0,0 +1,332 @@
|
||||
# c-relay-pg Thread Pool Plan — db_ops Abstraction + SQLite Thread Pool
|
||||
|
||||
## Overview
|
||||
|
||||
This plan covers improvements to **c-relay-pg** (this repo) — the lean, single-binary, embedded-SQLite Nostr relay. Two phases:
|
||||
|
||||
1. **Phase 1: Database Abstraction Layer** — Consolidate all scattered `sqlite3_*` calls into a single `db_ops.h` / `db_ops.c` module. Pure refactor, no behavior change.
|
||||
2. **Phase 2: SQLite Thread Pool** — Add worker threads for concurrent reads, unblocking the `lws_service()` event loop.
|
||||
|
||||
This is the **final form of c-relay-pg**: an embedded-SQLite relay with clean architecture and non-blocking database access. The abstraction layer also enables a future fork to PostgreSQL — see [c-relay-pg-pg plan](c_relay_pg_pg_plan.md).
|
||||
|
||||
### Why This First?
|
||||
|
||||
The current architecture has a fundamental bottleneck documented in the [database architecture analysis](database_architecture_analysis.md): the single-threaded event loop blocks on every database call. A 672ms REQ query freezes every connected client. The thread pool fixes this without changing the database engine or deployment model.
|
||||
|
||||
| Metric | Current | After Thread Pool |
|
||||
|--------|---------|-------------------|
|
||||
| **Event loop blocking** | 0.1–672ms per query | Near-zero |
|
||||
| **Concurrent reads** | 1 | 4–8 |
|
||||
| **Deployment** | Single binary + DB file | Same |
|
||||
| **Database** | SQLite | Same |
|
||||
| **Code risk** | N/A | Low — additive change |
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph c-relay-pg - Final Form
|
||||
RELAY[c-relay-pg binary] -->|db_ops API| DBOPS[db_ops.c - abstraction layer]
|
||||
DBOPS -->|sqlite3 calls| SQLITE[SQLite WAL database]
|
||||
end
|
||||
```
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph c-relay-pg with Thread Pool
|
||||
LWS[lws_service event loop] -->|REQ arrives| Q[Thread-safe job queue]
|
||||
LWS -->|EVENT arrives| WQ[Write queue - single writer]
|
||||
Q --> T1[Reader Thread 1 - own sqlite3*]
|
||||
Q --> T2[Reader Thread 2 - own sqlite3*]
|
||||
Q --> T3[Reader Thread 3 - own sqlite3*]
|
||||
Q --> T4[Reader Thread 4 - own sqlite3*]
|
||||
WQ --> TW[Writer Thread - own sqlite3*]
|
||||
T1 -->|results| CB[Callback to lws event loop]
|
||||
T2 -->|results| CB
|
||||
T3 -->|results| CB
|
||||
T4 -->|results| CB
|
||||
TW -->|OK/error| CB
|
||||
CB -->|queue_message| LWS
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Database Abstraction Layer
|
||||
|
||||
### Goal
|
||||
|
||||
Consolidate all 258 scattered `sqlite3_*` calls across 8 files into a single `db_ops.h` / `db_ops.c` module with a backend-agnostic interface. SQLite continues to work — this is a pure refactor.
|
||||
|
||||
### Files That Currently Touch SQLite Directly
|
||||
|
||||
| File | `sqlite3_*` calls | Operations |
|
||||
|------|-------------------|------------|
|
||||
| [`src/main.c`](../src/main.c) | ~80 | Event store/retrieve, tag storage, REQ queries, COUNT queries, DB init, schema migration |
|
||||
| [`src/config.c`](../src/config.c) | ~60 | Config CRUD, auth rules CRUD, WoT sync, relay key storage, startup sequence |
|
||||
| [`src/api.c`](../src/api.c) | ~40 | Stats queries, monitoring views, admin SQL execution, config management |
|
||||
| [`src/dm_admin.c`](../src/dm_admin.c) | ~20 | Auth rule management, WoT whitelist operations |
|
||||
| [`src/websockets.c`](../src/websockets.c) | ~15 | COUNT queries, IP ban stats, connection tracking |
|
||||
| [`src/subscriptions.c`](../src/subscriptions.c) | ~15 | Subscription logging — create/close/disconnect |
|
||||
| [`src/nip009.c`](../src/nip009.c) | ~10 | Event deletion — by ID and by address |
|
||||
| [`src/request_validator.c`](../src/request_validator.c) | ~8 | Blacklist/whitelist checks |
|
||||
| [`src/ip_ban.c`](../src/ip_ban.c) | ~15 | IP ban table CRUD, persistence |
|
||||
|
||||
### Abstraction Layer Design
|
||||
|
||||
New files: `src/db_ops.h` and `src/db_ops.c`
|
||||
|
||||
```c
|
||||
// src/db_ops.h — Database operations abstraction layer
|
||||
#ifndef DB_OPS_H
|
||||
#define DB_OPS_H
|
||||
|
||||
#include "../nostr_core_lib/cjson/cJSON.h"
|
||||
|
||||
// ============================================================
|
||||
// Lifecycle
|
||||
// ============================================================
|
||||
int db_init(const char* connection_string); // SQLite: file path, PG: connection string
|
||||
void db_close(void);
|
||||
int db_is_available(void);
|
||||
|
||||
// ============================================================
|
||||
// Schema / Migration
|
||||
// ============================================================
|
||||
int db_ensure_schema(void);
|
||||
int db_get_schema_version(void);
|
||||
int db_execute_raw(const char* sql); // For schema DDL only
|
||||
|
||||
// ============================================================
|
||||
// Event Operations
|
||||
// ============================================================
|
||||
int db_store_event(cJSON* event);
|
||||
int db_event_exists(const char* event_id);
|
||||
char* db_get_event_json(const char* event_id); // Caller must free
|
||||
int db_delete_event_by_id(const char* event_id, const char* requester_pubkey);
|
||||
int db_delete_events_by_address(const char* pubkey, int kind,
|
||||
const char* d_tag, long before_timestamp);
|
||||
|
||||
// ============================================================
|
||||
// Event Queries - REQ handling
|
||||
// ============================================================
|
||||
typedef struct {
|
||||
int* kinds; int kind_count;
|
||||
char** authors; int author_count;
|
||||
char** ids; int id_count;
|
||||
char** tag_names; // Parallel arrays: tag_names[i] has tag_values[i][]
|
||||
char*** tag_values;
|
||||
int* tag_value_counts;
|
||||
int tag_filter_count;
|
||||
long since; // 0 = not set
|
||||
long until; // 0 = not set
|
||||
int limit; // 0 = default 500
|
||||
char* search; // NIP-50 search term, NULL = not set
|
||||
} db_event_filter_t;
|
||||
|
||||
typedef struct db_result db_result_t;
|
||||
|
||||
db_result_t* db_query_events(const db_event_filter_t* filter);
|
||||
const char* db_result_next_json(db_result_t* result); // Returns event_json, NULL when done
|
||||
int db_result_row_count(db_result_t* result);
|
||||
void db_result_free(db_result_t* result);
|
||||
|
||||
int db_count_events(const db_event_filter_t* filter);
|
||||
|
||||
// ============================================================
|
||||
// Config Operations
|
||||
// ============================================================
|
||||
char* db_get_config(const char* key); // Caller must free, NULL if not found
|
||||
int db_set_config(const char* key, const char* value, const char* data_type,
|
||||
const char* description, const char* category, int requires_restart);
|
||||
int db_update_config(const char* key, const char* value);
|
||||
int db_config_exists(const char* key);
|
||||
int db_get_config_count(void);
|
||||
|
||||
// ============================================================
|
||||
// Auth Rules Operations
|
||||
// ============================================================
|
||||
int db_add_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
|
||||
int db_remove_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
|
||||
int db_auth_rule_exists(const char* rule_type, const char* pattern_type, const char* pattern_value);
|
||||
int db_clear_auth_rules(const char* rule_type); // NULL = clear all
|
||||
int db_is_blacklisted(const char* pubkey);
|
||||
int db_is_whitelisted(const char* pubkey);
|
||||
int db_has_any_whitelist(void);
|
||||
|
||||
// ============================================================
|
||||
// Relay Key Storage
|
||||
// ============================================================
|
||||
int db_store_relay_private_key(const char* privkey_hex);
|
||||
char* db_get_relay_private_key(void); // Caller must free
|
||||
|
||||
// ============================================================
|
||||
// Subscription Logging
|
||||
// ============================================================
|
||||
int db_log_subscription_created(const char* sub_id, const char* wsi_ptr,
|
||||
const char* client_ip, const char* filter_json);
|
||||
int db_log_subscription_closed(const char* sub_id, const char* client_ip);
|
||||
int db_log_subscription_disconnected(const char* client_ip);
|
||||
int db_update_subscription_events_sent(const char* sub_id, int events_sent);
|
||||
int db_cleanup_orphaned_subscriptions(void);
|
||||
|
||||
// ============================================================
|
||||
// IP Ban Persistence
|
||||
// ============================================================
|
||||
int db_ensure_ip_ban_table(void);
|
||||
int db_load_ip_bans(void* ban_table, int table_size); // Populates in-memory table
|
||||
int db_save_ip_bans(const void* ban_table, int table_size);
|
||||
|
||||
// ============================================================
|
||||
// Analytics / Stats - for dashboard
|
||||
// ============================================================
|
||||
cJSON* db_get_event_kind_distribution(void);
|
||||
cJSON* db_get_time_based_stats(void);
|
||||
cJSON* db_get_top_pubkeys(int limit);
|
||||
cJSON* db_get_subscription_details(void);
|
||||
int db_get_total_event_count(void);
|
||||
|
||||
// ============================================================
|
||||
// Admin SQL Query
|
||||
// ============================================================
|
||||
cJSON* db_execute_admin_query(const char* sql, char* error_msg, size_t error_size);
|
||||
|
||||
#endif // DB_OPS_H
|
||||
```
|
||||
|
||||
### Migration Strategy for Phase 1
|
||||
|
||||
The key principle: **change the interface, not the behavior**. Each function in `db_ops.c` initially just wraps the existing SQLite calls.
|
||||
|
||||
**Step-by-step for each file:**
|
||||
|
||||
1. **Create `db_ops.h` and `db_ops.c`** with the interface above
|
||||
2. **Implement each `db_ops` function** by moving the existing SQLite code from the source files into `db_ops.c`
|
||||
3. **Replace direct `sqlite3_*` calls** in each source file with `db_ops_*` calls
|
||||
4. **Remove `extern sqlite3* g_db`** from each file — only `db_ops.c` knows about `g_db`
|
||||
5. **Remove `#include <sqlite3.h>`** from each file — only `db_ops.c` includes it
|
||||
6. **Update Makefile** to compile `db_ops.c`
|
||||
|
||||
### Order of Migration (by risk, lowest first)
|
||||
|
||||
1. **`src/ip_ban.c`** — Self-contained, simple CRUD. Good warmup.
|
||||
2. **`src/subscriptions.c`** — Logging only, no critical path.
|
||||
3. **`src/nip009.c`** — Event deletion, small file.
|
||||
4. **`src/request_validator.c`** — Auth checks, small file.
|
||||
5. **`src/api.c`** — Stats/monitoring queries. Larger but read-only.
|
||||
6. **`src/dm_admin.c`** — Auth rules + WoT. Medium complexity.
|
||||
7. **`src/config.c`** — Config CRUD. Large but well-structured.
|
||||
8. **`src/websockets.c`** — COUNT queries, minimal DB usage.
|
||||
9. **`src/main.c`** — Event store/retrieve, REQ queries. The big one — do last.
|
||||
|
||||
### Testing Phase 1
|
||||
|
||||
After Phase 1, the relay should behave **identically** to before. Run:
|
||||
- `tests/run_all_tests.sh` — Full test suite
|
||||
- `tests/performance_benchmarks.sh` — Verify no performance regression
|
||||
- Manual testing with production-like traffic
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: SQLite Thread Pool
|
||||
|
||||
### Goal
|
||||
|
||||
Add a pool of N worker threads, each with its own `sqlite3*` connection to the same database file. SQLite WAL mode (already enabled) supports **concurrent readers**. The event loop dispatches database work to the pool and remains responsive.
|
||||
|
||||
### Key Design Points
|
||||
|
||||
1. **Read path**: REQ queries dispatched to thread pool. Each worker opens its own `sqlite3*` connection. SQLite WAL allows unlimited concurrent readers.
|
||||
|
||||
2. **Write path**: EVENT inserts go through a single dedicated writer thread. SQLite only allows one writer at a time anyway — this serializes writes cleanly.
|
||||
|
||||
3. **Result delivery**: Worker threads cannot call `lws_write()` directly (libwebsockets is not thread-safe). Instead, they push results into a per-session message queue and call `lws_cancel_service()` to wake the event loop, which then drains the queue.
|
||||
|
||||
4. **Connection lifecycle**: Each thread opens its own connection with `PRAGMA journal_mode=WAL` and `PRAGMA busy_timeout=5000`.
|
||||
|
||||
### What Changes in the Codebase
|
||||
|
||||
| Component | Current | Thread Pool |
|
||||
|-----------|---------|-------------|
|
||||
| [`g_db`](../src/main.c:49) | Single global connection | One per thread + writer connection |
|
||||
| [`handle_req_message()`](../src/main.c:1101) | Synchronous SQL in callback | Package filter into job, dispatch to pool |
|
||||
| [`store_event()`](../src/main.c:787) | Synchronous INSERT in callback | Dispatch to writer thread |
|
||||
| [`handle_count_message()`](../src/websockets.c:2863) | Synchronous COUNT in callback | Dispatch to pool |
|
||||
| Result delivery | Direct `queue_message()` | Worker pushes to queue + `lws_cancel_service()` |
|
||||
| Config reads | Direct `sqlite3_prepare` on `g_db` | Can stay synchronous with own connection or cache |
|
||||
|
||||
### New Files
|
||||
|
||||
- `src/thread_pool.h` — Thread pool interface
|
||||
- `src/thread_pool.c` — Thread pool implementation (job queue, worker lifecycle, result delivery)
|
||||
|
||||
### Thread Pool Interface (sketch)
|
||||
|
||||
```c
|
||||
// src/thread_pool.h
|
||||
#ifndef THREAD_POOL_H
|
||||
#define THREAD_POOL_H
|
||||
|
||||
typedef enum {
|
||||
JOB_TYPE_REQ_QUERY,
|
||||
JOB_TYPE_COUNT_QUERY,
|
||||
JOB_TYPE_STORE_EVENT,
|
||||
JOB_TYPE_DELETE_EVENT,
|
||||
} job_type_t;
|
||||
|
||||
typedef struct {
|
||||
job_type_t type;
|
||||
void* session; // lws per-session data pointer
|
||||
db_event_filter_t filter; // For REQ/COUNT jobs
|
||||
cJSON* event; // For STORE jobs
|
||||
char event_id[65]; // For DELETE jobs
|
||||
} db_job_t;
|
||||
|
||||
int thread_pool_init(int num_readers, const char* db_path);
|
||||
void thread_pool_shutdown(void);
|
||||
int thread_pool_submit_read(db_job_t* job);
|
||||
int thread_pool_submit_write(db_job_t* job);
|
||||
|
||||
#endif // THREAD_POOL_H
|
||||
```
|
||||
|
||||
### Performance Characteristics
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| **Concurrent reads** | N readers in parallel (N = thread count, typically 4–8) |
|
||||
| **Write throughput** | Same as current — SQLite serializes writes regardless |
|
||||
| **Event loop latency** | Near-zero — REQ no longer blocks the loop |
|
||||
| **Max theoretical read throughput** | ~4–8x current (limited by disk I/O, not CPU) |
|
||||
| **Memory overhead** | ~50–100 MB per connection (page cache) |
|
||||
| **Latency per query** | Same as current per-query, but no head-of-line blocking |
|
||||
|
||||
### Limitations
|
||||
|
||||
- **Write contention**: SQLite still allows only ONE writer at a time. With WAL, readers don't block writers and writers don't block readers, but two simultaneous writes will serialize. At the current write rate (~56 events/hour), this is a non-issue.
|
||||
- **Database size**: The 2.7 GB index bloat problem remains. Thread pool doesn't fix the schema — it fixes the concurrency.
|
||||
- **Scaling ceiling**: Beyond ~8 reader threads, diminishing returns due to disk I/O contention on a single SQLite file.
|
||||
- **Complexity**: Need a proper job queue, thread lifecycle management, and careful handling of the lws ↔ worker thread boundary.
|
||||
|
||||
---
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|-----------|
|
||||
| Phase 1 introduces bugs | Each file migrated independently, tested after each. Full test suite runs after each file. |
|
||||
| Thread pool race conditions | Worker threads only touch their own `sqlite3*` connection. Result delivery uses a mutex-protected queue. `lws_cancel_service()` is documented as thread-safe. |
|
||||
| Performance regression from abstraction | Abstraction layer is thin wrappers — no extra allocations or copies. Benchmark before/after. |
|
||||
| lws thread safety issues | Workers never call `lws_write()` directly. They push to a queue and wake the event loop. This is the documented pattern for libwebsockets multi-threading. |
|
||||
| Rollback needed | Phase 1 is a pure refactor — easy to revert. Phase 2 thread pool is additive — can be disabled with a compile flag. |
|
||||
|
||||
---
|
||||
|
||||
## Relationship to c-relay-pg-pg
|
||||
|
||||
After Phase 1 (abstraction layer) is complete, the codebase is ready to be forked into a separate **c-relay-pg-pg** project that replaces the SQLite backend with PostgreSQL. See [c-relay-pg-pg plan](c_relay_pg_pg_plan.md) for details.
|
||||
|
||||
The `db_ops.h` interface is designed to work with any backend:
|
||||
- **SQLite** (this plan): `db_result_next_json()` copies from `sqlite3_column_text()`
|
||||
- **PostgreSQL** (c-relay-pg-pg): `db_result_next_json()` returns from `PQgetvalue()`
|
||||
- **LMDB** (future possibility): `db_result_next_json()` returns a zero-copy pointer into mmap'd memory
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,186 @@
|
||||
# Plan: Eliminate g_db from the Main Thread
|
||||
|
||||
## Problem Statement
|
||||
|
||||
All SQLite calls go through `db_ops.c`, but `db_active_connection()` falls back to `g_db` when `g_thread_db` is NULL. Since the main thread never sets `g_thread_db`, every `db_*` call from lws-main executes synchronous SQLite on the main thread. The thread pool workers have their own connections and work correctly — the problem is the **fallback path**.
|
||||
|
||||
Current perf data shows lws-main at **67.6% avg CPU**, dominated by SQLite symbols (`sqlite3BtreeTableMoveto`, `sqlite3VdbeExec`, `pcache1Fetch`).
|
||||
|
||||
## Goal
|
||||
|
||||
Remove `g_db` as a runtime connection used by the main thread. After this change:
|
||||
- The main thread has **no SQLite connection** and cannot execute synchronous queries
|
||||
- All DB work routes through thread pool workers (which have their own connections)
|
||||
- `g_db` is only used during startup/shutdown (before/after the event loop)
|
||||
- Any accidental `db_*` call from lws-main returns an error instead of silently blocking
|
||||
|
||||
## Architecture After Change
|
||||
|
||||
```
|
||||
lws-main thread:
|
||||
- WebSocket protocol handling
|
||||
- Message parsing/routing
|
||||
- Completion queue draining
|
||||
- NO SQLite access during event loop
|
||||
|
||||
db-read-0 thread:
|
||||
- REQ queries
|
||||
- COUNT queries
|
||||
- Config cache miss reads
|
||||
- Auth rule checks
|
||||
- Monitoring/stats queries
|
||||
|
||||
db-write thread:
|
||||
- EVENT inserts
|
||||
- Subscription logging
|
||||
- Config updates
|
||||
- Auth rule modifications
|
||||
|
||||
event-worker thread:
|
||||
- Signature validation
|
||||
- Duplicate check
|
||||
- Store via db-write
|
||||
```
|
||||
|
||||
## Categorized g_db Call Sites
|
||||
|
||||
### Category A: Startup-only — keep using g_db
|
||||
|
||||
These run before the event loop starts. They are fine.
|
||||
|
||||
| Function | File | Purpose |
|
||||
|----------|------|---------|
|
||||
| `db_init()` | db_ops.c:27 | Open database |
|
||||
| `db_exec_sql()` for PRAGMAs | main.c:854-882 | WAL, mmap, cache setup |
|
||||
| `db_table_exists()` | main.c:750 | Schema check |
|
||||
| `db_get_schema_version_dup()` | main.c:754 | Migration check |
|
||||
| `db_exec_sql()` for schema | main.c:842 | Create tables |
|
||||
| `populate_all_config_values_atomic()` | config.c:4290 | First-time config |
|
||||
| `populate_default_config_values()` | config.c:1657 | Default config |
|
||||
| `add_pubkeys_to_config_table()` | config.c:1778 | Pubkey storage |
|
||||
| `apply_cli_overrides_atomic()` | config.c:4211 | CLI overrides |
|
||||
| `db_store_relay_private_key_hex()` | config.c:495 | Key storage |
|
||||
| `db_populate_event_tags_from_existing()` | main.c:1156 | Tag migration |
|
||||
| `cleanup_all_subscriptions_on_startup()` | subscriptions.c:1092 | Orphan cleanup |
|
||||
|
||||
### Category B: Shutdown-only — keep using g_db
|
||||
|
||||
| Function | File | Purpose |
|
||||
|----------|------|---------|
|
||||
| `db_exec_sql()` WAL checkpoint | main.c:900 | Clean shutdown |
|
||||
| `db_close()` | main.c:904 | Close connection |
|
||||
|
||||
### Category C: Runtime hot path — must move off main thread
|
||||
|
||||
These are called during the event loop from lws-main context.
|
||||
|
||||
| Function | File | Called from | Fix strategy |
|
||||
|----------|------|------------|--------------|
|
||||
| `db_get_config_value_dup()` | db_ops.c:777 | config cache miss | Already cached with 5s TTL; pre-warm cache at startup so misses are rare |
|
||||
| `store_event()` for kind 14/1059 | main.c:983 | websockets.c sync path | Route through async event worker |
|
||||
| `store_event_post_actions()` | main.c | completion handler | Already runs on main; its DB calls need routing |
|
||||
| `db_log_subscription_created()` | db_ops.c:145 | subscriptions.c | Fire-and-forget via write queue |
|
||||
| `db_log_subscription_closed()` | db_ops.c:165 | subscriptions.c | Fire-and-forget via write queue |
|
||||
| `db_log_subscription_disconnected()` | db_ops.c:193 | subscriptions.c | Fire-and-forget via write queue |
|
||||
| `db_update_subscription_events_sent()` | db_ops.c:225 | subscriptions.c | Fire-and-forget via write queue |
|
||||
| `db_cleanup_orphaned_subscriptions()` | db_ops.c:244 | subscriptions.c | Move to startup only |
|
||||
| `generate_and_post_status_event()` | websockets.c:3408 | periodic timer | Submit to write worker |
|
||||
| `ip_ban_cleanup()` / `ip_ban_log_stats()` | websockets.c:3196 | periodic timer | Submit to write worker |
|
||||
| `db_get_total_event_count_ll()` | db_ops.c:589 | api.c monitoring | Submit to read worker |
|
||||
| `db_get_event_count_since()` | db_ops.c:606 | api.c monitoring | Submit to read worker |
|
||||
| `db_get_event_kind_distribution_rows()` | db_ops.c:625 | api.c monitoring | Submit to read worker |
|
||||
| `db_get_top_pubkeys_rows()` | db_ops.c:663 | api.c monitoring | Submit to read worker |
|
||||
| `db_get_subscription_details_rows()` | db_ops.c:697 | api.c monitoring | Submit to read worker |
|
||||
| `db_get_all_config_rows()` | db_ops.c:745 | api.c config query | Submit to read worker |
|
||||
| `db_execute_readonly_query_json()` | db_ops.c:442 | api.c SQL query | Submit to read worker |
|
||||
| `db_event_id_exists()` | db_ops.c:1041 | main.c event check | Already moved to event worker |
|
||||
| `db_retrieve_event_by_id()` | db_ops.c:1056 | main.c | Submit to read worker |
|
||||
| `db_get_event_pubkey()` | db_ops.c:262 | NIP-09 deletion | Submit to read worker |
|
||||
| `db_delete_event_by_id()` | db_ops.c:285 | NIP-09 deletion | Submit to write worker |
|
||||
| `db_delete_older_replaceable_events()` | db_ops.c:305 | store_event_core | Already on write worker |
|
||||
| `db_store_config_event()` | db_ops.c:888 | config.c | Submit to write worker |
|
||||
| `db_add_auth_rule()` | db_ops.c:1226 | config.c admin | Submit to write worker |
|
||||
| `db_remove_auth_rule()` | db_ops.c:1242 | config.c admin | Submit to write worker |
|
||||
| `db_delete_wot_whitelist_rules()` | db_ops.c:1258 | config.c WoT | Submit to write worker |
|
||||
| `db_count_wot_whitelist_rules()` | db_ops.c:1266 | config.c | Cache or read worker |
|
||||
| `db_store_event_tags_cjson()` | db_ops.c:1140 | main.c post-actions | Already on write worker path |
|
||||
| `db_get_config_row_count()` | db_ops.c:1121 | config.c diagnostics | Cache or read worker |
|
||||
| `db_set_config_value_full()` | db_ops.c:796 | config.c | Submit to write worker |
|
||||
| `db_update_config_value_only()` | db_ops.c:821 | config.c | Submit to write worker |
|
||||
| `db_upsert_config_value()` | db_ops.c:838 | api.c | Submit to write worker |
|
||||
| `db_exec_sql()` for transactions | config.c | admin events | Submit to write worker |
|
||||
| `db_count_with_sql()` | db_ops.c:415 | config.c admin | Submit to read worker |
|
||||
| `is_config_table_ready()` | config.c:4595 | config.c hybrid | Cache result at startup |
|
||||
| `generate_config_event_from_table()` | config.c:4933 | config.c | Cache or read worker |
|
||||
|
||||
### Category D: Auth rule checks — already use separate connections
|
||||
|
||||
These functions in `db_ops.c` open their own temporary read-only connections:
|
||||
|
||||
| Function | Line | Notes |
|
||||
|----------|------|-------|
|
||||
| `db_is_pubkey_blacklisted()` | 347 | Opens own connection |
|
||||
| `db_is_hash_blacklisted()` | 362 | Opens own connection |
|
||||
| `db_is_pubkey_whitelisted()` | 377 | Opens own connection |
|
||||
| `db_count_active_whitelist_rules()` | 392 | Opens own connection |
|
||||
|
||||
These are already safe — they don't use `g_db`. No change needed.
|
||||
|
||||
## Implementation Strategy
|
||||
|
||||
### Phase 1: Pre-warm caches and eliminate cache-miss DB reads
|
||||
|
||||
1. Pre-warm the config cache at startup by loading all config values into the in-memory cache before the event loop starts
|
||||
2. Pre-warm the NIP-11 cache at startup
|
||||
3. Pre-warm the auth rules fast cache at startup
|
||||
4. This eliminates the most frequent cache-miss `db_get_config_value_dup()` calls
|
||||
|
||||
### Phase 2: Route subscription logging through write worker
|
||||
|
||||
1. Add a `THREAD_POOL_JOB_FIRE_AND_FORGET` job type to thread_pool
|
||||
2. Create async wrappers: `db_log_subscription_created_async()`, etc.
|
||||
3. These submit SQL to the write worker queue and return immediately
|
||||
4. No completion callback needed — fire and forget
|
||||
|
||||
### Phase 3: Route special-kind EVENT store through async worker
|
||||
|
||||
1. Remove the `event_is_async_eligible()` exclusion for kinds 14, 1059, 23456
|
||||
2. For kind 23456: process admin command in completion handler on main thread after store
|
||||
3. For kind 14/1059: process NIP-17 DM in completion handler after store
|
||||
|
||||
### Phase 4: Route periodic timer DB work through workers
|
||||
|
||||
1. `generate_and_post_status_event()` — submit event creation to write worker
|
||||
2. `ip_ban_cleanup()` / `ip_ban_log_stats()` — submit to write worker
|
||||
3. Monitoring queries — submit to read worker with completion callback
|
||||
|
||||
### Phase 5: Null out g_db before event loop
|
||||
|
||||
1. After startup is complete and thread pool is initialized, set `g_db = NULL`
|
||||
2. Change `db_active_connection()` to assert/warn if both `g_thread_db` and `g_db` are NULL
|
||||
3. Any accidental main-thread DB call will now fail loudly instead of silently blocking
|
||||
4. Restore `g_db` briefly for shutdown checkpoint
|
||||
|
||||
## Implementation Order
|
||||
|
||||
1. Phase 1 — lowest risk, immediate benefit from eliminating cache misses
|
||||
2. Phase 2 — straightforward fire-and-forget pattern
|
||||
3. Phase 3 — moderate complexity, needs careful completion handler design
|
||||
4. Phase 4 — moderate complexity, periodic tasks need async patterns
|
||||
5. Phase 5 — the final enforcement step, only safe after phases 1-4
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- **Phase 1**: Very low risk — just pre-warming existing caches
|
||||
- **Phase 2**: Low risk — subscription logging is non-critical, fire-and-forget is safe
|
||||
- **Phase 3**: Medium risk — admin/DM event processing has complex state; needs careful testing
|
||||
- **Phase 4**: Medium risk — periodic tasks have side effects that need to complete
|
||||
- **Phase 5**: High risk if done prematurely — must verify ALL runtime paths are covered first
|
||||
|
||||
## Expected Impact
|
||||
|
||||
After all phases:
|
||||
- lws-main CPU should drop from ~67% to near 0% SQLite overhead
|
||||
- All SQLite work happens on db-read and db-write threads
|
||||
- Main thread only does WebSocket I/O, JSON parsing, and completion queue draining
|
||||
- Thread model becomes: lws-main = pure I/O, workers = all DB
|
||||
@@ -0,0 +1,579 @@
|
||||
# Event JSON Storage & Database Migration Plan
|
||||
|
||||
**Goal:** Store full event JSON in database for 2,500x faster retrieval + implement proper database migration system
|
||||
|
||||
---
|
||||
|
||||
## Decision: Fresh Start vs Migration
|
||||
|
||||
### Option A: Fresh Start (Recommended for This Change)
|
||||
|
||||
**Pros:**
|
||||
- ✅ Clean implementation (no migration complexity)
|
||||
- ✅ Fast deployment (no data conversion)
|
||||
- ✅ No risk of migration bugs
|
||||
- ✅ Opportunity to fix any schema issues
|
||||
- ✅ Smaller database (no legacy data)
|
||||
|
||||
**Cons:**
|
||||
- ❌ Lose existing events
|
||||
- ❌ Relay starts "empty"
|
||||
- ❌ Historical data lost
|
||||
|
||||
**Recommendation:** **Fresh start for this change** because:
|
||||
1. Your relay is still in development/testing phase
|
||||
2. The schema change is fundamental (affects every event)
|
||||
3. Migration would require reconstructing JSON for every existing event (expensive)
|
||||
4. You've been doing fresh starts anyway
|
||||
|
||||
### Option B: Implement Migration System
|
||||
|
||||
**Pros:**
|
||||
- ✅ Preserve existing events
|
||||
- ✅ No data loss
|
||||
- ✅ Professional approach
|
||||
- ✅ Reusable for future changes
|
||||
|
||||
**Cons:**
|
||||
- ❌ Complex implementation
|
||||
- ❌ Slow migration (reconstruct JSON for all events)
|
||||
- ❌ Risk of bugs during migration
|
||||
- ❌ Requires careful testing
|
||||
|
||||
**Recommendation:** **Implement migration system for FUTURE changes**, but start fresh for this one.
|
||||
|
||||
---
|
||||
|
||||
## Proposed Schema Change
|
||||
|
||||
### New Schema (v11)
|
||||
|
||||
```sql
|
||||
CREATE TABLE events (
|
||||
id TEXT PRIMARY KEY,
|
||||
pubkey TEXT NOT NULL,
|
||||
created_at INTEGER NOT NULL,
|
||||
kind INTEGER NOT NULL,
|
||||
event_type TEXT NOT NULL CHECK (event_type IN ('regular', 'replaceable', 'ephemeral', 'addressable')),
|
||||
content TEXT NOT NULL,
|
||||
sig TEXT NOT NULL,
|
||||
tags JSON NOT NULL DEFAULT '[]',
|
||||
event_json TEXT NOT NULL, -- NEW: Full event as JSON string
|
||||
first_seen INTEGER NOT NULL DEFAULT (strftime('%s', 'now'))
|
||||
);
|
||||
|
||||
-- Keep all existing indexes (they query the columns, not event_json)
|
||||
CREATE INDEX idx_events_pubkey ON events(pubkey);
|
||||
CREATE INDEX idx_events_kind ON events(kind);
|
||||
CREATE INDEX idx_events_created_at ON events(created_at DESC);
|
||||
CREATE INDEX idx_events_kind_created_at ON events(kind, created_at DESC);
|
||||
CREATE INDEX idx_events_pubkey_created_at ON events(pubkey, created_at DESC);
|
||||
```
|
||||
|
||||
### Why Keep Both Columns AND event_json?
|
||||
|
||||
**Columns (id, pubkey, kind, etc.):**
|
||||
- Used for **querying** (WHERE clauses, indexes)
|
||||
- Fast filtering and sorting
|
||||
- Required for SQL operations
|
||||
|
||||
**event_json:**
|
||||
- Used for **retrieval** (SELECT results)
|
||||
- Pre-serialized, ready to send
|
||||
- Eliminates JSON reconstruction
|
||||
|
||||
**This is a common pattern** in high-performance systems (denormalization for read performance).
|
||||
|
||||
---
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: Schema Update (v11)
|
||||
|
||||
**File:** `src/sql_schema.h`
|
||||
|
||||
```c
|
||||
#define EMBEDDED_SCHEMA_VERSION "11"
|
||||
|
||||
// In schema SQL:
|
||||
"CREATE TABLE events (\n\
|
||||
id TEXT PRIMARY KEY,\n\
|
||||
pubkey TEXT NOT NULL,\n\
|
||||
created_at INTEGER NOT NULL,\n\
|
||||
kind INTEGER NOT NULL,\n\
|
||||
event_type TEXT NOT NULL,\n\
|
||||
content TEXT NOT NULL,\n\
|
||||
sig TEXT NOT NULL,\n\
|
||||
tags JSON NOT NULL DEFAULT '[]',\n\
|
||||
event_json TEXT NOT NULL,\n\ -- NEW COLUMN
|
||||
first_seen INTEGER NOT NULL DEFAULT (strftime('%s', 'now'))\n\
|
||||
);\n\
|
||||
```
|
||||
|
||||
### Phase 2: Update store_event() Function
|
||||
|
||||
**File:** `src/main.c` (lines 660-773)
|
||||
|
||||
**Current:**
|
||||
```c
|
||||
int store_event(cJSON* event) {
|
||||
// Extract fields
|
||||
cJSON* id = cJSON_GetObjectItem(event, "id");
|
||||
// ... extract other fields ...
|
||||
|
||||
// INSERT with individual columns
|
||||
const char* sql = "INSERT INTO events (id, pubkey, ...) VALUES (?, ?, ...)";
|
||||
}
|
||||
```
|
||||
|
||||
**New:**
|
||||
```c
|
||||
int store_event(cJSON* event) {
|
||||
// Serialize event to JSON string ONCE
|
||||
char* event_json = cJSON_PrintUnformatted(event);
|
||||
if (!event_json) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
// Extract fields for indexed columns
|
||||
cJSON* id = cJSON_GetObjectItem(event, "id");
|
||||
// ... extract other fields ...
|
||||
|
||||
// INSERT with columns + event_json
|
||||
const char* sql = "INSERT INTO events (id, pubkey, ..., event_json) VALUES (?, ?, ..., ?)";
|
||||
|
||||
// ... bind parameters ...
|
||||
sqlite3_bind_text(stmt, 9, event_json, -1, SQLITE_TRANSIENT);
|
||||
|
||||
// ... execute ...
|
||||
free(event_json);
|
||||
}
|
||||
```
|
||||
|
||||
### Phase 3: Update handle_req_message() Function
|
||||
|
||||
**File:** `src/main.c` (lines 1302-1361)
|
||||
|
||||
**Current:**
|
||||
```c
|
||||
while (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
// Build event JSON from 7 columns
|
||||
cJSON* event = cJSON_CreateObject();
|
||||
cJSON_AddStringToObject(event, "id", (char*)sqlite3_column_text(stmt, 0));
|
||||
// ... 6 more fields ...
|
||||
cJSON* tags = cJSON_Parse(tags_json); // Parse tags
|
||||
cJSON_AddItemToObject(event, "tags", tags);
|
||||
|
||||
// Create EVENT message
|
||||
cJSON* event_msg = cJSON_CreateArray();
|
||||
cJSON_AddItemToArray(event_msg, cJSON_CreateString("EVENT"));
|
||||
cJSON_AddItemToArray(event_msg, cJSON_CreateString(sub_id));
|
||||
cJSON_AddItemToArray(event_msg, event);
|
||||
|
||||
char* msg_str = cJSON_Print(event_msg);
|
||||
queue_message(wsi, pss, msg_str, msg_len, LWS_WRITE_TEXT);
|
||||
}
|
||||
```
|
||||
|
||||
**New:**
|
||||
```c
|
||||
// Update SQL to select event_json
|
||||
const char* sql = "SELECT event_json FROM events WHERE ...";
|
||||
|
||||
while (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
const char* event_json = (char*)sqlite3_column_text(stmt, 0);
|
||||
|
||||
// Build EVENT message with pre-serialized event
|
||||
// Format: ["EVENT","sub_id",{...event_json...}]
|
||||
size_t msg_len = 12 + strlen(sub_id) + strlen(event_json); // ["EVENT","",""]
|
||||
char* msg_str = malloc(msg_len + 1);
|
||||
snprintf(msg_str, msg_len + 1, "[\"EVENT\",\"%s\",%s]", sub_id, event_json);
|
||||
|
||||
queue_message(wsi, pss, msg_str, strlen(msg_str), LWS_WRITE_TEXT);
|
||||
free(msg_str);
|
||||
}
|
||||
```
|
||||
|
||||
**Speedup:** 366 × (cJSON operations) eliminated!
|
||||
|
||||
---
|
||||
|
||||
## Database Migration System Design
|
||||
|
||||
### For Future Schema Changes
|
||||
|
||||
**File:** `src/migrations.c` (new file)
|
||||
|
||||
```c
|
||||
typedef struct {
|
||||
int from_version;
|
||||
int to_version;
|
||||
const char* description;
|
||||
int (*migrate_func)(sqlite3* db);
|
||||
} migration_t;
|
||||
|
||||
// Migration from v10 to v11: Add event_json column
|
||||
int migrate_v10_to_v11(sqlite3* db) {
|
||||
// Step 1: Add column
|
||||
const char* add_column_sql =
|
||||
"ALTER TABLE events ADD COLUMN event_json TEXT";
|
||||
|
||||
if (sqlite3_exec(db, add_column_sql, NULL, NULL, NULL) != SQLITE_OK) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
// Step 2: Populate event_json for existing events
|
||||
const char* select_sql =
|
||||
"SELECT id, pubkey, created_at, kind, content, sig, tags FROM events";
|
||||
|
||||
sqlite3_stmt* stmt;
|
||||
if (sqlite3_prepare_v2(db, select_sql, -1, &stmt, NULL) != SQLITE_OK) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
while (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
// Reconstruct JSON
|
||||
cJSON* event = cJSON_CreateObject();
|
||||
cJSON_AddStringToObject(event, "id", (char*)sqlite3_column_text(stmt, 0));
|
||||
// ... add other fields ...
|
||||
|
||||
char* event_json = cJSON_PrintUnformatted(event);
|
||||
|
||||
// Update row
|
||||
const char* update_sql = "UPDATE events SET event_json = ? WHERE id = ?";
|
||||
sqlite3_stmt* update_stmt;
|
||||
sqlite3_prepare_v2(db, update_sql, -1, &update_stmt, NULL);
|
||||
sqlite3_bind_text(update_stmt, 1, event_json, -1, SQLITE_TRANSIENT);
|
||||
sqlite3_bind_text(update_stmt, 2, (char*)sqlite3_column_text(stmt, 0), -1, SQLITE_STATIC);
|
||||
sqlite3_step(update_stmt);
|
||||
sqlite3_finalize(update_stmt);
|
||||
|
||||
free(event_json);
|
||||
cJSON_Delete(event);
|
||||
}
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
|
||||
// Step 3: Make column NOT NULL
|
||||
// (SQLite doesn't support ALTER COLUMN, so we'd need to recreate table)
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Migration registry
|
||||
static migration_t migrations[] = {
|
||||
{10, 11, "Add event_json column for fast retrieval", migrate_v10_to_v11},
|
||||
// Future migrations go here
|
||||
};
|
||||
|
||||
int run_migrations(sqlite3* db, int current_version, int target_version) {
|
||||
for (int i = 0; i < sizeof(migrations) / sizeof(migration_t); i++) {
|
||||
if (migrations[i].from_version >= current_version &&
|
||||
migrations[i].to_version <= target_version) {
|
||||
|
||||
printf("Running migration: %s\n", migrations[i].description);
|
||||
|
||||
if (migrations[i].migrate_func(db) != 0) {
|
||||
fprintf(stderr, "Migration failed: %s\n", migrations[i].description);
|
||||
return -1;
|
||||
}
|
||||
|
||||
// Update schema version
|
||||
char update_version_sql[256];
|
||||
snprintf(update_version_sql, sizeof(update_version_sql),
|
||||
"PRAGMA user_version = %d", migrations[i].to_version);
|
||||
sqlite3_exec(db, update_version_sql, NULL, NULL, NULL);
|
||||
}
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Recommendation: Hybrid Approach
|
||||
|
||||
### For This Change (v10 → v11): Fresh Start
|
||||
|
||||
**Rationale:**
|
||||
1. Your relay is still in development
|
||||
2. Migration would be slow (reconstruct JSON for all events)
|
||||
3. You've been doing fresh starts anyway
|
||||
4. Clean slate for performance testing
|
||||
|
||||
**Steps:**
|
||||
1. Update schema to v11 with event_json column
|
||||
2. Update store_event() to populate event_json
|
||||
3. Update handle_req_message() to use event_json
|
||||
4. Deploy with fresh database
|
||||
5. Test performance improvement
|
||||
|
||||
### For Future Changes: Use Migration System
|
||||
|
||||
**Rationale:**
|
||||
1. Once relay is in production, data preservation matters
|
||||
2. Migration system is reusable
|
||||
3. Professional approach for production relay
|
||||
|
||||
**Steps:**
|
||||
1. Create `src/migrations.c` and `src/migrations.h`
|
||||
2. Implement migration framework
|
||||
3. Add migration functions for each schema change
|
||||
4. Test migrations thoroughly before deployment
|
||||
|
||||
---
|
||||
|
||||
## Migration System Features
|
||||
|
||||
### Core Features
|
||||
|
||||
1. **Version Detection**
|
||||
- Read current schema version from database
|
||||
- Compare with embedded schema version
|
||||
- Determine which migrations to run
|
||||
|
||||
2. **Migration Chain**
|
||||
- Run migrations in sequence (v8 → v9 → v10 → v11)
|
||||
- Skip already-applied migrations
|
||||
- Stop on first failure
|
||||
|
||||
3. **Backup Before Migration**
|
||||
- Automatic database backup before migration
|
||||
- Rollback capability if migration fails
|
||||
- Backup retention policy
|
||||
|
||||
4. **Progress Reporting**
|
||||
- Log migration progress
|
||||
- Show estimated time remaining
|
||||
- Report success/failure
|
||||
|
||||
### Safety Features
|
||||
|
||||
1. **Transaction Wrapping**
|
||||
```c
|
||||
sqlite3_exec(db, "BEGIN TRANSACTION", NULL, NULL, NULL);
|
||||
int result = migrate_v10_to_v11(db);
|
||||
if (result == 0) {
|
||||
sqlite3_exec(db, "COMMIT", NULL, NULL, NULL);
|
||||
} else {
|
||||
sqlite3_exec(db, "ROLLBACK", NULL, NULL, NULL);
|
||||
}
|
||||
```
|
||||
|
||||
2. **Validation After Migration**
|
||||
- Verify row counts match
|
||||
- Check data integrity
|
||||
- Validate indexes created
|
||||
|
||||
3. **Dry-Run Mode**
|
||||
- Test migration without committing
|
||||
- Report what would be changed
|
||||
- Estimate migration time
|
||||
|
||||
---
|
||||
|
||||
## Implementation Timeline
|
||||
|
||||
### Immediate (Today): Fresh Start with event_json
|
||||
|
||||
**Changes:**
|
||||
1. Update schema to v11 (add event_json column)
|
||||
2. Update store_event() to populate event_json
|
||||
3. Update handle_req_message() to use event_json
|
||||
4. Deploy with fresh database
|
||||
|
||||
**Effort:** 4 hours
|
||||
**Impact:** 2,500x faster event retrieval
|
||||
|
||||
### This Week: Build Migration Framework
|
||||
|
||||
**Changes:**
|
||||
1. Create src/migrations.c and src/migrations.h
|
||||
2. Implement migration runner
|
||||
3. Add backup/rollback capability
|
||||
4. Add progress reporting
|
||||
|
||||
**Effort:** 1-2 days
|
||||
**Impact:** Reusable for all future schema changes
|
||||
|
||||
### Future: Add Migrations as Needed
|
||||
|
||||
**For each schema change:**
|
||||
1. Write migration function
|
||||
2. Add to migrations array
|
||||
3. Test thoroughly
|
||||
4. Deploy with automatic migration
|
||||
|
||||
---
|
||||
|
||||
## Code Structure
|
||||
|
||||
### File Organization
|
||||
|
||||
```
|
||||
src/
|
||||
├── migrations.c # NEW: Migration system
|
||||
├── migrations.h # NEW: Migration API
|
||||
├── sql_schema.h # Schema definition (v11)
|
||||
├── main.c # Updated store_event() and handle_req_message()
|
||||
└── ...
|
||||
```
|
||||
|
||||
### Migration API
|
||||
|
||||
```c
|
||||
// migrations.h
|
||||
int init_migration_system(sqlite3* db);
|
||||
int run_pending_migrations(sqlite3* db);
|
||||
int backup_database(const char* db_path, char* backup_path, size_t backup_path_size);
|
||||
int rollback_migration(sqlite3* db, const char* backup_path);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### For Fresh Start (v11)
|
||||
|
||||
1. **Local testing:**
|
||||
- Build with new schema
|
||||
- Post test events
|
||||
- Query events and measure performance
|
||||
- Verify event_json is populated correctly
|
||||
|
||||
2. **Performance testing:**
|
||||
- Query 366 events
|
||||
- Measure time (should be <10ms instead of 18s)
|
||||
- Check CPU usage (should be <20%)
|
||||
|
||||
3. **Production deployment:**
|
||||
- Stop relay
|
||||
- Delete old database
|
||||
- Start relay with v11 schema
|
||||
- Monitor performance
|
||||
|
||||
### For Migration System (Future)
|
||||
|
||||
1. **Unit tests:**
|
||||
- Test each migration function
|
||||
- Test rollback capability
|
||||
- Test error handling
|
||||
|
||||
2. **Integration tests:**
|
||||
- Create database with old schema
|
||||
- Run migration
|
||||
- Verify data integrity
|
||||
- Test rollback
|
||||
|
||||
3. **Performance tests:**
|
||||
- Measure migration time for large databases
|
||||
- Test with 10K, 100K, 1M events
|
||||
- Optimize slow migrations
|
||||
|
||||
---
|
||||
|
||||
## Migration Complexity Analysis
|
||||
|
||||
### For v10 → v11 Migration
|
||||
|
||||
**If we were to migrate existing data:**
|
||||
|
||||
```sql
|
||||
-- Step 1: Add column (fast)
|
||||
ALTER TABLE events ADD COLUMN event_json TEXT;
|
||||
|
||||
-- Step 2: Populate event_json (SLOW!)
|
||||
-- For each of N events:
|
||||
-- 1. SELECT 7 columns
|
||||
-- 2. Reconstruct JSON (cJSON operations)
|
||||
-- 3. Serialize to string (cJSON_Print)
|
||||
-- 4. UPDATE event_json column
|
||||
-- 5. Free memory
|
||||
|
||||
-- Estimated time:
|
||||
-- - 1000 events: ~10 seconds
|
||||
-- - 10000 events: ~100 seconds
|
||||
-- - 100000 events: ~1000 seconds (16 minutes)
|
||||
```
|
||||
|
||||
**Conclusion:** Migration is expensive for this change. Fresh start is better.
|
||||
|
||||
---
|
||||
|
||||
## Future Migration Examples
|
||||
|
||||
### Easy Migrations (Fast)
|
||||
|
||||
**Adding an index:**
|
||||
```c
|
||||
int migrate_add_index(sqlite3* db) {
|
||||
return sqlite3_exec(db,
|
||||
"CREATE INDEX idx_new ON events(new_column)",
|
||||
NULL, NULL, NULL);
|
||||
}
|
||||
```
|
||||
|
||||
**Adding a column with default:**
|
||||
```c
|
||||
int migrate_add_column(sqlite3* db) {
|
||||
return sqlite3_exec(db,
|
||||
"ALTER TABLE events ADD COLUMN new_col TEXT DEFAULT ''",
|
||||
NULL, NULL, NULL);
|
||||
}
|
||||
```
|
||||
|
||||
### Hard Migrations (Slow)
|
||||
|
||||
**Changing column type:**
|
||||
- Requires table recreation
|
||||
- Copy all data
|
||||
- Recreate indexes
|
||||
- Can take minutes for large databases
|
||||
|
||||
**Populating computed columns:**
|
||||
- Requires row-by-row processing
|
||||
- Can take minutes for large databases
|
||||
|
||||
---
|
||||
|
||||
## Recommendation Summary
|
||||
|
||||
### For This Change (event_json)
|
||||
|
||||
**Do:** Fresh start with v11 schema
|
||||
- Fast deployment
|
||||
- Clean implementation
|
||||
- Immediate performance benefit
|
||||
- No migration complexity
|
||||
|
||||
**Don't:** Migrate existing data
|
||||
- Too slow (reconstruct JSON for all events)
|
||||
- Too complex (first migration)
|
||||
- Not worth it (relay still in development)
|
||||
|
||||
### For Future Changes
|
||||
|
||||
**Do:** Implement migration system
|
||||
- Professional approach
|
||||
- Data preservation
|
||||
- Reusable framework
|
||||
- Required for production relay
|
||||
|
||||
**Timeline:**
|
||||
- **Today:** Deploy v11 with fresh start
|
||||
- **This week:** Build migration framework
|
||||
- **Future:** Use migrations for all schema changes
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. ✅ Update schema to v11 (add event_json column)
|
||||
2. ✅ Update store_event() to populate event_json
|
||||
3. ✅ Update handle_req_message() to use event_json
|
||||
4. ✅ Test locally with 366-event query
|
||||
5. ✅ Deploy to production with fresh database
|
||||
6. ✅ Measure performance improvement
|
||||
7. ⏳ Build migration system for future use
|
||||
|
||||
**Expected result:** 366-event retrieval time drops from 18s to <10ms (2,500x speedup)
|
||||
@@ -0,0 +1,312 @@
|
||||
# Event Tags Denormalization Plan
|
||||
|
||||
## Problem Statement
|
||||
|
||||
The relay is at 99% CPU with 1,178 WebSocket connections and ~120 new REQ subscriptions per minute. The root cause is that every tag-filtered REQ query (e.g., `#g`, `#e`, `#p`) uses `json_each(json(tags))` with `json_extract()` — a correlated subquery that parses the JSON tags column for every candidate row. With geohash-based subscriptions from a location app generating constant connection churn, this creates unsustainable CPU load.
|
||||
|
||||
### Evidence
|
||||
|
||||
- `EXPLAIN QUERY PLAN` shows: `CORRELATED SCALAR SUBQUERY → SCAN json_each VIRTUAL TABLE`
|
||||
- 8,077 subscription creates/hour but only 442 active at any time (connection churn)
|
||||
- Only 56 events stored/hour — the load is entirely from **read queries**, not writes
|
||||
- 14,819 events match the kind filter; each query parses JSON tags on candidate rows
|
||||
|
||||
## Solution: Denormalized `event_tags` Table
|
||||
|
||||
Replace `json_each()` queries with indexed lookups on a separate `event_tags` table. This is the standard approach used by strfry, nostream, and other production Nostr relays.
|
||||
|
||||
### Performance Impact
|
||||
|
||||
| Metric | Before | After |
|
||||
|--------|--------|-------|
|
||||
| Tag query type | `json_each()` correlated subquery | Indexed JOIN/subquery |
|
||||
| Complexity per query | O(rows × tags_per_event) JSON parsing | O(log n) B-tree lookup |
|
||||
| Expected CPU reduction | 99% | <10% for same traffic |
|
||||
|
||||
## Files to Modify
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `src/sql_schema.h` | Add `event_tags` table, indexes, cascade triggers; bump to schema v12 |
|
||||
| `src/main.c` | Add `store_event_tags()`, update `handle_req_message()` tag filter SQL, move config reads out of row loop, add startup tag population |
|
||||
| `src/nip009.c` | Add `DELETE FROM event_tags` alongside event deletions |
|
||||
| `src/dm_admin.c` | Downgrade NIP-17 decryption failure log level |
|
||||
| `src/websockets.c` | Downgrade NIP-17 error log to DEBUG at lines 780 and 1488 |
|
||||
|
||||
## Detailed Changes
|
||||
|
||||
### 1. Schema: `src/sql_schema.h`
|
||||
|
||||
Add the `event_tags` table after the events table definition. Bump schema version to 12.
|
||||
|
||||
```sql
|
||||
-- Denormalized event tags for fast indexed lookups
|
||||
-- Replaces json_each(json(tags)) queries which cause full JSON parsing per row
|
||||
CREATE TABLE event_tags (
|
||||
event_id TEXT NOT NULL,
|
||||
tag_name TEXT NOT NULL,
|
||||
tag_value TEXT NOT NULL,
|
||||
tag_index INTEGER NOT NULL DEFAULT 0,
|
||||
FOREIGN KEY (event_id) REFERENCES events(id) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
-- Primary lookup index: find events by tag name + value
|
||||
CREATE INDEX idx_event_tags_lookup ON event_tags(tag_name, tag_value);
|
||||
|
||||
-- Reverse lookup: find all tags for an event (for cleanup)
|
||||
CREATE INDEX idx_event_tags_event ON event_tags(event_id);
|
||||
|
||||
-- Composite index for common query pattern: tag + kind (via join)
|
||||
CREATE INDEX idx_event_tags_value_name ON event_tags(tag_value, tag_name);
|
||||
```
|
||||
|
||||
**Key design decisions:**
|
||||
- `ON DELETE CASCADE` handles cleanup when events are deleted (NIP-09, replaceable events)
|
||||
- `tag_index` stores the position within the tags array (useful for ordered tag access)
|
||||
- Only the first two elements of each tag are indexed (`tag_name` = `$[0]`, `tag_value` = `$[1]`) — this covers all standard Nostr tag filters (`#e`, `#p`, `#t`, `#g`, `#d`, etc.)
|
||||
- `PRAGMA foreign_keys = ON` is already in the schema, so CASCADE will work
|
||||
|
||||
### 2. Store Tags: `src/main.c` — New `store_event_tags()` Function
|
||||
|
||||
Add a new function called after successful event INSERT in `store_event()`:
|
||||
|
||||
```c
|
||||
// Insert denormalized tags into event_tags table for fast indexed lookups
|
||||
int store_event_tags(const char* event_id, cJSON* tags) {
|
||||
if (!g_db || !event_id || !tags || !cJSON_IsArray(tags)) {
|
||||
return 0; // Not an error if no tags
|
||||
}
|
||||
|
||||
const char* sql = "INSERT INTO event_tags (event_id, tag_name, tag_value, tag_index) VALUES (?, ?, ?, ?)";
|
||||
sqlite3_stmt* stmt;
|
||||
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
|
||||
if (rc != SQLITE_OK) {
|
||||
DEBUG_ERROR("Failed to prepare event_tags insert: %s", sqlite3_errmsg(g_db));
|
||||
return -1;
|
||||
}
|
||||
|
||||
int tag_index = 0;
|
||||
cJSON* tag = NULL;
|
||||
cJSON_ArrayForEach(tag, tags) {
|
||||
if (cJSON_IsArray(tag) && cJSON_GetArraySize(tag) >= 2) {
|
||||
cJSON* name = cJSON_GetArrayItem(tag, 0);
|
||||
cJSON* value = cJSON_GetArrayItem(tag, 1);
|
||||
|
||||
if (cJSON_IsString(name) && cJSON_IsString(value)) {
|
||||
sqlite3_reset(stmt);
|
||||
sqlite3_bind_text(stmt, 1, event_id, -1, SQLITE_STATIC);
|
||||
sqlite3_bind_text(stmt, 2, cJSON_GetStringValue(name), -1, SQLITE_STATIC);
|
||||
sqlite3_bind_text(stmt, 3, cJSON_GetStringValue(value), -1, SQLITE_STATIC);
|
||||
sqlite3_bind_int(stmt, 4, tag_index);
|
||||
|
||||
rc = sqlite3_step(stmt);
|
||||
if (rc != SQLITE_DONE) {
|
||||
DEBUG_ERROR("Failed to insert event tag: %s", sqlite3_errmsg(g_db));
|
||||
}
|
||||
}
|
||||
}
|
||||
tag_index++;
|
||||
}
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
**Call site** — in `store_event()` at line ~784, after `monitoring_on_event_stored()`:
|
||||
|
||||
```c
|
||||
// After successful event storage, insert denormalized tags
|
||||
store_event_tags(cJSON_GetStringValue(id), tags);
|
||||
```
|
||||
|
||||
### 3. Update REQ Query Builder: `src/main.c` — `handle_req_message()`
|
||||
|
||||
Replace the `json_each()` tag filter at lines 1243-1270 with an `event_tags` subquery:
|
||||
|
||||
**Before** (current code at line 1244):
|
||||
```c
|
||||
snprintf(sql_ptr, remaining,
|
||||
" AND EXISTS (SELECT 1 FROM json_each(json(tags)) "
|
||||
"WHERE json_extract(value, '$[0]') = ? "
|
||||
"AND json_extract(value, '$[1]') IN (");
|
||||
```
|
||||
|
||||
**After** (new code):
|
||||
```c
|
||||
snprintf(sql_ptr, remaining,
|
||||
" AND id IN (SELECT event_id FROM event_tags "
|
||||
"WHERE tag_name = ? AND tag_value IN (");
|
||||
```
|
||||
|
||||
The rest of the parameter binding code (lines 1248-1270) stays the same — the bind params are identical (`tag_name` then `tag_value(s)`). Only the SQL template changes.
|
||||
|
||||
The closing parenthesis changes from `"))` to `"))` — same syntax, just different semantics.
|
||||
|
||||
### 4. NIP-09 Delete Cascade: `src/nip009.c`
|
||||
|
||||
The `ON DELETE CASCADE` foreign key handles this automatically. When `DELETE FROM events WHERE id = ?` executes, SQLite will automatically delete matching rows from `event_tags`. **No code changes needed in nip009.c** as long as `PRAGMA foreign_keys = ON` is set (it already is in the schema).
|
||||
|
||||
However, verify the replaceable/addressable event triggers in the schema also cascade properly. The existing triggers at schema lines 97-119 use `DELETE FROM events WHERE ...` which will trigger the CASCADE.
|
||||
|
||||
### 5. Performance Fix: Move Config Reads Out of Row Loop
|
||||
|
||||
In `handle_req_message()` at lines 1400-1401, `get_config_bool()` is called **inside the `while (sqlite3_step())` loop** — meaning it executes a SQLite query for every row returned. Move these before the loop:
|
||||
|
||||
**Before** (inside loop):
|
||||
```c
|
||||
while (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
// ...
|
||||
int expiration_enabled = get_config_bool("expiration_enabled", 1); // SQLite query per row!
|
||||
int filter_responses = get_config_bool("expiration_filter", 1); // SQLite query per row!
|
||||
```
|
||||
|
||||
**After** (before loop):
|
||||
```c
|
||||
int expiration_enabled = get_config_bool("expiration_enabled", 1);
|
||||
int filter_responses = get_config_bool("expiration_filter", 1);
|
||||
|
||||
while (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
// ... use cached values
|
||||
```
|
||||
|
||||
### 6. Log Level Fixes
|
||||
|
||||
#### NIP-17 Decryption Failure — `src/websockets.c` lines 780 and 1488
|
||||
|
||||
Change from `DEBUG_ERROR` to `DEBUG_INFO`:
|
||||
```c
|
||||
// Before:
|
||||
DEBUG_ERROR("NIP-17 admin message processing failed");
|
||||
// After:
|
||||
DEBUG_INFO("NIP-17 admin message processing failed");
|
||||
```
|
||||
|
||||
This is expected behavior when non-admin gift wraps arrive — not an error condition.
|
||||
|
||||
#### Duplicate Event INSERT — `src/main.c` line 741
|
||||
|
||||
The `DEBUG_ERROR` at line 741 fires before the CONSTRAINT check at line 746. Suppress it for constraint violations:
|
||||
|
||||
```c
|
||||
// Before (line 738-741):
|
||||
if (rc != SQLITE_DONE) {
|
||||
const char* err_msg = sqlite3_errmsg(g_db);
|
||||
int extended_errcode = sqlite3_extended_errcode(g_db);
|
||||
DEBUG_ERROR("INSERT failed: rc=%d, extended_errcode=%d, msg=%s", rc, extended_errcode, err_msg);
|
||||
}
|
||||
|
||||
// After:
|
||||
if (rc != SQLITE_DONE) {
|
||||
const char* err_msg = sqlite3_errmsg(g_db);
|
||||
int extended_errcode = sqlite3_extended_errcode(g_db);
|
||||
if (rc != SQLITE_CONSTRAINT) {
|
||||
DEBUG_ERROR("INSERT failed: rc=%d, extended_errcode=%d, msg=%s", rc, extended_errcode, err_msg);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 7. Startup Tag Population
|
||||
|
||||
Since you said no migration code is needed (fresh deploy), the `event_tags` table will start empty and populate as new events arrive. Existing events won't have tags in the lookup table.
|
||||
|
||||
However, to avoid a period where old events don't appear in tag-filtered queries, add a one-time population function that runs at startup:
|
||||
|
||||
```c
|
||||
// Populate event_tags from existing events (run once at startup)
|
||||
int populate_event_tags_from_existing(void) {
|
||||
if (!g_db) return -1;
|
||||
|
||||
// Check if event_tags is already populated
|
||||
sqlite3_stmt* check_stmt;
|
||||
sqlite3_prepare_v2(g_db, "SELECT COUNT(*) FROM event_tags", -1, &check_stmt, NULL);
|
||||
if (sqlite3_step(check_stmt) == SQLITE_ROW && sqlite3_column_int(check_stmt, 0) > 0) {
|
||||
sqlite3_finalize(check_stmt);
|
||||
DEBUG_INFO("event_tags already populated, skipping");
|
||||
return 0;
|
||||
}
|
||||
sqlite3_finalize(check_stmt);
|
||||
|
||||
DEBUG_INFO("Populating event_tags from existing events...");
|
||||
|
||||
const char* sql = "SELECT id, tags FROM events WHERE tags != '[]'";
|
||||
sqlite3_stmt* stmt;
|
||||
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
|
||||
if (rc != SQLITE_OK) return -1;
|
||||
|
||||
// Use a transaction for bulk insert performance
|
||||
sqlite3_exec(g_db, "BEGIN TRANSACTION", NULL, NULL, NULL);
|
||||
|
||||
int event_count = 0;
|
||||
while (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
const char* event_id = (const char*)sqlite3_column_text(stmt, 0);
|
||||
const char* tags_json = (const char*)sqlite3_column_text(stmt, 1);
|
||||
|
||||
if (event_id && tags_json) {
|
||||
cJSON* tags = cJSON_Parse(tags_json);
|
||||
if (tags) {
|
||||
store_event_tags(event_id, tags);
|
||||
cJSON_Delete(tags);
|
||||
event_count++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
sqlite3_exec(g_db, "COMMIT", NULL, NULL, NULL);
|
||||
|
||||
DEBUG_INFO("Populated event_tags for %d events", event_count);
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
Call this during startup, after database initialization but before accepting connections.
|
||||
|
||||
## Architecture Diagram
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph "Current: O(n) per query"
|
||||
A1["REQ with #g filter"] --> B1["SQL: SELECT ... WHERE EXISTS"]
|
||||
B1 --> C1["json_each(json(tags))"]
|
||||
C1 --> D1["json_extract per row"]
|
||||
D1 --> E1["🔴 Full JSON parse per candidate row"]
|
||||
end
|
||||
|
||||
subgraph "New: O(log n) per query"
|
||||
A2["REQ with #g filter"] --> B2["SQL: SELECT ... WHERE id IN"]
|
||||
B2 --> C2["event_tags table"]
|
||||
C2 --> D2["idx_event_tags_lookup"]
|
||||
D2 --> E2["🟢 B-tree index lookup"]
|
||||
end
|
||||
|
||||
subgraph "Write Path (unchanged speed)"
|
||||
F["EVENT arrives"] --> G["store_event()"]
|
||||
G --> H["INSERT INTO events"]
|
||||
H --> I["store_event_tags()"]
|
||||
I --> J["INSERT INTO event_tags\n(one row per tag)"]
|
||||
end
|
||||
|
||||
subgraph "Delete Path (automatic)"
|
||||
K["NIP-09 DELETE"] --> L["DELETE FROM events"]
|
||||
L --> M["ON DELETE CASCADE"]
|
||||
M --> N["event_tags rows auto-deleted"]
|
||||
end
|
||||
```
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
1. Run existing test suite: `tests/run_all_tests.sh`
|
||||
2. Specifically run NIP tests that use tag filters: `tests/run_nip_tests.sh`
|
||||
3. Verify tag-filtered REQ queries return correct results
|
||||
4. Verify NIP-09 deletion cascades to event_tags
|
||||
5. Verify replaceable event triggers cascade to event_tags
|
||||
6. Monitor CPU usage after deployment with same traffic pattern
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- **Low risk**: The `event_tags` table is additive — it doesn't change the events table structure
|
||||
- **Low risk**: `ON DELETE CASCADE` is a well-tested SQLite feature
|
||||
- **Low risk**: The SQL change in `handle_req_message()` is a drop-in replacement (same bind params)
|
||||
- **Medium risk**: Startup population on a large database could take a few seconds — but with only 55K events, this should complete in under 1 second
|
||||
- **Write overhead**: Each event INSERT now also inserts ~5 rows into event_tags — negligible at 56 events/hour
|
||||
@@ -0,0 +1,585 @@
|
||||
# Plan: Ban Idle and Early-Disconnect Connections
|
||||
|
||||
## Problem Statement
|
||||
|
||||
The relay needs to defend against spam connections that:
|
||||
1. Connect via WebSocket and sit idle doing nothing (the original problem)
|
||||
2. Connect and immediately disconnect without subscribing or posting (the new requirement)
|
||||
|
||||
Both patterns indicate bot/scanner behavior that should result in IP bans.
|
||||
|
||||
## Goal
|
||||
|
||||
Implement a system that:
|
||||
- Bans IPs that connect but never send a REQ or EVENT (regardless of how the connection ends)
|
||||
- Works **independent of NIP-42 authentication** (no auth required)
|
||||
- Allows legitimate users to connect and use the relay normally
|
||||
- Uses existing IP ban infrastructure
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Client connects] --> B{IP banned?}
|
||||
B -->|Yes| C[Reject immediately]
|
||||
B -->|No| D[Set idle timeout timer]
|
||||
D --> E{Client sends REQ or EVENT?}
|
||||
E -->|Yes| F[Mark session ACTIVE<br>Cancel idle timer]
|
||||
E -->|No| G{Connection closes?}
|
||||
G --> H{Session was ACTIVE?}
|
||||
H -->|Yes| I[Clean close - no ban]
|
||||
H -->|No| J["ip_ban_record_failure()<br>→ may trigger ban"]
|
||||
F --> K[Normal operation]
|
||||
K --> L[Connection closes]
|
||||
L --> I
|
||||
```
|
||||
|
||||
## Key Insight
|
||||
|
||||
The distinction between "idle timeout" and "early disconnect" is minimal:
|
||||
- **Idle timeout**: Connection closed by server because client never became ACTIVE
|
||||
- **Early disconnect**: Connection closed by client because client never became ACTIVE
|
||||
|
||||
Both result in the same check: `if (!session_was_active) ban_ip()`
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### 1. Add Session Activity Tracking
|
||||
|
||||
**File: `src/websockets.h`**
|
||||
|
||||
Add to `struct per_session_data`:
|
||||
|
||||
```c
|
||||
// Session activity tracking for idle connection banning
|
||||
int session_active; // 1 if client sent REQ or EVENT, 0 otherwise
|
||||
int idle_timeout_sec; // Timeout value for this session (copied from config)
|
||||
```
|
||||
|
||||
### 2. Set Idle Timeout on All Connections
|
||||
|
||||
**File: `src/websockets.c` - `LWS_CALLBACK_ESTABLISHED`**
|
||||
|
||||
Currently (lines 561-572):
|
||||
```c
|
||||
// Only set timeout if auth is required
|
||||
if (pss->nip42_auth_required_events || pss->nip42_auth_required_subscriptions) {
|
||||
int auth_timeout = get_config_int("nip42_auth_timeout_sec", 10);
|
||||
if (auth_timeout > 0) {
|
||||
lws_set_timeout(wsi, PENDING_TIMEOUT_AWAITING_PING, auth_timeout);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Change to:
|
||||
```c
|
||||
// Initialize session activity tracking
|
||||
pss->session_active = 0;
|
||||
pss->idle_timeout_sec = get_config_int("idle_connection_timeout_sec", 30);
|
||||
|
||||
// Set idle timeout for ALL connections (not just auth-required)
|
||||
// This catches bots that connect and do nothing
|
||||
if (pss->idle_timeout_sec > 0) {
|
||||
lws_set_timeout(wsi, PENDING_TIMEOUT_AWAITING_PING, pss->idle_timeout_sec);
|
||||
DEBUG_TRACE("Idle timeout set: %d seconds for connection from %s",
|
||||
pss->idle_timeout_sec, pss->client_ip);
|
||||
}
|
||||
|
||||
// Also set auth timeout if auth is required (separate concern)
|
||||
if (pss->nip42_auth_required_events || pss->nip42_auth_required_subscriptions) {
|
||||
int auth_timeout = get_config_int("nip42_auth_timeout_sec", 10);
|
||||
// Use the shorter of the two timeouts
|
||||
int effective_timeout = (pss->idle_timeout_sec > 0 && pss->idle_timeout_sec < auth_timeout)
|
||||
? pss->idle_timeout_sec
|
||||
: auth_timeout;
|
||||
if (effective_timeout > 0) {
|
||||
lws_set_timeout(wsi, PENDING_TIMEOUT_AWAITING_PING, effective_timeout);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Mark Session as Active on Valid Activity
|
||||
|
||||
**File: `src/websockets.c` - `LWS_CALLBACK_RECEIVE`**
|
||||
|
||||
When processing REQ message (around line 1030):
|
||||
```c
|
||||
// Before handling REQ, mark session as active
|
||||
pthread_mutex_lock(&pss->session_lock);
|
||||
pss->session_active = 1;
|
||||
pthread_mutex_unlock(&pss->session_lock);
|
||||
|
||||
// Cancel idle timeout - this is legitimate usage
|
||||
lws_set_timeout(wsi, NO_PENDING_TIMEOUT, 0);
|
||||
```
|
||||
|
||||
When processing EVENT message (around line 1380):
|
||||
```c
|
||||
// Before handling EVENT, mark session as active
|
||||
pthread_mutex_lock(&pss->session_lock);
|
||||
pss->session_active = 1;
|
||||
pthread_mutex_unlock(&pss->session_lock);
|
||||
|
||||
// Cancel idle timeout - this is legitimate usage
|
||||
lws_set_timeout(wsi, NO_PENDING_TIMEOUT, 0);
|
||||
```
|
||||
|
||||
**Note**: We should also consider AUTH as "activity" since the client is engaging with the protocol:
|
||||
```c
|
||||
// In handle_nip42_auth_signed_event (nip042.c)
|
||||
// After successful auth, also mark session active
|
||||
pthread_mutex_lock(&pss->session_lock);
|
||||
pss->session_active = 1;
|
||||
pthread_mutex_unlock(&pss->session_lock);
|
||||
```
|
||||
|
||||
### 3.5. Separate Rate Limiting for Idle Connections
|
||||
|
||||
Per user feedback, idle/early-disconnect failures have **separate rate limiting** from auth failures. This requires:
|
||||
|
||||
**New Config Keys:**
|
||||
|
||||
| Config Key | Type | Default | Range | Description |
|
||||
|------------|------|---------|-------|-------------|
|
||||
| `idle_ban_threshold` | int | 3 | 1-100 | Idle failures before ban |
|
||||
| `idle_ban_window_sec` | int | 60 | 10-3600 | Window for counting idle failures |
|
||||
| `idle_ban_duration_sec` | int | 300 | 60-86400 | Initial idle ban duration |
|
||||
|
||||
**Add to `ip_ban_entry_t` in `src/ip_ban.c`:**
|
||||
|
||||
```c
|
||||
typedef struct {
|
||||
int state;
|
||||
char ip[46];
|
||||
|
||||
// Auth failure tracking (existing)
|
||||
int failure_count;
|
||||
time_t first_failure;
|
||||
time_t banned_until;
|
||||
int ban_count;
|
||||
|
||||
// NEW: Idle failure tracking (separate)
|
||||
int idle_failure_count;
|
||||
time_t idle_first_failure;
|
||||
time_t idle_banned_until;
|
||||
int idle_ban_count;
|
||||
|
||||
// Other existing fields...
|
||||
int has_authed_successfully;
|
||||
time_t last_success_at;
|
||||
int total_connections;
|
||||
int total_failures;
|
||||
int total_successes;
|
||||
time_t first_seen;
|
||||
} ip_ban_entry_t;
|
||||
```
|
||||
|
||||
**New function in `src/ip_ban.c`:**
|
||||
|
||||
```c
|
||||
// Record an idle/early-disconnect failure for an IP
|
||||
void ip_ban_record_idle_failure(const char* ip) {
|
||||
if (!ip || !g_initialized) return;
|
||||
if (!get_config_bool("idle_ban_enabled", 1)) return;
|
||||
|
||||
int threshold = get_config_int("idle_ban_threshold", 3);
|
||||
int window_sec = get_config_int("idle_ban_window_sec", 60);
|
||||
int ban_duration = get_config_int("idle_ban_duration_sec", 300);
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
ip_ban_entry_t* entry = get_or_create_entry(ip);
|
||||
if (!entry) {
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return;
|
||||
}
|
||||
|
||||
time_t now = time(NULL);
|
||||
|
||||
// Reset window if expired
|
||||
if (entry->idle_first_failure > 0 && (now - entry->idle_first_failure) > window_sec) {
|
||||
entry->idle_failure_count = 0;
|
||||
entry->idle_first_failure = now;
|
||||
}
|
||||
if (entry->idle_first_failure == 0) {
|
||||
entry->idle_first_failure = now;
|
||||
}
|
||||
|
||||
entry->idle_failure_count++;
|
||||
entry->total_failures++;
|
||||
|
||||
DEBUG_TRACE("IP %s idle failure count: %d/%d", ip, entry->idle_failure_count, threshold);
|
||||
|
||||
if (entry->idle_failure_count >= threshold) {
|
||||
int duration = ban_duration;
|
||||
for (int i = 0; i < entry->idle_ban_count && duration < 86400; i++) {
|
||||
duration *= 2;
|
||||
}
|
||||
if (duration > 86400) duration = 86400;
|
||||
|
||||
entry->idle_banned_until = now + duration;
|
||||
entry->idle_ban_count++;
|
||||
entry->idle_failure_count = 0;
|
||||
entry->idle_first_failure = 0;
|
||||
|
||||
DEBUG_WARN("IP %s banned for %d seconds (idle ban #%d) after %d idle failures",
|
||||
ip, duration, entry->idle_ban_count, threshold);
|
||||
}
|
||||
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
}
|
||||
```
|
||||
|
||||
**Update `ip_ban_is_banned()` to check BOTH ban types:**
|
||||
|
||||
```c
|
||||
int ip_ban_is_banned(const char* ip) {
|
||||
if (!ip || !g_initialized) return 0;
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
int idx = find_slot(ip);
|
||||
if (idx < 0 || g_ban_table[idx].state == IP_BAN_EMPTY) {
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return 0;
|
||||
}
|
||||
|
||||
ip_ban_entry_t* entry = &g_ban_table[idx];
|
||||
time_t now = time(NULL);
|
||||
int banned = 0;
|
||||
|
||||
// Check auth ban (if enabled)
|
||||
if (get_config_bool("auth_fail_ban_enabled", 1) &&
|
||||
entry->banned_until > 0 && now < entry->banned_until) {
|
||||
banned = 1;
|
||||
}
|
||||
|
||||
// Check idle ban (if enabled)
|
||||
if (get_config_bool("idle_ban_enabled", 1) &&
|
||||
entry->idle_banned_until > 0 && now < entry->idle_banned_until) {
|
||||
banned = 1;
|
||||
}
|
||||
|
||||
// Clear expired bans
|
||||
if (!banned) {
|
||||
if (entry->banned_until > 0 && now >= entry->banned_until) {
|
||||
entry->banned_until = 0;
|
||||
entry->failure_count = 0;
|
||||
entry->first_failure = 0;
|
||||
}
|
||||
if (entry->idle_banned_until > 0 && now >= entry->idle_banned_until) {
|
||||
entry->idle_banned_until = 0;
|
||||
entry->idle_failure_count = 0;
|
||||
entry->idle_first_failure = 0;
|
||||
}
|
||||
}
|
||||
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return banned;
|
||||
}
|
||||
```
|
||||
|
||||
**Update persistence:**
|
||||
- `ip_ban_load_from_db()`: Load idle_* fields from new columns
|
||||
- `ip_ban_save_to_db()`: Save idle_* fields to new columns
|
||||
- Add migration SQL to create new columns if they don't exist
|
||||
|
||||
**Add to `src/ip_ban.h`:**
|
||||
|
||||
```c
|
||||
// Record an idle/early-disconnect failure for an IP
|
||||
void ip_ban_record_idle_failure(const char* ip);
|
||||
```
|
||||
|
||||
### 4. Record Failure on Inactive Connection Close
|
||||
|
||||
**File: `src/websockets.c` - `LWS_CALLBACK_CLOSED`**
|
||||
|
||||
Currently (lines 2121-2128):
|
||||
```c
|
||||
// Record auth failure if connection closed while unauthenticated and auth was required
|
||||
if (!pss->authenticated &&
|
||||
(pss->nip42_auth_required_events || pss->nip42_auth_required_subscriptions) &&
|
||||
pss->auth_challenge_sent &&
|
||||
strlen(pss->client_ip) > 0) {
|
||||
ip_ban_record_failure(pss->client_ip);
|
||||
}
|
||||
```
|
||||
|
||||
Change to:
|
||||
```c
|
||||
// Record failure if connection closed without ever becoming active
|
||||
// This catches:
|
||||
// 1. Idle connections that timed out (never sent REQ/EVENT/AUTH)
|
||||
// 2. Early disconnects (client closed before sending REQ/EVENT/AUTH)
|
||||
if (!pss->session_active && strlen(pss->client_ip) > 0) {
|
||||
// Use separate idle failure recording (has its own threshold/duration)
|
||||
ip_ban_record_idle_failure(pss->client_ip);
|
||||
DEBUG_LOG("Recording idle/early-disconnect failure for IP %s (connected %ld seconds)",
|
||||
pss->client_ip,
|
||||
time(NULL) - pss->connection_established);
|
||||
}
|
||||
// Legacy: record auth failure if auth was required but not completed
|
||||
else if (!pss->authenticated &&
|
||||
(pss->nip42_auth_required_events || pss->nip42_auth_required_subscriptions) &&
|
||||
pss->auth_challenge_sent &&
|
||||
strlen(pss->client_ip) > 0) {
|
||||
ip_ban_record_failure(pss->client_ip);
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Add Configuration Keys
|
||||
|
||||
**File: `src/config.c` - Add validation for new keys**
|
||||
|
||||
In the config validation section, add:
|
||||
|
||||
```c
|
||||
// Idle connection ban settings
|
||||
if (strcmp(key, "idle_connection_timeout_sec") == 0) {
|
||||
if (!is_valid_positive_integer(value)) {
|
||||
snprintf(error_msg, error_size, "invalid idle_connection_timeout_sec '%s' (must be positive integer)", value);
|
||||
return -1;
|
||||
}
|
||||
int timeout = atoi(value);
|
||||
if (timeout < 5 || timeout > 300) {
|
||||
snprintf(error_msg, error_size, "idle_connection_timeout_sec must be between 5 and 300 seconds");
|
||||
return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (strcmp(key, "idle_ban_enabled") == 0) {
|
||||
if (!is_valid_boolean(value)) {
|
||||
snprintf(error_msg, error_size, "invalid boolean value '%s' for idle_ban_enabled", value);
|
||||
return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (strcmp(key, "idle_ban_threshold") == 0) {
|
||||
if (!is_valid_positive_integer(value)) {
|
||||
snprintf(error_msg, error_size, "invalid idle_ban_threshold '%s' (must be positive integer)", value);
|
||||
return -1;
|
||||
}
|
||||
int threshold = atoi(value);
|
||||
if (threshold < 1 || threshold > 100) {
|
||||
snprintf(error_msg, error_size, "idle_ban_threshold must be between 1 and 100");
|
||||
return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (strcmp(key, "idle_ban_window_sec") == 0) {
|
||||
if (!is_valid_positive_integer(value)) {
|
||||
snprintf(error_msg, error_size, "invalid idle_ban_window_sec '%s' (must be positive integer)", value);
|
||||
return -1;
|
||||
}
|
||||
int window = atoi(value);
|
||||
if (window < 10 || window > 3600) {
|
||||
snprintf(error_msg, error_size, "idle_ban_window_sec must be between 10 and 3600");
|
||||
return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (strcmp(key, "idle_ban_duration_sec") == 0) {
|
||||
if (!is_valid_positive_integer(value)) {
|
||||
snprintf(error_msg, error_size, "invalid idle_ban_duration_sec '%s' (must be positive integer)", value);
|
||||
return -1;
|
||||
}
|
||||
int duration = atoi(value);
|
||||
if (duration < 60 || duration > 86400) {
|
||||
snprintf(error_msg, error_size, "idle_ban_duration_sec must be between 60 and 86400");
|
||||
return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
Also update the config introspection/documentation functions to describe these keys.
|
||||
|
||||
### 6. Add API Documentation
|
||||
|
||||
**File: `src/api.c` - Add to config metadata**
|
||||
|
||||
```c
|
||||
{"idle_connection_timeout_sec", "int", 5, 300},
|
||||
{"idle_ban_enabled", "bool", 0, 1},
|
||||
{"idle_ban_threshold", "int", 1, 100},
|
||||
{"idle_ban_window_sec", "int", 10, 3600},
|
||||
{"idle_ban_duration_sec", "int", 60, 86400},
|
||||
```
|
||||
|
||||
Add descriptions in the config query handler:
|
||||
```c
|
||||
} else if (strcmp(key, "idle_connection_timeout_sec") == 0) {
|
||||
description = "Seconds before an idle connection (no REQ/EVENT) is closed and IP flagged. 0 to disable.";
|
||||
} else if (strcmp(key, "idle_ban_enabled") == 0) {
|
||||
description = "Whether to ban IPs that repeatedly connect without sending REQ or EVENT.";
|
||||
} else if (strcmp(key, "idle_ban_threshold") == 0) {
|
||||
description = "Number of idle/early-disconnect failures before banning an IP.";
|
||||
} else if (strcmp(key, "idle_ban_window_sec") == 0) {
|
||||
description = "Time window in seconds for counting idle failures.";
|
||||
} else if (strcmp(key, "idle_ban_duration_sec") == 0) {
|
||||
description = "Initial ban duration in seconds for idle failures (doubles each time).";
|
||||
}
|
||||
```
|
||||
|
||||
### 7. Update ip_ban Persistence and Cleanup
|
||||
|
||||
**File: `src/ip_ban.c` - Update `ip_ban_load_from_db()`**
|
||||
|
||||
Add migration SQL to create new columns if they don't exist, then load them:
|
||||
```sql
|
||||
ALTER TABLE ip_bans ADD COLUMN idle_failure_count INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE ip_bans ADD COLUMN idle_ban_count INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE ip_bans ADD COLUMN idle_banned_until INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE ip_bans ADD COLUMN idle_first_failure INTEGER NOT NULL DEFAULT 0;
|
||||
```
|
||||
|
||||
**File: `src/ip_ban.c` - Update `ip_ban_save_to_db()`**
|
||||
|
||||
Add the idle_* fields to the INSERT/REPLACE statement.
|
||||
|
||||
**File: `src/ip_ban.c` - Update `ip_ban_cleanup()`**
|
||||
|
||||
Add cleanup logic for idle ban entries (same pattern as auth ban cleanup).
|
||||
|
||||
**File: `src/ip_ban.c` - Update `ip_ban_log_stats()`**
|
||||
|
||||
Add idle ban count to the periodic log summary:
|
||||
```c
|
||||
DEBUG_WARN("IP BAN SUMMARY: %d auth-banned, %d idle-banned, %d tracked, %d trusted",
|
||||
auth_banned_count, idle_banned_count, tracked_count, trusted_count);
|
||||
```
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
### Idle Connection Settings
|
||||
|
||||
| Config Key | Type | Default | Range | Description |
|
||||
|------------|------|---------|-------|-------------|
|
||||
| `idle_connection_timeout_sec` | int | 30 | 5-300 | Seconds to wait for REQ/EVENT before closing connection. 0 = disable idle timeout. |
|
||||
| `idle_ban_enabled` | bool | true | true/false | Whether to ban IPs with repeated idle/early-disconnect failures. |
|
||||
| `idle_ban_threshold` | int | 3 | 1-100 | Idle failures before ban. |
|
||||
| `idle_ban_window_sec` | int | 60 | 10-3600 | Window for counting idle failures. |
|
||||
| `idle_ban_duration_sec` | int | 300 | 60-86400 | Initial idle ban duration. |
|
||||
|
||||
### Auth Failure Settings (Existing)
|
||||
|
||||
| Config Key | Type | Default | Range | Description |
|
||||
|------------|------|---------|-------|-------------|
|
||||
| `auth_fail_ban_enabled` | bool | true | true/false | Whether to ban IPs with failed auth. |
|
||||
| `auth_fail_ban_threshold` | int | 3 | 1-100 | Auth failures before ban. |
|
||||
| `auth_fail_window_sec` | int | 60 | 10-3600 | Window for counting auth failures. |
|
||||
| `auth_fail_ban_duration_sec` | int | 300 | 60-86400 | Initial auth ban duration. |
|
||||
|
||||
## Behavior Matrix
|
||||
|
||||
| Scenario | idle_timeout_sec | idle_ban_enabled | Result |
|
||||
|----------|------------------|------------------|--------|
|
||||
| Connect → do nothing → timeout | >0 | true | Connection closed, IP failure recorded, may be banned |
|
||||
| Connect → do nothing → timeout | >0 | false | Connection closed, no ban |
|
||||
| Connect → immediate disconnect | any | true | IP failure recorded, may be banned |
|
||||
| Connect → immediate disconnect | any | false | No ban |
|
||||
| Connect → send REQ | any | any | Session active, no ban |
|
||||
| Connect → send EVENT | any | any | Session active, no ban |
|
||||
| Connect → send AUTH (NIP-42) | any | any | Session active, no ban |
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
1. **Idle timeout test**: Connect via wscat, wait 30 seconds, verify disconnect and ban table entry
|
||||
2. **Early disconnect test**: Connect via wscat, immediately Ctrl-C, verify ban table entry
|
||||
3. **Active session test**: Connect, send REQ, disconnect immediately, verify NO ban
|
||||
4. **Config disable test**: Set `idle_ban_enabled=false`, repeat test 1, verify no ban
|
||||
5. **Timeout config test**: Set `idle_connection_timeout_sec=5`, verify faster disconnect
|
||||
|
||||
## Files to Modify
|
||||
|
||||
| File | Change Type |
|
||||
|------|-------------|
|
||||
| `src/websockets.h` | Add `session_active` and `idle_timeout_sec` fields to `per_session_data` |
|
||||
| `src/websockets.c` ~565 | Set idle timeout on ALL connections in `LWS_CALLBACK_ESTABLISHED` |
|
||||
| `src/websockets.c` ~1030 | Mark `session_active=1` and cancel idle timer on REQ |
|
||||
| `src/websockets.c` ~1380 | Mark `session_active=1` and cancel idle timer on EVENT |
|
||||
| `src/websockets.c` ~2121 | Call `ip_ban_record_idle_failure()` for inactive sessions in `LWS_CALLBACK_CLOSED` |
|
||||
| `src/nip042.c` ~130 | Mark `session_active=1` on successful AUTH |
|
||||
| `src/ip_ban.h` | Add `ip_ban_record_idle_failure()` declaration |
|
||||
| `src/ip_ban.c` | Add idle_* fields to `ip_ban_entry_t`, new `ip_ban_record_idle_failure()` function, update `ip_ban_is_banned()` to check both ban types, update persistence and cleanup |
|
||||
| `src/config.c` ~987+ | Add validation for 5 new idle_* config keys |
|
||||
| `src/api.c` ~664 | Add config metadata entries for idle_* keys |
|
||||
|
||||
## Deployment on Existing Server
|
||||
|
||||
### Config Keys: No Manual SQL Required
|
||||
|
||||
All config keys use [`get_config_int()`](src/config.c:300) and [`get_config_bool()`](src/config.c:313) which return **hardcoded defaults** when a key doesn't exist in the config table. The new code will work immediately with these defaults:
|
||||
|
||||
- `idle_connection_timeout_sec` → defaults to 30
|
||||
- `idle_ban_enabled` → defaults to true
|
||||
- `idle_ban_threshold` → defaults to 3
|
||||
- `idle_ban_window_sec` → defaults to 60
|
||||
- `idle_ban_duration_sec` → defaults to 300
|
||||
|
||||
If you want to **override** any defaults, you can insert them into the config table. From the same directory as the `.db` file:
|
||||
|
||||
```bash
|
||||
# Find your database file
|
||||
DB_FILE=$(ls *.db | head -1)
|
||||
|
||||
# Optional: Insert custom values (only if you want non-default settings)
|
||||
sqlite3 "$DB_FILE" "INSERT OR REPLACE INTO config (key, value) VALUES ('idle_connection_timeout_sec', '30');"
|
||||
sqlite3 "$DB_FILE" "INSERT OR REPLACE INTO config (key, value) VALUES ('idle_ban_enabled', 'true');"
|
||||
sqlite3 "$DB_FILE" "INSERT OR REPLACE INTO config (key, value) VALUES ('idle_ban_threshold', '3');"
|
||||
sqlite3 "$DB_FILE" "INSERT OR REPLACE INTO config (key, value) VALUES ('idle_ban_window_sec', '60');"
|
||||
sqlite3 "$DB_FILE" "INSERT OR REPLACE INTO config (key, value) VALUES ('idle_ban_duration_sec', '300');"
|
||||
```
|
||||
|
||||
Or change them via the admin API/web UI after deployment.
|
||||
|
||||
### ip_bans Table: Automatic Migration
|
||||
|
||||
The `ip_bans` table needs 4 new columns for idle tracking. The updated [`ip_ban_load_from_db()`](src/ip_ban.c:93) will run `ALTER TABLE` statements automatically on startup. These are safe because SQLite's `ALTER TABLE ADD COLUMN` is a no-op if the column already exists (we'll use error-tolerant execution).
|
||||
|
||||
If you prefer to run the migration manually before deploying:
|
||||
|
||||
```bash
|
||||
DB_FILE=$(ls *.db | head -1)
|
||||
|
||||
sqlite3 "$DB_FILE" <<'SQL'
|
||||
ALTER TABLE ip_bans ADD COLUMN idle_failure_count INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE ip_bans ADD COLUMN idle_ban_count INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE ip_bans ADD COLUMN idle_banned_until INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE ip_bans ADD COLUMN idle_first_failure INTEGER NOT NULL DEFAULT 0;
|
||||
SQL
|
||||
```
|
||||
|
||||
**Note:** SQLite will error on `ALTER TABLE ADD COLUMN` if the column already exists, but the code will handle this gracefully (ignore the error). Running it manually is optional — the relay will do it on startup.
|
||||
|
||||
### Deployment Steps
|
||||
|
||||
1. Build and deploy with `deploy_lt.sh` as usual
|
||||
2. The relay starts, `ip_ban_load_from_db()` auto-migrates the `ip_bans` table
|
||||
3. Config keys use defaults immediately — no manual SQL needed
|
||||
4. Optionally tune settings via admin API or direct SQL
|
||||
|
||||
## Backward Compatibility
|
||||
|
||||
- Default `idle_connection_timeout_sec=30` provides immediate protection
|
||||
- Default `idle_ban_enabled=true` maintains current spam protection behavior
|
||||
- Setting `idle_connection_timeout_sec=0` restores old behavior (no idle timeout)
|
||||
- Setting `idle_ban_enabled=false` allows connections without banning (for debugging)
|
||||
- Existing auth-based banning continues to work independently with its own thresholds
|
||||
- Database migration adds new columns with defaults — existing ban data preserved
|
||||
|
||||
## Summary
|
||||
|
||||
This plan implements a unified "session activity" tracking system with **separate rate limiting** for idle vs auth failures:
|
||||
|
||||
1. **Catches idle connections** via `lws_set_timeout()` on ALL connections (not just auth-required)
|
||||
2. **Catches early disconnects** via the same `!session_active` check on close
|
||||
3. **Respects legitimate users** who actually use the relay (REQ/EVENT/AUTH marks session active)
|
||||
4. **Separate idle ban tracking** — idle failures have their own threshold, window, and duration independent of auth failures
|
||||
5. **Extends existing ban infrastructure** — adds idle_* fields to `ip_ban_entry_t`, unified `ip_ban_is_banned()` checks both ban types
|
||||
6. **Fully configurable** — 5 new config keys for idle banning, all tunable via admin events
|
||||
|
||||
The key insight is that there's no meaningful difference between "idle timeout" and "early disconnect" — both indicate a client that connected but never actually used the relay. The same `!session_active` check handles both cases, and the separate rate limiting ensures idle bots don't interfere with auth failure tracking for legitimate clients who fail NIP-42.
|
||||
@@ -0,0 +1,194 @@
|
||||
# Main Thread CPU Offload Plan
|
||||
|
||||
## Problem Statement
|
||||
|
||||
The main `c_relay_pg` thread consumes ~56% CPU while the DB worker threads (`db-read-1..4`, `db-write`) sit near 0%. All protocol handling, JSON parsing, event validation, subscription matching, and message queueing happens synchronously inside `nostr_relay_callback()` on the main libwebsockets event-loop thread.
|
||||
|
||||
## Current Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
LWS[lws_service - main thread] --> CB[nostr_relay_callback]
|
||||
CB --> PARSE[JSON parse - cJSON_Parse]
|
||||
CB --> VALIDATE[Signature verify - nostr_validate_unified_request]
|
||||
CB --> STORE[store_event - builds payload]
|
||||
STORE --> SYNC_WRITE[thread_pool_execute_store_event_sync]
|
||||
SYNC_WRITE --> |blocks main thread| DB_WRITE[db-write thread]
|
||||
DB_WRITE --> |signal| SYNC_WRITE
|
||||
CB --> REQ[handle_req_message - builds SQL]
|
||||
REQ --> SYNC_READ[thread_pool_execute_req_sync]
|
||||
SYNC_READ --> |blocks main thread| DB_READ[db-read-N thread]
|
||||
DB_READ --> |signal| SYNC_READ
|
||||
CB --> BROADCAST[broadcast_event_to_subscriptions]
|
||||
BROADCAST --> QUEUE[queue_message per subscriber]
|
||||
CB --> WRITEABLE[LWS_CALLBACK_SERVER_WRITEABLE]
|
||||
WRITEABLE --> DRAIN[process_message_queue - lws_write]
|
||||
```
|
||||
|
||||
### Why the main thread is hot
|
||||
|
||||
| Work item | Where | Cost |
|
||||
|-----------|-------|------|
|
||||
| JSON parsing | `cJSON_Parse` in `LWS_CALLBACK_RECEIVE` | Medium - per message |
|
||||
| Signature verification | `nostr_validate_unified_request` - ed25519 crypto | **High** - per EVENT |
|
||||
| SQL query building | `handle_req_message` filter-to-SQL loop | Low-Medium |
|
||||
| Sync DB wait | `thread_pool_execute_*_sync` - pthread_cond_wait | Blocks but yields CPU |
|
||||
| Result iteration + expiration check | Row loop in `handle_req_message` with `cJSON_Parse` per row | Medium-High for large result sets |
|
||||
| Subscription matching | `broadcast_event_to_subscriptions` - filter matching | Medium - scales with subscriber count |
|
||||
| Message serialization + queueing | `snprintf` + `queue_message_take_ownership` per subscriber | Medium |
|
||||
| Config lookups | `get_config_int/bool` called repeatedly in hot paths | Low but frequent |
|
||||
|
||||
### Key constraint: libwebsockets is single-threaded
|
||||
|
||||
libwebsockets requires that **all `lws_write`, `lws_callback_on_writable`, and `lws_close_reason` calls happen from the service thread** (the thread running `lws_service`). This means we cannot directly write to WebSocket connections from worker threads. However, we **can** do computation on worker threads and post results back to the main thread for I/O.
|
||||
|
||||
## Offload Strategy
|
||||
|
||||
### Phase 1: Async EVENT Processing (Highest Impact)
|
||||
|
||||
Convert EVENT handling from synchronous to async. Currently the main thread does: parse → validate → store → broadcast, all blocking. Instead:
|
||||
|
||||
1. **Main thread**: Parse JSON (fast), extract event ID for dedup check, then submit a job to a new "event processing" worker thread
|
||||
2. **Worker thread**: Signature verification (expensive crypto), store_event (already goes to DB thread), prepare broadcast payload
|
||||
3. **Main thread callback**: Receive result via `wake_loop_cb` + `lws_cancel_service`, send OK response and broadcast to subscribers
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
LWS[lws_service - main thread] --> RECEIVE[LWS_CALLBACK_RECEIVE]
|
||||
RECEIVE --> PARSE[JSON parse + dedup check]
|
||||
PARSE --> SUBMIT[Submit to event-worker queue]
|
||||
SUBMIT --> LWS
|
||||
|
||||
WORKER[event-worker thread] --> VERIFY[Signature verification]
|
||||
VERIFY --> STORE_DB[store_event via DB pool]
|
||||
STORE_DB --> PREP[Prepare broadcast payload]
|
||||
PREP --> RESULT_Q[Push result to completion queue]
|
||||
RESULT_Q --> WAKE[lws_cancel_service]
|
||||
|
||||
LWS2[lws_service wakes] --> POLL[Poll completion queue]
|
||||
POLL --> OK[Send OK response via queue_message]
|
||||
POLL --> BCAST[broadcast_event_to_subscriptions]
|
||||
```
|
||||
|
||||
**What this offloads**: ed25519 signature verification (~the most expensive per-event operation), event classification, tag serialization, and the synchronous DB store wait.
|
||||
|
||||
**What stays on main thread**: JSON parse (needed to extract event ID for dedup), OK response writing, broadcast fan-out (requires lws access).
|
||||
|
||||
### Phase 2: Async REQ Query Execution (Medium Impact)
|
||||
|
||||
Convert REQ handling from sync to async:
|
||||
|
||||
1. **Main thread**: Parse filters, build SQL, create subscription, submit query job
|
||||
2. **DB reader thread**: Execute query (already happens, but currently blocks main thread via `_sync`)
|
||||
3. **Main thread callback**: Iterate results, queue EVENT messages, send EOSE
|
||||
|
||||
This is simpler than Phase 1 because the thread pool already supports async submission via `thread_pool_submit_read` with a `result_cb`. The `_sync` wrappers just add a condvar wait on top. We need to:
|
||||
- Use `thread_pool_submit_read` directly instead of `thread_pool_execute_req_sync`
|
||||
- Store pending REQ context (sub_id, wsi, pss) so the callback can complete the work
|
||||
- In the result callback, push results to a completion queue and call `lws_cancel_service`
|
||||
- On the main thread, drain the completion queue and send EVENT + EOSE messages
|
||||
|
||||
### Phase 3: Reduce Per-Row Overhead in REQ Results (Low-Medium Impact)
|
||||
|
||||
Currently each row from a REQ query gets `cJSON_Parse` just to check NIP-40 expiration. This is wasteful:
|
||||
|
||||
- Option A: Add an `expiration` column to the events table so expiration filtering can be done in SQL
|
||||
- Option B: Store expiration timestamp in a fast-parse format (extract during INSERT, store as integer column)
|
||||
- Option C: Use string search on the raw JSON for the expiration tag instead of full parse
|
||||
|
||||
### Phase 4: Config Value Caching (Low Impact, Easy Win)
|
||||
|
||||
`get_config_int` and `get_config_bool` are called on every message in hot paths. These do SQLite queries. Cache config values in memory with a TTL or invalidation signal, so the main loop only refreshes them periodically (already done for `debug_level` every 60s — extend to all hot-path config values).
|
||||
|
||||
## Implementation Priority
|
||||
|
||||
| Phase | Impact | Risk | Complexity |
|
||||
|-------|--------|------|------------|
|
||||
| Phase 1: Async EVENT | **High** - removes crypto from main thread | Medium - async state management | Medium-High |
|
||||
| Phase 2: Async REQ | **Medium** - unblocks main thread during queries | Low - infrastructure exists | Medium |
|
||||
| Phase 3: Expiration optimization | **Low-Medium** - reduces per-row parse cost | Low | Low |
|
||||
| Phase 4: Config caching | **Low** - reduces DB round-trips | Very Low | Low |
|
||||
|
||||
## Detailed Design: Phase 1 (Async EVENT Processing)
|
||||
|
||||
### New Components
|
||||
|
||||
#### Completion Queue (`src/completion_queue.h/.c`)
|
||||
A thread-safe FIFO queue for posting results from worker threads back to the main thread:
|
||||
|
||||
```c
|
||||
typedef struct {
|
||||
int type; // COMPLETION_TYPE_EVENT_RESULT, COMPLETION_TYPE_REQ_RESULT, etc.
|
||||
void* data; // Type-specific result data
|
||||
struct lws* wsi; // Target WebSocket connection
|
||||
void* pss; // Per-session data
|
||||
} completion_item_t;
|
||||
|
||||
int completion_queue_init(void);
|
||||
int completion_queue_push(completion_item_t* item);
|
||||
completion_item_t* completion_queue_pop(void); // Non-blocking
|
||||
void completion_queue_shutdown(void);
|
||||
```
|
||||
|
||||
#### Event Worker Thread
|
||||
A dedicated pthread that processes EVENT validation/storage:
|
||||
|
||||
```c
|
||||
typedef struct {
|
||||
cJSON* event; // Parsed event JSON - ownership transferred
|
||||
cJSON* full_message; // Full message JSON for context
|
||||
struct lws* wsi;
|
||||
void* pss;
|
||||
char sub_id[64]; // For response routing
|
||||
} event_work_item_t;
|
||||
```
|
||||
|
||||
#### Main Loop Integration
|
||||
Add a completion queue drain step to the main event loop. After `lws_service` returns (either from timeout or `lws_cancel_service` wake), check the completion queue:
|
||||
|
||||
```c
|
||||
while (g_server_running && !g_shutdown_flag) {
|
||||
int result = lws_service(ws_context, 1000);
|
||||
|
||||
// NEW: Drain completion queue
|
||||
completion_item_t* item;
|
||||
while ((item = completion_queue_pop()) != NULL) {
|
||||
process_completion(item); // Send OK, broadcast, etc.
|
||||
free(item);
|
||||
}
|
||||
|
||||
// ... existing periodic checks ...
|
||||
}
|
||||
```
|
||||
|
||||
### Changes to Existing Code
|
||||
|
||||
1. **`nostr_relay_callback` EVENT path**: After JSON parse and dedup check, instead of calling `nostr_validate_unified_request` + `store_event` + `broadcast_event_to_subscriptions` synchronously, submit an `event_work_item_t` to the event worker queue and return 0 immediately.
|
||||
|
||||
2. **`store_event`**: No changes needed — it already uses `thread_pool_execute_store_event_sync` which will run on the DB writer thread. The event worker thread will call it.
|
||||
|
||||
3. **`broadcast_event_to_subscriptions`**: No changes needed — it will be called from the main thread when processing the completion item, which is correct since it calls `queue_message_take_ownership` (requires lws thread).
|
||||
|
||||
### Thread Safety Considerations
|
||||
|
||||
- The `cJSON* event` object must be fully owned by the worker thread during processing. The main thread must not access it after submission.
|
||||
- The `wsi` and `pss` pointers could become invalid if the client disconnects while the event is being processed. The completion handler must validate that the connection is still alive before sending the OK response.
|
||||
- A generation counter or epoch on `pss` can detect stale references.
|
||||
|
||||
## What Cannot Be Offloaded
|
||||
|
||||
- **`lws_write` / `queue_message`**: Must happen on the lws service thread
|
||||
- **`lws_callback_on_writable`**: Must happen on the lws service thread
|
||||
- **`lws_close_reason`**: Must happen on the lws service thread
|
||||
- **Subscription list iteration for broadcast**: Accesses `lws_wsi_user` which is lws-internal
|
||||
|
||||
These are fundamental libwebsockets constraints. The pattern is always: do computation off-thread, post result to completion queue, wake main thread, do I/O on main thread.
|
||||
|
||||
## Expected Impact
|
||||
|
||||
With Phase 1 alone, the main thread would no longer perform:
|
||||
- ed25519 signature verification (~100-500μs per event depending on CPU)
|
||||
- Synchronous DB store wait (~50-200μs per event)
|
||||
- Event classification, tag serialization, JSON serialization for storage
|
||||
|
||||
This should reduce main-thread CPU by roughly 30-50% for EVENT-heavy workloads, shifting that work to the event worker thread and DB threads.
|
||||
@@ -0,0 +1,207 @@
|
||||
# C-Relay-PG Memory Leak Investigation & Fix Plan
|
||||
|
||||
## Problem Statement
|
||||
|
||||
c-relay-pg reached **1.56 GB** of its 1.5 GB `MemoryHigh` cgroup limit after only **1 hour 37 minutes** of runtime. The kernel is actively throttling the process with swap pressure (1.8M swap used). This strongly indicates one or more memory leaks causing unbounded growth.
|
||||
|
||||
## Root Cause Analysis
|
||||
|
||||
After thorough code review, I identified **three categories** of memory issues:
|
||||
|
||||
1. **Confirmed `get_config_value()` leaks** — the most pervasive issue
|
||||
2. **Potential structural leaks** in subscription/connection lifecycle
|
||||
3. **Stack pressure** from large stack-allocated arrays
|
||||
|
||||
---
|
||||
|
||||
## Category 1: `get_config_value()` Leaks (HIGH SEVERITY — Most Likely Root Cause)
|
||||
|
||||
### The Core Problem
|
||||
|
||||
[`get_config_value()`](src/config.c:293) calls [`get_config_value_from_table()`](src/config.c:1690) which returns `strdup(value)` — a **heap-allocated string that the caller must free**. Many call sites treat the return value as a borrowed pointer and never free it.
|
||||
|
||||
**Every unfree'd call leaks ~64-256 bytes per invocation.** On a busy relay processing hundreds of events/second, this accumulates to GB-scale leaks within hours.
|
||||
|
||||
### Confirmed Leak Sites
|
||||
|
||||
#### `websockets.c` — Called on EVERY incoming event
|
||||
|
||||
| Line | Code | Frequency | Severity |
|
||||
|------|------|-----------|----------|
|
||||
| [1527](src/websockets.c:1527) | `get_config_value("relay_pubkey")` — auth bypass check for kind 14 DMs | Every kind-14 event | **HIGH** |
|
||||
| [1549](src/websockets.c:1549) | `get_config_value("admin_pubkey")` — auth bypass check for kind 23456 | Every kind-23456 event | **MEDIUM** |
|
||||
| [2704](src/websockets.c:2704) | `get_config_value("relay_pubkey")` — DM stats command processing | Every DM to relay | **MEDIUM** |
|
||||
| [2742](src/websockets.c:2742) | `get_config_value("admin_pubkey")` — DM admin check | Every DM to relay | **MEDIUM** |
|
||||
|
||||
#### `main.c` — Called on EVERY admin event check
|
||||
|
||||
| Line | Code | Frequency | Severity |
|
||||
|------|------|-----------|----------|
|
||||
| [1718](src/main.c:1718) | `get_config_value("relay_pubkey")` — inside a loop iterating tags | Every admin event, per tag | **CRITICAL** |
|
||||
| [1742](src/main.c:1742) | `get_config_value("admin_pubkey")` — admin authorization | Every admin event | **HIGH** |
|
||||
|
||||
#### `config.c` — Called on EVERY event kind check
|
||||
|
||||
| Line | Code | Frequency | Severity |
|
||||
|------|------|-----------|----------|
|
||||
| [377](src/config.c:377) | `get_config_value("nip42_auth_required_kinds")` — NIP-42 kind check | Every incoming event | **CRITICAL** |
|
||||
|
||||
#### `ip_ban.c` — Called on EVERY ban check
|
||||
|
||||
| Line | Code | Frequency | Severity |
|
||||
|------|------|-----------|----------|
|
||||
| [255](src/ip_ban.c:255) | `get_config_value("idle_ban_whitelist")` — whitelist check | Every connection check | **HIGH** |
|
||||
|
||||
#### `nip042.c` — Called on every auth verification
|
||||
|
||||
| Line | Code | Frequency | Severity |
|
||||
|------|------|-----------|----------|
|
||||
| [99](src/nip042.c:99) | `get_config_value("relay_url")` — relay URL for auth verification | Every NIP-42 auth | **MEDIUM** |
|
||||
|
||||
#### `request_validator.c` — Called on every auth rules check
|
||||
|
||||
| Line | Code | Frequency | Severity |
|
||||
|------|------|-----------|----------|
|
||||
| [208](src/request_validator.c:208) | `get_config_value("auth_enabled")` — properly freed | N/A | OK |
|
||||
| [216](src/request_validator.c:216) | `get_config_value("auth_rules_enabled")` — properly freed | N/A | OK |
|
||||
| [304](src/request_validator.c:304) | `get_config_value("admin_pubkey")` — NOT freed | Every event validation | **HIGH** |
|
||||
| [320](src/request_validator.c:320) | `get_config_value("nip42_auth_enabled")` — NOT freed | Every event validation | **HIGH** |
|
||||
|
||||
### Files That DO Free Correctly (for reference)
|
||||
|
||||
- [`nip011.c`](src/nip011.c:127) — Properly frees all `get_config_value()` results
|
||||
- [`nip013.c`](src/nip013.c:43) — Properly frees `pow_mode`
|
||||
- [`request_validator.c`](src/request_validator.c:191) — Properly frees `nip42_timeout` and `nip42_tolerance`
|
||||
|
||||
### Estimated Impact
|
||||
|
||||
On a relay processing ~100 events/second:
|
||||
- `is_nip42_auth_required_for_kind()` leaks ~100-200 bytes × 100/sec = **~10-20 KB/sec**
|
||||
- `ip_is_whitelisted()` leaks ~100 bytes × connections/sec
|
||||
- `is_authorized_admin_event()` leaks inside tag loop — potentially **multiple leaks per event**
|
||||
- Combined: **~50-100 KB/sec → ~180-360 MB/hour** — consistent with the observed 1.56 GB in 97 minutes
|
||||
|
||||
---
|
||||
|
||||
## Category 2: Structural Lifecycle Leaks (MEDIUM SEVERITY)
|
||||
|
||||
### 2a. `LWS_CALLBACK_CLOSED` — Inactive Subscription Skip
|
||||
|
||||
In [`websockets.c:2339`](src/websockets.c:2339), the cleanup handler only collects subscription IDs where `sub->active` is true:
|
||||
|
||||
```c
|
||||
if (sub->active) { // Only process active subscriptions
|
||||
temp_sub_id_t* temp = malloc(sizeof(temp_sub_id_t));
|
||||
```
|
||||
|
||||
If a subscription was marked inactive by another thread between the time it was added and the connection close, it will be **skipped** — never removed from the global manager, never freed. The subscription object, its filters, and all cJSON objects within leak permanently.
|
||||
|
||||
### 2b. `connection_list_remove` Potential
|
||||
|
||||
The `connection_list_remove(wsi)` call at [`websockets.c:2252`](src/websockets.c:2252) happens before pss cleanup. Need to verify the connection list implementation doesn't hold references that prevent cleanup.
|
||||
|
||||
### 2c. `ip_connection_info_t` Leak on Disconnect
|
||||
|
||||
The per-IP connection tracking in [`subscriptions.h:79`](src/subscriptions.h:79) creates `ip_connection_info_t` nodes via `calloc`. These are only removed by `remove_ip_connection()` — need to verify this is called on every disconnect path.
|
||||
|
||||
---
|
||||
|
||||
## Category 3: Stack Pressure (LOW SEVERITY but notable)
|
||||
|
||||
### 3a. `candidates_to_check` Stack Array
|
||||
|
||||
In [`subscriptions.c:810`](src/subscriptions.c:810):
|
||||
|
||||
```c
|
||||
subscription_t* candidates_to_check[MAX_TOTAL_SUBSCRIPTIONS]; // 5000 × 8 bytes = 40 KB
|
||||
```
|
||||
|
||||
This allocates **40 KB on the stack** for every `broadcast_event_to_subscriptions()` call. While not a heap leak, it contributes to high memory pressure under concurrent load.
|
||||
|
||||
### 3b. `added_kinds` Bitmap
|
||||
|
||||
In [`subscriptions.c:68`](src/subscriptions.c:68):
|
||||
|
||||
```c
|
||||
unsigned char added_kinds[8192] = {0}; // 8 KB on stack
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Fix Implementation Plan
|
||||
|
||||
### Phase 1: Fix `get_config_value()` Leaks (Highest Impact)
|
||||
|
||||
**Strategy A — Preferred: Add a config cache layer**
|
||||
|
||||
Instead of fixing 20+ call sites, add a **cached config system** that reads values once and caches them in memory, invalidating on config change events. This eliminates both the leak AND the repeated SQLite queries per event.
|
||||
|
||||
```
|
||||
Config Cache Architecture:
|
||||
- Static hash table of key-value pairs
|
||||
- Populated at startup and on config change events
|
||||
- get_config_value_cached() returns const pointer to cached value (no allocation)
|
||||
- Existing get_config_value() kept for dynamic/rare lookups
|
||||
```
|
||||
|
||||
**Strategy B — Quick fix: Free at every call site**
|
||||
|
||||
Add `free((char*)result)` after every `get_config_value()` call that doesn't already free. This is more error-prone but faster to implement.
|
||||
|
||||
**Recommendation: Implement Strategy A for hot-path values (relay_pubkey, admin_pubkey, auth settings), Strategy B for cold-path values.**
|
||||
|
||||
### Phase 2: Fix Structural Leaks
|
||||
|
||||
1. **Fix inactive subscription skip in CLOSED handler** — Remove the `if (sub->active)` guard, or add a second pass that also collects inactive subscriptions for cleanup
|
||||
2. **Verify `remove_ip_connection()` is called on all disconnect paths**
|
||||
3. **Add defensive cleanup** — periodic sweep of orphaned subscriptions
|
||||
|
||||
### Phase 3: Add Memory Monitoring
|
||||
|
||||
1. **Add a `/stats` endpoint** that reports:
|
||||
- Current RSS/VSZ from `/proc/self/status`
|
||||
- Active subscription count
|
||||
- Message queue depth across all sessions
|
||||
- Config cache hit/miss ratio
|
||||
2. **Add periodic memory logging** to the service loop
|
||||
3. **Consider integrating jemalloc** for better allocation tracking in production
|
||||
|
||||
### Phase 4: Reduce Stack Pressure
|
||||
|
||||
1. Move `candidates_to_check` to heap allocation with `malloc`/`free`
|
||||
2. Consider reducing `MAX_TOTAL_SUBSCRIPTIONS` or using a dynamic array
|
||||
|
||||
---
|
||||
|
||||
## Verification Strategy
|
||||
|
||||
1. **Build with AddressSanitizer** (`-fsanitize=address`) for development testing
|
||||
2. **Run under Valgrind** with `--leak-check=full` for comprehensive leak detection
|
||||
3. **Monitor RSS over time** after fixes — should plateau rather than grow linearly
|
||||
4. **Load test** with `tests/load_tests.sh` while monitoring memory
|
||||
|
||||
---
|
||||
|
||||
## Immediate Mitigation
|
||||
|
||||
While fixes are being implemented:
|
||||
1. **Raise `MemoryHigh`** to 2 GB in the systemd unit to prevent throttling
|
||||
2. **Add a cron job** to restart the service every 12-24 hours
|
||||
3. **Monitor with**: `watch -n 5 'cat /proc/$(pgrep c_relay_pg)/status | grep -E "VmRSS|VmSwap"'`
|
||||
|
||||
---
|
||||
|
||||
## Implementation Priority Order
|
||||
|
||||
| Priority | Task | Impact |
|
||||
|----------|------|--------|
|
||||
| P0 | Fix `is_nip42_auth_required_for_kind()` leak in config.c:377 | Called on every event |
|
||||
| P0 | Fix `ip_is_whitelisted()` leak in ip_ban.c:255 | Called on every connection check |
|
||||
| P0 | Fix `is_authorized_admin_event()` leaks in main.c:1718,1742 | Called per admin event, leaks in loop |
|
||||
| P1 | Fix websockets.c:1527,1549 auth bypass leaks | Called on every event with auth |
|
||||
| P1 | Fix request_validator.c:304,320 leaks | Called on every event validation |
|
||||
| P1 | Fix nip042.c:99 relay_url leak | Called on every NIP-42 auth |
|
||||
| P2 | Implement config cache for hot-path values | Eliminates leak class entirely |
|
||||
| P2 | Fix inactive subscription cleanup in CLOSED handler | Prevents slow subscription leak |
|
||||
| P3 | Add memory monitoring endpoint | Ongoing visibility |
|
||||
| P3 | Move large stack arrays to heap | Reduces stack pressure |
|
||||
@@ -0,0 +1,201 @@
|
||||
# Multi-Admin Support Plan
|
||||
|
||||
## Overview
|
||||
|
||||
Enable multiple Nostr pubkeys to act as relay administrators. Currently the relay stores a single `admin_pubkey` string in the `config` table and uses `strcmp()` equality checks throughout the codebase. This plan converts that to a comma-separated list of pubkeys stored in the same config key, with a centralized `is_admin_pubkey()` helper that all authorization gates call.
|
||||
|
||||
## Design Principles
|
||||
|
||||
1. **Minimal schema change** — keep using the existing `config` table with key `admin_pubkey`. The value changes from a single 64-char hex string to a comma-separated list of 64-char hex strings.
|
||||
2. **Primary admin concept** — the first pubkey in the list is the "primary admin" (the original one). Only the primary admin can add/remove other admins. This prevents privilege escalation.
|
||||
3. **Centralized check** — introduce one function `is_admin_pubkey(const char* pubkey)` that all 8 authorization gates call instead of inline `strcmp()`.
|
||||
4. **In-memory cache** — parse the comma-separated list once on startup/config-change into a static array for O(1)-ish lookups without repeated string parsing.
|
||||
5. **Backward compatible** — a single pubkey with no commas works identically to today.
|
||||
|
||||
## Data Model
|
||||
|
||||
```
|
||||
config table:
|
||||
key = "admin_pubkey"
|
||||
value = "hex1,hex2,hex3" (comma-separated, no spaces)
|
||||
data_type = "string"
|
||||
```
|
||||
|
||||
Maximum admins: 10 (compile-time constant `MAX_ADMIN_PUBKEYS`).
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Config Table: admin_pubkey = hex1,hex2,hex3] --> B[reload_admin_pubkeys_cache]
|
||||
B --> C[Static array: g_admin_pubkeys with count]
|
||||
D[Any authorization check] --> E[is_admin_pubkey - pubkey]
|
||||
E --> C
|
||||
F[CLI: -a flag] --> G[first_time_startup_sequence]
|
||||
G --> A
|
||||
H[Admin command: add_admin / remove_admin] --> I[modify_admin_list]
|
||||
I --> A
|
||||
I --> B
|
||||
```
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Step 1: Add centralized admin check infrastructure in config.c / config.h
|
||||
|
||||
**New constants and types:**
|
||||
```c
|
||||
#define MAX_ADMIN_PUBKEYS 10
|
||||
```
|
||||
|
||||
**New functions in config.h:**
|
||||
```c
|
||||
int is_admin_pubkey(const char* pubkey); // Returns 1 if pubkey is in admin list
|
||||
int get_admin_pubkey_count(void); // Returns number of admins
|
||||
const char* get_primary_admin_pubkey(void); // Returns first admin pubkey
|
||||
int reload_admin_pubkeys_cache(void); // Parse config value into cache
|
||||
int add_admin_pubkey(const char* pubkey); // Add pubkey to admin list (primary admin only)
|
||||
int remove_admin_pubkey(const char* pubkey); // Remove pubkey from admin list (primary admin only)
|
||||
```
|
||||
|
||||
**Implementation in config.c:**
|
||||
- Static array `g_admin_pubkeys[MAX_ADMIN_PUBKEYS][65]` and `g_admin_pubkey_count`
|
||||
- `reload_admin_pubkeys_cache()` reads `admin_pubkey` from config table, splits on commas, validates each is 64 hex chars, populates array
|
||||
- `is_admin_pubkey()` iterates the cached array with `strcmp()`
|
||||
- `get_primary_admin_pubkey()` returns `g_admin_pubkeys[0]` — used where only the primary admin matters (WoT sync, etc.)
|
||||
- `add_admin_pubkey()` / `remove_admin_pubkey()` modify the comma-separated string in the config table and call `reload_admin_pubkeys_cache()`
|
||||
|
||||
### Step 2: Replace all 8 authorization strcmp gates
|
||||
|
||||
Each of these currently does:
|
||||
```c
|
||||
const char* admin_pubkey = get_config_value("admin_pubkey");
|
||||
if (strcmp(sender_pubkey, admin_pubkey) == 0) { /* authorized */ }
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```c
|
||||
if (is_admin_pubkey(sender_pubkey)) { /* authorized */ }
|
||||
```
|
||||
|
||||
**Files and locations:**
|
||||
|
||||
| File | Function | Line | Change |
|
||||
|------|----------|------|--------|
|
||||
| src/config.c | `process_admin_config_event()` | ~1441 | Replace strcmp with `is_admin_pubkey()` |
|
||||
| src/config.c | `handle_kind_23456_unified()` | ~2727 | Replace strcmp with `is_admin_pubkey()` |
|
||||
| src/main.c | `verify_admin_event()` | ~1905 | Replace strcmp with `is_admin_pubkey()` |
|
||||
| src/main.c | Event storage / WoT trigger | ~1174 | Replace strcmp with `is_admin_pubkey()` |
|
||||
| src/websockets.c | NIP-42 auth bypass | ~2324 | Replace strcmp with `is_admin_pubkey()` |
|
||||
| src/websockets.c | NIP-17 DM admin check | ~3560 | Replace strcmp with `is_admin_pubkey()` |
|
||||
| src/dm_admin.c | `process_admin_dm()` | ~503 | Replace strcmp with `is_admin_pubkey()` |
|
||||
| src/request_validator.c | Event validation bypass | ~307 | Replace strcmp with `is_admin_pubkey()` |
|
||||
|
||||
Each replacement also eliminates the `get_config_value("admin_pubkey")` call and its associated `free()`, simplifying the code.
|
||||
|
||||
### Step 3: Update startup / initialization pipeline
|
||||
|
||||
**config.c `first_time_startup_sequence()`:**
|
||||
- No change needed — it already stores a single pubkey. The comma-separated format with one entry is identical to the current format.
|
||||
|
||||
**config.c `populate_all_config_values_atomic()`:**
|
||||
- No change needed — inserts the initial admin_pubkey as a single value.
|
||||
|
||||
**config.c `add_pubkeys_to_config_table()`:**
|
||||
- No change needed — stores single pubkey on migration.
|
||||
|
||||
**config.c `apply_cli_overrides_atomic()`:**
|
||||
- No change needed — CLI override replaces the entire admin_pubkey value.
|
||||
|
||||
**main.c startup validation:**
|
||||
- Update the pre-warm validation at ~line 2438 to call `reload_admin_pubkeys_cache()` and validate that at least one admin pubkey exists, rather than checking for exactly 64 chars.
|
||||
|
||||
**Call `reload_admin_pubkeys_cache()`** after:
|
||||
- `populate_all_config_values_atomic()` completes
|
||||
- `reload_config_from_table()` completes
|
||||
- Any admin add/remove operation
|
||||
|
||||
### Step 4: Add admin management commands to kind 23456 API
|
||||
|
||||
Add two new admin commands that can be sent as encrypted kind 23456 events:
|
||||
|
||||
```json
|
||||
["add_admin", "pubkey_hex"]
|
||||
["remove_admin", "pubkey_hex"]
|
||||
["list_admins"]
|
||||
```
|
||||
|
||||
**Authorization rule:** Only the primary admin (first in list) can add/remove other admins. This is enforced in the command handler, not in the general `is_admin_pubkey()` check.
|
||||
|
||||
**Implementation location:** `handle_system_command_unified()` in config.c, alongside existing commands like `system_status`, `graceful_shutdown`, etc.
|
||||
|
||||
### Step 5: Update WoT sync to handle multiple admins
|
||||
|
||||
In `wot_sync_from_admin_kind3()`:
|
||||
- Currently queries kind 3 events from the single admin pubkey
|
||||
- Change to use `get_primary_admin_pubkey()` — WoT trust anchor should remain the primary admin
|
||||
- This is a deliberate design choice: secondary admins can manage the relay but the WoT trust graph is rooted in the primary admin's social graph
|
||||
|
||||
### Step 6: Update CLI to accept multiple admin pubkeys
|
||||
|
||||
**Option A - Comma-separated single flag (recommended):**
|
||||
```bash
|
||||
./c_relay_pg -a npub1...,npub2...,npub3...
|
||||
```
|
||||
|
||||
**Changes:**
|
||||
- `cli_options_t.admin_pubkey_override` becomes `char admin_pubkeys_override[MAX_ADMIN_PUBKEYS * 65]` (larger buffer for comma-separated list)
|
||||
- CLI parsing in main.c splits on commas, decodes each npub/hex, reassembles as comma-separated hex
|
||||
- `make_and_restart_relay.sh` `-a` flag documentation updated
|
||||
|
||||
### Step 7: Update config validation in api.c
|
||||
|
||||
In the config validation table at line ~912:
|
||||
```c
|
||||
{"admin_pubkey", "string", 0, 65},
|
||||
```
|
||||
Change max length to accommodate multiple pubkeys:
|
||||
```c
|
||||
{"admin_pubkey", "string", 0, MAX_ADMIN_PUBKEYS * 65},
|
||||
```
|
||||
|
||||
### Step 8: Update get_admin_pubkey_cached
|
||||
|
||||
`get_admin_pubkey_cached()` currently returns the raw config value. Update to return only the primary admin pubkey for backward compatibility with any callers that expect a single 64-char string.
|
||||
|
||||
### Step 9: Update test scripts
|
||||
|
||||
Update test scripts that hardcode admin keys to work with the multi-admin system:
|
||||
- `tests/sql_test.sh`
|
||||
- `tests/white_black_test.sh`
|
||||
- `tests/17_nip_test.sh`
|
||||
- `tests/run_all_tests.sh`
|
||||
- `tests/.test_keys.txt`
|
||||
|
||||
Add new test cases for:
|
||||
- Adding a second admin via kind 23456 command
|
||||
- Verifying second admin can send admin commands
|
||||
- Verifying non-admin cannot send admin commands
|
||||
- Removing a secondary admin
|
||||
- Verifying primary admin cannot be removed
|
||||
- Verifying secondary admin cannot add/remove admins
|
||||
|
||||
### Step 10: Update documentation
|
||||
|
||||
- `AGENTS.md` — update admin API event structure section
|
||||
- `docs/configuration_guide.md` — document multi-admin setup
|
||||
- `docs/user_guide.md` — document admin management commands
|
||||
- `API.md` — document new add_admin/remove_admin/list_admins commands
|
||||
- `README.md` — mention multi-admin capability
|
||||
|
||||
## Migration / Backward Compatibility
|
||||
|
||||
- Existing databases with a single `admin_pubkey` value work without any migration — a string with no commas is treated as a single-admin list
|
||||
- The `is_admin_pubkey()` function handles both formats transparently
|
||||
- No schema changes needed — the config table already stores strings of arbitrary length
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Primary admin privilege** — only the first pubkey in the list can add/remove admins, preventing secondary admins from locking out the primary
|
||||
2. **No self-removal** — the primary admin cannot remove themselves (prevents accidental lockout)
|
||||
3. **Max admin limit** — capped at 10 to prevent abuse and keep the linear scan fast
|
||||
4. **Validation** — each pubkey in the list is validated as 64 hex characters before storage
|
||||
@@ -0,0 +1,410 @@
|
||||
# C-Relay-PG: PostgreSQL Implementation Plan
|
||||
|
||||
## Current State Assessment
|
||||
|
||||
### What We Have
|
||||
- **PostgreSQL 18.3** installed and running on localhost:5432
|
||||
- **libpq development headers** available via pkg-config (`/usr/include/postgresql`, `/usr/lib/x86_64-linux-gnu`)
|
||||
- **`db_ops.h`** abstraction layer with 40+ functions already defined
|
||||
- **`db_ops.c`** — single 1,398-line file implementing all functions against SQLite
|
||||
- **`ip_ban.c`** — already uses `db_ops` API (not raw SQLite) for persistence
|
||||
- **`config.c`** — uses `db_ops` API for all database access
|
||||
|
||||
### SQLite Coupling Points (Must Fix)
|
||||
|
||||
| Location | Issue |
|
||||
|----------|-------|
|
||||
| `src/main.c:11` | `#include <sqlite3.h>` |
|
||||
| `src/main.c:51` | `sqlite3* g_db = NULL;` — global handle owned by main.c |
|
||||
| `src/main.c:897` | `if (g_db)` — direct handle check |
|
||||
| `src/main.c:2228` | `if (!g_db)` — direct handle check |
|
||||
| `src/main.c:2358` | `g_db = NULL;` — strict mode nullification |
|
||||
| `src/main.c:743` | Log message references `sqlite3_open()` |
|
||||
| `src/main.c:854-885` | SQLite PRAGMAs (WAL, mmap_size, cache_size) |
|
||||
| `src/main.c:795-824` | SQLite-specific DDL in migration code |
|
||||
| `src/main.c:900` | `PRAGMA wal_checkpoint(TRUNCATE)` |
|
||||
| `src/config.c:4727` | `sqlite_master` query |
|
||||
| `src/config.c:4388` | `INSERT OR REPLACE` SQL |
|
||||
| `src/db_ops.c:13` | `extern sqlite3* g_db;` |
|
||||
| `src/db_ops.c:871` | `strftime('%s', 'now')` — SQLite-specific |
|
||||
| `src/db_ops.c:889,909,956` | `INSERT OR REPLACE` — SQLite-specific |
|
||||
| `src/db_ops.c:1341` | `sqlite_master` — SQLite-specific |
|
||||
| `src/db_ops.c:1387-1395` | `sqlite3_wal_checkpoint_v2()` — SQLite-only |
|
||||
| `src/ip_ban.c:122,205` | `strftime('%s', 'now')` and `INSERT OR REPLACE` in SQL strings |
|
||||
| `src/sql_schema.h` | Entire embedded schema is SQLite DDL |
|
||||
| `src/subscriptions.c:1187-1197` | Commented-out raw SQLite code |
|
||||
|
||||
### What Already Works Through `db_ops`
|
||||
- `ip_ban.c` — uses `db_exec_sql()`, `db_prepare()`, `db_step_stmt()`, etc.
|
||||
- `config.c` — uses `db_get_config_value_dup()`, `db_set_config_value_full()`, etc.
|
||||
- `subscriptions.c` — uses `db_log_subscription_*()` functions
|
||||
- `api.c` — uses `db_execute_readonly_query_json()`, stat helpers
|
||||
- `nip009.c` — uses `db_delete_event_by_id()`, `db_get_event_pubkey()`
|
||||
|
||||
---
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 0: PostgreSQL Setup & Smoke Test
|
||||
|
||||
Create the PostgreSQL role, database, and verify connectivity from C.
|
||||
|
||||
```
|
||||
Tasks:
|
||||
0.1 Create PostgreSQL role "crelay" with password
|
||||
0.2 Create database "crelay" owned by role "crelay"
|
||||
0.3 Write a minimal C test program that connects via libpq and runs SELECT 1
|
||||
0.4 Verify the test compiles and links with: gcc test_pg.c -lpq -o test_pg
|
||||
```
|
||||
|
||||
### Phase 1: Internalize g_db — Move Database Ownership Into db_ops
|
||||
|
||||
**Goal**: Eliminate `extern sqlite3* g_db` from main.c. The `db_ops` module must own its connection handle internally. This is the critical prerequisite — without it, swapping backends is impossible.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Before
|
||||
MAIN[main.c owns sqlite3* g_db] --> DBOPS[db_ops.c uses extern g_db]
|
||||
end
|
||||
subgraph After
|
||||
MAIN2[main.c calls db_init/db_close] --> DBOPS2[db_ops.c owns static g_db internally]
|
||||
end
|
||||
```
|
||||
|
||||
```
|
||||
Tasks:
|
||||
1.1 Move `sqlite3* g_db` from main.c into db_ops.c as a static variable
|
||||
1.2 Move `char g_database_path[512]` from config.c into db_ops.c as static
|
||||
1.3 Remove `#include <sqlite3.h>` from main.c
|
||||
1.4 Replace all `g_db` references in main.c with db_ops API calls:
|
||||
- `if (g_db)` → `if (db_is_available())`
|
||||
- `g_db = NULL` → `db_close()` (or new `db_detach_main()`)
|
||||
1.5 Replace SQLite PRAGMA calls in main.c init_database() with new db_ops functions:
|
||||
- `db_set_journal_mode_wal()`
|
||||
- `db_set_mmap_size(long size)`
|
||||
- `db_set_cache_size(int kb)`
|
||||
1.6 Move schema migration logic from main.c into db_ops (new `db_apply_schema()`)
|
||||
1.7 Replace `sqlite_master` query in config.c:4727 with `db_table_exists()`
|
||||
1.8 Replace `INSERT OR REPLACE` in config.c:4388 with `db_prepare()`/`db_step_stmt()`
|
||||
1.9 Build and run existing test suite to confirm no regressions
|
||||
```
|
||||
|
||||
### Phase 2: Rename db_ops.c → db_ops_sqlite.c, Create Dispatcher
|
||||
|
||||
**Goal**: Split the single `db_ops.c` into a backend-specific file and a thin dispatcher, preparing the architecture for a second backend.
|
||||
|
||||
```
|
||||
New file layout:
|
||||
src/db_ops.h — unchanged public API
|
||||
src/db_ops_sqlite.c — renamed from db_ops.c (all SQLite implementation)
|
||||
src/db_ops_postgres.c — new file (stub initially)
|
||||
src/db_ops.c — thin dispatcher that forwards to active backend
|
||||
```
|
||||
|
||||
```
|
||||
Tasks:
|
||||
2.1 Rename src/db_ops.c → src/db_ops_sqlite.c
|
||||
2.2 Prefix all function names in db_ops_sqlite.c with `sqlite_` (e.g., `sqlite_db_init()`)
|
||||
2.3 Create src/db_ops.c dispatcher with compile-time backend selection:
|
||||
- #ifdef DB_BACKEND_POSTGRES → call postgres_* functions
|
||||
- #else → call sqlite_* functions (default)
|
||||
2.4 Create src/db_ops_postgres.c with stub implementations that return DB_ERROR
|
||||
2.5 Update Makefile with DB_BACKEND variable:
|
||||
- `DB_BACKEND ?= sqlite`
|
||||
- Conditional source file inclusion and -lpq linking
|
||||
2.6 Build with DB_BACKEND=sqlite — verify identical behavior
|
||||
2.7 Build with DB_BACKEND=postgres — verify it compiles (stubs return errors)
|
||||
```
|
||||
|
||||
### Phase 3: PostgreSQL Schema
|
||||
|
||||
**Goal**: Create the PostgreSQL schema file and apply it to the database.
|
||||
|
||||
```
|
||||
Tasks:
|
||||
3.1 Create src/pg_schema.sql with PostgreSQL DDL from the plan:
|
||||
- events table with JSONB tags column
|
||||
- GIN index on tags
|
||||
- config, auth_rules, relay_seckey, subscriptions, ip_bans tables
|
||||
- Materialized views for dashboard
|
||||
3.2 Create src/pg_schema.h — embedded schema as C string (like sql_schema.h)
|
||||
3.3 Add `postgres_db_apply_schema()` function in db_ops_postgres.c
|
||||
3.4 Apply schema to the crelay database and verify with psql
|
||||
```
|
||||
|
||||
### Phase 4: Implement db_ops_postgres.c — Core Functions
|
||||
|
||||
**Goal**: Implement the PostgreSQL backend for all `db_ops.h` functions using libpq.
|
||||
|
||||
This is the largest phase. Functions are grouped by dependency order.
|
||||
|
||||
#### 4A: Connection Management
|
||||
```
|
||||
Tasks:
|
||||
4A.1 Implement postgres_db_init() — PQconnectdb() with connection string
|
||||
4A.2 Implement postgres_db_close() — PQfinish()
|
||||
4A.3 Implement postgres_db_is_available() — PQstatus() check
|
||||
4A.4 Implement postgres_db_last_error() — PQerrorMessage()
|
||||
4A.5 Implement postgres_db_get_database_path() — return connection string
|
||||
4A.6 Implement thread-local connection pool:
|
||||
- postgres_db_set_thread_connection()
|
||||
- postgres_db_clear_thread_connection()
|
||||
- postgres_db_open_worker_connection() — new PQconnectdb per worker
|
||||
- postgres_db_close_worker_connection() — PQfinish per worker
|
||||
```
|
||||
|
||||
#### 4B: Statement API (db_stmt_t wrapper for PGresult)
|
||||
```
|
||||
Tasks:
|
||||
4B.1 Define postgres db_stmt_t struct wrapping PGresult + cursor state
|
||||
4B.2 Implement postgres_db_prepare() — PQprepare() with auto-generated name
|
||||
4B.3 Implement postgres_db_bind_text_param/int_param/int64_param
|
||||
Note: libpq uses $1,$2 params, not ?. Need parameter array approach.
|
||||
4B.4 Implement postgres_db_step_stmt() — PQexecPrepared() + row iteration
|
||||
4B.5 Implement postgres_db_column_text_value/int_value/int64_value/double_value
|
||||
4B.6 Implement postgres_db_reset_stmt() and postgres_db_finalize_stmt()
|
||||
4B.7 Implement postgres_db_exec_sql() — PQexec() for DDL/DML
|
||||
```
|
||||
|
||||
**Key design decision**: libpq doesn't have a step-based cursor like SQLite. Two approaches:
|
||||
1. **PQexecPrepared** returns all rows at once — iterate with row index
|
||||
2. **DECLARE CURSOR / FETCH** for streaming large result sets
|
||||
|
||||
For this relay's workload (most queries return <1000 rows), approach 1 is simpler and sufficient.
|
||||
|
||||
#### 4C: Config & Key Storage
|
||||
```
|
||||
Tasks:
|
||||
4C.1 Implement postgres_db_get_all_config_rows()
|
||||
4C.2 Implement postgres_db_get_config_value_dup()
|
||||
4C.3 Implement postgres_db_set_config_value_full()
|
||||
— Use INSERT ... ON CONFLICT (key) DO UPDATE instead of INSERT OR REPLACE
|
||||
4C.4 Implement postgres_db_update_config_value_only()
|
||||
— Use EXTRACT(EPOCH FROM NOW()) instead of strftime('%s','now')
|
||||
4C.5 Implement postgres_db_upsert_config_value()
|
||||
4C.6 Implement postgres_db_store_relay_private_key_hex()
|
||||
4C.7 Implement postgres_db_get_relay_private_key_hex_dup()
|
||||
4C.8 Implement postgres_db_store_config_event()
|
||||
4C.9 Implement postgres_db_get_config_row_count()
|
||||
```
|
||||
|
||||
#### 4D: Event Storage & Retrieval
|
||||
```
|
||||
Tasks:
|
||||
4D.1 Implement postgres_db_insert_event_with_json()
|
||||
— INSERT ... ON CONFLICT (id) DO NOTHING
|
||||
— Map extended_errcode to DB_CONSTRAINT for duplicates
|
||||
4D.2 Implement postgres_db_event_id_exists()
|
||||
4D.3 Implement postgres_db_retrieve_event_by_id()
|
||||
4D.4 Implement postgres_db_get_event_pubkey()
|
||||
4D.5 Implement postgres_db_get_event_time_bounds()
|
||||
4D.6 Implement postgres_db_get_latest_event_pubkey_for_kind_dup()
|
||||
```
|
||||
|
||||
#### 4E: Event Deletion (NIP-09)
|
||||
```
|
||||
Tasks:
|
||||
4E.1 Implement postgres_db_delete_event_by_id()
|
||||
4E.2 Implement postgres_db_delete_events_by_address()
|
||||
```
|
||||
|
||||
#### 4F: Tag Operations
|
||||
```
|
||||
Tasks:
|
||||
4F.1 Implement postgres_db_store_event_tags_cjson()
|
||||
— Note: With JSONB+GIN, this may become a no-op for PostgreSQL
|
||||
— Tags are already stored in the events.tags JSONB column
|
||||
4F.2 Implement postgres_db_populate_event_tags_from_existing()
|
||||
— No-op for PostgreSQL (no separate event_tags table needed)
|
||||
```
|
||||
|
||||
#### 4G: Auth Rules
|
||||
```
|
||||
Tasks:
|
||||
4G.1 Implement postgres_db_is_pubkey_blacklisted()
|
||||
4G.2 Implement postgres_db_is_hash_blacklisted()
|
||||
4G.3 Implement postgres_db_is_pubkey_whitelisted()
|
||||
4G.4 Implement postgres_db_count_active_whitelist_rules()
|
||||
4G.5 Implement postgres_db_add_auth_rule()
|
||||
4G.6 Implement postgres_db_remove_auth_rule()
|
||||
4G.7 Implement postgres_db_delete_wot_whitelist_rules()
|
||||
4G.8 Implement postgres_db_count_wot_whitelist_rules()
|
||||
```
|
||||
|
||||
#### 4H: Subscription Logging
|
||||
```
|
||||
Tasks:
|
||||
4H.1 Implement postgres_db_log_subscription_created()
|
||||
— Use INSERT ... ON CONFLICT DO UPDATE
|
||||
4H.2 Implement postgres_db_log_subscription_closed()
|
||||
— Use EXTRACT(EPOCH FROM NOW()) for timestamps
|
||||
4H.3 Implement postgres_db_log_subscription_disconnected()
|
||||
4H.4 Implement postgres_db_update_subscription_events_sent()
|
||||
4H.5 Implement postgres_db_cleanup_orphaned_subscriptions()
|
||||
```
|
||||
|
||||
#### 4I: Monitoring & Stats
|
||||
```
|
||||
Tasks:
|
||||
4I.1 Implement postgres_db_get_total_event_count_ll()
|
||||
4I.2 Implement postgres_db_get_event_count_since()
|
||||
4I.3 Implement postgres_db_get_event_kind_distribution_rows()
|
||||
4I.4 Implement postgres_db_get_top_pubkeys_rows()
|
||||
4I.5 Implement postgres_db_get_subscription_details_rows()
|
||||
4I.6 Implement postgres_db_count_with_sql()
|
||||
4I.7 Implement postgres_db_execute_readonly_query_json()
|
||||
— Map PG column types instead of SQLITE_INTEGER/SQLITE_TEXT/etc.
|
||||
```
|
||||
|
||||
#### 4J: Schema & DDL Helpers
|
||||
```
|
||||
Tasks:
|
||||
4J.1 Implement postgres_db_table_exists()
|
||||
— Use information_schema.tables instead of sqlite_master
|
||||
4J.2 Implement postgres_db_get_schema_version_dup()
|
||||
4J.3 Implement postgres_db_wal_checkpoint_passive() — no-op for PostgreSQL
|
||||
4J.4 Implement postgres_db_wal_checkpoint_truncate() — no-op for PostgreSQL
|
||||
```
|
||||
|
||||
### Phase 5: Fix SQLite-Specific SQL in Callers
|
||||
|
||||
**Goal**: Ensure SQL strings passed through `db_prepare()` / `db_exec_sql()` from outside `db_ops` are portable.
|
||||
|
||||
```
|
||||
Tasks:
|
||||
5.1 Audit ip_ban.c SQL strings:
|
||||
— Replace `strftime('%s', 'now')` with `EXTRACT(EPOCH FROM NOW())::BIGINT`
|
||||
or pass timestamp as bind parameter from C (time(NULL))
|
||||
— Replace `INSERT OR REPLACE` with `INSERT ... ON CONFLICT DO UPDATE`
|
||||
5.2 Audit config.c SQL strings:
|
||||
— Replace `sqlite_master` query with db_table_exists()
|
||||
— Replace `INSERT OR REPLACE` with ON CONFLICT syntax
|
||||
5.3 Audit main.c init_database():
|
||||
— Move all PRAGMA calls behind db_ops backend-specific init
|
||||
— Move schema migration behind db_apply_schema()
|
||||
5.4 Strategy decision: use C-side timestamps (time(NULL)) as bind params
|
||||
instead of database-side timestamp functions for portability
|
||||
```
|
||||
|
||||
### Phase 6: CLI Connection String Support
|
||||
|
||||
**Goal**: Accept PostgreSQL connection parameters from the command line.
|
||||
|
||||
```
|
||||
Tasks:
|
||||
6.1 Add --db-host, --db-port, --db-name, --db-user, --db-password CLI options
|
||||
6.2 Add --db-connstring option for full libpq connection string
|
||||
6.3 Construct connection string from individual params if --db-connstring not given
|
||||
6.4 Pass connection string to db_init() (works for both backends)
|
||||
6.5 Update make_and_restart_relay.sh to support PostgreSQL connection params
|
||||
6.6 Update AGENTS.md with new CLI options
|
||||
```
|
||||
|
||||
### Phase 7: Integration Testing
|
||||
|
||||
**Goal**: Verify the PostgreSQL backend passes all existing tests.
|
||||
|
||||
```
|
||||
Tasks:
|
||||
7.1 Build with DB_BACKEND=postgres
|
||||
7.2 Start relay with PostgreSQL connection string
|
||||
7.3 Run tests/run_nip_tests.sh — verify NIP compliance
|
||||
7.4 Run tests/run_all_tests.sh — verify full test suite
|
||||
7.5 Run tests/performance_benchmarks.sh — compare with SQLite baseline
|
||||
7.6 Test first-time startup flow (key generation, schema creation)
|
||||
7.7 Test config event storage and retrieval
|
||||
7.8 Test auth rules (whitelist/blacklist)
|
||||
7.9 Test IP ban persistence across restarts
|
||||
7.10 Test thread pool worker connections with PostgreSQL
|
||||
```
|
||||
|
||||
### Phase 8: Query Builder — JSONB Tag Queries
|
||||
|
||||
**Goal**: Replace the `event_tags` JOIN queries with PostgreSQL JSONB containment queries for tag filtering in REQ handling.
|
||||
|
||||
```
|
||||
Tasks:
|
||||
8.1 Identify where REQ filter → SQL translation happens (websockets.c / subscriptions.c)
|
||||
8.2 Create backend-aware query builder or use #ifdef for tag filter SQL:
|
||||
— SQLite: `id IN (SELECT event_id FROM event_tags WHERE tag_name=? AND tag_value IN (?))`
|
||||
— PostgreSQL: `tags @> '[["p", "value"]]'::jsonb`
|
||||
8.3 Test tag-filtered REQ queries against PostgreSQL
|
||||
8.4 Verify GIN index is being used (EXPLAIN ANALYZE)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## SQL Dialect Mapping Reference
|
||||
|
||||
| SQLite | PostgreSQL |
|
||||
|--------|-----------|
|
||||
| `INSERT OR REPLACE INTO t ...` | `INSERT INTO t ... ON CONFLICT (pk) DO UPDATE SET ...` |
|
||||
| `strftime('%s', 'now')` | `EXTRACT(EPOCH FROM NOW())::BIGINT` |
|
||||
| `SELECT 1 FROM sqlite_master WHERE type='table' AND name=?` | `SELECT 1 FROM information_schema.tables WHERE table_name=?` |
|
||||
| `PRAGMA journal_mode=WAL` | N/A (PostgreSQL handles WAL internally) |
|
||||
| `PRAGMA mmap_size=N` | N/A |
|
||||
| `PRAGMA cache_size=-N` | `SET shared_buffers` (server-level, not per-connection) |
|
||||
| `sqlite3_wal_checkpoint_v2()` | N/A (no-op) |
|
||||
| `sqlite3_changes()` | `PQcmdTuples(result)` |
|
||||
| `sqlite3_extended_errcode()` | `PQresultErrorField(result, PG_DIAG_SQLSTATE)` |
|
||||
| `?` bind parameter | `$1`, `$2`, ... positional parameters |
|
||||
| `JSON` column type | `JSONB` column type |
|
||||
| `json_each()` for tag queries | `@>` containment operator with GIN index |
|
||||
| `INTEGER PRIMARY KEY` auto-increment | `SERIAL` or `BIGSERIAL` |
|
||||
|
||||
---
|
||||
|
||||
## Architecture After Implementation
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph Application Layer
|
||||
MAIN[main.c] --> DBOPS_H[db_ops.h - public API]
|
||||
CONFIG[config.c] --> DBOPS_H
|
||||
IPBAN[ip_ban.c] --> DBOPS_H
|
||||
SUBS[subscriptions.c] --> DBOPS_H
|
||||
API[api.c] --> DBOPS_H
|
||||
NIP09[nip009.c] --> DBOPS_H
|
||||
WS[websockets.c] --> DBOPS_H
|
||||
end
|
||||
|
||||
subgraph Database Abstraction
|
||||
DBOPS_H --> DISPATCH[db_ops.c - dispatcher]
|
||||
DISPATCH -->|DB_BACKEND_SQLITE| SQLITE[db_ops_sqlite.c]
|
||||
DISPATCH -->|DB_BACKEND_POSTGRES| PG[db_ops_postgres.c]
|
||||
end
|
||||
|
||||
subgraph Backends
|
||||
SQLITE --> SQLITE3[libsqlite3]
|
||||
PG --> LIBPQ[libpq]
|
||||
LIBPQ --> PGSERVER[PostgreSQL Server]
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|-----------|
|
||||
| Breaking SQLite backend during refactor | Phase 1-2 are pure refactors — test suite must pass after each |
|
||||
| libpq statement API mismatch | Use PQexecParams with row-index iteration, not cursor-based |
|
||||
| SQL dialect differences in caller code | Phase 5 audits all SQL outside db_ops; use C timestamps for portability |
|
||||
| Connection failures | Implement reconnection with exponential backoff in postgres_db_init |
|
||||
| Thread safety | Each worker thread gets its own PGconn via db_open_worker_connection |
|
||||
| Performance regression | Phase 7 benchmarks compare SQLite vs PostgreSQL on same workload |
|
||||
|
||||
---
|
||||
|
||||
## Build Commands
|
||||
|
||||
```bash
|
||||
# SQLite backend (default, unchanged behavior)
|
||||
make
|
||||
|
||||
# PostgreSQL backend
|
||||
make DB_BACKEND=postgres
|
||||
|
||||
# Run with PostgreSQL
|
||||
./build/c_relay_pg_x86 --db-connstring "host=localhost dbname=crelay user=crelay password=secret"
|
||||
```
|
||||
@@ -0,0 +1,257 @@
|
||||
# PostgreSQL Nostr Relay Architecture Analysis
|
||||
|
||||
This document summarizes common approaches and best practices for building Nostr relays backed by PostgreSQL, based on industry-standard designs and existing open-source relay implementations.
|
||||
|
||||
## Identified Projects
|
||||
* **Nostrss**: High-performance Rust-based relay.
|
||||
* **Nostrgres**: Focused on PostgreSQL integration with complex event filtering.
|
||||
* **Relay-rs (Postgres branch)**: General purpose high-throughput relay.
|
||||
* **Nostream** ([github.com/Cameri/nostream](https://github.com/Cameri/nostream)): Production-ready TypeScript relay backed by PostgreSQL and Redis.
|
||||
* **Nostr-Relay-NestJS** ([github.com/CodyTseng/nostr-relay-nestjs](https://github.com/CodyTseng/nostr-relay-nestjs)): High-performance relay built with NestJS and PostgreSQL, utilizing Kysely for type-safe database interactions.
|
||||
|
||||
## Detailed Analysis
|
||||
|
||||
### 1. Common Schema Patterns
|
||||
* **Event Storage (JSONB)**: Most PostgreSQL-backed relays store the raw event as a JSONB object in a central `events` table. This provides flexibility for the evolving Nostr spec while allowing efficient extraction of fields.
|
||||
* **Tag Denormalization**: While the raw event is in JSONB, performant relays extract tags (like `p`, `e`, `t`, `d`) into a separate `tags` table with foreign key relationships to the `events` table. This avoids slow JSONB traversal during complex filter queries.
|
||||
* **Author/Publisher Tracking**: A dedicated `authors` or `pubkeys` table is typically used to maintain indexing on the `pubkey` field, enabling quick lookups of user activity.
|
||||
|
||||
### 2. Performance Optimization
|
||||
* **GIN Indexes on JSONB**: Crucial for filtering on specific event tags or custom properties stored within the event object. GIN indexes with `jsonb_path_ops` are generally preferred for equality checks.
|
||||
* **Time-Series Partitioning**: Given the append-only nature of Nostr, partitioning the `events` table by time (e.g., daily or weekly chunks) is highly recommended. This significantly improves query performance for recent data and simplifies data expiration/deletion.
|
||||
* **Clustering**: Clustering the `events` table by the timestamp index can reduce disk I/O, as it keeps temporally related events physically adjacent on the disk.
|
||||
|
||||
### 3. Schema Management Approaches
|
||||
* **Migrations**: Most mature relays utilize migration tools (like `Flyway`, `Diesel migrations`, or `Golang-migrate`) to version control database schema changes. This is critical for production stability.
|
||||
* **Auto-generation**: Some lightweight prototypes auto-generate schemas at startup. This is generally discouraged for production due to the risk of destructive changes, data loss, or blocking DDL operations on large tables.
|
||||
|
||||
## Recommendations for Building a New Relay
|
||||
1. **Prioritize Denormalization**: Do not rely solely on JSONB queries for high-traffic filters. Extract indexed tags into dedicated columns or tables.
|
||||
2. **Use Partitioning from Day One**: Implementing native PostgreSQL partitioning (e.g., using `pg_partman`) is much easier before the dataset grows to millions of rows.
|
||||
3. **Connection Pooling**: PostgreSQL requires aggressive connection management. Use a high-performance pooler like `PgBouncer` to handle the large number of concurrent, short-lived connections common in WebSocket-based relay traffic.
|
||||
4. **Asynchronous Writes**: Decouple the WebSocket ingestion thread from the database writer thread to ensure that slow database writes do not impact the relay's responsiveness.
|
||||
|
||||
## Nostream ([github.com/Cameri/nostream](https://github.com/Cameri/nostream))
|
||||
|
||||
A production-ready Nostr relay written in TypeScript (Node.js), backed by PostgreSQL 14+ and Redis. Uses Knex.js for versioned migrations.
|
||||
|
||||
### PostgreSQL Schema
|
||||
|
||||
The schema is fully normalized — no JSONB blob for the whole event. Tags are stored separately and populated by a trigger.
|
||||
|
||||
**`events` table** — one row per event, tags kept as a JSONB column for the trigger to process:
|
||||
```sql
|
||||
CREATE TABLE events (
|
||||
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
|
||||
event_id BYTEA UNIQUE NOT NULL, -- 32-byte hash
|
||||
event_pubkey BYTEA NOT NULL,
|
||||
event_kind INTEGER UNSIGNED NOT NULL,
|
||||
event_created_at INTEGER UNSIGNED NOT NULL,
|
||||
event_content TEXT NOT NULL,
|
||||
event_tags JSONB, -- raw tags array, source of truth for trigger
|
||||
event_signature BYTEA NOT NULL,
|
||||
remote_address TEXT,
|
||||
expires_at INTEGER,
|
||||
deleted_at TIMESTAMP,
|
||||
event_deduplication JSONB, -- used for replaceable event conflict key
|
||||
first_seen TIMESTAMP DEFAULT now()
|
||||
);
|
||||
|
||||
-- Indexes
|
||||
CREATE INDEX ON events (event_pubkey);
|
||||
CREATE INDEX ON events (event_kind);
|
||||
CREATE INDEX ON events (event_created_at);
|
||||
CREATE INDEX ON events (expires_at);
|
||||
CREATE INDEX event_tags_idx ON events USING GIN (event_tags); -- GIN on raw JSONB
|
||||
|
||||
-- Unique index enforcing NIP-16 replaceable event semantics
|
||||
CREATE UNIQUE INDEX replaceable_events_idx ON events (event_pubkey, event_kind)
|
||||
WHERE event_kind = 0 OR event_kind = 3
|
||||
OR (event_kind >= 10000 AND event_kind < 20000);
|
||||
```
|
||||
|
||||
**`event_tags` table** — denormalized single-letter tags, populated by a PostgreSQL trigger on `events`:
|
||||
```sql
|
||||
CREATE TABLE event_tags (
|
||||
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
|
||||
event_id BYTEA NOT NULL,
|
||||
tag_name TEXT NOT NULL, -- single-letter only (e.g. 'e', 'p', 't')
|
||||
tag_value TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX ON event_tags (event_id);
|
||||
CREATE INDEX ON event_tags (tag_name, tag_value);
|
||||
|
||||
-- Trigger: on INSERT/UPDATE/DELETE to events, repopulate event_tags
|
||||
CREATE OR REPLACE FUNCTION process_event_tags() RETURNS TRIGGER AS $$
|
||||
DECLARE
|
||||
tag_element jsonb;
|
||||
BEGIN
|
||||
DELETE FROM event_tags WHERE event_id = OLD.event_id;
|
||||
IF TG_OP = 'INSERT' OR TG_OP = 'UPDATE' THEN
|
||||
FOR tag_element IN SELECT jsonb_array_elements(NEW.event_tags) LOOP
|
||||
IF length(trim('"' FROM (tag_element->0)::text)) = 1
|
||||
AND (tag_element->1)::text IS NOT NULL THEN
|
||||
INSERT INTO event_tags (event_id, tag_name, tag_value)
|
||||
VALUES (NEW.event_id,
|
||||
trim('"' FROM (tag_element->0)::text),
|
||||
trim('"' FROM (tag_element->1)::text));
|
||||
END IF;
|
||||
END LOOP;
|
||||
END IF;
|
||||
RETURN NEW;
|
||||
END;
|
||||
$$ LANGUAGE plpgsql;
|
||||
|
||||
CREATE TRIGGER insert_event_tags
|
||||
AFTER INSERT OR UPDATE OR DELETE ON events
|
||||
FOR EACH ROW EXECUTE FUNCTION process_event_tags();
|
||||
```
|
||||
|
||||
Tag queries (`#e`, `#p`, etc.) join `event_tags` rather than traversing the JSONB array.
|
||||
|
||||
**`users` and `invoices` tables** — for paid relay admission (Lightning payments):
|
||||
```sql
|
||||
CREATE TABLE users (
|
||||
pubkey BYTEA PRIMARY KEY,
|
||||
is_admitted BOOLEAN DEFAULT FALSE,
|
||||
balance BIGINT DEFAULT 0, -- in msats
|
||||
tos_accepted_at DATETIME
|
||||
);
|
||||
|
||||
CREATE TABLE invoices (
|
||||
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
|
||||
pubkey BYTEA NOT NULL,
|
||||
bolt11 TEXT NOT NULL,
|
||||
amount_requested BIGINT UNSIGNED NOT NULL,
|
||||
amount_paid BIGINT UNSIGNED,
|
||||
unit ENUM('msats','sats','btc'),
|
||||
status ENUM('pending','completed','expired'),
|
||||
description TEXT,
|
||||
confirmed_at DATETIME,
|
||||
expires_at DATETIME
|
||||
);
|
||||
```
|
||||
|
||||
### How Redis Is Used
|
||||
|
||||
Redis is **not** used for subscription fan-out. Subscription delivery is handled entirely in-process: each `WebSocketAdapter` holds a `Map<subscriptionId, filters[]>`, and a broadcast event walks all connected clients and filters locally.
|
||||
|
||||
Redis has two actual roles:
|
||||
|
||||
1. **Sliding-window rate limiting** — The `SlidingWindowRateLimiter` uses Redis sorted sets to track request timestamps per key (pubkey, IP, etc.) within a rolling time window:
|
||||
- `ZREMRANGEBYSCORE key 0 (now - period)` — evict old entries
|
||||
- `ZADD key timestamp member` — record this hit
|
||||
- `ZRANGE key 0 -1` — count hits in window
|
||||
- `EXPIRE key period` — auto-cleanup
|
||||
- Keys follow the pattern `<pubkey>:events:<period>` or `<ip>:message:<period>`
|
||||
|
||||
2. **General-purpose cache** — A `RedisAdapter` wraps the client with `get`/`set`/`hasKey` helpers, used for caching admission checks (e.g. `<pubkey>:is-admitted`). The event handler has a `TODO` comment noting that the `userRepository.findByPubkey()` call should be replaced with a cache lookup — meaning this is partially implemented.
|
||||
|
||||
---
|
||||
|
||||
## Schema Definitions
|
||||
|
||||
### Relay-rs (Postgres)
|
||||
```sql
|
||||
-- Events table
|
||||
CREATE TABLE "event" (
|
||||
id bytea NOT NULL,
|
||||
pub_key bytea NOT NULL,
|
||||
created_at timestamp with time zone NOT NULL,
|
||||
kind integer NOT NULL,
|
||||
"content" bytea NOT NULL,
|
||||
hidden bit(1) NOT NULL DEFAULT 0::bit(1),
|
||||
delegated_by bytea NULL,
|
||||
first_seen timestamp with time zone NOT NULL DEFAULT now(),
|
||||
expires_at timestamp(0) with time zone,
|
||||
CONSTRAINT event_pkey PRIMARY KEY (id)
|
||||
);
|
||||
CREATE INDEX event_created_at_idx ON "event" (created_at,kind);
|
||||
CREATE INDEX event_pub_key_idx ON "event" (pub_key);
|
||||
CREATE INDEX event_delegated_by_idx ON "event" (delegated_by);
|
||||
CREATE INDEX event_expires_at_idx ON "event" (expires_at);
|
||||
|
||||
-- Tags table
|
||||
CREATE TABLE "tag" (
|
||||
id int8 NOT NULL GENERATED BY DEFAULT AS IDENTITY,
|
||||
event_id bytea NOT NULL,
|
||||
"name" varchar NOT NULL,
|
||||
value bytea NULL,
|
||||
value_hex bytea NULL,
|
||||
CONSTRAINT tag_fk FOREIGN KEY (event_id) REFERENCES "event"(id) ON DELETE CASCADE,
|
||||
CONSTRAINT unique_constraint_name UNIQUE (event_id, "name", value, value_hex)
|
||||
);
|
||||
CREATE INDEX tag_event_id_idx ON tag USING btree (event_id, name);
|
||||
CREATE INDEX tag_value_idx ON tag USING btree (value);
|
||||
CREATE INDEX tag_value_hex_idx ON tag USING btree (value_hex);
|
||||
|
||||
-- Account table
|
||||
CREATE TABLE "account" (
|
||||
pubkey varchar NOT NULL,
|
||||
is_admitted BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
balance BIGINT NOT NULL DEFAULT 0,
|
||||
tos_accepted_at TIMESTAMP,
|
||||
CONSTRAINT account_pkey PRIMARY KEY (pubkey)
|
||||
);
|
||||
|
||||
-- Invoice table
|
||||
CREATE TYPE status AS ENUM ('Paid', 'Unpaid', 'Expired');
|
||||
CREATE TABLE "invoice" (
|
||||
payment_hash varchar NOT NULL,
|
||||
pubkey varchar NOT NULL,
|
||||
invoice varchar NOT NULL,
|
||||
amount BIGINT NOT NULL,
|
||||
status status NOT NULL DEFAULT 'Unpaid',
|
||||
description varchar,
|
||||
created_at timestamp,
|
||||
confirmed_at timestamp,
|
||||
CONSTRAINT invoice_payment_hash PRIMARY KEY (payment_hash),
|
||||
CONSTRAINT invoice_pubkey_fkey FOREIGN KEY (pubkey) REFERENCES account (pubkey) ON DELETE CASCADE
|
||||
);
|
||||
```
|
||||
|
||||
### Nostr-Relay-NestJS (PostgreSQL via Kysely)
|
||||
```sql
|
||||
-- Events table
|
||||
CREATE TABLE events (
|
||||
id CHAR(64) PRIMARY KEY,
|
||||
pubkey CHAR(64) NOT NULL,
|
||||
created_at BIGINT NOT NULL,
|
||||
kind INTEGER NOT NULL,
|
||||
tags JSONB NOT NULL DEFAULT '[]',
|
||||
generic_tags TEXT[] NOT NULL DEFAULT '{}',
|
||||
content TEXT NOT NULL DEFAULT '',
|
||||
sig CHAR(128) NOT NULL,
|
||||
expired_at BIGINT,
|
||||
d_tag_value TEXT,
|
||||
delegator CHAR(64),
|
||||
create_date TIMESTAMP NOT NULL DEFAULT now(),
|
||||
update_date TIMESTAMP NOT NULL DEFAULT now(),
|
||||
delete_date TIMESTAMP
|
||||
);
|
||||
|
||||
-- Indices
|
||||
CREATE EXTENSION IF NOT EXISTS btree_gin;
|
||||
CREATE INDEX generic_tags_kind_idx ON events USING gin (generic_tags, kind);
|
||||
CREATE INDEX pubkey_kind_created_at_idx ON events (pubkey, kind, created_at);
|
||||
CREATE INDEX delegator_kind_created_at_idx ON events (delegator, kind, created_at);
|
||||
CREATE INDEX created_at_kind_idx ON events (created_at, kind);
|
||||
|
||||
-- Generic tags table (for optimized tag queries)
|
||||
CREATE TABLE generic_tags (
|
||||
id SERIAL PRIMARY KEY,
|
||||
tag TEXT NOT NULL,
|
||||
author CHAR(64) NOT NULL,
|
||||
kind INTEGER NOT NULL,
|
||||
event_id CHAR(64) NOT NULL REFERENCES events(id),
|
||||
created_at BIGINT NOT NULL
|
||||
);
|
||||
|
||||
-- Indices for tag filtering
|
||||
CREATE UNIQUE INDEX g_tag_event_id_idx ON generic_tags (tag, event_id);
|
||||
CREATE INDEX g_tag_kind_created_at_idx ON generic_tags (tag, kind, created_at);
|
||||
CREATE INDEX g_tag_created_at_idx ON generic_tags (tag, created_at);
|
||||
```
|
||||
Nostr-Relay-NestJS features a hybrid storage model where tags are stored both in a JSONB field (for general lookup) and a normalized `generic_tags` table (for optimized tag query performance). The schema uses Kysely for type-safe interaction.
|
||||
|
||||
@@ -0,0 +1,342 @@
|
||||
# Database Query Performance Analysis Report
|
||||
|
||||
**Analysis Date:** 2026-02-02
|
||||
**Log Duration:** ~6 minutes (15:24:50 - 15:30:58)
|
||||
**Total Queries:** 366 queries
|
||||
**Data Source:** serverlog.txt
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The relay is experiencing moderate performance issues with an average query time of **10.4ms** and a maximum query time of **672ms**. The primary bottlenecks are:
|
||||
|
||||
1. **Tag-based searches using `json_each()`** - 53% of all queries (194/366)
|
||||
2. **Monitoring system queries** - Taking 540-550ms each
|
||||
3. **Multiple pubkey lookups** - Kind 10002 queries with 15-50 pubkeys
|
||||
|
||||
---
|
||||
|
||||
## Query Performance Metrics
|
||||
|
||||
### Overall Statistics
|
||||
- **Total Queries:** 366
|
||||
- **Average Query Time:** 10,440 μs (10.4 ms)
|
||||
- **Minimum Query Time:** 14 μs
|
||||
- **Maximum Query Time:** 672,846 μs (672.8 ms)
|
||||
- **Slow Queries (>10ms):** 8 queries (2.2%)
|
||||
|
||||
### Query Type Breakdown
|
||||
| Type | Count | Percentage |
|
||||
|------|-------|------------|
|
||||
| REQ | 359 | 98.1% |
|
||||
| MONITOR | 7 | 1.9% |
|
||||
|
||||
---
|
||||
|
||||
## Critical Performance Issues
|
||||
|
||||
### 1. **SLOWEST QUERY: 672ms Tag Search (IP: 192.42.116.178)**
|
||||
|
||||
```sql
|
||||
SELECT id, pubkey, created_at, kind, content, sig, tags
|
||||
FROM events
|
||||
WHERE 1=1
|
||||
AND (kind < 20000 OR kind >= 30000)
|
||||
AND kind IN (5,6300,7000,2004,1622)
|
||||
AND EXISTS (
|
||||
SELECT 1 FROM json_each(json(tags))
|
||||
WHERE json_extract(value, '$[0]') = ?
|
||||
AND json_extract(value, '$[1]') IN (?)
|
||||
)
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 100
|
||||
```
|
||||
|
||||
**Problem:** Full table scan with JSON parsing for every row
|
||||
**Impact:** 672ms for 0 results (wasted computation)
|
||||
**Root Cause:** No index on tag values, requires scanning all events
|
||||
|
||||
---
|
||||
|
||||
### 2. **Monitoring System Queries: 540-550ms Each**
|
||||
|
||||
```sql
|
||||
SELECT * FROM active_subscriptions_log
|
||||
ORDER BY created_at DESC
|
||||
```
|
||||
|
||||
**Occurrences:** 4 queries in 6 minutes
|
||||
**Average Time:** 545ms
|
||||
**Rows Returned:** 20-52 rows
|
||||
**Problem:** Extremely slow for small result sets
|
||||
**Root Cause:** Likely missing index on `created_at` column
|
||||
|
||||
---
|
||||
|
||||
### 3. **Tag-Based Searches (json_each) - 53% of All Queries**
|
||||
|
||||
- **Total:** 194 queries (53% of all queries)
|
||||
- **Pattern:** `EXISTS (SELECT 1 FROM json_each(json(tags)) WHERE ...)`
|
||||
- **Most Common:** Kind 1984 (105 queries), Kind 1111 (47 queries)
|
||||
- **Problem:** Every tag search requires full JSON parsing
|
||||
|
||||
**Example Pattern:**
|
||||
```sql
|
||||
WHERE kind IN (1984)
|
||||
AND pubkey IN (?)
|
||||
AND EXISTS (
|
||||
SELECT 1 FROM json_each(json(tags))
|
||||
WHERE json_extract(value, '$[0]') = ?
|
||||
AND json_extract(value, '$[1]') IN (?,?,?,?,?,?,?,?,?,?,?)
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4. **Multiple Pubkey Lookups (Kind 10002)**
|
||||
|
||||
- **Total:** 64 queries for kind 10002
|
||||
- **Average Time:** 2,500-3,300 μs (2.5-3.3ms)
|
||||
- **Pattern:** Queries with 15-50 pubkeys in `IN` clause
|
||||
- **Problem:** Large `IN` clauses without proper indexing
|
||||
|
||||
**Example:**
|
||||
```sql
|
||||
WHERE kind IN (10002)
|
||||
AND pubkey IN (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Client Activity Analysis
|
||||
|
||||
### Top Query Sources (by IP)
|
||||
| IP Address | Query Count | Percentage | Notes |
|
||||
|------------|-------------|------------|-------|
|
||||
| 45.84.107.222 | 101 | 27.6% | **Highest activity** |
|
||||
| 23.234.109.54 | 69 | 18.9% | Second highest |
|
||||
| 185.220.101.38 | 56 | 15.3% | Third highest |
|
||||
| 192.42.116.178 | 51 | 13.9% | **Source of 672ms query** |
|
||||
| 149.22.80.85 | 34 | 9.3% | |
|
||||
| 174.138.53.241 | 24 | 6.6% | |
|
||||
| Others | 31 | 8.5% | 6 other IPs |
|
||||
|
||||
**Observation:** Top 3 IPs account for 61.8% of all queries
|
||||
|
||||
---
|
||||
|
||||
## Most Common Query Patterns
|
||||
|
||||
| Kind Filter | Query Count | Percentage |
|
||||
|-------------|-------------|------------|
|
||||
| kind IN (1984) | 105 | 28.7% |
|
||||
| kind IN (10002) | 64 | 17.5% |
|
||||
| kind IN (1111) | 47 | 12.8% |
|
||||
| kind IN (0,2,3,10002) | 24 | 6.6% |
|
||||
| kind IN (9735) | 23 | 6.3% |
|
||||
| kind IN (0,30315,10002,10050) | 20 | 5.5% |
|
||||
| Others | 83 | 22.7% |
|
||||
|
||||
---
|
||||
|
||||
## Optimization Recommendations
|
||||
|
||||
### Priority 1: Critical (Immediate Action Required)
|
||||
|
||||
#### 1.1 Add Index on `active_subscriptions_log.created_at`
|
||||
**Impact:** Will reduce monitoring queries from 540ms to <10ms
|
||||
**Effort:** Low
|
||||
**SQL:**
|
||||
```sql
|
||||
CREATE INDEX IF NOT EXISTS idx_active_subscriptions_created_at
|
||||
ON active_subscriptions_log(created_at DESC);
|
||||
```
|
||||
|
||||
#### 1.2 Implement Tag Indexing System
|
||||
**Impact:** Will reduce tag searches from 100-600ms to <10ms
|
||||
**Effort:** High
|
||||
**Options:**
|
||||
- **Option A:** Create separate `event_tags` table with indexes
|
||||
```sql
|
||||
CREATE TABLE event_tags (
|
||||
event_id TEXT NOT NULL,
|
||||
tag_name TEXT NOT NULL,
|
||||
tag_value TEXT NOT NULL,
|
||||
FOREIGN KEY (event_id) REFERENCES events(id)
|
||||
);
|
||||
CREATE INDEX idx_event_tags_lookup ON event_tags(tag_name, tag_value);
|
||||
```
|
||||
- **Option B:** Use SQLite JSON1 extension with generated columns (if available)
|
||||
- **Option C:** Implement application-level tag caching
|
||||
|
||||
**Recommended:** Option A (most reliable and performant)
|
||||
|
||||
---
|
||||
|
||||
### Priority 2: High (Implement Within Week)
|
||||
|
||||
#### 2.1 Optimize Multiple Pubkey Queries
|
||||
**Current:** `pubkey IN (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)`
|
||||
**Problem:** Large IN clauses are inefficient
|
||||
**Solution:**
|
||||
- Add composite index: `CREATE INDEX idx_events_kind_pubkey ON events(kind, pubkey, created_at DESC);`
|
||||
- Consider query rewriting for >10 pubkeys
|
||||
|
||||
#### 2.2 Add Query Result Caching
|
||||
**Target Queries:**
|
||||
- Kind 0 (profile) lookups - frequently repeated
|
||||
- Kind 10002 (relay lists) - relatively static
|
||||
- Kind 1984 (reports) - can be cached for 30-60 seconds
|
||||
|
||||
**Implementation:**
|
||||
- Use in-memory LRU cache (e.g., 1000 entries)
|
||||
- Cache key: hash of SQL + parameters
|
||||
- TTL: 30-60 seconds for most queries
|
||||
|
||||
---
|
||||
|
||||
### Priority 3: Medium (Optimize Over Time)
|
||||
|
||||
#### 3.1 Disable or Throttle Monitoring Queries
|
||||
**Current:** Monitoring queries run every ~60 seconds
|
||||
**Impact:** Each query takes 540ms
|
||||
**Options:**
|
||||
- Increase throttle to 300 seconds (5 minutes)
|
||||
- Disable monitoring entirely if not actively used
|
||||
- Optimize `active_subscriptions_log` table structure
|
||||
|
||||
#### 3.2 Implement Query Complexity Limits
|
||||
**Problem:** Some queries scan entire table (e.g., 672ms query returned 0 results)
|
||||
**Solution:**
|
||||
- Reject queries without time bounds (require `since` or `until`)
|
||||
- Limit number of kinds in single query (max 10)
|
||||
- Limit number of pubkeys in single query (max 20)
|
||||
|
||||
#### 3.3 Add Composite Indexes for Common Patterns
|
||||
```sql
|
||||
-- For kind + created_at queries (most common pattern)
|
||||
CREATE INDEX idx_events_kind_created ON events(kind, created_at DESC);
|
||||
|
||||
-- For kind + pubkey + created_at queries
|
||||
CREATE INDEX idx_events_kind_pubkey_created ON events(kind, pubkey, created_at DESC);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Database Schema Recommendations
|
||||
|
||||
### Current Issues
|
||||
1. **No tag indexing** - Forces full table scans with JSON parsing
|
||||
2. **Missing created_at indexes** - Monitoring queries are extremely slow
|
||||
3. **No composite indexes** - Multi-condition queries inefficient
|
||||
|
||||
### Recommended Schema Changes
|
||||
|
||||
```sql
|
||||
-- 1. Add tag indexing table
|
||||
CREATE TABLE IF NOT EXISTS event_tags (
|
||||
event_id TEXT NOT NULL,
|
||||
tag_name TEXT NOT NULL,
|
||||
tag_value TEXT NOT NULL,
|
||||
FOREIGN KEY (event_id) REFERENCES events(id) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
CREATE INDEX idx_event_tags_lookup ON event_tags(tag_name, tag_value);
|
||||
CREATE INDEX idx_event_tags_event_id ON event_tags(event_id);
|
||||
|
||||
-- 2. Add monitoring table index
|
||||
CREATE INDEX IF NOT EXISTS idx_active_subscriptions_created_at
|
||||
ON active_subscriptions_log(created_at DESC);
|
||||
|
||||
-- 3. Add composite indexes for common query patterns
|
||||
CREATE INDEX IF NOT EXISTS idx_events_kind_created
|
||||
ON events(kind, created_at DESC);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_events_kind_pubkey_created
|
||||
ON events(kind, pubkey, created_at DESC);
|
||||
|
||||
-- 4. Add index for pubkey lookups
|
||||
CREATE INDEX IF NOT EXISTS idx_events_pubkey_created
|
||||
ON events(pubkey, created_at DESC);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Expected Performance Improvements
|
||||
|
||||
| Optimization | Current Avg | Expected Avg | Improvement |
|
||||
|--------------|-------------|--------------|-------------|
|
||||
| Tag searches (with event_tags table) | 100-600ms | 5-20ms | **95-97%** |
|
||||
| Monitoring queries (with index) | 540ms | 5-10ms | **98%** |
|
||||
| Multiple pubkey queries (with index) | 2.5-3.3ms | 0.5-1ms | **70-80%** |
|
||||
| Overall average query time | 10.4ms | 2-3ms | **70-80%** |
|
||||
|
||||
---
|
||||
|
||||
## Client Behavior Analysis
|
||||
|
||||
### Potentially Abusive Patterns
|
||||
|
||||
#### IP: 192.42.116.178 (51 queries)
|
||||
- **Issue:** Generated the slowest query (672ms)
|
||||
- **Pattern:** Complex tag searches with multiple kinds
|
||||
- **Recommendation:** Monitor for repeated expensive queries
|
||||
|
||||
#### IP: 45.84.107.222 (101 queries - 27.6% of total)
|
||||
- **Issue:** Highest query volume
|
||||
- **Pattern:** Mix of kind 10002, 1984, and various other kinds
|
||||
- **Recommendation:** Acceptable if queries are efficient; monitor for abuse
|
||||
|
||||
### Normal Behavior
|
||||
- Most queries are <1ms (fast)
|
||||
- Majority return 0-10 rows (reasonable)
|
||||
- Query patterns match typical Nostr client behavior
|
||||
|
||||
---
|
||||
|
||||
## Action Plan
|
||||
|
||||
### Immediate (Today)
|
||||
1. ✅ Add index on `active_subscriptions_log.created_at`
|
||||
2. ✅ Increase monitoring throttle from 60s to 300s (or disable)
|
||||
3. ✅ Monitor IP 192.42.116.178 for repeated expensive queries
|
||||
|
||||
### This Week
|
||||
1. ⏳ Design and implement `event_tags` table
|
||||
2. ⏳ Add composite indexes for common query patterns
|
||||
3. ⏳ Implement query complexity limits (require time bounds)
|
||||
|
||||
### This Month
|
||||
1. ⏳ Implement query result caching (LRU cache)
|
||||
2. ⏳ Add query cost estimation and rejection
|
||||
3. ⏳ Optimize subscription matching algorithm
|
||||
|
||||
---
|
||||
|
||||
## Monitoring Recommendations
|
||||
|
||||
### Key Metrics to Track
|
||||
1. **Average query time** - Target: <5ms
|
||||
2. **P95 query time** - Target: <50ms
|
||||
3. **P99 query time** - Target: <100ms
|
||||
4. **Queries >100ms** - Target: <1% of queries
|
||||
5. **Tag search percentage** - Target: <30% after optimization
|
||||
|
||||
### Alert Thresholds
|
||||
- **Critical:** Average query time >20ms for 5 minutes
|
||||
- **Warning:** Any single query >1000ms (1 second)
|
||||
- **Info:** Client making >100 queries/minute
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
The relay is experiencing performance issues primarily due to:
|
||||
1. **Lack of tag indexing** (53% of queries affected)
|
||||
2. **Missing indexes on monitoring tables** (540ms queries)
|
||||
3. **Inefficient multiple pubkey lookups**
|
||||
|
||||
Implementing the recommended optimizations will reduce average query time from **10.4ms to 2-3ms** (70-80% improvement) and eliminate the 500-600ms slow queries entirely.
|
||||
|
||||
**Priority:** Implement tag indexing system and add missing indexes within 1 week to prevent performance degradation as database grows.
|
||||
@@ -0,0 +1,204 @@
|
||||
# Remaining Implementation Plan — Zero SQLite on Main Thread + PG Fork Readiness
|
||||
|
||||
## Overview
|
||||
|
||||
This plan covers all remaining work to achieve two goals:
|
||||
1. **Zero direct SQLite calls on lws-main** during the event loop
|
||||
2. **Clean `db_ops` abstraction boundary** so the PG fork has no raw `sqlite3_*` calls outside `db_ops.c`
|
||||
|
||||
## Current State
|
||||
|
||||
### Completed (Fixes 1-5, 7 + Phases 1-3, 5)
|
||||
- Global config cache with 5s TTL
|
||||
- Dedup check moved to event worker
|
||||
- Async COUNT queries via thread pool
|
||||
- WoT auth rules cached per-session
|
||||
- Special-kind EVENT store through async worker
|
||||
- Subscription logging via fire-and-forget write queue
|
||||
- IP ban periodic save via write queue
|
||||
- g_db nulled before event loop
|
||||
|
||||
### Remaining
|
||||
- Fix 6: Reduce reader threads (trivial)
|
||||
- Fix 8: Move periodic tasks off main thread (partial — IP ban save done, monitoring/status not)
|
||||
- Fix 9: Cache NIP-11 response
|
||||
- Sync `store_event()` calls from admin/DM/NIP-09 paths
|
||||
- Raw `sqlite3_*` calls in `thread_pool.c` and `websockets.c`
|
||||
- COUNT query path in `websockets.c` still uses `json_each()` instead of `event_tags`
|
||||
|
||||
---
|
||||
|
||||
## Phase A: Move Remaining Periodic Tasks Off Main Thread
|
||||
|
||||
**Goal:** Eliminate the last regular SQLite calls from the lws-main event loop timer.
|
||||
|
||||
### A1: Move `generate_and_post_status_event()` to write worker
|
||||
|
||||
Currently called synchronously from the main event loop at `websockets.c:3455`.
|
||||
|
||||
**Changes:**
|
||||
- Add `THREAD_POOL_JOB_STATUS_POST` to `thread_pool.h` job type enum
|
||||
- Add executor in `thread_pool.c` that calls `generate_and_post_status_event()`
|
||||
- In `websockets.c` main loop, replace direct call with `thread_pool_submit_write()` job
|
||||
- The write worker has a DB connection, so `generate_stats_json()` → `store_event()` → `broadcast_event_to_subscriptions()` all work naturally
|
||||
- Note: `broadcast_event_to_subscriptions()` is thread-safe (uses its own mutex)
|
||||
|
||||
**Files:** `thread_pool.h`, `thread_pool.c`, `websockets.c`
|
||||
|
||||
### A2: Move monitoring queries to read/write worker
|
||||
|
||||
`monitoring_on_event_stored()` and `monitoring_on_subscription_change()` in `api.c` call `generate_event_driven_monitoring()` / `generate_subscription_driven_monitoring()` which do DB queries and then broadcast ephemeral events.
|
||||
|
||||
**Changes:**
|
||||
- Add `THREAD_POOL_JOB_MONITORING` job type
|
||||
- Submit monitoring generation as a read job (it mostly does SELECTs)
|
||||
- The completion broadcasts the ephemeral event via `lws_cancel_service()` + completion queue
|
||||
- Alternative simpler approach: since these are throttled (default 5s) and only fire when kind 24567 subscribers exist, they could run on the write worker as fire-and-forget
|
||||
|
||||
**Files:** `thread_pool.h`, `thread_pool.c`, `api.c`, `websockets.c`
|
||||
|
||||
### A3: Move `ip_ban_cleanup()` and `ip_ban_log_stats()` to write worker
|
||||
|
||||
Currently called from the 60-second periodic timer at `websockets.c:3479-3480`.
|
||||
|
||||
**Changes:**
|
||||
- Add `THREAD_POOL_JOB_IP_BAN_CLEANUP` job type (or reuse a generic callback job)
|
||||
- Submit as fire-and-forget write job from the periodic timer
|
||||
- `ip_ban_cleanup()` does in-memory cleanup + optional DB writes
|
||||
- `ip_ban_log_stats()` is purely logging, no DB
|
||||
|
||||
**Files:** `thread_pool.h`, `thread_pool.c`, `ip_ban.c`, `websockets.c`
|
||||
|
||||
---
|
||||
|
||||
## Phase B: Route Remaining Sync `store_event()` Calls Through Async Worker
|
||||
|
||||
**Goal:** Eliminate all synchronous `store_event()` calls from lws-main context.
|
||||
|
||||
### Current sync `store_event()` call sites on main thread
|
||||
|
||||
| Call site | File | Context | Frequency |
|
||||
|-----------|------|---------|-----------|
|
||||
| Kind 1059 gift wrap store | `websockets.c:1795,1803,1819` | NIP-17 DM processing | Low |
|
||||
| Kind 14 DM store | `websockets.c:1843` | NIP-17 DM processing | Low |
|
||||
| Regular event fallback | `websockets.c:1862,1876` | Non-async-eligible events | Low |
|
||||
| Kind 1059 (auth path) | `websockets.c:2594,2602,2618` | NIP-42 auth + DM | Low |
|
||||
| Kind 14 (auth path) | `websockets.c:2642` | NIP-42 auth + DM | Low |
|
||||
| Regular (auth path) | `websockets.c:2661,2675` | NIP-42 auth fallback | Low |
|
||||
| Admin response store | `config.c:2585` | Admin API response | Very low |
|
||||
| DM admin gift wrap | `dm_admin.c:389` | Admin DM response | Very low |
|
||||
| NIP-09 deletion store | `nip009.c:135` | Deletion events | Low |
|
||||
| Status event store | `api.c:595` | Periodic status post | Very low (handled by Phase A1) |
|
||||
| Admin response event | `api.c:908` | Admin API | Very low |
|
||||
| DM chat response | `api.c:1377` | Admin DM chat | Very low |
|
||||
| Relay-generated event | `api.c:2350` | Relay metadata | Very low |
|
||||
| DM stats response | `websockets.c:3648` | DM stats command | Very low |
|
||||
|
||||
### Strategy
|
||||
|
||||
Rather than converting each call site individually, create a **synchronous-to-async bridge**:
|
||||
|
||||
**Changes:**
|
||||
- Create `store_event_async(cJSON* event, store_event_callback_t cb, void* ctx)` that submits the event to the write worker
|
||||
- For call sites that need the result (e.g., to broadcast after store), use a completion callback
|
||||
- For fire-and-forget sites (admin responses, DM responses), use NULL callback
|
||||
- The existing `store_event()` wrapper becomes: submit to write worker + block on completion (for backward compat during transition)
|
||||
- Eventually, all callers migrate to the async version
|
||||
|
||||
**Alternative simpler approach:** Since these are all low-frequency paths, and the write worker already has a DB connection, we can submit them as generic write jobs with a callback that does the store + broadcast. The main thread just needs to hand off the event JSON.
|
||||
|
||||
**Files:** `main.c`, `websockets.c`, `api.c`, `config.c`, `dm_admin.c`, `nip009.c`, `thread_pool.h`, `thread_pool.c`
|
||||
|
||||
---
|
||||
|
||||
## Phase C: `db_ops` Abstraction Cleanup for PG Fork
|
||||
|
||||
**Goal:** Ensure all raw `sqlite3_*` calls are inside `db_ops.c` only. External code uses only `db_ops.h` functions.
|
||||
|
||||
### C1: Add `db_open_worker_connection()` / `db_close_worker_connection()` to `db_ops.h`
|
||||
|
||||
Currently, `thread_pool.c:208` and `websockets.c:537` call `sqlite3_open_v2()` directly.
|
||||
|
||||
**Changes:**
|
||||
- Add to `db_ops.h`:
|
||||
```c
|
||||
void* db_open_worker_connection(void); // Returns opaque connection handle
|
||||
void db_close_worker_connection(void* conn);
|
||||
```
|
||||
- Implementation in `db_ops.c`: opens a new SQLite connection with WAL + busy timeout
|
||||
- `thread_pool.c` worker init calls `db_open_worker_connection()` instead of raw `sqlite3_open_v2()`
|
||||
- `websockets.c` event-worker calls `db_open_worker_connection()` instead of raw `sqlite3_open_v2()`
|
||||
- Remove `#include <sqlite3.h>` from `thread_pool.c` and `websockets.c`
|
||||
|
||||
**Files:** `db_ops.h`, `db_ops.c`, `thread_pool.c`, `websockets.c`
|
||||
|
||||
### C2: Audit and remove remaining `sqlite3` includes outside `db_ops.c`
|
||||
|
||||
After C1, grep for any remaining `sqlite3` references outside `db_ops.c` and eliminate them.
|
||||
|
||||
**Files:** All `.c` files
|
||||
|
||||
### C3: Unify COUNT query tag path
|
||||
|
||||
The COUNT query builder in `websockets.c:3849` still uses `json_each(json(tags))` while the REQ query builder in `main.c:1604` uses the indexed `event_tags` table. Unify to use `event_tags` for consistency.
|
||||
|
||||
**Files:** `websockets.c`
|
||||
|
||||
---
|
||||
|
||||
## Phase D: Quick Wins
|
||||
|
||||
### D1: Fix 6 — Reduce reader threads from 4 to 1
|
||||
|
||||
Change default in `thread_pool_init()` or make configurable. With all reads going through the pool, a single reader is sufficient unless profiling shows otherwise.
|
||||
|
||||
**Files:** `thread_pool.c` or config default
|
||||
|
||||
### D2: Fix 9 — Cache NIP-11 response
|
||||
|
||||
Cache the serialized NIP-11 JSON string with a 60-second TTL. Invalidate on admin config change.
|
||||
|
||||
**Files:** `nip011.c`
|
||||
|
||||
---
|
||||
|
||||
## Implementation Order
|
||||
|
||||
| # | Task | Risk | Dependencies |
|
||||
|---|------|------|-------------|
|
||||
| 1 | A3: ip_ban_cleanup to write worker | Very Low | None |
|
||||
| 2 | A1: status post to write worker | Low | None |
|
||||
| 3 | A2: monitoring to worker | Low | None |
|
||||
| 4 | D1: reduce reader threads to 1 | Very Low | None |
|
||||
| 5 | D2: cache NIP-11 response | Very Low | None |
|
||||
| 6 | C1: db_open/close_worker_connection | Low | None |
|
||||
| 7 | C3: unify COUNT tag query path | Low | None |
|
||||
| 8 | C2: audit/remove sqlite3 includes | Very Low | C1 |
|
||||
| 9 | B: route sync store_event through async | Medium | A1 (status post already handled) |
|
||||
|
||||
## Expected Outcome
|
||||
|
||||
After all phases:
|
||||
- **lws-main has zero SQLite calls** during the event loop
|
||||
- **All `sqlite3_*` calls are inside `db_ops.c`** — clean abstraction boundary
|
||||
- **PG fork can replace `db_ops.c`** without touching any other file
|
||||
- Thread model is fully realized:
|
||||
- `lws-main` = WebSocket I/O + JSON parse + subscription match + completion drain
|
||||
- `event-worker` = signature verify + dedup + store submission
|
||||
- `db-read` = REQ queries + COUNT queries + monitoring reads
|
||||
- `db-write` = event INSERTs + subscription logging + periodic tasks + admin writes
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
CLIENT[Nostr Clients] --> LWS[lws-main<br/>Zero SQLite<br/>WS I/O + JSON + Sub Match]
|
||||
|
||||
LWS -->|EVENT| EW[event-worker<br/>Verify + Dedup]
|
||||
EW -->|store job| DW[db-write<br/>INSERT events<br/>Sub logging<br/>Periodic tasks<br/>Admin writes]
|
||||
EW -->|completion| LWS
|
||||
DW -->|completion| LWS
|
||||
|
||||
LWS -->|REQ/COUNT| DR[db-read<br/>SELECT queries<br/>Monitoring reads]
|
||||
DR -->|completion| LWS
|
||||
|
||||
LWS -->|OK/EVENT/EOSE/COUNT| CLIENT
|
||||
```
|
||||
@@ -0,0 +1,379 @@
|
||||
# Plan: Remove All SQLite from lws-main Thread
|
||||
|
||||
## Problem Statement
|
||||
|
||||
After implementing the 4 offload fixes (v2.1.4), `lws-main` still consumes **53.87% avg CPU** (down from 68.71%). Perf profiling shows **100% of the top symbols are SQLite** — B-tree traversal, page cache, VFS reads. The worker threads (`db-read`, `db-write`, `event-worker`) are nearly idle at 0.0–0.1%.
|
||||
|
||||
The goal is to eliminate all synchronous SQLite access from `lws-main` so it only does: JSON parsing, WebSocket I/O, subscription matching (in-memory), and message queuing.
|
||||
|
||||
## Thread Architecture Change
|
||||
|
||||
Reduce from 7 threads to 4:
|
||||
|
||||
```
|
||||
Current: lws-main, db-read-1, db-read-2, db-read-3, db-read-4, db-write, event-worker
|
||||
Proposed: lws-main, db-read, db-write, event-worker
|
||||
```
|
||||
|
||||
The 4 reader threads are overkill — they average 0.07–0.10% CPU each. A single `db-read` thread is sufficient for the current workload. This can be made configurable later if needed.
|
||||
|
||||
## Remaining SQLite Calls on lws-main
|
||||
|
||||
Every synchronous SQLite call that currently runs on `lws-main`:
|
||||
|
||||
### Hot Path — Per-Event (every incoming EVENT message)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `event_id_exists_in_db()` | websockets.c:1324, 2134 | Every EVENT | ~10-50us per B-tree lookup |
|
||||
| `get_config_value("relay_pubkey")` | websockets.c:2050 | Every kind-14 EVENT | SQLite SELECT |
|
||||
| `get_config_value("admin_pubkey")` | websockets.c:2073 | Every kind-23456 EVENT | SQLite SELECT |
|
||||
|
||||
### Hot Path — Per-Connection (every new WebSocket connection)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `get_config_bool("trust_proxy_headers")` | websockets.c:1071 | Every connection | SQLite SELECT |
|
||||
| `get_config_bool("nip42_auth_required_events")` | websockets.c:1134 | Every connection | SQLite SELECT |
|
||||
| `get_config_bool("nip42_auth_required_subscriptions")` | websockets.c:1135 | Every connection | SQLite SELECT |
|
||||
|
||||
### Hot Path — Per-REQ (every subscription request)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `get_config_bool("expiration_enabled")` | main.c:1424 | Every REQ | SQLite SELECT |
|
||||
| `get_config_bool("expiration_filter")` | main.c:1425 | Every REQ | SQLite SELECT |
|
||||
| `check_database_auth_rules()` | websockets.c:1706 | Every REQ when WoT enabled | SQLite SELECT |
|
||||
|
||||
### Medium Path — Per-COUNT (NIP-45)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `thread_pool_execute_count_sync()` | websockets.c:3689 | Every COUNT message | Blocks main thread waiting for db-read |
|
||||
|
||||
### Low Path — Periodic (every 60s or on timer)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `refresh_hot_config_if_needed()` | websockets.c:228-247 | Every 5s | 8 SQLite SELECTs |
|
||||
| `generate_and_post_status_event()` | websockets.c:3213 | Every N hours | store_event + broadcast |
|
||||
| `ip_ban_cleanup/log_stats` | websockets.c:3004-3005 | Every 60s | SQLite reads/writes |
|
||||
|
||||
### Per-EVENT — Validator Path (called from event-worker AND sync path)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `get_config_value("admin_pubkey")` | request_validator.c:308 | Every kind-23456 EVENT | SQLite SELECT |
|
||||
| `get_config_value("nip42_auth_enabled")` | request_validator.c:326 | Every EVENT | SQLite SELECT |
|
||||
| `get_config_bool("pow_enabled")` | request_validator.c:370 | Every EVENT | SQLite SELECT |
|
||||
| `get_config_int("pow_min_difficulty")` | request_validator.c:371 | Every EVENT | SQLite SELECT |
|
||||
| `get_config_int("pow_validation_flags")` | request_validator.c:372 | Every EVENT | SQLite SELECT |
|
||||
| `get_config_int("nip40_expiration_grace_period")` | request_validator.c:432 | Every EVENT with expiration | SQLite SELECT |
|
||||
| `get_config_value("auth_enabled")` | request_validator.c:212 | Every EVENT (reload_auth_config) | SQLite SELECT |
|
||||
| `get_config_value("auth_rules_enabled")` | request_validator.c:220 | Every EVENT (reload_auth_config) | SQLite SELECT |
|
||||
| `db_count_active_whitelist_rules()` | request_validator.c:536 | Every 5s (cached) | Opens separate SQLite connection |
|
||||
|
||||
**Note:** These run on the **event-worker** thread for async-eligible events, but on **lws-main** for special kinds (14, 1059, 23456) that go through the sync validation path. They also run on lws-main for the `nostr_validate_unified_request()` call in the sync fallback.
|
||||
|
||||
### Per-EVENT — NIP-13 PoW Validation
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `get_config_bool("pow_enabled")` | nip013.c:27 | Every EVENT | SQLite SELECT |
|
||||
| `get_config_int("pow_min_difficulty")` | nip013.c:28 | Every EVENT | SQLite SELECT |
|
||||
| `get_config_value("pow_mode")` | nip013.c:29 | Every EVENT | SQLite SELECT |
|
||||
|
||||
### Per-Broadcast — NIP-40 Expiration Check
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `get_config_bool("expiration_enabled")` | subscriptions.c:764 | Every broadcast | SQLite SELECT |
|
||||
| `get_config_bool("expiration_filter")` | subscriptions.c:765 | Every broadcast | SQLite SELECT |
|
||||
|
||||
### Per-NIP-11 Request (HTTP, not WebSocket)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| 15+ `get_config_value()` calls | nip011.c:96-110 | Every NIP-11 HTTP request | 15 SQLite SELECTs |
|
||||
| 10+ `get_config_int()`/`get_config_bool()` calls | nip011.c:113-123 | Every NIP-11 HTTP request | 10 SQLite SELECTs |
|
||||
|
||||
### Per-NIP-42 Auth Event
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `get_config_value("relay_url")` | nip042.c:99 | Every AUTH event | SQLite SELECT |
|
||||
| `get_config_int("relay_port")` | nip042.c:103 | Every AUTH event (fallback) | SQLite SELECT |
|
||||
|
||||
### Per-NIP-40 Expiration Check (EVENT submission)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| 5 `get_config_bool()`/`get_config_int()` calls | nip040.c:37-41 | Every EVENT with expiration tag | 5 SQLite SELECTs |
|
||||
|
||||
### Subscription Lifecycle (per sub create/close/disconnect)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `db_log_subscription_created()` | subscriptions.c:1043 | Every REQ | SQLite INSERT |
|
||||
| `db_log_subscription_closed()` | subscriptions.c:1053 | Every CLOSE | SQLite INSERT + UPDATE |
|
||||
| `db_log_subscription_disconnected()` | subscriptions.c:1060 | Every disconnect | SQLite UPDATE + INSERT |
|
||||
| `db_update_subscription_events_sent()` | subscriptions.c:1088 | Every broadcast match | SQLite UPDATE |
|
||||
| `db_cleanup_orphaned_subscriptions()` | subscriptions.c:1100 | Startup only | SQLite DELETE |
|
||||
|
||||
### IP Ban System (per-connection and periodic)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `get_config_value("idle_ban_whitelist")` | ip_ban.c:254 | Every ban check | SQLite SELECT |
|
||||
| `get_config_bool("auth_fail_ban_enabled")` | ip_ban.c:301, 343 | Every ban check + failure | SQLite SELECT |
|
||||
| `get_config_bool("idle_ban_enabled")` | ip_ban.c:307, 394 | Every ban check + idle failure | SQLite SELECT |
|
||||
| 6+ `get_config_int()` calls | ip_ban.c:345-399, 464-465 | Every failure recording | SQLite SELECTs |
|
||||
| `db_prepare()`/`db_step_stmt()` for ip_bans | ip_ban.c:141-238 | Load at startup + periodic save | SQLite reads/writes |
|
||||
|
||||
### Special Kind Sync Path (kinds 14, 1059, 23456)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `store_event()` | websockets.c:1548-1629 | Every special-kind EVENT | Full sync store |
|
||||
| `process_admin_event_in_config()` | websockets.c:1468 | Every kind-23456 | Config DB writes |
|
||||
| `process_nip17_admin_message()` | websockets.c:1523 | Every kind-1059 when enabled | Decrypt + DB |
|
||||
| `process_dm_stats_command()` | websockets.c:3240 | Admin DM stats | Decrypt + query + encrypt |
|
||||
| `generate_stats_json()` | api.c:1167 (via dm_admin.c:805) | Admin stats command | Multiple SQLite queries |
|
||||
|
||||
### DM Admin Path (called from lws-main for kind 1059/14)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `get_config_value("relay_pubkey")` | dm_admin.c:358, 451 | Every admin DM | SQLite SELECT |
|
||||
| `get_config_value("admin_pubkey")` | dm_admin.c:504 | Every admin DM | SQLite SELECT |
|
||||
| `get_config_int("nip59_timestamp_max_delay_sec")` | dm_admin.c:373 | Every admin DM | SQLite SELECT |
|
||||
| `get_config_int("wot_enabled")` | dm_admin.c:613, 640, 660 | WoT commands | SQLite SELECT |
|
||||
|
||||
### API/Monitoring (called from lws-main periodic timer)
|
||||
|
||||
| Call | Location | Frequency | Cost |
|
||||
|------|----------|-----------|------|
|
||||
| `generate_stats_json()` | api.c:1167 | Status post + admin stats | ~10 SQLite queries |
|
||||
| `query_time_based_statistics()` | api.c:102 | Monitoring events | SQLite queries |
|
||||
| `query_subscription_details()` | api.c:197 | Monitoring events | SQLite queries |
|
||||
| `get_config_value("relay_pubkey")` | api.c:419, 821, 1291 | Every monitoring event | SQLite SELECT |
|
||||
| `get_config_int("kind_1_status_posts_hours")` | api.c:547 | Status post check | SQLite SELECT |
|
||||
| `get_config_int("kind_24567_reporting_throttle_sec")` | api.c:60 | Monitoring throttle | SQLite SELECT |
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Fix 1: Create Global Config Cache — Eliminate ALL get_config_* SQLite Calls
|
||||
|
||||
**Impact: Very High** — eliminates 50+ distinct `get_config_value/int/bool` call sites across 8 source files
|
||||
|
||||
The audit found `get_config_value/int/bool` calls in: `websockets.c`, `main.c`, `subscriptions.c`, `request_validator.c`, `nip011.c`, `nip013.c`, `nip040.c`, `nip042.c`, `ip_ban.c`, `dm_admin.c`, `api.c`, `config.c`. Every single one does a `SELECT value FROM config WHERE key=?` SQLite query.
|
||||
|
||||
**Approach:** Replace `get_config_value_from_table()` with an in-memory hash map that is loaded once at startup and refreshed periodically (every 5s) or on admin config change events.
|
||||
|
||||
**Changes:**
|
||||
|
||||
1. **New: `config_cache` in config.c** — A simple hash map or fixed array of key-value pairs loaded from the config table:
|
||||
```c
|
||||
typedef struct {
|
||||
char key[64];
|
||||
char value[512];
|
||||
} config_cache_entry_t;
|
||||
|
||||
static config_cache_entry_t g_config_cache[256];
|
||||
static int g_config_cache_count = 0;
|
||||
static time_t g_config_cache_last_refresh = 0;
|
||||
static pthread_mutex_t g_config_cache_mutex;
|
||||
#define CONFIG_CACHE_TTL_SEC 5
|
||||
```
|
||||
|
||||
2. **Modify `get_config_value_from_table()`** — Instead of querying SQLite, look up in the in-memory cache. If cache is stale, refresh from SQLite (but only once per TTL period, not per call).
|
||||
|
||||
3. **Add `config_cache_refresh()`** — Loads all rows from `SELECT key, value FROM config` into the hash map. Called:
|
||||
- Once at startup after config table is populated
|
||||
- Every 5 seconds from the existing `refresh_hot_config_if_needed()` timer
|
||||
- Immediately after `process_admin_event_in_config()` modifies config
|
||||
|
||||
4. **Thread safety:** The cache is read from lws-main and event-worker threads. Use a read-write lock or double-buffer pattern. Since config changes are rare, a simple mutex with short hold time is fine.
|
||||
|
||||
5. **Remove the existing `hot_config_cache_t`** in websockets.c — it becomes redundant since the global config cache serves the same purpose with broader coverage.
|
||||
|
||||
This single change eliminates SQLite calls from:
|
||||
- Per-connection: `trust_proxy_headers`, `nip42_auth_required_*` (websockets.c)
|
||||
- Per-REQ: `expiration_enabled`, `expiration_filter` (main.c)
|
||||
- Per-EVENT: `admin_pubkey`, `relay_pubkey`, `nip42_auth_enabled`, `pow_*` (request_validator.c, nip013.c)
|
||||
- Per-broadcast: `expiration_enabled`, `expiration_filter` (subscriptions.c)
|
||||
- Per-NIP-11: 25+ config values (nip011.c)
|
||||
- Per-NIP-42: `relay_url`, `relay_port` (nip042.c)
|
||||
- Per-NIP-40: 5 expiration config values (nip040.c)
|
||||
- Per-IP-ban: `idle_ban_*`, `auth_fail_*` config values (ip_ban.c)
|
||||
- Per-admin-DM: `relay_pubkey`, `admin_pubkey`, `wot_enabled` (dm_admin.c)
|
||||
- Per-monitoring: `relay_pubkey`, throttle values (api.c)
|
||||
|
||||
### Fix 2: Move Duplicate Check to Event Worker Thread
|
||||
|
||||
**Impact: High** — eliminates the most frequent per-EVENT SQLite call from lws-main
|
||||
|
||||
Currently `event_id_exists_in_db()` runs on lws-main before async submission. Move it into the `async_event_worker_main()` function, which already has its own SQLite connection.
|
||||
|
||||
**Changes:**
|
||||
- Remove `event_id_exists_in_db()` calls from both EVENT paths in websockets.c (lines 1324 and 2134)
|
||||
- Add the duplicate check at the start of `async_event_worker_main()` before `nostr_validate_unified_request()`
|
||||
- If duplicate found, set `completion->success = 1` and `completion->error_message = "duplicate: already have this event"` and skip crypto
|
||||
- The completion handler already sends OK responses, so this works seamlessly
|
||||
|
||||
**Trade-off:** Without the early dedup check on lws-main, duplicate events will be submitted to the event worker queue and consume a queue slot + thread wakeup before being detected. This is acceptable because:
|
||||
- The worker thread dedup check is still fast (B-tree index lookup)
|
||||
- It avoids the much larger cost of running SQLite on lws-main for every event
|
||||
- Queue depth is 4096, so even under heavy duplicate traffic this won't overflow
|
||||
|
||||
### Fix 3: Make COUNT Queries Async
|
||||
|
||||
**Impact: Medium** — eliminates blocking wait on lws-main for NIP-45 COUNT
|
||||
|
||||
Currently `handle_count_message()` calls `thread_pool_execute_count_sync()` which blocks lws-main waiting for the db-read worker. Convert to the same async pattern used for REQ queries.
|
||||
|
||||
**Changes:**
|
||||
- Create `count_async_state_t` similar to `req_async_state_t`
|
||||
- Create `count_async_completion_t` with count result
|
||||
- Add `submit_count_query_async()` that uses `thread_pool_submit_read()` with a callback
|
||||
- The callback pushes to a count completion queue
|
||||
- Add `process_count_async_completions()` to drain the queue on lws-main and send COUNT responses
|
||||
- Call it from the main event loop alongside `process_req_async_completions()`
|
||||
|
||||
### Fix 4: Cache Auth Rules Check Result
|
||||
|
||||
**Impact: Low-Medium** — eliminates per-REQ SQLite when WoT is enabled
|
||||
|
||||
`check_database_auth_rules()` at websockets.c:1706 runs a SQLite query for every REQ when `wot_enabled == 2`. Cache the result per-session.
|
||||
|
||||
**Changes:**
|
||||
- Add `wot_checked` and `wot_allowed` fields to `per_session_data`
|
||||
- On first REQ, call `check_database_auth_rules()` and cache the result
|
||||
- On subsequent REQs from the same session, use the cached value
|
||||
- Reset cache when auth state changes (e.g., after NIP-42 AUTH)
|
||||
|
||||
### Fix 5: Move Special-Kind EVENT Store to Event Worker
|
||||
|
||||
**Impact: Low** — special kinds are rare but currently do full sync store on lws-main
|
||||
|
||||
Kinds 14, 1059, and 23456 are currently excluded from async processing because they have special handling. However, the `store_event()` call within those paths still blocks lws-main.
|
||||
|
||||
**Changes:**
|
||||
- For kind 1059 and kind 14: after special processing completes, submit the store to the event worker instead of calling `store_event()` synchronously
|
||||
- For kind 23456: admin events are not stored (processed by admin API), so no change needed
|
||||
- This requires the special processing (decryption, admin check) to still happen on lws-main (it needs `pss` context), but the final DB write goes async
|
||||
|
||||
### Fix 6: Reduce Reader Threads from 4 to 1
|
||||
|
||||
**Impact: Resource savings** — 3 fewer threads, 3 fewer SQLite connections
|
||||
|
||||
**Changes:**
|
||||
- Change default `reader_threads` in `thread_pool_init()` from 4 to 1
|
||||
- Or make it configurable via config with default 1
|
||||
- The single `db-read` thread handles all REQ and COUNT queries sequentially
|
||||
- If future profiling shows the read thread becoming a bottleneck, increase back to 2+
|
||||
|
||||
### Fix 7: Move Subscription Logging Off Main Thread
|
||||
|
||||
**Impact: Medium** — eliminates per-REQ INSERT, per-CLOSE UPDATE, per-broadcast UPDATE
|
||||
|
||||
The subscription lifecycle functions in subscriptions.c do synchronous SQLite writes on lws-main:
|
||||
- `db_log_subscription_created()` — INSERT on every REQ
|
||||
- `db_log_subscription_closed()` — INSERT + UPDATE on every CLOSE
|
||||
- `db_log_subscription_disconnected()` — UPDATE + INSERT on every disconnect
|
||||
- `db_update_subscription_events_sent()` — UPDATE on every broadcast match
|
||||
|
||||
**Changes:**
|
||||
- Submit these as fire-and-forget write jobs to the db-write thread via `thread_pool_submit_write()`
|
||||
- Create a new job type `THREAD_POOL_JOB_SUBSCRIPTION_LOG` or simply use a generic callback job
|
||||
- The main thread doesn't need to wait for these to complete — they're purely observational logging
|
||||
|
||||
### Fix 8: Move Periodic Tasks Off Main Thread
|
||||
|
||||
**Impact: Low** — these run every 60s but can still cause latency spikes
|
||||
|
||||
- `generate_and_post_status_event()`: Submit as an async job to the event worker
|
||||
- `ip_ban_cleanup()`/`ip_ban_log_stats()`/`ip_ban_save()`: These do SQLite reads/writes. Move to a periodic job on the db-write thread
|
||||
- Config cache refresh (Fix 1) replaces the old `refresh_hot_config_if_needed()` SQLite queries
|
||||
|
||||
### Fix 9: Cache NIP-11 Response
|
||||
|
||||
**Impact: Low-Medium** — eliminates 25+ SQLite SELECTs per NIP-11 HTTP request
|
||||
|
||||
`generate_relay_info_json()` in nip011.c calls `get_config_value()` 15 times and `get_config_int()`/`get_config_bool()` 10 times. After Fix 1 (global config cache), these become in-memory lookups. But we can go further:
|
||||
|
||||
**Changes:**
|
||||
- Cache the serialized NIP-11 JSON response string with a TTL (e.g., 60 seconds)
|
||||
- On NIP-11 request, return the cached string directly without rebuilding
|
||||
- Invalidate on admin config change events
|
||||
|
||||
## Implementation Order
|
||||
|
||||
| # | Fix | Risk | Complexity |
|
||||
|---|-----|------|------------|
|
||||
| 1 | Global config cache (replaces all get_config_* SQLite) | Low | Medium |
|
||||
| 2 | Move dedup check to event worker | Low | Low |
|
||||
| 3 | Async COUNT queries | Low | Medium |
|
||||
| 4 | Cache WoT auth rules per-session | Very Low | Low |
|
||||
| 5 | Async store for special kinds | Medium | Medium |
|
||||
| 6 | Reduce reader threads to 1 | Very Low | Trivial |
|
||||
| 7 | Move subscription logging off main thread | Low | Low-Medium |
|
||||
| 8 | Move periodic tasks off main thread | Low | Medium |
|
||||
| 9 | Cache NIP-11 response | Very Low | Low |
|
||||
|
||||
## Expected Impact
|
||||
|
||||
After all 9 fixes, `lws-main` should have **zero direct SQLite calls**. Its work becomes:
|
||||
- JSON parse incoming messages (~fast)
|
||||
- In-memory subscription matching (~fast)
|
||||
- In-memory config lookups (~fast, hash map)
|
||||
- Message formatting and queuing (~fast)
|
||||
- `lws_service()` I/O (~fast)
|
||||
- Draining completion queues (~fast)
|
||||
|
||||
### Impact by fix
|
||||
|
||||
| Fix | SQLite calls eliminated | Frequency |
|
||||
|-----|------------------------|-----------|
|
||||
| Fix 1: Global config cache | ~50+ `get_config_*` call sites across 8 files | Per-event, per-connection, per-REQ, per-broadcast, per-NIP-11 |
|
||||
| Fix 2: Dedup to event worker | 2 `event_id_exists_in_db()` calls | Per-EVENT |
|
||||
| Fix 3: Async COUNT | 1 `thread_pool_execute_count_sync()` | Per-COUNT message |
|
||||
| Fix 4: WoT cache per-session | 1 `check_database_auth_rules()` | Per-REQ when WoT enabled |
|
||||
| Fix 5: Async special-kind store | ~6 `store_event()` calls | Per special-kind EVENT |
|
||||
| Fix 6: Reduce readers | N/A (resource savings) | N/A |
|
||||
| Fix 7: Async subscription logging | 4 `db_log_*`/`db_update_*` calls | Per-REQ, per-CLOSE, per-broadcast |
|
||||
| Fix 8: Async periodic tasks | `generate_stats_json()`, `ip_ban_*` | Every 60s |
|
||||
| Fix 9: Cache NIP-11 | 25+ config lookups per request | Per NIP-11 HTTP request |
|
||||
|
||||
Estimated main-thread CPU reduction: from **54% to ~5-10%** (mostly WebSocket I/O, subscription matching, and JSON parsing).
|
||||
|
||||
## Thread Model After Changes
|
||||
|
||||
```
|
||||
lws-main — WebSocket I/O, JSON parse, subscription match, message queue
|
||||
NO SQLite. Pure compute + I/O.
|
||||
|
||||
db-read — All SELECT queries: REQ results, COUNT results, config refresh,
|
||||
auth rule checks, event dedup checks via completion queue
|
||||
|
||||
db-write — All INSERT/UPDATE: event storage, tag inserts, IP ban persistence,
|
||||
status event generation
|
||||
|
||||
event-worker — EVENT validation: signature verification, then submits store to
|
||||
db-write via completion queue, results flow back to lws-main
|
||||
```
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
CLIENT[Nostr Clients] --> LWS[lws-main<br/>WebSocket I/O<br/>JSON parse<br/>Sub matching<br/>NO SQLite]
|
||||
|
||||
LWS -->|EVENT submit| EW[event-worker<br/>Sig verify<br/>Dedup check]
|
||||
EW -->|store job| DW[db-write<br/>INSERT events<br/>INSERT tags<br/>IP ban save]
|
||||
EW -->|completion| LWS
|
||||
DW -->|completion| LWS
|
||||
|
||||
LWS -->|REQ/COUNT submit| DR[db-read<br/>SELECT queries<br/>Config refresh<br/>Auth checks]
|
||||
DR -->|completion| LWS
|
||||
|
||||
LWS -->|OK/EVENT/EOSE| CLIENT
|
||||
```
|
||||
@@ -0,0 +1,215 @@
|
||||
# Thread Pool PostgreSQL Optimization Plan
|
||||
|
||||
## Current State
|
||||
|
||||
### Thread Inventory
|
||||
| Thread | Name | Role | PG Connection | Config Key |
|
||||
|--------|------|------|--------------|------------|
|
||||
| lws-main | `lws-main` | WebSocket event loop, HTTP, NIP-11, admin events, subscription matching, broadcast, monitoring event generation | `g_pg_conn` (main) | N/A — always runs |
|
||||
| reader-1 | `db-read-1` | Async REQ and COUNT query execution | `g_thread_pg_conn` (TLS) | `thread_pool_readers` (default: 1) |
|
||||
| writer | `db-write` | Async EVENT store, delete, sub logging, IP ban save, WAL checkpoint | `g_thread_pg_conn` (TLS) | N/A — always 1 |
|
||||
| event-worker | `event-worker` | Async EVENT ingestion — duplicate check, signature validation, `store_event_core()`, ephemeral event handling | Own `worker_db` via `db_open_worker_connection()` | N/A — always 1 |
|
||||
|
||||
**Total: 4 threads, 4 PG connections**
|
||||
|
||||
### event-worker Thread (websockets.c — separate from thread pool)
|
||||
|
||||
The `event-worker` is a dedicated async pipeline for incoming EVENT messages, defined in [`src/websockets.c`](src/websockets.c:543). It is **not** part of the thread pool — it has its own job queue, mutex, and condvar.
|
||||
|
||||
**What it does for each incoming EVENT:**
|
||||
1. Checks if event ID already exists in DB (`event_id_exists_in_db()`)
|
||||
2. Validates the event with `nostr_validate_unified_request()` (signature check)
|
||||
3. For ephemeral events (kind 20000-29999): marks for broadcast only, no storage
|
||||
4. For regular events: calls `store_event_core()` to insert into PostgreSQL
|
||||
5. Pushes a completion struct and wakes `lws-main` via `lws_cancel_service()`
|
||||
6. `lws-main` picks up the completion, sends OK response, and broadcasts to subscribers
|
||||
|
||||
**Why it exists separately from the thread pool writer:**
|
||||
- The thread pool writer handles generic DB operations (sub logging, IP bans, etc.)
|
||||
- The event-worker handles the full EVENT ingestion pipeline including validation and duplicate detection — not just the DB insert
|
||||
- It was added to move CPU-intensive signature verification off the main thread
|
||||
|
||||
**Relationship to thread pool writer:**
|
||||
There is overlap — both can write to the events table. The event-worker calls `store_event_core()` directly, while the thread pool writer uses `db_insert_event_with_json()`. Some EVENT paths still go through the thread pool writer (sync path for admin/DM/protected events).
|
||||
|
||||
### What Runs on lws-main Today (Shouldn't Block)
|
||||
- WebSocket accept/read/write for all Nostr clients
|
||||
- JSON parsing of REQ/EVENT/CLOSE/COUNT messages
|
||||
- Event signature verification (CPU-bound)
|
||||
- Subscription matching and broadcast (iterates all subscriptions)
|
||||
- NIP-11 HTTP responses
|
||||
- NIP-42 auth challenge generation
|
||||
- Admin event processing (kind 23456) — includes NIP-44 decrypt + config changes
|
||||
- **Monitoring event generation** — queries DB, signs events, broadcasts (BLOCKS)
|
||||
- **Sync config reads** — `get_config_value()` hits PG on main thread (BLOCKS)
|
||||
|
||||
### SQLite-Era Constraints (No Longer Apply)
|
||||
- Single writer thread was required (SQLite serializes writes)
|
||||
- WAL checkpoint job was required (SQLite-specific)
|
||||
- Reader/writer separation was critical for avoiding SQLITE_BUSY
|
||||
- `db_wal_checkpoint_truncate()` on writer shutdown — SQLite-only
|
||||
|
||||
---
|
||||
|
||||
## Changes for PostgreSQL
|
||||
|
||||
### Phase 1: Increase Reader Thread Count
|
||||
|
||||
**What:** Change default reader count from 1 to 4.
|
||||
|
||||
**Why:** With SQLite, multiple readers had diminishing returns due to file-level locking. PostgreSQL handles concurrent reads perfectly — each reader has its own connection and queries execute in parallel.
|
||||
|
||||
**Files:**
|
||||
- `src/main.c` line ~2524: Change default from 1 to 4
|
||||
- Config key `thread_pool_readers` already exists — just change the default
|
||||
|
||||
**Risk:** Low. Each reader opens its own PG connection via `db_open_worker_connection()`. PostgreSQL default `max_connections = 100` easily handles 4-8 readers.
|
||||
|
||||
### Phase 2: Allow Multiple Writer Threads
|
||||
|
||||
**What:** Make writer thread count configurable (default: 2). Convert the single `pthread_t writer` to an array like readers.
|
||||
|
||||
**Why:** PostgreSQL supports concurrent writes with row-level locking. The single-writer bottleneck was a SQLite constraint. With multiple writers, EVENT inserts from different clients can execute in parallel.
|
||||
|
||||
**Files:**
|
||||
- `src/thread_pool.h`: Add `int writer_threads` to `thread_pool_config_t`
|
||||
- `src/thread_pool.c`:
|
||||
- Change `pthread_t writer` to `pthread_t* writers` + `int writer_count`
|
||||
- `thread_pool_init()`: create N writer threads
|
||||
- `thread_pool_shutdown()`: join all writer threads
|
||||
- `writer_worker_main()`: set thread name to `db-write-N`
|
||||
- `src/main.c`: Add config key `thread_pool_writers` (default: 2)
|
||||
|
||||
**Risk:** Medium. Need to verify that no write job assumes serialized execution. Current write jobs:
|
||||
- `STORE_EVENT` — PG handles concurrent inserts with ON CONFLICT
|
||||
- `LOG_SUB_CREATED/CLOSED/DISCONNECTED` — independent rows, safe
|
||||
- `UPDATE_SUB_EVENTS_SENT` — updates by sub_id, safe
|
||||
- `IP_BAN_SAVE` — writes to ip_bans table, safe
|
||||
- `WAL_CHECKPOINT` — SQLite-only, skip for PG (see Phase 3)
|
||||
|
||||
All write jobs operate on independent rows or use conflict resolution. Multiple writers are safe.
|
||||
|
||||
### Phase 3: Remove SQLite-Specific WAL Checkpoint Logic
|
||||
|
||||
**What:** Make WAL checkpoint a no-op when running PostgreSQL backend.
|
||||
|
||||
**Why:** PostgreSQL manages its own WAL internally. The `db_wal_checkpoint_passive()` and `db_wal_checkpoint_truncate()` calls are SQLite-specific.
|
||||
|
||||
**Files:**
|
||||
- `src/thread_pool.c`:
|
||||
- `execute_wal_checkpoint_job()`: Return OK immediately if PG backend
|
||||
- `writer_worker_main()` shutdown path: Skip `db_wal_checkpoint_truncate()` if PG backend
|
||||
- `src/db_ops.c` or `src/db_ops_postgres.c`: Ensure WAL checkpoint functions are no-ops for PG
|
||||
|
||||
**Risk:** Low. These are already effectively no-ops since the PG backend doesn't implement them, but making it explicit avoids log noise.
|
||||
|
||||
### Phase 4: Move Monitoring Event Generation Off Main Thread
|
||||
|
||||
**What:** Run monitoring queries and event generation on a reader thread instead of lws-main.
|
||||
|
||||
**Why:** The monitoring system (`generate_monitoring_event_for_type()` in `src/api.c`) currently:
|
||||
1. Queries PostgreSQL for stats (blocks main thread 5-50ms)
|
||||
2. Creates and signs a Nostr event (CPU work on main thread)
|
||||
3. Broadcasts to subscriptions (main thread)
|
||||
|
||||
Step 1 and 2 should happen on a worker thread. Only step 3 (broadcast) needs to happen on lws-main.
|
||||
|
||||
**Approach:** Add a new job type `THREAD_POOL_JOB_MONITORING_QUERY` to the read queue. The reader thread executes the query and prepares the event JSON. The completion callback on lws-main just broadcasts the pre-built event.
|
||||
|
||||
**Files:**
|
||||
- `src/thread_pool.h`: Add `THREAD_POOL_JOB_MONITORING_QUERY` job type and payload struct
|
||||
- `src/thread_pool.c`: Add `execute_monitoring_query_job()` handler
|
||||
- `src/api.c`: Refactor `generate_monitoring_event_for_type()` to submit async job
|
||||
- `src/websockets.c`: Handle monitoring completion in the event loop callback
|
||||
|
||||
**Risk:** Medium. The monitoring event signing requires the relay private key. Need to pass it to the worker thread or pre-compute the unsigned event and sign on completion.
|
||||
|
||||
### Phase 5: Add Configurable Thread Pool Sizing via Config Events
|
||||
|
||||
**What:** Allow thread pool reader/writer counts to be set via the admin config event system.
|
||||
|
||||
**Config keys:**
|
||||
- `thread_pool_readers` — number of reader threads (default: 4)
|
||||
- `thread_pool_writers` — number of writer threads (default: 2)
|
||||
- `thread_pool_max_queue_depth` — max pending jobs per queue (default: 4096)
|
||||
|
||||
**Note:** Thread count changes require relay restart (cannot dynamically resize thread pool).
|
||||
|
||||
**Files:**
|
||||
- `src/default_config_event.h`: Add defaults for new config keys
|
||||
- `src/main.c`: Read config values when initializing thread pool
|
||||
|
||||
**Risk:** Low. Config keys already work this way for `thread_pool_readers`.
|
||||
|
||||
### Phase 6: Fix Error Messages (Cosmetic)
|
||||
|
||||
**What:** Update SQLite-specific error messages in thread pool to be backend-neutral.
|
||||
|
||||
**Files:**
|
||||
- `src/thread_pool.c` line 508: "Reader worker failed to open SQLite connection" → "Reader worker failed to open database connection"
|
||||
- `src/thread_pool.c` line 530: "Writer worker failed to open SQLite connection" → "Writer worker failed to open database connection"
|
||||
|
||||
**Risk:** None.
|
||||
|
||||
---
|
||||
|
||||
## Target Thread Layout After Implementation
|
||||
|
||||
```
|
||||
c_relay_pg process
|
||||
├── lws-main — WebSocket event loop (never blocks on DB)
|
||||
├── db-read-1 — Async REQ/COUNT/monitoring queries
|
||||
├── db-read-2 — Async REQ/COUNT/monitoring queries
|
||||
├── db-read-3 — Async REQ/COUNT/monitoring queries
|
||||
├── db-read-4 — Async REQ/COUNT/monitoring queries
|
||||
├── event-worker — Async EVENT ingestion (validate + store)
|
||||
├── db-write-1 — Async sub logging, IP bans, misc writes
|
||||
└── db-write-2 — Async sub logging, IP bans, misc writes
|
||||
```
|
||||
|
||||
**Total: 8 threads, 8 PG connections** (configurable)
|
||||
|
||||
---
|
||||
|
||||
## PostgreSQL Connection Budget
|
||||
|
||||
| Component | Connections | Notes |
|
||||
|-----------|------------|-------|
|
||||
| lws-main | 1 | Sync config reads, admin event processing |
|
||||
| Reader threads (4) | 4 | One per reader |
|
||||
| event-worker | 1 | EVENT ingestion pipeline |
|
||||
| Writer threads (2) | 2 | Sub logging, IP bans, misc |
|
||||
| **Total per instance** | **8** | Well within PG default max_connections=100 |
|
||||
| Future: admin-ws | +1 | When/if added later |
|
||||
| Future: pg-listener | +1 | When/if added later |
|
||||
| **Future total** | **10** | Still very comfortable |
|
||||
|
||||
For load balancing with N relay instances: N × 8 connections. With 10 instances = 80 connections. PgBouncer recommended at that scale.
|
||||
|
||||
---
|
||||
|
||||
## Implementation Order
|
||||
|
||||
| Phase | Description | Complexity | Impact |
|
||||
|-------|-------------|-----------|--------|
|
||||
| 1 | Increase reader count to 4 | Trivial — one line change | High — parallel query execution |
|
||||
| 2 | Multiple writer threads | Medium — refactor writer array | Medium — parallel event inserts |
|
||||
| 3 | Remove WAL checkpoint logic | Low — conditional no-op | Low — cleaner code |
|
||||
| 4 | Move monitoring off main thread | Medium — new job type + async flow | High — unblocks event loop |
|
||||
| 5 | Config event integration | Low — add config keys | Low — operational convenience |
|
||||
| 6 | Fix error messages | Trivial | None — cosmetic |
|
||||
|
||||
**Recommended start:** Phase 1 + 6 (trivial, immediate benefit), then Phase 3 (cleanup), then Phase 4 (biggest architectural improvement), then Phase 2 (nice-to-have), then Phase 5 (polish).
|
||||
|
||||
---
|
||||
|
||||
## Future: Load Balancing Considerations
|
||||
|
||||
The thread pool design is already compatible with horizontal scaling because:
|
||||
|
||||
1. **No shared in-process state between instances** — all state is in PostgreSQL
|
||||
2. **Subscription matching is per-instance** — each instance tracks its own connected clients
|
||||
3. **Cross-instance event notification** via PostgreSQL LISTEN/NOTIFY (future phase)
|
||||
4. **Connection pooling** via PgBouncer when connection count becomes a concern
|
||||
|
||||
The only addition needed for multi-instance is a LISTEN/NOTIFY subscriber thread that receives "new event stored" notifications from PostgreSQL and triggers subscription matching on the local instance. This is the `pg-listener` thread discussed earlier — it can be added independently of the thread pool changes in this plan.
|
||||
@@ -0,0 +1,107 @@
|
||||
# WAL Checkpoint Fix — db-write Thread CPU Plan
|
||||
|
||||
## Problem
|
||||
|
||||
Fresh profiling of the production server shows the `db-write` thread consuming **77% average CPU** (peaking at 100%) while `lws-main` sits at 0.3%.
|
||||
|
||||
The `perf` callgraph confirms 100% of the hot symbols are SQLite **B-tree read operations** running on the `db-write` thread:
|
||||
|
||||
| % CPU | Symbol | Role |
|
||||
|-------|--------|------|
|
||||
| 13.88 | `sqlite3BtreeTableMoveto` | B-tree seek |
|
||||
| 10.05 | `sqlite3VdbeExec` | VM execution |
|
||||
| 7.77 | `rep_movs_alternative` | kernel memcpy in pread |
|
||||
| 7.76 | `sqlite3GetVarint` | varint decode |
|
||||
| 6.66 | `pcache1Fetch` | page cache lookup |
|
||||
| 6.02 | `memcpy` | data copy |
|
||||
|
||||
These are **read-path** functions, not write-path. The `event-worker` thread accumulated only 3 CPU ticks over 5 minutes, meaning very few events are actually being stored.
|
||||
|
||||
## Root Cause
|
||||
|
||||
SQLite WAL auto-checkpoint. By default SQLite triggers a checkpoint every 1000 WAL pages. The checkpoint is executed by the **next writer** — which is always the `db-write` thread since all writes are funnelled through it.
|
||||
|
||||
A WAL checkpoint reads every dirty page from the WAL file and writes it back to the main database file. This involves B-tree traversals to locate pages, which explains the `BtreeTableMoveto` dominance.
|
||||
|
||||
The current `db_open_worker_connection` in `src/db_ops.c` sets `PRAGMA journal_mode=WAL` and `busy_timeout=5000` but does **not** configure `wal_autocheckpoint`. SQLite's default of 1000 pages applies.
|
||||
|
||||
In earlier profiling runs the same CPU burn appeared on `lws-main` at 55-67% — because that thread was doing the writes directly before the thread-pool refactoring. The work simply moved threads; the total cost is unchanged.
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Step 1 — Disable auto-checkpoint on the write connection
|
||||
|
||||
In `db_open_worker_connection` add:
|
||||
|
||||
```c
|
||||
sqlite3_exec(db, "PRAGMA wal_autocheckpoint=0;", NULL, NULL, NULL);
|
||||
```
|
||||
|
||||
This prevents the write thread from ever running a checkpoint inline with normal writes.
|
||||
|
||||
### Step 2 — Add a periodic checkpoint job type to the thread pool
|
||||
|
||||
Add a new job type `THREAD_POOL_JOB_WAL_CHECKPOINT` to `thread_pool.h`.
|
||||
|
||||
The handler in `thread_pool.c` calls:
|
||||
|
||||
```c
|
||||
sqlite3_wal_checkpoint_v2(db, NULL, SQLITE_CHECKPOINT_PASSIVE, NULL, NULL);
|
||||
```
|
||||
|
||||
`PASSIVE` mode checkpoints only pages that are not currently being read, so it never blocks readers.
|
||||
|
||||
### Step 3 — Submit the checkpoint job on a timer
|
||||
|
||||
In the existing 60-second periodic timer in `websockets.c`, add a call to submit a WAL checkpoint job to the write queue:
|
||||
|
||||
```c
|
||||
thread_pool_submit_wal_checkpoint();
|
||||
```
|
||||
|
||||
This runs the checkpoint once per minute on the write thread, but as a **bounded, predictable** operation rather than being triggered unpredictably by every INSERT.
|
||||
|
||||
### Step 4 — Also disable auto-checkpoint on reader connections
|
||||
|
||||
In `db_open_worker_connection`, the same `wal_autocheckpoint=0` applies to reader connections too. Readers can also trigger checkpoints in SQLite; disabling it on all connections ensures only the explicit periodic job does checkpointing.
|
||||
|
||||
### Step 5 — Run a TRUNCATE checkpoint at shutdown
|
||||
|
||||
In `thread_pool_shutdown`, before closing the write connection, run:
|
||||
|
||||
```c
|
||||
sqlite3_wal_checkpoint_v2(db, NULL, SQLITE_CHECKPOINT_TRUNCATE, NULL, NULL);
|
||||
```
|
||||
|
||||
This ensures the WAL is fully flushed and truncated on clean shutdown, keeping the database file compact.
|
||||
|
||||
## Files Changed
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `src/db_ops.c` | Add `PRAGMA wal_autocheckpoint=0` in `db_open_worker_connection` |
|
||||
| `src/thread_pool.h` | Add `THREAD_POOL_JOB_WAL_CHECKPOINT` enum value |
|
||||
| `src/thread_pool.c` | Add `execute_wal_checkpoint_job` handler; add checkpoint-at-shutdown in `thread_pool_shutdown`; add `thread_pool_submit_wal_checkpoint` helper |
|
||||
| `src/websockets.c` | Call `thread_pool_submit_wal_checkpoint()` in the 60-second periodic timer |
|
||||
|
||||
## Expected Impact
|
||||
|
||||
- `db-write` CPU should drop from ~77% to near 0% when idle (no events to store)
|
||||
- Periodic checkpoint bursts of a few seconds every 60s instead of continuous burn
|
||||
- No change to data durability — WAL still protects against crashes; checkpoint just moves data from WAL to main DB file
|
||||
- Readers are unaffected — PASSIVE checkpoint never blocks them
|
||||
|
||||
## Risk
|
||||
|
||||
- If the relay crashes between checkpoints, the WAL file may be larger than before (up to 60s of accumulated writes). This is safe — SQLite replays the WAL on next open. The WAL file size is bounded by the write rate, which is low.
|
||||
- PASSIVE checkpoint may not checkpoint all pages if readers hold them. This is fine — the next checkpoint will catch up.
|
||||
|
||||
## Verification
|
||||
|
||||
After deploying, re-run the profiler:
|
||||
|
||||
```bash
|
||||
./deploy_lt_debug.sh && DURATION=300 INTERVAL=5 ./tests/thread_cpu_profile.sh
|
||||
```
|
||||
|
||||
Expected: `db-write` avg CPU drops below 5%.
|
||||
@@ -0,0 +1,298 @@
|
||||
# Web of Trust (WoT) Implementation Plan
|
||||
|
||||
## Feature Description
|
||||
|
||||
When enabled, the relay restricts access to pubkeys that the admin follows. The admin's kind 3 (contact list) event is used as the source of truth — all `p` tags in that event become whitelisted pubkeys. The admin themselves is always whitelisted.
|
||||
|
||||
Single config variable `wot_enabled` with three levels:
|
||||
|
||||
| Value | Mode | Effect |
|
||||
|-------|------|--------|
|
||||
| `0` | Off | Open relay — anyone can read and write |
|
||||
| `1` | Write-only | Only followed pubkeys can **publish** events. Anyone can read/subscribe. |
|
||||
| `2` | Full | Only followed pubkeys can **publish AND subscribe**. Requires NIP-42 auth for subscriptions. Eliminates subscription churn from anonymous connections. |
|
||||
|
||||
## How It Works
|
||||
|
||||
### WoT Sync Flow
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["Admin publishes kind 3 event\n- contact list -"] --> B{wot_enabled > 0?}
|
||||
B -->|No| C["Store event normally"]
|
||||
B -->|Yes| D["Extract p tags from kind 3"]
|
||||
D --> E["Clear existing WoT whitelist rules"]
|
||||
E --> F["Insert new whitelist rules\nfor each followed pubkey"]
|
||||
F --> G["Enable auth_enabled"]
|
||||
G --> H{wot_enabled == 2?}
|
||||
H -->|Yes| I["Enable nip42_auth_required_subscriptions"]
|
||||
H -->|No| J["Done - write-only restriction"]
|
||||
```
|
||||
|
||||
### Event Publishing Flow — Write Restriction (wot_enabled >= 1)
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["EVENT message arrives"] --> B{auth_enabled?}
|
||||
B -->|No| C["Accept event"]
|
||||
B -->|Yes| D["check_database_auth_rules - pubkey -"]
|
||||
D --> E{Pubkey in whitelist\nor wot_whitelist?}
|
||||
E -->|Yes| C
|
||||
E -->|No| F{Any whitelist rules exist?}
|
||||
F -->|Yes| G["REJECT: not whitelisted"]
|
||||
F -->|No| C
|
||||
```
|
||||
|
||||
### Subscription Flow — Read Restriction (wot_enabled == 2)
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["REQ message arrives"] --> B{wot_enabled == 2?}
|
||||
B -->|No| C["Allow subscription"]
|
||||
B -->|Yes| D{NIP-42 authenticated?}
|
||||
D -->|No| E["Send AUTH challenge"]
|
||||
D -->|Yes| F["check_database_auth_rules\nusing authenticated_pubkey"]
|
||||
F --> G{Pubkey in whitelist\nor wot_whitelist?}
|
||||
G -->|Yes| C
|
||||
G -->|No| H["REJECT: not authorized\nfor subscriptions"]
|
||||
```
|
||||
|
||||
## Design Decisions
|
||||
|
||||
### Leveraging Existing Infrastructure
|
||||
|
||||
The relay already has everything needed:
|
||||
|
||||
1. **`auth_rules` table** — stores whitelist/blacklist rules with `rule_type`, `pattern_type`, `pattern_value`
|
||||
2. **`check_database_auth_rules()`** in [`request_validator.c:529`](src/request_validator.c:529) — already implements the logic: "if whitelist rules exist and pubkey is not whitelisted, deny"
|
||||
3. **`add_auth_rule_from_config()`** / **`remove_auth_rule_from_config()`** in [`config.c`](src/config.c:2082) — already manage auth rules
|
||||
4. **`event_tags` table** — can efficiently query `p` tags from kind 3 events
|
||||
5. **Config system** — `auth_enabled` flag already controls whether auth rules are checked
|
||||
6. **NIP-42 auth** — `nip42_auth_required_subscriptions` already gates REQ access, `pss->authenticated_pubkey` stores the authenticated pubkey
|
||||
|
||||
### WoT-Specific Whitelist Rules
|
||||
|
||||
To distinguish WoT-generated whitelist rules from manually-added ones, we use a new `rule_type` value: `wot_whitelist`. This allows:
|
||||
- Clearing all WoT rules without affecting manual whitelist/blacklist rules
|
||||
- Querying WoT status separately
|
||||
- The existing `check_database_auth_rules()` function already checks for `rule_type = 'whitelist'` — we need to also match `wot_whitelist` in the whitelist check
|
||||
|
||||
### Trigger Mechanism
|
||||
|
||||
The WoT sync happens when:
|
||||
1. **A kind 3 event from the admin is stored** — detected in `store_event()` after successful INSERT
|
||||
2. **Startup** — if `wot_enabled > 0`, sync from the most recent admin kind 3 event in the database
|
||||
3. **Admin DM command** — `wot sync` to force a manual resync
|
||||
|
||||
### What Gets Whitelisted
|
||||
|
||||
- All pubkeys in `p` tags of the admin's kind 3 event
|
||||
- The admin pubkey itself (always whitelisted)
|
||||
- The relay pubkey (always whitelisted — for admin DM responses)
|
||||
|
||||
### What Is NOT Blocked
|
||||
|
||||
Even with WoT enabled, these are always allowed:
|
||||
- Admin events (kind 23456) — already bypassed in [`request_validator.c:303`](src/request_validator.c:303)
|
||||
- NIP-42 auth events (kind 22242) — already bypassed in [`request_validator.c:318`](src/request_validator.c:318)
|
||||
- Kind 3 events from the admin — needed to update the follow list itself
|
||||
- Kind 1059 gift wraps addressed to the relay — needed for admin DMs
|
||||
|
||||
## Files to Modify
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `src/default_config_event.h` | Add `wot_enabled` config key (default: `0`) |
|
||||
| `src/config.c` | Add `wot_sync_from_admin_kind3()` function |
|
||||
| `src/main.c` | Add WoT trigger in `store_event()` when admin kind 3 is stored, add startup WoT sync |
|
||||
| `src/request_validator.c` | Update whitelist SQL to also match `wot_whitelist` rule type |
|
||||
| `src/sql_schema.h` | Update `auth_rules` CHECK constraint to include `wot_whitelist` |
|
||||
| `src/websockets.c` | Add WoT pubkey check after NIP-42 auth check in REQ handler |
|
||||
| `src/dm_admin.c` | Add `wot 0`, `wot 1`, `wot 2`, `wot sync`, `wot status` DM commands |
|
||||
|
||||
## Detailed Changes
|
||||
|
||||
### 1. Schema: `src/sql_schema.h`
|
||||
|
||||
Update the `auth_rules` table CHECK constraint to allow `wot_whitelist`:
|
||||
|
||||
```sql
|
||||
-- Before:
|
||||
rule_type TEXT NOT NULL CHECK (rule_type IN ('whitelist', 'blacklist', 'rate_limit', 'auth_required'))
|
||||
|
||||
-- After:
|
||||
rule_type TEXT NOT NULL CHECK (rule_type IN ('whitelist', 'blacklist', 'rate_limit', 'auth_required', 'wot_whitelist'))
|
||||
```
|
||||
|
||||
### 2. Config: `src/default_config_event.h`
|
||||
|
||||
Add WoT configuration:
|
||||
|
||||
```c
|
||||
// Web of Trust Settings
|
||||
// 0 = off, 1 = write-only (followed pubkeys can publish), 2 = full (followed pubkeys can publish AND subscribe)
|
||||
{"wot_enabled", "0"},
|
||||
```
|
||||
|
||||
### 3. Core WoT Sync Function: `src/config.c`
|
||||
|
||||
New function `wot_sync_from_admin_kind3()`:
|
||||
|
||||
```c
|
||||
int wot_sync_from_admin_kind3(void) {
|
||||
int wot_level = get_config_int("wot_enabled", 0);
|
||||
if (wot_level <= 0) return 0; // WoT disabled
|
||||
|
||||
// 1. Get admin pubkey from config
|
||||
// 2. Query event_tags for p tags from admin's latest kind 3 event:
|
||||
// SELECT DISTINCT et.tag_value FROM event_tags et
|
||||
// JOIN events e ON et.event_id = e.id
|
||||
// WHERE e.kind = 3 AND e.pubkey = ? AND et.tag_name = 'p'
|
||||
// ORDER BY e.created_at DESC
|
||||
// 3. BEGIN TRANSACTION
|
||||
// 4. DELETE FROM auth_rules WHERE rule_type = 'wot_whitelist'
|
||||
// 5. INSERT wot_whitelist for admin pubkey
|
||||
// 6. INSERT wot_whitelist for relay pubkey
|
||||
// 7. For each p tag: INSERT INTO auth_rules (rule_type, pattern_type, pattern_value)
|
||||
// VALUES ('wot_whitelist', 'pubkey', ?)
|
||||
// 8. COMMIT
|
||||
// 9. Set auth_enabled = true
|
||||
// 10. If wot_level == 2: set nip42_auth_required_subscriptions = true
|
||||
// 11. Log count of whitelisted pubkeys
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Trigger on Kind 3 Store: `src/main.c`
|
||||
|
||||
In `store_event()`, after successful INSERT and after `store_event_tags()`:
|
||||
|
||||
```c
|
||||
// Check if this is a kind 3 event from the admin — trigger WoT sync
|
||||
if ((int)cJSON_GetNumberValue(kind) == 3) {
|
||||
int wot_level = get_config_int("wot_enabled", 0);
|
||||
if (wot_level > 0) {
|
||||
const char* admin_pubkey = get_config_value("admin_pubkey");
|
||||
if (admin_pubkey && strcmp(cJSON_GetStringValue(pubkey), admin_pubkey) == 0) {
|
||||
DEBUG_INFO("Admin kind 3 event stored — triggering WoT sync");
|
||||
wot_sync_from_admin_kind3();
|
||||
}
|
||||
if (admin_pubkey) free((char*)admin_pubkey);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Startup WoT Sync: `src/main.c`
|
||||
|
||||
In `main()`, after `populate_event_tags_from_existing()`:
|
||||
|
||||
```c
|
||||
// Sync Web of Trust whitelist if enabled
|
||||
int wot_level = get_config_int("wot_enabled", 0);
|
||||
if (wot_level > 0) {
|
||||
wot_sync_from_admin_kind3();
|
||||
}
|
||||
```
|
||||
|
||||
### 6. Update Whitelist Check: `src/request_validator.c`
|
||||
|
||||
Update the whitelist SQL queries to also match `wot_whitelist`:
|
||||
|
||||
```c
|
||||
// Before:
|
||||
"SELECT rule_type FROM auth_rules WHERE rule_type = 'whitelist' AND pattern_type = 'pubkey' AND pattern_value = ? AND active = 1 LIMIT 1"
|
||||
|
||||
// After:
|
||||
"SELECT rule_type FROM auth_rules WHERE rule_type IN ('whitelist', 'wot_whitelist') AND pattern_type = 'pubkey' AND pattern_value = ? AND active = 1 LIMIT 1"
|
||||
```
|
||||
|
||||
And the whitelist-exists check:
|
||||
```c
|
||||
// Before:
|
||||
"SELECT COUNT(*) FROM auth_rules WHERE rule_type = 'whitelist' AND pattern_type = 'pubkey' AND active = 1 LIMIT 1"
|
||||
|
||||
// After:
|
||||
"SELECT COUNT(*) FROM auth_rules WHERE rule_type IN ('whitelist', 'wot_whitelist') AND pattern_type = 'pubkey' AND active = 1 LIMIT 1"
|
||||
```
|
||||
|
||||
### 7. REQ Handler WoT Check: `src/websockets.c`
|
||||
|
||||
When `wot_enabled == 2`, add a pubkey whitelist check **after** the existing NIP-42 auth check in the REQ handler. The existing code at line 903 already requires NIP-42 auth when `nip42_auth_required_subscriptions` is set. We add a WoT check right after:
|
||||
|
||||
```c
|
||||
// Existing NIP-42 auth check (line 903):
|
||||
if (pss && pss->nip42_auth_required_subscriptions && !pss->authenticated) {
|
||||
// ... send AUTH challenge or NOTICE ...
|
||||
return 0;
|
||||
}
|
||||
|
||||
// NEW: WoT read restriction check (wot_enabled == 2)
|
||||
if (pss && pss->authenticated && get_config_int("wot_enabled", 0) == 2) {
|
||||
// Client is authenticated — check if their pubkey is in the WoT whitelist
|
||||
int wot_result = check_database_auth_rules(pss->authenticated_pubkey, "subscription", NULL);
|
||||
if (wot_result != NOSTR_SUCCESS) {
|
||||
send_notice_message(wsi, pss, "restricted: your pubkey is not in this relay's web of trust");
|
||||
DEBUG_INFO("REQ rejected: pubkey %s not in WoT whitelist", pss->authenticated_pubkey);
|
||||
cJSON_Delete(json);
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 8. Admin DM Commands: `src/dm_admin.c`
|
||||
|
||||
Add plain text DM commands:
|
||||
|
||||
| Command | Action |
|
||||
|---------|--------|
|
||||
| `wot 0` or `wot off` | Disable WoT, delete all `wot_whitelist` rules, reset `nip42_auth_required_subscriptions` |
|
||||
| `wot 1` or `wot write` | Write-only WoT — sync follow list, set `auth_enabled=true` |
|
||||
| `wot 2` or `wot full` | Full WoT — sync follow list, set `auth_enabled=true`, set `nip42_auth_required_subscriptions=true` |
|
||||
| `wot sync` | Force resync from admin's kind 3 event |
|
||||
| `wot status` | Show WoT level (0/1/2), count of whitelisted pubkeys |
|
||||
|
||||
### 9. NIP-11 Update
|
||||
|
||||
When WoT is enabled (level 1 or 2), the relay should indicate this in NIP-11 relay info. Add to the `limitation` object:
|
||||
```json
|
||||
"auth_required": true
|
||||
```
|
||||
|
||||
## Edge Cases
|
||||
|
||||
1. **Admin has no kind 3 event in database**: WoT sync does nothing, logs a warning. No whitelist rules created = open relay.
|
||||
|
||||
2. **Admin updates follow list**: New kind 3 event triggers full resync — old WoT rules are cleared, new ones inserted. This is atomic (transaction).
|
||||
|
||||
3. **WoT disabled (set to 0)**: Clears all `wot_whitelist` rules. Manual `whitelist` rules are preserved. `nip42_auth_required_subscriptions` is reset to false.
|
||||
|
||||
4. **Admin unfollows someone**: Next kind 3 event triggers resync, the unfollowed pubkey's `wot_whitelist` rule is removed (full clear + reinsert).
|
||||
|
||||
5. **Kind 1059 gift wraps**: These need special handling — the relay needs to accept gift wraps addressed to it even from non-whitelisted pubkeys (for admin DMs). The `is_nip17_gift_wrap_for_relay()` check should bypass WoT.
|
||||
|
||||
6. **Read restriction without NIP-42 support (level 2)**: If a client doesn't support NIP-42, they cannot authenticate and therefore cannot subscribe. This is by design — it's the most effective way to reduce subscription churn from anonymous connections.
|
||||
|
||||
7. **NIP-11 discovery**: Clients can check NIP-11 to see if `auth_required` is true before connecting, avoiding wasted connections.
|
||||
|
||||
8. **Changing level from 2 to 1**: `nip42_auth_required_subscriptions` is reset to false, allowing anonymous subscriptions again. WoT whitelist rules remain for write restriction.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
1. Enable write-only WoT via DM: `wot 1`
|
||||
2. Verify admin can still publish events
|
||||
3. Verify followed pubkeys can publish events
|
||||
4. Verify non-followed pubkeys are rejected for EVENT
|
||||
5. Verify non-followed pubkeys can still subscribe (REQ)
|
||||
6. Enable full WoT via DM: `wot 2`
|
||||
7. Verify unauthenticated clients get AUTH challenge on REQ
|
||||
8. Verify authenticated non-followed pubkeys are rejected for REQ
|
||||
9. Verify authenticated followed pubkeys can subscribe
|
||||
10. Verify `wot status` shows correct level and count
|
||||
11. Publish new kind 3 event, verify auto-resync
|
||||
12. `wot 0` — verify all pubkeys can publish and subscribe again
|
||||
13. Verify admin DMs still work when WoT is enabled
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
- WoT sync is O(n) where n = number of follows (typically 100-2000)
|
||||
- Uses a transaction for bulk insert — fast even for large follow lists
|
||||
- Auth rule check is already indexed: `idx_auth_rules_pattern ON auth_rules(pattern_type, pattern_value)`
|
||||
- No additional per-event overhead — the existing `check_database_auth_rules()` already runs on every event when auth is enabled
|
||||
- **Level 2 directly addresses CPU load**: With `wot_enabled=2`, unauthenticated connections cannot create subscriptions, eliminating the ~120 REQ/minute churn from anonymous connections that was causing 99% CPU
|
||||
@@ -0,0 +1,388 @@
|
||||
# Web of Trust — Admin Web UI Plan
|
||||
|
||||
## Overview
|
||||
|
||||
Add a **Web of Trust** section to the existing Authorization page in the admin web interface. The section lets the admin see whether their kind 3 contact list is on the relay, select a WoT level, trigger a sync, and view the current whitelist count.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Data Flow
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A[Web UI] -->|config_set wot_enabled N| B[Kind 23456 Event]
|
||||
A -->|system_command wot_sync| B
|
||||
A -->|system_command wot_status| B
|
||||
B --> C[Relay Backend]
|
||||
C -->|Kind 23457 Response| D[Web UI updates display]
|
||||
```
|
||||
|
||||
### How It Works
|
||||
|
||||
1. **On page load** — the Authorization page already calls `loadAuthRules`. We add a parallel call to `loadWotStatus` which sends a `system_command` / `wot_status` event.
|
||||
2. **Backend responds** with a kind 23457 JSON containing: `wot_enabled` level, `wot_whitelist_count`, and `admin_kind3_exists` boolean.
|
||||
3. **UI renders** a card showing:
|
||||
- Whether the admin's kind 3 event exists on the relay
|
||||
- Current WoT level as a 3-option radio/button group
|
||||
- Count of whitelisted pubkeys
|
||||
- A SYNC button to force resync
|
||||
4. **Changing level** sends `config_set` / `wot_enabled` / `0|1|2` via the existing admin API, then triggers `wot_sync` if level > 0.
|
||||
5. **SYNC button** sends `system_command` / `wot_sync`.
|
||||
|
||||
## Detailed Changes
|
||||
|
||||
### 1. Backend: New system commands in `src/config.c`
|
||||
|
||||
Add two new branches to `handle_system_command_unified`:
|
||||
|
||||
#### `wot_status` command
|
||||
|
||||
Returns JSON response:
|
||||
```json
|
||||
{
|
||||
"command": "wot_status",
|
||||
"status": "success",
|
||||
"wot_enabled": 2,
|
||||
"admin_kind3_exists": true,
|
||||
"wot_whitelist_count": 347,
|
||||
"timestamp": 1234567890
|
||||
}
|
||||
```
|
||||
|
||||
Implementation:
|
||||
- Read `wot_enabled` from config table
|
||||
- Query `SELECT COUNT(*) FROM events WHERE kind = 3 AND pubkey = ?` with admin_pubkey to check kind 3 existence
|
||||
- Query `SELECT COUNT(*) FROM auth_rules WHERE rule_type = 'wot_whitelist' AND active = 1` for whitelist count
|
||||
- Return as kind 23457 signed response
|
||||
|
||||
#### `wot_sync` command
|
||||
|
||||
Calls `wot_sync_from_admin_kind3` and returns result:
|
||||
```json
|
||||
{
|
||||
"command": "wot_sync",
|
||||
"status": "success",
|
||||
"wot_whitelist_count": 347,
|
||||
"timestamp": 1234567890
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Backend: Hook `config_set` for `wot_enabled`
|
||||
|
||||
In `handle_config_set_unified`, after the config value is updated, add a check:
|
||||
```c
|
||||
// After successful update, trigger WoT sync if wot_enabled changed
|
||||
if (strcmp(config_key, "wot_enabled") == 0) {
|
||||
int new_level = atoi(config_value);
|
||||
if (new_level > 0) {
|
||||
wot_sync_from_admin_kind3();
|
||||
} else {
|
||||
// Level 0: clear wot_whitelist rules and reset auth flags
|
||||
sqlite3_exec(g_db, "DELETE FROM auth_rules WHERE rule_type = 'wot_whitelist'", NULL, NULL, NULL);
|
||||
update_config_in_table("nip42_auth_required_subscriptions", "false");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Frontend: HTML — WoT section in `api/index.html`
|
||||
|
||||
Add a new div **inside** the `authRulesSection`, before the auth rules table. This keeps WoT visually grouped with authorization:
|
||||
|
||||
```html
|
||||
<!-- Web of Trust Section -->
|
||||
<div id="wotSection" class="input-group">
|
||||
<div class="section-header" style="font-size: 14px; margin-bottom: 10px;">
|
||||
WEB OF TRUST
|
||||
</div>
|
||||
|
||||
<!-- Kind 3 Status Indicator -->
|
||||
<div id="wotKind3Status" class="wot-status-row">
|
||||
<span>Admin Contact List (kind 3):</span>
|
||||
<span id="wotKind3Indicator" class="wot-indicator wot-unknown">Checking...</span>
|
||||
</div>
|
||||
|
||||
<!-- WoT Level Selector -->
|
||||
<div class="wot-level-selector">
|
||||
<label>WoT Level:</label>
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="wotLevel0Btn" class="wot-level-btn" onclick="setWotLevel(0)">
|
||||
OFF
|
||||
</button>
|
||||
<button type="button" id="wotLevel1Btn" class="wot-level-btn" onclick="setWotLevel(1)">
|
||||
WRITE ONLY
|
||||
</button>
|
||||
<button type="button" id="wotLevel2Btn" class="wot-level-btn" onclick="setWotLevel(2)">
|
||||
FULL
|
||||
</button>
|
||||
</div>
|
||||
<div class="wot-level-description" id="wotLevelDescription">
|
||||
Level 0: Open relay — anyone can read and write
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- WoT Stats -->
|
||||
<div class="wot-stats-row">
|
||||
<span>Whitelisted Pubkeys:</span>
|
||||
<span id="wotWhitelistCount">—</span>
|
||||
</div>
|
||||
|
||||
<!-- Sync Button -->
|
||||
<div class="inline-buttons">
|
||||
<button type="button" id="wotSyncBtn" onclick="syncWot()">SYNC FROM KIND 3</button>
|
||||
<button type="button" id="wotRefreshBtn" onclick="loadWotStatus()">REFRESH STATUS</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<hr style="margin: 15px 0; border-color: var(--border-color);">
|
||||
```
|
||||
|
||||
### 4. Frontend: CSS additions in `api/index.css`
|
||||
|
||||
```css
|
||||
/* Web of Trust styles */
|
||||
.wot-status-row, .wot-stats-row {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: 8px 0;
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.wot-indicator {
|
||||
padding: 2px 10px;
|
||||
border-radius: 4px;
|
||||
font-weight: bold;
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
.wot-indicator.wot-found { background: #28a745; color: white; }
|
||||
.wot-indicator.wot-missing { background: #dc3545; color: white; }
|
||||
.wot-indicator.wot-unknown { background: #6c757d; color: white; }
|
||||
|
||||
.wot-level-selector { padding: 10px 0; }
|
||||
.wot-level-selector label { display: block; margin-bottom: 5px; font-weight: bold; }
|
||||
|
||||
.wot-level-btn { min-width: 100px; }
|
||||
.wot-level-btn.active {
|
||||
background: var(--accent-color, #007bff);
|
||||
color: white;
|
||||
border-color: var(--accent-color, #007bff);
|
||||
}
|
||||
|
||||
.wot-level-description {
|
||||
font-size: 12px;
|
||||
color: var(--text-secondary);
|
||||
margin-top: 5px;
|
||||
font-style: italic;
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Frontend: JavaScript functions in `api/index.js`
|
||||
|
||||
#### `loadWotStatus` — query current WoT state
|
||||
|
||||
```javascript
|
||||
async function loadWotStatus() {
|
||||
try {
|
||||
if (!isLoggedIn || !userPubkey || !relayPool) return;
|
||||
|
||||
const command_array = ["system_command", "wot_status"];
|
||||
const encrypted_content = await encryptForRelay(JSON.stringify(command_array));
|
||||
if (!encrypted_content) return;
|
||||
|
||||
const event = {
|
||||
kind: 23456,
|
||||
pubkey: userPubkey,
|
||||
created_at: Math.floor(Date.now() / 1000),
|
||||
tags: [["p", getRelayPubkey()]],
|
||||
content: encrypted_content
|
||||
};
|
||||
|
||||
const signedEvent = await window.nostr.signEvent(event);
|
||||
const url = relayConnectionUrl.value.trim();
|
||||
await relayPool.publish([url], signedEvent);
|
||||
|
||||
log('WoT status query sent', 'INFO');
|
||||
} catch (error) {
|
||||
log('Failed to load WoT status: ' + error.message, 'ERROR');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### `setWotLevel` — change WoT level via config_set
|
||||
|
||||
```javascript
|
||||
async function setWotLevel(level) {
|
||||
try {
|
||||
if (!isLoggedIn || !userPubkey || !relayPool) return;
|
||||
|
||||
// Send config_set for wot_enabled
|
||||
const command_array = ["config_set", "wot_enabled", String(level)];
|
||||
const encrypted_content = await encryptForRelay(JSON.stringify(command_array));
|
||||
if (!encrypted_content) return;
|
||||
|
||||
const event = {
|
||||
kind: 23456,
|
||||
pubkey: userPubkey,
|
||||
created_at: Math.floor(Date.now() / 1000),
|
||||
tags: [["p", getRelayPubkey()]],
|
||||
content: encrypted_content
|
||||
};
|
||||
|
||||
const signedEvent = await window.nostr.signEvent(event);
|
||||
const url = relayConnectionUrl.value.trim();
|
||||
await relayPool.publish([url], signedEvent);
|
||||
|
||||
log('WoT level set to ' + level, 'INFO');
|
||||
|
||||
// Refresh status after a short delay
|
||||
setTimeout(() => loadWotStatus(), 1500);
|
||||
} catch (error) {
|
||||
log('Failed to set WoT level: ' + error.message, 'ERROR');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### `syncWot` — force WoT sync
|
||||
|
||||
```javascript
|
||||
async function syncWot() {
|
||||
try {
|
||||
if (!isLoggedIn || !userPubkey || !relayPool) return;
|
||||
|
||||
const command_array = ["system_command", "wot_sync"];
|
||||
const encrypted_content = await encryptForRelay(JSON.stringify(command_array));
|
||||
if (!encrypted_content) return;
|
||||
|
||||
const event = {
|
||||
kind: 23456,
|
||||
pubkey: userPubkey,
|
||||
created_at: Math.floor(Date.now() / 1000),
|
||||
tags: [["p", getRelayPubkey()]],
|
||||
content: encrypted_content
|
||||
};
|
||||
|
||||
const signedEvent = await window.nostr.signEvent(event);
|
||||
const url = relayConnectionUrl.value.trim();
|
||||
await relayPool.publish([url], signedEvent);
|
||||
|
||||
log('WoT sync command sent', 'INFO');
|
||||
|
||||
// Refresh status after sync completes
|
||||
setTimeout(() => loadWotStatus(), 2000);
|
||||
} catch (error) {
|
||||
log('Failed to sync WoT: ' + error.message, 'ERROR');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### `handleWotStatusResponse` — process backend response
|
||||
|
||||
```javascript
|
||||
function handleWotStatusResponse(responseData) {
|
||||
const kind3Indicator = document.getElementById('wotKind3Indicator');
|
||||
const whitelistCount = document.getElementById('wotWhitelistCount');
|
||||
const levelDesc = document.getElementById('wotLevelDescription');
|
||||
|
||||
// Update kind 3 indicator
|
||||
if (kind3Indicator) {
|
||||
if (responseData.admin_kind3_exists) {
|
||||
kind3Indicator.textContent = 'Found ✓';
|
||||
kind3Indicator.className = 'wot-indicator wot-found';
|
||||
} else {
|
||||
kind3Indicator.textContent = 'Not Found ✗';
|
||||
kind3Indicator.className = 'wot-indicator wot-missing';
|
||||
}
|
||||
}
|
||||
|
||||
// Update whitelist count
|
||||
if (whitelistCount) {
|
||||
whitelistCount.textContent = responseData.wot_whitelist_count || 0;
|
||||
}
|
||||
|
||||
// Update level buttons
|
||||
const level = responseData.wot_enabled || 0;
|
||||
['wotLevel0Btn', 'wotLevel1Btn', 'wotLevel2Btn'].forEach((id, i) => {
|
||||
const btn = document.getElementById(id);
|
||||
if (btn) {
|
||||
btn.classList.toggle('active', i === level);
|
||||
}
|
||||
});
|
||||
|
||||
// Update description
|
||||
const descriptions = [
|
||||
'Level 0: Open relay — anyone can read and write',
|
||||
'Level 1: Write-only — only followed pubkeys can publish events',
|
||||
'Level 2: Full — only followed pubkeys can publish AND subscribe (NIP-42 required)'
|
||||
];
|
||||
if (levelDesc) {
|
||||
levelDesc.textContent = descriptions[level] || descriptions[0];
|
||||
}
|
||||
|
||||
// Enable/disable sync button based on kind 3 existence
|
||||
const syncBtn = document.getElementById('wotSyncBtn');
|
||||
if (syncBtn) {
|
||||
syncBtn.disabled = !responseData.admin_kind3_exists;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Wire into existing response handler
|
||||
|
||||
In the existing `handleSystemCommandResponse` function, add:
|
||||
```javascript
|
||||
if (responseData.command === 'wot_status' || responseData.command === 'wot_sync') {
|
||||
handleWotStatusResponse(responseData);
|
||||
}
|
||||
```
|
||||
|
||||
#### Wire into page navigation
|
||||
|
||||
In the `authorization` case of the page switch handler, add:
|
||||
```javascript
|
||||
case 'authorization':
|
||||
loadAuthRules().catch(...);
|
||||
loadWotStatus().catch(error => {
|
||||
console.log('Auto-load WoT status failed: ' + error.message);
|
||||
});
|
||||
break;
|
||||
```
|
||||
|
||||
## Files to Modify
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `src/config.c` | Add `wot_status` and `wot_sync` branches to `handle_system_command_unified`; add WoT sync hook to `handle_config_set_unified` |
|
||||
| `api/index.html` | Add WoT section HTML inside `authRulesSection` |
|
||||
| `api/index.css` | Add WoT-specific CSS styles |
|
||||
| `api/index.js` | Add `loadWotStatus`, `setWotLevel`, `syncWot`, `handleWotStatusResponse` functions; wire into page nav and response handler |
|
||||
|
||||
## Edge Cases
|
||||
|
||||
1. **Admin not logged in** — WoT section shows but buttons are disabled; status shows "Checking..."
|
||||
2. **No kind 3 event** — Indicator shows red "Not Found ✗"; SYNC button is disabled; level selector still works but sync will whitelist only admin + relay
|
||||
3. **Level change from 2 to 0** — Backend clears all wot_whitelist rules and resets nip42_auth_required_subscriptions
|
||||
4. **Concurrent config_set and wot_sync** — The backend handles these sequentially via SQLite transactions; no race condition
|
||||
|
||||
## UI Mockup
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ WEB OF TRUST │
|
||||
│ │
|
||||
│ Admin Contact List (kind 3): [Found ✓] │
|
||||
│ │
|
||||
│ WoT Level: │
|
||||
│ [ OFF ] [ WRITE ONLY ] [ FULL ] │
|
||||
│ Level 2: Full — only followed pubkeys can │
|
||||
│ publish AND subscribe (NIP-42 required) │
|
||||
│ │
|
||||
│ Whitelisted Pubkeys: 347 │
|
||||
│ │
|
||||
│ [ SYNC FROM KIND 3 ] [ REFRESH STATUS ] │
|
||||
├─────────────────────────────────────────────┤
|
||||
│ ─────────────────────────────────────────── │
|
||||
│ AUTH RULES MANAGEMENT │
|
||||
│ ... existing auth rules table ... │
|
||||
└─────────────────────────────────────────────┘
|
||||
```
|
||||
@@ -0,0 +1,10 @@
|
||||
https://github.com/rushmi0/Fenrir-s
|
||||
https://github.com/barkyq/gnost-relay
|
||||
https://github.com/lpicanco/knostr
|
||||
https://github.com/bezysoftware/netstr
|
||||
https://github.com/lebrunel/nex
|
||||
https://github.com/CodyTseng/nostr-relay-nestjs
|
||||
https://github.com/mattn/nostr-relay
|
||||
https://github.com/Cameri/nostream
|
||||
https://github.com/Giszmo/NostrPostr/tree/master/NostrRelay
|
||||
https://github.com/fiatjaf/relayer/tree/master/examples/basic
|
||||
+4937
File diff suppressed because it is too large
Load Diff
@@ -64,6 +64,12 @@ void monitoring_on_event_stored(void);
|
||||
void monitoring_on_subscription_change(void);
|
||||
int get_monitoring_throttle_seconds(void);
|
||||
|
||||
// Dedicated API worker thread lifecycle
|
||||
int start_api_worker(void);
|
||||
void stop_api_worker(void);
|
||||
void api_worker_process_completions(void);
|
||||
int api_worker_enqueue_status_post(void);
|
||||
|
||||
// Kind 1 status posts
|
||||
int generate_and_post_status_event(void);
|
||||
|
||||
|
||||
+1603
-686
File diff suppressed because it is too large
Load Diff
+24
-4
@@ -1,7 +1,6 @@
|
||||
#ifndef CONFIG_H
|
||||
#define CONFIG_H
|
||||
|
||||
#include <sqlite3.h>
|
||||
#include <cjson/cJSON.h>
|
||||
#include <time.h>
|
||||
#include <pthread.h>
|
||||
@@ -16,6 +15,9 @@ struct lws;
|
||||
#define RELAY_URL_MAX_LENGTH 512
|
||||
#define RELAY_PUBKEY_MAX_LENGTH 65
|
||||
#define RELAY_CONTACT_MAX_LENGTH 256
|
||||
#define MAX_ADMIN_PUBKEYS 10
|
||||
#define ADMIN_PUBKEY_HEX_LEN 64
|
||||
#define ADMIN_PUBKEY_LIST_MAX_LENGTH ((MAX_ADMIN_PUBKEYS * ADMIN_PUBKEY_HEX_LEN) + (MAX_ADMIN_PUBKEYS - 1) + 1)
|
||||
#define SUBSCRIPTION_ID_MAX_LENGTH 64
|
||||
#define CLIENT_IP_MAX_LENGTH 46
|
||||
#define MAX_SUBSCRIPTIONS_PER_CLIENT 25
|
||||
@@ -30,10 +32,16 @@ extern char g_database_path[512];
|
||||
// Command line options structure for first-time startup
|
||||
typedef struct {
|
||||
int port_override; // -1 = not set, >0 = port value
|
||||
char admin_pubkey_override[65]; // Empty string = not set, 64-char hex = override
|
||||
char admin_pubkey_override[ADMIN_PUBKEY_LIST_MAX_LENGTH]; // Empty string = not set, comma-separated 64-char hex pubkeys
|
||||
char relay_privkey_override[65]; // Empty string = not set, 64-char hex = override
|
||||
int strict_port; // 0 = allow port increment, 1 = fail if exact port unavailable
|
||||
int debug_level; // 0-5, default 0 (no debug output)
|
||||
char db_connstring_override[1024];
|
||||
char db_host[128];
|
||||
int db_port; // 0 = not set
|
||||
char db_name[128];
|
||||
char db_user[128];
|
||||
char db_password[128];
|
||||
} cli_options_t;
|
||||
|
||||
// Core configuration functions (temporary compatibility)
|
||||
@@ -62,7 +70,7 @@ int get_config_int(const char* key, int default_value);
|
||||
int get_config_bool(const char* key, int default_value);
|
||||
|
||||
// First-time startup functions
|
||||
int is_first_time_startup(void);
|
||||
int is_first_time_startup(const char* db_connection_target);
|
||||
int first_time_startup_sequence(const cli_options_t* cli_options, char* admin_pubkey_out, char* relay_pubkey_out, char* relay_privkey_out);
|
||||
int startup_existing_relay(const char* relay_pubkey, const cli_options_t* cli_options);
|
||||
|
||||
@@ -78,6 +86,7 @@ char* extract_pubkey_from_filename(const char* filename);
|
||||
int store_relay_private_key(const char* relay_privkey_hex);
|
||||
char* get_relay_private_key(void);
|
||||
const char* get_temp_relay_private_key(void); // For first-time startup only
|
||||
int preload_relay_private_key_cache(void); // Preload relay private key before strict DB mode
|
||||
|
||||
// NIP-42 authentication configuration functions
|
||||
int parse_auth_required_kinds(const char* kinds_str, int* kinds_array, int max_kinds);
|
||||
@@ -116,7 +125,10 @@ cJSON* build_query_response(const char* query_type, cJSON* results_array, int to
|
||||
int add_auth_rule_from_config(const char* rule_type, const char* pattern_type,
|
||||
const char* pattern_value);
|
||||
int remove_auth_rule_from_config(const char* rule_type, const char* pattern_type,
|
||||
const char* pattern_value);
|
||||
const char* pattern_value);
|
||||
|
||||
// Web of Trust (WoT) sync function
|
||||
int wot_sync_from_admin_kind3(void);
|
||||
|
||||
// Unified configuration cache management
|
||||
void force_config_cache_refresh(void);
|
||||
@@ -125,6 +137,14 @@ const char* get_relay_pubkey_cached(void);
|
||||
void invalidate_config_cache(void);
|
||||
int reload_config_from_table(void);
|
||||
|
||||
// Multi-admin helpers
|
||||
int reload_admin_pubkeys_cache(void);
|
||||
int is_admin_pubkey(const char* pubkey);
|
||||
int get_admin_pubkey_count(void);
|
||||
const char* get_primary_admin_pubkey(void);
|
||||
int add_admin_pubkey(const char* pubkey);
|
||||
int remove_admin_pubkey(const char* pubkey);
|
||||
|
||||
// Hybrid config access functions
|
||||
const char* get_config_value_hybrid(const char* key);
|
||||
int is_config_table_ready(void);
|
||||
|
||||
+315
@@ -0,0 +1,315 @@
|
||||
#define _GNU_SOURCE
|
||||
|
||||
#include "db_ops.h"
|
||||
#include "sqlite_db_ops.h"
|
||||
#include "db_ops_postgres.h"
|
||||
|
||||
#ifdef DB_BACKEND_POSTGRES
|
||||
|
||||
int db_init(const char* connection_string) { return postgres_db_init(connection_string); }
|
||||
void db_close(void) { postgres_db_close(); }
|
||||
int db_is_available(void) { return postgres_db_is_available(); }
|
||||
const char* db_last_error(void) { return postgres_db_last_error(); }
|
||||
const char* db_get_database_path(void) { return postgres_db_get_database_path(); }
|
||||
|
||||
int db_set_thread_connection(void* connection) { return postgres_db_set_thread_connection(connection); }
|
||||
void db_clear_thread_connection(void) { postgres_db_clear_thread_connection(); }
|
||||
|
||||
int db_open_worker_connection(const char* db_path, void** out_connection) {
|
||||
return postgres_db_open_worker_connection(db_path, out_connection);
|
||||
}
|
||||
void db_close_worker_connection(void* connection) { postgres_db_close_worker_connection(connection); }
|
||||
|
||||
int db_prepare(const char* sql, db_stmt_t** out_stmt) {
|
||||
return postgres_db_prepare(sql, (postgres_db_stmt_t**)out_stmt);
|
||||
}
|
||||
int db_bind_text_param(db_stmt_t* stmt, int index, const char* value) {
|
||||
return postgres_db_bind_text_param((postgres_db_stmt_t*)stmt, index, value);
|
||||
}
|
||||
int db_bind_int_param(db_stmt_t* stmt, int index, int value) {
|
||||
return postgres_db_bind_int_param((postgres_db_stmt_t*)stmt, index, value);
|
||||
}
|
||||
int db_bind_int64_param(db_stmt_t* stmt, int index, long long value) {
|
||||
return postgres_db_bind_int64_param((postgres_db_stmt_t*)stmt, index, value);
|
||||
}
|
||||
int db_step_stmt(db_stmt_t* stmt) { return postgres_db_step_stmt((postgres_db_stmt_t*)stmt); }
|
||||
int db_reset_stmt(db_stmt_t* stmt) { return postgres_db_reset_stmt((postgres_db_stmt_t*)stmt); }
|
||||
const char* db_column_text_value(db_stmt_t* stmt, int col) {
|
||||
return postgres_db_column_text_value((postgres_db_stmt_t*)stmt, col);
|
||||
}
|
||||
int db_column_int_value(db_stmt_t* stmt, int col) {
|
||||
return postgres_db_column_int_value((postgres_db_stmt_t*)stmt, col);
|
||||
}
|
||||
long long db_column_int64_value(db_stmt_t* stmt, int col) {
|
||||
return postgres_db_column_int64_value((postgres_db_stmt_t*)stmt, col);
|
||||
}
|
||||
double db_column_double_value(db_stmt_t* stmt, int col) {
|
||||
return postgres_db_column_double_value((postgres_db_stmt_t*)stmt, col);
|
||||
}
|
||||
void db_finalize_stmt(db_stmt_t* stmt) { postgres_db_finalize_stmt((postgres_db_stmt_t*)stmt); }
|
||||
|
||||
int db_log_subscription_created(const char* sub_id, const char* wsi_ptr,
|
||||
const char* client_ip, const char* filter_json) {
|
||||
return postgres_db_log_subscription_created(sub_id, wsi_ptr, client_ip, filter_json);
|
||||
}
|
||||
int db_log_subscription_closed(const char* sub_id, const char* client_ip) {
|
||||
return postgres_db_log_subscription_closed(sub_id, client_ip);
|
||||
}
|
||||
int db_log_subscription_disconnected(const char* client_ip) {
|
||||
return postgres_db_log_subscription_disconnected(client_ip);
|
||||
}
|
||||
int db_update_subscription_events_sent(const char* sub_id, int events_sent) {
|
||||
return postgres_db_update_subscription_events_sent(sub_id, events_sent);
|
||||
}
|
||||
int db_cleanup_orphaned_subscriptions(void) { return postgres_db_cleanup_orphaned_subscriptions(); }
|
||||
|
||||
int db_get_event_pubkey(const char* event_id, char* pubkey_out, size_t pubkey_out_size) {
|
||||
return postgres_db_get_event_pubkey(event_id, pubkey_out, pubkey_out_size);
|
||||
}
|
||||
int db_delete_event_by_id(const char* event_id, const char* requester_pubkey) {
|
||||
return postgres_db_delete_event_by_id(event_id, requester_pubkey);
|
||||
}
|
||||
int db_delete_events_by_address(const char* pubkey, int kind, const char* d_tag, long before_timestamp) {
|
||||
return postgres_db_delete_events_by_address(pubkey, kind, d_tag, before_timestamp);
|
||||
}
|
||||
|
||||
int db_is_pubkey_blacklisted(const char* pubkey) { return postgres_db_is_pubkey_blacklisted(pubkey); }
|
||||
int db_is_hash_blacklisted(const char* resource_hash) { return postgres_db_is_hash_blacklisted(resource_hash); }
|
||||
int db_is_pubkey_whitelisted(const char* pubkey) { return postgres_db_is_pubkey_whitelisted(pubkey); }
|
||||
int db_count_active_whitelist_rules(void) { return postgres_db_count_active_whitelist_rules(); }
|
||||
|
||||
int db_count_with_sql(const char* sql, const char** bind_params, int bind_param_count, int* out_count) {
|
||||
return postgres_db_count_with_sql(sql, bind_params, bind_param_count, out_count);
|
||||
}
|
||||
char* db_execute_readonly_query_json(const char* query, const char* request_id,
|
||||
char* error_message, size_t error_size,
|
||||
int max_rows, int timeout_ms) {
|
||||
return postgres_db_execute_readonly_query_json(query, request_id, error_message, error_size,
|
||||
max_rows, timeout_ms);
|
||||
}
|
||||
|
||||
int db_get_total_event_count_ll(long long* out_count) { return postgres_db_get_total_event_count_ll(out_count); }
|
||||
int db_get_event_count_since(time_t cutoff, long long* out_count) {
|
||||
return postgres_db_get_event_count_since(cutoff, out_count);
|
||||
}
|
||||
cJSON* db_get_event_kind_distribution_rows(long long* out_total_events) {
|
||||
return postgres_db_get_event_kind_distribution_rows(out_total_events);
|
||||
}
|
||||
cJSON* db_get_top_pubkeys_rows(int limit) { return postgres_db_get_top_pubkeys_rows(limit); }
|
||||
cJSON* db_get_subscription_details_rows(void) { return postgres_db_get_subscription_details_rows(); }
|
||||
|
||||
cJSON* db_get_all_config_rows(void) { return postgres_db_get_all_config_rows(); }
|
||||
char* db_get_config_value_dup(const char* key) { return postgres_db_get_config_value_dup(key); }
|
||||
int db_set_config_value_full(const char* key, const char* value, const char* data_type,
|
||||
const char* description, const char* category, int requires_restart) {
|
||||
return postgres_db_set_config_value_full(key, value, data_type, description, category, requires_restart);
|
||||
}
|
||||
int db_update_config_value_only(const char* key, const char* value) {
|
||||
return postgres_db_update_config_value_only(key, value);
|
||||
}
|
||||
int db_upsert_config_value(const char* key, const char* value, const char* data_type) {
|
||||
return postgres_db_upsert_config_value(key, value, data_type);
|
||||
}
|
||||
int db_store_relay_private_key_hex(const char* relay_privkey_hex) {
|
||||
return postgres_db_store_relay_private_key_hex(relay_privkey_hex);
|
||||
}
|
||||
char* db_get_relay_private_key_hex_dup(void) { return postgres_db_get_relay_private_key_hex_dup(); }
|
||||
int db_store_config_event(const cJSON* event) { return postgres_db_store_config_event(event); }
|
||||
|
||||
int db_insert_event_with_json(const char* id, const char* pubkey, long long created_at,
|
||||
int kind, const char* event_type, const char* content,
|
||||
const char* sig, const char* tags_json, const char* event_json,
|
||||
int* out_step_rc, int* out_extended_errcode) {
|
||||
return postgres_db_insert_event_with_json(id, pubkey, created_at, kind, event_type, content,
|
||||
sig, tags_json, event_json, out_step_rc, out_extended_errcode);
|
||||
}
|
||||
int db_get_event_time_bounds(long long* out_min_created_at, long long* out_max_created_at) {
|
||||
return postgres_db_get_event_time_bounds(out_min_created_at, out_max_created_at);
|
||||
}
|
||||
int db_event_id_exists(const char* event_id, int* out_exists) {
|
||||
return postgres_db_event_id_exists(event_id, out_exists);
|
||||
}
|
||||
cJSON* db_retrieve_event_by_id(const char* event_id) { return postgres_db_retrieve_event_by_id(event_id); }
|
||||
char* db_get_latest_event_pubkey_for_kind_dup(int kind) {
|
||||
return postgres_db_get_latest_event_pubkey_for_kind_dup(kind);
|
||||
}
|
||||
|
||||
int db_get_config_row_count(int* out_count) { return postgres_db_get_config_row_count(out_count); }
|
||||
|
||||
int db_store_event_tags_cjson(const char* event_id, const cJSON* tags) {
|
||||
return postgres_db_store_event_tags_cjson(event_id, tags);
|
||||
}
|
||||
int db_populate_event_tags_from_existing(void) { return postgres_db_populate_event_tags_from_existing(); }
|
||||
|
||||
int db_add_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value) {
|
||||
return postgres_db_add_auth_rule(rule_type, pattern_type, pattern_value);
|
||||
}
|
||||
int db_remove_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value) {
|
||||
return postgres_db_remove_auth_rule(rule_type, pattern_type, pattern_value);
|
||||
}
|
||||
int db_delete_wot_whitelist_rules(void) { return postgres_db_delete_wot_whitelist_rules(); }
|
||||
int db_count_wot_whitelist_rules(void) { return postgres_db_count_wot_whitelist_rules(); }
|
||||
|
||||
int db_table_exists(const char* table_name, int* out_exists) {
|
||||
return postgres_db_table_exists(table_name, out_exists);
|
||||
}
|
||||
char* db_get_schema_version_dup(void) { return postgres_db_get_schema_version_dup(); }
|
||||
int db_exec_sql(const char* sql) { return postgres_db_exec_sql(sql); }
|
||||
int db_wal_checkpoint_passive(void) { return postgres_db_wal_checkpoint_passive(); }
|
||||
int db_wal_checkpoint_truncate(void) { return postgres_db_wal_checkpoint_truncate(); }
|
||||
|
||||
#else
|
||||
|
||||
int db_init(const char* connection_string) { return sqlite_db_init(connection_string); }
|
||||
void db_close(void) { sqlite_db_close(); }
|
||||
int db_is_available(void) { return sqlite_db_is_available(); }
|
||||
const char* db_last_error(void) { return sqlite_db_last_error(); }
|
||||
const char* db_get_database_path(void) { return sqlite_db_get_database_path(); }
|
||||
|
||||
int db_set_thread_connection(void* connection) { return sqlite_db_set_thread_connection(connection); }
|
||||
void db_clear_thread_connection(void) { sqlite_db_clear_thread_connection(); }
|
||||
|
||||
int db_open_worker_connection(const char* db_path, void** out_connection) {
|
||||
return sqlite_db_open_worker_connection(db_path, out_connection);
|
||||
}
|
||||
void db_close_worker_connection(void* connection) { sqlite_db_close_worker_connection(connection); }
|
||||
|
||||
int db_prepare(const char* sql, db_stmt_t** out_stmt) {
|
||||
return sqlite_db_prepare(sql, (sqlite_db_stmt_t**)out_stmt);
|
||||
}
|
||||
int db_bind_text_param(db_stmt_t* stmt, int index, const char* value) {
|
||||
return sqlite_db_bind_text_param((sqlite_db_stmt_t*)stmt, index, value);
|
||||
}
|
||||
int db_bind_int_param(db_stmt_t* stmt, int index, int value) {
|
||||
return sqlite_db_bind_int_param((sqlite_db_stmt_t*)stmt, index, value);
|
||||
}
|
||||
int db_bind_int64_param(db_stmt_t* stmt, int index, long long value) {
|
||||
return sqlite_db_bind_int64_param((sqlite_db_stmt_t*)stmt, index, value);
|
||||
}
|
||||
int db_step_stmt(db_stmt_t* stmt) { return sqlite_db_step_stmt((sqlite_db_stmt_t*)stmt); }
|
||||
int db_reset_stmt(db_stmt_t* stmt) { return sqlite_db_reset_stmt((sqlite_db_stmt_t*)stmt); }
|
||||
const char* db_column_text_value(db_stmt_t* stmt, int col) {
|
||||
return sqlite_db_column_text_value((sqlite_db_stmt_t*)stmt, col);
|
||||
}
|
||||
int db_column_int_value(db_stmt_t* stmt, int col) {
|
||||
return sqlite_db_column_int_value((sqlite_db_stmt_t*)stmt, col);
|
||||
}
|
||||
long long db_column_int64_value(db_stmt_t* stmt, int col) {
|
||||
return sqlite_db_column_int64_value((sqlite_db_stmt_t*)stmt, col);
|
||||
}
|
||||
double db_column_double_value(db_stmt_t* stmt, int col) {
|
||||
return sqlite_db_column_double_value((sqlite_db_stmt_t*)stmt, col);
|
||||
}
|
||||
void db_finalize_stmt(db_stmt_t* stmt) { sqlite_db_finalize_stmt((sqlite_db_stmt_t*)stmt); }
|
||||
|
||||
int db_log_subscription_created(const char* sub_id, const char* wsi_ptr,
|
||||
const char* client_ip, const char* filter_json) {
|
||||
return sqlite_db_log_subscription_created(sub_id, wsi_ptr, client_ip, filter_json);
|
||||
}
|
||||
int db_log_subscription_closed(const char* sub_id, const char* client_ip) {
|
||||
return sqlite_db_log_subscription_closed(sub_id, client_ip);
|
||||
}
|
||||
int db_log_subscription_disconnected(const char* client_ip) {
|
||||
return sqlite_db_log_subscription_disconnected(client_ip);
|
||||
}
|
||||
int db_update_subscription_events_sent(const char* sub_id, int events_sent) {
|
||||
return sqlite_db_update_subscription_events_sent(sub_id, events_sent);
|
||||
}
|
||||
int db_cleanup_orphaned_subscriptions(void) { return sqlite_db_cleanup_orphaned_subscriptions(); }
|
||||
|
||||
int db_get_event_pubkey(const char* event_id, char* pubkey_out, size_t pubkey_out_size) {
|
||||
return sqlite_db_get_event_pubkey(event_id, pubkey_out, pubkey_out_size);
|
||||
}
|
||||
int db_delete_event_by_id(const char* event_id, const char* requester_pubkey) {
|
||||
return sqlite_db_delete_event_by_id(event_id, requester_pubkey);
|
||||
}
|
||||
int db_delete_events_by_address(const char* pubkey, int kind, const char* d_tag, long before_timestamp) {
|
||||
return sqlite_db_delete_events_by_address(pubkey, kind, d_tag, before_timestamp);
|
||||
}
|
||||
|
||||
int db_is_pubkey_blacklisted(const char* pubkey) { return sqlite_db_is_pubkey_blacklisted(pubkey); }
|
||||
int db_is_hash_blacklisted(const char* resource_hash) { return sqlite_db_is_hash_blacklisted(resource_hash); }
|
||||
int db_is_pubkey_whitelisted(const char* pubkey) { return sqlite_db_is_pubkey_whitelisted(pubkey); }
|
||||
int db_count_active_whitelist_rules(void) { return sqlite_db_count_active_whitelist_rules(); }
|
||||
|
||||
int db_count_with_sql(const char* sql, const char** bind_params, int bind_param_count, int* out_count) {
|
||||
return sqlite_db_count_with_sql(sql, bind_params, bind_param_count, out_count);
|
||||
}
|
||||
char* db_execute_readonly_query_json(const char* query, const char* request_id,
|
||||
char* error_message, size_t error_size,
|
||||
int max_rows, int timeout_ms) {
|
||||
return sqlite_db_execute_readonly_query_json(query, request_id, error_message, error_size,
|
||||
max_rows, timeout_ms);
|
||||
}
|
||||
|
||||
int db_get_total_event_count_ll(long long* out_count) { return sqlite_db_get_total_event_count_ll(out_count); }
|
||||
int db_get_event_count_since(time_t cutoff, long long* out_count) {
|
||||
return sqlite_db_get_event_count_since(cutoff, out_count);
|
||||
}
|
||||
cJSON* db_get_event_kind_distribution_rows(long long* out_total_events) {
|
||||
return sqlite_db_get_event_kind_distribution_rows(out_total_events);
|
||||
}
|
||||
cJSON* db_get_top_pubkeys_rows(int limit) { return sqlite_db_get_top_pubkeys_rows(limit); }
|
||||
cJSON* db_get_subscription_details_rows(void) { return sqlite_db_get_subscription_details_rows(); }
|
||||
|
||||
cJSON* db_get_all_config_rows(void) { return sqlite_db_get_all_config_rows(); }
|
||||
char* db_get_config_value_dup(const char* key) { return sqlite_db_get_config_value_dup(key); }
|
||||
int db_set_config_value_full(const char* key, const char* value, const char* data_type,
|
||||
const char* description, const char* category, int requires_restart) {
|
||||
return sqlite_db_set_config_value_full(key, value, data_type, description, category, requires_restart);
|
||||
}
|
||||
int db_update_config_value_only(const char* key, const char* value) {
|
||||
return sqlite_db_update_config_value_only(key, value);
|
||||
}
|
||||
int db_upsert_config_value(const char* key, const char* value, const char* data_type) {
|
||||
return sqlite_db_upsert_config_value(key, value, data_type);
|
||||
}
|
||||
int db_store_relay_private_key_hex(const char* relay_privkey_hex) {
|
||||
return sqlite_db_store_relay_private_key_hex(relay_privkey_hex);
|
||||
}
|
||||
char* db_get_relay_private_key_hex_dup(void) { return sqlite_db_get_relay_private_key_hex_dup(); }
|
||||
int db_store_config_event(const cJSON* event) { return sqlite_db_store_config_event(event); }
|
||||
|
||||
int db_insert_event_with_json(const char* id, const char* pubkey, long long created_at,
|
||||
int kind, const char* event_type, const char* content,
|
||||
const char* sig, const char* tags_json, const char* event_json,
|
||||
int* out_step_rc, int* out_extended_errcode) {
|
||||
return sqlite_db_insert_event_with_json(id, pubkey, created_at, kind, event_type, content,
|
||||
sig, tags_json, event_json, out_step_rc, out_extended_errcode);
|
||||
}
|
||||
int db_get_event_time_bounds(long long* out_min_created_at, long long* out_max_created_at) {
|
||||
return sqlite_db_get_event_time_bounds(out_min_created_at, out_max_created_at);
|
||||
}
|
||||
int db_event_id_exists(const char* event_id, int* out_exists) {
|
||||
return sqlite_db_event_id_exists(event_id, out_exists);
|
||||
}
|
||||
cJSON* db_retrieve_event_by_id(const char* event_id) { return sqlite_db_retrieve_event_by_id(event_id); }
|
||||
char* db_get_latest_event_pubkey_for_kind_dup(int kind) {
|
||||
return sqlite_db_get_latest_event_pubkey_for_kind_dup(kind);
|
||||
}
|
||||
|
||||
int db_get_config_row_count(int* out_count) { return sqlite_db_get_config_row_count(out_count); }
|
||||
|
||||
int db_store_event_tags_cjson(const char* event_id, const cJSON* tags) {
|
||||
return sqlite_db_store_event_tags_cjson(event_id, tags);
|
||||
}
|
||||
int db_populate_event_tags_from_existing(void) { return sqlite_db_populate_event_tags_from_existing(); }
|
||||
|
||||
int db_add_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value) {
|
||||
return sqlite_db_add_auth_rule(rule_type, pattern_type, pattern_value);
|
||||
}
|
||||
int db_remove_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value) {
|
||||
return sqlite_db_remove_auth_rule(rule_type, pattern_type, pattern_value);
|
||||
}
|
||||
int db_delete_wot_whitelist_rules(void) { return sqlite_db_delete_wot_whitelist_rules(); }
|
||||
int db_count_wot_whitelist_rules(void) { return sqlite_db_count_wot_whitelist_rules(); }
|
||||
|
||||
int db_table_exists(const char* table_name, int* out_exists) {
|
||||
return sqlite_db_table_exists(table_name, out_exists);
|
||||
}
|
||||
char* db_get_schema_version_dup(void) { return sqlite_db_get_schema_version_dup(); }
|
||||
int db_exec_sql(const char* sql) { return sqlite_db_exec_sql(sql); }
|
||||
int db_wal_checkpoint_passive(void) { return sqlite_db_wal_checkpoint_passive(); }
|
||||
int db_wal_checkpoint_truncate(void) { return sqlite_db_wal_checkpoint_truncate(); }
|
||||
|
||||
#endif
|
||||
+120
@@ -0,0 +1,120 @@
|
||||
#ifndef DB_OPS_H
|
||||
#define DB_OPS_H
|
||||
|
||||
#include <stddef.h>
|
||||
#include <time.h>
|
||||
#include <cjson/cJSON.h>
|
||||
|
||||
// Generic helpers
|
||||
int db_init(const char* connection_string);
|
||||
void db_close(void);
|
||||
int db_is_available(void);
|
||||
const char* db_last_error(void);
|
||||
const char* db_get_database_path(void);
|
||||
|
||||
// Per-thread connection override (used by thread pool workers)
|
||||
int db_set_thread_connection(void* connection);
|
||||
void db_clear_thread_connection(void);
|
||||
|
||||
// Worker/runtime SQLite connection lifecycle (kept internal to db_ops.c)
|
||||
int db_open_worker_connection(const char* db_path, void** out_connection);
|
||||
void db_close_worker_connection(void* connection);
|
||||
|
||||
// DB result codes (backend-agnostic)
|
||||
#define DB_OK 0
|
||||
#define DB_ERROR 1
|
||||
#define DB_CONSTRAINT 19
|
||||
#define DB_MISUSE 21
|
||||
#define DB_ROW 100
|
||||
#define DB_DONE 101
|
||||
|
||||
typedef struct db_stmt db_stmt_t;
|
||||
|
||||
// Generic statement helpers (Phase 1 migration)
|
||||
int db_prepare(const char* sql, db_stmt_t** out_stmt);
|
||||
int db_bind_text_param(db_stmt_t* stmt, int index, const char* value);
|
||||
int db_bind_int_param(db_stmt_t* stmt, int index, int value);
|
||||
int db_bind_int64_param(db_stmt_t* stmt, int index, long long value);
|
||||
int db_step_stmt(db_stmt_t* stmt);
|
||||
int db_reset_stmt(db_stmt_t* stmt);
|
||||
const char* db_column_text_value(db_stmt_t* stmt, int col);
|
||||
int db_column_int_value(db_stmt_t* stmt, int col);
|
||||
long long db_column_int64_value(db_stmt_t* stmt, int col);
|
||||
double db_column_double_value(db_stmt_t* stmt, int col);
|
||||
void db_finalize_stmt(db_stmt_t* stmt);
|
||||
|
||||
// Subscription logging
|
||||
int db_log_subscription_created(const char* sub_id, const char* wsi_ptr,
|
||||
const char* client_ip, const char* filter_json);
|
||||
int db_log_subscription_closed(const char* sub_id, const char* client_ip);
|
||||
int db_log_subscription_disconnected(const char* client_ip);
|
||||
int db_update_subscription_events_sent(const char* sub_id, int events_sent);
|
||||
int db_cleanup_orphaned_subscriptions(void);
|
||||
|
||||
// NIP-09 event deletion helpers
|
||||
int db_get_event_pubkey(const char* event_id, char* pubkey_out, size_t pubkey_out_size);
|
||||
int db_delete_event_by_id(const char* event_id, const char* requester_pubkey);
|
||||
int db_delete_events_by_address(const char* pubkey, int kind,
|
||||
const char* d_tag, long before_timestamp);
|
||||
|
||||
// Auth rule checks for request validator
|
||||
int db_is_pubkey_blacklisted(const char* pubkey);
|
||||
int db_is_hash_blacklisted(const char* resource_hash);
|
||||
int db_is_pubkey_whitelisted(const char* pubkey);
|
||||
int db_count_active_whitelist_rules(void);
|
||||
|
||||
// Generic prepared COUNT helper
|
||||
int db_count_with_sql(const char* sql, const char** bind_params, int bind_param_count, int* out_count);
|
||||
char* db_execute_readonly_query_json(const char* query, const char* request_id,
|
||||
char* error_message, size_t error_size,
|
||||
int max_rows, int timeout_ms);
|
||||
|
||||
// Monitoring/stat helpers (Phase 1 api.c migration)
|
||||
int db_get_total_event_count_ll(long long* out_count);
|
||||
int db_get_event_count_since(time_t cutoff, long long* out_count);
|
||||
cJSON* db_get_event_kind_distribution_rows(long long* out_total_events);
|
||||
cJSON* db_get_top_pubkeys_rows(int limit);
|
||||
cJSON* db_get_subscription_details_rows(void);
|
||||
|
||||
// Config helpers
|
||||
cJSON* db_get_all_config_rows(void);
|
||||
char* db_get_config_value_dup(const char* key);
|
||||
int db_set_config_value_full(const char* key, const char* value, const char* data_type,
|
||||
const char* description, const char* category, int requires_restart);
|
||||
int db_update_config_value_only(const char* key, const char* value);
|
||||
int db_upsert_config_value(const char* key, const char* value, const char* data_type);
|
||||
int db_store_relay_private_key_hex(const char* relay_privkey_hex);
|
||||
char* db_get_relay_private_key_hex_dup(void);
|
||||
int db_store_config_event(const cJSON* event);
|
||||
|
||||
// Event storage/timestamp/retrieval helpers
|
||||
int db_insert_event_with_json(const char* id, const char* pubkey, long long created_at,
|
||||
int kind, const char* event_type, const char* content,
|
||||
const char* sig, const char* tags_json, const char* event_json,
|
||||
int* out_step_rc, int* out_extended_errcode);
|
||||
int db_get_event_time_bounds(long long* out_min_created_at, long long* out_max_created_at);
|
||||
int db_event_id_exists(const char* event_id, int* out_exists);
|
||||
cJSON* db_retrieve_event_by_id(const char* event_id);
|
||||
char* db_get_latest_event_pubkey_for_kind_dup(int kind);
|
||||
|
||||
// Config table helpers
|
||||
int db_get_config_row_count(int* out_count);
|
||||
|
||||
// Event tag helpers
|
||||
int db_store_event_tags_cjson(const char* event_id, const cJSON* tags);
|
||||
int db_populate_event_tags_from_existing(void);
|
||||
|
||||
// Auth/WoT rule helpers
|
||||
int db_add_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
|
||||
int db_remove_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
|
||||
int db_delete_wot_whitelist_rules(void);
|
||||
int db_count_wot_whitelist_rules(void);
|
||||
|
||||
// Schema/DDL helpers
|
||||
int db_table_exists(const char* table_name, int* out_exists);
|
||||
char* db_get_schema_version_dup(void);
|
||||
int db_exec_sql(const char* sql);
|
||||
int db_wal_checkpoint_passive(void);
|
||||
int db_wal_checkpoint_truncate(void);
|
||||
|
||||
#endif // DB_OPS_H
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,97 @@
|
||||
#ifndef DB_OPS_POSTGRES_H
|
||||
#define DB_OPS_POSTGRES_H
|
||||
|
||||
#include "db_ops.h"
|
||||
|
||||
typedef struct postgres_db_stmt postgres_db_stmt_t;
|
||||
|
||||
int postgres_db_init(const char* connection_string);
|
||||
void postgres_db_close(void);
|
||||
int postgres_db_is_available(void);
|
||||
const char* postgres_db_last_error(void);
|
||||
const char* postgres_db_get_database_path(void);
|
||||
|
||||
int postgres_db_apply_schema(void);
|
||||
|
||||
int postgres_db_set_thread_connection(void* connection);
|
||||
void postgres_db_clear_thread_connection(void);
|
||||
|
||||
int postgres_db_open_worker_connection(const char* db_path, void** out_connection);
|
||||
void postgres_db_close_worker_connection(void* connection);
|
||||
|
||||
int postgres_db_prepare(const char* sql, postgres_db_stmt_t** out_stmt);
|
||||
int postgres_db_bind_text_param(postgres_db_stmt_t* stmt, int index, const char* value);
|
||||
int postgres_db_bind_int_param(postgres_db_stmt_t* stmt, int index, int value);
|
||||
int postgres_db_bind_int64_param(postgres_db_stmt_t* stmt, int index, long long value);
|
||||
int postgres_db_step_stmt(postgres_db_stmt_t* stmt);
|
||||
int postgres_db_reset_stmt(postgres_db_stmt_t* stmt);
|
||||
const char* postgres_db_column_text_value(postgres_db_stmt_t* stmt, int col);
|
||||
int postgres_db_column_int_value(postgres_db_stmt_t* stmt, int col);
|
||||
long long postgres_db_column_int64_value(postgres_db_stmt_t* stmt, int col);
|
||||
double postgres_db_column_double_value(postgres_db_stmt_t* stmt, int col);
|
||||
void postgres_db_finalize_stmt(postgres_db_stmt_t* stmt);
|
||||
|
||||
int postgres_db_log_subscription_created(const char* sub_id, const char* wsi_ptr,
|
||||
const char* client_ip, const char* filter_json);
|
||||
int postgres_db_log_subscription_closed(const char* sub_id, const char* client_ip);
|
||||
int postgres_db_log_subscription_disconnected(const char* client_ip);
|
||||
int postgres_db_update_subscription_events_sent(const char* sub_id, int events_sent);
|
||||
int postgres_db_cleanup_orphaned_subscriptions(void);
|
||||
|
||||
int postgres_db_get_event_pubkey(const char* event_id, char* pubkey_out, size_t pubkey_out_size);
|
||||
int postgres_db_delete_event_by_id(const char* event_id, const char* requester_pubkey);
|
||||
int postgres_db_delete_events_by_address(const char* pubkey, int kind,
|
||||
const char* d_tag, long before_timestamp);
|
||||
|
||||
int postgres_db_is_pubkey_blacklisted(const char* pubkey);
|
||||
int postgres_db_is_hash_blacklisted(const char* resource_hash);
|
||||
int postgres_db_is_pubkey_whitelisted(const char* pubkey);
|
||||
int postgres_db_count_active_whitelist_rules(void);
|
||||
|
||||
int postgres_db_count_with_sql(const char* sql, const char** bind_params, int bind_param_count, int* out_count);
|
||||
char* postgres_db_execute_readonly_query_json(const char* query, const char* request_id,
|
||||
char* error_message, size_t error_size,
|
||||
int max_rows, int timeout_ms);
|
||||
|
||||
int postgres_db_get_total_event_count_ll(long long* out_count);
|
||||
int postgres_db_get_event_count_since(time_t cutoff, long long* out_count);
|
||||
cJSON* postgres_db_get_event_kind_distribution_rows(long long* out_total_events);
|
||||
cJSON* postgres_db_get_top_pubkeys_rows(int limit);
|
||||
cJSON* postgres_db_get_subscription_details_rows(void);
|
||||
|
||||
cJSON* postgres_db_get_all_config_rows(void);
|
||||
char* postgres_db_get_config_value_dup(const char* key);
|
||||
int postgres_db_set_config_value_full(const char* key, const char* value, const char* data_type,
|
||||
const char* description, const char* category, int requires_restart);
|
||||
int postgres_db_update_config_value_only(const char* key, const char* value);
|
||||
int postgres_db_upsert_config_value(const char* key, const char* value, const char* data_type);
|
||||
int postgres_db_store_relay_private_key_hex(const char* relay_privkey_hex);
|
||||
char* postgres_db_get_relay_private_key_hex_dup(void);
|
||||
int postgres_db_store_config_event(const cJSON* event);
|
||||
|
||||
int postgres_db_insert_event_with_json(const char* id, const char* pubkey, long long created_at,
|
||||
int kind, const char* event_type, const char* content,
|
||||
const char* sig, const char* tags_json, const char* event_json,
|
||||
int* out_step_rc, int* out_extended_errcode);
|
||||
int postgres_db_get_event_time_bounds(long long* out_min_created_at, long long* out_max_created_at);
|
||||
int postgres_db_event_id_exists(const char* event_id, int* out_exists);
|
||||
cJSON* postgres_db_retrieve_event_by_id(const char* event_id);
|
||||
char* postgres_db_get_latest_event_pubkey_for_kind_dup(int kind);
|
||||
|
||||
int postgres_db_get_config_row_count(int* out_count);
|
||||
|
||||
int postgres_db_store_event_tags_cjson(const char* event_id, const cJSON* tags);
|
||||
int postgres_db_populate_event_tags_from_existing(void);
|
||||
|
||||
int postgres_db_add_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
|
||||
int postgres_db_remove_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
|
||||
int postgres_db_delete_wot_whitelist_rules(void);
|
||||
int postgres_db_count_wot_whitelist_rules(void);
|
||||
|
||||
int postgres_db_table_exists(const char* table_name, int* out_exists);
|
||||
char* postgres_db_get_schema_version_dup(void);
|
||||
int postgres_db_exec_sql(const char* sql);
|
||||
int postgres_db_wal_checkpoint_passive(void);
|
||||
int postgres_db_wal_checkpoint_truncate(void);
|
||||
|
||||
#endif // DB_OPS_POSTGRES_H
|
||||
+1397
File diff suppressed because it is too large
Load Diff
@@ -65,6 +65,9 @@ static const struct {
|
||||
{"max_total_subscriptions", "5000"},
|
||||
{"max_filters_per_subscription", "10"},
|
||||
|
||||
// Connection Management
|
||||
{"max_connection_seconds", "86400"}, // 24 hours (0 = disabled)
|
||||
|
||||
// Event Processing Limits
|
||||
{"max_event_tags", "100"},
|
||||
{"max_content_length", "8196"},
|
||||
@@ -80,11 +83,58 @@ static const struct {
|
||||
// IP-based rate limiting or access control (which would require firewall protection anyway)
|
||||
{"trust_proxy_headers", "true"},
|
||||
|
||||
// Debug Level (0=none, 1=errors, 2=warnings, 3=info, 4=debug, 5=trace)
|
||||
// Can be changed at runtime without restart via config_set admin command
|
||||
{"debug_level", "3"},
|
||||
|
||||
// IP Auth Failure Ban Settings
|
||||
// Ban IPs that repeatedly fail NIP-42 authentication
|
||||
{"auth_fail_ban_enabled", "true"},
|
||||
{"auth_fail_ban_threshold", "3"}, // failures before ban
|
||||
{"auth_fail_window_sec", "60"}, // window to count failures in
|
||||
{"auth_fail_ban_duration_sec", "300"}, // initial ban duration (doubles each time, max 24h)
|
||||
|
||||
// NIP-42 Authentication Timeout
|
||||
// Seconds after connection before unauthenticated clients are disconnected (0 = disabled)
|
||||
// Prevents unauthenticated connections from accumulating under heavy load
|
||||
{"nip42_auth_timeout_sec", "10"},
|
||||
|
||||
// Idle Connection Ban Settings
|
||||
// Ban IPs that connect but never send REQ or EVENT (idle or early disconnect)
|
||||
{"idle_connection_timeout_sec", "0"}, // Seconds before idle connection is closed (0 = disabled)
|
||||
{"idle_ban_enabled", "false"}, // Whether to ban IPs with idle failures
|
||||
{"idle_ban_threshold", "1"}, // Idle failures before ban (1 = ban on first offense)
|
||||
{"idle_ban_window_sec", "30"}, // Window to count idle failures in
|
||||
{"idle_ban_duration_sec", "300"}, // Initial ban duration (doubles each time, max 24h)
|
||||
|
||||
// SQLite Performance Tuning
|
||||
// mmap_size: bytes of database file to memory-map (0 = disabled, 268435456 = 256MB recommended)
|
||||
// Eliminates pread64 syscall overhead for database reads — significant CPU savings under load
|
||||
{"sqlite_mmap_size", "268435456"},
|
||||
// cache_size_kb: SQLite page cache size in KB (negative = KB, positive = pages of 4KB each)
|
||||
// Default 2000KB is too small for a busy relay; 65536KB (64MB) keeps hot data in memory
|
||||
{"sqlite_cache_size_kb", "65536"},
|
||||
|
||||
// NIP-59 Gift Wrap Timestamp Configuration
|
||||
{"nip59_timestamp_max_delay_sec", "0"},
|
||||
|
||||
// Kind 1 Status Posts
|
||||
{"kind_1_status_posts_hours", "1"}
|
||||
{"kind_1_status_posts_hours", "1"},
|
||||
|
||||
// Web of Trust Settings
|
||||
// 0 = off, 1 = write-only (followed pubkeys can publish), 2 = full (followed pubkeys can publish AND subscribe)
|
||||
{"wot_enabled", "0"},
|
||||
|
||||
// NIP-17 Admin DM Settings
|
||||
// When false (default), Kind 1059 gift wrap events are stored normally without expensive decryption attempts.
|
||||
// Enable only if you intend to use NIP-17 DMs to send admin commands to the relay.
|
||||
{"nip17_admin_enabled", "false"},
|
||||
|
||||
// Thread Pool Settings
|
||||
{"thread_pool_enabled", "true"},
|
||||
{"thread_pool_readers", "4"},
|
||||
{"thread_pool_writers", "2"},
|
||||
{"thread_pool_max_queue_depth", "4096"}
|
||||
};
|
||||
|
||||
// Number of default configuration values
|
||||
|
||||
+154
-15
@@ -5,6 +5,7 @@
|
||||
#include "config.h"
|
||||
#include "debug.h"
|
||||
#include "api.h"
|
||||
#include "db_ops.h"
|
||||
#include "../nostr_core_lib/nostr_core/nostr_core.h"
|
||||
#include "../nostr_core_lib/nostr_core/nip017.h"
|
||||
#include "../nostr_core_lib/nostr_core/nip044.h"
|
||||
@@ -17,9 +18,6 @@
|
||||
#include <cjson/cJSON.h>
|
||||
#include <libwebsockets.h>
|
||||
|
||||
// External database connection (from main.c)
|
||||
extern sqlite3* g_db;
|
||||
|
||||
// Forward declarations for unified handlers
|
||||
extern int handle_auth_query_unified(cJSON* event, const char* query_type, char* error_message, size_t error_size, struct lws* wsi);
|
||||
extern int handle_config_query_unified(cJSON* event, const char* query_type, char* error_message, size_t error_size, struct lws* wsi);
|
||||
@@ -37,10 +35,13 @@ extern const char* get_tag_value(cJSON* event, const char* tag_name, int value_i
|
||||
extern char* get_relay_private_key(void);
|
||||
extern const char* get_config_value(const char* key);
|
||||
extern int get_config_bool(const char* key, int default_value);
|
||||
extern int get_config_int(const char* key, int default_value);
|
||||
extern int update_config_in_table(const char* key, const char* value);
|
||||
|
||||
// Forward declarations for database functions
|
||||
extern int store_event(cJSON* event);
|
||||
extern int broadcast_event_to_subscriptions(cJSON* event);
|
||||
extern int store_event_tags(const char* event_id, cJSON* tags);
|
||||
|
||||
// Forward declarations for stats generation (moved to api.c)
|
||||
extern char* generate_stats_json(void);
|
||||
@@ -138,7 +139,7 @@ int process_dm_admin_command(cJSON* command_array, cJSON* event, char* error_mes
|
||||
cJSON_AddItemToArray(synthetic_tags, command_tag);
|
||||
|
||||
// Add existing event tags
|
||||
cJSON* existing_tags = cJSON_GetObjectItem(event, "tags");
|
||||
cJSON* existing_tags = cJSON_GetObjectItemCaseSensitive(event, "tags");
|
||||
if (existing_tags && cJSON_IsArray(existing_tags)) {
|
||||
cJSON* tag = NULL;
|
||||
cJSON_ArrayForEach(tag, existing_tags) {
|
||||
@@ -301,7 +302,7 @@ cJSON* process_nip17_admin_message(cJSON* gift_wrap_event, char* error_message,
|
||||
// Only create a generic response for other command types that don't handle their own responses
|
||||
if (result == 0) {
|
||||
// Extract content to check if it's a plain text command
|
||||
cJSON* content_obj = cJSON_GetObjectItem(inner_dm, "content");
|
||||
cJSON* content_obj = cJSON_GetObjectItemCaseSensitive(inner_dm, "content");
|
||||
if (content_obj && cJSON_IsString(content_obj)) {
|
||||
const char* dm_content = cJSON_GetStringValue(content_obj);
|
||||
|
||||
@@ -344,7 +345,7 @@ cJSON* process_nip17_admin_message(cJSON* gift_wrap_event, char* error_message,
|
||||
return NULL;
|
||||
|
||||
// Get sender pubkey for response from the decrypted DM event
|
||||
cJSON* sender_pubkey_obj = cJSON_GetObjectItem(inner_dm, "pubkey");
|
||||
cJSON* sender_pubkey_obj = cJSON_GetObjectItemCaseSensitive(inner_dm, "pubkey");
|
||||
if (sender_pubkey_obj && cJSON_IsString(sender_pubkey_obj)) {
|
||||
const char* sender_pubkey = cJSON_GetStringValue(sender_pubkey_obj);
|
||||
|
||||
@@ -436,13 +437,13 @@ int is_nip17_gift_wrap_for_relay(cJSON* event) {
|
||||
}
|
||||
|
||||
// Check kind
|
||||
cJSON* kind_obj = cJSON_GetObjectItem(event, "kind");
|
||||
cJSON* kind_obj = cJSON_GetObjectItemCaseSensitive(event, "kind");
|
||||
if (!kind_obj || !cJSON_IsNumber(kind_obj) || (int)cJSON_GetNumberValue(kind_obj) != 1059) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Check tags for "p" tag with relay pubkey
|
||||
cJSON* tags = cJSON_GetObjectItem(event, "tags");
|
||||
cJSON* tags = cJSON_GetObjectItemCaseSensitive(event, "tags");
|
||||
if (!tags || !cJSON_IsArray(tags)) {
|
||||
return 0;
|
||||
}
|
||||
@@ -481,7 +482,7 @@ int process_nip17_admin_command(cJSON* dm_event, char* error_message, size_t err
|
||||
DEBUG_INFO("DM_ADMIN: Processing NIP-17 admin command from decrypted DM");
|
||||
|
||||
// Extract content from DM
|
||||
cJSON* content_obj = cJSON_GetObjectItem(dm_event, "content");
|
||||
cJSON* content_obj = cJSON_GetObjectItemCaseSensitive(dm_event, "content");
|
||||
if (!content_obj || !cJSON_IsString(content_obj)) {
|
||||
DEBUG_INFO("DM_ADMIN: DM missing content field");
|
||||
strncpy(error_message, "NIP-17: DM missing content", error_size - 1);
|
||||
@@ -492,7 +493,7 @@ int process_nip17_admin_command(cJSON* dm_event, char* error_message, size_t err
|
||||
DEBUG_INFO("DM_ADMIN: Extracted DM content: %.100s%s", dm_content, strlen(dm_content) > 100 ? "..." : "");
|
||||
|
||||
// Check if sender is admin before processing any commands
|
||||
cJSON* sender_pubkey_obj = cJSON_GetObjectItem(dm_event, "pubkey");
|
||||
cJSON* sender_pubkey_obj = cJSON_GetObjectItemCaseSensitive(dm_event, "pubkey");
|
||||
if (!sender_pubkey_obj || !cJSON_IsString(sender_pubkey_obj)) {
|
||||
DEBUG_INFO("DM_ADMIN: DM missing sender pubkey - treating as user DM");
|
||||
return 0; // Not an error, just treat as user DM
|
||||
@@ -500,8 +501,7 @@ int process_nip17_admin_command(cJSON* dm_event, char* error_message, size_t err
|
||||
const char* sender_pubkey = cJSON_GetStringValue(sender_pubkey_obj);
|
||||
|
||||
// Check if sender is admin
|
||||
const char* admin_pubkey = get_config_value("admin_pubkey");
|
||||
int is_admin = admin_pubkey && strlen(admin_pubkey) > 0 && strcmp(sender_pubkey, admin_pubkey) == 0;
|
||||
int is_admin = is_admin_pubkey(sender_pubkey);
|
||||
|
||||
DEBUG_INFO("DM_ADMIN: Sender pubkey: %.16s... (admin: %s)", sender_pubkey, is_admin ? "YES" : "NO");
|
||||
|
||||
@@ -598,6 +598,145 @@ int process_nip17_admin_command(cJSON* dm_event, char* error_message, size_t err
|
||||
DEBUG_INFO("DM_ADMIN: Status command processed successfully");
|
||||
return 0;
|
||||
}
|
||||
// Check for WoT (Web of Trust) commands
|
||||
else if (strstr(content_lower, "wot") != NULL) {
|
||||
DEBUG_INFO("DM_ADMIN: Processing WoT command");
|
||||
|
||||
// Skip "wot" prefix and find the subcommand
|
||||
char* wot_cmd = strstr(content_lower, "wot");
|
||||
if (wot_cmd) {
|
||||
wot_cmd += 3; // Skip "wot"
|
||||
while (*wot_cmd == ' ') wot_cmd++; // Skip spaces
|
||||
|
||||
char response_msg[1024];
|
||||
int wot_level = get_config_int("wot_enabled", 0);
|
||||
extern int wot_sync_from_admin_kind3(void);
|
||||
|
||||
if (strcmp(wot_cmd, "0") == 0 || strcmp(wot_cmd, "off") == 0) {
|
||||
// Disable WoT
|
||||
update_config_in_table("wot_enabled", "0");
|
||||
update_config_in_table("auth_enabled", "false");
|
||||
update_config_in_table("nip42_auth_required_subscriptions", "false");
|
||||
|
||||
// Clear wot_whitelist rules
|
||||
db_delete_wot_whitelist_rules();
|
||||
|
||||
snprintf(response_msg, sizeof(response_msg),
|
||||
"🔓 Web of Trust Disabled\n"
|
||||
"━━━━━━━━━━━━━━━━━━━━━━\n"
|
||||
"\n"
|
||||
"WoT has been turned off.\n"
|
||||
"\n"
|
||||
"The relay is now open to all pubkeys for reading and writing.\n"
|
||||
"Manual whitelist/blacklist rules are preserved.");
|
||||
send_nip17_response(sender_pubkey, response_msg, NULL, 0);
|
||||
DEBUG_INFO("DM_ADMIN: WoT disabled");
|
||||
}
|
||||
else if (strcmp(wot_cmd, "1") == 0 || strcmp(wot_cmd, "write") == 0) {
|
||||
// Write-only mode
|
||||
update_config_in_table("wot_enabled", "1");
|
||||
wot_sync_from_admin_kind3();
|
||||
|
||||
wot_level = get_config_int("wot_enabled", 0);
|
||||
snprintf(response_msg, sizeof(response_msg),
|
||||
"✍️ Web of Trust: Write-Only Mode\n"
|
||||
"━━━━━━━━━━━━━━━━━━━━━━\n"
|
||||
"\n"
|
||||
"Level: 1 (Write-only restriction)\n"
|
||||
"\n"
|
||||
"Only pubkeys you follow can publish events.\n"
|
||||
"Anyone can still subscribe and read events.\n"
|
||||
"\n"
|
||||
"The relay will now sync from your kind 3 (contact list) event.");
|
||||
send_nip17_response(sender_pubkey, response_msg, NULL, 0);
|
||||
DEBUG_INFO("DM_ADMIN: WoT set to write-only mode");
|
||||
}
|
||||
else if (strcmp(wot_cmd, "2") == 0 || strcmp(wot_cmd, "full") == 0) {
|
||||
// Full mode (write + read restriction)
|
||||
update_config_in_table("wot_enabled", "2");
|
||||
wot_sync_from_admin_kind3();
|
||||
|
||||
wot_level = get_config_int("wot_enabled", 0);
|
||||
snprintf(response_msg, sizeof(response_msg),
|
||||
"🔒 Web of Trust: Full Mode\n"
|
||||
"━━━━━━━━━━━━━━━━━━━━━━\n"
|
||||
"\n"
|
||||
"Level: 2 (Full restriction)\n"
|
||||
"\n"
|
||||
"Only pubkeys you follow can:\n"
|
||||
" • Publish events\n"
|
||||
" • Subscribe to events (requires NIP-42 auth)\n"
|
||||
"\n"
|
||||
"This eliminates anonymous subscription churn.\n"
|
||||
"\n"
|
||||
"The relay will now sync from your kind 3 (contact list) event.");
|
||||
send_nip17_response(sender_pubkey, response_msg, NULL, 0);
|
||||
DEBUG_INFO("DM_ADMIN: WoT set to full mode");
|
||||
}
|
||||
else if (strcmp(wot_cmd, "sync") == 0) {
|
||||
// Force resync
|
||||
wot_sync_from_admin_kind3();
|
||||
|
||||
snprintf(response_msg, sizeof(response_msg),
|
||||
"🔄 Web of Trust: Sync Complete\n"
|
||||
"━━━━━━━━━━━━━━━━━━━━━━\n"
|
||||
"\n"
|
||||
"WoT whitelist has been refreshed from your\n"
|
||||
"kind 3 (contact list) event.\n"
|
||||
"\n"
|
||||
"Current level: %d", wot_level);
|
||||
send_nip17_response(sender_pubkey, response_msg, NULL, 0);
|
||||
DEBUG_INFO("DM_ADMIN: WoT manual sync completed");
|
||||
}
|
||||
else if (strcmp(wot_cmd, "status") == 0 || strlen(wot_cmd) == 0) {
|
||||
// Show status
|
||||
// Count whitelisted pubkeys
|
||||
int whitelist_count = db_count_wot_whitelist_rules();
|
||||
|
||||
const char* level_str;
|
||||
if (wot_level == 0) level_str = "Off (open relay)";
|
||||
else if (wot_level == 1) level_str = "Write-only (followed pubkeys can publish)";
|
||||
else if (wot_level == 2) level_str = "Full (followed pubkeys can publish AND subscribe)";
|
||||
else level_str = "Unknown";
|
||||
|
||||
snprintf(response_msg, sizeof(response_msg),
|
||||
"📊 Web of Trust Status\n"
|
||||
"━━━━━━━━━━━━━━━━━━━━━━\n"
|
||||
"\n"
|
||||
"Level: %d\n"
|
||||
"%s\n"
|
||||
"\n"
|
||||
"Whitelisted pubkeys: %d\n"
|
||||
"\n"
|
||||
"Commands:\n"
|
||||
" wot 0/off - Disable WoT\n"
|
||||
" wot 1/write - Write-only mode\n"
|
||||
" wot 2/full - Full restriction mode\n"
|
||||
" wot sync - Force resync from kind 3\n"
|
||||
" wot status - Show this status",
|
||||
wot_level, level_str, whitelist_count);
|
||||
send_nip17_response(sender_pubkey, response_msg, NULL, 0);
|
||||
DEBUG_INFO("DM_ADMIN: WoT status displayed");
|
||||
}
|
||||
else {
|
||||
// Unknown wot subcommand
|
||||
snprintf(response_msg, sizeof(response_msg),
|
||||
"❌ Unknown WoT Command\n"
|
||||
"━━━━━━━━━━━━━━━━━━━━━━\n"
|
||||
"\n"
|
||||
"Usage: wot [command]\n"
|
||||
"\n"
|
||||
"Commands:\n"
|
||||
" 0, off - Disable WoT\n"
|
||||
" 1, write - Write-only mode\n"
|
||||
" 2, full - Full restriction mode\n"
|
||||
" sync - Force resync from kind 3\n"
|
||||
" status - Show current status");
|
||||
send_nip17_response(sender_pubkey, response_msg, NULL, 0);
|
||||
}
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
else {
|
||||
DEBUG_INFO("DM_ADMIN: Checking for confirmation or config change requests");
|
||||
// Check if it's a confirmation response (yes/no)
|
||||
@@ -652,7 +791,7 @@ int process_nip17_admin_command(cJSON* dm_event, char* error_message, size_t err
|
||||
if (cJSON_IsString(first_item) && strcmp(cJSON_GetStringValue(first_item), "stats") == 0) {
|
||||
DEBUG_INFO("DM_ADMIN: Processing JSON stats command");
|
||||
// Get sender pubkey for response
|
||||
cJSON* sender_pubkey_obj = cJSON_GetObjectItem(dm_event, "pubkey");
|
||||
cJSON* sender_pubkey_obj = cJSON_GetObjectItemCaseSensitive(dm_event, "pubkey");
|
||||
if (!sender_pubkey_obj || !cJSON_IsString(sender_pubkey_obj)) {
|
||||
cJSON_Delete(command_array);
|
||||
DEBUG_INFO("DM_ADMIN: DM missing sender pubkey for stats command");
|
||||
@@ -694,13 +833,13 @@ int process_nip17_admin_command(cJSON* dm_event, char* error_message, size_t err
|
||||
cJSON_AddStringToObject(synthetic_event, "content", dm_content);
|
||||
|
||||
// Copy pubkey from DM
|
||||
cJSON* pubkey_obj = cJSON_GetObjectItem(dm_event, "pubkey");
|
||||
cJSON* pubkey_obj = cJSON_GetObjectItemCaseSensitive(dm_event, "pubkey");
|
||||
if (pubkey_obj && cJSON_IsString(pubkey_obj)) {
|
||||
cJSON_AddStringToObject(synthetic_event, "pubkey", cJSON_GetStringValue(pubkey_obj));
|
||||
}
|
||||
|
||||
// Copy tags from DM
|
||||
cJSON* tags = cJSON_GetObjectItem(dm_event, "tags");
|
||||
cJSON* tags = cJSON_GetObjectItemCaseSensitive(dm_event, "tags");
|
||||
if (tags) {
|
||||
cJSON_AddItemToObject(synthetic_event, "tags", cJSON_Duplicate(tags, 1));
|
||||
}
|
||||
|
||||
File diff suppressed because one or more lines are too long
+620
@@ -0,0 +1,620 @@
|
||||
#define _GNU_SOURCE
|
||||
#include "ip_ban.h"
|
||||
#include "debug.h"
|
||||
#include "config.h"
|
||||
#include "db_ops.h"
|
||||
#include "thread_pool.h"
|
||||
#include <string.h>
|
||||
#include <stdlib.h>
|
||||
#include <pthread.h>
|
||||
|
||||
// ============================================================
|
||||
// IP Auth Failure Ban System
|
||||
//
|
||||
// Fixed-size open-addressing hash table. No malloc after init.
|
||||
// Thread-safe via a single mutex (low contention — only called
|
||||
// at connection open/close, not in the hot event path).
|
||||
//
|
||||
// State is persisted to the ip_bans SQLite table every 5 minutes
|
||||
// and loaded at startup so bans survive relay restarts.
|
||||
// ============================================================
|
||||
|
||||
#define IP_BAN_EMPTY 0
|
||||
#define IP_BAN_ACTIVE 1
|
||||
|
||||
typedef struct {
|
||||
int state; // IP_BAN_EMPTY or IP_BAN_ACTIVE
|
||||
char ip[46]; // IPv4 or IPv6 string
|
||||
|
||||
// Auth failure tracking (existing)
|
||||
int failure_count; // failures in current window
|
||||
time_t first_failure; // start of current failure window
|
||||
time_t banned_until; // 0 = not banned
|
||||
int ban_count; // escalation level (for exponential backoff)
|
||||
|
||||
// NEW: Idle failure tracking (separate)
|
||||
int idle_failure_count;
|
||||
time_t idle_first_failure;
|
||||
time_t idle_banned_until;
|
||||
int idle_ban_count;
|
||||
|
||||
// Other existing fields
|
||||
int has_authed_successfully; // 1 if this IP has ever authenticated
|
||||
time_t last_success_at; // timestamp of last successful auth
|
||||
int total_connections; // lifetime connection count
|
||||
int total_failures; // lifetime auth failure count
|
||||
int total_successes; // lifetime successful auth count
|
||||
time_t first_seen; // when this IP was first seen
|
||||
} ip_ban_entry_t;
|
||||
|
||||
static ip_ban_entry_t g_ban_table[IP_BAN_TABLE_SIZE];
|
||||
static pthread_mutex_t g_ban_mutex = PTHREAD_MUTEX_INITIALIZER;
|
||||
static int g_initialized = 0;
|
||||
|
||||
// Simple FNV-1a hash for IP strings
|
||||
static unsigned int ip_hash(const char* ip) {
|
||||
unsigned int hash = 2166136261u;
|
||||
while (*ip) {
|
||||
hash ^= (unsigned char)*ip++;
|
||||
hash *= 16777619u;
|
||||
}
|
||||
return hash % IP_BAN_TABLE_SIZE;
|
||||
}
|
||||
|
||||
// Find slot for IP (open addressing with linear probing)
|
||||
static int find_slot(const char* ip) {
|
||||
unsigned int start = ip_hash(ip);
|
||||
for (unsigned int i = 0; i < IP_BAN_TABLE_SIZE; i++) {
|
||||
unsigned int idx = (start + i) % IP_BAN_TABLE_SIZE;
|
||||
if (g_ban_table[idx].state == IP_BAN_EMPTY) {
|
||||
return (int)idx;
|
||||
}
|
||||
if (strcmp(g_ban_table[idx].ip, ip) == 0) {
|
||||
return (int)idx;
|
||||
}
|
||||
}
|
||||
return -1;
|
||||
}
|
||||
|
||||
// Get or create an entry for an IP. Returns NULL if table is full.
|
||||
// Caller must hold g_ban_mutex.
|
||||
static ip_ban_entry_t* get_or_create_entry(const char* ip) {
|
||||
int idx = find_slot(ip);
|
||||
if (idx < 0) {
|
||||
DEBUG_WARN("IP ban table full, cannot track %s", ip);
|
||||
return NULL;
|
||||
}
|
||||
ip_ban_entry_t* entry = &g_ban_table[idx];
|
||||
if (entry->state == IP_BAN_EMPTY) {
|
||||
entry->state = IP_BAN_ACTIVE;
|
||||
strncpy(entry->ip, ip, sizeof(entry->ip) - 1);
|
||||
entry->ip[sizeof(entry->ip) - 1] = '\0';
|
||||
entry->first_seen = time(NULL);
|
||||
}
|
||||
return entry;
|
||||
}
|
||||
|
||||
void ip_ban_init(void) {
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
memset(g_ban_table, 0, sizeof(g_ban_table));
|
||||
g_initialized = 1;
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
DEBUG_LOG("IP ban table initialized (%d slots)", IP_BAN_TABLE_SIZE);
|
||||
}
|
||||
|
||||
void ip_ban_load_from_db(void) {
|
||||
if (!db_is_available() || !g_initialized) return;
|
||||
|
||||
// Create table if it doesn't exist (handles existing databases)
|
||||
const char* create_sql =
|
||||
"CREATE TABLE IF NOT EXISTS ip_bans ("
|
||||
" ip TEXT PRIMARY KEY,"
|
||||
" failure_count INTEGER NOT NULL DEFAULT 0,"
|
||||
" ban_count INTEGER NOT NULL DEFAULT 0,"
|
||||
" banned_until INTEGER NOT NULL DEFAULT 0,"
|
||||
" first_failure INTEGER NOT NULL DEFAULT 0,"
|
||||
" has_authed_successfully INTEGER NOT NULL DEFAULT 0,"
|
||||
" last_success_at INTEGER NOT NULL DEFAULT 0,"
|
||||
" total_connections INTEGER NOT NULL DEFAULT 0,"
|
||||
" total_failures INTEGER NOT NULL DEFAULT 0,"
|
||||
" total_successes INTEGER NOT NULL DEFAULT 0,"
|
||||
" first_seen INTEGER NOT NULL DEFAULT 0,"
|
||||
" updated_at INTEGER NOT NULL DEFAULT 0"
|
||||
");";
|
||||
if (db_exec_sql(create_sql) != 0) {
|
||||
DEBUG_ERROR("Failed to create ip_bans table: %s", db_last_error());
|
||||
return;
|
||||
}
|
||||
|
||||
// Migration: Add idle_* columns if they don't exist (ignore errors if already exists)
|
||||
(void)db_exec_sql("ALTER TABLE ip_bans ADD COLUMN idle_failure_count INTEGER NOT NULL DEFAULT 0");
|
||||
(void)db_exec_sql("ALTER TABLE ip_bans ADD COLUMN idle_ban_count INTEGER NOT NULL DEFAULT 0");
|
||||
(void)db_exec_sql("ALTER TABLE ip_bans ADD COLUMN idle_banned_until INTEGER NOT NULL DEFAULT 0");
|
||||
(void)db_exec_sql("ALTER TABLE ip_bans ADD COLUMN idle_first_failure INTEGER NOT NULL DEFAULT 0");
|
||||
|
||||
const char* sql =
|
||||
"SELECT ip, failure_count, ban_count, banned_until, first_failure,"
|
||||
" has_authed_successfully, last_success_at, total_connections,"
|
||||
" total_failures, total_successes, first_seen,"
|
||||
" idle_failure_count, idle_ban_count, idle_banned_until, idle_first_failure"
|
||||
" FROM ip_bans";
|
||||
|
||||
db_stmt_t* stmt;
|
||||
if (db_prepare(sql, &stmt) != DB_OK) {
|
||||
DEBUG_ERROR("Failed to prepare ip_bans load query");
|
||||
return;
|
||||
}
|
||||
|
||||
int loaded = 0;
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
while (db_step_stmt(stmt) == DB_ROW) {
|
||||
const char* ip = db_column_text_value(stmt, 0);
|
||||
if (!ip) continue;
|
||||
|
||||
int idx = find_slot(ip);
|
||||
if (idx < 0) continue;
|
||||
|
||||
ip_ban_entry_t* entry = &g_ban_table[idx];
|
||||
entry->state = IP_BAN_ACTIVE;
|
||||
strncpy(entry->ip, ip, sizeof(entry->ip) - 1);
|
||||
entry->ip[sizeof(entry->ip) - 1] = '\0';
|
||||
entry->failure_count = db_column_int_value(stmt, 1);
|
||||
entry->ban_count = db_column_int_value(stmt, 2);
|
||||
entry->banned_until = (time_t)db_column_int64_value(stmt, 3);
|
||||
entry->first_failure = (time_t)db_column_int64_value(stmt, 4);
|
||||
entry->has_authed_successfully = db_column_int_value(stmt, 5);
|
||||
entry->last_success_at = (time_t)db_column_int64_value(stmt, 6);
|
||||
entry->total_connections = db_column_int_value(stmt, 7);
|
||||
entry->total_failures = db_column_int_value(stmt, 8);
|
||||
entry->total_successes = db_column_int_value(stmt, 9);
|
||||
entry->first_seen = (time_t)db_column_int64_value(stmt, 10);
|
||||
// Load idle tracking fields (default to 0 if columns don't exist yet)
|
||||
entry->idle_failure_count = db_column_int_value(stmt, 11);
|
||||
entry->idle_ban_count = db_column_int_value(stmt, 12);
|
||||
entry->idle_banned_until = (time_t)db_column_int64_value(stmt, 13);
|
||||
entry->idle_first_failure = (time_t)db_column_int64_value(stmt, 14);
|
||||
loaded++;
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
db_finalize_stmt(stmt);
|
||||
|
||||
// Count how many are still actively banned
|
||||
time_t now = time(NULL);
|
||||
int still_banned = 0;
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
for (int i = 0; i < IP_BAN_TABLE_SIZE; i++) {
|
||||
if (g_ban_table[i].state == IP_BAN_ACTIVE &&
|
||||
g_ban_table[i].banned_until > now) {
|
||||
still_banned++;
|
||||
}
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
|
||||
DEBUG_WARN("IP ban table loaded: %d IPs restored (%d still banned)", loaded, still_banned);
|
||||
}
|
||||
|
||||
void ip_ban_save_to_db(void) {
|
||||
if (!db_is_available() || !g_initialized) return;
|
||||
|
||||
const char* upsert_sql =
|
||||
"INSERT INTO ip_bans"
|
||||
" (ip, failure_count, ban_count, banned_until, first_failure,"
|
||||
" has_authed_successfully, last_success_at, total_connections,"
|
||||
" total_failures, total_successes, first_seen, updated_at,"
|
||||
" idle_failure_count, idle_ban_count, idle_banned_until, idle_first_failure)"
|
||||
" VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"
|
||||
" ON CONFLICT(ip) DO UPDATE SET"
|
||||
" failure_count=excluded.failure_count,"
|
||||
" ban_count=excluded.ban_count,"
|
||||
" banned_until=excluded.banned_until,"
|
||||
" first_failure=excluded.first_failure,"
|
||||
" has_authed_successfully=excluded.has_authed_successfully,"
|
||||
" last_success_at=excluded.last_success_at,"
|
||||
" total_connections=excluded.total_connections,"
|
||||
" total_failures=excluded.total_failures,"
|
||||
" total_successes=excluded.total_successes,"
|
||||
" first_seen=excluded.first_seen,"
|
||||
" updated_at=excluded.updated_at,"
|
||||
" idle_failure_count=excluded.idle_failure_count,"
|
||||
" idle_ban_count=excluded.idle_ban_count,"
|
||||
" idle_banned_until=excluded.idle_banned_until,"
|
||||
" idle_first_failure=excluded.idle_first_failure";
|
||||
|
||||
db_stmt_t* stmt;
|
||||
if (db_prepare(upsert_sql, &stmt) != DB_OK) {
|
||||
DEBUG_ERROR("Failed to prepare ip_bans save query");
|
||||
return;
|
||||
}
|
||||
|
||||
(void)db_exec_sql("BEGIN TRANSACTION");
|
||||
|
||||
int saved = 0;
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
for (int i = 0; i < IP_BAN_TABLE_SIZE; i++) {
|
||||
ip_ban_entry_t* entry = &g_ban_table[i];
|
||||
if (entry->state != IP_BAN_ACTIVE) continue;
|
||||
|
||||
(void)db_reset_stmt(stmt);
|
||||
db_bind_text_param(stmt, 1, entry->ip);
|
||||
db_bind_int_param(stmt, 2, entry->failure_count);
|
||||
db_bind_int_param(stmt, 3, entry->ban_count);
|
||||
db_bind_int64_param(stmt, 4, (long long)entry->banned_until);
|
||||
db_bind_int64_param(stmt, 5, (long long)entry->first_failure);
|
||||
db_bind_int_param(stmt, 6, entry->has_authed_successfully);
|
||||
db_bind_int64_param(stmt, 7, (long long)entry->last_success_at);
|
||||
db_bind_int_param(stmt, 8, entry->total_connections);
|
||||
db_bind_int_param(stmt, 9, entry->total_failures);
|
||||
db_bind_int_param(stmt, 10, entry->total_successes);
|
||||
db_bind_int64_param(stmt, 11, (long long)entry->first_seen);
|
||||
db_bind_int64_param(stmt, 12, (long long)time(NULL));
|
||||
// Idle tracking fields
|
||||
db_bind_int_param(stmt, 13, entry->idle_failure_count);
|
||||
db_bind_int_param(stmt, 14, entry->idle_ban_count);
|
||||
db_bind_int64_param(stmt, 15, (long long)entry->idle_banned_until);
|
||||
db_bind_int64_param(stmt, 16, (long long)entry->idle_first_failure);
|
||||
|
||||
if (db_step_stmt(stmt) != DB_DONE) {
|
||||
DEBUG_WARN("Failed to save ip_ban entry for %s", entry->ip);
|
||||
} else {
|
||||
saved++;
|
||||
}
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
|
||||
db_finalize_stmt(stmt);
|
||||
(void)db_exec_sql("COMMIT");
|
||||
|
||||
DEBUG_TRACE("IP ban table saved: %d entries written to DB", saved);
|
||||
}
|
||||
|
||||
// Check if an IP is in the idle_ban_whitelist config (comma-separated list)
|
||||
static int ip_is_whitelisted(const char* ip) {
|
||||
if (!ip) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Always trust loopback to avoid local test/dev self-bans.
|
||||
if (strcmp(ip, "127.0.0.1") == 0 || strcmp(ip, "::1") == 0) {
|
||||
return 1;
|
||||
}
|
||||
|
||||
const char* whitelist = get_config_value("idle_ban_whitelist");
|
||||
if (!whitelist || whitelist[0] == '\0') {
|
||||
if (whitelist) free((char*)whitelist);
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Make a mutable copy to tokenize
|
||||
char buf[1024];
|
||||
strncpy(buf, whitelist, sizeof(buf) - 1);
|
||||
buf[sizeof(buf) - 1] = '\0';
|
||||
|
||||
int is_match = 0;
|
||||
char* token = strtok(buf, ",");
|
||||
while (token) {
|
||||
// Trim leading/trailing spaces
|
||||
while (*token == ' ') token++;
|
||||
char* end = token + strlen(token) - 1;
|
||||
while (end > token && *end == ' ') { *end = '\0'; end--; }
|
||||
if (strcmp(token, ip) == 0) {
|
||||
is_match = 1;
|
||||
break;
|
||||
}
|
||||
token = strtok(NULL, ",");
|
||||
}
|
||||
|
||||
free((char*)whitelist);
|
||||
return is_match;
|
||||
}
|
||||
|
||||
int ip_ban_is_banned(const char* ip) {
|
||||
if (!ip || !g_initialized) return 0;
|
||||
|
||||
// Whitelisted IPs are never banned
|
||||
if (ip_is_whitelisted(ip)) return 0;
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
int idx = find_slot(ip);
|
||||
if (idx < 0 || g_ban_table[idx].state == IP_BAN_EMPTY) {
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return 0;
|
||||
}
|
||||
|
||||
ip_ban_entry_t* entry = &g_ban_table[idx];
|
||||
time_t now = time(NULL);
|
||||
int banned = 0;
|
||||
|
||||
// Check auth ban (if enabled)
|
||||
if (get_config_bool("auth_fail_ban_enabled", 1) &&
|
||||
entry->banned_until > 0 && now < entry->banned_until) {
|
||||
banned = 1;
|
||||
}
|
||||
|
||||
// Check idle ban (if enabled)
|
||||
if (get_config_bool("idle_ban_enabled", 1) &&
|
||||
entry->idle_banned_until > 0 && now < entry->idle_banned_until) {
|
||||
banned = 1;
|
||||
}
|
||||
|
||||
// Clear expired bans
|
||||
if (!banned) {
|
||||
if (entry->banned_until > 0 && now >= entry->banned_until) {
|
||||
entry->banned_until = 0;
|
||||
entry->failure_count = 0;
|
||||
entry->first_failure = 0;
|
||||
}
|
||||
if (entry->idle_banned_until > 0 && now >= entry->idle_banned_until) {
|
||||
entry->idle_banned_until = 0;
|
||||
entry->idle_failure_count = 0;
|
||||
entry->idle_first_failure = 0;
|
||||
}
|
||||
}
|
||||
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return banned;
|
||||
}
|
||||
|
||||
void ip_ban_record_connection(const char* ip) {
|
||||
if (!ip || !g_initialized) return;
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
ip_ban_entry_t* entry = get_or_create_entry(ip);
|
||||
if (entry) {
|
||||
entry->total_connections++;
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
}
|
||||
|
||||
void ip_ban_record_failure(const char* ip) {
|
||||
if (!ip || !g_initialized) return;
|
||||
if (!get_config_bool("auth_fail_ban_enabled", 1)) return;
|
||||
|
||||
int threshold = get_config_int("auth_fail_ban_threshold", 3);
|
||||
int window_sec = get_config_int("auth_fail_window_sec", 60);
|
||||
int ban_duration = get_config_int("auth_fail_ban_duration_sec", 300);
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
ip_ban_entry_t* entry = get_or_create_entry(ip);
|
||||
if (!entry) {
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return;
|
||||
}
|
||||
|
||||
time_t now = time(NULL);
|
||||
|
||||
// Reset window if expired
|
||||
if (entry->first_failure > 0 && (now - entry->first_failure) > window_sec) {
|
||||
entry->failure_count = 0;
|
||||
entry->first_failure = now;
|
||||
}
|
||||
if (entry->first_failure == 0) {
|
||||
entry->first_failure = now;
|
||||
}
|
||||
|
||||
entry->failure_count++;
|
||||
entry->total_failures++;
|
||||
|
||||
DEBUG_TRACE("IP %s auth failure count: %d/%d", ip, entry->failure_count, threshold);
|
||||
|
||||
if (entry->failure_count >= threshold) {
|
||||
int duration = ban_duration;
|
||||
for (int i = 0; i < entry->ban_count && duration < 86400; i++) {
|
||||
duration *= 2;
|
||||
}
|
||||
if (duration > 86400) duration = 86400;
|
||||
|
||||
entry->banned_until = now + duration;
|
||||
entry->ban_count++;
|
||||
entry->failure_count = 0;
|
||||
entry->first_failure = 0;
|
||||
|
||||
DEBUG_WARN("IP %s banned for %d seconds (ban #%d) after %d auth failures",
|
||||
ip, duration, entry->ban_count, threshold);
|
||||
}
|
||||
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
}
|
||||
|
||||
// Record an idle/early-disconnect failure for an IP
|
||||
void ip_ban_record_idle_failure(const char* ip) {
|
||||
if (!ip || !g_initialized) return;
|
||||
if (!get_config_bool("idle_ban_enabled", 1)) return;
|
||||
if (ip_is_whitelisted(ip)) return; // Never record idle failures for whitelisted IPs
|
||||
|
||||
int threshold = get_config_int("idle_ban_threshold", 1);
|
||||
int window_sec = get_config_int("idle_ban_window_sec", 30);
|
||||
int ban_duration = get_config_int("idle_ban_duration_sec", 300);
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
ip_ban_entry_t* entry = get_or_create_entry(ip);
|
||||
if (!entry) {
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return;
|
||||
}
|
||||
|
||||
time_t now = time(NULL);
|
||||
|
||||
// Reset window if expired
|
||||
if (entry->idle_first_failure > 0 && (now - entry->idle_first_failure) > window_sec) {
|
||||
entry->idle_failure_count = 0;
|
||||
entry->idle_first_failure = now;
|
||||
}
|
||||
if (entry->idle_first_failure == 0) {
|
||||
entry->idle_first_failure = now;
|
||||
}
|
||||
|
||||
entry->idle_failure_count++;
|
||||
entry->total_failures++;
|
||||
|
||||
DEBUG_TRACE("IP %s idle failure count: %d/%d", ip, entry->idle_failure_count, threshold);
|
||||
|
||||
if (entry->idle_failure_count >= threshold) {
|
||||
int duration = ban_duration;
|
||||
for (int i = 0; i < entry->idle_ban_count && duration < 86400; i++) {
|
||||
duration *= 2;
|
||||
}
|
||||
if (duration > 86400) duration = 86400;
|
||||
|
||||
entry->idle_banned_until = now + duration;
|
||||
entry->idle_ban_count++;
|
||||
entry->idle_failure_count = 0;
|
||||
entry->idle_first_failure = 0;
|
||||
|
||||
DEBUG_WARN("IP %s banned for %d seconds (idle ban #%d) after %d idle failures",
|
||||
ip, duration, entry->idle_ban_count, threshold);
|
||||
}
|
||||
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
}
|
||||
|
||||
void ip_ban_record_success(const char* ip) {
|
||||
if (!ip || !g_initialized) return;
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
ip_ban_entry_t* entry = get_or_create_entry(ip);
|
||||
if (entry) {
|
||||
entry->failure_count = 0;
|
||||
entry->first_failure = 0;
|
||||
entry->has_authed_successfully = 1;
|
||||
entry->last_success_at = time(NULL);
|
||||
entry->total_successes++;
|
||||
DEBUG_TRACE("IP %s authenticated successfully — failure count cleared", ip);
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
}
|
||||
|
||||
void ip_ban_cleanup(void) {
|
||||
if (!g_initialized) return;
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
time_t now = time(NULL);
|
||||
int window_sec = get_config_int("auth_fail_window_sec", 60);
|
||||
int idle_window_sec = get_config_int("idle_ban_window_sec", 60);
|
||||
int cleaned = 0;
|
||||
|
||||
for (int i = 0; i < IP_BAN_TABLE_SIZE; i++) {
|
||||
if (g_ban_table[i].state != IP_BAN_ACTIVE) continue;
|
||||
ip_ban_entry_t* entry = &g_ban_table[i];
|
||||
|
||||
// Check auth failure window expiration
|
||||
int auth_ban_expired = (entry->banned_until == 0 || now >= entry->banned_until);
|
||||
int auth_window_expired = (entry->first_failure == 0 || (now - entry->first_failure) > window_sec * 10);
|
||||
|
||||
// Check idle failure window expiration
|
||||
int idle_ban_expired = (entry->idle_banned_until == 0 || now >= entry->idle_banned_until);
|
||||
int idle_window_expired = (entry->idle_first_failure == 0 || (now - entry->idle_first_failure) > idle_window_sec * 10);
|
||||
|
||||
if (auth_ban_expired && auth_window_expired && entry->failure_count == 0 &&
|
||||
idle_ban_expired && idle_window_expired && entry->idle_failure_count == 0) {
|
||||
int retain_sec = 86400; // 24 hours
|
||||
int last_auth_ban_expired_long_ago = (entry->banned_until == 0 ||
|
||||
(now - entry->banned_until) > retain_sec);
|
||||
int last_idle_ban_expired_long_ago = (entry->idle_banned_until == 0 ||
|
||||
(now - entry->idle_banned_until) > retain_sec);
|
||||
|
||||
if (last_auth_ban_expired_long_ago && last_idle_ban_expired_long_ago &&
|
||||
!entry->has_authed_successfully &&
|
||||
entry->ban_count == 0 && entry->idle_ban_count == 0 &&
|
||||
entry->total_connections <= 1) {
|
||||
// Fully clean — never banned, never authenticated, only seen once
|
||||
memset(entry, 0, sizeof(ip_ban_entry_t));
|
||||
cleaned++;
|
||||
} else {
|
||||
// Keep entry permanently — preserve ban_count for escalation.
|
||||
// An IP that has been banned before will always get at least a 24-hour ban
|
||||
// if it fails auth again, regardless of how long it has been away.
|
||||
entry->failure_count = 0;
|
||||
entry->first_failure = 0;
|
||||
entry->idle_failure_count = 0;
|
||||
entry->idle_first_failure = 0;
|
||||
if (last_auth_ban_expired_long_ago) {
|
||||
entry->banned_until = 0;
|
||||
// ban_count intentionally NOT reset — permanent escalation
|
||||
}
|
||||
if (last_idle_ban_expired_long_ago) {
|
||||
entry->idle_banned_until = 0;
|
||||
// idle_ban_count intentionally NOT reset — permanent escalation
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (cleaned > 0) {
|
||||
DEBUG_TRACE("IP ban cleanup: freed %d stale entries", cleaned);
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
}
|
||||
|
||||
int ip_ban_get_banned_count(void) {
|
||||
if (!g_initialized) return 0;
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
time_t now = time(NULL);
|
||||
int count = 0;
|
||||
for (int i = 0; i < IP_BAN_TABLE_SIZE; i++) {
|
||||
if (g_ban_table[i].state != IP_BAN_ACTIVE) continue;
|
||||
// Count if either auth banned or idle banned
|
||||
if (g_ban_table[i].banned_until > now ||
|
||||
g_ban_table[i].idle_banned_until > now) {
|
||||
count++;
|
||||
}
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return count;
|
||||
}
|
||||
|
||||
int ip_ban_get_tracked_count(void) {
|
||||
if (!g_initialized) return 0;
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
int count = 0;
|
||||
for (int i = 0; i < IP_BAN_TABLE_SIZE; i++) {
|
||||
if (g_ban_table[i].state == IP_BAN_ACTIVE) count++;
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
return count;
|
||||
}
|
||||
|
||||
void ip_ban_log_stats(void) {
|
||||
if (!g_initialized) return;
|
||||
|
||||
static time_t last_log = 0;
|
||||
time_t now = time(NULL);
|
||||
|
||||
if (now - last_log < 300) return;
|
||||
last_log = now;
|
||||
|
||||
// Save to DB every 5 minutes
|
||||
if (thread_pool_is_running()) {
|
||||
thread_pool_job_t job;
|
||||
memset(&job, 0, sizeof(job));
|
||||
job.type = THREAD_POOL_JOB_IP_BAN_SAVE;
|
||||
|
||||
thread_pool_status_t rc = thread_pool_submit_write(&job, NULL);
|
||||
if (rc != THREAD_POOL_STATUS_OK) {
|
||||
DEBUG_WARN("Failed to queue IP_BAN_SAVE job (%d); saving synchronously", rc);
|
||||
ip_ban_save_to_db();
|
||||
}
|
||||
} else {
|
||||
ip_ban_save_to_db();
|
||||
}
|
||||
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
int auth_banned_count = 0;
|
||||
int idle_banned_count = 0;
|
||||
int tracked_count = 0;
|
||||
int trusted_count = 0;
|
||||
|
||||
for (int i = 0; i < IP_BAN_TABLE_SIZE; i++) {
|
||||
if (g_ban_table[i].state != IP_BAN_ACTIVE) continue;
|
||||
tracked_count++;
|
||||
if (g_ban_table[i].banned_until > now) auth_banned_count++;
|
||||
if (g_ban_table[i].idle_banned_until > now) idle_banned_count++;
|
||||
if (g_ban_table[i].has_authed_successfully) trusted_count++;
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
|
||||
int total_banned = auth_banned_count + idle_banned_count;
|
||||
if (total_banned > 0 || tracked_count > 0) {
|
||||
DEBUG_WARN("IP BAN SUMMARY: %d auth-banned, %d idle-banned, %d tracked, %d trusted (ever authed)",
|
||||
auth_banned_count, idle_banned_count, tracked_count, trusted_count);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
#ifndef IP_BAN_H
|
||||
#define IP_BAN_H
|
||||
|
||||
// IP Auth Failure Ban System
|
||||
//
|
||||
// Tracks auth failures per IP address and temporarily bans IPs that repeatedly
|
||||
// fail NIP-42 authentication. Uses an in-memory fixed-size hash table — no
|
||||
// database writes on the hot path. State is persisted to the ip_bans table
|
||||
// every 5 minutes and loaded at startup.
|
||||
//
|
||||
// Config keys (all read from the config table at runtime):
|
||||
// auth_fail_ban_enabled bool default: true (0 = disabled)
|
||||
// auth_fail_ban_threshold int default: 3 (failures before ban)
|
||||
// auth_fail_window_sec int default: 60 (window to count failures)
|
||||
// auth_fail_ban_duration_sec int default: 300 (initial ban duration, doubles each time)
|
||||
|
||||
#include <time.h>
|
||||
|
||||
// Maximum number of IPs tracked simultaneously (fixed-size, no malloc)
|
||||
#define IP_BAN_TABLE_SIZE 4096
|
||||
|
||||
// Initialize the IP ban table (call once at startup, before loading from DB)
|
||||
void ip_ban_init(void);
|
||||
|
||||
// Load ban state from the ip_bans database table.
|
||||
// Call after ip_ban_init() and after the database is open.
|
||||
void ip_ban_load_from_db(void);
|
||||
|
||||
// Save current ban state to the ip_bans database table.
|
||||
// Called every 5 minutes from the maintenance timer.
|
||||
void ip_ban_save_to_db(void);
|
||||
|
||||
// Check if an IP is currently banned.
|
||||
// Returns 1 if banned (connection should be rejected), 0 if allowed.
|
||||
int ip_ban_is_banned(const char* ip);
|
||||
|
||||
// Record an auth failure for an IP.
|
||||
// Called when a connection is closed due to auth timeout.
|
||||
// May trigger a ban if the threshold is exceeded.
|
||||
void ip_ban_record_failure(const char* ip);
|
||||
|
||||
// Record an idle/early-disconnect failure for an IP.
|
||||
// Called when a connection closes without ever sending REQ or EVENT.
|
||||
// May trigger a ban if the threshold is exceeded.
|
||||
void ip_ban_record_idle_failure(const char* ip);
|
||||
|
||||
// Record a successful auth for an IP.
|
||||
// Sets has_authed_successfully=1 and clears failure count.
|
||||
void ip_ban_record_success(const char* ip);
|
||||
|
||||
// Record a new connection from an IP (increment total_connections).
|
||||
void ip_ban_record_connection(const char* ip);
|
||||
|
||||
// Periodic cleanup: expire old entries (call from the connection age checker).
|
||||
void ip_ban_cleanup(void);
|
||||
|
||||
// Emit a periodic WARN-level log summary of banned IPs (every 5 minutes).
|
||||
// Also saves state to DB.
|
||||
void ip_ban_log_stats(void);
|
||||
|
||||
// Get stats for logging/monitoring
|
||||
int ip_ban_get_banned_count(void);
|
||||
int ip_ban_get_tracked_count(void);
|
||||
|
||||
#endif // IP_BAN_H
|
||||
+1251
-438
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user