Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
cbd260c1ae | ||
|
|
bc9ac290ab |
Submodule
+1
Submodule nostr-rs-relay added at 64cfcaf44a
@@ -0,0 +1,257 @@
|
||||
# PostgreSQL Nostr Relay Architecture Analysis
|
||||
|
||||
This document summarizes common approaches and best practices for building Nostr relays backed by PostgreSQL, based on industry-standard designs and existing open-source relay implementations.
|
||||
|
||||
## Identified Projects
|
||||
* **Nostrss**: High-performance Rust-based relay.
|
||||
* **Nostrgres**: Focused on PostgreSQL integration with complex event filtering.
|
||||
* **Relay-rs (Postgres branch)**: General purpose high-throughput relay.
|
||||
* **Nostream** ([github.com/Cameri/nostream](https://github.com/Cameri/nostream)): Production-ready TypeScript relay backed by PostgreSQL and Redis.
|
||||
* **Nostr-Relay-NestJS** ([github.com/CodyTseng/nostr-relay-nestjs](https://github.com/CodyTseng/nostr-relay-nestjs)): High-performance relay built with NestJS and PostgreSQL, utilizing Kysely for type-safe database interactions.
|
||||
|
||||
## Detailed Analysis
|
||||
|
||||
### 1. Common Schema Patterns
|
||||
* **Event Storage (JSONB)**: Most PostgreSQL-backed relays store the raw event as a JSONB object in a central `events` table. This provides flexibility for the evolving Nostr spec while allowing efficient extraction of fields.
|
||||
* **Tag Denormalization**: While the raw event is in JSONB, performant relays extract tags (like `p`, `e`, `t`, `d`) into a separate `tags` table with foreign key relationships to the `events` table. This avoids slow JSONB traversal during complex filter queries.
|
||||
* **Author/Publisher Tracking**: A dedicated `authors` or `pubkeys` table is typically used to maintain indexing on the `pubkey` field, enabling quick lookups of user activity.
|
||||
|
||||
### 2. Performance Optimization
|
||||
* **GIN Indexes on JSONB**: Crucial for filtering on specific event tags or custom properties stored within the event object. GIN indexes with `jsonb_path_ops` are generally preferred for equality checks.
|
||||
* **Time-Series Partitioning**: Given the append-only nature of Nostr, partitioning the `events` table by time (e.g., daily or weekly chunks) is highly recommended. This significantly improves query performance for recent data and simplifies data expiration/deletion.
|
||||
* **Clustering**: Clustering the `events` table by the timestamp index can reduce disk I/O, as it keeps temporally related events physically adjacent on the disk.
|
||||
|
||||
### 3. Schema Management Approaches
|
||||
* **Migrations**: Most mature relays utilize migration tools (like `Flyway`, `Diesel migrations`, or `Golang-migrate`) to version control database schema changes. This is critical for production stability.
|
||||
* **Auto-generation**: Some lightweight prototypes auto-generate schemas at startup. This is generally discouraged for production due to the risk of destructive changes, data loss, or blocking DDL operations on large tables.
|
||||
|
||||
## Recommendations for Building a New Relay
|
||||
1. **Prioritize Denormalization**: Do not rely solely on JSONB queries for high-traffic filters. Extract indexed tags into dedicated columns or tables.
|
||||
2. **Use Partitioning from Day One**: Implementing native PostgreSQL partitioning (e.g., using `pg_partman`) is much easier before the dataset grows to millions of rows.
|
||||
3. **Connection Pooling**: PostgreSQL requires aggressive connection management. Use a high-performance pooler like `PgBouncer` to handle the large number of concurrent, short-lived connections common in WebSocket-based relay traffic.
|
||||
4. **Asynchronous Writes**: Decouple the WebSocket ingestion thread from the database writer thread to ensure that slow database writes do not impact the relay's responsiveness.
|
||||
|
||||
## Nostream ([github.com/Cameri/nostream](https://github.com/Cameri/nostream))
|
||||
|
||||
A production-ready Nostr relay written in TypeScript (Node.js), backed by PostgreSQL 14+ and Redis. Uses Knex.js for versioned migrations.
|
||||
|
||||
### PostgreSQL Schema
|
||||
|
||||
The schema is fully normalized — no JSONB blob for the whole event. Tags are stored separately and populated by a trigger.
|
||||
|
||||
**`events` table** — one row per event, tags kept as a JSONB column for the trigger to process:
|
||||
```sql
|
||||
CREATE TABLE events (
|
||||
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
|
||||
event_id BYTEA UNIQUE NOT NULL, -- 32-byte hash
|
||||
event_pubkey BYTEA NOT NULL,
|
||||
event_kind INTEGER UNSIGNED NOT NULL,
|
||||
event_created_at INTEGER UNSIGNED NOT NULL,
|
||||
event_content TEXT NOT NULL,
|
||||
event_tags JSONB, -- raw tags array, source of truth for trigger
|
||||
event_signature BYTEA NOT NULL,
|
||||
remote_address TEXT,
|
||||
expires_at INTEGER,
|
||||
deleted_at TIMESTAMP,
|
||||
event_deduplication JSONB, -- used for replaceable event conflict key
|
||||
first_seen TIMESTAMP DEFAULT now()
|
||||
);
|
||||
|
||||
-- Indexes
|
||||
CREATE INDEX ON events (event_pubkey);
|
||||
CREATE INDEX ON events (event_kind);
|
||||
CREATE INDEX ON events (event_created_at);
|
||||
CREATE INDEX ON events (expires_at);
|
||||
CREATE INDEX event_tags_idx ON events USING GIN (event_tags); -- GIN on raw JSONB
|
||||
|
||||
-- Unique index enforcing NIP-16 replaceable event semantics
|
||||
CREATE UNIQUE INDEX replaceable_events_idx ON events (event_pubkey, event_kind)
|
||||
WHERE event_kind = 0 OR event_kind = 3
|
||||
OR (event_kind >= 10000 AND event_kind < 20000);
|
||||
```
|
||||
|
||||
**`event_tags` table** — denormalized single-letter tags, populated by a PostgreSQL trigger on `events`:
|
||||
```sql
|
||||
CREATE TABLE event_tags (
|
||||
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
|
||||
event_id BYTEA NOT NULL,
|
||||
tag_name TEXT NOT NULL, -- single-letter only (e.g. 'e', 'p', 't')
|
||||
tag_value TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX ON event_tags (event_id);
|
||||
CREATE INDEX ON event_tags (tag_name, tag_value);
|
||||
|
||||
-- Trigger: on INSERT/UPDATE/DELETE to events, repopulate event_tags
|
||||
CREATE OR REPLACE FUNCTION process_event_tags() RETURNS TRIGGER AS $$
|
||||
DECLARE
|
||||
tag_element jsonb;
|
||||
BEGIN
|
||||
DELETE FROM event_tags WHERE event_id = OLD.event_id;
|
||||
IF TG_OP = 'INSERT' OR TG_OP = 'UPDATE' THEN
|
||||
FOR tag_element IN SELECT jsonb_array_elements(NEW.event_tags) LOOP
|
||||
IF length(trim('"' FROM (tag_element->0)::text)) = 1
|
||||
AND (tag_element->1)::text IS NOT NULL THEN
|
||||
INSERT INTO event_tags (event_id, tag_name, tag_value)
|
||||
VALUES (NEW.event_id,
|
||||
trim('"' FROM (tag_element->0)::text),
|
||||
trim('"' FROM (tag_element->1)::text));
|
||||
END IF;
|
||||
END LOOP;
|
||||
END IF;
|
||||
RETURN NEW;
|
||||
END;
|
||||
$$ LANGUAGE plpgsql;
|
||||
|
||||
CREATE TRIGGER insert_event_tags
|
||||
AFTER INSERT OR UPDATE OR DELETE ON events
|
||||
FOR EACH ROW EXECUTE FUNCTION process_event_tags();
|
||||
```
|
||||
|
||||
Tag queries (`#e`, `#p`, etc.) join `event_tags` rather than traversing the JSONB array.
|
||||
|
||||
**`users` and `invoices` tables** — for paid relay admission (Lightning payments):
|
||||
```sql
|
||||
CREATE TABLE users (
|
||||
pubkey BYTEA PRIMARY KEY,
|
||||
is_admitted BOOLEAN DEFAULT FALSE,
|
||||
balance BIGINT DEFAULT 0, -- in msats
|
||||
tos_accepted_at DATETIME
|
||||
);
|
||||
|
||||
CREATE TABLE invoices (
|
||||
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
|
||||
pubkey BYTEA NOT NULL,
|
||||
bolt11 TEXT NOT NULL,
|
||||
amount_requested BIGINT UNSIGNED NOT NULL,
|
||||
amount_paid BIGINT UNSIGNED,
|
||||
unit ENUM('msats','sats','btc'),
|
||||
status ENUM('pending','completed','expired'),
|
||||
description TEXT,
|
||||
confirmed_at DATETIME,
|
||||
expires_at DATETIME
|
||||
);
|
||||
```
|
||||
|
||||
### How Redis Is Used
|
||||
|
||||
Redis is **not** used for subscription fan-out. Subscription delivery is handled entirely in-process: each `WebSocketAdapter` holds a `Map<subscriptionId, filters[]>`, and a broadcast event walks all connected clients and filters locally.
|
||||
|
||||
Redis has two actual roles:
|
||||
|
||||
1. **Sliding-window rate limiting** — The `SlidingWindowRateLimiter` uses Redis sorted sets to track request timestamps per key (pubkey, IP, etc.) within a rolling time window:
|
||||
- `ZREMRANGEBYSCORE key 0 (now - period)` — evict old entries
|
||||
- `ZADD key timestamp member` — record this hit
|
||||
- `ZRANGE key 0 -1` — count hits in window
|
||||
- `EXPIRE key period` — auto-cleanup
|
||||
- Keys follow the pattern `<pubkey>:events:<period>` or `<ip>:message:<period>`
|
||||
|
||||
2. **General-purpose cache** — A `RedisAdapter` wraps the client with `get`/`set`/`hasKey` helpers, used for caching admission checks (e.g. `<pubkey>:is-admitted`). The event handler has a `TODO` comment noting that the `userRepository.findByPubkey()` call should be replaced with a cache lookup — meaning this is partially implemented.
|
||||
|
||||
---
|
||||
|
||||
## Schema Definitions
|
||||
|
||||
### Relay-rs (Postgres)
|
||||
```sql
|
||||
-- Events table
|
||||
CREATE TABLE "event" (
|
||||
id bytea NOT NULL,
|
||||
pub_key bytea NOT NULL,
|
||||
created_at timestamp with time zone NOT NULL,
|
||||
kind integer NOT NULL,
|
||||
"content" bytea NOT NULL,
|
||||
hidden bit(1) NOT NULL DEFAULT 0::bit(1),
|
||||
delegated_by bytea NULL,
|
||||
first_seen timestamp with time zone NOT NULL DEFAULT now(),
|
||||
expires_at timestamp(0) with time zone,
|
||||
CONSTRAINT event_pkey PRIMARY KEY (id)
|
||||
);
|
||||
CREATE INDEX event_created_at_idx ON "event" (created_at,kind);
|
||||
CREATE INDEX event_pub_key_idx ON "event" (pub_key);
|
||||
CREATE INDEX event_delegated_by_idx ON "event" (delegated_by);
|
||||
CREATE INDEX event_expires_at_idx ON "event" (expires_at);
|
||||
|
||||
-- Tags table
|
||||
CREATE TABLE "tag" (
|
||||
id int8 NOT NULL GENERATED BY DEFAULT AS IDENTITY,
|
||||
event_id bytea NOT NULL,
|
||||
"name" varchar NOT NULL,
|
||||
value bytea NULL,
|
||||
value_hex bytea NULL,
|
||||
CONSTRAINT tag_fk FOREIGN KEY (event_id) REFERENCES "event"(id) ON DELETE CASCADE,
|
||||
CONSTRAINT unique_constraint_name UNIQUE (event_id, "name", value, value_hex)
|
||||
);
|
||||
CREATE INDEX tag_event_id_idx ON tag USING btree (event_id, name);
|
||||
CREATE INDEX tag_value_idx ON tag USING btree (value);
|
||||
CREATE INDEX tag_value_hex_idx ON tag USING btree (value_hex);
|
||||
|
||||
-- Account table
|
||||
CREATE TABLE "account" (
|
||||
pubkey varchar NOT NULL,
|
||||
is_admitted BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
balance BIGINT NOT NULL DEFAULT 0,
|
||||
tos_accepted_at TIMESTAMP,
|
||||
CONSTRAINT account_pkey PRIMARY KEY (pubkey)
|
||||
);
|
||||
|
||||
-- Invoice table
|
||||
CREATE TYPE status AS ENUM ('Paid', 'Unpaid', 'Expired');
|
||||
CREATE TABLE "invoice" (
|
||||
payment_hash varchar NOT NULL,
|
||||
pubkey varchar NOT NULL,
|
||||
invoice varchar NOT NULL,
|
||||
amount BIGINT NOT NULL,
|
||||
status status NOT NULL DEFAULT 'Unpaid',
|
||||
description varchar,
|
||||
created_at timestamp,
|
||||
confirmed_at timestamp,
|
||||
CONSTRAINT invoice_payment_hash PRIMARY KEY (payment_hash),
|
||||
CONSTRAINT invoice_pubkey_fkey FOREIGN KEY (pubkey) REFERENCES account (pubkey) ON DELETE CASCADE
|
||||
);
|
||||
```
|
||||
|
||||
### Nostr-Relay-NestJS (PostgreSQL via Kysely)
|
||||
```sql
|
||||
-- Events table
|
||||
CREATE TABLE events (
|
||||
id CHAR(64) PRIMARY KEY,
|
||||
pubkey CHAR(64) NOT NULL,
|
||||
created_at BIGINT NOT NULL,
|
||||
kind INTEGER NOT NULL,
|
||||
tags JSONB NOT NULL DEFAULT '[]',
|
||||
generic_tags TEXT[] NOT NULL DEFAULT '{}',
|
||||
content TEXT NOT NULL DEFAULT '',
|
||||
sig CHAR(128) NOT NULL,
|
||||
expired_at BIGINT,
|
||||
d_tag_value TEXT,
|
||||
delegator CHAR(64),
|
||||
create_date TIMESTAMP NOT NULL DEFAULT now(),
|
||||
update_date TIMESTAMP NOT NULL DEFAULT now(),
|
||||
delete_date TIMESTAMP
|
||||
);
|
||||
|
||||
-- Indices
|
||||
CREATE EXTENSION IF NOT EXISTS btree_gin;
|
||||
CREATE INDEX generic_tags_kind_idx ON events USING gin (generic_tags, kind);
|
||||
CREATE INDEX pubkey_kind_created_at_idx ON events (pubkey, kind, created_at);
|
||||
CREATE INDEX delegator_kind_created_at_idx ON events (delegator, kind, created_at);
|
||||
CREATE INDEX created_at_kind_idx ON events (created_at, kind);
|
||||
|
||||
-- Generic tags table (for optimized tag queries)
|
||||
CREATE TABLE generic_tags (
|
||||
id SERIAL PRIMARY KEY,
|
||||
tag TEXT NOT NULL,
|
||||
author CHAR(64) NOT NULL,
|
||||
kind INTEGER NOT NULL,
|
||||
event_id CHAR(64) NOT NULL REFERENCES events(id),
|
||||
created_at BIGINT NOT NULL
|
||||
);
|
||||
|
||||
-- Indices for tag filtering
|
||||
CREATE UNIQUE INDEX g_tag_event_id_idx ON generic_tags (tag, event_id);
|
||||
CREATE INDEX g_tag_kind_created_at_idx ON generic_tags (tag, kind, created_at);
|
||||
CREATE INDEX g_tag_created_at_idx ON generic_tags (tag, created_at);
|
||||
```
|
||||
Nostr-Relay-NestJS features a hybrid storage model where tags are stored both in a JSONB field (for general lookup) and a normalized `generic_tags` table (for optimized tag query performance). The schema uses Kysely for type-safe interaction.
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
https://github.com/rushmi0/Fenrir-s
|
||||
https://github.com/barkyq/gnost-relay
|
||||
https://github.com/lpicanco/knostr
|
||||
https://github.com/bezysoftware/netstr
|
||||
https://github.com/lebrunel/nex
|
||||
https://github.com/CodyTseng/nostr-relay-nestjs
|
||||
https://github.com/mattn/nostr-relay
|
||||
https://github.com/Cameri/nostream
|
||||
https://github.com/Giszmo/NostrPostr/tree/master/NostrRelay
|
||||
https://github.com/fiatjaf/relayer/tree/master/examples/basic
|
||||
@@ -14,7 +14,6 @@ extern void log_query_execution(const char* query_type, const char* sub_id,
|
||||
#include <libwebsockets.h>
|
||||
#include <cjson/cJSON.h>
|
||||
int get_active_connection_count(void);
|
||||
#include <sqlite3.h>
|
||||
#include <time.h>
|
||||
#include <sys/stat.h>
|
||||
#include <unistd.h>
|
||||
@@ -688,7 +687,6 @@ static const config_definition_t known_configs[] = {
|
||||
{NULL, NULL, 0, 0}
|
||||
};
|
||||
// External database connection (from main.c)
|
||||
extern sqlite3* g_db;
|
||||
extern char g_database_path[512];
|
||||
|
||||
// Forward declarations for database functions
|
||||
@@ -993,138 +991,12 @@ char* execute_sql_query(const char* query, const char* request_id, char* error_m
|
||||
return NULL;
|
||||
}
|
||||
|
||||
if (!g_db) {
|
||||
if (!db_is_available()) {
|
||||
snprintf(error_message, error_size, "Database not available");
|
||||
return NULL;
|
||||
}
|
||||
|
||||
// Set busy timeout to prevent long-running queries (5 seconds)
|
||||
sqlite3_busy_timeout(g_db, 5000);
|
||||
|
||||
// Prepare statement
|
||||
sqlite3_stmt* stmt;
|
||||
int rc = sqlite3_prepare_v2(g_db, query, -1, &stmt, NULL);
|
||||
if (rc != SQLITE_OK) {
|
||||
const char* err_msg = sqlite3_errmsg(g_db);
|
||||
snprintf(error_message, error_size, "SQL prepare failed: %s", err_msg);
|
||||
return NULL;
|
||||
}
|
||||
|
||||
// Execute query and collect results
|
||||
cJSON* response = cJSON_CreateObject();
|
||||
cJSON_AddStringToObject(response, "query_type", "sql_query");
|
||||
cJSON_AddStringToObject(response, "request_id", request_id);
|
||||
cJSON_AddNumberToObject(response, "timestamp", (double)time(NULL));
|
||||
cJSON_AddStringToObject(response, "query", query);
|
||||
|
||||
// Get column information
|
||||
int col_count = sqlite3_column_count(stmt);
|
||||
cJSON* columns = cJSON_CreateArray();
|
||||
for (int i = 0; i < col_count; i++) {
|
||||
const char* col_name = sqlite3_column_name(stmt, i);
|
||||
cJSON_AddItemToArray(columns, cJSON_CreateString(col_name ? col_name : ""));
|
||||
}
|
||||
cJSON_AddItemToObject(response, "columns", columns);
|
||||
|
||||
// Execute and collect rows (with limit)
|
||||
cJSON* rows = cJSON_CreateArray();
|
||||
int row_count = 0;
|
||||
const int MAX_ROWS = 1000; // Configurable limit
|
||||
|
||||
struct timespec start_time;
|
||||
clock_gettime(CLOCK_MONOTONIC, &start_time);
|
||||
|
||||
while ((rc = sqlite3_step(stmt)) == SQLITE_ROW && row_count < MAX_ROWS) {
|
||||
cJSON* row = cJSON_CreateArray();
|
||||
|
||||
for (int i = 0; i < col_count; i++) {
|
||||
int col_type = sqlite3_column_type(stmt, i);
|
||||
|
||||
switch (col_type) {
|
||||
case SQLITE_INTEGER:
|
||||
cJSON_AddItemToArray(row, cJSON_CreateNumber((double)sqlite3_column_int64(stmt, i)));
|
||||
break;
|
||||
case SQLITE_FLOAT:
|
||||
cJSON_AddItemToArray(row, cJSON_CreateNumber(sqlite3_column_double(stmt, i)));
|
||||
break;
|
||||
case SQLITE_TEXT: {
|
||||
const char* text = (const char*)sqlite3_column_text(stmt, i);
|
||||
cJSON_AddItemToArray(row, cJSON_CreateString(text ? text : ""));
|
||||
break;
|
||||
}
|
||||
case SQLITE_BLOB: {
|
||||
// Convert blob to hex string for JSON compatibility
|
||||
const void* blob = sqlite3_column_blob(stmt, i);
|
||||
int blob_size = sqlite3_column_bytes(stmt, i);
|
||||
if (blob && blob_size > 0) {
|
||||
char* hex_str = malloc(blob_size * 2 + 1);
|
||||
if (hex_str) {
|
||||
for (int j = 0; j < blob_size; j++) {
|
||||
sprintf(hex_str + j * 2, "%02x", ((unsigned char*)blob)[j]);
|
||||
}
|
||||
hex_str[blob_size * 2] = '\0';
|
||||
cJSON_AddItemToArray(row, cJSON_CreateString(hex_str));
|
||||
free(hex_str);
|
||||
} else {
|
||||
cJSON_AddItemToArray(row, cJSON_CreateString("[BLOB]"));
|
||||
}
|
||||
} else {
|
||||
cJSON_AddItemToArray(row, cJSON_CreateString(""));
|
||||
}
|
||||
break;
|
||||
}
|
||||
case SQLITE_NULL:
|
||||
cJSON_AddItemToArray(row, cJSON_CreateNull());
|
||||
break;
|
||||
default:
|
||||
cJSON_AddItemToArray(row, cJSON_CreateString("[UNKNOWN]"));
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
cJSON_AddItemToArray(rows, row);
|
||||
row_count++;
|
||||
|
||||
// Check timeout (additional safety check)
|
||||
struct timespec current_time;
|
||||
clock_gettime(CLOCK_MONOTONIC, ¤t_time);
|
||||
double elapsed = (current_time.tv_sec - start_time.tv_sec) +
|
||||
(current_time.tv_nsec - start_time.tv_nsec) / 1e9;
|
||||
if (elapsed > 4.5) { // 4.5 seconds to allow for cleanup
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
|
||||
// Check for execution errors
|
||||
if (rc != SQLITE_DONE && rc != SQLITE_ROW) {
|
||||
const char* err_msg = sqlite3_errmsg(g_db);
|
||||
snprintf(error_message, error_size, "SQL execution failed: %s", err_msg);
|
||||
cJSON_Delete(response);
|
||||
return NULL;
|
||||
}
|
||||
|
||||
// Check row limit
|
||||
if (row_count >= MAX_ROWS) {
|
||||
cJSON_AddStringToObject(response, "warning", "Result truncated to maximum row limit");
|
||||
}
|
||||
|
||||
// Add metadata
|
||||
cJSON_AddNumberToObject(response, "row_count", row_count);
|
||||
cJSON_AddNumberToObject(response, "execution_time_ms", 0); // Will be set by caller
|
||||
cJSON_AddItemToObject(response, "rows", rows);
|
||||
|
||||
// Convert to JSON string
|
||||
char* json_result = cJSON_Print(response);
|
||||
cJSON_Delete(response);
|
||||
|
||||
if (!json_result) {
|
||||
snprintf(error_message, error_size, "Failed to generate JSON response");
|
||||
return NULL;
|
||||
}
|
||||
|
||||
return json_result;
|
||||
return db_execute_readonly_query_json(query, request_id, error_message, error_size, 1000, 5000);
|
||||
}
|
||||
|
||||
// Unified handler for SQL query commands
|
||||
@@ -1304,10 +1176,10 @@ char* generate_stats_json(void) {
|
||||
cJSON_AddNumberToObject(response, "timestamp", (double)time(NULL));
|
||||
|
||||
// Get database file size
|
||||
extern char g_database_path[512];
|
||||
const char* db_path = db_get_database_path();
|
||||
struct stat db_stat;
|
||||
long long db_size = 0;
|
||||
if (stat(g_database_path, &db_stat) == 0) {
|
||||
if (db_path && db_path[0] != '\0' && stat(db_path, &db_stat) == 0) {
|
||||
db_size = db_stat.st_size;
|
||||
}
|
||||
cJSON_AddNumberToObject(response, "database_size_bytes", db_size);
|
||||
|
||||
+306
-446
File diff suppressed because it is too large
Load Diff
@@ -1,7 +1,6 @@
|
||||
#ifndef CONFIG_H
|
||||
#define CONFIG_H
|
||||
|
||||
#include <sqlite3.h>
|
||||
#include <cjson/cJSON.h>
|
||||
#include <time.h>
|
||||
#include <pthread.h>
|
||||
|
||||
+480
-2
@@ -4,6 +4,7 @@
|
||||
#include "debug.h"
|
||||
#include "config.h"
|
||||
|
||||
#include <sqlite3.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
@@ -12,12 +13,90 @@
|
||||
extern sqlite3* g_db;
|
||||
extern char g_database_path[512];
|
||||
|
||||
typedef struct db_stmt {
|
||||
sqlite3_stmt* stmt;
|
||||
} db_stmt_t;
|
||||
|
||||
int db_is_available(void) {
|
||||
return g_db != NULL;
|
||||
}
|
||||
|
||||
sqlite3* db_get_handle(void) {
|
||||
return g_db;
|
||||
const char* db_last_error(void) {
|
||||
if (!g_db) return "database not available";
|
||||
return sqlite3_errmsg(g_db);
|
||||
}
|
||||
|
||||
const char* db_get_database_path(void) {
|
||||
return g_database_path;
|
||||
}
|
||||
|
||||
int db_prepare(const char* sql, db_stmt_t** out_stmt) {
|
||||
if (!g_db || !sql || !out_stmt) return DB_MISUSE;
|
||||
|
||||
sqlite3_stmt* raw_stmt = NULL;
|
||||
int rc = sqlite3_prepare_v2(g_db, sql, -1, &raw_stmt, NULL);
|
||||
if (rc != SQLITE_OK) return rc;
|
||||
|
||||
db_stmt_t* wrapper = (db_stmt_t*)malloc(sizeof(db_stmt_t));
|
||||
if (!wrapper) {
|
||||
sqlite3_finalize(raw_stmt);
|
||||
return DB_ERROR;
|
||||
}
|
||||
|
||||
wrapper->stmt = raw_stmt;
|
||||
*out_stmt = wrapper;
|
||||
return DB_OK;
|
||||
}
|
||||
|
||||
int db_bind_text_param(db_stmt_t* stmt, int index, const char* value) {
|
||||
if (!stmt || !stmt->stmt) return DB_MISUSE;
|
||||
return sqlite3_bind_text(stmt->stmt, index, value ? value : "", -1, SQLITE_TRANSIENT);
|
||||
}
|
||||
|
||||
int db_bind_int_param(db_stmt_t* stmt, int index, int value) {
|
||||
if (!stmt || !stmt->stmt) return DB_MISUSE;
|
||||
return sqlite3_bind_int(stmt->stmt, index, value);
|
||||
}
|
||||
|
||||
int db_bind_int64_param(db_stmt_t* stmt, int index, long long value) {
|
||||
if (!stmt || !stmt->stmt) return DB_MISUSE;
|
||||
return sqlite3_bind_int64(stmt->stmt, index, (sqlite3_int64)value);
|
||||
}
|
||||
|
||||
int db_step_stmt(db_stmt_t* stmt) {
|
||||
if (!stmt || !stmt->stmt) return DB_MISUSE;
|
||||
return sqlite3_step(stmt->stmt);
|
||||
}
|
||||
|
||||
int db_reset_stmt(db_stmt_t* stmt) {
|
||||
if (!stmt || !stmt->stmt) return DB_MISUSE;
|
||||
return sqlite3_reset(stmt->stmt);
|
||||
}
|
||||
|
||||
const char* db_column_text_value(db_stmt_t* stmt, int col) {
|
||||
if (!stmt || !stmt->stmt) return NULL;
|
||||
return (const char*)sqlite3_column_text(stmt->stmt, col);
|
||||
}
|
||||
|
||||
int db_column_int_value(db_stmt_t* stmt, int col) {
|
||||
if (!stmt || !stmt->stmt) return 0;
|
||||
return sqlite3_column_int(stmt->stmt, col);
|
||||
}
|
||||
|
||||
long long db_column_int64_value(db_stmt_t* stmt, int col) {
|
||||
if (!stmt || !stmt->stmt) return 0;
|
||||
return (long long)sqlite3_column_int64(stmt->stmt, col);
|
||||
}
|
||||
|
||||
double db_column_double_value(db_stmt_t* stmt, int col) {
|
||||
if (!stmt || !stmt->stmt) return 0.0;
|
||||
return sqlite3_column_double(stmt->stmt, col);
|
||||
}
|
||||
|
||||
void db_finalize_stmt(db_stmt_t* stmt) {
|
||||
if (!stmt) return;
|
||||
if (stmt->stmt) sqlite3_finalize(stmt->stmt);
|
||||
free(stmt);
|
||||
}
|
||||
|
||||
int db_log_subscription_created(const char* sub_id, const char* wsi_ptr,
|
||||
@@ -315,6 +394,155 @@ int db_count_with_sql(const char* sql, const char** bind_params, int bind_param_
|
||||
return 0;
|
||||
}
|
||||
|
||||
char* db_execute_readonly_query_json(const char* query, const char* request_id,
|
||||
char* error_message, size_t error_size,
|
||||
int max_rows, int timeout_ms) {
|
||||
if (!g_db || !query || !request_id || !error_message) return NULL;
|
||||
|
||||
sqlite3_busy_timeout(g_db, timeout_ms > 0 ? timeout_ms : 5000);
|
||||
|
||||
sqlite3_stmt* stmt = NULL;
|
||||
int rc = sqlite3_prepare_v2(g_db, query, -1, &stmt, NULL);
|
||||
if (rc != SQLITE_OK) {
|
||||
const char* err_msg = sqlite3_errmsg(g_db);
|
||||
snprintf(error_message, error_size, "SQL prepare failed: %s", err_msg ? err_msg : "unknown");
|
||||
return NULL;
|
||||
}
|
||||
|
||||
cJSON* response = cJSON_CreateObject();
|
||||
if (!response) {
|
||||
sqlite3_finalize(stmt);
|
||||
snprintf(error_message, error_size, "Failed to allocate query response object");
|
||||
return NULL;
|
||||
}
|
||||
|
||||
cJSON_AddStringToObject(response, "query_type", "sql_query");
|
||||
cJSON_AddStringToObject(response, "request_id", request_id);
|
||||
cJSON_AddNumberToObject(response, "timestamp", (double)time(NULL));
|
||||
cJSON_AddStringToObject(response, "query", query);
|
||||
|
||||
int col_count = sqlite3_column_count(stmt);
|
||||
cJSON* columns = cJSON_CreateArray();
|
||||
if (!columns) {
|
||||
sqlite3_finalize(stmt);
|
||||
cJSON_Delete(response);
|
||||
snprintf(error_message, error_size, "Failed to allocate columns array");
|
||||
return NULL;
|
||||
}
|
||||
for (int i = 0; i < col_count; i++) {
|
||||
const char* col_name = sqlite3_column_name(stmt, i);
|
||||
cJSON_AddItemToArray(columns, cJSON_CreateString(col_name ? col_name : ""));
|
||||
}
|
||||
cJSON_AddItemToObject(response, "columns", columns);
|
||||
|
||||
cJSON* rows = cJSON_CreateArray();
|
||||
if (!rows) {
|
||||
sqlite3_finalize(stmt);
|
||||
cJSON_Delete(response);
|
||||
snprintf(error_message, error_size, "Failed to allocate rows array");
|
||||
return NULL;
|
||||
}
|
||||
|
||||
const int row_limit = (max_rows > 0) ? max_rows : 1000;
|
||||
int row_count = 0;
|
||||
|
||||
struct timespec start_time;
|
||||
clock_gettime(CLOCK_MONOTONIC, &start_time);
|
||||
|
||||
while ((rc = sqlite3_step(stmt)) == SQLITE_ROW && row_count < row_limit) {
|
||||
cJSON* row = cJSON_CreateArray();
|
||||
if (!row) {
|
||||
sqlite3_finalize(stmt);
|
||||
cJSON_Delete(rows);
|
||||
cJSON_Delete(response);
|
||||
snprintf(error_message, error_size, "Failed to allocate row array");
|
||||
return NULL;
|
||||
}
|
||||
|
||||
for (int i = 0; i < col_count; i++) {
|
||||
int col_type = sqlite3_column_type(stmt, i);
|
||||
switch (col_type) {
|
||||
case SQLITE_INTEGER:
|
||||
cJSON_AddItemToArray(row, cJSON_CreateNumber((double)sqlite3_column_int64(stmt, i)));
|
||||
break;
|
||||
case SQLITE_FLOAT:
|
||||
cJSON_AddItemToArray(row, cJSON_CreateNumber(sqlite3_column_double(stmt, i)));
|
||||
break;
|
||||
case SQLITE_TEXT: {
|
||||
const char* text = (const char*)sqlite3_column_text(stmt, i);
|
||||
cJSON_AddItemToArray(row, cJSON_CreateString(text ? text : ""));
|
||||
break;
|
||||
}
|
||||
case SQLITE_BLOB: {
|
||||
const void* blob = sqlite3_column_blob(stmt, i);
|
||||
int blob_size = sqlite3_column_bytes(stmt, i);
|
||||
if (blob && blob_size > 0) {
|
||||
char* hex_str = malloc((size_t)blob_size * 2 + 1);
|
||||
if (hex_str) {
|
||||
for (int j = 0; j < blob_size; j++) {
|
||||
sprintf(hex_str + j * 2, "%02x", ((const unsigned char*)blob)[j]);
|
||||
}
|
||||
hex_str[(size_t)blob_size * 2] = '\0';
|
||||
cJSON_AddItemToArray(row, cJSON_CreateString(hex_str));
|
||||
free(hex_str);
|
||||
} else {
|
||||
cJSON_AddItemToArray(row, cJSON_CreateString("[BLOB]"));
|
||||
}
|
||||
} else {
|
||||
cJSON_AddItemToArray(row, cJSON_CreateString(""));
|
||||
}
|
||||
break;
|
||||
}
|
||||
case SQLITE_NULL:
|
||||
cJSON_AddItemToArray(row, cJSON_CreateNull());
|
||||
break;
|
||||
default:
|
||||
cJSON_AddItemToArray(row, cJSON_CreateString("[UNKNOWN]"));
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
cJSON_AddItemToArray(rows, row);
|
||||
row_count++;
|
||||
|
||||
struct timespec current_time;
|
||||
clock_gettime(CLOCK_MONOTONIC, ¤t_time);
|
||||
double elapsed = (current_time.tv_sec - start_time.tv_sec) +
|
||||
(current_time.tv_nsec - start_time.tv_nsec) / 1e9;
|
||||
if (elapsed > 4.5) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
|
||||
if (rc != SQLITE_DONE && rc != SQLITE_ROW) {
|
||||
const char* err_msg = sqlite3_errmsg(g_db);
|
||||
snprintf(error_message, error_size, "SQL execution failed: %s", err_msg ? err_msg : "unknown");
|
||||
cJSON_Delete(rows);
|
||||
cJSON_Delete(response);
|
||||
return NULL;
|
||||
}
|
||||
|
||||
if (row_count >= row_limit) {
|
||||
cJSON_AddStringToObject(response, "warning", "Result truncated to maximum row limit");
|
||||
}
|
||||
|
||||
cJSON_AddNumberToObject(response, "row_count", row_count);
|
||||
cJSON_AddNumberToObject(response, "execution_time_ms", 0);
|
||||
cJSON_AddItemToObject(response, "rows", rows);
|
||||
|
||||
char* json_result = cJSON_Print(response);
|
||||
cJSON_Delete(response);
|
||||
|
||||
if (!json_result) {
|
||||
snprintf(error_message, error_size, "Failed to generate JSON response");
|
||||
return NULL;
|
||||
}
|
||||
|
||||
return json_result;
|
||||
}
|
||||
|
||||
int db_get_total_event_count_ll(long long* out_count) {
|
||||
if (!g_db || !out_count) return -1;
|
||||
|
||||
@@ -654,6 +882,47 @@ int db_store_config_event(const cJSON* event) {
|
||||
return (rc == SQLITE_DONE) ? 0 : -1;
|
||||
}
|
||||
|
||||
int db_insert_event_with_json(const char* id, const char* pubkey, long long created_at,
|
||||
int kind, const char* event_type, const char* content,
|
||||
const char* sig, const char* tags_json, const char* event_json,
|
||||
int* out_step_rc, int* out_extended_errcode) {
|
||||
if (!g_db || !id || !pubkey || !event_type || !content || !sig || !tags_json || !event_json) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
const char* sql =
|
||||
"INSERT INTO events (id, pubkey, created_at, kind, event_type, content, sig, tags, event_json) "
|
||||
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)";
|
||||
|
||||
sqlite3_stmt* stmt = NULL;
|
||||
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
sqlite3_bind_text(stmt, 1, id, -1, SQLITE_TRANSIENT);
|
||||
sqlite3_bind_text(stmt, 2, pubkey, -1, SQLITE_TRANSIENT);
|
||||
sqlite3_bind_int64(stmt, 3, (sqlite3_int64)created_at);
|
||||
sqlite3_bind_int(stmt, 4, kind);
|
||||
sqlite3_bind_text(stmt, 5, event_type, -1, SQLITE_TRANSIENT);
|
||||
sqlite3_bind_text(stmt, 6, content, -1, SQLITE_TRANSIENT);
|
||||
sqlite3_bind_text(stmt, 7, sig, -1, SQLITE_TRANSIENT);
|
||||
sqlite3_bind_text(stmt, 8, tags_json, -1, SQLITE_TRANSIENT);
|
||||
sqlite3_bind_text(stmt, 9, event_json, -1, SQLITE_TRANSIENT);
|
||||
|
||||
int step_rc = sqlite3_step(stmt);
|
||||
int extended_errcode = sqlite3_extended_errcode(g_db);
|
||||
sqlite3_finalize(stmt);
|
||||
|
||||
if (out_step_rc) {
|
||||
*out_step_rc = step_rc;
|
||||
}
|
||||
if (out_extended_errcode) {
|
||||
*out_extended_errcode = extended_errcode;
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
int db_get_event_time_bounds(long long* out_min_created_at, long long* out_max_created_at) {
|
||||
if (!g_db || !out_min_created_at || !out_max_created_at) return -1;
|
||||
|
||||
@@ -676,6 +945,86 @@ int db_get_event_time_bounds(long long* out_min_created_at, long long* out_max_c
|
||||
return 0;
|
||||
}
|
||||
|
||||
int db_event_id_exists(const char* event_id, int* out_exists) {
|
||||
if (!g_db || !event_id || !out_exists) return -1;
|
||||
|
||||
sqlite3_stmt* stmt = NULL;
|
||||
const char* sql = "SELECT 1 FROM events WHERE id=? LIMIT 1";
|
||||
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) return -1;
|
||||
|
||||
sqlite3_bind_text(stmt, 1, event_id, 64, SQLITE_STATIC);
|
||||
int exists = (sqlite3_step(stmt) == SQLITE_ROW) ? 1 : 0;
|
||||
sqlite3_finalize(stmt);
|
||||
|
||||
*out_exists = exists;
|
||||
return 0;
|
||||
}
|
||||
|
||||
cJSON* db_retrieve_event_by_id(const char* event_id) {
|
||||
if (!g_db || !event_id) return NULL;
|
||||
|
||||
const char* sql =
|
||||
"SELECT id, pubkey, created_at, kind, content, sig, tags FROM events WHERE id = ?";
|
||||
|
||||
sqlite3_stmt* stmt = NULL;
|
||||
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) {
|
||||
return NULL;
|
||||
}
|
||||
|
||||
sqlite3_bind_text(stmt, 1, event_id, -1, SQLITE_STATIC);
|
||||
|
||||
cJSON* event = NULL;
|
||||
if (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
event = cJSON_CreateObject();
|
||||
if (event) {
|
||||
cJSON_AddStringToObject(event, "id", (const char*)sqlite3_column_text(stmt, 0));
|
||||
cJSON_AddStringToObject(event, "pubkey", (const char*)sqlite3_column_text(stmt, 1));
|
||||
cJSON_AddNumberToObject(event, "created_at", sqlite3_column_int64(stmt, 2));
|
||||
cJSON_AddNumberToObject(event, "kind", sqlite3_column_int(stmt, 3));
|
||||
cJSON_AddStringToObject(event, "content", (const char*)sqlite3_column_text(stmt, 4));
|
||||
cJSON_AddStringToObject(event, "sig", (const char*)sqlite3_column_text(stmt, 5));
|
||||
|
||||
const char* tags_json = (const char*)sqlite3_column_text(stmt, 6);
|
||||
if (tags_json) {
|
||||
cJSON* tags = cJSON_Parse(tags_json);
|
||||
if (tags) {
|
||||
cJSON_AddItemToObject(event, "tags", tags);
|
||||
} else {
|
||||
cJSON_AddItemToObject(event, "tags", cJSON_CreateArray());
|
||||
}
|
||||
} else {
|
||||
cJSON_AddItemToObject(event, "tags", cJSON_CreateArray());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
return event;
|
||||
}
|
||||
|
||||
char* db_get_latest_event_pubkey_for_kind_dup(int kind) {
|
||||
if (!g_db) return NULL;
|
||||
|
||||
const char* sql = "SELECT pubkey FROM events WHERE kind = ? ORDER BY created_at DESC LIMIT 1";
|
||||
sqlite3_stmt* stmt = NULL;
|
||||
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) {
|
||||
return NULL;
|
||||
}
|
||||
|
||||
sqlite3_bind_int(stmt, 1, kind);
|
||||
|
||||
char* pubkey_dup = NULL;
|
||||
if (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
const char* pubkey = (const char*)sqlite3_column_text(stmt, 0);
|
||||
if (pubkey) {
|
||||
pubkey_dup = strdup(pubkey);
|
||||
}
|
||||
}
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
return pubkey_dup;
|
||||
}
|
||||
|
||||
int db_get_config_row_count(int* out_count) {
|
||||
if (!g_db || !out_count) return -1;
|
||||
|
||||
@@ -695,6 +1044,92 @@ int db_get_config_row_count(int* out_count) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
int db_store_event_tags_cjson(const char* event_id, const cJSON* tags) {
|
||||
if (!g_db || !event_id || !tags || !cJSON_IsArray(tags)) {
|
||||
return 0; // Not an error if no tags
|
||||
}
|
||||
|
||||
const char* sql = "INSERT INTO event_tags (event_id, tag_name, tag_value, tag_index) VALUES (?, ?, ?, ?)";
|
||||
sqlite3_stmt* stmt = NULL;
|
||||
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
|
||||
if (rc != SQLITE_OK) {
|
||||
DEBUG_ERROR("Failed to prepare event_tags insert: %s", sqlite3_errmsg(g_db));
|
||||
return -1;
|
||||
}
|
||||
|
||||
int tag_index = 0;
|
||||
cJSON* tag = NULL;
|
||||
cJSON_ArrayForEach(tag, tags) {
|
||||
if (cJSON_IsArray(tag) && cJSON_GetArraySize(tag) >= 2) {
|
||||
cJSON* name = cJSON_GetArrayItem(tag, 0);
|
||||
cJSON* value = cJSON_GetArrayItem(tag, 1);
|
||||
|
||||
if (cJSON_IsString(name) && cJSON_IsString(value)) {
|
||||
sqlite3_reset(stmt);
|
||||
sqlite3_bind_text(stmt, 1, event_id, -1, SQLITE_STATIC);
|
||||
sqlite3_bind_text(stmt, 2, cJSON_GetStringValue(name), -1, SQLITE_STATIC);
|
||||
sqlite3_bind_text(stmt, 3, cJSON_GetStringValue(value), -1, SQLITE_STATIC);
|
||||
sqlite3_bind_int(stmt, 4, tag_index);
|
||||
|
||||
rc = sqlite3_step(stmt);
|
||||
if (rc != SQLITE_DONE) {
|
||||
DEBUG_ERROR("Failed to insert event tag: %s", sqlite3_errmsg(g_db));
|
||||
}
|
||||
}
|
||||
}
|
||||
tag_index++;
|
||||
}
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int db_populate_event_tags_from_existing(void) {
|
||||
if (!g_db) return -1;
|
||||
|
||||
sqlite3_stmt* check_stmt = NULL;
|
||||
if (sqlite3_prepare_v2(g_db, "SELECT COUNT(*) FROM event_tags", -1, &check_stmt, NULL) != SQLITE_OK) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
if (sqlite3_step(check_stmt) == SQLITE_ROW && sqlite3_column_int(check_stmt, 0) > 0) {
|
||||
sqlite3_finalize(check_stmt);
|
||||
DEBUG_INFO("event_tags already populated, skipping");
|
||||
return 0;
|
||||
}
|
||||
sqlite3_finalize(check_stmt);
|
||||
|
||||
DEBUG_INFO("Populating event_tags from existing events...");
|
||||
|
||||
const char* sql = "SELECT id, tags FROM events WHERE tags != '[]'";
|
||||
sqlite3_stmt* stmt = NULL;
|
||||
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
|
||||
if (rc != SQLITE_OK) return -1;
|
||||
|
||||
sqlite3_exec(g_db, "BEGIN TRANSACTION", NULL, NULL, NULL);
|
||||
|
||||
int event_count = 0;
|
||||
while (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
const char* event_id = (const char*)sqlite3_column_text(stmt, 0);
|
||||
const char* tags_json = (const char*)sqlite3_column_text(stmt, 1);
|
||||
|
||||
if (event_id && tags_json) {
|
||||
cJSON* tags = cJSON_Parse(tags_json);
|
||||
if (tags) {
|
||||
db_store_event_tags_cjson(event_id, tags);
|
||||
cJSON_Delete(tags);
|
||||
event_count++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
sqlite3_exec(g_db, "COMMIT", NULL, NULL, NULL);
|
||||
|
||||
DEBUG_INFO("Populated event_tags for %d events", event_count);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int db_add_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value) {
|
||||
if (!g_db || !rule_type || !pattern_type || !pattern_value) return -1;
|
||||
|
||||
@@ -745,3 +1180,46 @@ int db_count_wot_whitelist_rules(void) {
|
||||
}
|
||||
return count;
|
||||
}
|
||||
|
||||
int db_table_exists(const char* table_name, int* out_exists) {
|
||||
if (!g_db || !table_name || !out_exists) return -1;
|
||||
|
||||
const char* sql = "SELECT 1 FROM sqlite_master WHERE type='table' AND name=? LIMIT 1";
|
||||
sqlite3_stmt* stmt = NULL;
|
||||
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
sqlite3_bind_text(stmt, 1, table_name, -1, SQLITE_TRANSIENT);
|
||||
*out_exists = (sqlite3_step(stmt) == SQLITE_ROW) ? 1 : 0;
|
||||
sqlite3_finalize(stmt);
|
||||
return 0;
|
||||
}
|
||||
|
||||
char* db_get_schema_version_dup(void) {
|
||||
if (!g_db) return NULL;
|
||||
|
||||
const char* sql = "SELECT value FROM schema_info WHERE key = 'version'";
|
||||
sqlite3_stmt* stmt = NULL;
|
||||
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) {
|
||||
return NULL;
|
||||
}
|
||||
|
||||
char* result = NULL;
|
||||
if (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
const char* version = (const char*)sqlite3_column_text(stmt, 0);
|
||||
if (version) {
|
||||
result = strdup(version);
|
||||
}
|
||||
}
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
return result;
|
||||
}
|
||||
|
||||
int db_exec_sql(const char* sql) {
|
||||
if (!g_db || !sql) return -1;
|
||||
|
||||
int rc = sqlite3_exec(g_db, sql, NULL, NULL, NULL);
|
||||
return (rc == SQLITE_OK) ? 0 : -1;
|
||||
}
|
||||
|
||||
+45
-3
@@ -3,12 +3,35 @@
|
||||
|
||||
#include <stddef.h>
|
||||
#include <time.h>
|
||||
#include <sqlite3.h>
|
||||
#include <cjson/cJSON.h>
|
||||
|
||||
// Generic helpers
|
||||
int db_is_available(void);
|
||||
sqlite3* db_get_handle(void);
|
||||
const char* db_last_error(void);
|
||||
const char* db_get_database_path(void);
|
||||
|
||||
// DB result codes (backend-agnostic)
|
||||
#define DB_OK 0
|
||||
#define DB_ERROR 1
|
||||
#define DB_CONSTRAINT 19
|
||||
#define DB_MISUSE 21
|
||||
#define DB_ROW 100
|
||||
#define DB_DONE 101
|
||||
|
||||
typedef struct db_stmt db_stmt_t;
|
||||
|
||||
// Generic statement helpers (Phase 1 migration)
|
||||
int db_prepare(const char* sql, db_stmt_t** out_stmt);
|
||||
int db_bind_text_param(db_stmt_t* stmt, int index, const char* value);
|
||||
int db_bind_int_param(db_stmt_t* stmt, int index, int value);
|
||||
int db_bind_int64_param(db_stmt_t* stmt, int index, long long value);
|
||||
int db_step_stmt(db_stmt_t* stmt);
|
||||
int db_reset_stmt(db_stmt_t* stmt);
|
||||
const char* db_column_text_value(db_stmt_t* stmt, int col);
|
||||
int db_column_int_value(db_stmt_t* stmt, int col);
|
||||
long long db_column_int64_value(db_stmt_t* stmt, int col);
|
||||
double db_column_double_value(db_stmt_t* stmt, int col);
|
||||
void db_finalize_stmt(db_stmt_t* stmt);
|
||||
|
||||
// Subscription logging
|
||||
int db_log_subscription_created(const char* sub_id, const char* wsi_ptr,
|
||||
@@ -32,6 +55,9 @@ int db_count_active_whitelist_rules(void);
|
||||
|
||||
// Generic prepared COUNT helper
|
||||
int db_count_with_sql(const char* sql, const char** bind_params, int bind_param_count, int* out_count);
|
||||
char* db_execute_readonly_query_json(const char* query, const char* request_id,
|
||||
char* error_message, size_t error_size,
|
||||
int max_rows, int timeout_ms);
|
||||
|
||||
// Monitoring/stat helpers (Phase 1 api.c migration)
|
||||
int db_get_total_event_count_ll(long long* out_count);
|
||||
@@ -51,16 +77,32 @@ int db_store_relay_private_key_hex(const char* relay_privkey_hex);
|
||||
char* db_get_relay_private_key_hex_dup(void);
|
||||
int db_store_config_event(const cJSON* event);
|
||||
|
||||
// Event timestamp helpers
|
||||
// Event storage/timestamp/retrieval helpers
|
||||
int db_insert_event_with_json(const char* id, const char* pubkey, long long created_at,
|
||||
int kind, const char* event_type, const char* content,
|
||||
const char* sig, const char* tags_json, const char* event_json,
|
||||
int* out_step_rc, int* out_extended_errcode);
|
||||
int db_get_event_time_bounds(long long* out_min_created_at, long long* out_max_created_at);
|
||||
int db_event_id_exists(const char* event_id, int* out_exists);
|
||||
cJSON* db_retrieve_event_by_id(const char* event_id);
|
||||
char* db_get_latest_event_pubkey_for_kind_dup(int kind);
|
||||
|
||||
// Config table helpers
|
||||
int db_get_config_row_count(int* out_count);
|
||||
|
||||
// Event tag helpers
|
||||
int db_store_event_tags_cjson(const char* event_id, const cJSON* tags);
|
||||
int db_populate_event_tags_from_existing(void);
|
||||
|
||||
// Auth/WoT rule helpers
|
||||
int db_add_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
|
||||
int db_remove_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
|
||||
int db_delete_wot_whitelist_rules(void);
|
||||
int db_count_wot_whitelist_rules(void);
|
||||
|
||||
// Schema/DDL helpers
|
||||
int db_table_exists(const char* table_name, int* out_exists);
|
||||
char* db_get_schema_version_dup(void);
|
||||
int db_exec_sql(const char* sql);
|
||||
|
||||
#endif // DB_OPS_H
|
||||
|
||||
+49
-53
@@ -102,8 +102,7 @@ void ip_ban_init(void) {
|
||||
}
|
||||
|
||||
void ip_ban_load_from_db(void) {
|
||||
sqlite3* db = db_get_handle();
|
||||
if (!db || !g_initialized) return;
|
||||
if (!db_is_available() || !g_initialized) return;
|
||||
|
||||
// Create table if it doesn't exist (handles existing databases)
|
||||
const char* create_sql =
|
||||
@@ -121,18 +120,16 @@ void ip_ban_load_from_db(void) {
|
||||
" first_seen INTEGER NOT NULL DEFAULT 0,"
|
||||
" updated_at INTEGER NOT NULL DEFAULT (strftime('%s', 'now'))"
|
||||
");";
|
||||
char* err = NULL;
|
||||
if (sqlite3_exec(db, create_sql, NULL, NULL, &err) != SQLITE_OK) {
|
||||
DEBUG_ERROR("Failed to create ip_bans table: %s", err ? err : "unknown");
|
||||
if (err) sqlite3_free(err);
|
||||
if (db_exec_sql(create_sql) != 0) {
|
||||
DEBUG_ERROR("Failed to create ip_bans table: %s", db_last_error());
|
||||
return;
|
||||
}
|
||||
|
||||
// Migration: Add idle_* columns if they don't exist (ignore errors if already exists)
|
||||
sqlite3_exec(db, "ALTER TABLE ip_bans ADD COLUMN idle_failure_count INTEGER NOT NULL DEFAULT 0", NULL, NULL, NULL);
|
||||
sqlite3_exec(db, "ALTER TABLE ip_bans ADD COLUMN idle_ban_count INTEGER NOT NULL DEFAULT 0", NULL, NULL, NULL);
|
||||
sqlite3_exec(db, "ALTER TABLE ip_bans ADD COLUMN idle_banned_until INTEGER NOT NULL DEFAULT 0", NULL, NULL, NULL);
|
||||
sqlite3_exec(db, "ALTER TABLE ip_bans ADD COLUMN idle_first_failure INTEGER NOT NULL DEFAULT 0", NULL, NULL, NULL);
|
||||
(void)db_exec_sql("ALTER TABLE ip_bans ADD COLUMN idle_failure_count INTEGER NOT NULL DEFAULT 0");
|
||||
(void)db_exec_sql("ALTER TABLE ip_bans ADD COLUMN idle_ban_count INTEGER NOT NULL DEFAULT 0");
|
||||
(void)db_exec_sql("ALTER TABLE ip_bans ADD COLUMN idle_banned_until INTEGER NOT NULL DEFAULT 0");
|
||||
(void)db_exec_sql("ALTER TABLE ip_bans ADD COLUMN idle_first_failure INTEGER NOT NULL DEFAULT 0");
|
||||
|
||||
const char* sql =
|
||||
"SELECT ip, failure_count, ban_count, banned_until, first_failure,"
|
||||
@@ -141,16 +138,16 @@ void ip_ban_load_from_db(void) {
|
||||
" idle_failure_count, idle_ban_count, idle_banned_until, idle_first_failure"
|
||||
" FROM ip_bans";
|
||||
|
||||
sqlite3_stmt* stmt;
|
||||
if (sqlite3_prepare_v2(db, sql, -1, &stmt, NULL) != SQLITE_OK) {
|
||||
db_stmt_t* stmt;
|
||||
if (db_prepare(sql, &stmt) != DB_OK) {
|
||||
DEBUG_ERROR("Failed to prepare ip_bans load query");
|
||||
return;
|
||||
}
|
||||
|
||||
int loaded = 0;
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
while (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
const char* ip = (const char*)sqlite3_column_text(stmt, 0);
|
||||
while (db_step_stmt(stmt) == DB_ROW) {
|
||||
const char* ip = db_column_text_value(stmt, 0);
|
||||
if (!ip) continue;
|
||||
|
||||
int idx = find_slot(ip);
|
||||
@@ -160,25 +157,25 @@ void ip_ban_load_from_db(void) {
|
||||
entry->state = IP_BAN_ACTIVE;
|
||||
strncpy(entry->ip, ip, sizeof(entry->ip) - 1);
|
||||
entry->ip[sizeof(entry->ip) - 1] = '\0';
|
||||
entry->failure_count = sqlite3_column_int(stmt, 1);
|
||||
entry->ban_count = sqlite3_column_int(stmt, 2);
|
||||
entry->banned_until = (time_t)sqlite3_column_int64(stmt, 3);
|
||||
entry->first_failure = (time_t)sqlite3_column_int64(stmt, 4);
|
||||
entry->has_authed_successfully = sqlite3_column_int(stmt, 5);
|
||||
entry->last_success_at = (time_t)sqlite3_column_int64(stmt, 6);
|
||||
entry->total_connections = sqlite3_column_int(stmt, 7);
|
||||
entry->total_failures = sqlite3_column_int(stmt, 8);
|
||||
entry->total_successes = sqlite3_column_int(stmt, 9);
|
||||
entry->first_seen = (time_t)sqlite3_column_int64(stmt, 10);
|
||||
entry->failure_count = db_column_int_value(stmt, 1);
|
||||
entry->ban_count = db_column_int_value(stmt, 2);
|
||||
entry->banned_until = (time_t)db_column_int64_value(stmt, 3);
|
||||
entry->first_failure = (time_t)db_column_int64_value(stmt, 4);
|
||||
entry->has_authed_successfully = db_column_int_value(stmt, 5);
|
||||
entry->last_success_at = (time_t)db_column_int64_value(stmt, 6);
|
||||
entry->total_connections = db_column_int_value(stmt, 7);
|
||||
entry->total_failures = db_column_int_value(stmt, 8);
|
||||
entry->total_successes = db_column_int_value(stmt, 9);
|
||||
entry->first_seen = (time_t)db_column_int64_value(stmt, 10);
|
||||
// Load idle tracking fields (default to 0 if columns don't exist yet)
|
||||
entry->idle_failure_count = sqlite3_column_int(stmt, 11);
|
||||
entry->idle_ban_count = sqlite3_column_int(stmt, 12);
|
||||
entry->idle_banned_until = (time_t)sqlite3_column_int64(stmt, 13);
|
||||
entry->idle_first_failure = (time_t)sqlite3_column_int64(stmt, 14);
|
||||
entry->idle_failure_count = db_column_int_value(stmt, 11);
|
||||
entry->idle_ban_count = db_column_int_value(stmt, 12);
|
||||
entry->idle_banned_until = (time_t)db_column_int64_value(stmt, 13);
|
||||
entry->idle_first_failure = (time_t)db_column_int64_value(stmt, 14);
|
||||
loaded++;
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
sqlite3_finalize(stmt);
|
||||
db_finalize_stmt(stmt);
|
||||
|
||||
// Count how many are still actively banned
|
||||
time_t now = time(NULL);
|
||||
@@ -196,8 +193,7 @@ void ip_ban_load_from_db(void) {
|
||||
}
|
||||
|
||||
void ip_ban_save_to_db(void) {
|
||||
sqlite3* db = db_get_handle();
|
||||
if (!db || !g_initialized) return;
|
||||
if (!db_is_available() || !g_initialized) return;
|
||||
|
||||
const char* upsert_sql =
|
||||
"INSERT OR REPLACE INTO ip_bans"
|
||||
@@ -207,13 +203,13 @@ void ip_ban_save_to_db(void) {
|
||||
" idle_failure_count, idle_ban_count, idle_banned_until, idle_first_failure)"
|
||||
" VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, strftime('%s', 'now'), ?, ?, ?, ?)";
|
||||
|
||||
sqlite3_stmt* stmt;
|
||||
if (sqlite3_prepare_v2(db, upsert_sql, -1, &stmt, NULL) != SQLITE_OK) {
|
||||
db_stmt_t* stmt;
|
||||
if (db_prepare(upsert_sql, &stmt) != DB_OK) {
|
||||
DEBUG_ERROR("Failed to prepare ip_bans save query");
|
||||
return;
|
||||
}
|
||||
|
||||
sqlite3_exec(db, "BEGIN TRANSACTION", NULL, NULL, NULL);
|
||||
(void)db_exec_sql("BEGIN TRANSACTION");
|
||||
|
||||
int saved = 0;
|
||||
pthread_mutex_lock(&g_ban_mutex);
|
||||
@@ -221,25 +217,25 @@ void ip_ban_save_to_db(void) {
|
||||
ip_ban_entry_t* entry = &g_ban_table[i];
|
||||
if (entry->state != IP_BAN_ACTIVE) continue;
|
||||
|
||||
sqlite3_reset(stmt);
|
||||
sqlite3_bind_text(stmt, 1, entry->ip, -1, SQLITE_STATIC);
|
||||
sqlite3_bind_int(stmt, 2, entry->failure_count);
|
||||
sqlite3_bind_int(stmt, 3, entry->ban_count);
|
||||
sqlite3_bind_int64(stmt, 4, (sqlite3_int64)entry->banned_until);
|
||||
sqlite3_bind_int64(stmt, 5, (sqlite3_int64)entry->first_failure);
|
||||
sqlite3_bind_int(stmt, 6, entry->has_authed_successfully);
|
||||
sqlite3_bind_int64(stmt, 7, (sqlite3_int64)entry->last_success_at);
|
||||
sqlite3_bind_int(stmt, 8, entry->total_connections);
|
||||
sqlite3_bind_int(stmt, 9, entry->total_failures);
|
||||
sqlite3_bind_int(stmt, 10, entry->total_successes);
|
||||
sqlite3_bind_int64(stmt, 11, (sqlite3_int64)entry->first_seen);
|
||||
(void)db_reset_stmt(stmt);
|
||||
db_bind_text_param(stmt, 1, entry->ip);
|
||||
db_bind_int_param(stmt, 2, entry->failure_count);
|
||||
db_bind_int_param(stmt, 3, entry->ban_count);
|
||||
db_bind_int64_param(stmt, 4, (long long)entry->banned_until);
|
||||
db_bind_int64_param(stmt, 5, (long long)entry->first_failure);
|
||||
db_bind_int_param(stmt, 6, entry->has_authed_successfully);
|
||||
db_bind_int64_param(stmt, 7, (long long)entry->last_success_at);
|
||||
db_bind_int_param(stmt, 8, entry->total_connections);
|
||||
db_bind_int_param(stmt, 9, entry->total_failures);
|
||||
db_bind_int_param(stmt, 10, entry->total_successes);
|
||||
db_bind_int64_param(stmt, 11, (long long)entry->first_seen);
|
||||
// Idle tracking fields
|
||||
sqlite3_bind_int(stmt, 12, entry->idle_failure_count);
|
||||
sqlite3_bind_int(stmt, 13, entry->idle_ban_count);
|
||||
sqlite3_bind_int64(stmt, 14, (sqlite3_int64)entry->idle_banned_until);
|
||||
sqlite3_bind_int64(stmt, 15, (sqlite3_int64)entry->idle_first_failure);
|
||||
db_bind_int_param(stmt, 12, entry->idle_failure_count);
|
||||
db_bind_int_param(stmt, 13, entry->idle_ban_count);
|
||||
db_bind_int64_param(stmt, 14, (long long)entry->idle_banned_until);
|
||||
db_bind_int64_param(stmt, 15, (long long)entry->idle_first_failure);
|
||||
|
||||
if (sqlite3_step(stmt) != SQLITE_DONE) {
|
||||
if (db_step_stmt(stmt) != DB_DONE) {
|
||||
DEBUG_WARN("Failed to save ip_ban entry for %s", entry->ip);
|
||||
} else {
|
||||
saved++;
|
||||
@@ -247,8 +243,8 @@ void ip_ban_save_to_db(void) {
|
||||
}
|
||||
pthread_mutex_unlock(&g_ban_mutex);
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
sqlite3_exec(db, "COMMIT", NULL, NULL, NULL);
|
||||
db_finalize_stmt(stmt);
|
||||
(void)db_exec_sql("COMMIT");
|
||||
|
||||
DEBUG_TRACE("IP ban table saved: %d entries written to DB", saved);
|
||||
}
|
||||
|
||||
+86
-275
@@ -426,7 +426,7 @@ int init_database(const char* database_path_override) {
|
||||
cleanup_stale_wal_files(db_path);
|
||||
|
||||
int rc = sqlite3_open(db_path, &g_db);
|
||||
if (rc != SQLITE_OK) {
|
||||
if (rc != DB_OK) {
|
||||
DEBUG_ERROR("Cannot open database");
|
||||
DEBUG_TRACE("Exiting init_database() - failed to open database");
|
||||
return -1;
|
||||
@@ -445,64 +445,41 @@ int init_database(const char* database_path_override) {
|
||||
// DEBUG_GUARD_END
|
||||
|
||||
// Check if database is already initialized by looking for the events table
|
||||
const char* check_sql = "SELECT name FROM sqlite_master WHERE type='table' AND name='events'";
|
||||
sqlite3_stmt* check_stmt;
|
||||
rc = sqlite3_prepare_v2(g_db, check_sql, -1, &check_stmt, NULL);
|
||||
if (rc == SQLITE_OK) {
|
||||
int has_events_table = (sqlite3_step(check_stmt) == SQLITE_ROW);
|
||||
sqlite3_finalize(check_stmt);
|
||||
int has_events_table = 0;
|
||||
if (db_table_exists("events", &has_events_table) == 0) {
|
||||
|
||||
if (has_events_table) {
|
||||
// Check existing schema version and migrate if needed
|
||||
const char* version_sql = "SELECT value FROM schema_info WHERE key = 'version'";
|
||||
sqlite3_stmt* version_stmt;
|
||||
const char* db_version = NULL;
|
||||
char* db_version = db_get_schema_version_dup();
|
||||
int needs_migration = 0;
|
||||
|
||||
if (sqlite3_prepare_v2(g_db, version_sql, -1, &version_stmt, NULL) == SQLITE_OK) {
|
||||
if (sqlite3_step(version_stmt) == SQLITE_ROW) {
|
||||
db_version = (char*)sqlite3_column_text(version_stmt, 0);
|
||||
|
||||
// Check if migration is needed
|
||||
if (!db_version || strcmp(db_version, "5") == 0) {
|
||||
needs_migration = 1;
|
||||
} else if (strcmp(db_version, "6") == 0) {
|
||||
// Database is at schema version v6 (compatible)
|
||||
} else if (strcmp(db_version, "7") == 0) {
|
||||
// Database is at schema version v7 (compatible)
|
||||
} else if (strcmp(db_version, "8") == 0) {
|
||||
// Database is at schema version v8 (compatible)
|
||||
} else if (strcmp(db_version, "9") == 0) {
|
||||
// Database is at schema version v9 (compatible)
|
||||
} else if (strcmp(db_version, "10") == 0) {
|
||||
// Database is at schema version v10 (compatible)
|
||||
} else if (strcmp(db_version, EMBEDDED_SCHEMA_VERSION) == 0) {
|
||||
// Database is at current schema version
|
||||
} else {
|
||||
char warning_msg[256];
|
||||
snprintf(warning_msg, sizeof(warning_msg), "Unknown database schema version: %s (expected %s)",
|
||||
db_version, EMBEDDED_SCHEMA_VERSION);
|
||||
DEBUG_WARN(warning_msg);
|
||||
}
|
||||
} else {
|
||||
needs_migration = 1;
|
||||
}
|
||||
sqlite3_finalize(version_stmt);
|
||||
} else {
|
||||
// Check if migration is needed
|
||||
if (!db_version || strcmp(db_version, "5") == 0) {
|
||||
needs_migration = 1;
|
||||
} else if (strcmp(db_version, "6") == 0) {
|
||||
// Database is at schema version v6 (compatible)
|
||||
} else if (strcmp(db_version, "7") == 0) {
|
||||
// Database is at schema version v7 (compatible)
|
||||
} else if (strcmp(db_version, "8") == 0) {
|
||||
// Database is at schema version v8 (compatible)
|
||||
} else if (strcmp(db_version, "9") == 0) {
|
||||
// Database is at schema version v9 (compatible)
|
||||
} else if (strcmp(db_version, "10") == 0) {
|
||||
// Database is at schema version v10 (compatible)
|
||||
} else if (strcmp(db_version, EMBEDDED_SCHEMA_VERSION) == 0) {
|
||||
// Database is at current schema version
|
||||
} else {
|
||||
char warning_msg[256];
|
||||
snprintf(warning_msg, sizeof(warning_msg), "Unknown database schema version: %s (expected %s)",
|
||||
db_version, EMBEDDED_SCHEMA_VERSION);
|
||||
DEBUG_WARN(warning_msg);
|
||||
}
|
||||
|
||||
// Perform migration if needed
|
||||
if (needs_migration) {
|
||||
// Check if auth_rules table already exists
|
||||
const char* check_auth_rules_sql = "SELECT name FROM sqlite_master WHERE type='table' AND name='auth_rules'";
|
||||
sqlite3_stmt* check_stmt;
|
||||
int has_auth_rules = 0;
|
||||
|
||||
if (sqlite3_prepare_v2(g_db, check_auth_rules_sql, -1, &check_stmt, NULL) == SQLITE_OK) {
|
||||
has_auth_rules = (sqlite3_step(check_stmt) == SQLITE_ROW);
|
||||
sqlite3_finalize(check_stmt);
|
||||
}
|
||||
(void)db_table_exists("auth_rules", &has_auth_rules);
|
||||
|
||||
if (!has_auth_rules) {
|
||||
// Add auth_rules table matching sql_schema.h
|
||||
@@ -519,14 +496,9 @@ int init_database(const char* database_path_override) {
|
||||
" updated_at INTEGER NOT NULL DEFAULT (strftime('%s', 'now'))"
|
||||
");";
|
||||
|
||||
char* error_msg = NULL;
|
||||
int rc = sqlite3_exec(g_db, create_auth_rules_sql, NULL, NULL, &error_msg);
|
||||
if (rc != SQLITE_OK) {
|
||||
char error_log[512];
|
||||
snprintf(error_log, sizeof(error_log), "Failed to create auth_rules table: %s",
|
||||
error_msg ? error_msg : "unknown error");
|
||||
DEBUG_ERROR(error_log);
|
||||
if (error_msg) sqlite3_free(error_msg);
|
||||
if (db_exec_sql(create_auth_rules_sql) != 0) {
|
||||
DEBUG_ERROR("Failed to create auth_rules table");
|
||||
if (db_version) free(db_version);
|
||||
return -1;
|
||||
}
|
||||
|
||||
@@ -536,14 +508,9 @@ int init_database(const char* database_path_override) {
|
||||
"CREATE INDEX IF NOT EXISTS idx_auth_rules_type ON auth_rules(rule_type);"
|
||||
"CREATE INDEX IF NOT EXISTS idx_auth_rules_active ON auth_rules(active);";
|
||||
|
||||
char* index_error_msg = NULL;
|
||||
int index_rc = sqlite3_exec(g_db, create_auth_rules_indexes_sql, NULL, NULL, &index_error_msg);
|
||||
if (index_rc != SQLITE_OK) {
|
||||
char index_error_log[512];
|
||||
snprintf(index_error_log, sizeof(index_error_log), "Failed to create auth_rules indexes: %s",
|
||||
index_error_msg ? index_error_msg : "unknown error");
|
||||
DEBUG_ERROR(index_error_log);
|
||||
if (index_error_msg) sqlite3_free(index_error_msg);
|
||||
if (db_exec_sql(create_auth_rules_indexes_sql) != 0) {
|
||||
DEBUG_ERROR("Failed to create auth_rules indexes");
|
||||
if (db_version) free(db_version);
|
||||
return -1;
|
||||
}
|
||||
} else {
|
||||
@@ -555,32 +522,24 @@ int init_database(const char* database_path_override) {
|
||||
"INSERT OR REPLACE INTO schema_info (key, value, updated_at) "
|
||||
"VALUES ('version', '6', strftime('%s', 'now'))";
|
||||
|
||||
char* error_msg = NULL;
|
||||
int rc = sqlite3_exec(g_db, update_version_sql, NULL, NULL, &error_msg);
|
||||
if (rc != SQLITE_OK) {
|
||||
char error_log[512];
|
||||
snprintf(error_log, sizeof(error_log), "Failed to update schema version: %s",
|
||||
error_msg ? error_msg : "unknown error");
|
||||
DEBUG_ERROR(error_log);
|
||||
if (error_msg) sqlite3_free(error_msg);
|
||||
if (db_exec_sql(update_version_sql) != 0) {
|
||||
DEBUG_ERROR("Failed to update schema version");
|
||||
if (db_version) free(db_version);
|
||||
return -1;
|
||||
}
|
||||
|
||||
if (db_version) {
|
||||
free(db_version);
|
||||
}
|
||||
} else if (db_version) {
|
||||
free(db_version);
|
||||
}
|
||||
} else {
|
||||
// Initialize database schema using embedded SQL
|
||||
|
||||
// Execute the embedded schema SQL
|
||||
char* error_msg = NULL;
|
||||
rc = sqlite3_exec(g_db, EMBEDDED_SCHEMA_SQL, NULL, NULL, &error_msg);
|
||||
if (rc != SQLITE_OK) {
|
||||
char error_log[512];
|
||||
snprintf(error_log, sizeof(error_log), "Failed to initialize database schema: %s",
|
||||
error_msg ? error_msg : "unknown error");
|
||||
DEBUG_ERROR(error_log);
|
||||
if (error_msg) {
|
||||
sqlite3_free(error_msg);
|
||||
}
|
||||
if (db_exec_sql(EMBEDDED_SCHEMA_SQL) != 0) {
|
||||
DEBUG_ERROR("Failed to initialize database schema");
|
||||
return -1;
|
||||
}
|
||||
|
||||
@@ -591,14 +550,8 @@ int init_database(const char* database_path_override) {
|
||||
}
|
||||
|
||||
// Enable WAL mode for better concurrency and crash recovery
|
||||
char* wal_error = NULL;
|
||||
rc = sqlite3_exec(g_db, "PRAGMA journal_mode=WAL;", NULL, NULL, &wal_error);
|
||||
if (rc != SQLITE_OK) {
|
||||
char error_msg[256];
|
||||
snprintf(error_msg, sizeof(error_msg), "Failed to enable WAL mode: %s",
|
||||
wal_error ? wal_error : "unknown error");
|
||||
DEBUG_WARN(error_msg);
|
||||
if (wal_error) sqlite3_free(wal_error);
|
||||
if (db_exec_sql("PRAGMA journal_mode=WAL;") != 0) {
|
||||
DEBUG_WARN("Failed to enable WAL mode");
|
||||
// Continue anyway - WAL mode is optional
|
||||
} else {
|
||||
DEBUG_LOG("SQLite WAL mode enabled");
|
||||
@@ -611,11 +564,8 @@ int init_database(const char* database_path_override) {
|
||||
if (mmap_size > 0) {
|
||||
char mmap_pragma[64];
|
||||
snprintf(mmap_pragma, sizeof(mmap_pragma), "PRAGMA mmap_size=%ld;", mmap_size);
|
||||
char* mmap_error = NULL;
|
||||
rc = sqlite3_exec(g_db, mmap_pragma, NULL, NULL, &mmap_error);
|
||||
if (rc != SQLITE_OK) {
|
||||
DEBUG_WARN("Failed to set mmap_size: %s", mmap_error ? mmap_error : "unknown");
|
||||
if (mmap_error) sqlite3_free(mmap_error);
|
||||
if (db_exec_sql(mmap_pragma) != 0) {
|
||||
DEBUG_WARN("Failed to set mmap_size");
|
||||
} else {
|
||||
DEBUG_LOG("SQLite mmap_size set to %ld bytes", mmap_size);
|
||||
}
|
||||
@@ -628,11 +578,8 @@ int init_database(const char* database_path_override) {
|
||||
char cache_pragma[64];
|
||||
// Use negative value so SQLite interprets it as KB rather than page count
|
||||
snprintf(cache_pragma, sizeof(cache_pragma), "PRAGMA cache_size=-%d;", cache_size_kb > 0 ? cache_size_kb : -cache_size_kb);
|
||||
char* cache_error = NULL;
|
||||
rc = sqlite3_exec(g_db, cache_pragma, NULL, NULL, &cache_error);
|
||||
if (rc != SQLITE_OK) {
|
||||
DEBUG_WARN("Failed to set cache_size: %s", cache_error ? cache_error : "unknown");
|
||||
if (cache_error) sqlite3_free(cache_error);
|
||||
if (db_exec_sql(cache_pragma) != 0) {
|
||||
DEBUG_WARN("Failed to set cache_size");
|
||||
} else {
|
||||
DEBUG_LOG("SQLite cache_size set to %d KB", cache_size_kb);
|
||||
}
|
||||
@@ -649,14 +596,8 @@ void close_database() {
|
||||
if (g_db) {
|
||||
// Perform WAL checkpoint to minimize stale files on next startup
|
||||
DEBUG_LOG("Performing WAL checkpoint before database close");
|
||||
char* checkpoint_error = NULL;
|
||||
int rc = sqlite3_exec(g_db, "PRAGMA wal_checkpoint(TRUNCATE);", NULL, NULL, &checkpoint_error);
|
||||
if (rc != SQLITE_OK) {
|
||||
char error_msg[256];
|
||||
snprintf(error_msg, sizeof(error_msg), "WAL checkpoint warning: %s",
|
||||
checkpoint_error ? checkpoint_error : "unknown error");
|
||||
DEBUG_WARN(error_msg);
|
||||
if (checkpoint_error) sqlite3_free(checkpoint_error);
|
||||
if (db_exec_sql("PRAGMA wal_checkpoint(TRUNCATE);") != 0) {
|
||||
DEBUG_WARN("WAL checkpoint warning");
|
||||
}
|
||||
|
||||
sqlite3_close(g_db);
|
||||
@@ -731,43 +672,7 @@ const char* extract_d_tag_value(cJSON* tags) {
|
||||
|
||||
// Insert denormalized tags into event_tags table for fast indexed lookups
|
||||
int store_event_tags(const char* event_id, cJSON* tags) {
|
||||
if (!g_db || !event_id || !tags || !cJSON_IsArray(tags)) {
|
||||
return 0; // Not an error if no tags
|
||||
}
|
||||
|
||||
const char* sql = "INSERT INTO event_tags (event_id, tag_name, tag_value, tag_index) VALUES (?, ?, ?, ?)";
|
||||
sqlite3_stmt* stmt;
|
||||
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
|
||||
if (rc != SQLITE_OK) {
|
||||
DEBUG_ERROR("Failed to prepare event_tags insert: %s", sqlite3_errmsg(g_db));
|
||||
return -1;
|
||||
}
|
||||
|
||||
int tag_index = 0;
|
||||
cJSON* tag = NULL;
|
||||
cJSON_ArrayForEach(tag, tags) {
|
||||
if (cJSON_IsArray(tag) && cJSON_GetArraySize(tag) >= 2) {
|
||||
cJSON* name = cJSON_GetArrayItem(tag, 0);
|
||||
cJSON* value = cJSON_GetArrayItem(tag, 1);
|
||||
|
||||
if (cJSON_IsString(name) && cJSON_IsString(value)) {
|
||||
sqlite3_reset(stmt);
|
||||
sqlite3_bind_text(stmt, 1, event_id, -1, SQLITE_STATIC);
|
||||
sqlite3_bind_text(stmt, 2, cJSON_GetStringValue(name), -1, SQLITE_STATIC);
|
||||
sqlite3_bind_text(stmt, 3, cJSON_GetStringValue(value), -1, SQLITE_STATIC);
|
||||
sqlite3_bind_int(stmt, 4, tag_index);
|
||||
|
||||
rc = sqlite3_step(stmt);
|
||||
if (rc != SQLITE_DONE) {
|
||||
DEBUG_ERROR("Failed to insert event tag: %s", sqlite3_errmsg(g_db));
|
||||
}
|
||||
}
|
||||
}
|
||||
tag_index++;
|
||||
}
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
return 0;
|
||||
return db_store_event_tags_cjson(event_id, tags);
|
||||
}
|
||||
|
||||
// Store event in database
|
||||
@@ -825,43 +730,34 @@ int store_event(cJSON* event) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
// Prepare SQL statement for event insertion
|
||||
const char* sql =
|
||||
"INSERT INTO events (id, pubkey, created_at, kind, event_type, content, sig, tags, event_json) "
|
||||
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)";
|
||||
|
||||
sqlite3_stmt* stmt;
|
||||
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
|
||||
if (rc != SQLITE_OK) {
|
||||
DEBUG_ERROR("Failed to prepare event insert statement");
|
||||
int rc = DB_ERROR;
|
||||
int extended_errcode = 0;
|
||||
if (db_insert_event_with_json(cJSON_GetStringValue(id),
|
||||
cJSON_GetStringValue(pubkey),
|
||||
(long long)cJSON_GetNumberValue(created_at),
|
||||
(int)cJSON_GetNumberValue(kind),
|
||||
event_type_to_string(type),
|
||||
cJSON_GetStringValue(content),
|
||||
cJSON_GetStringValue(sig),
|
||||
tags_json,
|
||||
event_json,
|
||||
&rc,
|
||||
&extended_errcode) != 0) {
|
||||
DEBUG_ERROR("Failed to execute event insert operation");
|
||||
free(tags_json);
|
||||
free(event_json);
|
||||
return -1;
|
||||
}
|
||||
|
||||
// Bind parameters
|
||||
sqlite3_bind_text(stmt, 1, cJSON_GetStringValue(id), -1, SQLITE_STATIC);
|
||||
sqlite3_bind_text(stmt, 2, cJSON_GetStringValue(pubkey), -1, SQLITE_STATIC);
|
||||
sqlite3_bind_int64(stmt, 3, (sqlite3_int64)cJSON_GetNumberValue(created_at));
|
||||
sqlite3_bind_int(stmt, 4, (int)cJSON_GetNumberValue(kind));
|
||||
sqlite3_bind_text(stmt, 5, event_type_to_string(type), -1, SQLITE_STATIC);
|
||||
sqlite3_bind_text(stmt, 6, cJSON_GetStringValue(content), -1, SQLITE_STATIC);
|
||||
sqlite3_bind_text(stmt, 7, cJSON_GetStringValue(sig), -1, SQLITE_STATIC);
|
||||
sqlite3_bind_text(stmt, 8, tags_json, -1, SQLITE_TRANSIENT);
|
||||
sqlite3_bind_text(stmt, 9, event_json, -1, SQLITE_TRANSIENT);
|
||||
|
||||
// Execute statement
|
||||
rc = sqlite3_step(stmt);
|
||||
if (rc != SQLITE_DONE) {
|
||||
const char* err_msg = sqlite3_errmsg(g_db);
|
||||
int extended_errcode = sqlite3_extended_errcode(g_db);
|
||||
if (rc != SQLITE_CONSTRAINT) {
|
||||
|
||||
if (rc != DB_DONE) {
|
||||
const char* err_msg = db_last_error();
|
||||
if (rc != DB_CONSTRAINT) {
|
||||
DEBUG_ERROR("INSERT failed: rc=%d, extended_errcode=%d, msg=%s", rc, extended_errcode, err_msg);
|
||||
}
|
||||
}
|
||||
sqlite3_finalize(stmt);
|
||||
|
||||
if (rc != SQLITE_DONE) {
|
||||
if (rc == SQLITE_CONSTRAINT) {
|
||||
if (rc != DB_DONE) {
|
||||
if (rc == DB_CONSTRAINT) {
|
||||
DEBUG_WARN("Event already exists in database");
|
||||
|
||||
// Add TRACE level debug to show both events
|
||||
@@ -888,7 +784,7 @@ int store_event(cJSON* event) {
|
||||
return 0; // Not an error, just duplicate
|
||||
}
|
||||
char error_msg[256];
|
||||
snprintf(error_msg, sizeof(error_msg), "Failed to insert event: %s", sqlite3_errmsg(g_db));
|
||||
snprintf(error_msg, sizeof(error_msg), "Failed to insert event: %s", db_last_error());
|
||||
DEBUG_ERROR(error_msg);
|
||||
free(tags_json);
|
||||
free(event_json);
|
||||
@@ -929,60 +825,16 @@ int store_event(cJSON* event) {
|
||||
int event_id_exists_in_db(const char* event_id) {
|
||||
if (!g_db || !event_id || strlen(event_id) != 64) return 0;
|
||||
|
||||
sqlite3_stmt* stmt;
|
||||
const char* sql = "SELECT 1 FROM events WHERE id=? LIMIT 1";
|
||||
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) return 0;
|
||||
|
||||
sqlite3_bind_text(stmt, 1, event_id, 64, SQLITE_STATIC);
|
||||
int exists = (sqlite3_step(stmt) == SQLITE_ROW) ? 1 : 0;
|
||||
sqlite3_finalize(stmt);
|
||||
int exists = 0;
|
||||
if (db_event_id_exists(event_id, &exists) != 0) {
|
||||
return 0;
|
||||
}
|
||||
return exists;
|
||||
}
|
||||
|
||||
// Populate event_tags from existing events (run once at startup)
|
||||
int populate_event_tags_from_existing(void) {
|
||||
if (!g_db) return -1;
|
||||
|
||||
// Check if event_tags is already populated
|
||||
sqlite3_stmt* check_stmt;
|
||||
sqlite3_prepare_v2(g_db, "SELECT COUNT(*) FROM event_tags", -1, &check_stmt, NULL);
|
||||
if (sqlite3_step(check_stmt) == SQLITE_ROW && sqlite3_column_int(check_stmt, 0) > 0) {
|
||||
sqlite3_finalize(check_stmt);
|
||||
DEBUG_INFO("event_tags already populated, skipping");
|
||||
return 0;
|
||||
}
|
||||
sqlite3_finalize(check_stmt);
|
||||
|
||||
DEBUG_INFO("Populating event_tags from existing events...");
|
||||
|
||||
const char* sql = "SELECT id, tags FROM events WHERE tags != '[]'";
|
||||
sqlite3_stmt* stmt;
|
||||
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
|
||||
if (rc != SQLITE_OK) return -1;
|
||||
|
||||
// Use a transaction for bulk insert performance
|
||||
sqlite3_exec(g_db, "BEGIN TRANSACTION", NULL, NULL, NULL);
|
||||
|
||||
int event_count = 0;
|
||||
while (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
const char* event_id = (const char*)sqlite3_column_text(stmt, 0);
|
||||
const char* tags_json = (const char*)sqlite3_column_text(stmt, 1);
|
||||
|
||||
if (event_id && tags_json) {
|
||||
cJSON* tags = cJSON_Parse(tags_json);
|
||||
if (tags) {
|
||||
store_event_tags(event_id, tags);
|
||||
cJSON_Delete(tags);
|
||||
event_count++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
sqlite3_exec(g_db, "COMMIT", NULL, NULL, NULL);
|
||||
|
||||
DEBUG_INFO("Populated event_tags for %d events", event_count);
|
||||
return 0;
|
||||
return db_populate_event_tags_from_existing();
|
||||
}
|
||||
|
||||
/////////////////////////////////////////////////////////////////////////////////////////
|
||||
@@ -992,48 +844,7 @@ int populate_event_tags_from_existing(void) {
|
||||
/////////////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
cJSON* retrieve_event(const char* event_id) {
|
||||
if (!g_db || !event_id) {
|
||||
return NULL;
|
||||
}
|
||||
|
||||
const char* sql =
|
||||
"SELECT id, pubkey, created_at, kind, content, sig, tags FROM events WHERE id = ?";
|
||||
|
||||
sqlite3_stmt* stmt;
|
||||
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
|
||||
if (rc != SQLITE_OK) {
|
||||
return NULL;
|
||||
}
|
||||
|
||||
sqlite3_bind_text(stmt, 1, event_id, -1, SQLITE_STATIC);
|
||||
|
||||
cJSON* event = NULL;
|
||||
if (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
event = cJSON_CreateObject();
|
||||
|
||||
cJSON_AddStringToObject(event, "id", (char*)sqlite3_column_text(stmt, 0));
|
||||
cJSON_AddStringToObject(event, "pubkey", (char*)sqlite3_column_text(stmt, 1));
|
||||
cJSON_AddNumberToObject(event, "created_at", sqlite3_column_int64(stmt, 2));
|
||||
cJSON_AddNumberToObject(event, "kind", sqlite3_column_int(stmt, 3));
|
||||
cJSON_AddStringToObject(event, "content", (char*)sqlite3_column_text(stmt, 4));
|
||||
cJSON_AddStringToObject(event, "sig", (char*)sqlite3_column_text(stmt, 5));
|
||||
|
||||
// Parse tags JSON
|
||||
const char* tags_json = (char*)sqlite3_column_text(stmt, 6);
|
||||
if (tags_json) {
|
||||
cJSON* tags = cJSON_Parse(tags_json);
|
||||
if (tags) {
|
||||
cJSON_AddItemToObject(event, "tags", tags);
|
||||
} else {
|
||||
cJSON_AddItemToObject(event, "tags", cJSON_CreateArray());
|
||||
}
|
||||
} else {
|
||||
cJSON_AddItemToObject(event, "tags", cJSON_CreateArray());
|
||||
}
|
||||
}
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
return event;
|
||||
return db_retrieve_event_by_id(event_id);
|
||||
}
|
||||
|
||||
|
||||
@@ -1548,11 +1359,11 @@ int handle_req_message(const char* sub_id, cJSON* filters, struct lws *wsi, stru
|
||||
clock_gettime(CLOCK_MONOTONIC, &query_start);
|
||||
|
||||
// Execute query and send events
|
||||
sqlite3_stmt* stmt;
|
||||
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
|
||||
if (rc != SQLITE_OK) {
|
||||
db_stmt_t* stmt;
|
||||
int rc = db_prepare(sql, &stmt);
|
||||
if (rc != DB_OK) {
|
||||
char error_msg[256];
|
||||
snprintf(error_msg, sizeof(error_msg), "Failed to prepare subscription query: %s", sqlite3_errmsg(g_db));
|
||||
snprintf(error_msg, sizeof(error_msg), "Failed to prepare subscription query: %s", db_last_error());
|
||||
DEBUG_ERROR(error_msg);
|
||||
|
||||
// Log the failed query so we can see what SQL was generated
|
||||
@@ -1565,7 +1376,7 @@ int handle_req_message(const char* sub_id, cJSON* filters, struct lws *wsi, stru
|
||||
timestamp,
|
||||
sub_id,
|
||||
pss ? pss->client_ip : "N/A",
|
||||
sqlite3_errmsg(g_db),
|
||||
db_last_error(),
|
||||
sql);
|
||||
fflush(stderr);
|
||||
}
|
||||
@@ -1580,7 +1391,7 @@ int handle_req_message(const char* sub_id, cJSON* filters, struct lws *wsi, stru
|
||||
|
||||
// Bind parameters
|
||||
for (int i = 0; i < bind_param_count; i++) {
|
||||
sqlite3_bind_text(stmt, i + 1, bind_params[i], -1, SQLITE_TRANSIENT);
|
||||
db_bind_text_param(stmt, i + 1, bind_params[i]);
|
||||
}
|
||||
|
||||
// Cache config values outside the row loop (performance fix)
|
||||
@@ -1588,7 +1399,7 @@ int handle_req_message(const char* sub_id, cJSON* filters, struct lws *wsi, stru
|
||||
int filter_responses = get_config_bool("expiration_filter", 1);
|
||||
|
||||
int row_count = 0;
|
||||
while (sqlite3_step(stmt) == SQLITE_ROW) {
|
||||
while (db_step_stmt(stmt) == DB_ROW) {
|
||||
row_count++;
|
||||
|
||||
// Track rows returned for abuse detection
|
||||
@@ -1597,7 +1408,7 @@ int handle_req_message(const char* sub_id, cJSON* filters, struct lws *wsi, stru
|
||||
}
|
||||
|
||||
// Get pre-serialized event JSON (no reconstruction needed!)
|
||||
const char* event_json_str = (char*)sqlite3_column_text(stmt, 0);
|
||||
const char* event_json_str = db_column_text_value(stmt, 0);
|
||||
if (!event_json_str) {
|
||||
DEBUG_ERROR("Event has NULL event_json field");
|
||||
continue;
|
||||
@@ -1645,7 +1456,7 @@ int handle_req_message(const char* sub_id, cJSON* filters, struct lws *wsi, stru
|
||||
events_sent++;
|
||||
}
|
||||
|
||||
sqlite3_finalize(stmt);
|
||||
db_finalize_stmt(stmt);
|
||||
|
||||
// Stop query timing and log
|
||||
clock_gettime(CLOCK_MONOTONIC, &query_end);
|
||||
|
||||
+2
-2
@@ -13,8 +13,8 @@
|
||||
// Using CRELAY_ prefix to avoid conflicts with nostr_core_lib VERSION macros
|
||||
#define CRELAY_VERSION_MAJOR 2
|
||||
#define CRELAY_VERSION_MINOR 0
|
||||
#define CRELAY_VERSION_PATCH 2
|
||||
#define CRELAY_VERSION "v2.0.2"
|
||||
#define CRELAY_VERSION_PATCH 4
|
||||
#define CRELAY_VERSION "v2.0.4"
|
||||
|
||||
// Relay metadata (authoritative source for NIP-11 information)
|
||||
#define RELAY_NAME "C-Relay"
|
||||
|
||||
@@ -10,7 +10,6 @@
|
||||
#include <signal.h>
|
||||
#include <time.h>
|
||||
#include <pthread.h>
|
||||
#include <sqlite3.h>
|
||||
|
||||
// Include libwebsockets after pthread.h to ensure pthread_rwlock_t is defined
|
||||
#include <libwebsockets.h>
|
||||
|
||||
Reference in New Issue
Block a user