laantungir/c-relay
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 10 Wiki Activity

v2.0.3 - Complete Phase 1 SQLite abstraction cleanup across config/api/websockets and add db path accessor

Browse Source
This commit is contained in:
Laan Tungir
2026-04-01 07:47:21 -04:00
parent 192eeb248d
commit bc9ac290ab
11 changed files with 960 additions and 820 deletions
Download Patch File Download Diff File Expand all files Collapse all files
nostr-rs-relay
Submodule
+1
Submodule nostr-rs-relay added at 64cfcaf44a
postgres_relays.md
+91
View File
@@ -0,0 +1,91 @@
# PostgreSQL Nostr Relay Architecture Analysis
This document summarizes common approaches and best practices for building Nostr relays backed by PostgreSQL, based on industry-standard designs and existing open-source relay implementations.
## Identified Projects
* **Nostrss**: High-performance Rust-based relay.
* **Nostrgres**: Focused on PostgreSQL integration with complex event filtering.
* **Relay-rs (Postgres branch)**: General purpose high-throughput relay.
## Detailed Analysis
### 1. Common Schema Patterns
* **Event Storage (JSONB)**: Most PostgreSQL-backed relays store the raw event as a JSONB object in a central `events` table. This provides flexibility for the evolving Nostr spec while allowing efficient extraction of fields.
* **Tag Denormalization**: While the raw event is in JSONB, performant relays extract tags (like `p`, `e`, `t`, `d`) into a separate `tags` table with foreign key relationships to the `events` table. This avoids slow JSONB traversal during complex filter queries.
* **Author/Publisher Tracking**: A dedicated `authors` or `pubkeys` table is typically used to maintain indexing on the `pubkey` field, enabling quick lookups of user activity.
### 2. Performance Optimization
* **GIN Indexes on JSONB**: Crucial for filtering on specific event tags or custom properties stored within the event object. GIN indexes with `jsonb_path_ops` are generally preferred for equality checks.
* **Time-Series Partitioning**: Given the append-only nature of Nostr, partitioning the `events` table by time (e.g., daily or weekly chunks) is highly recommended. This significantly improves query performance for recent data and simplifies data expiration/deletion.
* **Clustering**: Clustering the `events` table by the timestamp index can reduce disk I/O, as it keeps temporally related events physically adjacent on the disk.
### 3. Schema Management Approaches
* **Migrations**: Most mature relays utilize migration tools (like `Flyway`, `Diesel migrations`, or `Golang-migrate`) to version control database schema changes. This is critical for production stability.
* **Auto-generation**: Some lightweight prototypes auto-generate schemas at startup. This is generally discouraged for production due to the risk of destructive changes, data loss, or blocking DDL operations on large tables.
## Recommendations for Building a New Relay
1. **Prioritize Denormalization**: Do not rely solely on JSONB queries for high-traffic filters. Extract indexed tags into dedicated columns or tables.
2. **Use Partitioning from Day One**: Implementing native PostgreSQL partitioning (e.g., using `pg_partman`) is much easier before the dataset grows to millions of rows.
3. **Connection Pooling**: PostgreSQL requires aggressive connection management. Use a high-performance pooler like `PgBouncer` to handle the large number of concurrent, short-lived connections common in WebSocket-based relay traffic.
4. **Asynchronous Writes**: Decouple the WebSocket ingestion thread from the database writer thread to ensure that slow database writes do not impact the relay's responsiveness.
## Schema Definitions
### Relay-rs (Postgres)
```sql
-- Events table
CREATE TABLE "event" (
id bytea NOT NULL,
pub_key bytea NOT NULL,
created_at timestamp with time zone NOT NULL,
kind integer NOT NULL,
"content" bytea NOT NULL,
hidden bit(1) NOT NULL DEFAULT 0::bit(1),
delegated_by bytea NULL,
first_seen timestamp with time zone NOT NULL DEFAULT now(),
expires_at timestamp(0) with time zone,
CONSTRAINT event_pkey PRIMARY KEY (id)
);
CREATE INDEX event_created_at_idx ON "event" (created_at,kind);
CREATE INDEX event_pub_key_idx ON "event" (pub_key);
CREATE INDEX event_delegated_by_idx ON "event" (delegated_by);
CREATE INDEX event_expires_at_idx ON "event" (expires_at);
-- Tags table
CREATE TABLE "tag" (
id int8 NOT NULL GENERATED BY DEFAULT AS IDENTITY,
event_id bytea NOT NULL,
"name" varchar NOT NULL,
value bytea NULL,
value_hex bytea NULL,
CONSTRAINT tag_fk FOREIGN KEY (event_id) REFERENCES "event"(id) ON DELETE CASCADE,
CONSTRAINT unique_constraint_name UNIQUE (event_id, "name", value, value_hex)
);
CREATE INDEX tag_event_id_idx ON tag USING btree (event_id, name);
CREATE INDEX tag_value_idx ON tag USING btree (value);
CREATE INDEX tag_value_hex_idx ON tag USING btree (value_hex);
-- Account table
CREATE TABLE "account" (
pubkey varchar NOT NULL,
is_admitted BOOLEAN NOT NULL DEFAULT FALSE,
balance BIGINT NOT NULL DEFAULT 0,
tos_accepted_at TIMESTAMP,
CONSTRAINT account_pkey PRIMARY KEY (pubkey)
);
-- Invoice table
CREATE TYPE status AS ENUM ('Paid', 'Unpaid', 'Expired');
CREATE TABLE "invoice" (
payment_hash varchar NOT NULL,
pubkey varchar NOT NULL,
invoice varchar NOT NULL,
amount BIGINT NOT NULL,
status status NOT NULL DEFAULT 'Unpaid',
description varchar,
created_at timestamp,
confirmed_at timestamp,
CONSTRAINT invoice_payment_hash PRIMARY KEY (payment_hash),
CONSTRAINT invoice_pubkey_fkey FOREIGN KEY (pubkey) REFERENCES account (pubkey) ON DELETE CASCADE
);
```
projects.txt
+10
View File
@@ -0,0 +1,10 @@
https://github.com/rushmi0/Fenrir-s
https://github.com/barkyq/gnost-relay
https://github.com/lpicanco/knostr
https://github.com/bezysoftware/netstr
https://github.com/lebrunel/nex
https://github.com/CodyTseng/nostr-relay-nestjs
https://github.com/mattn/nostr-relay
https://github.com/Cameri/nostream
https://github.com/Giszmo/NostrPostr/tree/master/NostrRelay
https://github.com/fiatjaf/relayer/tree/master/examples/basic
relay.pid
+1 -1
View File
@@ -1 +1 @@
457096
484178
src/api.c
+4 -132
View File
@@ -14,7 +14,6 @@ extern void log_query_execution(const char* query_type, const char* sub_id,
#include <libwebsockets.h>
#include <cjson/cJSON.h>
int get_active_connection_count(void);
#include <sqlite3.h>
#include <time.h>
#include <sys/stat.h>
#include <unistd.h>
@@ -688,7 +687,6 @@ static const config_definition_t known_configs[] = {
{NULL, NULL, 0, 0}
};
// External database connection (from main.c)
extern sqlite3* g_db;
extern char g_database_path[512];
// Forward declarations for database functions
@@ -993,138 +991,12 @@ char* execute_sql_query(const char* query, const char* request_id, char* error_m
return NULL;
}
if (!g_db) {
if (!db_is_available()) {
snprintf(error_message, error_size, "Database not available");
return NULL;
}
// Set busy timeout to prevent long-running queries (5 seconds)
sqlite3_busy_timeout(g_db, 5000);
// Prepare statement
sqlite3_stmt* stmt;
int rc = sqlite3_prepare_v2(g_db, query, -1, &stmt, NULL);
if (rc != SQLITE_OK) {
const char* err_msg = sqlite3_errmsg(g_db);
snprintf(error_message, error_size, "SQL prepare failed: %s", err_msg);
return NULL;
}
// Execute query and collect results
cJSON* response = cJSON_CreateObject();
cJSON_AddStringToObject(response, "query_type", "sql_query");
cJSON_AddStringToObject(response, "request_id", request_id);
cJSON_AddNumberToObject(response, "timestamp", (double)time(NULL));
cJSON_AddStringToObject(response, "query", query);
// Get column information
int col_count = sqlite3_column_count(stmt);
cJSON* columns = cJSON_CreateArray();
for (int i = 0; i < col_count; i++) {
const char* col_name = sqlite3_column_name(stmt, i);
cJSON_AddItemToArray(columns, cJSON_CreateString(col_name ? col_name : ""));
}
cJSON_AddItemToObject(response, "columns", columns);
// Execute and collect rows (with limit)
cJSON* rows = cJSON_CreateArray();
int row_count = 0;
const int MAX_ROWS = 1000; // Configurable limit
struct timespec start_time;
clock_gettime(CLOCK_MONOTONIC, &start_time);
while ((rc = sqlite3_step(stmt)) == SQLITE_ROW && row_count < MAX_ROWS) {
cJSON* row = cJSON_CreateArray();
for (int i = 0; i < col_count; i++) {
int col_type = sqlite3_column_type(stmt, i);
switch (col_type) {
case SQLITE_INTEGER:
cJSON_AddItemToArray(row, cJSON_CreateNumber((double)sqlite3_column_int64(stmt, i)));
break;
case SQLITE_FLOAT:
cJSON_AddItemToArray(row, cJSON_CreateNumber(sqlite3_column_double(stmt, i)));
break;
case SQLITE_TEXT: {
const char* text = (const char*)sqlite3_column_text(stmt, i);
cJSON_AddItemToArray(row, cJSON_CreateString(text ? text : ""));
break;
}
case SQLITE_BLOB: {
// Convert blob to hex string for JSON compatibility
const void* blob = sqlite3_column_blob(stmt, i);
int blob_size = sqlite3_column_bytes(stmt, i);
if (blob && blob_size > 0) {
char* hex_str = malloc(blob_size * 2 + 1);
if (hex_str) {
for (int j = 0; j < blob_size; j++) {
sprintf(hex_str + j * 2, "%02x", ((unsigned char*)blob)[j]);
}
hex_str[blob_size * 2] = '\0';
cJSON_AddItemToArray(row, cJSON_CreateString(hex_str));
free(hex_str);
} else {
cJSON_AddItemToArray(row, cJSON_CreateString("[BLOB]"));
}
} else {
cJSON_AddItemToArray(row, cJSON_CreateString(""));
}
break;
}
case SQLITE_NULL:
cJSON_AddItemToArray(row, cJSON_CreateNull());
break;
default:
cJSON_AddItemToArray(row, cJSON_CreateString("[UNKNOWN]"));
break;
}
}
cJSON_AddItemToArray(rows, row);
row_count++;
// Check timeout (additional safety check)
struct timespec current_time;
clock_gettime(CLOCK_MONOTONIC, &current_time);
double elapsed = (current_time.tv_sec - start_time.tv_sec) +
(current_time.tv_nsec - start_time.tv_nsec) / 1e9;
if (elapsed > 4.5) { // 4.5 seconds to allow for cleanup
break;
}
}
sqlite3_finalize(stmt);
// Check for execution errors
if (rc != SQLITE_DONE && rc != SQLITE_ROW) {
const char* err_msg = sqlite3_errmsg(g_db);
snprintf(error_message, error_size, "SQL execution failed: %s", err_msg);
cJSON_Delete(response);
return NULL;
}
// Check row limit
if (row_count >= MAX_ROWS) {
cJSON_AddStringToObject(response, "warning", "Result truncated to maximum row limit");
}
// Add metadata
cJSON_AddNumberToObject(response, "row_count", row_count);
cJSON_AddNumberToObject(response, "execution_time_ms", 0); // Will be set by caller
cJSON_AddItemToObject(response, "rows", rows);
// Convert to JSON string
char* json_result = cJSON_Print(response);
cJSON_Delete(response);
if (!json_result) {
snprintf(error_message, error_size, "Failed to generate JSON response");
return NULL;
}
return json_result;
return db_execute_readonly_query_json(query, request_id, error_message, error_size, 1000, 5000);
}
// Unified handler for SQL query commands
@@ -1304,10 +1176,10 @@ char* generate_stats_json(void) {
cJSON_AddNumberToObject(response, "timestamp", (double)time(NULL));
// Get database file size
extern char g_database_path[512];
const char* db_path = db_get_database_path();
struct stat db_stat;
long long db_size = 0;
if (stat(g_database_path, &db_stat) == 0) {
if (db_path && db_path[0] != '\0' && stat(db_path, &db_stat) == 0) {
db_size = db_stat.st_size;
}
cJSON_AddNumberToObject(response, "database_size_bytes", db_size);
src/config.c
+275 -415
View File
File diff suppressed because it is too large Load Diff
src/db_ops.c
+462
View File
@@ -20,6 +20,69 @@ sqlite3* db_get_handle(void) {
return g_db;
}
const char* db_last_error(void) {
if (!g_db) return "database not available";
return sqlite3_errmsg(g_db);
}
const char* db_get_database_path(void) {
return g_database_path;
}
int db_prepare(const char* sql, sqlite3_stmt** out_stmt) {
if (!g_db || !sql || !out_stmt) return SQLITE_MISUSE;
return sqlite3_prepare_v2(g_db, sql, -1, out_stmt, NULL);
}
int db_bind_text_param(sqlite3_stmt* stmt, int index, const char* value) {
if (!stmt) return SQLITE_MISUSE;
return sqlite3_bind_text(stmt, index, value ? value : "", -1, SQLITE_TRANSIENT);
}
int db_bind_int_param(sqlite3_stmt* stmt, int index, int value) {
if (!stmt) return SQLITE_MISUSE;
return sqlite3_bind_int(stmt, index, value);
}
int db_bind_int64_param(sqlite3_stmt* stmt, int index, long long value) {
if (!stmt) return SQLITE_MISUSE;
return sqlite3_bind_int64(stmt, index, (sqlite3_int64)value);
}
int db_step_stmt(sqlite3_stmt* stmt) {
if (!stmt) return SQLITE_MISUSE;
return sqlite3_step(stmt);
}
int db_reset_stmt(sqlite3_stmt* stmt) {
if (!stmt) return SQLITE_MISUSE;
return sqlite3_reset(stmt);
}
const char* db_column_text_value(sqlite3_stmt* stmt, int col) {
if (!stmt) return NULL;
return (const char*)sqlite3_column_text(stmt, col);
}
int db_column_int_value(sqlite3_stmt* stmt, int col) {
if (!stmt) return 0;
return sqlite3_column_int(stmt, col);
}
long long db_column_int64_value(sqlite3_stmt* stmt, int col) {
if (!stmt) return 0;
return (long long)sqlite3_column_int64(stmt, col);
}
double db_column_double_value(sqlite3_stmt* stmt, int col) {
if (!stmt) return 0.0;
return sqlite3_column_double(stmt, col);
}
void db_finalize_stmt(sqlite3_stmt* stmt) {
if (stmt) sqlite3_finalize(stmt);
}
int db_log_subscription_created(const char* sub_id, const char* wsi_ptr,
const char* client_ip, const char* filter_json) {
if (!g_db || !sub_id || !wsi_ptr || !client_ip) return -1;
@@ -315,6 +378,155 @@ int db_count_with_sql(const char* sql, const char** bind_params, int bind_param_
return 0;
}
char* db_execute_readonly_query_json(const char* query, const char* request_id,
char* error_message, size_t error_size,
int max_rows, int timeout_ms) {
if (!g_db || !query || !request_id || !error_message) return NULL;
sqlite3_busy_timeout(g_db, timeout_ms > 0 ? timeout_ms : 5000);
sqlite3_stmt* stmt = NULL;
int rc = sqlite3_prepare_v2(g_db, query, -1, &stmt, NULL);
if (rc != SQLITE_OK) {
const char* err_msg = sqlite3_errmsg(g_db);
snprintf(error_message, error_size, "SQL prepare failed: %s", err_msg ? err_msg : "unknown");
return NULL;
}
cJSON* response = cJSON_CreateObject();
if (!response) {
sqlite3_finalize(stmt);
snprintf(error_message, error_size, "Failed to allocate query response object");
return NULL;
}
cJSON_AddStringToObject(response, "query_type", "sql_query");
cJSON_AddStringToObject(response, "request_id", request_id);
cJSON_AddNumberToObject(response, "timestamp", (double)time(NULL));
cJSON_AddStringToObject(response, "query", query);
int col_count = sqlite3_column_count(stmt);
cJSON* columns = cJSON_CreateArray();
if (!columns) {
sqlite3_finalize(stmt);
cJSON_Delete(response);
snprintf(error_message, error_size, "Failed to allocate columns array");
return NULL;
}
for (int i = 0; i < col_count; i++) {
const char* col_name = sqlite3_column_name(stmt, i);
cJSON_AddItemToArray(columns, cJSON_CreateString(col_name ? col_name : ""));
}
cJSON_AddItemToObject(response, "columns", columns);
cJSON* rows = cJSON_CreateArray();
if (!rows) {
sqlite3_finalize(stmt);
cJSON_Delete(response);
snprintf(error_message, error_size, "Failed to allocate rows array");
return NULL;
}
const int row_limit = (max_rows > 0) ? max_rows : 1000;
int row_count = 0;
struct timespec start_time;
clock_gettime(CLOCK_MONOTONIC, &start_time);
while ((rc = sqlite3_step(stmt)) == SQLITE_ROW && row_count < row_limit) {
cJSON* row = cJSON_CreateArray();
if (!row) {
sqlite3_finalize(stmt);
cJSON_Delete(rows);
cJSON_Delete(response);
snprintf(error_message, error_size, "Failed to allocate row array");
return NULL;
}
for (int i = 0; i < col_count; i++) {
int col_type = sqlite3_column_type(stmt, i);
switch (col_type) {
case SQLITE_INTEGER:
cJSON_AddItemToArray(row, cJSON_CreateNumber((double)sqlite3_column_int64(stmt, i)));
break;
case SQLITE_FLOAT:
cJSON_AddItemToArray(row, cJSON_CreateNumber(sqlite3_column_double(stmt, i)));
break;
case SQLITE_TEXT: {
const char* text = (const char*)sqlite3_column_text(stmt, i);
cJSON_AddItemToArray(row, cJSON_CreateString(text ? text : ""));
break;
}
case SQLITE_BLOB: {
const void* blob = sqlite3_column_blob(stmt, i);
int blob_size = sqlite3_column_bytes(stmt, i);
if (blob && blob_size > 0) {
char* hex_str = malloc((size_t)blob_size * 2 + 1);
if (hex_str) {
for (int j = 0; j < blob_size; j++) {
sprintf(hex_str + j * 2, "%02x", ((const unsigned char*)blob)[j]);
}
hex_str[(size_t)blob_size * 2] = '\0';
cJSON_AddItemToArray(row, cJSON_CreateString(hex_str));
free(hex_str);
} else {
cJSON_AddItemToArray(row, cJSON_CreateString("[BLOB]"));
}
} else {
cJSON_AddItemToArray(row, cJSON_CreateString(""));
}
break;
}
case SQLITE_NULL:
cJSON_AddItemToArray(row, cJSON_CreateNull());
break;
default:
cJSON_AddItemToArray(row, cJSON_CreateString("[UNKNOWN]"));
break;
}
}
cJSON_AddItemToArray(rows, row);
row_count++;
struct timespec current_time;
clock_gettime(CLOCK_MONOTONIC, &current_time);
double elapsed = (current_time.tv_sec - start_time.tv_sec) +
(current_time.tv_nsec - start_time.tv_nsec) / 1e9;
if (elapsed > 4.5) {
break;
}
}
sqlite3_finalize(stmt);
if (rc != SQLITE_DONE && rc != SQLITE_ROW) {
const char* err_msg = sqlite3_errmsg(g_db);
snprintf(error_message, error_size, "SQL execution failed: %s", err_msg ? err_msg : "unknown");
cJSON_Delete(rows);
cJSON_Delete(response);
return NULL;
}
if (row_count >= row_limit) {
cJSON_AddStringToObject(response, "warning", "Result truncated to maximum row limit");
}
cJSON_AddNumberToObject(response, "row_count", row_count);
cJSON_AddNumberToObject(response, "execution_time_ms", 0);
cJSON_AddItemToObject(response, "rows", rows);
char* json_result = cJSON_Print(response);
cJSON_Delete(response);
if (!json_result) {
snprintf(error_message, error_size, "Failed to generate JSON response");
return NULL;
}
return json_result;
}
int db_get_total_event_count_ll(long long* out_count) {
if (!g_db || !out_count) return -1;
@@ -654,6 +866,47 @@ int db_store_config_event(const cJSON* event) {
return (rc == SQLITE_DONE) ? 0 : -1;
}
int db_insert_event_with_json(const char* id, const char* pubkey, long long created_at,
int kind, const char* event_type, const char* content,
const char* sig, const char* tags_json, const char* event_json,
int* out_step_rc, int* out_extended_errcode) {
if (!g_db || !id || !pubkey || !event_type || !content || !sig || !tags_json || !event_json) {
return -1;
}
const char* sql =
"INSERT INTO events (id, pubkey, created_at, kind, event_type, content, sig, tags, event_json) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)";
sqlite3_stmt* stmt = NULL;
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) {
return -1;
}
sqlite3_bind_text(stmt, 1, id, -1, SQLITE_TRANSIENT);
sqlite3_bind_text(stmt, 2, pubkey, -1, SQLITE_TRANSIENT);
sqlite3_bind_int64(stmt, 3, (sqlite3_int64)created_at);
sqlite3_bind_int(stmt, 4, kind);
sqlite3_bind_text(stmt, 5, event_type, -1, SQLITE_TRANSIENT);
sqlite3_bind_text(stmt, 6, content, -1, SQLITE_TRANSIENT);
sqlite3_bind_text(stmt, 7, sig, -1, SQLITE_TRANSIENT);
sqlite3_bind_text(stmt, 8, tags_json, -1, SQLITE_TRANSIENT);
sqlite3_bind_text(stmt, 9, event_json, -1, SQLITE_TRANSIENT);
int step_rc = sqlite3_step(stmt);
int extended_errcode = sqlite3_extended_errcode(g_db);
sqlite3_finalize(stmt);
if (out_step_rc) {
*out_step_rc = step_rc;
}
if (out_extended_errcode) {
*out_extended_errcode = extended_errcode;
}
return 0;
}
int db_get_event_time_bounds(long long* out_min_created_at, long long* out_max_created_at) {
if (!g_db || !out_min_created_at || !out_max_created_at) return -1;
@@ -676,6 +929,86 @@ int db_get_event_time_bounds(long long* out_min_created_at, long long* out_max_c
return 0;
}
int db_event_id_exists(const char* event_id, int* out_exists) {
if (!g_db || !event_id || !out_exists) return -1;
sqlite3_stmt* stmt = NULL;
const char* sql = "SELECT 1 FROM events WHERE id=? LIMIT 1";
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) return -1;
sqlite3_bind_text(stmt, 1, event_id, 64, SQLITE_STATIC);
int exists = (sqlite3_step(stmt) == SQLITE_ROW) ? 1 : 0;
sqlite3_finalize(stmt);
*out_exists = exists;
return 0;
}
cJSON* db_retrieve_event_by_id(const char* event_id) {
if (!g_db || !event_id) return NULL;
const char* sql =
"SELECT id, pubkey, created_at, kind, content, sig, tags FROM events WHERE id = ?";
sqlite3_stmt* stmt = NULL;
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) {
return NULL;
}
sqlite3_bind_text(stmt, 1, event_id, -1, SQLITE_STATIC);
cJSON* event = NULL;
if (sqlite3_step(stmt) == SQLITE_ROW) {
event = cJSON_CreateObject();
if (event) {
cJSON_AddStringToObject(event, "id", (const char*)sqlite3_column_text(stmt, 0));
cJSON_AddStringToObject(event, "pubkey", (const char*)sqlite3_column_text(stmt, 1));
cJSON_AddNumberToObject(event, "created_at", sqlite3_column_int64(stmt, 2));
cJSON_AddNumberToObject(event, "kind", sqlite3_column_int(stmt, 3));
cJSON_AddStringToObject(event, "content", (const char*)sqlite3_column_text(stmt, 4));
cJSON_AddStringToObject(event, "sig", (const char*)sqlite3_column_text(stmt, 5));
const char* tags_json = (const char*)sqlite3_column_text(stmt, 6);
if (tags_json) {
cJSON* tags = cJSON_Parse(tags_json);
if (tags) {
cJSON_AddItemToObject(event, "tags", tags);
} else {
cJSON_AddItemToObject(event, "tags", cJSON_CreateArray());
}
} else {
cJSON_AddItemToObject(event, "tags", cJSON_CreateArray());
}
}
}
sqlite3_finalize(stmt);
return event;
}
char* db_get_latest_event_pubkey_for_kind_dup(int kind) {
if (!g_db) return NULL;
const char* sql = "SELECT pubkey FROM events WHERE kind = ? ORDER BY created_at DESC LIMIT 1";
sqlite3_stmt* stmt = NULL;
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) {
return NULL;
}
sqlite3_bind_int(stmt, 1, kind);
char* pubkey_dup = NULL;
if (sqlite3_step(stmt) == SQLITE_ROW) {
const char* pubkey = (const char*)sqlite3_column_text(stmt, 0);
if (pubkey) {
pubkey_dup = strdup(pubkey);
}
}
sqlite3_finalize(stmt);
return pubkey_dup;
}
int db_get_config_row_count(int* out_count) {
if (!g_db || !out_count) return -1;
@@ -695,6 +1028,92 @@ int db_get_config_row_count(int* out_count) {
return 0;
}
int db_store_event_tags_cjson(const char* event_id, const cJSON* tags) {
if (!g_db || !event_id || !tags || !cJSON_IsArray(tags)) {
return 0; // Not an error if no tags
}
const char* sql = "INSERT INTO event_tags (event_id, tag_name, tag_value, tag_index) VALUES (?, ?, ?, ?)";
sqlite3_stmt* stmt = NULL;
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
if (rc != SQLITE_OK) {
DEBUG_ERROR("Failed to prepare event_tags insert: %s", sqlite3_errmsg(g_db));
return -1;
}
int tag_index = 0;
cJSON* tag = NULL;
cJSON_ArrayForEach(tag, tags) {
if (cJSON_IsArray(tag) && cJSON_GetArraySize(tag) >= 2) {
cJSON* name = cJSON_GetArrayItem(tag, 0);
cJSON* value = cJSON_GetArrayItem(tag, 1);
if (cJSON_IsString(name) && cJSON_IsString(value)) {
sqlite3_reset(stmt);
sqlite3_bind_text(stmt, 1, event_id, -1, SQLITE_STATIC);
sqlite3_bind_text(stmt, 2, cJSON_GetStringValue(name), -1, SQLITE_STATIC);
sqlite3_bind_text(stmt, 3, cJSON_GetStringValue(value), -1, SQLITE_STATIC);
sqlite3_bind_int(stmt, 4, tag_index);
rc = sqlite3_step(stmt);
if (rc != SQLITE_DONE) {
DEBUG_ERROR("Failed to insert event tag: %s", sqlite3_errmsg(g_db));
}
}
}
tag_index++;
}
sqlite3_finalize(stmt);
return 0;
}
int db_populate_event_tags_from_existing(void) {
if (!g_db) return -1;
sqlite3_stmt* check_stmt = NULL;
if (sqlite3_prepare_v2(g_db, "SELECT COUNT(*) FROM event_tags", -1, &check_stmt, NULL) != SQLITE_OK) {
return -1;
}
if (sqlite3_step(check_stmt) == SQLITE_ROW && sqlite3_column_int(check_stmt, 0) > 0) {
sqlite3_finalize(check_stmt);
DEBUG_INFO("event_tags already populated, skipping");
return 0;
}
sqlite3_finalize(check_stmt);
DEBUG_INFO("Populating event_tags from existing events...");
const char* sql = "SELECT id, tags FROM events WHERE tags != '[]'";
sqlite3_stmt* stmt = NULL;
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
if (rc != SQLITE_OK) return -1;
sqlite3_exec(g_db, "BEGIN TRANSACTION", NULL, NULL, NULL);
int event_count = 0;
while (sqlite3_step(stmt) == SQLITE_ROW) {
const char* event_id = (const char*)sqlite3_column_text(stmt, 0);
const char* tags_json = (const char*)sqlite3_column_text(stmt, 1);
if (event_id && tags_json) {
cJSON* tags = cJSON_Parse(tags_json);
if (tags) {
db_store_event_tags_cjson(event_id, tags);
cJSON_Delete(tags);
event_count++;
}
}
}
sqlite3_finalize(stmt);
sqlite3_exec(g_db, "COMMIT", NULL, NULL, NULL);
DEBUG_INFO("Populated event_tags for %d events", event_count);
return 0;
}
int db_add_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value) {
if (!g_db || !rule_type || !pattern_type || !pattern_value) return -1;
@@ -745,3 +1164,46 @@ int db_count_wot_whitelist_rules(void) {
}
return count;
}
int db_table_exists(const char* table_name, int* out_exists) {
if (!g_db || !table_name || !out_exists) return -1;
const char* sql = "SELECT 1 FROM sqlite_master WHERE type='table' AND name=? LIMIT 1";
sqlite3_stmt* stmt = NULL;
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) {
return -1;
}
sqlite3_bind_text(stmt, 1, table_name, -1, SQLITE_TRANSIENT);
*out_exists = (sqlite3_step(stmt) == SQLITE_ROW) ? 1 : 0;
sqlite3_finalize(stmt);
return 0;
}
char* db_get_schema_version_dup(void) {
if (!g_db) return NULL;
const char* sql = "SELECT value FROM schema_info WHERE key = 'version'";
sqlite3_stmt* stmt = NULL;
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) {
return NULL;
}
char* result = NULL;
if (sqlite3_step(stmt) == SQLITE_ROW) {
const char* version = (const char*)sqlite3_column_text(stmt, 0);
if (version) {
result = strdup(version);
}
}
sqlite3_finalize(stmt);
return result;
}
int db_exec_sql(const char* sql) {
if (!g_db || !sql) return -1;
int rc = sqlite3_exec(g_db, sql, NULL, NULL, NULL);
return (rc == SQLITE_OK) ? 0 : -1;
}
src/db_ops.h
+35 -1
View File
@@ -9,6 +9,21 @@
// Generic helpers
int db_is_available(void);
sqlite3* db_get_handle(void);
const char* db_last_error(void);
const char* db_get_database_path(void);
// Generic statement helpers (Phase 1 migration)
int db_prepare(const char* sql, sqlite3_stmt** out_stmt);
int db_bind_text_param(sqlite3_stmt* stmt, int index, const char* value);
int db_bind_int_param(sqlite3_stmt* stmt, int index, int value);
int db_bind_int64_param(sqlite3_stmt* stmt, int index, long long value);
int db_step_stmt(sqlite3_stmt* stmt);
int db_reset_stmt(sqlite3_stmt* stmt);
const char* db_column_text_value(sqlite3_stmt* stmt, int col);
int db_column_int_value(sqlite3_stmt* stmt, int col);
long long db_column_int64_value(sqlite3_stmt* stmt, int col);
double db_column_double_value(sqlite3_stmt* stmt, int col);
void db_finalize_stmt(sqlite3_stmt* stmt);
// Subscription logging
int db_log_subscription_created(const char* sub_id, const char* wsi_ptr,
@@ -32,6 +47,9 @@ int db_count_active_whitelist_rules(void);
// Generic prepared COUNT helper
int db_count_with_sql(const char* sql, const char** bind_params, int bind_param_count, int* out_count);
char* db_execute_readonly_query_json(const char* query, const char* request_id,
char* error_message, size_t error_size,
int max_rows, int timeout_ms);
// Monitoring/stat helpers (Phase 1 api.c migration)
int db_get_total_event_count_ll(long long* out_count);
@@ -51,16 +69,32 @@ int db_store_relay_private_key_hex(const char* relay_privkey_hex);
char* db_get_relay_private_key_hex_dup(void);
int db_store_config_event(const cJSON* event);
// Event timestamp helpers
// Event storage/timestamp/retrieval helpers
int db_insert_event_with_json(const char* id, const char* pubkey, long long created_at,
int kind, const char* event_type, const char* content,
const char* sig, const char* tags_json, const char* event_json,
int* out_step_rc, int* out_extended_errcode);
int db_get_event_time_bounds(long long* out_min_created_at, long long* out_max_created_at);
int db_event_id_exists(const char* event_id, int* out_exists);
cJSON* db_retrieve_event_by_id(const char* event_id);
char* db_get_latest_event_pubkey_for_kind_dup(int kind);
// Config table helpers
int db_get_config_row_count(int* out_count);
// Event tag helpers
int db_store_event_tags_cjson(const char* event_id, const cJSON* tags);
int db_populate_event_tags_from_existing(void);
// Auth/WoT rule helpers
int db_add_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
int db_remove_auth_rule(const char* rule_type, const char* pattern_type, const char* pattern_value);
int db_delete_wot_whitelist_rules(void);
int db_count_wot_whitelist_rules(void);
// Schema/DDL helpers
int db_table_exists(const char* table_name, int* out_exists);
char* db_get_schema_version_dup(void);
int db_exec_sql(const char* sql);
#endif // DB_OPS_H
src/main.c
+79 -268
View File
@@ -445,64 +445,41 @@ int init_database(const char* database_path_override) {
// DEBUG_GUARD_END
// Check if database is already initialized by looking for the events table
const char* check_sql = "SELECT name FROM sqlite_master WHERE type='table' AND name='events'";
sqlite3_stmt* check_stmt;
rc = sqlite3_prepare_v2(g_db, check_sql, -1, &check_stmt, NULL);
if (rc == SQLITE_OK) {
int has_events_table = (sqlite3_step(check_stmt) == SQLITE_ROW);
sqlite3_finalize(check_stmt);
int has_events_table = 0;
if (db_table_exists("events", &has_events_table) == 0) {
if (has_events_table) {
// Check existing schema version and migrate if needed
const char* version_sql = "SELECT value FROM schema_info WHERE key = 'version'";
sqlite3_stmt* version_stmt;
const char* db_version = NULL;
char* db_version = db_get_schema_version_dup();
int needs_migration = 0;
if (sqlite3_prepare_v2(g_db, version_sql, -1, &version_stmt, NULL) == SQLITE_OK) {
if (sqlite3_step(version_stmt) == SQLITE_ROW) {
db_version = (char*)sqlite3_column_text(version_stmt, 0);
// Check if migration is needed
if (!db_version || strcmp(db_version, "5") == 0) {
needs_migration = 1;
} else if (strcmp(db_version, "6") == 0) {
// Database is at schema version v6 (compatible)
} else if (strcmp(db_version, "7") == 0) {
// Database is at schema version v7 (compatible)
} else if (strcmp(db_version, "8") == 0) {
// Database is at schema version v8 (compatible)
} else if (strcmp(db_version, "9") == 0) {
// Database is at schema version v9 (compatible)
} else if (strcmp(db_version, "10") == 0) {
// Database is at schema version v10 (compatible)
} else if (strcmp(db_version, EMBEDDED_SCHEMA_VERSION) == 0) {
// Database is at current schema version
} else {
char warning_msg[256];
snprintf(warning_msg, sizeof(warning_msg), "Unknown database schema version: %s (expected %s)",
db_version, EMBEDDED_SCHEMA_VERSION);
DEBUG_WARN(warning_msg);
}
} else {
needs_migration = 1;
}
sqlite3_finalize(version_stmt);
} else {
// Check if migration is needed
if (!db_version || strcmp(db_version, "5") == 0) {
needs_migration = 1;
} else if (strcmp(db_version, "6") == 0) {
// Database is at schema version v6 (compatible)
} else if (strcmp(db_version, "7") == 0) {
// Database is at schema version v7 (compatible)
} else if (strcmp(db_version, "8") == 0) {
// Database is at schema version v8 (compatible)
} else if (strcmp(db_version, "9") == 0) {
// Database is at schema version v9 (compatible)
} else if (strcmp(db_version, "10") == 0) {
// Database is at schema version v10 (compatible)
} else if (strcmp(db_version, EMBEDDED_SCHEMA_VERSION) == 0) {
// Database is at current schema version
} else {
char warning_msg[256];
snprintf(warning_msg, sizeof(warning_msg), "Unknown database schema version: %s (expected %s)",
db_version, EMBEDDED_SCHEMA_VERSION);
DEBUG_WARN(warning_msg);
}
// Perform migration if needed
if (needs_migration) {
// Check if auth_rules table already exists
const char* check_auth_rules_sql = "SELECT name FROM sqlite_master WHERE type='table' AND name='auth_rules'";
sqlite3_stmt* check_stmt;
int has_auth_rules = 0;
if (sqlite3_prepare_v2(g_db, check_auth_rules_sql, -1, &check_stmt, NULL) == SQLITE_OK) {
has_auth_rules = (sqlite3_step(check_stmt) == SQLITE_ROW);
sqlite3_finalize(check_stmt);
}
(void)db_table_exists("auth_rules", &has_auth_rules);
if (!has_auth_rules) {
// Add auth_rules table matching sql_schema.h
@@ -519,14 +496,9 @@ int init_database(const char* database_path_override) {
" updated_at INTEGER NOT NULL DEFAULT (strftime('%s', 'now'))"
");";
char* error_msg = NULL;
int rc = sqlite3_exec(g_db, create_auth_rules_sql, NULL, NULL, &error_msg);
if (rc != SQLITE_OK) {
char error_log[512];
snprintf(error_log, sizeof(error_log), "Failed to create auth_rules table: %s",
error_msg ? error_msg : "unknown error");
DEBUG_ERROR(error_log);
if (error_msg) sqlite3_free(error_msg);
if (db_exec_sql(create_auth_rules_sql) != 0) {
DEBUG_ERROR("Failed to create auth_rules table");
if (db_version) free(db_version);
return -1;
}
@@ -536,14 +508,9 @@ int init_database(const char* database_path_override) {
"CREATE INDEX IF NOT EXISTS idx_auth_rules_type ON auth_rules(rule_type);"
"CREATE INDEX IF NOT EXISTS idx_auth_rules_active ON auth_rules(active);";
char* index_error_msg = NULL;
int index_rc = sqlite3_exec(g_db, create_auth_rules_indexes_sql, NULL, NULL, &index_error_msg);
if (index_rc != SQLITE_OK) {
char index_error_log[512];
snprintf(index_error_log, sizeof(index_error_log), "Failed to create auth_rules indexes: %s",
index_error_msg ? index_error_msg : "unknown error");
DEBUG_ERROR(index_error_log);
if (index_error_msg) sqlite3_free(index_error_msg);
if (db_exec_sql(create_auth_rules_indexes_sql) != 0) {
DEBUG_ERROR("Failed to create auth_rules indexes");
if (db_version) free(db_version);
return -1;
}
} else {
@@ -555,32 +522,24 @@ int init_database(const char* database_path_override) {
"INSERT OR REPLACE INTO schema_info (key, value, updated_at) "
"VALUES ('version', '6', strftime('%s', 'now'))";
char* error_msg = NULL;
int rc = sqlite3_exec(g_db, update_version_sql, NULL, NULL, &error_msg);
if (rc != SQLITE_OK) {
char error_log[512];
snprintf(error_log, sizeof(error_log), "Failed to update schema version: %s",
error_msg ? error_msg : "unknown error");
DEBUG_ERROR(error_log);
if (error_msg) sqlite3_free(error_msg);
if (db_exec_sql(update_version_sql) != 0) {
DEBUG_ERROR("Failed to update schema version");
if (db_version) free(db_version);
return -1;
}
if (db_version) {
free(db_version);
}
} else if (db_version) {
free(db_version);
}
} else {
// Initialize database schema using embedded SQL
// Execute the embedded schema SQL
char* error_msg = NULL;
rc = sqlite3_exec(g_db, EMBEDDED_SCHEMA_SQL, NULL, NULL, &error_msg);
if (rc != SQLITE_OK) {
char error_log[512];
snprintf(error_log, sizeof(error_log), "Failed to initialize database schema: %s",
error_msg ? error_msg : "unknown error");
DEBUG_ERROR(error_log);
if (error_msg) {
sqlite3_free(error_msg);
}
if (db_exec_sql(EMBEDDED_SCHEMA_SQL) != 0) {
DEBUG_ERROR("Failed to initialize database schema");
return -1;
}
@@ -591,14 +550,8 @@ int init_database(const char* database_path_override) {
}
// Enable WAL mode for better concurrency and crash recovery
char* wal_error = NULL;
rc = sqlite3_exec(g_db, "PRAGMA journal_mode=WAL;", NULL, NULL, &wal_error);
if (rc != SQLITE_OK) {
char error_msg[256];
snprintf(error_msg, sizeof(error_msg), "Failed to enable WAL mode: %s",
wal_error ? wal_error : "unknown error");
DEBUG_WARN(error_msg);
if (wal_error) sqlite3_free(wal_error);
if (db_exec_sql("PRAGMA journal_mode=WAL;") != 0) {
DEBUG_WARN("Failed to enable WAL mode");
// Continue anyway - WAL mode is optional
} else {
DEBUG_LOG("SQLite WAL mode enabled");
@@ -611,11 +564,8 @@ int init_database(const char* database_path_override) {
if (mmap_size > 0) {
char mmap_pragma[64];
snprintf(mmap_pragma, sizeof(mmap_pragma), "PRAGMA mmap_size=%ld;", mmap_size);
char* mmap_error = NULL;
rc = sqlite3_exec(g_db, mmap_pragma, NULL, NULL, &mmap_error);
if (rc != SQLITE_OK) {
DEBUG_WARN("Failed to set mmap_size: %s", mmap_error ? mmap_error : "unknown");
if (mmap_error) sqlite3_free(mmap_error);
if (db_exec_sql(mmap_pragma) != 0) {
DEBUG_WARN("Failed to set mmap_size");
} else {
DEBUG_LOG("SQLite mmap_size set to %ld bytes", mmap_size);
}
@@ -628,11 +578,8 @@ int init_database(const char* database_path_override) {
char cache_pragma[64];
// Use negative value so SQLite interprets it as KB rather than page count
snprintf(cache_pragma, sizeof(cache_pragma), "PRAGMA cache_size=-%d;", cache_size_kb > 0 ? cache_size_kb : -cache_size_kb);
char* cache_error = NULL;
rc = sqlite3_exec(g_db, cache_pragma, NULL, NULL, &cache_error);
if (rc != SQLITE_OK) {
DEBUG_WARN("Failed to set cache_size: %s", cache_error ? cache_error : "unknown");
if (cache_error) sqlite3_free(cache_error);
if (db_exec_sql(cache_pragma) != 0) {
DEBUG_WARN("Failed to set cache_size");
} else {
DEBUG_LOG("SQLite cache_size set to %d KB", cache_size_kb);
}
@@ -649,14 +596,8 @@ void close_database() {
if (g_db) {
// Perform WAL checkpoint to minimize stale files on next startup
DEBUG_LOG("Performing WAL checkpoint before database close");
char* checkpoint_error = NULL;
int rc = sqlite3_exec(g_db, "PRAGMA wal_checkpoint(TRUNCATE);", NULL, NULL, &checkpoint_error);
if (rc != SQLITE_OK) {
char error_msg[256];
snprintf(error_msg, sizeof(error_msg), "WAL checkpoint warning: %s",
checkpoint_error ? checkpoint_error : "unknown error");
DEBUG_WARN(error_msg);
if (checkpoint_error) sqlite3_free(checkpoint_error);
if (db_exec_sql("PRAGMA wal_checkpoint(TRUNCATE);") != 0) {
DEBUG_WARN("WAL checkpoint warning");
}
sqlite3_close(g_db);
@@ -731,43 +672,7 @@ const char* extract_d_tag_value(cJSON* tags) {
// Insert denormalized tags into event_tags table for fast indexed lookups
int store_event_tags(const char* event_id, cJSON* tags) {
if (!g_db || !event_id || !tags || !cJSON_IsArray(tags)) {
return 0; // Not an error if no tags
}
const char* sql = "INSERT INTO event_tags (event_id, tag_name, tag_value, tag_index) VALUES (?, ?, ?, ?)";
sqlite3_stmt* stmt;
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
if (rc != SQLITE_OK) {
DEBUG_ERROR("Failed to prepare event_tags insert: %s", sqlite3_errmsg(g_db));
return -1;
}
int tag_index = 0;
cJSON* tag = NULL;
cJSON_ArrayForEach(tag, tags) {
if (cJSON_IsArray(tag) && cJSON_GetArraySize(tag) >= 2) {
cJSON* name = cJSON_GetArrayItem(tag, 0);
cJSON* value = cJSON_GetArrayItem(tag, 1);
if (cJSON_IsString(name) && cJSON_IsString(value)) {
sqlite3_reset(stmt);
sqlite3_bind_text(stmt, 1, event_id, -1, SQLITE_STATIC);
sqlite3_bind_text(stmt, 2, cJSON_GetStringValue(name), -1, SQLITE_STATIC);
sqlite3_bind_text(stmt, 3, cJSON_GetStringValue(value), -1, SQLITE_STATIC);
sqlite3_bind_int(stmt, 4, tag_index);
rc = sqlite3_step(stmt);
if (rc != SQLITE_DONE) {
DEBUG_ERROR("Failed to insert event tag: %s", sqlite3_errmsg(g_db));
}
}
}
tag_index++;
}
sqlite3_finalize(stmt);
return 0;
return db_store_event_tags_cjson(event_id, tags);
}
// Store event in database
@@ -825,40 +730,31 @@ int store_event(cJSON* event) {
return -1;
}
// Prepare SQL statement for event insertion
const char* sql =
"INSERT INTO events (id, pubkey, created_at, kind, event_type, content, sig, tags, event_json) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)";
sqlite3_stmt* stmt;
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
if (rc != SQLITE_OK) {
DEBUG_ERROR("Failed to prepare event insert statement");
int rc = SQLITE_ERROR;
int extended_errcode = 0;
if (db_insert_event_with_json(cJSON_GetStringValue(id),
cJSON_GetStringValue(pubkey),
(long long)cJSON_GetNumberValue(created_at),
(int)cJSON_GetNumberValue(kind),
event_type_to_string(type),
cJSON_GetStringValue(content),
cJSON_GetStringValue(sig),
tags_json,
event_json,
&rc,
&extended_errcode) != 0) {
DEBUG_ERROR("Failed to execute event insert operation");
free(tags_json);
free(event_json);
return -1;
}
// Bind parameters
sqlite3_bind_text(stmt, 1, cJSON_GetStringValue(id), -1, SQLITE_STATIC);
sqlite3_bind_text(stmt, 2, cJSON_GetStringValue(pubkey), -1, SQLITE_STATIC);
sqlite3_bind_int64(stmt, 3, (sqlite3_int64)cJSON_GetNumberValue(created_at));
sqlite3_bind_int(stmt, 4, (int)cJSON_GetNumberValue(kind));
sqlite3_bind_text(stmt, 5, event_type_to_string(type), -1, SQLITE_STATIC);
sqlite3_bind_text(stmt, 6, cJSON_GetStringValue(content), -1, SQLITE_STATIC);
sqlite3_bind_text(stmt, 7, cJSON_GetStringValue(sig), -1, SQLITE_STATIC);
sqlite3_bind_text(stmt, 8, tags_json, -1, SQLITE_TRANSIENT);
sqlite3_bind_text(stmt, 9, event_json, -1, SQLITE_TRANSIENT);
// Execute statement
rc = sqlite3_step(stmt);
if (rc != SQLITE_DONE) {
const char* err_msg = sqlite3_errmsg(g_db);
int extended_errcode = sqlite3_extended_errcode(g_db);
const char* err_msg = db_last_error();
if (rc != SQLITE_CONSTRAINT) {
DEBUG_ERROR("INSERT failed: rc=%d, extended_errcode=%d, msg=%s", rc, extended_errcode, err_msg);
}
}
sqlite3_finalize(stmt);
if (rc != SQLITE_DONE) {
if (rc == SQLITE_CONSTRAINT) {
@@ -888,7 +784,7 @@ int store_event(cJSON* event) {
return 0; // Not an error, just duplicate
}
char error_msg[256];
snprintf(error_msg, sizeof(error_msg), "Failed to insert event: %s", sqlite3_errmsg(g_db));
snprintf(error_msg, sizeof(error_msg), "Failed to insert event: %s", db_last_error());
DEBUG_ERROR(error_msg);
free(tags_json);
free(event_json);
@@ -929,60 +825,16 @@ int store_event(cJSON* event) {
int event_id_exists_in_db(const char* event_id) {
if (!g_db || !event_id || strlen(event_id) != 64) return 0;
sqlite3_stmt* stmt;
const char* sql = "SELECT 1 FROM events WHERE id=? LIMIT 1";
if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) != SQLITE_OK) return 0;
sqlite3_bind_text(stmt, 1, event_id, 64, SQLITE_STATIC);
int exists = (sqlite3_step(stmt) == SQLITE_ROW) ? 1 : 0;
sqlite3_finalize(stmt);
int exists = 0;
if (db_event_id_exists(event_id, &exists) != 0) {
return 0;
}
return exists;
}
// Populate event_tags from existing events (run once at startup)
int populate_event_tags_from_existing(void) {
if (!g_db) return -1;
// Check if event_tags is already populated
sqlite3_stmt* check_stmt;
sqlite3_prepare_v2(g_db, "SELECT COUNT(*) FROM event_tags", -1, &check_stmt, NULL);
if (sqlite3_step(check_stmt) == SQLITE_ROW && sqlite3_column_int(check_stmt, 0) > 0) {
sqlite3_finalize(check_stmt);
DEBUG_INFO("event_tags already populated, skipping");
return 0;
}
sqlite3_finalize(check_stmt);
DEBUG_INFO("Populating event_tags from existing events...");
const char* sql = "SELECT id, tags FROM events WHERE tags != '[]'";
sqlite3_stmt* stmt;
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
if (rc != SQLITE_OK) return -1;
// Use a transaction for bulk insert performance
sqlite3_exec(g_db, "BEGIN TRANSACTION", NULL, NULL, NULL);
int event_count = 0;
while (sqlite3_step(stmt) == SQLITE_ROW) {
const char* event_id = (const char*)sqlite3_column_text(stmt, 0);
const char* tags_json = (const char*)sqlite3_column_text(stmt, 1);
if (event_id && tags_json) {
cJSON* tags = cJSON_Parse(tags_json);
if (tags) {
store_event_tags(event_id, tags);
cJSON_Delete(tags);
event_count++;
}
}
}
sqlite3_finalize(stmt);
sqlite3_exec(g_db, "COMMIT", NULL, NULL, NULL);
DEBUG_INFO("Populated event_tags for %d events", event_count);
return 0;
return db_populate_event_tags_from_existing();
}
/////////////////////////////////////////////////////////////////////////////////////////
@@ -992,48 +844,7 @@ int populate_event_tags_from_existing(void) {
/////////////////////////////////////////////////////////////////////////////////////////
cJSON* retrieve_event(const char* event_id) {
if (!g_db || !event_id) {
return NULL;
}
const char* sql =
"SELECT id, pubkey, created_at, kind, content, sig, tags FROM events WHERE id = ?";
sqlite3_stmt* stmt;
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
if (rc != SQLITE_OK) {
return NULL;
}
sqlite3_bind_text(stmt, 1, event_id, -1, SQLITE_STATIC);
cJSON* event = NULL;
if (sqlite3_step(stmt) == SQLITE_ROW) {
event = cJSON_CreateObject();
cJSON_AddStringToObject(event, "id", (char*)sqlite3_column_text(stmt, 0));
cJSON_AddStringToObject(event, "pubkey", (char*)sqlite3_column_text(stmt, 1));
cJSON_AddNumberToObject(event, "created_at", sqlite3_column_int64(stmt, 2));
cJSON_AddNumberToObject(event, "kind", sqlite3_column_int(stmt, 3));
cJSON_AddStringToObject(event, "content", (char*)sqlite3_column_text(stmt, 4));
cJSON_AddStringToObject(event, "sig", (char*)sqlite3_column_text(stmt, 5));
// Parse tags JSON
const char* tags_json = (char*)sqlite3_column_text(stmt, 6);
if (tags_json) {
cJSON* tags = cJSON_Parse(tags_json);
if (tags) {
cJSON_AddItemToObject(event, "tags", tags);
} else {
cJSON_AddItemToObject(event, "tags", cJSON_CreateArray());
}
} else {
cJSON_AddItemToObject(event, "tags", cJSON_CreateArray());
}
}
sqlite3_finalize(stmt);
return event;
return db_retrieve_event_by_id(event_id);
}
@@ -1549,10 +1360,10 @@ int handle_req_message(const char* sub_id, cJSON* filters, struct lws *wsi, stru
// Execute query and send events
sqlite3_stmt* stmt;
int rc = sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL);
int rc = db_prepare(sql, &stmt);
if (rc != SQLITE_OK) {
char error_msg[256];
snprintf(error_msg, sizeof(error_msg), "Failed to prepare subscription query: %s", sqlite3_errmsg(g_db));
snprintf(error_msg, sizeof(error_msg), "Failed to prepare subscription query: %s", db_last_error());
DEBUG_ERROR(error_msg);
// Log the failed query so we can see what SQL was generated
@@ -1565,7 +1376,7 @@ int handle_req_message(const char* sub_id, cJSON* filters, struct lws *wsi, stru
timestamp,
sub_id,
pss ? pss->client_ip : "N/A",
sqlite3_errmsg(g_db),
db_last_error(),
sql);
fflush(stderr);
}
@@ -1580,7 +1391,7 @@ int handle_req_message(const char* sub_id, cJSON* filters, struct lws *wsi, stru
// Bind parameters
for (int i = 0; i < bind_param_count; i++) {
sqlite3_bind_text(stmt, i + 1, bind_params[i], -1, SQLITE_TRANSIENT);
db_bind_text_param(stmt, i + 1, bind_params[i]);
}
// Cache config values outside the row loop (performance fix)
@@ -1588,7 +1399,7 @@ int handle_req_message(const char* sub_id, cJSON* filters, struct lws *wsi, stru
int filter_responses = get_config_bool("expiration_filter", 1);
int row_count = 0;
while (sqlite3_step(stmt) == SQLITE_ROW) {
while (db_step_stmt(stmt) == SQLITE_ROW) {
row_count++;
// Track rows returned for abuse detection
@@ -1597,7 +1408,7 @@ int handle_req_message(const char* sub_id, cJSON* filters, struct lws *wsi, stru
}
// Get pre-serialized event JSON (no reconstruction needed!)
const char* event_json_str = (char*)sqlite3_column_text(stmt, 0);
const char* event_json_str = db_column_text_value(stmt, 0);
if (!event_json_str) {
DEBUG_ERROR("Event has NULL event_json field");
continue;
@@ -1645,7 +1456,7 @@ int handle_req_message(const char* sub_id, cJSON* filters, struct lws *wsi, stru
events_sent++;
}
sqlite3_finalize(stmt);
db_finalize_stmt(stmt);
// Stop query timing and log
clock_gettime(CLOCK_MONOTONIC, &query_end);
src/main.h
+2 -2
View File
@@ -13,8 +13,8 @@
// Using CRELAY_ prefix to avoid conflicts with nostr_core_lib VERSION macros
#define CRELAY_VERSION_MAJOR 2
#define CRELAY_VERSION_MINOR 0
#define CRELAY_VERSION_PATCH 2
#define CRELAY_VERSION "v2.0.2"
#define CRELAY_VERSION_PATCH 3
#define CRELAY_VERSION "v2.0.3"
// Relay metadata (authoritative source for NIP-11 information)
#define RELAY_NAME "C-Relay"
src/websockets.c
-1
View File
@@ -10,7 +10,6 @@
#include <signal.h>
#include <time.h>
#include <pthread.h>
#include <sqlite3.h>
// Include libwebsockets after pthread.h to ensure pthread_rwlock_t is defined
#include <libwebsockets.h>