sovereign_browser/plans/sqlite-migration.md

Plan: Migrate All Storage to SQLite + Remove Key Persistence

Overview

Consolidate all browser data storage into the SQLite database (~/.sovereign_browser/browser.db) and remove all key persistence — private keys live only in RAM and are gone when the browser quits.

Current state

File	Format	Stores	Action
identity.json	JSON	Nostr private key, mnemonic, bunker URL	Delete the code + file — keys must never touch disk
history.txt	Plain text	Recent URLs (max 50)	Migrate to SQLite history table
session.txt	Plain text	Open tab URLs	Migrate to SQLite session table
settings.conf	key=value	Browser preferences	Migrate to SQLite key_value table (already exists)
browser.db	SQLite	Nostr events, tags, key_value	Keep + extend

Phase 1: Remove key persistence

src/key_store.h / src/key_store.c

Remove:

• key_store_path() — no file path needed
• key_store_save() — never called (dead code)
• key_store_load() — never called (dead code)
• key_store_clear() — replace with a no-op or remove the calls

Keep:

• key_store_identity_t struct (in-memory only)
• key_store_method_t enum
• key_store_create_signer() — creates a signer from an in-memory identity

src/cli.h / src/cli.c

Remove:

• --no-save-identity flag and no_save_identity field (meaningless now)
• The TODO comment about key_store_save()

src/main.c / src/agent_login.c

Remove:

• key_store_clear() calls in app_menu_logout_proxy() and agent_login.c logout

Defensive cleanup

On startup, delete ~/.sovereign_browser/identity.json if it exists (in case a previous version created it).

Phase 2: Migrate history to SQLite

src/db.h / src/db.c

Add a history table to the schema:

CREATE TABLE IF NOT EXISTS history (
    id          INTEGER PRIMARY KEY AUTOINCREMENT,
    url         TEXT NOT NULL UNIQUE,
    title       TEXT,
    visited_at INTEGER NOT NULL,
    visit_count INTEGER DEFAULT 1
);
CREATE INDEX IF NOT EXISTS idx_history_visited_at ON history(visited_at DESC);

Add functions:

int  db_history_add(const char *url, const char *title);
/* Returns most-recent-first. Fills urls_out (caller frees each + array). */
char **db_history_get(int *count_out, int limit);
int  db_history_clear(void);

db_history_add does an UPSERT: on conflict (URL already exists), increment visit_count and update visited_at.

src/history.h / src/history.c

Rewrite to use db_history_add / db_history_get / db_history_clear. Remove the flat-file history_path(), fopen(), the g_history[50][2048] array, and history_load(). The history_add() function now takes an optional title parameter (or we keep the existing signature and pass NULL for title from on_load_changed).

Remove HISTORY_MAX_ENTRIES (no cap — SQLite handles it).

Phase 3: Migrate session to SQLite

src/db.h / src/db.c

Add a session table:

CREATE TABLE IF NOT EXISTS session (
    tab_index INTEGER PRIMARY KEY,
    url       TEXT NOT NULL,
    title     TEXT
);

Add functions:

int  db_session_save(const char **urls, const char **titles, int count);
int  db_session_load(char ***urls_out, char ***titles_out, int *count_out);
int  db_session_clear(void);

db_session_save clears the table then inserts all current tabs. db_session_load reads them back in tab_index order.

src/session.h / src/session.c

Rewrite session_save() and session_restore() to use the SQLite functions. Remove session_path() and fopen().

Phase 4: Migrate settings to SQLite

src/settings.h / src/settings.c

Rewrite settings_load() and settings_save() to use db_kv_get / db_kv_set for each field. The key_value table already exists in the schema.

Each setting is stored as a key-value pair:

• restore_session → "true" / "false"
• new_tab_url → the URL string
• tab_bar_position → "top" / "bottom" / etc.
• bootstrap_relays → newline-separated URLs
• etc.

Important: settings_load() must be called after db_init() (the DB must be open first). This changes the startup order in main.c:

settings_load()  →  db_init()  →  settings_load()  (revised order)

Actually: db_init() first, then settings_load() reads from the DB.

Remove settings_path() and fopen() from settings.c.

Phase 5: Cleanup

• Delete ~/.sovereign_browser/identity.json on startup (defensive)
• Delete ~/.sovereign_browser/history.txt on startup (one-time migration)
• Delete ~/.sovereign_browser/session.txt on startup (one-time migration)
• Delete ~/.sovereign_browser/settings.conf on startup (one-time migration)
• Or: leave the old files in place (they're just ignored) — simpler, less destructive

Startup order change in main.c

Current:

settings_load();
history_load();
db_init();

New:

db_init();           /* open the database first */
settings_load();     /* reads from key_value table */
/* history_load() removed — history is queried from SQLite on demand */

File change summary

File	Change
src/key_store.h	Remove key_store_save, key_store_load, key_store_clear, key_store_path
src/key_store.c	Remove file I/O functions; keep only key_store_create_signer
src/db.h	Add db_history_*, db_session_* functions
src/db.c	Add history + session tables to schema; implement new functions
src/history.h	Update API (remove history_load, add title param)
src/history.c	Rewrite to use SQLite; remove flat-file code
src/session.h	No API change
src/session.c	Rewrite to use SQLite; remove flat-file code
src/settings.h	No API change
src/settings.c	Rewrite to use db_kv_get/db_kv_set; remove flat-file code
src/main.c	Reorder startup (db_init before settings_load); remove key_store_clear call; delete old files defensively
src/agent_login.c	Remove key_store_clear call
src/cli.h	Remove no_save_identity field
src/cli.c	Remove --no-save-identity flag