sovereign_browser/plans/embedded-agent.md

# Embedded Agent — In-Browser LLM with First-Class MCP Access

## Concurrency model — long-running tasks

The agent loop runs on a **background GThread**, not the GTK main thread. This
is critical for long-running tasks like "follow each link, download images,
write a summary for each" which may involve dozens of LLM calls and hundreds
of tool calls.

```mermaid
sequenceDiagram
    participant UI as Chat UI main thread
    participant Chat as agent_chat.c
    participant Thread as Agent Loop background thread
    participant LLM as LLM API
    participant Main as GTK main loop
    participant Tools as agent_tools_dispatch

    UI->>Chat: POST sovereign://agents/send?text=...
    Chat->>Thread: g_thread_new agent_loop_run
    Chat-->>UI: 200 OK session started
    loop ReAct iterations
        Thread->>LLM: POST chat/completions blocking
        LLM-->>Thread: response + tool_calls
        Thread->>Thread: persist assistant message
        alt has tool_calls
            loop each tool call
                Thread->>Main: g_idle_add dispatch_tool
                Main->>Tools: agent_tools_dispatch sync JS eval
                Tools-->>Main: result
                Main-->>Thread: result via GAsyncQueue
                Thread->>Thread: persist tool result
            end
        else no tool_calls
            Thread->>Thread: mark session complete
        end
    end
    Thread->>Main: g_idle_add update_ui final
```

### Threading rules

1. **LLM HTTP calls** happen on the background thread — `soup_session_send`
   blocks in-thread, the GTK UI stays responsive.

2. **Browser tool dispatch** (snapshot, click, eval, etc.) MUST run on the
   GTK main thread because WebKitGTK is not thread-safe. The background
   thread schedules each tool call via `g_idle_add()` and waits for the
   result on a `GAsyncQueue` or a per-call `GMainLoop` (same pattern the
   MCP HTTP handler uses for sync JS eval).

3. **Filesystem + shell tools** run directly on the background thread — they
   don't touch GTK or WebKit, so no main-thread hop needed. This keeps
   long shell commands from blocking the UI.

4. **SQLite writes** use the existing `SQLITE_OPEN_FULLMUTEX` connection
   (thread-safe). The background thread can call `db_kv_set` /
   `agent_chat_store_*` directly.

5. **UI updates** happen via `g_idle_add()` — the background thread pushes
   status updates (current tool, iteration count, partial output) to a
   shared struct, and an idle callback renders them in the chat page.

6. **Cancellation** — a `g_atomic_int` cancel flag is checked at the top of
   each loop iteration. The chat UI's "Stop" button sets it. The background
   thread exits cleanly at the next check point.

### Iteration cap

The default `max_iterations` is **100** (configurable on
`sovereign://agents`), not 20. Long tasks like "follow 30 links" need
multiple tool calls per link (open, snapshot, extract, download, write) —
easily 100+ calls. The cap is a safety valve, not a tight limit. The user
can raise it in settings for very long batch jobs.

### Status polling

The chat UI polls `sovereign://agents/status?session=...` every 500ms while
a session is active. The response includes:
- `state`: `idle` | `thinking` | `tool_call` | `complete` | `error` | `cancelled`
- `iteration`: current iteration number
- `current_tool`: name of the tool being executed (if any)
- `last_message`: most recent assistant text (for progressive display)
- `error`: error message if state is `error`

This gives the user live visibility into long-running tasks without
streaming complexity.

## Goal

Embed an LLM-powered agent directly inside sovereign_browser. The user types
`; <message>` in the URL bar to talk to the agent. The agent has first-class
access to the browser's own MCP tool set (snapshot, click, eval, tabs, etc.)
plus full filesystem and shell access (the browser runs in a dedicated Qubes
qube, so arbitrary command execution is acceptable and intended).

Provider config (base URL, API key, model name) is managed on a new
`sovereign://agents` internal page. Chat history is persisted per-session in
SQLite so it can be fed back as context on follow-up messages.

## User-facing flow

```mermaid
flowchart LR
    User[User types in URL bar] --> Check{Starts with semicolon?}
    Check -- yes --> Agent[Embedded Agent Chat UI]
    Check -- no --> Nav[Normal navigation]
    Agent --> LLM[OpenAI-compatible API call]
    LLM --> ToolLoop{Tool calls in response?}
    ToolLoop -- yes --> Dispatch[agent_tools_dispatch via internal MCP loopback]
    Dispatch --> ToolLoop
    ToolLoop -- no --> Render[Render assistant message in chat UI]
    Render --> User
```

## Architecture

```mermaid
flowchart TB
    subgraph URLBar
        Entry[GtkEntry on_url_activate]
    end
    subgraph AgentModule
        Router[agent_chat_route_input]
        Client[agent_llm.c — HTTP client]
        Loop[agent_loop.c — tool-call loop]
        Store[agent_chat_store.c — SQLite persistence]
    end
    subgraph BrowserCore
        Tools[agent_tools_dispatch]
        FsTools[agent_fs_tools.c — fs + shell tools]
        Bridge[nostr_bridge.c — sovereign:// scheme]
    end
    subgraph External
        API[OpenAI-compatible API]
    end
    subgraph UI
        ChatPage[sovereign://agents/chat — HTML/JS chat UI]
        ConfigPage[sovereign://agents — provider config]
    end

    Entry --> Router
    Router -->|chat message| Client
    Client -->|HTTPS POST| API
    API -->|JSON response| Client
    Client --> Loop
    Loop -->|tool call| Tools
    Loop -->|fs/shell call| FsTools
    Tools --> BrowserCore
    Loop --> Store
    Store --> ChatPage
    Bridge --> ChatPage
    Bridge --> ConfigPage
```

### Key design decisions

1. **Reuse `agent_tools_dispatch()`** — The embedded agent calls the exact same
   C dispatch function the external MCP server uses. No parallel tool
   implementation. The agent passes `conn = NULL` so async JS tools use the
   sync `agent_js_eval_sync()` path (same as the MCP HTTP handler).

2. **New system tools (`agent_fs_tools.c`)** — `fs_read`, `fs_write`,
   `fs_list`, `fs_mkdir`, `fs_delete`, `shell_exec`. These are registered
   alongside the existing browser tools so both the embedded agent and
   external MCP clients can use them. Full shell access — the Qubes qube
   provides the sandbox.

3. **OpenAI-compatible HTTP client (`agent_llm.c`)** — Uses libsoup-3.0
   (already linked) to POST to `{base_url}/chat/completions` with the
   `tools` array built from the same `tool_defs[]` catalog in
   [`agent_mcp.c`](src/agent_mcp.c:130). Streams or polls; parses tool calls
   from the response. One client covers OpenAI, OpenRouter, Ollama, LM Studio,
   Groq, etc. via base URL + key.

4. **Tool-call loop (`agent_loop.c`)** — Standard ReAct loop, runs on a
   background `GThread` (see Concurrency model above):
   - Build messages array (system prompt + persisted history + new user msg).
   - Call LLM (blocking HTTP on the background thread).
   - If response contains `tool_calls`, dispatch each:
     - Browser tools → hop to GTK main thread via `g_idle_add()` + wait on
       `GAsyncQueue` (WebKitGTK is not thread-safe).
     - Filesystem/shell tools → run directly on the background thread.
   - Append tool results to messages, call LLM again.
   - Repeat until no more tool calls → render final assistant text.
   - Cap at N iterations (default 100, configurable) to prevent infinite
     loops. Check cancel flag at the top of each iteration.

5. **Chat persistence (`agent_chat_store.c`)** — New SQLite tables
   (`agent_sessions`, `agent_messages`) in the per-user `browser.db`. Each
   session has an id, title, created_at. Messages store role
   (user/assistant/tool), content, and tool-call JSON. The URL-bar `;` command
   reuses the most recent session (or starts a new one if none exists).

6. **`sovereign://agents` config page** — Rendered by `nostr_bridge.c` (same
   pattern as `sovereign://settings`). Fields: provider name, base URL, API
   key, model name, system prompt (optional). Saved via
   `sovereign://agents/set?key=...&value=...` to the `key_value` table
   (existing `db_kv_set`). API key stored in the key_value table; since this
   is a dedicated qube, that's acceptable.

7. **`sovereign://agents/chat` chat UI** — A lightweight HTML/JS page
   (rendered by `nostr_bridge.c`) that:
   - Shows the message history (loaded from SQLite via a
     `sovereign://agents/messages?session=...` endpoint).
   - Has an input box for follow-up messages (POSTed to
     `sovereign://agents/send?session=...&text=...`).
   - Polls `sovereign://agents/status?session=...` for in-progress tool calls
     and streams the assistant's final response.
   - The URL-bar `;` shortcut navigates here with the message pre-filled and
     auto-submits.

8. **URL-bar `;` routing** — In [`on_url_activate`](src/tab_manager.c:913),
   check if `text[0] == ';'`. If so, extract the message (`text + 1`, trimmed)
   and call `agent_chat_route_input(message)` instead of `normalize_url()`.
   The function opens `sovereign://agents/chat` in the active tab (or a new
   tab if the active tab isn't already the chat page) and kicks off the
   agent loop. If the message is empty (`;` alone), just open the chat page
   without sending.

9. **System prompt** — A default system prompt explains the agent's
   capabilities: it controls a web browser via MCP tools, has filesystem and
   shell access, and should use the snapshot+ref pattern for page
   interaction. The user can override this on `sovereign://agents`.

## New files

| File | Purpose |
|------|---------|
| `src/agent_llm.h` / `src/agent_llm.c` | OpenAI-compatible HTTP client (libsoup). Sends chat-completions request with tools, parses response + tool_calls. |
| `src/agent_loop.h` / `src/agent_loop.c` | ReAct tool-call loop. Orchestrates LLM calls ↔ tool dispatch. |
| `src/agent_chat_store.h` / `src/agent_chat_store.c` | SQLite persistence for chat sessions and messages. |
| `src/agent_fs_tools.h` / `src/agent_fs_tools.c` | Filesystem + shell tools (fs_read, fs_write, fs_list, fs_mkdir, fs_delete, shell_exec). |
| `src/agent_chat.h` / `src/agent_chat.c` | High-level entry point: `agent_chat_route_input()`, session management, bridges URL-bar → loop → UI. |

## Modified files

| File | Change |
|------|--------|
| [`src/tab_manager.c`](src/tab_manager.c:913) | `on_url_activate`: detect `;` prefix, route to `agent_chat_route_input()`. |
| [`src/nostr_bridge.c`](src/nostr_bridge.c:1723) | Add `sovereign://agents`, `sovereign://agents/chat`, `sovereign://agents/set`, `sovereign://agents/messages`, `sovereign://agents/send`, `sovereign://agents/status` route handlers. |
| [`src/agent_mcp.c`](src/agent_mcp.c:130) | Export `tool_defs[]` / `build_tools_list()` (or move to a shared header) so `agent_llm.c` can build the OpenAI tools array from the same catalog. Add fs/shell tool defs. |
| [`src/agent_tools.c`](src/agent_tools.c) | Dispatch fs/shell tools (or route them via a new `agent_fs_tools_dispatch()` called from the same dispatcher). |
| [`src/db.c`](src/db.c) / [`src/db.h`](src/db.h) | Add `agent_sessions` + `agent_messages` tables and CRUD functions. |
| [`src/settings.h`](src/settings.h) / [`src/settings.c`](src/settings.c) | Add agent provider settings fields (base_url, api_key, model, system_prompt). |
| [`Makefile`](Makefile) | Add new `.c` files to `SRC`. |

## SQLite schema

```sql
CREATE TABLE IF NOT EXISTS agent_sessions (
    id          TEXT PRIMARY KEY,   -- UUID
    title       TEXT,
    created_at  INTEGER,
    updated_at  INTEGER
);

CREATE TABLE IF NOT EXISTS agent_messages (
    id          INTEGER PRIMARY KEY AUTOINCREMENT,
    session_id  TEXT,
    role        TEXT,               -- 'user' | 'assistant' | 'tool' | 'system'
    content     TEXT,               -- message text (or tool result JSON)
    tool_calls  TEXT,               -- JSON array of tool calls (assistant msgs)
    tool_call_id TEXT,              -- for role='tool': which call this answers
    created_at  INTEGER,
    FOREIGN KEY (session_id) REFERENCES agent_sessions(id)
);
CREATE INDEX IF NOT EXISTS idx_agent_messages_session
    ON agent_messages(session_id, created_at);
```

## Tool catalog additions (fs + shell)

| Tool | Description |
|------|-------------|
| `fs_read` | Read a file's contents (text). Params: `path`. |
| `fs_write` | Write text to a file (overwrite). Params: `path`, `content`. |
| `fs_list` | List directory entries. Params: `path`. |
| `fs_mkdir` | Create a directory (recursive). Params: `path`. |
| `fs_delete` | Delete a file or empty directory. Params: `path`. |
| `shell_exec` | Run a shell command, return stdout+stderr+exit code. Params: `command`, `timeout_ms` (default 30000). |

## OpenAI tools array format

The `agent_llm.c` client builds the `tools` field from the shared
`tool_defs[]` array, converting each entry to the OpenAI format:

```json
{
  "type": "function",
  "function": {
    "name": "snapshot",
    "description": "Get the accessibility tree...",
    "parameters": { ...schema... }
  }
}
```

## Concurrency note

See the **Concurrency model** section at the top of this document for the
full design. Summary: the agent loop runs on a background `GThread`. LLM
HTTP calls block in-thread. Browser tool calls hop to the GTK main thread
via `g_idle_add()` + `GAsyncQueue` (WebKitGTK is not thread-safe).
Filesystem and shell tools run directly on the background thread. The UI
polls `sovereign://agents/status` for live progress. A cancel flag allows
the user to stop long-running tasks.

## Phasing

The work breaks into two phases that can be implemented sequentially:

**Phase 1 — Foundation:** fs/shell tools, LLM client, tool-call loop, chat
persistence, `sovereign://agents` config page, URL-bar `;` routing, basic
chat UI. After Phase 1 the example task ("save links to ~/temp/links.txt")
works end-to-end.

**Phase 2 — Polish:** streaming responses, tool-call progress display in the
chat UI, session list / history sidebar, editable system prompt per session,
multi-session support from the URL bar, error recovery UI.