sovereign_browser/plans/agent-tools.md

# Agent Tools Implementation Plan

## Overview

Make sovereign_browser agentically capable by embedding a WebSocket server
in the browser process. External AI agents (LLM tools, scripts, other
processes) connect to `ws://localhost:PORT` and send JSON tool commands to
navigate, inspect, and interact with web pages.

The design is inspired by [agent-browser](https://github.com/vercel-labs/agent-browser),
which uses a **snapshot + ref** workflow: take a snapshot (accessibility tree
with element refs), then interact by ref (click `@e2`, fill `@e3`). This is
the optimal pattern for LLM-driven browser automation.

## Architecture

```mermaid
flowchart TB
    subgraph AgentProcess
        Agent[AI Agent / Script / CLI]
    end

    subgraph BrowserProcess
        WSServer[WebSocket Server — SoupServer]
        ToolDispatch[Tool Dispatcher]
        Snapshot[Snapshot Engine]
        Tools[Tool Implementations]
        TabMgr[Tab Manager]
    end

    subgraph WebKitGTK
        Webview[WebKitWebView — active tab]
        JSEval[webkit_web_view_evaluate_javascript]
    end

    Agent -->|ws://localhost:PORT JSON| WSServer
    WSServer --> ToolDispatch
    ToolDispatch --> Tools
    Tools --> Snapshot
    Tools --> TabMgr
    Tools -->|JS eval| JSEval
    JSEval --> Webview
    Snapshot -->|inject + extract| JSEval
    TabMgr --> Webview
```

### Key design decisions

0. **Agent tools must use the same code path as the GUI** — Agent tools
   should call the same C functions the GUI calls, not a parallel
   implementation. This ensures that any error a user would see, the
   agent also sees. Debugging WebKit internals is not our concern; our
   C code is. If the agent and GUI take different paths, bugs can hide
   from the agent. The login tools are the one exception (backend-only,
   bypassing the GTK dialog) — all browser tools go through the same
   `WebKitWebView` and `tab_manager` APIs.

1. **WebSocket server via libsoup** — `SoupServer` with
   `soup_server_add_websocket_handler()` provides a production-ready
   WebSocket server. libsoup is already linked (WebKitGTK dependency).
   No new dependencies, no hand-rolling the WebSocket protocol.

2. **JSON protocol via cJSON** — cJSON is already vendored in
   `nostr_core_lib/cjson/`. All tool requests and responses are JSON.

3. **Snapshot + ref pattern** — The core workflow:
   - `snapshot` injects a JS script that walks the accessibility tree,
     assigns sequential refs (`e1`, `e2`, ...) to interactive elements,
     and returns a text representation of the tree.
   - `click @e2` looks up the ref in a per-tab ref map, finds the DOM
     element, and clicks it via `element.click()`.
   - Refs are per-tab and persist until the next `snapshot` call.

4. **Per-tab ref storage** — Each `tab_info_t` gets a ref map (hash table
   from ref string → element selector or DOM node reference). Since JS
   execution is per-webview, refs are scoped to the tab's webview.

5. **Async JS execution** — `webkit_web_view_evaluate_javascript()` is
   async. For tools that need the return value (snapshot, get_text, eval),
   we use a callback-based approach: the tool injects JS that stores the
   result in a known global variable, then reads it back. Alternatively,
   we can use a Promise-based approach with a polling mechanism.

6. **Simple HTTP fallback** — The WebSocket server also handles plain HTTP
   GET requests to `http://localhost:PORT/` returning a status JSON
   (server info, connected clients, tab count). This makes it easy to
   discover the server with `curl` without a WebSocket client.

## WebSocket protocol

### Request format

```json
{
  "id": 1,
  "tool": "snapshot",
  "params": {
    "interactive": true,
    "compact": true
  }
}
```

### Response format

```json
{
  "id": 1,
  "success": true,
  "data": {
    "snapshot": "- heading \"Example Domain\" [ref=e1] [level=1]\n- link \"More information...\" [ref=e2]",
    "refs": {
      "e1": {"role": "heading", "name": "Example Domain", "level": 1},
      "e2": {"role": "link", "name": "More information...", "href": "https://example.com"}
    }
  }
}
```

### Error response

```json
{
  "id": 1,
  "success": false,
  "error": {
    "code": "ELEMENT_NOT_FOUND",
    "message": "No element with ref @e99"
  }
}
```

### Push events (server → client)

```json
{
  "type": "event",
  "event": "load",
  "data": {"url": "https://example.com", "title": "Example Domain", "tab": 0}
}
```

```json
{
  "type": "event",
  "event": "tab_changed",
  "data": {"tab": 1, "url": "https://example.com", "title": "Example"}
}
```

## Tool catalog — full comparison with agent-browser

The table below lists every agent-browser CLI command and whether we plan
to implement it. "Phase 1" tools are in the initial implementation; "Phase 2"
tools are deferred but designed for; "N/A" tools don't apply to our
architecture (CDP-specific, cloud providers, etc.).

### Core commands

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `open <url>` | `open` | 1 | Navigate active tab |
| `click <sel>` | `click` | 1 | By ref or CSS selector |
| `dblclick <sel>` | `dblclick` | 2 | Double-click |
| `focus <sel>` | `focus` | 1 | Focus element |
| `type <sel> <text>` | `type` | 1 | Type into element |
| `fill <sel> <text>` | `fill` | 1 | Clear and fill |
| `press <key>` | `press` | 1 | Press keyboard key |
| `keyboard type <text>` | `keyboard_type` | 2 | Type with real keystrokes |
| `keyboard inserttext <text>` | `insert_text` | 2 | Insert text without key events |
| `keydown <key>` | `keydown` | 2 | Hold key down |
| `keyup <key>` | `keyup` | 2 | Release key |
| `hover <sel>` | `hover` | 1 | Hover element |
| `select <sel> <val>` | `select` | 2 | Select dropdown option |
| `check <sel>` | `check` | 2 | Check checkbox |
| `uncheck <sel>` | `uncheck` | 2 | Uncheck checkbox |
| `scroll <dir> [px]` | `scroll` | 1 | Scroll page |
| `scrollintoview <sel>` | `scroll_into_view` | 2 | Scroll element into view |
| `drag <src> <tgt>` | `drag` | 2 | Drag and drop |
| `upload <sel> <files>` | `upload` | 2 | Upload files |
| `screenshot [path]` | `screenshot` | 1 | Take screenshot |
| `screenshot --annotate` | `screenshot_annotated` | 2 | Annotated with element labels |
| `pdf <path>` | `pdf` | 2 | Save as PDF |
| `snapshot` | `snapshot` | 1 | Accessibility tree with refs |
| `eval <js>` | `eval` | 1 | Run JavaScript |
| `connect <port>` | N/A | — | CDP-specific, we are the server |
| `stream enable` | `stream_enable` | 2 | WebSocket viewport streaming |
| `stream status` | `stream_status` | 2 | Streaming state |
| `stream disable` | `stream_disable` | 2 | Stop streaming |
| `close` | `close` | 1 | Close active tab |
| `close --all` | `close_all` | 2 | Close all tabs |

### Get info commands

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `get text <sel>` | `get_text` | 1 | Get text content |
| `get html <sel>` | `get_html` | 1 | Get innerHTML |
| `get value <sel>` | `get_value` | 2 | Get input value |
| `get attr <sel> <attr>` | `get_attr` | 1 | Get attribute |
| `get title` | `get_title` | 1 | Get page title |
| `get url` | `get_url` | 1 | Get current URL |
| `get cdp-url` | N/A | — | CDP-specific |
| `get count <sel>` | `get_count` | 2 | Count matching elements |
| `get box <sel>` | `get_box` | 2 | Get bounding box |
| `get styles <sel>` | `get_styles` | 2 | Get computed styles |

### Check state commands

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `is visible <sel>` | `is_visible` | 2 | Check if visible |
| `is enabled <sel>` | `is_enabled` | 2 | Check if enabled |
| `is checked <sel>` | `is_checked` | 2 | Check if checked |

### Find elements (semantic locators)

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `find role <role> ...` | `find_role` | 2 | By ARIA role |
| `find text <text> ...` | `find_text` | 2 | By text content |
| `find label <label> ...` | `find_label` | 2 | By label |
| `find placeholder <ph> ...` | `find_placeholder` | 2 | By placeholder |
| `find alt <text> ...` | `find_alt` | 2 | By alt text |
| `find title <text> ...` | `find_title` | 2 | By title attr |
| `find testid <id> ...` | `find_testid` | 2 | By data-testid |
| `find first <sel> ...` | `find_first` | 2 | First match |
| `find last <sel> ...` | `find_last` | 2 | Last match |
| `find nth <n> <sel> ...` | `find_nth` | 2 | Nth match |

### Wait commands

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `wait <selector>` | `wait_for` | 1 | Wait for element visible |
| `wait <ms>` | `wait` | 1 | Wait for time |
| `wait --text "..."` | `wait_for_text` | 2 | Wait for text to appear |
| `wait --url "..."` | `wait_for_url` | 2 | Wait for URL pattern |
| `wait --load networkidle` | `wait_for_load` | 2 | Wait for load state |
| `wait --fn "..."` | `wait_for_fn` | 2 | Wait for JS condition |

### Batch execution

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `batch --json` | `batch` | 2 | Execute multiple commands |

### Clipboard

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `clipboard read` | `clipboard_read` | 2 | Read clipboard |
| `clipboard write "..."` | `clipboard_write` | 2 | Write clipboard |
| `clipboard copy` | `clipboard_copy` | 2 | Copy selection |
| `clipboard paste` | `clipboard_paste` | 2 | Paste from clipboard |

### Mouse control

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `mouse move <x> <y>` | `mouse_move` | 2 | Move mouse |
| `mouse down [button]` | `mouse_down` | 2 | Press button |
| `mouse up [button]` | `mouse_up` | 2 | Release button |
| `mouse wheel <dy> [dx]` | `mouse_wheel` | 2 | Scroll wheel |

### Browser settings

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `set viewport <w> <h>` | `set_viewport` | 2 | Set viewport size |
| `set device <name>` | `set_device` | N/A | Device emulation is CDP-specific |
| `set geo <lat> <lng>` | `set_geo` | N/A | Geolocation emulation |
| `set offline [on\|off]` | `set_offline` | 2 | Toggle offline |
| `set headers <json>` | `set_headers` | 2 | Extra HTTP headers |
| `set credentials <u> <p>` | `set_credentials` | 2 | HTTP basic auth |
| `set media [dark\|light]` | `set_media` | 2 | Emulate color scheme |

### Cookies and storage

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `cookies` | `cookies_get` | 2 | Get all cookies |
| `cookies set <name> <val>` | `cookies_set` | 2 | Set cookie |
| `cookies clear` | `cookies_clear` | 2 | Clear cookies |
| `storage local` | `storage_local_get` | 2 | Get localStorage |
| `storage local <key>` | `storage_local_get_key` | 2 | Get specific key |
| `storage local set <k> <v>` | `storage_local_set` | 2 | Set value |
| `storage local clear` | `storage_local_clear` | 2 | Clear all |
| `storage session` | `storage_session_*` | 2 | Same for sessionStorage |

### Network

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `network route <url>` | `network_route` | N/A | Request interception is CDP-specific |
| `network unroute` | `network_unroute` | N/A | CDP-specific |
| `network requests` | `network_requests` | N/A | CDP-specific |
| `network request <id>` | `network_request_detail` | N/A | CDP-specific |
| `network har start` | `har_start` | N/A | CDP-specific |
| `network har stop` | `har_stop` | N/A | CDP-specific |

### Tabs and windows

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `tab` | `tab_list` | 1 | List tabs |
| `tab new [url]` | `tab_new` | 1 | New tab |
| `tab <n>` | `tab_switch` | 1 | Switch to tab n |
| `tab close [n]` | `tab_close` | 1 | Close tab |
| `window new` | `window_new` | N/A | Multi-window is future roadmap |

### Frames

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `frame <sel>` | `frame_switch` | 2 | Switch to iframe |
| `frame main` | `frame_main` | 2 | Back to main frame |

### Dialogs

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `dialog accept [text]` | `dialog_accept` | 2 | Accept dialog |
| `dialog dismiss` | `dialog_dismiss` | 2 | Dismiss dialog |
| `dialog status` | `dialog_status` | 2 | Check if dialog open |

### Diff

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `diff snapshot` | `diff_snapshot` | N/A | Future, after snapshot is stable |
| `diff screenshot` | `diff_screenshot` | N/A | Future |
| `diff url` | `diff_url` | N/A | Future |

### Debug

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `trace start/stop` | N/A | — | Chrome trace, not available in WebKitGTK |
| `profiler start/stop` | N/A | — | Chrome DevTools profiler |
| `console` | `console` | 2 | View console messages |
| `errors` | `errors` | 2 | View page errors |
| `highlight <sel>` | `highlight` | 2 | Highlight element |
| `inspect` | N/A | — | Opens Chrome DevTools |
| `state save/load/list` | `state_save/load/list` | 2 | Save/restore auth state |

### Navigation

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `back` | `back` | 1 | Go back |
| `forward` | `forward` | 1 | Go forward |
| `reload` | `reload` | 1 | Reload page |

### Setup

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `install` | N/A | — | We bundle WebKitGTK, no install needed |
| `upgrade` | N/A | — | Package manager handles updates |

### Authentication

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `auth save` | N/A | — | We use Nostr identity, not credential vault |
| `auth login` | N/A | — | Nostr login is our native flow |

### Sessions

| agent-browser command | Our tool | Phase | Notes |
|------------------------|----------|-------|-------|
| `--session <name>` | N/A | — | We have one browser process, tabs instead |
| `--session-name <name>` | N/A | — | Session persistence is our session.c |
| `--profile <path>` | N/A | — | WebKitGTK handles profile data |
| `--state <path>` | `state_save/load` | 2 | Could implement via WebKitGTK data manager |

### Login tools (sovereign_browser-specific, not in agent-browser)

These tools are unique to sovereign_browser. They allow an agent to handle
the Nostr login flow programmatically — essential because the browser
shows a modal login dialog on startup that blocks all other tools until
the user (or agent) signs in.

The agent server starts **before** the login dialog, so the agent can
authenticate without any human interaction. The login tools call the same
`nostr_core_lib` functions that the GTK login dialog uses, bypassing the
dialog entirely.

| Tool | Params | Description |
|------|--------|-------------|
| `login_status` | `{}` | Check if logged in; return method, pubkey, readonly |
| `login` | `{"method": "local", "nsec": "nsec1..."}` | Sign in with local key |
| `login` | `{"method": "seed", "mnemonic": "word1 word2 ...", "passphrase": "..."}` | Sign in with BIP-39 seed phrase |
| `login` | `{"method": "readonly", "npub": "npub1..."}` | Sign in read-only with npub |
| `login` | `{"method": "nip46", "bunker_url": "bunker://..."}` | Sign in with NIP-46 remote signer |
| `login` | `{"method": "nsigner", "transport": "qrexec", "device": "nostr_signer", "service": "qubes.NsignerRpc", "index": 0}` | Sign in with n_signer hardware |
| `logout` | `{}` | Log out and clear signer |
| `switch_identity` | `{"method": "local", "nsec": "nsec1..."}` | Switch to a new identity (same params as login) |

**Login response:**
```json
{
  "id": 1,
  "success": true,
  "data": {
    "method": "local",
    "pubkey": "a1b2c3...",
    "npub": "npub1...",
    "readonly": false
  }
}
```

**Architecture change:** The agent server must start *before* the login
dialog. In `main.c`, the startup sequence becomes:

1. `gtk_init()` + load settings/history
2. Create the main window (but don't show it yet)
3. **Start the agent server** (so agents can connect)
4. If `agent_server_enabled` and an agent connects within a timeout,
   wait for the agent to call `login`
5. Otherwise (no agent, or timeout), show the GTK login dialog as before
6. After login (via agent or dialog), create tabs and show the window

This means the agent server has two states:
- **Pre-login:** Only `login_status`, `login`, `logout` tools are available
- **Post-login:** All browser tools become available

### Summary

| Category | agent-browser commands | Our Phase 1 | Our Phase 2 | Our Phase 3 | N/A |
|----------|----------------------|-------------|-------------|-------------|-----|
| Login (unique) | 0 (8 new) | 8 | 0 | 0 | 0 |
| Core | 29 | 10 | 0 | 14 | 5 |
| Get info | 9 | 4 | 0 | 4 | 1 |
| Check state | 3 | 0 | 0 | 3 | 0 |
| Find elements | 10 | 0 | 0 | 10 | 0 |
| Wait | 6 | 2 | 0 | 4 | 0 |
| Batch | 1 | 0 | 0 | 1 | 0 |
| Clipboard | 4 | 0 | 0 | 4 | 0 |
| Mouse | 4 | 0 | 0 | 4 | 0 |
| Settings | 7 | 0 | 0 | 4 | 3 |
| Cookies/storage | 8 | 0 | 0 | 11 | 0 |
| Network | 6 | 0 | 0 | 0 | 6 |
| Tabs/windows | 5 | 4 | 0 | 0 | 1 |
| Frames | 2 | 0 | 0 | 2 | 0 |
| Dialogs | 3 | 0 | 0 | 3 | 0 |
| Diff | 3 | 0 | 0 | 0 | 3 |
| Debug | 8 | 0 | 0 | 4 | 4 |
| Navigation | 3 | 3 | 0 | 0 | 0 |
| Setup | 2 | 0 | 0 | 0 | 2 |
| Auth | 2 | 0 | 0 | 0 | 2 |
| Sessions | 4 | 0 | 0 | 2 | 2 |
| **Total** | **119 + 8** | **30** | **0** | **70** | **29** |

**Phase 1 (30 tools):** Login tools (8) + the minimal set for agent-driven
browser automation (22) — login, navigate, snapshot, inspect, interact,
and manage tabs, plus `screenshot`. This is enough for an LLM to log in,
browse, read, fill forms, click buttons, and multi-tab. The agent server
starts before login so the agent can authenticate without human
interaction.

**Phase 2 (0 tools):** Reserved for refinements to Phase 1 tools based on
real agent usage — better snapshot formatting, error handling, edge cases.

**Phase 3 (70 tools):** Everything else that has a WebKitGTK equivalent.
The original plan counted 67, but implementation added 3 extra
`storage_session_*` tools (the plan listed storage as 8 local-only tools;
we implemented 11 by adding `storage_session_get`, `storage_session_get_key`,
`storage_session_set`, and `storage_session_clear`). Deferred until Phase 1
is stable and tested with real agents.

**N/A (29 tools):** CDP-specific features (network interception, Chrome
DevTools profiler, trace), cloud provider integrations, and features that
don't map to our architecture (device emulation, multi-window). Some could
be revisited if WebKitGTK adds equivalent APIs.

## Snapshot engine

The snapshot is the heart of the agent system. It injects a JavaScript
script into the active tab's webview that:

1. Walks the DOM accessibility tree (using `TreeWalker` or recursive
   `querySelectorAll`)
2. For each interactive element (links, buttons, inputs, selects, textareas,
   elements with role/aria-label), assigns a sequential ref (`e1`, `e2`, ...)
3. Stores a mapping from ref → unique CSS selector (for later re-querying)
4. Returns a text representation of the tree:

```
- heading "Example Domain" [ref=e1] [level=1]
- paragraph "This domain is for use in..."
- link "More information..." [ref=e2]
- textbox "Search" [ref=e3]
- button "Submit" [ref=e4]
```

### Ref storage

The ref map is stored as a JSON string in a hidden `window.__agentRefs`
global on the page. When a tool like `click @e2` is called:

1. The tool injects JS: `var el = document.querySelector(window.__agentRefs['e2']); el.click();`
2. If the element is gone (page navigated), the tool returns an error
   suggesting a re-snapshot.

### Interactive-only mode

When `interactive: true`, only elements that are focusable or have an
ARIA role are included. This dramatically reduces the snapshot size for
LLM context windows.

## New files

### `src/agent_server.h` / `src/agent_server.c`

```c
/*
 * WebSocket server for agent tool commands.
 * Uses libsoup SoupServer with a websocket handler at /agent.
 */

void agent_server_start(int port);
void agent_server_stop(void);
int  agent_server_get_port(void);
gboolean agent_server_is_running(void);

/* Broadcast a push event to all connected clients. */
void agent_server_emit_event(const char *event_name, cJSON *data);
```

### `src/agent_tools.h` / `src/agent_tools.c`

```c
/*
 * Tool dispatch: receives a JSON request, executes the tool,
 * returns a JSON response.
 */

/* Process a tool request JSON and return response JSON.
 * The caller frees the returned cJSON*. */
cJSON *agent_tools_dispatch(cJSON *request);
```

### `src/agent_snapshot.h` / `src/agent_snapshot.c`

```c
/*
 * Accessibility tree snapshot via JS injection.
 * Assigns refs to interactive elements and returns a text tree.
 */

/* The JS script injected for snapshot. Exposed for testing. */
extern const char *AGENT_SNAPSHOT_JS;

/* Parse a snapshot result JSON from the JS execution. */
cJSON *agent_snapshot_parse(const char *js_result);
```

## New files (login-specific)

### `src/agent_login.h` / `src/agent_login.c`

```c
/*
 * Agent-driven Nostr login — calls nostr_core_lib directly,
 * bypassing the GTK login dialog. Used by the agent server
 * so an AI agent can authenticate without human interaction.
 */

/* Check current login state. Returns JSON response. */
cJSON *agent_login_status(void);

/* Perform login with the given method and params.
 * params is a cJSON object with "method" and method-specific fields.
 * Returns JSON response with pubkey/npub on success. */
cJSON *agent_login(cJSON *params);

/* Log out — clear signer, reset state. Returns JSON response. */
cJSON *agent_logout(void);

/* Switch identity — same as login but frees the old signer first. */
cJSON *agent_switch_identity(cJSON *params);

/* Returns TRUE if logged in (signer or readonly loaded). */
gboolean agent_is_logged_in(void);
```

## Modified files

### `src/settings.h` / `src/settings.c`

Add settings:
- `agent_server_enabled` (bool, default: true)
- `agent_server_port` (int, default: 17777)
- `agent_server_allowed_origins` (string, default: "*" for localhost)
- `agent_login_timeout_ms` (int, default: 30000) — how long to wait
  for an agent to log in before falling back to the GTK dialog

### `src/tab_manager.h` / `src/tab_manager.c`

- Add `GHashTable *ref_map` to `tab_info_t` for per-tab element refs
- Emit events to `agent_server` on load, title change, tab switch

### `src/main.c`

Major change to startup sequence:
- Call `agent_server_start()` **before** `do_login()` (not after)
- After starting the server, wait for either:
  - An agent to call `login` within `agent_login_timeout_ms`, OR
  - The timeout to expire, then show the GTK login dialog
- After login (via agent or dialog), proceed to create tabs and show window
- Call `agent_server_stop()` in `on_window_destroy()`

### `Makefile`

Add `src/agent_server.c src/agent_tools.c src/agent_snapshot.c src/agent_login.c` to `SRC`.
Add `libsoup-3.0` to pkg-config if not already included.

## Implementation order

### Phase 1 — Login + core browser automation

1. **settings** — add agent server settings (port, enabled, allowed origins)
2. **agent_server** — WebSocket server using libsoup SoupServer, JSON protocol
3. **agent_login** — login tools (login_status, login, logout, switch_identity)
   that call nostr_core_lib directly, bypassing the GTK dialog
4. **main.c integration (pre-login)** — start agent server before login dialog,
   support agent-driven login with fallback to GTK dialog
5. **agent_snapshot** — the JS injection script + ref assignment
6. **agent_tools** — tool dispatch + core tools (open, snapshot, click, fill,
   type, eval, get_text, get_html, get_attr, get_url, get_title)
7. **main.c integration (post-login)** — wire browser tools to tab_manager,
   emit events (load, tab_changed)
8. **Navigation tools** — back, forward, reload, scroll, press, wait, wait_for
9. **Tab tools** — tab_list, tab_new, tab_switch, tab_close
10. **Interaction tools** — hover, focus, close
11. **Screenshot tool** — WebKitGTK snapshot API
12. **UI status indicator** — show server port in toolbar
13. **Makefile + docs** — build and documentation

### Phase 2 — Refinements

14. Polish Phase 1 tools based on real agent usage
15. Better snapshot formatting and error messages
16. Edge case handling (page navigation during interaction, stale refs)

### Phase 3 — Extended tool catalog (COMPLETE)

17. All remaining tools from the comparison table (70 tools — 67 planned
    + 3 extra `storage_session_*` tools)
18. Find elements, check state, clipboard, mouse control, cookies/storage,
    frames, dialogs, console/errors, batch, state save/load, etc.
19. Implemented in 8 batches (see [`plans/phase3-tools.md`](plans/phase3-tools.md)):
    - Batch 1: Extended interaction (11 tools)
    - Batch 2: Get info + check state (7 tools)
    - Batch 3: Find elements (10 tools)
    - Batch 4: Wait + batch (5 tools)
    - Batch 5: Cookies + storage (11 tools)
    - Batch 6: Mouse + clipboard + settings (13 tools)
    - Batch 7: Frames + dialogs + debug (10 tools)
    - Batch 8: Complex tools (3 tools)

## Testing the tools

The agent server starts before login, so the first thing an agent does is
authenticate. Once logged in, all browser tools become available.

### Login flow

```bash
# Using websocat (install: cargo install websocat)

# Check login status (before logging in)
echo '{"id":1,"tool":"login_status","params":{}}' | websocat ws://localhost:17777/agent

# Login with local key (nsec)
echo '{"id":2,"tool":"login","params":{"method":"local","nsec":"nsec1..."}}' | websocat ws://localhost:17777/agent

# Login read-only with npub
echo '{"id":3,"tool":"login","params":{"method":"readonly","npub":"npub1..."}}' | websocat ws://localhost:17777/agent

# Login with seed phrase
echo '{"id":4,"tool":"login","params":{"method":"seed","mnemonic":"abandon abandon abandon ... abandon about"}}' | websocat ws://localhost:17777/agent

# Login with NIP-46 remote signer
echo '{"id":5,"tool":"login","params":{"method":"nip46","bunker_url":"bunker://..."}}' | websocat ws://localhost:17777/agent

# Login with n_signer via Other Qube (qrexec)
echo '{"id":6,"tool":"login","params":{"method":"nsigner","transport":"qrexec","device":"nostr_signer","service":"qubes.NsignerRpc","index":0}}' | websocat ws://localhost:17777/agent
```

### Browser automation flow (after login)

```bash
# Open a page
echo '{"id":10,"tool":"open","params":{"url":"https://example.com"}}' | websocat ws://localhost:17777/agent

# Snapshot — get accessibility tree with refs
echo '{"id":11,"tool":"snapshot","params":{"interactive":true}}' | websocat ws://localhost:17777/agent

# Click by ref
echo '{"id":12,"tool":"click","params":{"ref":"@e2"}}' | websocat ws://localhost:17777/agent

# Fill an input
echo '{"id":13,"tool":"fill","params":{"ref":"@e3","value":"test@example.com"}}' | websocat ws://localhost:17777/agent

# Get text
echo '{"id":14,"tool":"get_text","params":{"ref":"@e1"}}' | websocat ws://localhost:17777/agent

# List tabs
echo '{"id":15,"tool":"tab_list","params":{}}' | websocat ws://localhost:17777/agent

# Open new tab
echo '{"id":16,"tool":"tab_new","params":{"url":"https://example.org"}}' | websocat ws://localhost:17777/agent
```

### Python example (persistent connection)

```python
import websocket, json

ws = websocket.create_connection("ws://localhost:17777/agent")

# Login first
ws.send(json.dumps({"id": 1, "tool": "login", "params": {
    "method": "readonly", "npub": "npub1..."
}}))
print(json.loads(ws.recv()))

# Then browse
ws.send(json.dumps({"id": 2, "tool": "open", "params": {
    "url": "https://example.com"
}}))
print(json.loads(ws.recv()))

ws.send(json.dumps({"id": 3, "tool": "snapshot", "params": {
    "interactive": True
}}))
print(json.loads(ws.recv()))
```

## Future: Nostr relay integration

Since the WebSocket server is already in the browser process, a future
enhancement could expose Nostr relay connections through the same server.
An agent could subscribe to Nostr events and react to them — e.g., a
remote agent controlling the browser via Nostr DMs. This is a natural
extension of the "sovereign identity" thesis: your Nostr identity controls
your browser, not just your signing.

## Implementation Status

**All phases complete. 100 tools total.**

| Phase | Tools | Status |
|-------|-------|--------|
| Phase 1 — Login + core | 30 | ✅ Complete |
| Phase 2 — Refinements | 0 | ✅ N/A (folded into Phase 1/3) |
| Phase 3 — Extended catalog | 70 | ✅ Complete (8 batches) |
| **Total** | **100** | ✅ |

### Phase 3 batches (all complete)

| Batch | Category | Tools | Count |
|-------|----------|-------|-------|
| 1 | Extended interaction | dblclick, select, check, uncheck, scroll_into_view, keyboard_type, insert_text, keydown, keyup, drag, close_all | 11 |
| 2 | Get info + check state | get_value, get_count, get_box, get_styles, is_visible, is_enabled, is_checked | 7 |
| 3 | Find elements | find_role, find_text, find_label, find_placeholder, find_alt, find_title, find_testid, find_first, find_last, find_nth | 10 |
| 4 | Wait + batch | wait_for_text, wait_for_url, wait_for_load, wait_for_fn, batch | 5 |
| 5 | Cookies + storage | cookies_get, cookies_set, cookies_clear, storage_local_get, storage_local_get_key, storage_local_set, storage_local_clear, storage_session_get, storage_session_get_key, storage_session_set, storage_session_clear | 11 |
| 6 | Mouse + clipboard + settings | mouse_move, mouse_down, mouse_up, mouse_wheel, clipboard_read, clipboard_write, clipboard_copy, clipboard_paste, set_viewport, set_offline, set_headers, set_credentials, set_media | 13 |
| 7 | Frames + dialogs + debug | frame_switch, frame_main, dialog_accept, dialog_dismiss, dialog_status, console, errors, highlight, state_save, state_load | 10 |
| 8 | Complex tools | upload, pdf, screenshot_annotated | 3 |
| **Total** | | | **70** |

### Verification

- `make clean && make` — builds cleanly (only pre-existing warnings in
  `login_dialog.c` and `agent_mcp.c`).
- `tools/list` returns exactly 100 tools.
- `initialize` / `ping` / `login_status` work without login.
- Browser tools (`dblclick`, etc.) correctly return `NOT_LOGGED_IN`
  before login.
- `batch` dispatches without login; sub-commands enforce login
  individually.
- Roo Code MCP config (`mcp_settings.json`) `alwaysAllow` updated with
  all 100 tool names.