From 372339aa3585070047a7e92a8f6efafb2a2460bd Mon Sep 17 00:00:00 2001 From: Laan Tungir Date: Sun, 12 Jul 2026 09:20:14 -0400 Subject: [PATCH] v0.0.12 - MCP server: 100 tools, Streamable HTTP transport, favicons, refresh button, webview sizing fix - MCP protocol compliance: ping, resources/list, prompts/list, logging/setLevel, 202 notifications - Screenshot tool: WebKit snapshot to PNG to base64 MCP image content - Session management: Mcp-Session-Id header, SSE response format, GET/DELETE endpoints - Phase 3: 70 new tools across 8 batches (extended interaction, get/state, find elements, wait/batch, cookies/storage, mouse/clipboard/settings, frames/dialogs/debug, complex) - Favicon support: enabled WebKitFaviconDatabase, notify::favicon signal handler - Tab UI: favicons in tab labels, evenly distributed tabs, immediate title from URL host - Refresh button: left-click reload, right-click hard reload menu (bypass cache, clear site data) - Webview sizing fix: gtk_widget_set_vexpand/hexpand to prevent 1px height blank page - Updated Roo Code MCP config (100 tools in alwaysAllow) - Updated plans: mcp-server.md, agent-tools.md, phase3-tools.md --- VERSION | 2 +- plans/agent-tools.md | 75 +- plans/mcp-server.md | 54 +- plans/phase3-tools.md | 290 +++ src/agent_mcp.c | 558 +++++- src/agent_tools.c | 4217 +++++++++++++++++++++++++++++++++++++++++ src/main.c | 8 + src/tab_manager.c | 186 +- src/tab_manager.h | 7 + src/version.h | 4 +- 10 files changed, 5356 insertions(+), 45 deletions(-) create mode 100644 plans/phase3-tools.md diff --git a/VERSION b/VERSION index 2cfabea..8cbf02c 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.0.11 +0.0.12 diff --git a/plans/agent-tools.md b/plans/agent-tools.md index 866c88c..49fe646 100644 --- a/plans/agent-tools.md +++ b/plans/agent-tools.md @@ -441,7 +441,7 @@ This means the agent server has two states: | Clipboard | 4 | 0 | 0 | 4 | 0 | | Mouse | 4 | 0 | 0 | 4 | 0 | | Settings | 7 | 0 | 0 | 4 | 3 | -| Cookies/storage | 8 | 0 | 0 | 8 | 0 | +| Cookies/storage | 8 | 0 | 0 | 11 | 0 | | Network | 6 | 0 | 0 | 0 | 6 | | Tabs/windows | 5 | 4 | 0 | 0 | 1 | | Frames | 2 | 0 | 0 | 2 | 0 | @@ -452,19 +452,24 @@ This means the agent server has two states: | Setup | 2 | 0 | 0 | 0 | 2 | | Auth | 2 | 0 | 0 | 0 | 2 | | Sessions | 4 | 0 | 0 | 2 | 2 | -| **Total** | **119 + 8** | **31** | **0** | **67** | **29** | +| **Total** | **119 + 8** | **30** | **0** | **70** | **29** | -**Phase 1 (31 tools):** Login tools (8) + the minimal set for agent-driven -browser automation (23) — login, navigate, snapshot, inspect, interact, -and manage tabs. This is enough for an LLM to log in, browse, read, fill -forms, click buttons, and multi-tab. The agent server starts before login -so the agent can authenticate without human interaction. +**Phase 1 (30 tools):** Login tools (8) + the minimal set for agent-driven +browser automation (22) — login, navigate, snapshot, inspect, interact, +and manage tabs, plus `screenshot`. This is enough for an LLM to log in, +browse, read, fill forms, click buttons, and multi-tab. The agent server +starts before login so the agent can authenticate without human +interaction. **Phase 2 (0 tools):** Reserved for refinements to Phase 1 tools based on real agent usage — better snapshot formatting, error handling, edge cases. -**Phase 3 (67 tools):** Everything else that has a WebKitGTK equivalent. -Deferred until Phase 1 is stable and tested with real agents. +**Phase 3 (70 tools):** Everything else that has a WebKitGTK equivalent. +The original plan counted 67, but implementation added 3 extra +`storage_session_*` tools (the plan listed storage as 8 local-only tools; +we implemented 11 by adding `storage_session_get`, `storage_session_get_key`, +`storage_session_set`, and `storage_session_clear`). Deferred until Phase 1 +is stable and tested with real agents. **N/A (29 tools):** CDP-specific features (network interception, Chrome DevTools profiler, trace), cloud provider integrations, and features that @@ -641,11 +646,21 @@ Add `libsoup-3.0` to pkg-config if not already included. 15. Better snapshot formatting and error messages 16. Edge case handling (page navigation during interaction, stale refs) -### Phase 3 — Extended tool catalog +### Phase 3 — Extended tool catalog (COMPLETE) -17. All remaining tools from the comparison table (67 tools) +17. All remaining tools from the comparison table (70 tools — 67 planned + + 3 extra `storage_session_*` tools) 18. Find elements, check state, clipboard, mouse control, cookies/storage, frames, dialogs, console/errors, batch, state save/load, etc. +19. Implemented in 8 batches (see [`plans/phase3-tools.md`](plans/phase3-tools.md)): + - Batch 1: Extended interaction (11 tools) + - Batch 2: Get info + check state (7 tools) + - Batch 3: Find elements (10 tools) + - Batch 4: Wait + batch (5 tools) + - Batch 5: Cookies + storage (11 tools) + - Batch 6: Mouse + clipboard + settings (13 tools) + - Batch 7: Frames + dialogs + debug (10 tools) + - Batch 8: Complex tools (3 tools) ## Testing the tools @@ -734,3 +749,41 @@ An agent could subscribe to Nostr events and react to them — e.g., a remote agent controlling the browser via Nostr DMs. This is a natural extension of the "sovereign identity" thesis: your Nostr identity controls your browser, not just your signing. + +## Implementation Status + +**All phases complete. 100 tools total.** + +| Phase | Tools | Status | +|-------|-------|--------| +| Phase 1 — Login + core | 30 | ✅ Complete | +| Phase 2 — Refinements | 0 | ✅ N/A (folded into Phase 1/3) | +| Phase 3 — Extended catalog | 70 | ✅ Complete (8 batches) | +| **Total** | **100** | ✅ | + +### Phase 3 batches (all complete) + +| Batch | Category | Tools | Count | +|-------|----------|-------|-------| +| 1 | Extended interaction | dblclick, select, check, uncheck, scroll_into_view, keyboard_type, insert_text, keydown, keyup, drag, close_all | 11 | +| 2 | Get info + check state | get_value, get_count, get_box, get_styles, is_visible, is_enabled, is_checked | 7 | +| 3 | Find elements | find_role, find_text, find_label, find_placeholder, find_alt, find_title, find_testid, find_first, find_last, find_nth | 10 | +| 4 | Wait + batch | wait_for_text, wait_for_url, wait_for_load, wait_for_fn, batch | 5 | +| 5 | Cookies + storage | cookies_get, cookies_set, cookies_clear, storage_local_get, storage_local_get_key, storage_local_set, storage_local_clear, storage_session_get, storage_session_get_key, storage_session_set, storage_session_clear | 11 | +| 6 | Mouse + clipboard + settings | mouse_move, mouse_down, mouse_up, mouse_wheel, clipboard_read, clipboard_write, clipboard_copy, clipboard_paste, set_viewport, set_offline, set_headers, set_credentials, set_media | 13 | +| 7 | Frames + dialogs + debug | frame_switch, frame_main, dialog_accept, dialog_dismiss, dialog_status, console, errors, highlight, state_save, state_load | 10 | +| 8 | Complex tools | upload, pdf, screenshot_annotated | 3 | +| **Total** | | | **70** | + +### Verification + +- `make clean && make` — builds cleanly (only pre-existing warnings in + `login_dialog.c` and `agent_mcp.c`). +- `tools/list` returns exactly 100 tools. +- `initialize` / `ping` / `login_status` work without login. +- Browser tools (`dblclick`, etc.) correctly return `NOT_LOGGED_IN` + before login. +- `batch` dispatches without login; sub-commands enforce login + individually. +- Roo Code MCP config (`mcp_settings.json`) `alwaysAllow` updated with + all 100 tool names. diff --git a/plans/mcp-server.md b/plans/mcp-server.md index edffe51..b053931 100644 --- a/plans/mcp-server.md +++ b/plans/mcp-server.md @@ -56,7 +56,7 @@ This is ideal for us — we already have a SoupServer running. {"jsonrpc": "2.0", "id": 2, "method": "tools/list"} ``` -4. **Server responds** with the full tool catalog (31 tools). +4. **Server responds** with the full tool catalog (30 tools). 5. **Client sends `tools/call`**: ```json @@ -238,9 +238,16 @@ void agent_mcp_register(SoupServer *server); The MCP handler at `/mcp` processes HTTP POST requests with JSON-RPC bodies. It handles three methods: -1. **`initialize`** — returns server info and capabilities -2. **`tools/list`** — returns the tool catalog (all 31 tools with schemas) +1. **`initialize`** — returns server info and capabilities, creates a session, returns `Mcp-Session-Id` header +2. **`tools/list`** — returns the tool catalog (all 30 tools with schemas) 3. **`tools/call`** — dispatches to `agent_tools_dispatch()` (same as WebSocket) +4. **`ping`** — health check, returns empty result +5. **`resources/list`** / **`resources/templates/list`** — returns empty lists (no resources exposed) +6. **`prompts/list`** — returns empty list (no prompts exposed) +7. **`logging/setLevel`** — acknowledged, no filtering applied +8. **`notifications/*`** — returns 202 Accepted with no body +9. **`GET /mcp`** — returns SSE content-type with `: connected` comment +10. **`DELETE /mcp`** — terminates the session (requires `Mcp-Session-Id` header) For `tools/call`, the MCP handler: 1. Extracts the tool name and arguments from the JSON-RPC params @@ -313,3 +320,44 @@ configuration: 5. Register the MCP handler in `agent_server_start()` 6. Test with Roo Code or a manual MCP client 7. Update Makefile and docs + +## Implementation Status + +**Completed** — all items in the implementation order above are done. + +### What's implemented + +- **MCP Streamable HTTP transport** on `/mcp` (port 17777), coexisting with the existing `/agent` WebSocket endpoint. +- **Session management**: `initialize` creates a session and returns a `Mcp-Session-Id` header. Subsequent requests are validated against the session store. Unknown/expired sessions return 404. Sessions have a 60-second idle timeout. +- **SSE responses**: all JSON-RPC responses (requests with an `id`) are wrapped in `event: message\ndata: \n\n` SSE format with `Content-Type: text/event-stream`. +- **GET /mcp**: returns a `text/event-stream` response with a `: connected` comment (satisfies clients probing for SSE push support). +- **DELETE /mcp**: terminates a session by `Mcp-Session-Id` header. Returns 200 on success, 400 if header missing, 404 if session unknown. +- **Notifications**: `notifications/*` and any request without an `id` return 202 Accepted with no body. +- **Tool catalog**: 30 tools exposed (29 original + `screenshot`). The `screenshot` tool returns an MCP image content block (`type: "image"`, base64 PNG, `mimeType: "image/png"`) instead of a text block. +- **Lenient mode**: requests without a `Mcp-Session-Id` header are accepted with a warning (backward compatibility with older clients). +- **Roo Code integration**: configured in `mcp_settings.json` with `transport: "streamable-http"` and all 30 tools in `alwaysAllow`. + +### Test results (2026-07-12) + +All 10 integration tests pass: + +| Test | Description | Result | +|------|-------------|--------| +| 1 | Initialize — returns `Mcp-Session-Id` header + SSE body | ✅ Pass | +| 2 | `tools/list` — 30 tools including `screenshot` | ✅ Pass | +| 3 | `ping` with session — SSE empty result | ✅ Pass | +| 4 | `resources/list` — empty resources array | ✅ Pass | +| 5 | `prompts/list` — empty prompts array | ✅ Pass | +| 6 | `notifications/initialized` — 202 Accepted | ✅ Pass | +| 7 | `GET /mcp` — `text/event-stream` content type | ✅ Pass | +| 8 | `DELETE /mcp` — 200 OK | ✅ Pass | +| 9 | `ping` after DELETE — 404 (session terminated) | ✅ Pass | +| 10 | `login_status` tool call — returns login state | ✅ Pass | + +### Files + +- [`src/agent_mcp.c`](src/agent_mcp.c) — MCP handler (request routing, session management, SSE responses) +- [`src/agent_mcp.h`](src/agent_mcp.h) — public API +- [`src/agent_tools.c`](src/agent_tools.c) — tool dispatch (shared with WebSocket endpoint) +- [`src/agent_server.c`](src/agent_server.c) — registers MCP handler on the SoupServer +- [`Makefile`](Makefile) — includes `src/agent_mcp.c` in the build diff --git a/plans/phase3-tools.md b/plans/phase3-tools.md new file mode 100644 index 0000000..63cfd51 --- /dev/null +++ b/plans/phase3-tools.md @@ -0,0 +1,290 @@ +# Phase 3 — Extended Tool Catalog (70 tools) + +> **Status: Complete** — All 8 batches implemented, built, and verified. +> 100 total tools (30 Phase 1 + 70 Phase 3). See the verification table +> at the bottom and [`plans/agent-tools.md`](agent-tools.md) for the full +> status. + +## Overview + +Implement all remaining tools from the agent-browser comparison table. +The original plan counted 67 tools; the final implementation delivered 70 +(11+7+10+5+11+13+10+3). The difference of 3 comes from expanding the +storage category: the plan listed 8 local-only storage tools, but we +added 4 `storage_session_*` tools (get, get_key, set, clear) for full +session storage parity, bringing the storage batch to 11. These extend +sovere_browser's MCP server from 30 tools to 100 tools, covering the full +agent-browser feature set that has a WebKitGTK equivalent. + +## Implementation patterns + +All Phase 3 tools follow one of these patterns: + +### Pattern A: JS eval (sync) +The majority of tools. Resolve ref/selector → build JS string → `agent_js_eval_sync()` → parse result → return cJSON. Same pattern as existing `click`, `fill`, `get_text`, etc. + +### Pattern B: WebKitGTK C API +Some tools need WebKitGTK C APIs instead of JS: +- Cookies → `WebKitCookieManager` +- Settings → `WebKitSettings` +- Dialogs → `WebKitScriptDialog` (via signal handler) +- Console → `WebKitConsoleMessage` (via signal handler) +- PDF → `webkit_web_view_save()` / print API +- State → `WebKitWebsiteDataManager` + +### Pattern C: GDK/GTK API +- Clipboard → `GtkClipboard` / `GdkClipboard` +- Mouse events → `gdk_event_new()` + `gtk_main_do_event()` (synthesized) +- Viewport → `gtk_window_resize()` + +### Pattern D: Composite +- `batch` — calls `agent_tools_dispatch()` in a loop +- `find_*` — builds CSS selector from semantic criteria, then uses Pattern A +- `wait_for_*` — polls with `agent_js_eval_sync()` in a loop (like existing `wait_for`) + +## Batches + +### Batch 1: Extended interaction (11 tools) +All Pattern A (JS eval). Straightforward extensions of existing interaction tools. + +| Tool | Params | JS approach | Notes | +|------|--------|-------------|-------| +| `dblclick` | ref/selector | `el.dispatchEvent(new MouseEvent('dblclick',{bubbles:true}))` | Like click but dblclick event | +| `select` | ref/selector, value | Set `