sovereign_browser/plans/phase3-tools.md

Phase 3 — Extended Tool Catalog (70 tools)

Status: Complete — All 8 batches implemented, built, and verified. 100 total tools (30 Phase 1 + 70 Phase 3). See the verification table at the bottom and plans/agent-tools.md for the full status.

Overview

Implement all remaining tools from the agent-browser comparison table. The original plan counted 67 tools; the final implementation delivered 70 (11+7+10+5+11+13+10+3). The difference of 3 comes from expanding the storage category: the plan listed 8 local-only storage tools, but we added 4 storage_session_* tools (get, get_key, set, clear) for full session storage parity, bringing the storage batch to 11. These extend sovere_browser's MCP server from 30 tools to 100 tools, covering the full agent-browser feature set that has a WebKitGTK equivalent.

Implementation patterns

All Phase 3 tools follow one of these patterns:

Pattern A: JS eval (sync)

The majority of tools. Resolve ref/selector → build JS string → agent_js_eval_sync() → parse result → return cJSON. Same pattern as existing click, fill, get_text, etc.

Pattern B: WebKitGTK C API

Some tools need WebKitGTK C APIs instead of JS:

• Cookies → WebKitCookieManager
• Settings → WebKitSettings
• Dialogs → WebKitScriptDialog (via signal handler)
• Console → WebKitConsoleMessage (via signal handler)
• PDF → webkit_web_view_save() / print API
• State → WebKitWebsiteDataManager

Pattern C: GDK/GTK API

• Clipboard → GtkClipboard / GdkClipboard
• Mouse events → gdk_event_new() + gtk_main_do_event() (synthesized)
• Viewport → gtk_window_resize()

Pattern D: Composite

• batch — calls agent_tools_dispatch() in a loop
• find_* — builds CSS selector from semantic criteria, then uses Pattern A
• wait_for_* — polls with agent_js_eval_sync() in a loop (like existing wait_for)

Batches

Batch 1: Extended interaction (11 tools)

All Pattern A (JS eval). Straightforward extensions of existing interaction tools.

Tool	Params	JS approach	Notes
dblclick	ref/selector	el.dispatchEvent(new MouseEvent('dblclick',{bubbles:true}))	Like click but dblclick event
select	ref/selector, value	Set <select> .value + dispatch change event	For dropdowns
check	ref/selector	el.checked = true; el.dispatchEvent(new Event('change'))	Checkboxes
uncheck	ref/selector	el.checked = false; el.dispatchEvent(new Event('change'))	Checkboxes
scroll_into_view	ref/selector	el.scrollIntoView({behavior:'smooth',block:'center'})	Scroll element into view
keyboard_type	value	Synthesize keydown+keypress+keyup per char on focused element	Real keystrokes
insert_text	value	document.execCommand('insertText', false, text)	Insert without key events
keydown	key	new KeyboardEvent('keydown',{key:key,bubbles:true})	Hold key down
keyup	key	new KeyboardEvent('keyup',{key:key,bubbles:true})	Release key
drag	src_ref/selector, tgt_ref/selector	Synthesize dragstart→dragenter→dragover→drop→dragend	HTML5 drag and drop
close_all	(none)	Close all tabs via tab_manager_close_all()	New tab_manager function needed

Batch 2: Get info + check state (7 tools)

All Pattern A (JS eval). Read-only queries returning element state.

Tool	Params	JS approach	Returns
get_value	ref/selector	el.value	{"value":"..."}
get_count	selector	document.querySelectorAll(sel).length	{"count":N}
get_box	ref/selector	el.getBoundingClientRect()	{"x":N,"y":N,"width":N,"height":N}
get_styles	ref/selector	getComputedStyle(el) → JSON	{"styles":{...}}
is_visible	ref/selector	Check offsetParent, computed display/visibility	{"visible":true/false}
is_enabled	ref/selector	!el.disabled	{"enabled":true/false}
is_checked	ref/selector	el.checked	{"checked":true/false}

Batch 3: Find elements (10 tools)

Pattern D (composite). Build CSS selector from semantic criteria, then query.

Tool	Params	Selector approach
find_role	role, name?	[role="X"] or semantic HTML (button, nav, etc.)
find_text	text, exact?	:contains() via JS (no native CSS) — walk DOM
find_label	label	label[for] matching + aria-label + aria-labelledby
find_placeholder	placeholder	[placeholder*="X"]
find_alt	alt	[alt*="X"]
find_title	title	[title*="X"]
find_testid	testid	[data-testid="X"]
find_first	selector	document.querySelector(sel)
find_last	selector	document.querySelectorAll(sel)[last]
find_nth	n, selector	document.querySelectorAll(sel)[n]

All return {"ref":"@eN","selector":"...","role":"...","name":"..."} — assigns a ref so the result can be used with click/fill/etc. This requires integration with the snapshot ref system (window.__agentRefs).

Batch 4: Wait + batch (5 tools)

Tool	Params	Approach	Pattern
wait_for_text	text, timeout?	Poll JS: document.body.innerText.includes(text)	D (poll)
wait_for_url	url_pattern, timeout?	Poll webkit_web_view_get_uri() against regex	D (poll)
wait_for_load	state?, timeout?	Wait for WebKitWebView load event	B (signal)
wait_for_fn	script, timeout?	Poll JS: eval user script, check truthy	D (poll)
batch	commands[]	Loop: call agent_tools_dispatch() per command	D (composite)

Batch 5: Cookies + storage (11 tools)

Pattern B (WebKit API) for cookies, Pattern A (JS eval) for storage.

Tool	Params	Approach	Pattern
cookies_get	(none)	WebKitCookieManager async API	B
cookies_set	name, value, domain, path, ...	JS: document.cookie = "..."	A
cookies_clear	(none)	webkit_cookie_manager_clear()	B
storage_local_get	(none)	JS: JSON.stringify(localStorage)	A
storage_local_get_key	key	JS: localStorage.getItem(key)	A
storage_local_set	key, value	JS: localStorage.setItem(key, value)	A
storage_local_clear	(none)	JS: localStorage.clear()	A
storage_session_get	(none)	JS: JSON.stringify(sessionStorage)	A
storage_session_get_key	key	JS: sessionStorage.getItem(key)	A
storage_session_set	key, value	JS: sessionStorage.setItem(key, value)	A
storage_session_clear	(none)	JS: sessionStorage.clear()	A

Note: The original plan collapsed storage_session_* into a single row and counted 8 total for the batch (3 cookies + 4 local + 1 session row). The implementation expanded the session row into 4 concrete tools, giving 11 total (3 cookies + 4 local + 4 session). This is the source of the +3 difference between the planned 67 and the implemented 70.

Batch 6: Mouse + clipboard + settings (12 tools)

Tool	Params	Approach	Pattern
mouse_move	x, y	JS: dispatch mousemove at coordinates	A
mouse_down	button?	JS: dispatch mousedown	A
mouse_up	button?	JS: dispatch mouseup	A
mouse_wheel	dy, dx?	JS: dispatch wheel event	A
clipboard_read	(none)	gdk_clipboard_read_text_async()	C
clipboard_write	text	gdk_clipboard_set_text()	C
clipboard_copy	(none)	JS: document.execCommand('copy')	A
clipboard_paste	(none)	JS: document.execCommand('paste')	A
set_viewport	width, height	gtk_window_resize()	C
set_offline	on/off	WebKitWebContext network policy	B
set_headers	headers_json	WebKitUserContentManager injection or custom	B
set_credentials	user, pass	WebKitAuthenticationRequest handler	B
set_media	dark/light	WebKitSettings + CSS media query emulation	B

Note: set_media is 1 tool, set_credentials is 1 tool. Total for this batch: 13 tools (mouse 4 + clipboard 4 + settings 5).

Batch 7: Frames + dialogs + debug (10 tools)

Tool	Params	Approach	Pattern
frame_switch	ref/selector	Track current frame in C, eval JS in frame context	B
frame_main	(none)	Reset to main frame	B
dialog_accept	text?	webkit_script_dialog_confirm_set_confirmed() etc.	B
dialog_dismiss	(none)	Same API, set false	B
dialog_status	(none)	Check if a script dialog is pending	B
console	(none)	Collect console messages (need signal handler)	B
errors	(none)	Collect page errors (need signal handler)	B
highlight	ref/selector	JS: add temporary outline/border	A
state_save	(none)	WebKitWebsiteDataManager persist	B
state_load	(none)	Restore from persisted data	B

Note: state_save/load/list = 3 tools. Total: 10 tools (frames 2 + dialogs 3 + debug 2 + highlight 1 + state 2).

Batch 8: Complex tools (3 tools)

Tool	Params	Approach	Pattern
upload	ref/selector, files[]	Set file input .files via JS (limited) or WebKit API	B/A
pdf	path?	webkit_web_view_save() or print-to-PDF API	B
screenshot_annotated	(none)	Screenshot + overlay element labels via JS canvas	A+B

Tool count verification

Batch	Tools	Count	Status
1	Extended interaction	11	✅ Complete
2	Get info + check state	7	✅ Complete
3	Find elements	10	✅ Complete
4	Wait + batch	5	✅ Complete
5	Cookies + storage	11	✅ Complete (was 8 in plan; +3 storage_session_*)
6	Mouse + clipboard + settings	13	✅ Complete
7	Frames + dialogs + debug	10	✅ Complete
8	Complex	3	✅ Complete
Total		70	✅ All batches complete

Final verification (post-implementation)

• make clean && make — builds cleanly (only pre-existing warnings in login_dialog.c and agent_mcp.c).
• tools/list returns exactly 100 tools (30 Phase 1 + 70 Phase 3).
• initialize returns a session ID; ping returns an empty result.
• login_status works without login (logged_in: false).
• Browser tools (e.g. dblclick) return NOT_LOGGED_IN before login.
• batch dispatches without login; sub-commands enforce login individually (bug fix: batch was incorrectly gated by login — added to the login exception list in agent_tools_dispatch()).
• Roo Code MCP config alwaysAllow updated with all 100 tool names.

Implementation order

Batches 1-4 are all Pattern A (JS eval) or simple composites — no new WebKit API knowledge needed. They should be done first.

Batches 5-8 require WebKitGTK C API knowledge and are more complex. They should be done after the JS-based tools are stable.

Recommended execution order

1. Batch 1 (11 tools) — extended interaction, all JS eval
2. Batch 2 (7 tools) — get info + check state, all JS eval
3. Batch 3 (10 tools) — find elements, JS eval + ref integration
4. Batch 4 (5 tools) — wait + batch, polling + composite
5. Batch 5 (11 tools) — cookies + storage, mixed WebKit API + JS
6. Batch 6 (13 tools) — mouse + clipboard + settings, mixed
7. Batch 7 (10 tools) — frames + dialogs + debug, WebKit API
8. Batch 8 (3 tools) — complex tools, WebKit API

Files to modify

• src/agent_tools.c — all tool implementations + dispatch table entries
• src/agent_mcp.c — tool catalog entries (tool_defs[]) + schemas
• src/agent_tools.h — no changes needed (dispatch API unchanged)
• src/tab_manager.c / src/tab_manager.h — add tab_manager_close_all() for close_all
• src/agent_server.c — may need console/error message signal handlers
• ~/.config/VSCodium/User/globalStorage/rooveterinaryinc.roo-cline/settings/mcp_settings.json — add new tools to alwaysAllow
• plans/agent-tools.md — update Phase 3 status

Key design decisions

Find tools and ref integration

The find_* tools need to assign refs so results work with click @eN etc. This means they must register the found element in window.__agentRefs (the same mechanism the snapshot uses). The find tool will:

1. Run JS that finds the element and generates a unique CSS selector for it
2. Assign a new ref (incrementing counter from window.__agentRefCounter)
3. Store {selector: "...", role: "...", name: "..."} in window.__agentRefs[eN]
4. Return the ref to the caller

Batch tool

The batch tool takes an array of tool requests and executes them sequentially:

{"tool":"batch","params":{"commands":[
  {"tool":"open","params":{"url":"https://example.com"}},
  {"tool":"snapshot","params":{}},
  {"tool":"click","params":{"ref":"@e1"}}
]}}

Returns an array of responses. Each command is dispatched via agent_tools_dispatch(). If a command fails, the batch stops (unless continueOnError: true).

Console and error collection

Need to connect to WebKitWebView's console-message signal and notify::title or load-failed signals. Store messages in a per-webview GArray or GPtrArray. The console and errors tools return the collected messages and optionally clear them.

Dialog handling

WebKitGTK emits script-dialog signal for alert/confirm/prompt. We need to:

1. Connect to the signal
2. Store the pending dialog (type, message, default text)
3. The dialog_accept/dialog_dismiss tools call the appropriate webkit_script_dialog_*_set_*() function
4. dialog_status returns whether a dialog is pending and its details

Frame switching

WebKitGTK doesn't have a direct "switch to frame" API like Selenium. We'll track a "current frame" selector in C state. All JS eval tools will wrap their scripts to execute within the frame context:

(function() {
  var frame = document.querySelector('<frame_selector>');
  var doc = frame ? frame.contentDocument : document;
  // ... run tool JS within doc ...
})();

frame_main resets the frame selector to NULL (use main document).

Testing

After each batch:

1. make clean && make — verify build
2. Start browser, test each new tool with curl against /mcp
3. Verify tools/list shows the new tools
4. Update mcp_settings.json alwaysAllow array

Mermaid: Implementation flow

flowchart TB
    Start[Phase 3 Start] --> B1[Batch 1: Extended Interaction<br/>11 tools, JS eval]
    B1 --> B2[Batch 2: Get Info + Check State<br/>7 tools, JS eval]
    B2 --> B3[Batch 3: Find Elements<br/>10 tools, JS eval + refs]
    B3 --> B4[Batch 4: Wait + Batch<br/>5 tools, polling + composite]
    B4 --> B5[Batch 5: Cookies + Storage<br/>11 tools, WebKit API + JS]
    B5 --> B6[Batch 6: Mouse + Clipboard + Settings<br/>13 tools, mixed]
    B6 --> B7[Batch 7: Frames + Dialogs + Debug<br/>10 tools, WebKit API]
    B7 --> B8[Batch 8: Complex<br/>3 tools, WebKit API]
    B8 --> Final[Final: Build, Test, Update Config + Docs]
    Final --> Done[100 tools total<br/>30 Phase 1 + 70 Phase 3]