v0.0.65 - Release: fix segfault in settings sync — dispatch merge to main thread via g_idle_add to avoid data race on g_settings

VERSION

@@ -1 +1 @@
-0.0.63
+0.0.65

browser.sh

@@ -240,9 +240,18 @@ cmd_start() {
     fi
 
     if [[ "$software_rendering" -eq 1 ]]; then
-        echo "[browser] Software rendering detected — disabling WebKit compositor."
-        export WEBKIT_DISABLE_COMPOSITING_MODE=1
-        export WEBKIT_DISABLE_DMABUF_RENDERER=1
+        # On llvmpipe/softpipe, WebKit compositing is still faster than the
+        # non-composited fallback (confirmed by A/B testing on Qubes llvmpipe).
+        # We no longer disable compositing — just log a note.
+        # The DMABUF renderer is still disabled since there's no real GPU.
+        echo "[browser] Software rendering detected (llvmpipe/softpipe) — compositing kept enabled for performance."
+        if [[ -z "${WEBKIT_DISABLE_DMABUF_RENDERER:-}" ]]; then
+            export WEBKIT_DISABLE_DMABUF_RENDERER=1
+        fi
+        # Allow override via pre-set env var for A/B testing.
+        if [[ -z "${WEBKIT_DISABLE_COMPOSITING_MODE:-}" ]]; then
+            export WEBKIT_DISABLE_COMPOSITING_MODE=0
+        fi
     fi
 
     # Build first

plans/debug-responsiveness.md

@@ -0,0 +1,81 @@
+# Debugging Responsiveness — sovereign_browser
+
+## Symptoms (from user)
+1. **Hover lag**: a page that highlights buttons on mouseover responds ~2× slower than in Brave.
+2. **Load progress invisible until complete**: page loads "take a while" and show no progress until the download finishes.
+
+## Confirmed root-cause candidates (from code review)
+
+### A. Hover/scroll lag — WebKitGTK rendering pipeline
+[`src/tab_manager.c:2912`](src/tab_manager.c:2912) configures `WebKitSettings` but only sets:
+- `enable_developer_extras`, `enable_javascript`, `javascript_can_open_windows_automatically`
+- `allow_file_access_from_file_urls`, `allow_universal_access_from_file_urls`, `allow_modal_dialogs`
+
+**Not set** (left at WebKitGTK defaults):
+- `enable_accelerated_2d_canvas` (default FALSE on many builds)
+- `enable_smooth_scrolling` (default FALSE)
+- compositing mode — controlled by env var `WEBKIT_DISABLE_COMPOSITING_MODE=1`, which if set forces non-accelerated compositing
+
+WebKitGTK ships software-rendered by default on many distros; Chromium/Brave use GPU compositing + Skia by default. This alone explains ~2× paint lag on hover effects (each `:hover` triggers a repaint; software repaint is much slower).
+
+### B. No load-progress UI
+[`on_load_changed`](src/tab_manager.c:1635) handles `LOAD_STARTED` (→ stop icon), `LOAD_COMMITTED` (→ URL bar/title), `LOAD_FINISHED`/`LOAD_FAILED` (→ reload icon). There is **no** `notify::estimated-load-progress` signal handler and **no** progress bar/spinner in the tab strip. So between STARTED and FINISHED the user sees only a static stop icon — "nothing happens until done."
+
+## Debugging plan (ordered: cheap → invasive)
+
+### Step 1 — Confirm the rendering hypothesis with env vars (no code change)
+Run the browser with GPU compositing forced on and compare hover responsiveness:
+
+```bash
+WEBKIT_DISABLE_COMPOSITING_MODE=0 ./browser.sh start --login-method generate --url <hover-test-page>
+# vs
+WEBKIT_DISABLE_COMPOSITING_MODE=1 ./browser.sh start --login-method generate --url <hover-test-page>
+```
+
+Also try `WEBKIT_FORCE_SANDBOX=0` only if the above is inconclusive (rules out sandbox-overhead noise). If forcing compositing on closes the gap with Brave, the fix is enabling accelerated settings + ensuring the compositor is active.
+
+### Step 2 — Use the existing perf-probe to quantify
+The browser already ships [`www/js/perf-probe.js`](www/js/perf-probe.js) (long-task observer, rAF FPS, heartbeat main-thread-blockage fallback for WebKitGTK). Enable it (Settings → perf_probe_enabled, or it's gated by `settings_get()->perf_probe_enabled` at [`tab_manager.c:1337`](src/tab_manager.c:1337)) and load the hover-test page. Compare:
+- `cpu_busy_percent` (heartbeat blockage) — high = main thread is being starved
+- `fps` — low fps during hover = paint-bound, points at rendering pipeline
+- `active_timer_count` — high = page is timer-heavy (not our bug)
+
+Call the MCP `processes.tabs` tool ([`agent_mcp.c:583`](src/agent_mcp.c:583)) to read the probe values per tab while hovering.
+
+### Step 3 — System-level profiling to rule out the C/GTK main thread
+Even if rendering is the cause, confirm the GTK main thread isn't also blocked. Two cheap checks:
+
+```bash
+# a) Is the UI process CPU-bound during hover?
+top -p $(pgrep -f sovereign_browser) -H -d 0.5
+
+# b) perf record the UI process for 10s while hovering, then look at hotspots
+sudo perf record -p $(pgrep -f sovereign_browser) -g -- sleep 10
+sudo perf report --no-children
+```
+
+Look for time in `webkit*`/`WebCore*`/`cairo`/`GLib` dispatchers. If time is dominated by `g_main_context_*` or our own handlers, the C layer is implicated; if it's `WebCore::paint`/`cairo`, it's the renderer.
+
+### Step 4 — Check for main-thread blocking from sync JS eval
+[`agent_js_eval_sync`](src/agent_snapshot.c:80) runs a nested `g_main_loop` on the default context. The agent loop ([`agent_loop.c:333`](src/agent_loop.c:333)) calls this from a background thread while pumping the main loop. If an agent/MCP tool is running during normal browsing, this can re-dispatch unrelated sources and cause UI hitches. Verify by checking whether sluggishness only occurs while an agent loop is active (watch `[agent-loop] iter` log lines). If correlated, the fix is to route sync JS eval through a dedicated `GMainContext` instead of the default one.
+
+### Step 5 — Fix candidates (after diagnosis confirms which)
+
+**Rendering (likely):**
+- In [`src/tab_manager.c:2912`](src/tab_manager.c:2912) add:
+  ```c
+  webkit_settings_set_enable_accelerated_2d_canvas(settings, TRUE);
+  webkit_settings_set_enable_smooth_scrolling(settings, TRUE);
+  ```
+  and ensure `WEBKIT_DISABLE_COMPOSITING_MODE` is **not** set in the launch environment (check `browser.sh`).
+- Verify GPU is actually being used: `WEBKIT_DEBUG=compositing` or check `glxinfo`/EGL logs.
+
+**Load progress (confirmed gap):**
+- Add a `notify::estimated-load-progress` handler on each webview that drives a thin progress bar (or the tab label background) between `LOAD_STARTED` and `LOAD_FINISHED`. WebKitGTK emits this continuously during fetch; wiring it gives the missing "loading" feedback. Add the handler next to the `load-changed` connect at [`src/tab_manager.c:3136`](src/tab_manager.c:3136).
+
+## Recommended order
+1. Step 1 (env-var A/B test) — fastest signal, no rebuild.
+2. Step 2 (perf-probe) — quantifies, uses existing infra.
+3. Step 3 (perf/top) — rules out C-layer blocking.
+4. Step 4 — only if sluggishness correlates with agent activity.
+5. Step 5 — implement the fix(es) confirmed by 1–4.

plans/keyboard-shortcuts.md

@@ -131,6 +131,8 @@ A fixed enum of browser actions the user can bind. Each has: id, display label,
 | Action ID            | Label                  | Default binding           |
 |----------------------|------------------------|---------------------------|
 | `new_tab`            | New tab                | `<Control>t`              |
+| `new_window`         | New window             | `<Control>n`              |
+| `open_file`          | Open file              | `<Control>o`              |
 | `close_tab`          | Close tab              | `<Control>w`              |
 | `focus_url`          | Focus URL bar          | `<Control>l`              |
 | `next_tab`           | Next tab               | `<Control>Tab`            |

src/main.c

@@ -533,6 +533,10 @@ gboolean on_key_press(GtkWidget *widget, GdkEventKey *event,
         tab_manager_new_window_blank();
         return TRUE;
 
+    case SHORTCUT_OPEN_FILE:
+        tab_manager_open_file();
+        return TRUE;
+
     case SHORTCUT_CLOSE_TAB:
         tab_manager_close_active();
         return TRUE;

src/nostr_bridge.c

@@ -1255,6 +1255,10 @@ static void handle_settings_set(WebKitURISchemeRequest *request,
         }
         /* Trigger debounced NIP-78 sync. */
         settings_sync_publish();
+    } else if (strcmp(key, "perf_probe.enabled") == 0) {
+        bs->perf_probe_enabled = (strcmp(value, "true") == 0 ||
+                                   strcmp(value, "1") == 0);
+        settings_save();
     } else {
         respond_error_json(request, -1, "Unknown key");
         g_free(key); g_free(value);

src/relay_fetch.c

@@ -28,6 +28,10 @@
 #define MAX_RELAYS 32
 
 /* Relay query timeout in seconds. */
+
+/* Forward declaration — defined below. */
+static gboolean merge_settings_idle(gpointer data);
+
 #define RELAY_TIMEOUT_SECONDS 15
 
 /* Forward declaration — defined after relay_fetch_bootstrap. */
@@ -120,11 +124,18 @@ int relay_fetch_bootstrap(const char *pubkey_hex,
             /* Store in SQLite first, then try to merge into local settings.
              * settings_sync_merge_from_nostr checks the d-tag and only
              * merges if it's the shared "user-settings" event (or the
-             * legacy "sovereign_browser" event, which is migrated). */
+             * legacy "sovereign_browser" event, which is migrated).
+             *
+             * IMPORTANT: The merge calls settings_load() which writes to
+             * the global g_settings struct. This must run on the main
+             * thread to avoid a data race with the GTK main loop. */
             if (db_store_event(results[i]) == 0) {
                 stored++;
             }
-            settings_sync_merge_from_nostr(results[i]);
+            cJSON *event_copy = cJSON_Duplicate(results[i], 1);
+            if (event_copy) {
+                g_idle_add(merge_settings_idle, event_copy);
+            }
         } else {
             if (db_store_event(results[i]) == 0) {
                 stored++;
@@ -208,6 +219,19 @@ static gboolean avatar_refresh_idle(gpointer data) {
     return G_SOURCE_REMOVE;
 }
 
+/* Idle callback to merge NIP-78 settings on the main thread.
+ * settings_sync_merge_from_nostr calls settings_load() which writes to
+ * the global g_settings struct, so it must run on the GTK main thread
+ * to avoid data races with the main loop. Takes ownership of event_cjson. */
+static gboolean merge_settings_idle(gpointer data) {
+    cJSON *event = (cJSON *)data;
+    if (event) {
+        settings_sync_merge_from_nostr(event);
+        cJSON_Delete(event);
+    }
+    return FALSE;  /* run once */
+}
+
 /* ── Background thread ─────────────────────────────────────────────── */
 
 gpointer relay_fetch_thread(gpointer data) {

src/shortcuts.c

@@ -30,6 +30,10 @@ static const shortcut_meta_t g_registry[SHORTCUT_COUNT] = {
         "new_window", "New window",
         "Open a new browser window", "<Control>n"
     },
+    [SHORTCUT_OPEN_FILE] = {
+        "open_file", "Open file",
+        "Open a local file in the active tab", "<Control>o"
+    },
     [SHORTCUT_CLOSE_TAB] = {
         "close_tab", "Close tab",
         "Close the active tab", "<Control>w"

src/shortcuts.h

@@ -29,6 +29,7 @@ extern "C" {
 typedef enum {
     SHORTCUT_NEW_TAB = 0,
     SHORTCUT_NEW_WINDOW,
+    SHORTCUT_OPEN_FILE,
     SHORTCUT_CLOSE_TAB,
     SHORTCUT_FOCUS_URL,
     SHORTCUT_NEXT_TAB,

src/tab_manager.c

@@ -1632,6 +1632,30 @@ static gboolean on_refresh_button_press(GtkWidget *widget,
     return FALSE;  /* Let left-click propagate to "clicked" signal */
 }
 
+/* ── Load progress bar ──────────────────────────────────────────────── *
+ * WebKitGTK emits notify::estimated-load-progress continuously during
+ * page fetch. We drive a thin progress bar between LOAD_STARTED and
+ * LOAD_FINISHED so the user sees activity even when the page takes a
+ * while to download. The bar is hidden on LOAD_FINISHED / LOAD_FAILED. */
+static void on_load_progress_changed(WebKitWebView *webview,
+                                     GParamSpec    *pspec,
+                                     gpointer       user_data) {
+    (void)pspec;
+    tab_info_t *tab = (tab_info_t *)user_data;
+    if (tab == NULL || tab->progress_bar == NULL) return;
+
+    gdouble progress = webkit_web_view_get_estimated_load_progress(webview);
+    if (progress < 0.01) {
+        /* Not yet loading — keep hidden. */
+        return;
+    }
+
+    if (!gtk_widget_get_visible(tab->progress_bar)) {
+        gtk_widget_set_visible(tab->progress_bar, TRUE);
+    }
+    gtk_progress_bar_set_fraction(GTK_PROGRESS_BAR(tab->progress_bar), progress);
+}
+
 static void on_load_changed(WebKitWebView *webview,
                             WebKitLoadEvent load_event,
                             gpointer user_data) {
@@ -1699,6 +1723,12 @@ static void on_load_changed(WebKitWebView *webview,
         /* Restore the reload button visual now that loading is done. */
         tab_refresh_button_update(tab);
 
+        /* Hide the load-progress bar. */
+        if (tab != NULL && tab->progress_bar != NULL &&
+            gtk_widget_get_visible(tab->progress_bar)) {
+            gtk_widget_set_visible(tab->progress_bar, FALSE);
+        }
+
         /* Update tab title if already available (works for tabs in any
          * window's notebook). The notify::title handler covers the case
          * where the title arrives after load-finished. */
@@ -1739,6 +1769,12 @@ static gboolean on_load_failed(WebKitWebView *webview,
      * user clicks the X to cancel a load. */
     tab_refresh_button_update(tab);
 
+    /* Hide the load-progress bar on failure/cancellation. */
+    if (tab != NULL && tab->progress_bar != NULL &&
+        gtk_widget_get_visible(tab->progress_bar)) {
+        gtk_widget_set_visible(tab->progress_bar, FALSE);
+    }
+
     /* Keep the failed URL in the address bar so the user can see what
      * failed and edit/retry. Without this, WebKit reverts to about:blank
      * and the user loses the URL they typed or clicked. */
@@ -2105,9 +2141,11 @@ static void on_menu_toggle_sidebar(GtkMenuItem *item, gpointer data) {
     tab_manager_toggle_sidebar();
 }
 
-static void on_menu_open_file(GtkMenuItem *item, gpointer data) {
-    (void)item;
-    (void)data;
+/* Show the "Open File" dialog and load the chosen file into the active
+ * tab. Shared by the hamburger-menu callback and the Ctrl+O keyboard
+ * shortcut. Parents the dialog to the focused window so it appears over
+ * the window the user invoked it from. No-op if there is no window. */
+void tab_manager_open_file(void) {
     /* Parent the dialog to the currently focused window (g_active_window)
      * rather than always the main window (g_window), so the file chooser
      * appears over the window the user invoked it from. g_active_window is
@@ -2145,6 +2183,12 @@ static void on_menu_open_file(GtkMenuItem *item, gpointer data) {
     gtk_widget_destroy(dialog);
 }
 
+static void on_menu_open_file(GtkMenuItem *item, gpointer data) {
+    (void)item;
+    (void)data;
+    tab_manager_open_file();
+}
+
 static void on_menu_history_item(GtkMenuItem *item, gpointer data) {
     (void)data;
     tab_info_t *tab = tab_manager_get_active();
@@ -2916,6 +2960,13 @@ static tab_info_t *tab_create(const char *url) {
     webkit_settings_set_allow_file_access_from_file_urls(settings, TRUE);
     webkit_settings_set_allow_universal_access_from_file_urls(settings, TRUE);
     webkit_settings_set_allow_modal_dialogs(settings, TRUE);
+    /* Rendering performance: force hardware acceleration always on and
+     * enable smooth scrolling. These are off by default in WebKitGTK on
+     * many distros but significantly improve hover/scroll responsiveness
+     * even on llvmpipe software rendering. */
+    webkit_settings_set_hardware_acceleration_policy(settings,
+        WEBKIT_HARDWARE_ACCELERATION_POLICY_ALWAYS);
+    webkit_settings_set_enable_smooth_scrolling(settings, TRUE);
 
     /* Enhance native application/json documents at document end. The
      * embedded file is not NUL-terminated, so copy it before passing it to
@@ -3099,6 +3150,16 @@ static tab_info_t *tab_create(const char *url) {
     gtk_box_pack_start(GTK_BOX(tab->page), tab->bookmark_bar, FALSE, FALSE, 0);
     bookmark_bar_refresh(tab);
 
+    /* Thin load-progress bar between the bookmark bar and the webview.
+     * Hidden by default; shown during page loads via the
+     * notify::estimated-load-progress signal. */
+    tab->progress_bar = gtk_progress_bar_new();
+    gtk_progress_bar_set_show_text(GTK_PROGRESS_BAR(tab->progress_bar), FALSE);
+    gtk_widget_set_size_request(tab->progress_bar, -1, 3);
+    gtk_widget_set_no_show_all(tab->progress_bar, TRUE);
+    gtk_widget_set_visible(tab->progress_bar, FALSE);
+    gtk_box_pack_start(GTK_BOX(tab->page), tab->progress_bar, FALSE, FALSE, 0);
+
     /* Apply the active window's menu-bar visibility so a tab opened in a
      * hidden-chrome window is also hidden. The tab strip itself is
      * per-notebook (gtk_notebook_set_show_tabs), already set by the
@@ -3137,6 +3198,8 @@ static tab_info_t *tab_create(const char *url) {
                      G_CALLBACK(on_load_changed), tab);
     g_signal_connect(tab->webview, "load-failed",
                      G_CALLBACK(on_load_failed), tab);
+    g_signal_connect(tab->webview, "notify::estimated-load-progress",
+                     G_CALLBACK(on_load_progress_changed), tab);
     g_signal_connect(tab->webview, "notify::favicon",
                      G_CALLBACK(on_favicon_changed), tab);
     g_signal_connect(tab->webview, "notify::title",

src/tab_manager.h

@@ -31,6 +31,7 @@ typedef struct {
     GtkWidget     *refresh_btn;  /* toolbar reload/stop button (per-tab) */
     GtkWidget     *toolbar;      /* #main-toolbar (hamburger + nav + URL) */
     GtkWidget     *bookmark_bar; /* bookmarks toolbar (below URL toolbar) */
+    GtkWidget     *progress_bar; /* thin load-progress bar (below bookmark bar, above webview) */
     gboolean       refresh_shows_stop; /* cached visual state, avoids redundant updates */
     char           current_url[TAB_URL_MAX];
     char           title[TAB_TITLE_MAX];
@@ -80,6 +81,14 @@ void tab_manager_open_in_new_window(int index);
  */
 void tab_manager_new_window_blank(void);
 
+/*
+ * Show the "Open File" dialog and load the chosen local file into the
+ * active tab via a file:// URI. The dialog is parented to the focused
+ * window. Used by the hamburger-menu "Open File…" item and the Ctrl+O
+ * keyboard shortcut. No-op if there is no window or active tab.
+ */
+void tab_manager_open_file(void);
+
 /*
  * Close the tab at the given index. If it's the last tab, the window
  * destroy handler will fire (caller is responsible for quitting).

src/version.h

@@ -11,9 +11,9 @@
 #ifndef SOVEREIGN_BROWSER_VERSION_H
 #define SOVEREIGN_BROWSER_VERSION_H
 
-#define SB_VERSION         "v0.0.63"
+#define SB_VERSION         "v0.0.65"
 #define SB_VERSION_MAJOR   0
 #define SB_VERSION_MINOR   0
-#define SB_VERSION_PATCH   63
+#define SB_VERSION_PATCH   65
 
 #endif /* SOVEREIGN_BROWSER_VERSION_H */

tests/local-site/hover-test.html

@@ -0,0 +1,123 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="UTF-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>Hover Responsiveness Test</title>
+<style>
+  * { margin: 0; padding: 0; box-sizing: border-box; }
+  body {
+    font-family: sans-serif;
+    background: #1a1a2e;
+    color: #eee;
+    padding: 40px;
+  }
+  h1 { margin-bottom: 10px; }
+  p { margin-bottom: 30px; color: #aaa; }
+
+  .grid {
+    display: grid;
+    grid-template-columns: repeat(auto-fill, minmax(180px, 1fr));
+    gap: 16px;
+    max-width: 1000px;
+  }
+
+  .card {
+    background: #16213e;
+    border: 2px solid #0f3460;
+    border-radius: 12px;
+    padding: 24px 16px;
+    text-align: center;
+    cursor: pointer;
+    transition: background 0.05s, border-color 0.05s, transform 0.05s, box-shadow 0.05s;
+    user-select: none;
+  }
+
+  /* Hover effect: quick background + border + shadow change */
+  .card:hover {
+    background: #e94560;
+    border-color: #e94560;
+    transform: scale(1.05);
+    box-shadow: 0 0 20px rgba(233, 69, 96, 0.5);
+  }
+
+  .card:active {
+    transform: scale(0.98);
+  }
+
+  .card .icon {
+    font-size: 32px;
+    margin-bottom: 8px;
+  }
+  .card .label {
+    font-size: 14px;
+    font-weight: 600;
+  }
+
+  #fps-display {
+    position: fixed;
+    top: 12px;
+    right: 12px;
+    background: rgba(0,0,0,0.7);
+    color: #0f0;
+    font-family: monospace;
+    font-size: 18px;
+    padding: 6px 12px;
+    border-radius: 6px;
+    z-index: 999;
+  }
+
+  #instructions {
+    position: fixed;
+    bottom: 20px;
+    left: 50%;
+    transform: translateX(-50%);
+    background: rgba(0,0,0,0.8);
+    color: #ff0;
+    padding: 10px 20px;
+    border-radius: 8px;
+    font-size: 14px;
+    z-index: 999;
+  }
+</style>
+</head>
+<body>
+
+<div id="fps-display">FPS: --</div>
+<div id="instructions">Move mouse rapidly across the cards — watch hover highlight latency</div>
+
+<h1>🖱️ Hover Responsiveness Test</h1>
+<p>Each card has a <code>transition: 0.05s</code> on background, border, transform, and box-shadow.
+Compare how quickly the hover highlight appears vs Brave/Chrome.</p>
+
+<div class="grid" id="grid"></div>
+
+<script>
+  // Generate 60 cards
+  var grid = document.getElementById('grid');
+  var icons = ['🌟','🔥','💎','⚡','🎯','🚀','🌈','🍀','🎨','🦋','🌺','⭐','💫','✨','🎪','🏆','🥇','💡','🔮','🎭'];
+  for (var i = 0; i < 60; i++) {
+    var card = document.createElement('div');
+    card.className = 'card';
+    card.innerHTML = '<div class="icon">' + icons[i % icons.length] + '</div><div class="label">Card ' + (i+1) + '</div>';
+    grid.appendChild(card);
+  }
+
+  // Simple FPS counter
+  var fpsEl = document.getElementById('fps-display');
+  var frameCount = 0;
+  var lastFpsTime = performance.now();
+  function rafLoop() {
+    frameCount++;
+    var now = performance.now();
+    if (now - lastFpsTime >= 1000) {
+      fpsEl.textContent = 'FPS: ' + frameCount;
+      frameCount = 0;
+      lastFpsTime = now;
+    }
+    requestAnimationFrame(rafLoop);
+  }
+  requestAnimationFrame(rafLoop);
+</script>
+</body>
+</html>