mirror of
https://github.com/vitorpamplona/amethyst.git
synced 2026-08-09 08:04:45 +00:00
Merge pull request #3378 from vitorpamplona/fix/embed-ime-selection
Embedded text selection: native-parity IME + selection UI + magnifier, and platform-bug fixes
This commit is contained in:
Vitor Pamplona
@@ -0,0 +1,256 @@
|
||||
# Embedded text selection — native-Android parity
|
||||
|
||||
**Status:** core feature-complete. The working set landed in `fix(embed): IME typing +
|
||||
host-drawn text selection for embedded surfaces` (commit `e0a2a9ab81`); subsequent
|
||||
sessions added the magnifier, the `SelectionUiState` refactor, hybrid word+char
|
||||
handle-extend, and a run of polish/bug fixes (below). **The one open platform bug —
|
||||
full-screen round-trip kills selection paint — is now FIXED** (root cause was
|
||||
process-global `pauseTimers()` + an attached `WebView.destroy()`; see that section).
|
||||
|
||||
**2026-06-26 fixes (branch `fix/embed-ime-selection`):**
|
||||
- **No-blink word-select** — the overlay handles + toolbar blinked 2–3× on long-press
|
||||
word-select; cause was the shim's selection-*reveal* scrolls (a `<textarea>` auto-
|
||||
scrolling to show a forming/re-asserted range) tripping the hide-on-scroll path. The
|
||||
shim now timestamps selection activity (`lastSelActivityAt`) and treats a scroll within
|
||||
350 ms as a reveal-scroll (reposition, don't hide, don't re-arm the timer). Plus a
|
||||
`RemoteImeView` range-lost debounce as a safety net. (#2, #3)
|
||||
- **Page selection clears on field focus** — focusing a field left the page-text handles
|
||||
+ Copy bar up (the shim's page `selectionchange` is muted once a field is focused), and
|
||||
being z-above they STOLE the field handle's drag. Fixed: `focusin` emits `pagesel:false`
|
||||
(+ resets the scroll state); host `ImeEvent.Focus` also drops the page overlay. (#2)
|
||||
- **Caret handle drag** moved the loupe but not the caret — the unified `awaitEachGesture`
|
||||
did `change.consume()` BEFORE `change.positionChange()`, and `positionChange()` returns
|
||||
`Offset.Zero` once consumed, so `fp` never accumulated. Fixed with
|
||||
`positionChangeIgnoreConsumed()` (also immune to the sandbox surface consuming the move). (#10)
|
||||
- **Hybrid word+char handle-extend** completed (#5, below).
|
||||
- **Full-screen round-trip corruption FIXED** (was the open bug; see section).
|
||||
- **Embed WebViews follow the APP theme** — separate from selection, but same surfaces:
|
||||
see `embed-webview-prefers-color-scheme-limitation` memo / `EmbedWebViewTheme.kt`.
|
||||
|
||||
## Why we draw selection ourselves
|
||||
|
||||
Editable fields inside an embedded napplet/nsite/browser tab live in the keyless
|
||||
`:napplet` process and render through `SurfaceControlViewHost` /
|
||||
`SandboxedSdkView` (privacy-sandbox UI). A WebView rendered into an off-window
|
||||
surface like this **cannot host the soft keyboard and cannot present Chrome's
|
||||
own text-selection UI** (handles, the floating action-mode toolbar, the
|
||||
magnifier). Chrome detects it has nowhere to put that UI and collapses the
|
||||
selection to the focus endpoint.
|
||||
|
||||
So, exactly like Flutter's `TextInputPlugin` did for its virtual-display era, we
|
||||
relay editing to the main process: an invisible `EditText` (`RemoteImeView`)
|
||||
hosts the keyboard, `shim.js` mirrors DOM selection/caret geometry out over the
|
||||
Messenger channel, and `EmbeddedTabLayer` draws the selection UI in Compose on
|
||||
top of the surface. Everything we want for parity, we draw — the platform gives
|
||||
us nothing here.
|
||||
|
||||
## What native Android gives a text field (the parity target)
|
||||
|
||||
This is the full feature inventory we are cloning, with activation/deactivation
|
||||
rules, so we can check off coverage. Native impl lives in `android.widget.Editor`
|
||||
(+ `SelectionActionModeHelper`, `android.widget.Magnifier`,
|
||||
`PopupTouchHandleDrawable` on the Chrome side).
|
||||
|
||||
| # | Native feature | Activates | Deactivates | Our status |
|
||||
|---|----------------|-----------|-------------|------------|
|
||||
| 1 | **Insertion handle** (the teardrop "blob" under the caret) | tap in editable text; tap again to re-show | typing, scroll start, focus loss, ~4s inactivity timeout | ✅ `InsertionHandle`. **Native availability rule now matched (2026-06-25):** only shown when the field is NON-EMPTY (`Editor` gates the handle behind `text.length() > 0`, via `SelectionUiState.fieldHasText`) — fixes it popping up on focus of an empty box; hides on typing (`onEdited`), scroll (`scrolling`), focus loss, and ~4s inactivity (`hideCaret` timeout), re-showing on the next tap — via an explicit `ime.carettap` shim signal (DOM `click`), so a tap that doesn't move the caret still re-shows it. Device-verified. |
|
||||
| 2 | **Selection handles** (asymmetric left/right teardrops) | long-press word, double-tap word, drag-extend | tap-collapse, typing, new selection | ✅ `SelectionHandle(isStart)` + drag-to-extend, for BOTH plain page text (`pageExtend`) AND in-field `<input>`/`<textarea>` selections (`fieldExtend`, 2026-06-25). The shim reports the selection's caret feet (`sx/sb`,`ex/eb`, flagged `rng`) via the same mirror-div as the caret; the host holds the range geometry separately so Chrome's transient collapse-to-caret (the re-assert fight) doesn't yank the handles to the field edges. **Tap-to-collapse (2026-06-25):** a single tap inside a selection dismisses it to a caret at the tapped offset + insertion handle — the shim's `click` handler collapses explicitly via `offsetFromPoint` (off-window Chrome doesn't do it itself). Device-verified. |
|
||||
| 3 | **Floating toolbar** (Cut/Copy/Paste/Select-All/Share/…) | selection made, or tap insertion handle (Paste/Select-All) | scroll/fling (hides, returns on settle), handle drag (hides), tap-collapse | ⚠️ `EmbeddedSelectionToolbar` (Cut/Copy/Paste/Select-All). **Hide-during-handle-drag ✅ device-verified.** Hide-during-scroll via `SelectionUiState.scrolling` (shim `ime.scroll` + re-report on settle). **2026-06-26: the scroll path was hardened** — selection-*reveal* scrolls (forming/re-asserting a range auto-scrolls a `<textarea>`) are no longer treated as user scrolls (they blinked the overlays); the shim guards them via `lastSelActivityAt` and the hide self-heals instead of re-arming. (User content-scroll-hide still wants a clean on-device pass.) Still missing: overflow, Share/Web-Search/process-text. |
|
||||
| 4 | **Magnifier / loupe** (the zoom bubble above the finger while dragging a handle or the caret) | finger down + moving on a handle or the caret | finger up | ✅ **Built (2026-06-25).** `Magnifier` bubble in [EmbeddedMagnifier.kt] follows the dragged caret/selection handle, showing live magnified page pixels captured in the `:napplet` provider and shipped over IPC ([EmbeddedMagnifierProbe], option B). Both embed paths wired (browser + napplet); verified on device for the browser path. Capture Y is locked to the caret/selection line (X follows the finger). Possible further polish: RGB_565 to cut encode, themed crosshair, clamp capture X to the line so a fast drag past EOL doesn't show blank. |
|
||||
| 5 | **Word-granularity long-press** then char-extend | long-press | — | ✅ HYBRID word+char in-field handle-extend (2026-06-25, completed): `fieldExtend` keeps per-drag state (`fieldDragWordEnd`/`fieldDragWordStart`, reset on a >250ms gap or edge switch). The drag baselines at the current selection edge; sweeping PAST that word's far boundary snaps to the next whole word (`wordEndAt`/`wordStartAt`), while moving within/back from the furthest-reached word gives CHARACTER precision — so you can fine-tune to a single character (the previously-missing "then char" mode). Page-text extend stays char (no offset model). |
|
||||
| 6 | **Double-tap = word, long-press = word, (triple-tap/drag = paragraph)** | tap count | — | ✅ double-tap + long-press both select a word (2026-06-25). Chrome word-selects on the 2nd tap, then abandons it by collapsing to the end (off-window quirk); the host re-assert restores it. The shim `click` handler DEFERS its tap-to-collapse ~300ms and the real `dblclick` cancels that timer, so the word selection survives (a timing-only guard was flaky ~40%). Triple-tap/paragraph not done. |
|
||||
| 7 | **Smart selection / entity expansion** (`TextClassifier`: phone, URL, address, date → entity actions in toolbar) | selection lands on an entity | — | ❌ not built (low priority) |
|
||||
| 8 | **Drag selected text** (long-press a selection → drag-and-drop to move) | long-press on existing selection | drop | ❌ not built (low priority) |
|
||||
| 9 | **Auto-scroll while dragging to a viewport edge** | handle dragged near top/bottom edge | finger leaves edge / up | ✅ works (2026-06-25, user-confirmed). The drag driver (`onMagnify`) detects the finger in the surface's top/bottom edge zone and sends `ime.autoscroll`; the shim scrolls the textarea (else the window) and re-reports geometry, flagged so the hide-on-scroll path doesn't fire. Scrolls per drag-move in the edge zone (not on a perfectly-held finger). **Fixed alongside:** the nav drawer's left-edge swipe was hijacking the edge drag — `EmbeddedSelectionDrag.dragging` (set by `onMagnify`) now suspends the drawer's `gesturesEnabled` while a handle is dragged. |
|
||||
| 10 | **Caret snapping to character boundaries** | always during caret/handle drag | — | ✅ via `offsetFromPoint` binary search + Y-clamp. **2026-06-26 fix:** the insertion-handle drag stopped moving the caret (loupe showed, caret frozen) — the unified `awaitEachGesture` consumed the pointer change BEFORE reading `positionChange()`, which returns `Offset.Zero` once consumed, so the accumulated finger position never advanced. Now reads `positionChangeIgnoreConsumed()` first. |
|
||||
| 11 | **Themed handle/caret drawables + blink** | always | — | ✅/⚠️ The host-drawn handles use `colorScheme.primary` — which IS the native `textSelectHandle`/accent color — so the handle drawables are themed (the actionable part). The caret bar + selection-highlight are drawn by Chrome inside the off-window surface: the caret already blinks natively, and theming its color/the highlight would mean injecting CSS into arbitrary third-party pages (intrusive; `::selection` was already ruled out as non-painting), so those are intentionally left to Chrome. |
|
||||
| 12 | **Insertion-handle Paste/Select-All mini-popup** | tap the insertion handle | tap elsewhere | ✅ built (2026-06-25). Tapping the bare insertion handle toggles a Paste/Select-All bar above the caret (`SelectionUiState.insertionPopup`, toggled from the handle's unified tap/drag gesture); tap-elsewhere/typing/blur/selection/scroll dismiss it. Fixed a latent bug: toolbar items now consume the *down* (not just the up) so the tap doesn't bleed through to the surface and blur the field. Device-verified (Select-all selects all text, field stays focused). |
|
||||
|
||||
Legend: ✅ done · ⚠️ partial · ❌ missing.
|
||||
|
||||
### Activation/deactivation is the hard part
|
||||
|
||||
Most of the bugs we already fixed were activation-timing bugs (cursor-jumps-to-end,
|
||||
collapse-on-tap, focus-transfer races). The remaining features each carry their
|
||||
own state machine.
|
||||
|
||||
**Done (2026-06-25): `SelectionUiState`** (`SelectionUiState.kt`) now centralizes
|
||||
what used to be scattered flags in `EmbeddedTabLayer` (`showInsertionHandle`,
|
||||
`showSelectionToolbar`, `fieldGeometry`, `rangeFieldGeometry`, `pageSelection`).
|
||||
It holds the three mutually-exclusive contexts (insertion caret / in-field range /
|
||||
page-text range) plus the transient modifiers `dragging` and `scrolling`, and
|
||||
exposes derived visibility (`insertionHandle`, `fieldHandles`, `fieldToolbar`,
|
||||
`pageHandles`, `pageToolbar`) so the rules are expressed once:
|
||||
- **toolbar hides while a handle is dragged** (`dragging`, set from the same
|
||||
`OnMagnify` lifecycle that drives the loupe) — the dragged handle also hides its
|
||||
own teardrop (the loupe stands in), like Android; the other handle stays.
|
||||
- **all overlays hide while scrolling** (`scrolling`, from the shim's `ime.scroll`);
|
||||
the shim re-reports geometry just before `active=false` so they reappear
|
||||
repositioned.
|
||||
Still emergent / TODO: insertion-handle auto-timeout, tap-to-re-show.
|
||||
|
||||
**Selection-blink fix (2026-06-25).** A field selection — especially in a `<textarea>` —
|
||||
flickered: off-window Chrome abandons the selection by collapsing the caret to an
|
||||
endpoint every ~25 ms, and the FIELD re-assert round-tripped through the host EditText
|
||||
(`RemoteImeView.onPageState` → `ime.set` → page), leaving a visible collapsed frame each
|
||||
cycle. Fix: re-assert field selections **synchronously in the shim's `selectionchange`
|
||||
handler** (mirror of the page-text path that never blinked) — `lastFieldRange`/`lastFieldAt`
|
||||
tracked via `noteSel()`, and a collapse-to-endpoint within 1500 ms is reverted with `setSel`
|
||||
(guarded, not re-reported) so it reverts before paint. The host re-assert stays as a
|
||||
fallback. Device-verified: textarea selection is stable; input select still works.
|
||||
|
||||
## Priority order for parity work
|
||||
|
||||
1. **Magnifier (#4).** Biggest perceived gap. We already report caret/handle
|
||||
geometry; the magnifier needs a *magnified pixel view of the surface* at the
|
||||
drag point. Options:
|
||||
- **A. Compose-side zoom of a surface snapshot. ❌ RULED OUT (spiked 2026-06-25).**
|
||||
The surface is a `SurfaceControlViewHost` — we can't trivially `Bitmap`-grab a
|
||||
remote surface from the main process. We spiked `PixelCopy.request(SurfaceView, …)`
|
||||
against the live embedded surface (`SurfaceMagnifierProbe`, wired into
|
||||
`EmbeddedTabLayer` behind `BuildConfig.DEBUG`, fired on field focus). The capture
|
||||
target is the privacysandbox `ContentView extends SurfaceView` — the only real
|
||||
child of `SandboxedSdkView` once the session opens. **Result on a clearly-painted
|
||||
surface (1080×2088): every capture returns `ERROR_SOURCE_NO_DATA`** (center
|
||||
region, repeated 3×). Cause: the WebView pixels live in a *child* `SurfaceControl`
|
||||
reparented under the SurfaceView via `ContentView.setChildSurfacePackage(...)`; the
|
||||
host SurfaceView's *own* buffer is never drawn into, so `PixelCopy` on the parent
|
||||
reads an empty buffer. Host-side pixel capture of the sandboxed content is not
|
||||
available. (A `PixelCopy.request(Window, …)` against the host window would also
|
||||
miss it — the sandbox layer is a *separate* SurfaceControl z-ordered below the
|
||||
window.)
|
||||
- **B. Capture in `:napplet` and ship the loupe content. ✅ SPIKED & VIABLE
|
||||
(2026-06-25).** Inside the keyless provider the WebView IS a real in-window view, so
|
||||
`WebView.draw(Canvas)` into a software bitmap renders real DOM pixels. Spike added
|
||||
`MSG_MAGNIFIER_REQUEST`/`MSG_MAGNIFIER_FRAME` to `NappletBrowserContract`:
|
||||
`NappletBrowserService.onMagnifierRequest` draws a zoomed slice
|
||||
(`canvas.scale(zoom); translate(-(cx-box/2), -(cy-box/2)); webView.draw(canvas)`),
|
||||
PNG-encodes it, and ships the bytes back; `EmbeddedBrowserController` (now also an
|
||||
`EmbeddedMagnifierProbe`) requests on focus and `EmbeddedTabLayer` logs the result.
|
||||
**10-frame burst, 160px source × 1.5× zoom → 240×240 PNG, center `#FF111111`
|
||||
(real opaque content):** provider draw 0.7–1.2 ms steady (≈5 ms cold), provider
|
||||
total draw+PNG 3–4 ms steady (≈12 ms cold), client round-trip 4–8 ms steady
|
||||
(occasional ~18 ms), payload 8–15 KB (far under the 1 MB Binder limit). Comfortably
|
||||
within a frame budget if throttled to ~30 fps.
|
||||
|
||||
**✅ Real loupe shipped (2026-06-25).** `MagnifierUiState` + `Magnifier`
|
||||
(`EmbeddedMagnifier.kt`); the caret/selection handles call an `OnMagnify` callback
|
||||
on drag start/move/end; `EmbeddedTabLayer` tracks the drag point, throttles capture
|
||||
requests to one in flight (100 ms timeout), decodes each `MagnifierFrame` to an
|
||||
`ImageBitmap`, and floats the bubble above the finger (clamped, flips below near the
|
||||
top). Capture mirrored onto BOTH embed paths (browser:
|
||||
`NappletBrowserContract`/`NappletBrowserService`/`EmbeddedBrowserController`;
|
||||
napplet: `NappletEmbedContract`/`NappletHostService`/`EmbeddedNappletController`).
|
||||
Verified on device (browser): drag → bubble shows live magnified "ello world" with
|
||||
the caret, centered on the line → follows finger → hides on release. The dead
|
||||
Option-A probe (`SurfaceMagnifierProbe`) was removed. **Polish done (2026-06-25):**
|
||||
capture Y is locked to the authoritative caret/selection line (the handles pass a
|
||||
`lineHalfPx` so the box centers on the line, not the finger or the caret foot); X
|
||||
still follows the finger. Remaining nice-to-haves: RGB_565/raw to cut PNG encode,
|
||||
themed crosshair, clamp capture X to the line so a fast drag past EOL isn't blank,
|
||||
reuse one off-screen bitmap.
|
||||
- **Gesture-routing caveat (found during the spike).** The host-drawn handles'
|
||||
drag is fragile: the sandbox `ContentView.onTouchEvent` always returns `true`, so
|
||||
via Compose's `AndroidView` interop it consumes the drag-move pointer and cancels
|
||||
the overlay handle's `detectDragGestures` (synthetic `adb` drags on the handle
|
||||
never produced an `onDrag`). The magnifier trigger should ride the existing
|
||||
caret/selection-move path (which already round-trips through the shim), not a fresh
|
||||
Compose drag layered over the surface.
|
||||
2. **Toolbar state rules (#3 hide-during-drag/scroll) + insertion-handle popup
|
||||
(#12).** Pure Compose/state work, no platform unknowns. Do alongside the
|
||||
`SelectionUiState` refactor.
|
||||
3. **Handle inactivity timeout + tap-to-re-show (#1).** Small.
|
||||
4. **Double-tap-to-select (#6)** and **word-granularity drag (#5).**
|
||||
5. **Auto-scroll (#9), themed drawables/blink (#11).**
|
||||
6. Defer: smart selection (#7), drag-to-move (#8).
|
||||
|
||||
## ✅ FIXED — full-screen round-trip corrupted the embedded WebViews (2026-06-26)
|
||||
|
||||
**Symptom (was).** Open an embedded field's page in its own full-screen activity,
|
||||
then `back` to the embedded version. From then on **every** embedded surface in the
|
||||
`:napplet` process was broken — and it was far more than the selection highlight: DOM
|
||||
reads returned empty (a field that visibly showed text reported `value == ""`, so typing
|
||||
prepended at offset 0 and backspace did nothing), DNS died (`ERR_NAME_NOT_RESOLVED`), the
|
||||
selection highlight stopped painting, and IME broke. The page showed a stale last frame
|
||||
over a functionally-dead renderer.
|
||||
|
||||
**Root cause — two process-global defects in the full-screen hosts** corrupting the
|
||||
shared multiprocess WebView state the embedded surfaces rely on. (Diagnosed with temporary
|
||||
logging: page console → logcat, all `onReceivedError`/`onReceivedHttpError`, and the shim's
|
||||
focused-field type/caret. The user's own insight — "the activity is gone but the service
|
||||
doesn't come back; am I using something from the activity?" — pointed straight at it.)
|
||||
|
||||
1. **`pauseTimers()`/`resumeTimers()` are PROCESS-GLOBAL** (they pause JS, layout and
|
||||
parsing timers for *every* WebView in the process). `NappletBrowserActivity` /
|
||||
`NappletHostActivity` `onPause`/`onResume` and `NappletHostService`'s embed
|
||||
pause/resume all called them on their own lifecycle — so returning from full-screen
|
||||
*froze* the embedded surfaces, which had no resume of their own. **Fix:** removed ALL
|
||||
process-global timer calls; rely only on per-WebView `onPause()`/`onResume()` (which
|
||||
pause just that surface's JS/DOM — still meets the napplet background-security goal).
|
||||
2. **`WebView.destroy()` while still attached to the window** corrupts the shared
|
||||
multiprocess renderer (`cr_AwContents: "WebView.destroy() called while WebView is still
|
||||
attached to window"`). **Fix:** `stopLoading()` + `(parent as ViewGroup).removeView(...)`
|
||||
before `destroy()` in both full-screen activities' `onDestroy`.
|
||||
|
||||
Device-verified: the round-trip no longer corrupts the embeds. This supersedes the old
|
||||
"recreate the session on return" hypothesis and the ruled-out attempts (`::selection` CSS,
|
||||
surface resize, focus-cycle, `MSG_WAKE`) — none were the real cause.
|
||||
|
||||
## ✅ Embed WebViews follow the app theme (2026-06-26, separate concern)
|
||||
|
||||
Not text-selection, but the same off-window surfaces: embedded (and full-screen) WebViews
|
||||
rendered web content in the *device* theme, ignoring the app's DARK/LIGHT preference.
|
||||
**Root cause:** WebView's dark decision (`prefers-color-scheme` via algorithmic darkening)
|
||||
reads the context's **theme** (`?android:attr/isLightTheme`), NOT just `Configuration.uiMode`
|
||||
— and the off-window `SurfaceControlViewHost` surface context carries neither. The old
|
||||
`applyNightMode` used `UiModeManager.setNightMode` (permission-gated no-op). **Fix:** build
|
||||
every WebView from `nightThemedContext()` — `ContextThemeWrapper(createConfigurationContext(
|
||||
<night|day>), Theme.DeviceDefault.DayNight)` for the resolved theme — shared in
|
||||
`nappletHost/.../EmbedWebViewTheme.kt`, used by both embed services + both full-screen
|
||||
activities. The full debugging arc (config-only context fails; `setForceDark` gone at
|
||||
targetSdk 37; `setApplicationNightMode` does nothing; it's the unthemed context, not the
|
||||
process boundary) is in the `embed-webview-prefers-color-scheme-limitation` memo. Upstream
|
||||
WebView is still buggy here (a cross-process SCVH WebView ignores `uiMode`); the
|
||||
theme-wrapper is the app-side workaround.
|
||||
|
||||
## How to test
|
||||
|
||||
The on-device harness lives at **`tools/ime-test/`** (`index.html` + `README.md`).
|
||||
It's a single page with an `<input>`, a `<textarea>`, and an on-page log that
|
||||
timestamps focus/selection/input/composition events, **paint latency**,
|
||||
long-tasks, and main-thread blocks — the instrumentation that pinned the erase,
|
||||
caret-jump, and first-letter-freeze bugs, and exactly what we'll want when
|
||||
profiling the magnifier.
|
||||
|
||||
Run it (full details in `tools/ime-test/README.md`):
|
||||
|
||||
1. `cd tools/ime-test && python3 -m http.server 8765`
|
||||
2. Reach it: emulator → `http://10.0.2.2:8765`; USB device →
|
||||
`adb reverse tcp:8765 tcp:8765` then `http://localhost:8765`.
|
||||
3. Open that URL in the **in-app browser** to load it as an *embedded* tab. (Opening
|
||||
the same URL full-screen and pressing `back` used to reproduce the
|
||||
highlight/corruption bug — now fixed; it's still the regression test for it.)
|
||||
|
||||
Console log lines are tagged `[ImeDiag]` and surface in `adb logcat` (the
|
||||
`:napplet` process owns the WebView console). This is a dev tool — nothing under
|
||||
`tools/` ships, which is why those diagnostic strings are kept out of `src/`.
|
||||
|
||||
## Key files
|
||||
|
||||
- `commons/src/commonMain/composeResources/files/napplet/shim.js` — DOM bridge.
|
||||
Geometry sources: `caretCoords` (287), `offsetFromPoint` (318), `fieldGeom`
|
||||
(336), `reportState` (360), `pageGeom` (462), `sendPageSel` (470), `pageExtend`
|
||||
(494). A magnifier built via option B would add a loupe-render here.
|
||||
- `amethyst/.../embed/EmbeddedTabLayer.kt` — the Compose overlay. `InsertionHandle`
|
||||
(557), `SelectionHandle` (496), `EmbeddedSelectionToolbar` (616),
|
||||
`PageSelectionOverlay` (460), the `showInsertionHandle`/`showSelectionToolbar`
|
||||
state to be folded into a `SelectionUiState`. Magnifier popup (option A) lands
|
||||
here.
|
||||
- `amethyst/.../embed/RemoteImeView.kt` — invisible host `EditText`; selection
|
||||
re-assert + copy/cut/paste/select-all + edit callbacks.
|
||||
- `amethyst/.../embed/EmbeddedImeBridge.kt` — `SelectionGeometry` (caret + handle
|
||||
feet + viewport), `ImeEvent.{Focus,State,PageSelection}`, `parseSelectionGeometry`.
|
||||
- `amethyst/.../{browser/EmbeddedBrowserController,favorites/EmbeddedNappletController}.kt`
|
||||
— parse `ime.pagesel` + geometry off the Messenger channel.
|
||||
- Context: `amethyst/plans/2026-06-19-napplet-sandbox-host.md`,
|
||||
`2026-06-24-napplet-embedded-tabs.md`.
|
||||
+9
-3
@@ -44,6 +44,7 @@ import com.vitorpamplona.amethyst.ui.navigation.drawer.DrawerContent
|
||||
import com.vitorpamplona.amethyst.ui.navigation.navs.Nav
|
||||
import com.vitorpamplona.amethyst.ui.navigation.routes.Route
|
||||
import com.vitorpamplona.amethyst.ui.screen.AccountSessionManager
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedSelectionDrag
|
||||
import kotlinx.coroutines.launch
|
||||
|
||||
@OptIn(ExperimentalMaterial3Api::class)
|
||||
@@ -90,9 +91,14 @@ fun AccountSwitcherAndLeftDrawerLayout(
|
||||
dest.hasRoute<Route.Home>() || dest.hasRoute<Route.Message>()
|
||||
} ?: false
|
||||
val drawerGesturesEnabled =
|
||||
!isTabPagerRoute ||
|
||||
nav.drawerState.isOpen ||
|
||||
nav.drawerState.targetValue != nav.drawerState.currentValue
|
||||
(
|
||||
!isTabPagerRoute ||
|
||||
nav.drawerState.isOpen ||
|
||||
nav.drawerState.targetValue != nav.drawerState.currentValue
|
||||
) &&
|
||||
// Suspend the left-edge swipe while a selection/caret handle is dragged over an embedded surface,
|
||||
// so a handle drag near the left edge (or the auto-scroll edge drag) doesn't open the drawer.
|
||||
!EmbeddedSelectionDrag.dragging
|
||||
|
||||
ModalNavigationDrawer(
|
||||
drawerState = nav.drawerState,
|
||||
|
||||
+48
@@ -31,6 +31,7 @@ import android.os.IBinder
|
||||
import android.os.Looper
|
||||
import android.os.Message
|
||||
import android.os.Messenger
|
||||
import android.os.SystemClock
|
||||
import androidx.annotation.RequiresApi
|
||||
import androidx.compose.runtime.mutableStateListOf
|
||||
import androidx.privacysandbox.ui.client.SandboxedUiAdapterFactory
|
||||
@@ -41,8 +42,11 @@ import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.ConsoleBridge
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.ConsoleLogEntry
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedImeBridge
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedLoadStatus
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedMagnifierProbe
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedSurfaceController
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.ImeEvent
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.MagnifierFrame
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.parseSelectionGeometry
|
||||
import org.json.JSONObject
|
||||
import java.util.concurrent.atomic.AtomicLong
|
||||
|
||||
@@ -61,6 +65,7 @@ class EmbeddedWebAppController(
|
||||
private val themeType: String = "SYSTEM",
|
||||
) : EmbeddedSurfaceController,
|
||||
EmbeddedImeBridge,
|
||||
EmbeddedMagnifierProbe,
|
||||
ConsoleBridge {
|
||||
private val incoming = Messenger(Handler(Looper.getMainLooper(), ::onServiceMessage))
|
||||
private var serviceMessenger: Messenger? = null
|
||||
@@ -94,6 +99,9 @@ class EmbeddedWebAppController(
|
||||
|
||||
override var onImeEvent: ((ImeEvent) -> Unit)? = null
|
||||
|
||||
// SPIKE (magnifier #4, option B): provider-side capture round-trip. Removed once the loupe lands.
|
||||
override var onMagnifierFrame: ((MagnifierFrame) -> Unit)? = null
|
||||
|
||||
private val connection =
|
||||
object : ServiceConnection {
|
||||
override fun onServiceConnected(
|
||||
@@ -126,6 +134,7 @@ class EmbeddedWebAppController(
|
||||
pendingAdapter = null
|
||||
onUrlChanged = null
|
||||
onImeEvent = null
|
||||
onMagnifierFrame = null
|
||||
onLoadStatusChanged = null
|
||||
consoleLogs.clear()
|
||||
}
|
||||
@@ -192,6 +201,19 @@ class EmbeddedWebAppController(
|
||||
if (consoleLogs.size >= MAX_CONSOLE_LOGS) consoleLogs.removeAt(0)
|
||||
consoleLogs.add(ConsoleLogEntry(level, message, source, line))
|
||||
}
|
||||
NappletBrowserContract.MSG_MAGNIFIER_FRAME -> {
|
||||
val data = msg.data ?: return true
|
||||
val bytes = data.getByteArray(NappletBrowserContract.KEY_MAG_BYTES) ?: return true
|
||||
onMagnifierFrame?.invoke(
|
||||
MagnifierFrame(
|
||||
bytes = bytes,
|
||||
width = data.getInt(NappletBrowserContract.KEY_MAG_W),
|
||||
height = data.getInt(NappletBrowserContract.KEY_MAG_H),
|
||||
captureMs = data.getDouble(NappletBrowserContract.KEY_MAG_CAPTURE_MS),
|
||||
requestStampNanos = data.getLong(NappletBrowserContract.KEY_MAG_REQ_T),
|
||||
),
|
||||
)
|
||||
}
|
||||
else -> return false
|
||||
}
|
||||
return true
|
||||
@@ -245,6 +267,22 @@ class EmbeddedWebAppController(
|
||||
|
||||
override fun sendImeOp(json: String) = send(NappletBrowserContract.MSG_IME_OP) { putString(NappletBrowserContract.KEY_IME_PAYLOAD, json) }
|
||||
|
||||
// Stamp the client send time so the reply can be matched / stale frames dropped (same-process clock).
|
||||
override fun requestMagnifier(
|
||||
surfaceX: Float,
|
||||
surfaceY: Float,
|
||||
boxWidthPx: Int,
|
||||
boxHeightPx: Int,
|
||||
zoom: Float,
|
||||
) = send(NappletBrowserContract.MSG_MAGNIFIER_REQUEST) {
|
||||
putFloat(NappletBrowserContract.KEY_MAG_X, surfaceX)
|
||||
putFloat(NappletBrowserContract.KEY_MAG_Y, surfaceY)
|
||||
putInt(NappletBrowserContract.KEY_MAG_BOX_W, boxWidthPx)
|
||||
putInt(NappletBrowserContract.KEY_MAG_BOX_H, boxHeightPx)
|
||||
putFloat(NappletBrowserContract.KEY_MAG_ZOOM, zoom)
|
||||
putLong(NappletBrowserContract.KEY_MAG_REQ_T, SystemClock.elapsedRealtimeNanos())
|
||||
}
|
||||
|
||||
private fun parseImeEvent(payload: String): ImeEvent? {
|
||||
val o = runCatching { JSONObject(payload) }.getOrNull() ?: return null
|
||||
return when (o.optString("type")) {
|
||||
@@ -256,6 +294,7 @@ class EmbeddedWebAppController(
|
||||
text = o.optString("text", ""),
|
||||
selStart = o.optInt("selStart", 0),
|
||||
selEnd = o.optInt("selEnd", 0),
|
||||
geometry = parseSelectionGeometry(o.optJSONObject("geom")),
|
||||
)
|
||||
"ime.blur" -> ImeEvent.Blur
|
||||
"ime.state" ->
|
||||
@@ -263,7 +302,16 @@ class EmbeddedWebAppController(
|
||||
text = o.optString("text", ""),
|
||||
selStart = o.optInt("selStart", 0),
|
||||
selEnd = o.optInt("selEnd", 0),
|
||||
geometry = parseSelectionGeometry(o.optJSONObject("geom")),
|
||||
)
|
||||
"ime.pagesel" ->
|
||||
ImeEvent.PageSelection(
|
||||
active = o.optBoolean("active", false),
|
||||
text = o.optString("text", ""),
|
||||
geometry = parseSelectionGeometry(o.optJSONObject("geom")),
|
||||
)
|
||||
"ime.scroll" -> ImeEvent.Scroll(active = o.optBoolean("active", false))
|
||||
"ime.carettap" -> ImeEvent.CaretTap(geometry = parseSelectionGeometry(o.optJSONObject("geom")))
|
||||
else -> null
|
||||
}
|
||||
}
|
||||
|
||||
+79
@@ -20,6 +20,28 @@
|
||||
*/
|
||||
package com.vitorpamplona.amethyst.ui.screen.loggedIn.embed
|
||||
|
||||
import org.json.JSONObject
|
||||
|
||||
/** Parses the `geom` object of an `ime.pagesel` payload into a [SelectionGeometry], or null if absent. */
|
||||
fun parseSelectionGeometry(o: JSONObject?): SelectionGeometry? {
|
||||
if (o == null) return null
|
||||
return SelectionGeometry(
|
||||
left = o.optDouble("l", 0.0).toFloat(),
|
||||
top = o.optDouble("t", 0.0).toFloat(),
|
||||
right = o.optDouble("r", 0.0).toFloat(),
|
||||
bottom = o.optDouble("b", 0.0).toFloat(),
|
||||
startX = o.optDouble("sx", 0.0).toFloat(),
|
||||
startBottom = o.optDouble("sb", 0.0).toFloat(),
|
||||
endX = o.optDouble("ex", 0.0).toFloat(),
|
||||
endBottom = o.optDouble("eb", 0.0).toFloat(),
|
||||
viewportWidth = o.optDouble("vw", 0.0).toFloat(),
|
||||
caretX = if (o.has("cx")) o.optDouble("cx").toFloat() else null,
|
||||
caretTop = if (o.has("ct")) o.optDouble("ct").toFloat() else null,
|
||||
caretBottom = if (o.has("cb")) o.optDouble("cb").toFloat() else null,
|
||||
isRange = o.optBoolean("rng", false),
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* A cross-process editable: the focused field lives in the embedded WebView (a different process/window
|
||||
* that can't host the soft keyboard), so the main app keeps the keyboard and relays editing across this
|
||||
@@ -44,6 +66,7 @@ sealed interface ImeEvent {
|
||||
val text: String,
|
||||
val selStart: Int,
|
||||
val selEnd: Int,
|
||||
val geometry: SelectionGeometry? = null,
|
||||
) : ImeEvent
|
||||
|
||||
/** The field lost focus — dismiss the keyboard. */
|
||||
@@ -54,5 +77,61 @@ sealed interface ImeEvent {
|
||||
val text: String,
|
||||
val selStart: Int,
|
||||
val selEnd: Int,
|
||||
val geometry: SelectionGeometry? = null,
|
||||
) : ImeEvent
|
||||
|
||||
/**
|
||||
* Ordinary page text (not an input) gained or lost a selection. The embedded WebView can't present
|
||||
* Chrome's copy toolbar/handles in its cross-process surface, so the host draws its own over the page;
|
||||
* [text] is the selected text to put on the clipboard, [geometry] places the toolbar + drag handles.
|
||||
*/
|
||||
data class PageSelection(
|
||||
val active: Boolean,
|
||||
val text: String,
|
||||
val geometry: SelectionGeometry?,
|
||||
) : ImeEvent
|
||||
|
||||
/**
|
||||
* The embedded page started ([active] true) or finished ([active] false) scrolling. Host-drawn selection
|
||||
* UI hides while scrolling (its geometry is stale mid-scroll) and reappears, repositioned, on settle —
|
||||
* the shim re-reports geometry just before the `active=false` so the overlays land in the right place.
|
||||
*/
|
||||
data class Scroll(
|
||||
val active: Boolean,
|
||||
) : ImeEvent
|
||||
|
||||
/**
|
||||
* The user tapped the focused field, placing a bare caret. Re-shows the insertion handle even when the
|
||||
* tap didn't move the caret (so no [State] fired) — native shows the handle on every tap. [geometry]
|
||||
* carries the caret rect. Gated host-side to non-empty fields, like `Editor`.
|
||||
*/
|
||||
data class CaretTap(
|
||||
val geometry: SelectionGeometry?,
|
||||
) : ImeEvent
|
||||
}
|
||||
|
||||
/**
|
||||
* Selection geometry in the page's CSS px (viewport coords). The host scales by surface-width / [viewportWidth]
|
||||
* to screen px. [left]/[top]/[right]/[bottom] is the bounding box (toolbar anchors above it); ([startX],
|
||||
* [startBottom]) and ([endX], [endBottom]) are the caret feet where the two drag handles sit.
|
||||
*/
|
||||
data class SelectionGeometry(
|
||||
val left: Float,
|
||||
val top: Float,
|
||||
val right: Float,
|
||||
val bottom: Float,
|
||||
val startX: Float,
|
||||
val startBottom: Float,
|
||||
val endX: Float,
|
||||
val endBottom: Float,
|
||||
val viewportWidth: Float,
|
||||
// Present only when the field's selection is a bare caret (no range): the caret rect, so the host can
|
||||
// show a draggable insertion handle below it. Null for a range or page-text selection.
|
||||
val caretX: Float? = null,
|
||||
val caretTop: Float? = null,
|
||||
val caretBottom: Float? = null,
|
||||
// True when this is an in-field *range* selection whose [startX]/[startBottom]/[endX]/[endBottom] are the
|
||||
// real selection-endpoint feet (not the field-box placeholders). Lets the host hold the range geometry
|
||||
// through Chrome's transient collapse-to-caret reports during the re-assert fight.
|
||||
val isRange: Boolean = false,
|
||||
)
|
||||
|
||||
+114
@@ -0,0 +1,114 @@
|
||||
/*
|
||||
* Copyright (c) 2025 Vitor Pamplona
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
* this software and associated documentation files (the "Software"), to deal in
|
||||
* the Software without restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||
* subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in all
|
||||
* copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
package com.vitorpamplona.amethyst.ui.screen.loggedIn.embed
|
||||
|
||||
import androidx.compose.foundation.Image
|
||||
import androidx.compose.foundation.background
|
||||
import androidx.compose.foundation.border
|
||||
import androidx.compose.foundation.layout.Box
|
||||
import androidx.compose.foundation.layout.absoluteOffset
|
||||
import androidx.compose.foundation.layout.fillMaxSize
|
||||
import androidx.compose.foundation.layout.size
|
||||
import androidx.compose.foundation.shape.RoundedCornerShape
|
||||
import androidx.compose.material3.MaterialTheme
|
||||
import androidx.compose.runtime.Composable
|
||||
import androidx.compose.runtime.getValue
|
||||
import androidx.compose.runtime.mutableStateOf
|
||||
import androidx.compose.runtime.setValue
|
||||
import androidx.compose.ui.Modifier
|
||||
import androidx.compose.ui.draw.clip
|
||||
import androidx.compose.ui.draw.shadow
|
||||
import androidx.compose.ui.geometry.Offset
|
||||
import androidx.compose.ui.graphics.ImageBitmap
|
||||
import androidx.compose.ui.layout.ContentScale
|
||||
import androidx.compose.ui.platform.LocalDensity
|
||||
import androidx.compose.ui.unit.DpSize
|
||||
import androidx.compose.ui.unit.IntOffset
|
||||
import androidx.compose.ui.unit.IntSize
|
||||
import androidx.compose.ui.unit.dp
|
||||
import kotlin.math.roundToInt
|
||||
|
||||
/**
|
||||
* Hoisted state for the embedded-surface selection loupe (magnifier #4). The bubble shows a magnified,
|
||||
* live slice of the page captured in the `:napplet` provider (host-side `PixelCopy` can't read the sandbox
|
||||
* surface — see [EmbeddedMagnifierProbe]). While a caret/selection handle is dragged, [anchorPx] tracks the
|
||||
* drag point in this layer's px (so the bubble follows smoothly even between captures) and [image] is the
|
||||
* latest decoded frame. Throttle bookkeeping ([lastRequestUptimeMs]/[awaitingFrame]) lives here too but is
|
||||
* intentionally NOT Compose state — it must not trigger recomposition.
|
||||
*/
|
||||
class MagnifierUiState {
|
||||
var visible by mutableStateOf(false)
|
||||
var anchorPx by mutableStateOf(Offset.Zero)
|
||||
var image by mutableStateOf<ImageBitmap?>(null)
|
||||
|
||||
var lastRequestUptimeMs: Long = 0L
|
||||
var awaitingFrame: Boolean = false
|
||||
|
||||
fun hide() {
|
||||
visible = false
|
||||
image = null
|
||||
awaitingFrame = false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The loupe bubble: a rounded, elevated rectangle showing [MagnifierUiState.image], floated above the drag
|
||||
* point and centered on it horizontally, clamped inside [layerSize]. Like Android's `Magnifier`, it sits a
|
||||
* little above the finger; if there's no room above (drag near the top), it flips below. Nothing is drawn
|
||||
* until there's a frame to show.
|
||||
*/
|
||||
@Composable
|
||||
fun Magnifier(
|
||||
state: MagnifierUiState,
|
||||
layerSize: IntSize,
|
||||
bubbleSize: DpSize,
|
||||
) {
|
||||
if (!state.visible) return
|
||||
val img = state.image ?: return
|
||||
val density = LocalDensity.current
|
||||
val wPx = with(density) { bubbleSize.width.toPx() }
|
||||
val hPx = with(density) { bubbleSize.height.toPx() }
|
||||
val gapPx = with(density) { 28.dp.toPx() }
|
||||
|
||||
val left = (state.anchorPx.x - wPx / 2f).coerceIn(0f, (layerSize.width - wPx).coerceAtLeast(0f))
|
||||
val above = state.anchorPx.y - hPx - gapPx
|
||||
val top = if (above >= 0f) above else state.anchorPx.y + gapPx
|
||||
|
||||
val shape = RoundedCornerShape(12.dp)
|
||||
Box(
|
||||
Modifier
|
||||
.absoluteOffset { IntOffset(left.roundToInt(), top.roundToInt()) }
|
||||
.size(bubbleSize)
|
||||
.shadow(8.dp, shape)
|
||||
.clip(shape)
|
||||
.background(MaterialTheme.colorScheme.surface)
|
||||
.border(1.dp, MaterialTheme.colorScheme.outlineVariant, shape),
|
||||
) {
|
||||
// The captured frame is built so its px size ≈ the bubble's px size (source rect = bubble / zoom,
|
||||
// provider scaled by zoom), so FillBounds maps it 1:1 without distortion.
|
||||
Image(
|
||||
bitmap = img,
|
||||
contentDescription = null,
|
||||
contentScale = ContentScale.FillBounds,
|
||||
modifier = Modifier.fillMaxSize(),
|
||||
)
|
||||
}
|
||||
}
|
||||
+65
@@ -0,0 +1,65 @@
|
||||
/*
|
||||
* Copyright (c) 2025 Vitor Pamplona
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
* this software and associated documentation files (the "Software"), to deal in
|
||||
* the Software without restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||
* subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in all
|
||||
* copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
package com.vitorpamplona.amethyst.ui.screen.loggedIn.embed
|
||||
|
||||
/**
|
||||
* Magnifier (#4) capture path — see `amethyst/plans/2026-06-25-embed-text-selection-native-parity.md`.
|
||||
*
|
||||
* Host-side `PixelCopy` of the sandboxed surface returns `ERROR_SOURCE_NO_DATA` (the WebView pixels live in a
|
||||
* child SurfaceControl the host never draws into), so the selection loupe's content must be captured INSIDE
|
||||
* the keyless `:napplet` provider — where the WebView is a real in-window view with real pixels — and shipped
|
||||
* back over the Messenger channel. A controller whose surface can produce such captures implements this;
|
||||
* [EmbeddedTabLayer] discovers it by `as?` cast, mirroring how it treats [EmbeddedImeBridge].
|
||||
*/
|
||||
interface EmbeddedMagnifierProbe {
|
||||
/** A captured loupe frame arrived from the provider. */
|
||||
var onMagnifierFrame: ((MagnifierFrame) -> Unit)?
|
||||
|
||||
/**
|
||||
* Ask the provider for a zoomed slice of the live page centered on ([surfaceX], [surfaceY]) in surface px
|
||||
* (which equal the remote WebView's view px — the SCVH surface is 1:1 with the SandboxedSdkView).
|
||||
* [boxWidthPx]×[boxHeightPx] is the source rectangle; the returned image is that, scaled by [zoom].
|
||||
*/
|
||||
fun requestMagnifier(
|
||||
surfaceX: Float,
|
||||
surfaceY: Float,
|
||||
boxWidthPx: Int,
|
||||
boxHeightPx: Int,
|
||||
zoom: Float,
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* A loupe frame returned by the provider: a PNG ([bytes]) of [width]×[height], plus timing — [captureMs] is
|
||||
* the provider-side draw+encode cost and [requestStampNanos] echoes the client's send stamp so out-of-order
|
||||
* frames can be dropped.
|
||||
*/
|
||||
data class MagnifierFrame(
|
||||
val bytes: ByteArray,
|
||||
val width: Int,
|
||||
val height: Int,
|
||||
val captureMs: Double,
|
||||
val requestStampNanos: Long,
|
||||
) {
|
||||
override fun equals(other: Any?) = this === other
|
||||
|
||||
override fun hashCode() = System.identityHashCode(this)
|
||||
}
|
||||
+37
@@ -0,0 +1,37 @@
|
||||
/*
|
||||
* Copyright (c) 2025 Vitor Pamplona
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
* this software and associated documentation files (the "Software"), to deal in
|
||||
* the Software without restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||
* subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in all
|
||||
* copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
package com.vitorpamplona.amethyst.ui.screen.loggedIn.embed
|
||||
|
||||
import androidx.compose.runtime.getValue
|
||||
import androidx.compose.runtime.mutableStateOf
|
||||
import androidx.compose.runtime.setValue
|
||||
|
||||
/**
|
||||
* Process-level flag: true while the user is dragging a selection/caret handle over an embedded surface.
|
||||
*
|
||||
* The left navigation drawer's edge-swipe-to-open competes with a handle drag near the left edge (and with
|
||||
* the [auto-scroll][EmbeddedTabLayer] edge drag), opening the drawer mid-drag. The drawer reads this to
|
||||
* suspend its gesture while a handle is held. Kept out of the API-gated [EmbeddedTabHost] so non-R UI (the
|
||||
* drawer) can read it without a version guard; [EmbeddedTabLayer] is the only writer.
|
||||
*/
|
||||
object EmbeddedSelectionDrag {
|
||||
var dragging by mutableStateOf(false)
|
||||
}
|
||||
+581
-5
@@ -20,19 +20,38 @@
|
||||
*/
|
||||
package com.vitorpamplona.amethyst.ui.screen.loggedIn.embed
|
||||
|
||||
import android.content.ClipData
|
||||
import android.content.ClipboardManager
|
||||
import android.content.Context
|
||||
import android.graphics.BitmapFactory
|
||||
import android.os.Build
|
||||
import android.os.SystemClock
|
||||
import android.view.ViewGroup
|
||||
import androidx.annotation.RequiresApi
|
||||
import androidx.compose.foundation.BorderStroke
|
||||
import androidx.compose.foundation.Canvas
|
||||
import androidx.compose.foundation.background
|
||||
import androidx.compose.foundation.clickable
|
||||
import androidx.compose.foundation.gestures.awaitEachGesture
|
||||
import androidx.compose.foundation.gestures.awaitFirstDown
|
||||
import androidx.compose.foundation.gestures.detectDragGestures
|
||||
import androidx.compose.foundation.gestures.waitForUpOrCancellation
|
||||
import androidx.compose.foundation.interaction.MutableInteractionSource
|
||||
import androidx.compose.foundation.layout.Box
|
||||
import androidx.compose.foundation.layout.ExperimentalLayoutApi
|
||||
import androidx.compose.foundation.layout.Row
|
||||
import androidx.compose.foundation.layout.WindowInsets
|
||||
import androidx.compose.foundation.layout.absoluteOffset
|
||||
import androidx.compose.foundation.layout.fillMaxSize
|
||||
import androidx.compose.foundation.layout.height
|
||||
import androidx.compose.foundation.layout.imeAnimationTarget
|
||||
import androidx.compose.foundation.layout.padding
|
||||
import androidx.compose.foundation.layout.size
|
||||
import androidx.compose.foundation.layout.width
|
||||
import androidx.compose.foundation.shape.RoundedCornerShape
|
||||
import androidx.compose.material3.MaterialTheme
|
||||
import androidx.compose.material3.Surface
|
||||
import androidx.compose.material3.Text
|
||||
import androidx.compose.runtime.Composable
|
||||
import androidx.compose.runtime.DisposableEffect
|
||||
import androidx.compose.runtime.LaunchedEffect
|
||||
@@ -40,22 +59,59 @@ import androidx.compose.runtime.getValue
|
||||
import androidx.compose.runtime.key
|
||||
import androidx.compose.runtime.mutableStateOf
|
||||
import androidx.compose.runtime.remember
|
||||
import androidx.compose.runtime.rememberUpdatedState
|
||||
import androidx.compose.runtime.setValue
|
||||
import androidx.compose.ui.Alignment
|
||||
import androidx.compose.ui.Modifier
|
||||
import androidx.compose.ui.geometry.Offset
|
||||
import androidx.compose.ui.geometry.Rect
|
||||
import androidx.compose.ui.graphics.Path
|
||||
import androidx.compose.ui.graphics.asImageBitmap
|
||||
import androidx.compose.ui.input.pointer.pointerInput
|
||||
import androidx.compose.ui.input.pointer.positionChangeIgnoreConsumed
|
||||
import androidx.compose.ui.layout.Layout
|
||||
import androidx.compose.ui.layout.onGloballyPositioned
|
||||
import androidx.compose.ui.layout.positionInWindow
|
||||
import androidx.compose.ui.platform.LocalContext
|
||||
import androidx.compose.ui.platform.LocalDensity
|
||||
import androidx.compose.ui.unit.DpSize
|
||||
import androidx.compose.ui.unit.IntOffset
|
||||
import androidx.compose.ui.unit.IntSize
|
||||
import androidx.compose.ui.unit.dp
|
||||
import androidx.compose.ui.viewinterop.AndroidView
|
||||
import androidx.privacysandbox.ui.client.view.SandboxedSdkView
|
||||
import kotlinx.coroutines.delay
|
||||
import org.json.JSONObject
|
||||
import kotlin.math.roundToInt
|
||||
|
||||
// How far off-screen a parked (inactive) warm tab is shifted — well past any real screen width.
|
||||
private val OFFSCREEN_SHIFT = 10_000.dp
|
||||
|
||||
// Per-drag-move auto-scroll step (CSS px) when a handle is dragged into the surface's top/bottom edge zone.
|
||||
private const val AUTOSCROLL_STEP_CSS = 40.0
|
||||
|
||||
/**
|
||||
* Drives the selection loupe from a handle drag: [active] shows/hides the bubble, [fingerLayerPx] is the
|
||||
* drag point in the tab layer's px (where the bubble floats), and ([surfaceX], [surfaceY]) is the same point
|
||||
* in surface px (what the provider captures around). Called on drag start/move with `true`, on end with `false`.
|
||||
*/
|
||||
private typealias OnMagnify = (active: Boolean, fingerLayerPx: Offset, surfaceX: Float, surfaceY: Float) -> Unit
|
||||
|
||||
/** Sends a CSS-px positional IME op (caret move / selection extend) to the page; [edge] is null for a caret move. */
|
||||
private fun EmbeddedImeBridge.sendFieldOp(
|
||||
type: String,
|
||||
edge: String?,
|
||||
cssX: Float,
|
||||
cssY: Float,
|
||||
) = sendImeOp(
|
||||
JSONObject()
|
||||
.put("type", type)
|
||||
.apply { if (edge != null) put("edge", edge) }
|
||||
.put("x", cssX.toDouble())
|
||||
.put("y", cssY.toDouble())
|
||||
.toString(),
|
||||
)
|
||||
|
||||
/**
|
||||
* The persistent surface layer: a full-window overlay (mounted once in the app shell, below the
|
||||
* navigation drawer and dialogs) that renders **every** warm embedded session's [SandboxedSdkView] and
|
||||
@@ -121,10 +177,15 @@ fun EmbeddedTabLayer(barFavoriteIds: List<String>) {
|
||||
// surface re-render, no black flash between tabs. (Parking at 1dp forced a
|
||||
// resize + re-render on every switch, which flashed black for ~1s.)
|
||||
val left = (bounds.left - layerOrigin.x).toDp() + (if (active) 0.dp else OFFSCREEN_SHIFT)
|
||||
val imeOverlap = if (active) (imeBottomPx - (layerOrigin.y + layerSize.height - bounds.bottom)).coerceAtLeast(0f) else 0f
|
||||
// Do NOT shrink the surface for the keyboard: resizing reconfigures the cross-process
|
||||
// SurfaceControlViewHost surface, and the first frame presented after that reconfigure
|
||||
// stalls ~1s (the per-focus "freeze"). Keep the surface full-size and let the page bring
|
||||
// the focused field above the keyboard via the shim's scrollIntoView on focus.
|
||||
@Suppress("UNUSED_EXPRESSION")
|
||||
imeBottomPx
|
||||
Modifier
|
||||
.absoluteOffset(left, (bounds.top - layerOrigin.y).toDp())
|
||||
.size(bounds.width.toDp(), (bounds.height - imeOverlap).coerceAtLeast(1f).toDp())
|
||||
.size(bounds.width.toDp(), bounds.height.toDp())
|
||||
}
|
||||
} else {
|
||||
// No content bounds reported yet: park tiny off-screen until a tab is shown.
|
||||
@@ -267,17 +328,45 @@ fun EmbeddedTabLayer(barFavoriteIds: List<String>) {
|
||||
val context = LocalContext.current
|
||||
val imeBridge = EmbeddedTabHost.sessions.firstOrNull { it.id == activeId }?.controller as? EmbeddedImeBridge
|
||||
val imeView = remember { RemoteImeView(context) }
|
||||
// The embedded WebView can't show Chrome's copy/paste toolbar or selection handles in its cross-process
|
||||
// surface, so we draw our own over the page and route actions through the hidden EditText (which mirrors
|
||||
// the selection). All the show/hide state lives in one [SelectionUiState] so the rules — toolbar hides
|
||||
// while dragging or scrolling, handles hide while scrolling — are expressed in one place.
|
||||
val sel = remember { SelectionUiState() }
|
||||
DisposableEffect(imeBridge) {
|
||||
imeView.bind(imeBridge)
|
||||
imeView.onRangeSelectionChanged = { sel.onFieldRangeToggle(it) }
|
||||
imeView.onEdited = { sel.onEdited() }
|
||||
imeBridge?.onImeEvent = { event ->
|
||||
when (event) {
|
||||
is ImeEvent.Focus -> imeView.onPageFocus(event)
|
||||
ImeEvent.Blur -> imeView.onPageBlur()
|
||||
is ImeEvent.State -> imeView.onPageState(event)
|
||||
is ImeEvent.Focus -> {
|
||||
imeView.onPageFocus(event)
|
||||
// A field took focus → the browser dropped any page-text selection; clear its overlay so
|
||||
// the stale page handles/Copy bar don't sit above (and steal drags from) the field UI.
|
||||
sel.onPageSelection(null)
|
||||
// Cancel any in-flight scroll-hide from the page phase: otherwise the field's own
|
||||
// selection-reveal scrolls keep it armed and the new field handles/toolbar never appear.
|
||||
sel.scrolling = false
|
||||
sel.onFieldGeometry(event.geometry, event.text.isNotEmpty())
|
||||
}
|
||||
ImeEvent.Blur -> {
|
||||
imeView.onPageBlur()
|
||||
sel.onBlur()
|
||||
}
|
||||
is ImeEvent.State -> {
|
||||
imeView.onPageState(event)
|
||||
sel.onFieldGeometry(event.geometry, event.text.isNotEmpty())
|
||||
}
|
||||
is ImeEvent.PageSelection -> sel.onPageSelection(event.takeIf { it.active })
|
||||
is ImeEvent.Scroll -> sel.scrolling = event.active
|
||||
is ImeEvent.CaretTap -> sel.onCaretTap(event.geometry)
|
||||
}
|
||||
}
|
||||
onDispose {
|
||||
imeBridge?.onImeEvent = null
|
||||
imeView.onRangeSelectionChanged = null
|
||||
imeView.onEdited = null
|
||||
sel.reset()
|
||||
imeView.onPageBlur()
|
||||
imeView.bind(null)
|
||||
}
|
||||
@@ -293,5 +382,492 @@ fun EmbeddedTabLayer(barFavoriteIds: List<String>) {
|
||||
).size(1.dp),
|
||||
)
|
||||
}
|
||||
|
||||
// Native-style auto-hide: the insertion handle disappears after a few seconds of inactivity; the next
|
||||
// caret tap re-emits geometry and re-shows it. A drag keeps it alive (dragging gate) so it never
|
||||
// vanishes mid-drag; the key re-arms whenever the caret moves.
|
||||
val caretForTimeout = sel.insertionHandle
|
||||
LaunchedEffect(caretForTimeout, sel.dragging) {
|
||||
if (caretForTimeout != null && !sel.dragging) {
|
||||
delay(4_000)
|
||||
sel.hideCaret()
|
||||
}
|
||||
}
|
||||
|
||||
// Selection loupe (magnifier #4): while a caret/selection handle is dragged, show a magnified live
|
||||
// slice of the page. Host-side PixelCopy can't read the sandbox surface (ERROR_SOURCE_NO_DATA), so the
|
||||
// pixels are captured in the `:napplet` provider and shipped back here (see [EmbeddedMagnifierProbe]).
|
||||
// The handles report the drag point via [onMagnify]; we track it for the bubble position and ask the
|
||||
// provider for frames, throttled to one in flight (with a 100 ms timeout) so a fast drag can't flood
|
||||
// the IPC channel. The bubble follows the finger every move; the bitmap refreshes as frames land.
|
||||
val magProbe = imeBridge as? EmbeddedMagnifierProbe
|
||||
val magnifier = remember { MagnifierUiState() }
|
||||
val magBubble = DpSize(132.dp, 74.dp)
|
||||
val magZoom = 1.5f
|
||||
// Source rect (surface px) = bubble px / zoom, so the provider-scaled frame lands ≈ bubble-sized.
|
||||
val magSrcW = with(density) { (magBubble.width.toPx() / magZoom).roundToInt() }
|
||||
val magSrcH = with(density) { (magBubble.height.toPx() / magZoom).roundToInt() }
|
||||
DisposableEffect(magProbe) {
|
||||
magProbe?.onMagnifierFrame = { frame ->
|
||||
if (magnifier.visible) {
|
||||
magnifier.awaitingFrame = false
|
||||
BitmapFactory.decodeByteArray(frame.bytes, 0, frame.bytes.size)?.let { magnifier.image = it.asImageBitmap() }
|
||||
}
|
||||
}
|
||||
onDispose {
|
||||
magProbe?.onMagnifierFrame = null
|
||||
magnifier.hide()
|
||||
}
|
||||
}
|
||||
// Auto-scroll (#9): while dragging a handle near the surface's top/bottom edge, nudge the embedded
|
||||
// content so the selection can keep extending past the viewport — like Android. We scroll on each
|
||||
// drag-move that's in the edge zone (the finger is usually still micro-moving); the shim keeps the
|
||||
// overlays up during this programmatic scroll and re-reports geometry so they track.
|
||||
val edgeZonePx = with(density) { 56.dp.toPx() }
|
||||
val onMagnify: OnMagnify = { active, fingerPx, surfaceX, surfaceY ->
|
||||
// Drives the loupe, the toolbar-hide-while-dragging rule (and the dragged handle hides itself
|
||||
// locally), edge auto-scroll, and suspending the nav drawer's edge swipe — one drag lifecycle.
|
||||
sel.dragging = active
|
||||
EmbeddedSelectionDrag.dragging = active
|
||||
if (active && bounds.height > 0f) {
|
||||
val oy = bounds.top - layerOrigin.y
|
||||
val dy =
|
||||
when {
|
||||
fingerPx.y < oy + edgeZonePx -> -AUTOSCROLL_STEP_CSS
|
||||
fingerPx.y > oy + bounds.height - edgeZonePx -> AUTOSCROLL_STEP_CSS
|
||||
else -> 0.0
|
||||
}
|
||||
if (dy != 0.0) imeBridge?.sendImeOp(JSONObject().put("type", "ime.autoscroll").put("dy", dy).toString())
|
||||
}
|
||||
if (!active || magProbe == null) {
|
||||
magnifier.hide()
|
||||
} else {
|
||||
magnifier.visible = true
|
||||
magnifier.anchorPx = fingerPx
|
||||
val nowMs = SystemClock.uptimeMillis()
|
||||
if (!magnifier.awaitingFrame || nowMs - magnifier.lastRequestUptimeMs > 100L) {
|
||||
magnifier.awaitingFrame = true
|
||||
magnifier.lastRequestUptimeMs = nowMs
|
||||
magProbe.requestMagnifier(surfaceX, surfaceY, magSrcW, magSrcH, magZoom)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
val originX = bounds.left - layerOrigin.x
|
||||
val originY = bounds.top - layerOrigin.y
|
||||
val haveBounds = bounds.width > 0f && bounds.height > 0f
|
||||
|
||||
// In-field (<input>/<textarea>) range selection: cut/copy/paste/select-all routed to the hidden
|
||||
// EditText, plus draggable start/end handles (drag → `ime.fieldextend`). The toolbar hides while a
|
||||
// handle is dragged or the page scrolls; the handles hide only while scrolling.
|
||||
val fieldItems =
|
||||
listOf(
|
||||
"Cut" to {
|
||||
imeView.cutSelection()
|
||||
Unit
|
||||
},
|
||||
"Copy" to {
|
||||
imeView.copySelection()
|
||||
Unit
|
||||
},
|
||||
"Paste" to {
|
||||
imeView.pasteClipboard()
|
||||
Unit
|
||||
},
|
||||
"Select all" to {
|
||||
imeView.selectAllText()
|
||||
Unit
|
||||
},
|
||||
)
|
||||
val fieldHandles = sel.fieldHandles
|
||||
if (haveBounds && fieldHandles != null) {
|
||||
RangeSelectionOverlay(
|
||||
geometry = fieldHandles,
|
||||
surfaceOriginX = originX,
|
||||
surfaceOriginY = originY,
|
||||
scale = bounds.width / fieldHandles.viewportWidth,
|
||||
showToolbar = sel.fieldToolbar != null,
|
||||
toolbarItems = fieldItems,
|
||||
onMagnify = onMagnify,
|
||||
onExtend = { edge, cssX, cssY -> imeBridge?.sendFieldOp("ime.fieldextend", edge, cssX, cssY) },
|
||||
)
|
||||
} else if (haveBounds && sel.fieldToolbar != null) {
|
||||
// Authoritative range but no geometry yet → a centered fallback toolbar (no handles).
|
||||
EmbeddedSelectionToolbar(items = fieldItems, centerXpx = originX + bounds.width / 2f, topYpx = originY + with(density) { 8.dp.toPx() })
|
||||
}
|
||||
|
||||
// Bare caret in a field (no range): a draggable insertion handle under the cursor, like Android's.
|
||||
val caretGeom = sel.insertionHandle
|
||||
if (haveBounds && caretGeom?.caretX != null && caretGeom.caretBottom != null) {
|
||||
val scale = bounds.width / caretGeom.viewportWidth
|
||||
// Half the caret's line height (layer px), so the loupe capture centers on the text line.
|
||||
val caretLineHalf =
|
||||
(((caretGeom.caretTop?.let { caretGeom.caretBottom - it }) ?: 0f) * scale / 2f)
|
||||
.coerceIn(0f, with(density) { 28.dp.toPx() })
|
||||
InsertionHandle(
|
||||
tipPx = Offset(originX + caretGeom.caretX * scale, originY + caretGeom.caretBottom * scale),
|
||||
surfaceOriginX = originX,
|
||||
surfaceOriginY = originY,
|
||||
scale = scale,
|
||||
lineHalfPx = caretLineHalf,
|
||||
onMagnify = onMagnify,
|
||||
onTap = { sel.toggleInsertionPopup() },
|
||||
onDragTo = { cssX, cssY -> imeBridge?.sendFieldOp("ime.caretmove", null, cssX, cssY) },
|
||||
)
|
||||
}
|
||||
|
||||
// Tapping the bare insertion handle opens a small Paste / Select-all popup above the caret (native).
|
||||
val popupCaret = sel.insertionPopupAt
|
||||
if (haveBounds && popupCaret?.caretX != null && popupCaret.caretTop != null) {
|
||||
val scale = bounds.width / popupCaret.viewportWidth
|
||||
val gap = with(density) { 8.dp.toPx() }
|
||||
val toolbarH = with(density) { 44.dp.toPx() }
|
||||
val caretTopPx = originY + popupCaret.caretTop * scale
|
||||
val caretBottomPx = originY + (popupCaret.caretBottom ?: popupCaret.caretTop) * scale
|
||||
val topY = if (caretTopPx - toolbarH - gap > originY) caretTopPx - toolbarH - gap else caretBottomPx + gap
|
||||
EmbeddedSelectionToolbar(
|
||||
items =
|
||||
listOf(
|
||||
"Paste" to {
|
||||
imeView.pasteClipboard()
|
||||
sel.hideInsertionPopup()
|
||||
},
|
||||
"Select all" to {
|
||||
imeView.selectAllText()
|
||||
sel.hideInsertionPopup()
|
||||
},
|
||||
),
|
||||
centerXpx = originX + popupCaret.caretX * scale,
|
||||
topYpx = topY,
|
||||
)
|
||||
}
|
||||
|
||||
// Plain page-text selection: a Copy bar over the selection + a drag handle at each end (same rules).
|
||||
val pageHandles = sel.pageHandles
|
||||
val pageSel = sel.pageSelection
|
||||
if (haveBounds && pageHandles != null && pageSel != null) {
|
||||
RangeSelectionOverlay(
|
||||
geometry = pageHandles,
|
||||
surfaceOriginX = originX,
|
||||
surfaceOriginY = originY,
|
||||
scale = bounds.width / pageHandles.viewportWidth,
|
||||
showToolbar = sel.pageToolbar != null,
|
||||
toolbarItems =
|
||||
listOf(
|
||||
"Copy" to {
|
||||
val clipboard = context.getSystemService(Context.CLIPBOARD_SERVICE) as ClipboardManager
|
||||
clipboard.setPrimaryClip(ClipData.newPlainText("selection", pageSel.text))
|
||||
Unit
|
||||
},
|
||||
),
|
||||
onMagnify = onMagnify,
|
||||
onExtend = { edge, cssX, cssY -> imeBridge?.sendFieldOp("ime.pageextend", edge, cssX, cssY) },
|
||||
)
|
||||
}
|
||||
|
||||
// The loupe sits above everything else, following the active handle/caret drag.
|
||||
Magnifier(magnifier, layerSize, magBubble)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Host-drawn selection overlay shared by in-field (`<input>`/`<textarea>`) and plain page-text selections:
|
||||
* a floating toolbar above the selection (only when [showToolbar] — it's hidden mid-drag) plus a draggable
|
||||
* handle at each end. [geometry] is in the page's CSS px; [scale] and the surface origin map those to this
|
||||
* layer's px. Dragging a handle reports the new CSS-px target via [onExtend], which the shim turns into a
|
||||
* selection extension; the resulting geometry update repositions everything.
|
||||
*/
|
||||
@Composable
|
||||
private fun RangeSelectionOverlay(
|
||||
geometry: SelectionGeometry,
|
||||
surfaceOriginX: Float,
|
||||
surfaceOriginY: Float,
|
||||
scale: Float,
|
||||
showToolbar: Boolean,
|
||||
toolbarItems: List<Pair<String, () -> Unit>>,
|
||||
onMagnify: OnMagnify?,
|
||||
onExtend: (edge: String, cssX: Float, cssY: Float) -> Unit,
|
||||
) {
|
||||
val density = LocalDensity.current
|
||||
|
||||
fun mapX(css: Float) = surfaceOriginX + css * scale
|
||||
|
||||
fun mapY(css: Float) = surfaceOriginY + css * scale
|
||||
|
||||
if (showToolbar) {
|
||||
val toolbarH = with(density) { 44.dp.toPx() }
|
||||
val gap = with(density) { 8.dp.toPx() }
|
||||
val topPx = mapY(geometry.top)
|
||||
val toolbarY = if (topPx - toolbarH - gap > surfaceOriginY) topPx - toolbarH - gap else mapY(geometry.bottom) + gap
|
||||
EmbeddedSelectionToolbar(
|
||||
items = toolbarItems,
|
||||
centerXpx = mapX((geometry.left + geometry.right) / 2f),
|
||||
topYpx = toolbarY,
|
||||
)
|
||||
}
|
||||
|
||||
// Half the selection's line height (layer px), clamped — the bounding box spans every selected line, so
|
||||
// cap it at a sane single-line height so the loupe centers on the dragged endpoint's line, not the middle
|
||||
// of a tall multi-line box.
|
||||
val lineHalfPx = ((geometry.bottom - geometry.top) * scale / 2f).coerceAtMost(with(density) { 24.dp.toPx() })
|
||||
SelectionHandle(Offset(mapX(geometry.startX), mapY(geometry.startBottom)), isStart = true, surfaceOriginX, surfaceOriginY, scale, lineHalfPx, onMagnify) { x, y -> onExtend("start", x, y) }
|
||||
SelectionHandle(Offset(mapX(geometry.endX), mapY(geometry.endBottom)), isStart = false, surfaceOriginX, surfaceOriginY, scale, lineHalfPx, onMagnify) { x, y -> onExtend("end", x, y) }
|
||||
}
|
||||
|
||||
/**
|
||||
* A draggable selection-endpoint handle drawn as Android's teardrop: a disc with one squared corner that
|
||||
* points to the caret. The start handle points up-right (hangs to the left of the selection start); the end
|
||||
* handle points up-left. [tipPx] is the caret foot (this layer's px) where the point sits; a drag reports
|
||||
* the new tip position in the page's CSS px.
|
||||
*/
|
||||
@Composable
|
||||
private fun SelectionHandle(
|
||||
tipPx: Offset,
|
||||
isStart: Boolean,
|
||||
surfaceOriginX: Float,
|
||||
surfaceOriginY: Float,
|
||||
scale: Float,
|
||||
lineHalfPx: Float,
|
||||
onMagnify: OnMagnify?,
|
||||
onDragTo: (cssX: Float, cssY: Float) -> Unit,
|
||||
) {
|
||||
val sizeDp = 22.dp
|
||||
val sizePx = with(LocalDensity.current) { sizeDp.toPx() }
|
||||
val color = MaterialTheme.colorScheme.primary
|
||||
val currentTip by rememberUpdatedState(tipPx)
|
||||
val currentMagnify by rememberUpdatedState(onMagnify)
|
||||
var dragTip by remember { mutableStateOf<Offset?>(null) }
|
||||
val tip = dragTip ?: tipPx
|
||||
|
||||
// Loupe capture: X follows the finger, Y is locked to the authoritative endpoint's line ([currentTip] is
|
||||
// the foot, so lift by half the line height) — so the bubble shows the line being edited, not wherever the
|
||||
// finger drifts vertically.
|
||||
fun magnify(
|
||||
fingerPx: Offset,
|
||||
active: Boolean = true,
|
||||
) = currentMagnify?.invoke(active, fingerPx, fingerPx.x - surfaceOriginX, currentTip.y - surfaceOriginY - lineHalfPx)
|
||||
// Place the box so its pointed corner lands on the tip: start = top-right corner, end = top-left corner.
|
||||
val boxLeft = if (isStart) tip.x - sizePx else tip.x
|
||||
Box(
|
||||
Modifier
|
||||
.absoluteOffset { IntOffset(boxLeft.roundToInt(), tip.y.roundToInt()) }
|
||||
.size(sizeDp)
|
||||
.pointerInput(Unit) {
|
||||
detectDragGestures(
|
||||
onDragStart = {
|
||||
dragTip = currentTip
|
||||
magnify(currentTip)
|
||||
},
|
||||
onDrag = { change, delta ->
|
||||
change.consume()
|
||||
val np = (dragTip ?: currentTip) + delta
|
||||
dragTip = np
|
||||
onDragTo((np.x - surfaceOriginX) / scale, (np.y - surfaceOriginY) / scale)
|
||||
magnify(np)
|
||||
},
|
||||
onDragEnd = {
|
||||
dragTip = null
|
||||
magnify(Offset.Zero, active = false)
|
||||
},
|
||||
onDragCancel = {
|
||||
dragTip = null
|
||||
magnify(Offset.Zero, active = false)
|
||||
},
|
||||
)
|
||||
},
|
||||
) {
|
||||
// Hide the teardrop while THIS handle is being dragged — the loupe stands in for it, like Android.
|
||||
if (dragTip == null) {
|
||||
Canvas(Modifier.fillMaxSize()) {
|
||||
val d = size.minDimension
|
||||
val path =
|
||||
Path().apply {
|
||||
if (isStart) {
|
||||
moveTo(d, 0f)
|
||||
lineTo(d, d / 2f)
|
||||
arcTo(Rect(0f, 0f, d, d), 0f, 270f, false)
|
||||
close()
|
||||
} else {
|
||||
moveTo(0f, 0f)
|
||||
lineTo(0f, d / 2f)
|
||||
arcTo(Rect(0f, 0f, d, d), 180f, -270f, false)
|
||||
close()
|
||||
}
|
||||
}
|
||||
drawPath(path, color)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A draggable insertion (cursor) handle drawn as Android's upward teardrop, hanging below a bare caret with
|
||||
* its tip on the caret. [tipPx] is the caret foot (this layer's px); a drag reports the new caret position in
|
||||
* the page's CSS px, which the shim maps back to a character offset.
|
||||
*/
|
||||
@Composable
|
||||
private fun InsertionHandle(
|
||||
tipPx: Offset,
|
||||
surfaceOriginX: Float,
|
||||
surfaceOriginY: Float,
|
||||
scale: Float,
|
||||
lineHalfPx: Float,
|
||||
onMagnify: OnMagnify?,
|
||||
onTap: () -> Unit,
|
||||
onDragTo: (cssX: Float, cssY: Float) -> Unit,
|
||||
) {
|
||||
val density = LocalDensity.current
|
||||
val wPx = with(density) { 20.dp.toPx() }
|
||||
val hPx = wPx * 1.4f
|
||||
val color = MaterialTheme.colorScheme.primary
|
||||
val currentTip by rememberUpdatedState(tipPx)
|
||||
val currentMagnify by rememberUpdatedState(onMagnify)
|
||||
val currentTap by rememberUpdatedState(onTap)
|
||||
|
||||
// Loupe capture: X follows the finger, Y is locked to the authoritative caret's line ([currentTip] is the
|
||||
// caret foot, so lift by half the line height) — so the bubble shows the edited line, not wherever the
|
||||
// finger drifts vertically (e.g. a diagonal drag in a textarea).
|
||||
fun magnify(
|
||||
fingerPx: Offset,
|
||||
active: Boolean = true,
|
||||
) = currentMagnify?.invoke(active, fingerPx, fingerPx.x - surfaceOriginX, currentTip.y - surfaceOriginY - lineHalfPx)
|
||||
// Track the finger separately from what we draw: the handle renders at the AUTHORITATIVE caret (tipPx,
|
||||
// which snaps to a character position as the move round-trips through the shim), while the finger drives
|
||||
// the move. So the handle stays glued to the line/text and clamps to the field instead of trailing off.
|
||||
var fingerPx by remember { mutableStateOf<Offset?>(null) }
|
||||
Box(
|
||||
Modifier
|
||||
.absoluteOffset { IntOffset((tipPx.x - wPx / 2f).roundToInt(), tipPx.y.roundToInt()) }
|
||||
.size(with(density) { wPx.toDp() }, with(density) { hPx.toDp() })
|
||||
// One unified gesture so a TAP (toggle the Paste/Select-All popup) and a DRAG (move the caret)
|
||||
// don't fight each other. We consume the down so the tap never bleeds to the surface (which would
|
||||
// place a caret there and dismiss the popup we're opening).
|
||||
.pointerInput(Unit) {
|
||||
awaitEachGesture {
|
||||
val down = awaitFirstDown(requireUnconsumed = false)
|
||||
down.consume()
|
||||
var dragging = false
|
||||
var fp = currentTip
|
||||
while (true) {
|
||||
val change = awaitPointerEvent().changes.firstOrNull() ?: break
|
||||
if (!change.pressed) {
|
||||
if (!dragging) {
|
||||
currentTap()
|
||||
} else {
|
||||
fingerPx = null
|
||||
magnify(Offset.Zero, active = false)
|
||||
}
|
||||
break
|
||||
}
|
||||
if (!dragging && (change.position - down.position).getDistance() > viewConfiguration.touchSlop) {
|
||||
dragging = true
|
||||
fp = currentTip
|
||||
magnify(currentTip)
|
||||
}
|
||||
if (dragging) {
|
||||
// Read the delta BEFORE consuming: positionChange() returns Offset.Zero once the
|
||||
// change isConsumed, so consuming first (or the sandbox surface consuming the move in
|
||||
// an earlier pass) would freeze `fp` at the caret — the handle drags but the caret
|
||||
// never moves. positionChangeIgnoreConsumed() is immune to both.
|
||||
fp += change.positionChangeIgnoreConsumed()
|
||||
change.consume()
|
||||
fingerPx = fp
|
||||
onDragTo((fp.x - surfaceOriginX) / scale, (fp.y - surfaceOriginY) / scale)
|
||||
magnify(fp)
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
) {
|
||||
// Hide the teardrop while dragging — the loupe stands in for it, like Android.
|
||||
if (fingerPx == null) {
|
||||
Canvas(Modifier.fillMaxSize()) {
|
||||
val r = size.width / 2f
|
||||
val cx = size.width / 2f
|
||||
val cy = size.height - r
|
||||
val path =
|
||||
Path().apply {
|
||||
moveTo(cx, 0f)
|
||||
lineTo(cx + r, cy)
|
||||
arcTo(Rect(cx - r, cy - r, cx + r, cy + r), 0f, 180f, false)
|
||||
lineTo(cx, 0f)
|
||||
close()
|
||||
}
|
||||
drawPath(path, color)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Host-drawn copy/paste bar for an embedded page selection. Chrome can't present its own action-mode in the
|
||||
* cross-process surface, so this floats over the active tab; each action runs on the hidden [RemoteImeView]
|
||||
* (which mirrors the page's text + selection), so cut/paste relay back to the page through the normal path.
|
||||
* Uses pointer-input rather than `clickable` so tapping it doesn't pull focus off the EditText (which would
|
||||
* drop the keyboard and the selection).
|
||||
*/
|
||||
@Composable
|
||||
private fun EmbeddedSelectionToolbar(
|
||||
items: List<Pair<String, () -> Unit>>,
|
||||
centerXpx: Float,
|
||||
topYpx: Float,
|
||||
) {
|
||||
// Center in a SINGLE layout pass: measure the bar, then place it at centerX − width/2. (The old
|
||||
// measure-then-offset approach rendered it left-edge-at-centerX for one frame, so it visibly slid in
|
||||
// from the right each time the bar was (re)composed.)
|
||||
Layout(
|
||||
content = {
|
||||
Surface(
|
||||
shape = RoundedCornerShape(8.dp),
|
||||
color = MaterialTheme.colorScheme.surface,
|
||||
border = BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant),
|
||||
tonalElevation = 3.dp,
|
||||
shadowElevation = 6.dp,
|
||||
) {
|
||||
Row(verticalAlignment = Alignment.CenterVertically) {
|
||||
items.forEachIndexed { i, (label, action) ->
|
||||
if (i > 0) {
|
||||
Box(
|
||||
Modifier
|
||||
.width(1.dp)
|
||||
.height(20.dp)
|
||||
.background(MaterialTheme.colorScheme.outlineVariant),
|
||||
)
|
||||
}
|
||||
SelectionToolbarItem(label, action)
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
) { measurables, constraints ->
|
||||
val bar = measurables.first().measure(constraints.copy(minWidth = 0, minHeight = 0))
|
||||
layout(constraints.maxWidth, constraints.maxHeight) {
|
||||
bar.place((centerXpx - bar.width / 2f).roundToInt(), topYpx.roundToInt())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
private fun SelectionToolbarItem(
|
||||
label: String,
|
||||
onClick: () -> Unit,
|
||||
) {
|
||||
Text(
|
||||
text = label,
|
||||
color = MaterialTheme.colorScheme.onSurfaceVariant,
|
||||
modifier =
|
||||
Modifier
|
||||
// Consume the DOWN (not just the up) so the tap never bleeds through to the embedded surface
|
||||
// beneath — otherwise tapping the bar over non-editable page area blurs the field.
|
||||
.pointerInput(label) {
|
||||
awaitEachGesture {
|
||||
awaitFirstDown(requireUnconsumed = false).consume()
|
||||
val up = waitForUpOrCancellation()
|
||||
if (up != null) {
|
||||
up.consume()
|
||||
onClick()
|
||||
}
|
||||
}
|
||||
}.padding(horizontal = 12.dp, vertical = 10.dp),
|
||||
)
|
||||
}
|
||||
|
||||
+119
-4
@@ -22,6 +22,7 @@ package com.vitorpamplona.amethyst.ui.screen.loggedIn.embed
|
||||
|
||||
import android.annotation.SuppressLint
|
||||
import android.content.Context
|
||||
import android.os.SystemClock
|
||||
import android.text.Editable
|
||||
import android.text.InputType
|
||||
import android.text.TextWatcher
|
||||
@@ -93,7 +94,10 @@ class RemoteImeView(
|
||||
count: Int,
|
||||
) {}
|
||||
|
||||
override fun afterTextChanged(s: Editable?) = schedule()
|
||||
override fun afterTextChanged(s: Editable?) {
|
||||
if (!applyingRemote) onEdited?.invoke()
|
||||
schedule()
|
||||
}
|
||||
},
|
||||
)
|
||||
// The IME's "Go/Search/Send/Done" — the page submits/handles it (single-line has no newline).
|
||||
@@ -108,26 +112,101 @@ class RemoteImeView(
|
||||
this.bridge = bridge
|
||||
}
|
||||
|
||||
/**
|
||||
* Notified when the mirrored selection becomes (true) or stops being (false) a non-empty range. The
|
||||
* embedded WebView can't present Chrome's copy/paste toolbar in its cross-process surface, so the host
|
||||
* shows its own over the page and routes the actions back through [copy]/[cut]/[paste]/[selectAll],
|
||||
* which run on this EditText's Editable (and so relay to the page through the normal edit path).
|
||||
*/
|
||||
var onRangeSelectionChanged: ((Boolean) -> Unit)? = null
|
||||
private var hadRange = false
|
||||
|
||||
// Off-window Chrome abandons a selection by momentarily collapsing the caret to an endpoint, which we (or
|
||||
// the page shim) re-assert right back — so the mirrored selection flickers range→caret→range within a few
|
||||
// ms during a long-press/double-tap. The host-drawn handles/toolbar are gated on [hadRange], so reporting
|
||||
// every flip blinks them off and back on, in lock-step with each collapse cycle. Native never shows that
|
||||
// churn. So we DEFER the "range lost" signal by [RANGE_LOSS_DEBOUNCE_MS]: a re-assert that restores the
|
||||
// range first cancels the pending hide, and only a selection that truly STAYS collapsed hides the overlays.
|
||||
// Gaining a range is always reported immediately.
|
||||
private val reportRangeLost =
|
||||
Runnable {
|
||||
if (selectionStart == selectionEnd && hadRange) {
|
||||
hadRange = false
|
||||
onRangeSelectionChanged?.invoke(false)
|
||||
}
|
||||
}
|
||||
|
||||
/** Fired when the user edits text via the keyboard (not on programmatic page-state applies). The host
|
||||
* hides the insertion handle while typing, the way Android does. */
|
||||
var onEdited: (() -> Unit)? = null
|
||||
|
||||
fun copySelection(): Boolean = onTextContextMenuItem(android.R.id.copy)
|
||||
|
||||
fun cutSelection(): Boolean = onTextContextMenuItem(android.R.id.cut)
|
||||
|
||||
fun pasteClipboard(): Boolean = onTextContextMenuItem(android.R.id.paste)
|
||||
|
||||
fun selectAllText(): Boolean = onTextContextMenuItem(android.R.id.selectAll)
|
||||
|
||||
/** A page field focused: configure the keyboard, seed the buffer, and raise the IME. */
|
||||
fun onPageFocus(focus: ImeEvent.Focus) {
|
||||
configureFor(focus)
|
||||
applyRemote(focus.text, focus.selStart, focus.selEnd)
|
||||
// Focus the EditText BEFORE seeding text/selection. An EditText jumps its caret to the end when it
|
||||
// gains focus; if we seed first, that end-position then overrides the seed and gets shipped to the
|
||||
// page — so a tap mid-text lands the caret at the end of the field. Seeding AFTER focus makes the
|
||||
// tap position the final state (the focus-induced end-position only schedules a flush that then
|
||||
// coalesces to this seed, a no-op).
|
||||
requestFocus()
|
||||
imm.restartInput(this)
|
||||
applyRemote(focus.text, focus.selStart, focus.selEnd)
|
||||
// Post the show so it runs after focus/attachment has settled (showSoftInput can no-op otherwise).
|
||||
post {
|
||||
if (hasFocus()) imm.showSoftInput(this, InputMethodManager.SHOW_IMPLICIT)
|
||||
}
|
||||
}
|
||||
|
||||
// When the current selection first became a range, and how many of its collapse-abandonments we've
|
||||
// re-asserted. Chrome abandons a selection *immediately* (~60ms); a deliberate user tap-to-collapse comes
|
||||
// later — so we only re-assert within a short window of the range forming, bounded for safety.
|
||||
private var rangeBecameAt = 0L
|
||||
private var reassertCount = 0
|
||||
|
||||
/** The page changed the field itself (its JS, autofill): mirror it without echoing back. */
|
||||
fun onPageState(state: ImeEvent.State) {
|
||||
if (text?.toString() == state.text && selectionStart == state.selStart && selectionEnd == state.selEnd) return
|
||||
val curText = text?.toString() ?: ""
|
||||
// Chrome can't present selection handles in the embedded surface, so it abandons a selection by
|
||||
// collapsing the caret to one of the range's endpoints, right after it forms. We hold the
|
||||
// authoritative selection here, so a collapse to an endpoint of our current range (same text) within
|
||||
// the window is that abandonment, not a user action: re-assert our range instead of accepting it.
|
||||
val collapsedToEndpoint =
|
||||
selectionStart != selectionEnd &&
|
||||
state.selStart == state.selEnd &&
|
||||
state.text == curText &&
|
||||
(state.selStart == selectionStart || state.selStart == selectionEnd)
|
||||
if (collapsedToEndpoint && reassertCount < MAX_REASSERT &&
|
||||
SystemClock.uptimeMillis() - rangeBecameAt < REASSERT_WINDOW_MS
|
||||
) {
|
||||
reassertCount++
|
||||
// Keep the window alive across the fight: Chrome re-abandons the selection every ~600ms while the
|
||||
// gesture is held, so each re-assert restarts the clock — bounded by reassertCount so a page that
|
||||
// truly keeps the caret collapsed still wins.
|
||||
rangeBecameAt = SystemClock.uptimeMillis()
|
||||
lastSent = null // force a non-no-op flush so the range re-ships
|
||||
flushState()
|
||||
return
|
||||
}
|
||||
reassertCount = 0
|
||||
if (curText == state.text && selectionStart == state.selStart && selectionEnd == state.selEnd) return
|
||||
applyRemote(state.text, state.selStart, state.selEnd)
|
||||
}
|
||||
|
||||
/** The page field blurred: drop the keyboard. */
|
||||
fun onPageBlur() {
|
||||
removeCallbacks(reportRangeLost)
|
||||
if (hadRange) {
|
||||
hadRange = false
|
||||
onRangeSelectionChanged?.invoke(false)
|
||||
}
|
||||
clearFocus()
|
||||
imm.hideSoftInputFromWindow(windowToken, 0)
|
||||
}
|
||||
@@ -138,11 +217,18 @@ class RemoteImeView(
|
||||
selEnd: Int,
|
||||
) {
|
||||
applyingRemote = true
|
||||
setText(newText)
|
||||
// Only replace the buffer when the text actually changed. A page that rewrites its field's
|
||||
// selection on a timer (without changing the text) would otherwise force a full setText on every
|
||||
// update — clearing the composing region and restarting the IME — which needlessly churns the
|
||||
// keyboard and contends with the user's own typing. Reposition the cursor without touching the buffer.
|
||||
if (text?.toString() != newText) setText(newText)
|
||||
val len = text?.length ?: 0
|
||||
setSelection(selStart.coerceIn(0, len), selEnd.coerceIn(0, len))
|
||||
applyingRemote = false
|
||||
lastSent = stateJson().toString()
|
||||
// Start the abandonment window when a fresh range appears, so onPageState can tell Chrome's instant
|
||||
// collapse from a later user tap-to-collapse.
|
||||
if (selectionStart != selectionEnd) rangeBecameAt = SystemClock.uptimeMillis()
|
||||
}
|
||||
|
||||
private fun schedule() {
|
||||
@@ -156,6 +242,20 @@ class RemoteImeView(
|
||||
selEnd: Int,
|
||||
) {
|
||||
super.onSelectionChanged(selStart, selEnd)
|
||||
val isRange = selStart != selEnd
|
||||
if (isRange) {
|
||||
// A range is back (or still here): cancel any pending hide and show immediately.
|
||||
removeCallbacks(reportRangeLost)
|
||||
if (!hadRange) {
|
||||
hadRange = true
|
||||
onRangeSelectionChanged?.invoke(true)
|
||||
}
|
||||
} else if (hadRange) {
|
||||
// Collapsed — but this may be Chrome's transient abandonment we're about to re-assert. Defer the
|
||||
// hide; if the range returns within the window, the show branch above cancels this.
|
||||
removeCallbacks(reportRangeLost)
|
||||
postDelayed(reportRangeLost, RANGE_LOSS_DEBOUNCE_MS)
|
||||
}
|
||||
schedule()
|
||||
}
|
||||
|
||||
@@ -237,4 +337,19 @@ class RemoteImeView(
|
||||
return result
|
||||
}
|
||||
}
|
||||
|
||||
private companion object {
|
||||
// Chrome re-abandons a held selection repeatedly; cover a long-press hold (each re-assert restarts the
|
||||
// window) while still giving up if the page truly keeps re-collapsing forever.
|
||||
private const val MAX_REASSERT = 12
|
||||
|
||||
// Wider than Chrome's ~600ms re-collapse interval so consecutive abandonments stay inside the window,
|
||||
// yet short enough that a deliberate user tap-to-collapse (well after the gesture) is accepted.
|
||||
private const val REASSERT_WINDOW_MS = 800L
|
||||
|
||||
// How long a collapse must persist before we hide the host-drawn selection overlays. The shim/host
|
||||
// re-assert restores an abandonment collapse within a frame or two (one IPC hop), so this only needs to
|
||||
// outlast that round-trip — comfortably short enough that a genuine tap-to-collapse still feels instant.
|
||||
private const val RANGE_LOSS_DEBOUNCE_MS = 250L
|
||||
}
|
||||
}
|
||||
|
||||
+183
@@ -0,0 +1,183 @@
|
||||
/*
|
||||
* Copyright (c) 2025 Vitor Pamplona
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
* this software and associated documentation files (the "Software"), to deal in
|
||||
* the Software without restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||
* subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in all
|
||||
* copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
package com.vitorpamplona.amethyst.ui.screen.loggedIn.embed
|
||||
|
||||
import androidx.compose.runtime.Stable
|
||||
import androidx.compose.runtime.getValue
|
||||
import androidx.compose.runtime.mutableStateOf
|
||||
import androidx.compose.runtime.setValue
|
||||
|
||||
/**
|
||||
* Single source of truth for the host-drawn selection UI over an embedded surface. Folds together what used
|
||||
* to be scattered booleans/geometries (`showSelectionToolbar`, `showInsertionHandle`, `fieldGeometry`,
|
||||
* `rangeFieldGeometry`, `pageSelection`) so the show/hide rules are expressed in one place.
|
||||
*
|
||||
* Three mutually-exclusive selection contexts can be active: a bare caret in a field ([insertionGeometry]),
|
||||
* a range inside an `<input>`/`<textarea>` ([fieldRange], gated by [fieldHasRange] — the authoritative
|
||||
* "is there a selection" signal from the mirror [RemoteImeView]), or a plain page-text selection
|
||||
* ([pageSelection]). [dragging] and [scrolling] are the transient modifiers:
|
||||
*
|
||||
* - **dragging** a handle/caret hides the floating toolbar (and the dragged handle hides itself while the
|
||||
* loupe stands in), matching Android; the other handle stays so you can see the live range.
|
||||
* - **scrolling** hides every overlay (its geometry is stale mid-scroll); the shim re-reports geometry on
|
||||
* settle, then clears this, so they reappear in the right place.
|
||||
*/
|
||||
@Stable
|
||||
class SelectionUiState {
|
||||
/** Latest field geometry; carries the caret rect when collapsed, positioning the insertion handle. */
|
||||
var fieldGeometry by mutableStateOf<SelectionGeometry?>(null)
|
||||
private set
|
||||
|
||||
/** Held in-field range geometry (real endpoint feet) — survives Chrome's transient collapse-to-caret. */
|
||||
var fieldRange by mutableStateOf<SelectionGeometry?>(null)
|
||||
private set
|
||||
|
||||
/** Authoritative "the field selection is a non-empty range", from the mirror EditText. */
|
||||
var fieldHasRange by mutableStateOf(false)
|
||||
private set
|
||||
|
||||
/** A bare caret handle is showing — set by a tap-placed caret in non-empty text, cleared on typing/blur/timeout. */
|
||||
var caretShown by mutableStateOf(false)
|
||||
private set
|
||||
|
||||
/**
|
||||
* Whether the focused field currently has any text. Native (`Editor.onTouchUpEvent`) only offers the
|
||||
* insertion handle when `text.length() > 0`, so an empty field never gets a draggable cursor handle
|
||||
* (just the blinking caret). Gating on this is what stops the handle popping up on focus of an empty box.
|
||||
*/
|
||||
var fieldHasText by mutableStateOf(false)
|
||||
private set
|
||||
|
||||
/** Plain page-text (non-editable) selection. */
|
||||
var pageSelection by mutableStateOf<ImeEvent.PageSelection?>(null)
|
||||
private set
|
||||
|
||||
/** A handle/caret is being dragged. */
|
||||
var dragging by mutableStateOf(false)
|
||||
|
||||
/** The embedded surface is scrolling. */
|
||||
var scrolling by mutableStateOf(false)
|
||||
|
||||
/** Tapping the bare insertion handle opens a small Paste/Select-All popup (native); tap-elsewhere closes it. */
|
||||
var insertionPopup by mutableStateOf(false)
|
||||
private set
|
||||
|
||||
private val insertionGeometry: SelectionGeometry?
|
||||
get() = fieldGeometry?.takeIf { it.caretX != null && it.caretBottom != null && it.viewportWidth > 0f }
|
||||
|
||||
// ---- derived visibility (read in composition; track the backing state) ----
|
||||
|
||||
val insertionHandle: SelectionGeometry?
|
||||
get() = if (caretShown && fieldHasText && !fieldHasRange && !scrolling) insertionGeometry else null
|
||||
|
||||
/** Geometry for the insertion-handle Paste/Select-All popup, or null when it shouldn't show. */
|
||||
val insertionPopupAt: SelectionGeometry?
|
||||
get() = if (insertionPopup && !dragging) insertionHandle else null
|
||||
|
||||
val fieldHandles: SelectionGeometry?
|
||||
get() = if (fieldHasRange && !scrolling) fieldRange?.takeIf { it.viewportWidth > 0f } else null
|
||||
|
||||
val fieldToolbar: SelectionGeometry?
|
||||
get() = if (fieldHasRange && !dragging && !scrolling) (fieldRange ?: fieldGeometry) else null
|
||||
|
||||
private val pageGeom: SelectionGeometry?
|
||||
get() = pageSelection?.takeIf { it.active }?.geometry?.takeIf { it.viewportWidth > 0f }
|
||||
|
||||
val pageHandles: SelectionGeometry?
|
||||
get() = if (!scrolling) pageGeom else null
|
||||
|
||||
val pageToolbar: SelectionGeometry?
|
||||
get() = if (!dragging && !scrolling) pageGeom else null
|
||||
|
||||
// ---- signal sinks ----
|
||||
|
||||
fun onFieldGeometry(
|
||||
g: SelectionGeometry?,
|
||||
hasText: Boolean,
|
||||
) {
|
||||
fieldHasText = hasText
|
||||
if (g == null) return
|
||||
fieldGeometry = g
|
||||
if (g.isRange) fieldRange = g
|
||||
// Only a tap-placed caret in NON-empty text gets the handle — like native. (Typing suppresses the
|
||||
// echo, so this fires for taps, not keystrokes; see RemoteImeView / shim __nappletIme guard.)
|
||||
if (g.caretX != null && hasText) caretShown = true
|
||||
}
|
||||
|
||||
/** From [RemoteImeView.onRangeSelectionChanged]: the field gained/lost a non-empty selection. */
|
||||
fun onFieldRangeToggle(hasRange: Boolean) {
|
||||
fieldHasRange = hasRange
|
||||
if (!hasRange) fieldRange = null
|
||||
if (hasRange) insertionPopup = false // a selection formed → no insertion popup
|
||||
}
|
||||
|
||||
/**
|
||||
* The user tapped the focused field (placing a caret) — (re-)show the insertion handle even if the caret
|
||||
* didn't move. Gated to non-empty text by the [insertionHandle] getter, exactly like native.
|
||||
*/
|
||||
fun onCaretTap(g: SelectionGeometry?) {
|
||||
if (g != null) fieldGeometry = g
|
||||
caretShown = true
|
||||
insertionPopup = false // a tap in the text dismisses the popup (and moves the caret)
|
||||
}
|
||||
|
||||
/** The user typed: hide the insertion handle, like Android. */
|
||||
fun onEdited() {
|
||||
caretShown = false
|
||||
insertionPopup = false
|
||||
}
|
||||
|
||||
/** Native auto-hides the insertion handle after ~a few seconds of inactivity; a later tap re-shows it. */
|
||||
fun hideCaret() {
|
||||
caretShown = false
|
||||
insertionPopup = false
|
||||
}
|
||||
|
||||
/** Tap on the bare insertion handle toggles its Paste/Select-All popup. */
|
||||
fun toggleInsertionPopup() {
|
||||
insertionPopup = !insertionPopup
|
||||
}
|
||||
|
||||
fun hideInsertionPopup() {
|
||||
insertionPopup = false
|
||||
}
|
||||
|
||||
fun onBlur() {
|
||||
fieldGeometry = null
|
||||
fieldRange = null
|
||||
fieldHasRange = false
|
||||
fieldHasText = false
|
||||
caretShown = false
|
||||
insertionPopup = false
|
||||
}
|
||||
|
||||
fun onPageSelection(sel: ImeEvent.PageSelection?) {
|
||||
pageSelection = sel
|
||||
}
|
||||
|
||||
/** Full teardown when the active tab/controller changes. */
|
||||
fun reset() {
|
||||
onBlur()
|
||||
pageSelection = null
|
||||
dragging = false
|
||||
scrolling = false
|
||||
}
|
||||
}
|
||||
+47
-1
@@ -31,6 +31,7 @@ import android.os.IBinder
|
||||
import android.os.Looper
|
||||
import android.os.Message
|
||||
import android.os.Messenger
|
||||
import android.os.SystemClock
|
||||
import androidx.annotation.RequiresApi
|
||||
import androidx.privacysandbox.ui.client.SandboxedUiAdapterFactory
|
||||
import androidx.privacysandbox.ui.client.view.SandboxedSdkView
|
||||
@@ -39,8 +40,11 @@ import com.vitorpamplona.amethyst.napplethost.NappletEmbedContract
|
||||
import com.vitorpamplona.amethyst.napplethost.NappletHostContract
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedImeBridge
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedLoadStatus
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedMagnifierProbe
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedSurfaceController
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.ImeEvent
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.MagnifierFrame
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.parseSelectionGeometry
|
||||
import org.json.JSONObject
|
||||
import java.util.concurrent.atomic.AtomicLong
|
||||
|
||||
@@ -59,7 +63,8 @@ class EmbeddedNostrAppController(
|
||||
private val appContext: Context,
|
||||
private val params: Bundle,
|
||||
) : EmbeddedSurfaceController,
|
||||
EmbeddedImeBridge {
|
||||
EmbeddedImeBridge,
|
||||
EmbeddedMagnifierProbe {
|
||||
private val incoming = Messenger(Handler(Looper.getMainLooper(), ::onServiceMessage))
|
||||
private var serviceMessenger: Messenger? = null
|
||||
private var bound = false
|
||||
@@ -93,6 +98,8 @@ class EmbeddedNostrAppController(
|
||||
|
||||
override var onImeEvent: ((ImeEvent) -> Unit)? = null
|
||||
|
||||
override var onMagnifierFrame: ((MagnifierFrame) -> Unit)? = null
|
||||
|
||||
private val connection =
|
||||
object : ServiceConnection {
|
||||
override fun onServiceConnected(
|
||||
@@ -125,6 +132,7 @@ class EmbeddedNostrAppController(
|
||||
onStateChanged = null
|
||||
onNotice = null
|
||||
onImeEvent = null
|
||||
onMagnifierFrame = null
|
||||
onLoadStatusChanged = null
|
||||
}
|
||||
|
||||
@@ -183,6 +191,19 @@ class EmbeddedNostrAppController(
|
||||
val failed = msg.data?.getBoolean(NappletEmbedContract.KEY_LOAD_FAILED, false) ?: false
|
||||
onLoadState(isLoading, failed)
|
||||
}
|
||||
NappletEmbedContract.MSG_MAGNIFIER_FRAME -> {
|
||||
val data = msg.data ?: return true
|
||||
val bytes = data.getByteArray(NappletEmbedContract.KEY_MAG_BYTES) ?: return true
|
||||
onMagnifierFrame?.invoke(
|
||||
MagnifierFrame(
|
||||
bytes = bytes,
|
||||
width = data.getInt(NappletEmbedContract.KEY_MAG_W),
|
||||
height = data.getInt(NappletEmbedContract.KEY_MAG_H),
|
||||
captureMs = data.getDouble(NappletEmbedContract.KEY_MAG_CAPTURE_MS),
|
||||
requestStampNanos = data.getLong(NappletEmbedContract.KEY_MAG_REQ_T),
|
||||
),
|
||||
)
|
||||
}
|
||||
else -> return false
|
||||
}
|
||||
return true
|
||||
@@ -190,6 +211,21 @@ class EmbeddedNostrAppController(
|
||||
|
||||
override fun sendImeOp(json: String) = send(NappletEmbedContract.MSG_IME_OP) { putString(NappletEmbedContract.KEY_IME_PAYLOAD, json) }
|
||||
|
||||
override fun requestMagnifier(
|
||||
surfaceX: Float,
|
||||
surfaceY: Float,
|
||||
boxWidthPx: Int,
|
||||
boxHeightPx: Int,
|
||||
zoom: Float,
|
||||
) = send(NappletEmbedContract.MSG_MAGNIFIER_REQUEST) {
|
||||
putFloat(NappletEmbedContract.KEY_MAG_X, surfaceX)
|
||||
putFloat(NappletEmbedContract.KEY_MAG_Y, surfaceY)
|
||||
putInt(NappletEmbedContract.KEY_MAG_BOX_W, boxWidthPx)
|
||||
putInt(NappletEmbedContract.KEY_MAG_BOX_H, boxHeightPx)
|
||||
putFloat(NappletEmbedContract.KEY_MAG_ZOOM, zoom)
|
||||
putLong(NappletEmbedContract.KEY_MAG_REQ_T, SystemClock.elapsedRealtimeNanos())
|
||||
}
|
||||
|
||||
private fun parseImeEvent(payload: String): ImeEvent? {
|
||||
val o = runCatching { JSONObject(payload) }.getOrNull() ?: return null
|
||||
return when (o.optString("type")) {
|
||||
@@ -201,6 +237,7 @@ class EmbeddedNostrAppController(
|
||||
text = o.optString("text", ""),
|
||||
selStart = o.optInt("selStart", 0),
|
||||
selEnd = o.optInt("selEnd", 0),
|
||||
geometry = parseSelectionGeometry(o.optJSONObject("geom")),
|
||||
)
|
||||
"ime.blur" -> ImeEvent.Blur
|
||||
"ime.state" ->
|
||||
@@ -208,7 +245,16 @@ class EmbeddedNostrAppController(
|
||||
text = o.optString("text", ""),
|
||||
selStart = o.optInt("selStart", 0),
|
||||
selEnd = o.optInt("selEnd", 0),
|
||||
geometry = parseSelectionGeometry(o.optJSONObject("geom")),
|
||||
)
|
||||
"ime.pagesel" ->
|
||||
ImeEvent.PageSelection(
|
||||
active = o.optBoolean("active", false),
|
||||
text = o.optString("text", ""),
|
||||
geometry = parseSelectionGeometry(o.optJSONObject("geom")),
|
||||
)
|
||||
"ime.scroll" -> ImeEvent.Scroll(active = o.optBoolean("active", false))
|
||||
"ime.carettap" -> ImeEvent.CaretTap(geometry = parseSelectionGeometry(o.optJSONObject("geom")))
|
||||
else -> null
|
||||
}
|
||||
}
|
||||
|
||||
@@ -191,6 +191,7 @@
|
||||
if (IME_PROXY) (function(){
|
||||
var el = null; // the focused editable element, or null
|
||||
var inComposition = false;
|
||||
function perfNow(){ try { return performance.now(); } catch (_) { return 0; } }
|
||||
|
||||
function isEditable(n){
|
||||
if (!n) return false;
|
||||
@@ -205,7 +206,22 @@
|
||||
}
|
||||
function isCE(n){ return !!(n && n.isContentEditable); }
|
||||
function valOf(n){ return isCE(n) ? n.textContent : (n.value || ''); }
|
||||
function setVal(n, v){ if (isCE(n)) n.textContent = v; else n.value = v; }
|
||||
// Controlled-input frameworks (React, Preact, …) install an INSTANCE-level `value` setter on the
|
||||
// <input>/<textarea> that records the last value they wrote, and then suppress their onChange whenever
|
||||
// the element's value already equals that recorded value. A plain `n.value = v` assignment goes through
|
||||
// that tracker, so our programmatic edit looks like a no-op to the framework: onChange never fires, its
|
||||
// state stays stale, and on the next render it reconciles the field straight back to the stale value —
|
||||
// wiping what we just typed. Writing through the NATIVE prototype setter sets the real value without
|
||||
// touching the tracker, so the framework's input handler sees value != tracked, detects the change, and
|
||||
// commits it. Identical to `n.value = v` for plain (non-framework) pages.
|
||||
var nativeInputValueSet = (function(){ try { return Object.getOwnPropertyDescriptor(window.HTMLInputElement.prototype, 'value').set; } catch (_) { return null; } })();
|
||||
var nativeAreaValueSet = (function(){ try { return Object.getOwnPropertyDescriptor(window.HTMLTextAreaElement.prototype, 'value').set; } catch (_) { return null; } })();
|
||||
function setVal(n, v){
|
||||
if (isCE(n)) { n.textContent = v; return; }
|
||||
var set = (typeof HTMLTextAreaElement !== 'undefined' && n instanceof HTMLTextAreaElement) ? nativeAreaValueSet : nativeInputValueSet;
|
||||
if (set) { try { set.call(n, v); return; } catch (_) {} }
|
||||
n.value = v;
|
||||
}
|
||||
|
||||
// --- contenteditable selection/replacement, mapped through char offsets into textContent ---
|
||||
// We can't use setSelectionRange/value on a contenteditable root; instead we map a char offset
|
||||
@@ -261,31 +277,144 @@
|
||||
return [n.selectionStart || 0, n.selectionEnd || 0];
|
||||
}
|
||||
function setSel(n, s, e){
|
||||
// No-op if already there: re-applying the same selection still fires `select`/`selectionchange`, and
|
||||
// every such redundant event ripples into a host report → geometry update → toolbar/handle recompose
|
||||
// (visible churn). Our re-asserts/re-applies frequently target the current range, so guard them here.
|
||||
var cur = selOf(n);
|
||||
if (cur[0] === s && cur[1] === e) return;
|
||||
lastSelActivityAt = perfNow(); // a real selection write → the field may auto-scroll to reveal it
|
||||
if (isCE(n)) ceSetSel(n, s, e);
|
||||
else { try { n.setSelectionRange(s, e); } catch (_) {} }
|
||||
}
|
||||
// The DOM exposes no caret/selection rect for a position inside an <input>/<textarea>, so we mirror the
|
||||
// field into a hidden div (same font/padding/wrapping) and measure where a marker span lands — the
|
||||
// well-known "textarea-caret-position" technique. Used to place the insertion handle (and drag it).
|
||||
var CARET_PROPS = ['direction','boxSizing','width','height','overflowX','overflowY','borderTopWidth','borderRightWidth','borderBottomWidth','borderLeftWidth','paddingTop','paddingRight','paddingBottom','paddingLeft','fontStyle','fontVariant','fontWeight','fontStretch','fontSize','fontSizeAdjust','lineHeight','fontFamily','textAlign','textTransform','textIndent','textDecoration','letterSpacing','wordSpacing','tabSize','MozTabSize'];
|
||||
function caretCoords(n, position){
|
||||
try {
|
||||
var isInput = (n.nodeName || '').toUpperCase() === 'INPUT';
|
||||
var computed = window.getComputedStyle(n);
|
||||
var div = document.createElement('div');
|
||||
var s = div.style;
|
||||
s.position = 'absolute'; s.visibility = 'hidden'; s.whiteSpace = isInput ? 'nowrap' : 'pre-wrap'; s.wordWrap = 'break-word'; s.overflow = 'hidden';
|
||||
for (var i = 0; i < CARET_PROPS.length; i++) { s[CARET_PROPS[i]] = computed[CARET_PROPS[i]]; }
|
||||
var val = valOf(n);
|
||||
div.textContent = val.substring(0, position);
|
||||
if (isInput) div.textContent = div.textContent.replace(/\s/g, ' ');
|
||||
var span = document.createElement('span');
|
||||
span.textContent = val.substring(position) || '.';
|
||||
div.appendChild(span);
|
||||
document.body.appendChild(div);
|
||||
var caretL = span.offsetLeft, caretT = span.offsetTop;
|
||||
var lineHeight = parseInt(computed.lineHeight) || parseInt(computed.fontSize) || 16;
|
||||
document.body.removeChild(div);
|
||||
// offsetLeft/Top are measured from the mirror's *inner* (padding) edge, but getBoundingClientRect is the
|
||||
// *outer* border box — so add the field's border widths to land on the real caret.
|
||||
var bl = parseFloat(computed.borderLeftWidth) || 0, bt = parseFloat(computed.borderTopWidth) || 0;
|
||||
var rect = n.getBoundingClientRect();
|
||||
var x = rect.left + bl + caretL - n.scrollLeft;
|
||||
var top = rect.top + bt + caretT - n.scrollTop;
|
||||
return { x: x, top: top, bottom: top + lineHeight };
|
||||
} catch (_) { return null; }
|
||||
}
|
||||
// Inverse: the char offset whose caret is nearest the CSS-px point (x,y). Binary search in reading order
|
||||
// (offset increases left-to-right, top-to-bottom), so a handle drag maps back to a cursor position. Y is
|
||||
// first clamped into the field's text rows, so dragging the handle (which sits below the line) or off the
|
||||
// field keeps the cursor on the nearest line and lets X drive the column — like Android.
|
||||
function offsetFromPoint(n, x, y){
|
||||
var len = valOf(n).length;
|
||||
var first = caretCoords(n, 0), last = caretCoords(n, len);
|
||||
if (first && y < first.top) y = (first.top + first.bottom) / 2;
|
||||
else if (last && y > last.bottom) y = (last.top + last.bottom) / 2;
|
||||
var lo = 0, hi = len;
|
||||
while (lo < hi) {
|
||||
var mid = (lo + hi) >> 1, c = caretCoords(n, mid);
|
||||
if (!c) break;
|
||||
if (y < c.top) hi = mid;
|
||||
else if (y > c.bottom) lo = mid + 1;
|
||||
else if (x < c.x) hi = mid;
|
||||
else lo = mid + 1;
|
||||
}
|
||||
// The search lands on the boundary just RIGHT of x; round to the NEAREST boundary instead (native
|
||||
// getOffsetForPosition) so tapping the left half of a glyph doesn't advance the caret past it. Only
|
||||
// compare within the same line (skip when lo sits at a wrap, where lo-1 is on the previous row).
|
||||
if (lo > 0) {
|
||||
var cl = caretCoords(n, lo - 1), cr = caretCoords(n, lo);
|
||||
if (cl && cr && cl.top === cr.top && (x - cl.x) < (cr.x - x)) lo = lo - 1;
|
||||
}
|
||||
return lo;
|
||||
}
|
||||
// The focused field's bounding box in CSS px (toolbar anchor). When the selection is a bare caret it also
|
||||
// carries the caret rect (cx/ct/cb) so the host can show a draggable insertion handle.
|
||||
function fieldGeom(n){
|
||||
try {
|
||||
var b = n.getBoundingClientRect();
|
||||
var g = { l: b.left, t: b.top, r: b.right, b: b.bottom, sx: b.left, sb: b.bottom, ex: b.right, eb: b.bottom, vw: window.innerWidth };
|
||||
if (!isCE(n)) {
|
||||
var sel = selOf(n);
|
||||
if (sel[0] === sel[1]) {
|
||||
// Bare caret → carry the caret rect so the host shows the insertion handle.
|
||||
var c = caretCoords(n, sel[0]);
|
||||
if (c) { g.cx = c.x; g.ct = c.top; g.cb = c.bottom; }
|
||||
} else {
|
||||
// Range → carry the start/end caret feet so the host shows draggable selection handles, and
|
||||
// tighten the box to the selected line span so the toolbar anchors above the selection.
|
||||
var cs = caretCoords(n, sel[0]), ce = caretCoords(n, sel[1]);
|
||||
if (cs && ce) {
|
||||
g.rng = true; // marks a real range so the host keeps these feet through Chrome's collapse fight
|
||||
g.sx = cs.x; g.sb = cs.bottom; g.ex = ce.x; g.eb = ce.bottom;
|
||||
g.t = Math.min(cs.top, ce.top); g.b = Math.max(cs.bottom, ce.bottom);
|
||||
}
|
||||
}
|
||||
}
|
||||
return g;
|
||||
} catch (_) { return null; }
|
||||
}
|
||||
function focusInfo(n){
|
||||
var t = (n.tagName || '').toUpperCase();
|
||||
var multiline = isCE(n) || t === 'TEXTAREA';
|
||||
var inputType = isCE(n) ? 'text' : (t === 'TEXTAREA' ? 'textarea' : (n.type || 'text').toLowerCase());
|
||||
var sel = selOf(n);
|
||||
return { type:'ime.focus', inputType: inputType, enterKeyHint: (n.enterKeyHint || ''),
|
||||
multiline: multiline, text: valOf(n), selStart: sel[0], selEnd: sel[1] };
|
||||
multiline: multiline, text: valOf(n), selStart: sel[0], selEnd: sel[1], geom: fieldGeom(n) };
|
||||
}
|
||||
// Last selection we either applied (applyState) or already reported, so the asynchronous
|
||||
// selectionchange our own setSel triggers doesn't echo back to the host as a fresh edit.
|
||||
var lastSel = null;
|
||||
// The live non-collapsed field range we protect from Chrome's off-window abandonment (see the
|
||||
// selectionchange handler). Tracked centrally so handle-extends and select-all keep it current.
|
||||
var lastFieldRange = null, lastFieldAt = -1;
|
||||
// When the selection last changed (Chrome-driven OR our own setSel). Forming/re-asserting a selection makes
|
||||
// the browser auto-scroll the field to reveal it, firing `scroll` events that are NOT a user content scroll.
|
||||
// The hide-on-scroll path uses this to ignore those: hiding the host overlays on a selection-reveal scroll
|
||||
// makes the handles/toolbar blink off-and-on every time a selection settles. See onAnyScroll.
|
||||
var lastSelActivityAt = -1;
|
||||
function sameSel(a, b){ return !!(a && b && a[0] === b[0] && a[1] === b[1]); }
|
||||
function noteSel(sel){
|
||||
lastSel = sel;
|
||||
if (el && !isCE(el)) {
|
||||
if (sel[0] !== sel[1]) { lastFieldRange = [sel[0], sel[1]]; lastFieldAt = perfNow(); }
|
||||
else { lastFieldRange = null; }
|
||||
}
|
||||
}
|
||||
function reportState(){
|
||||
if (!el) return;
|
||||
var sel = selOf(el);
|
||||
lastSel = sel;
|
||||
send({ type:'ime.state', text: valOf(el), selStart: sel[0], selEnd: sel[1] });
|
||||
noteSel(sel);
|
||||
send({ type:'ime.state', text: valOf(el), selStart: sel[0], selEnd: sel[1], geom: fieldGeom(el) });
|
||||
}
|
||||
|
||||
document.addEventListener('focusin', function(e){
|
||||
if (isEditable(e.target)) {
|
||||
el = e.target; inComposition = false; lastSel = selOf(el); send(focusInfo(el));
|
||||
// Focusing a field clears any page-text selection in the browser. The page selectionchange handler is
|
||||
// muted while a field is focused (el is set), so it never emits the `active:false` — emit it here, or
|
||||
// the host's page handles + Copy bar linger ABOVE the field overlays (and, being z-above, steal the
|
||||
// caret/selection-handle drag so the caret can't be moved).
|
||||
if (pageSelText) { lastPageRange = null; sendPageSel(false, null); }
|
||||
// Cancel any in-flight scroll-hide from the page phase so the new field overlays aren't suppressed by a
|
||||
// stale `scrolling` state (its settle would otherwise keep getting re-armed by the field reveal-scrolls).
|
||||
scrolling = false; if (scrollTimer) { clearTimeout(scrollTimer); scrollTimer = null; }
|
||||
// The host shrinks the surface for the keyboard, but also nudge the field into view in case IME
|
||||
// insets aren't delivered (some hosts) so it never sits behind the keyboard.
|
||||
try { el.scrollIntoView({ block: 'center', inline: 'nearest' }); } catch (_) {}
|
||||
@@ -296,11 +425,71 @@
|
||||
}, true);
|
||||
// The page (its own JS, autofill) changed the field: resync the host keyboard's view of it.
|
||||
document.addEventListener('input', function(e){ if (e.target === el && !el.__nappletIme) reportState(); }, true);
|
||||
// Mirror selection changes inside the focused editable to the host. Off-window Chrome abandons a field
|
||||
// selection by collapsing the caret to one of its endpoints; we re-assert it RIGHT HERE, synchronously,
|
||||
// the same way the page-text path does — reverting before the collapse paints, so it doesn't blink (the
|
||||
// old path round-tripped through the host EditText, leaving a visible collapsed frame each cycle). The
|
||||
// host's own re-assert in RemoteImeView.onPageState stays as a fallback for collapses we don't catch.
|
||||
document.addEventListener('selectionchange', function(){
|
||||
if (!el || el.__nappletIme) return;
|
||||
if (sameSel(selOf(el), lastSel)) return; // our own applyState/setSel echoing back
|
||||
if (!el || el.__nappletIme) return; // our own applyState/setSel
|
||||
lastSelActivityAt = perfNow(); // Chrome moved the selection → an imminent reveal-scroll isn't a user scroll
|
||||
var sel = selOf(el);
|
||||
if (sameSel(sel, lastSel)) return; // echo of what we just applied
|
||||
if (!isCE(el) && lastFieldRange && sel[0] === sel[1] &&
|
||||
(sel[0] === lastFieldRange[0] || sel[0] === lastFieldRange[1]) &&
|
||||
(perfNow() - lastFieldAt) < 1500) {
|
||||
// Chrome's off-window abandonment collapsed our live range to an endpoint → snap it back
|
||||
// synchronously (reverts before paint, so no blink) and keep the window open. The host already
|
||||
// holds this range, so we don't re-report (which would feed the slow round-trip loop).
|
||||
el.__nappletIme = true;
|
||||
setSel(el, lastFieldRange[0], lastFieldRange[1]);
|
||||
el.__nappletIme = false;
|
||||
lastSel = selOf(el);
|
||||
lastFieldAt = perfNow();
|
||||
return;
|
||||
}
|
||||
reportState();
|
||||
}, true);
|
||||
// Tap handling on the focused editable. A single tap collapses any selection to a caret at the tap point
|
||||
// and shows the insertion handle (native; off-window Chrome won't collapse-on-tap itself). A DOUBLE tap
|
||||
// selects the word — but `click` fires before `dblclick`, so instead of guessing with timing we DEFER the
|
||||
// collapse and let the real `dblclick` cancel it. This is robust to Chrome's own double-click timing
|
||||
// (a timing guess raced it and sometimes ate the word selection → "cursor jumps to end of word").
|
||||
var collapseTimer = null;
|
||||
function clearCollapse() { if (collapseTimer) { clearTimeout(collapseTimer); collapseTimer = null; } }
|
||||
document.addEventListener('dblclick', function(e){
|
||||
if (!el || isCE(el)) return;
|
||||
clearCollapse(); // a real double-tap → don't collapse; keep Chrome's word selection
|
||||
var s = selOf(el);
|
||||
if (s[0] !== s[1] && !sameSel(s, lastSel)) reportState(); // report the word only if not already sent
|
||||
}, true);
|
||||
document.addEventListener('click', function(e){
|
||||
if (!el || isCE(el) || e.target !== el) return;
|
||||
var sel = selOf(el);
|
||||
if (sel[0] !== sel[1]) {
|
||||
// Tap landed on a selection. Defer the collapse: if a dblclick follows (within the tap window) it
|
||||
// cancels this and the word stays selected; otherwise this fires and collapses to the tapped offset.
|
||||
var x = e.clientX, y = e.clientY;
|
||||
clearCollapse();
|
||||
collapseTimer = setTimeout(function(){
|
||||
collapseTimer = null;
|
||||
if (!el) return;
|
||||
var s = selOf(el);
|
||||
if (s[0] === s[1]) return; // already collapsed
|
||||
var off = offsetFromPoint(el, x, y);
|
||||
el.__nappletIme = true;
|
||||
setSel(el, off, off);
|
||||
el.__nappletIme = false;
|
||||
lastSel = selOf(el);
|
||||
reportState();
|
||||
send({ type:'ime.carettap', geom: fieldGeom(el) });
|
||||
}, 300);
|
||||
} else {
|
||||
// Tap on a bare caret → (re-)show the insertion handle. If a double-tap follows, dblclick selects the
|
||||
// word and supersedes this.
|
||||
send({ type:'ime.carettap', geom: fieldGeom(el) });
|
||||
}
|
||||
}, true);
|
||||
|
||||
function enter(n){
|
||||
if (!n) return;
|
||||
@@ -363,12 +552,190 @@
|
||||
}
|
||||
setSel(n, msg.selStart, msg.selEnd);
|
||||
if (!composingActive && inComposition) { inComposition = false; fireComp(n, 'compositionend', d.inserted || ''); }
|
||||
} finally { n.__nappletIme = false; lastSel = selOf(n); }
|
||||
} finally { n.__nappletIme = false; noteSel(selOf(n)); }
|
||||
}
|
||||
|
||||
// --- Page (non-editable) text selection re-hosting ---
|
||||
// Chrome can't present its selection handles/toolbar in the cross-process embedded surface, so a
|
||||
// long-press on ordinary page text selects a word and then ~60ms later abandons (collapses) it, the
|
||||
// same way it does inside inputs. Mirror the document selection: re-assert it when it collapses right
|
||||
// after forming, and report the selected text so the host can show its own Copy bar over the page.
|
||||
var pageSelText = '', lastPageRange = null, lastPageAt = -1, pageReasserting = false;
|
||||
// Selection geometry in CSS px (viewport coords). The host maps these to screen px (scale = surface
|
||||
// width / vw) to draw the toolbar above the selection and a handle at each end. l/t/r/b is the bounding
|
||||
// box; (sx,sb) the start-caret foot, (ex,eb) the end-caret foot; vw lets the host derive the scale.
|
||||
function pageGeom(r){
|
||||
try {
|
||||
var b = r.getBoundingClientRect();
|
||||
var sr = r.cloneRange(); sr.collapse(true); var s = sr.getBoundingClientRect();
|
||||
var er = r.cloneRange(); er.collapse(false); var e = er.getBoundingClientRect();
|
||||
return { l: b.left, t: b.top, r: b.right, b: b.bottom, sx: s.left, sb: s.bottom, ex: e.left, eb: e.bottom, vw: window.innerWidth };
|
||||
} catch (_) { return null; }
|
||||
}
|
||||
function sendPageSel(active, r){
|
||||
var text = active ? String(window.getSelection()) : '';
|
||||
pageSelText = text;
|
||||
send({ type: 'ime.pagesel', active: active, text: text, geom: active && r ? pageGeom(r) : null });
|
||||
}
|
||||
document.addEventListener('selectionchange', function(){
|
||||
if (el || pageReasserting) return; // selections inside an editable are handled above
|
||||
lastSelActivityAt = perfNow(); // page selection moved → an imminent reveal-scroll isn't a user scroll
|
||||
var s = window.getSelection();
|
||||
if (s && s.rangeCount && !s.isCollapsed) {
|
||||
var r = s.getRangeAt(0);
|
||||
lastPageRange = r.cloneRange(); lastPageAt = perfNow();
|
||||
sendPageSel(true, r);
|
||||
} else if (lastPageRange && (perfNow() - lastPageAt) < 400) {
|
||||
pageReasserting = true;
|
||||
try { s.removeAllRanges(); s.addRange(lastPageRange); } catch (_) {}
|
||||
pageReasserting = false;
|
||||
lastPageAt = perfNow();
|
||||
} else if (pageSelText) {
|
||||
lastPageRange = null;
|
||||
sendPageSel(false, null);
|
||||
}
|
||||
}, true);
|
||||
// Host drag of a selection handle: move the dragged edge to the text position under (x,y) CSS px,
|
||||
// keeping the opposite edge anchored. setBaseAndExtent tolerates either drag direction.
|
||||
function pageExtend(edge, x, y){
|
||||
try {
|
||||
var pt = document.caretRangeFromPoint && document.caretRangeFromPoint(x, y);
|
||||
var s = window.getSelection();
|
||||
if (!pt || !s.rangeCount) return;
|
||||
var cur = s.getRangeAt(0);
|
||||
var aN, aO;
|
||||
if (edge === 'start') { aN = cur.endContainer; aO = cur.endOffset; } else { aN = cur.startContainer; aO = cur.startOffset; }
|
||||
pageReasserting = true;
|
||||
s.setBaseAndExtent(aN, aO, pt.startContainer, pt.startOffset);
|
||||
pageReasserting = false;
|
||||
if (!s.isCollapsed) {
|
||||
var nr = s.getRangeAt(0);
|
||||
lastPageRange = nr.cloneRange(); lastPageAt = perfNow();
|
||||
sendPageSel(true, nr);
|
||||
}
|
||||
} catch (_) {}
|
||||
}
|
||||
|
||||
// Word-granularity snapping (native: dragging a word selection's handle extends a word at a time). The
|
||||
// end handle snaps to the end of the word at/after the offset; the start handle to the start of the word
|
||||
// at/before it. Whitespace between words extends to the adjacent word so you never stop mid-gap.
|
||||
function isWordChar(c){ return c != null && /\S/.test(c); }
|
||||
function wordEndAt(text, off){
|
||||
var i = off;
|
||||
while (i < text.length && !isWordChar(text[i])) i++;
|
||||
while (i < text.length && isWordChar(text[i])) i++;
|
||||
return i;
|
||||
}
|
||||
function wordStartAt(text, off){
|
||||
var i = off;
|
||||
while (i > 0 && !isWordChar(text[i - 1])) i--;
|
||||
while (i > 0 && isWordChar(text[i - 1])) i--;
|
||||
return i;
|
||||
}
|
||||
|
||||
// Per-drag state for the hybrid word/char handle extend below. `fieldDragWordEnd`/`fieldDragWordStart`
|
||||
// remember how far the dragged edge has been word-snapped so far this gesture; a >250ms gap between
|
||||
// `ime.fieldextend` ops (or a switch of edge) means a NEW drag, so we re-baseline to the live selection.
|
||||
var fieldDragAt = -1, fieldDragEdge = null, fieldDragWordEnd = -1, fieldDragWordStart = -1;
|
||||
// Host drag of an in-field selection handle: move the dragged edge to the offset under (x,y) CSS px,
|
||||
// keeping the other edge anchored, clamped so it can't cross the anchor. HYBRID granularity, matching
|
||||
// native `Editor` word-selection drags (#5): the gesture starts anchored to the current selection edge,
|
||||
// and as the finger sweeps PAST that word's far boundary it snaps the dragged edge to the next WHOLE word
|
||||
// (so sweeping across words grabs them whole and never stops mid-gap); moving WITHIN or back from the
|
||||
// furthest-reached word gives CHARACTER precision (so you can fine-tune to a single character).
|
||||
function fieldExtend(edge, x, y){
|
||||
if (!el || isCE(el)) return;
|
||||
try {
|
||||
var off = offsetFromPoint(el, x, y);
|
||||
var text = valOf(el);
|
||||
var sel = selOf(el);
|
||||
var now = perfNow();
|
||||
var fresh = (now - fieldDragAt > 250) || edge !== fieldDragEdge;
|
||||
fieldDragAt = now; fieldDragEdge = edge;
|
||||
var s, e;
|
||||
if (edge === 'start') {
|
||||
e = sel[1];
|
||||
if (fresh) fieldDragWordStart = sel[0]; // baseline at the current selection start
|
||||
if (off < fieldDragWordStart) { s = wordStartAt(text, off); fieldDragWordStart = s; } // swept into a new word → snap whole
|
||||
else s = off; // within / back from the furthest word → character precision
|
||||
s = Math.max(0, Math.min(s, e));
|
||||
} else {
|
||||
s = sel[0];
|
||||
if (fresh) fieldDragWordEnd = sel[1]; // baseline at the current selection end
|
||||
if (off > fieldDragWordEnd) { e = wordEndAt(text, off); fieldDragWordEnd = e; } // swept into a new word → snap whole
|
||||
else e = off; // within / back from the furthest word → character precision
|
||||
e = Math.min(text.length, Math.max(e, s));
|
||||
}
|
||||
el.__nappletIme = true;
|
||||
setSel(el, s, e);
|
||||
el.__nappletIme = false;
|
||||
lastSel = selOf(el);
|
||||
reportState();
|
||||
} catch (_) {}
|
||||
}
|
||||
|
||||
// While the page scrolls, host-drawn selection UI (toolbar + handles) would float at stale positions, so
|
||||
// the host hides it on scroll-start and we re-report fresh geometry on scroll-idle so it reappears in the
|
||||
// right place — like Android. Only signal when there's a selection to hide (a field range or page text).
|
||||
var scrolling = false, scrollTimer = null, autoScrolling = false;
|
||||
// How long after a selection change a scroll is treated as the browser's auto-reveal of that selection
|
||||
// (not a user content scroll). Generous enough to catch the reveal-scroll that fires a frame or two later.
|
||||
var SCROLL_SEL_GUARD_MS = 350;
|
||||
function hasSelectionUi(){ return !!pageSelText || !!(el && (function(s){ return s[0] !== s[1]; })(selOf(el))); }
|
||||
function onAnyScroll(){
|
||||
if (autoScrolling) return; // our own drag-to-edge auto-scroll: keep the overlays up, don't hide them
|
||||
if (!hasSelectionUi()) return;
|
||||
if ((perfNow() - lastSelActivityAt) < SCROLL_SEL_GUARD_MS) {
|
||||
// The browser auto-scrolled to reveal a just-changed selection (forming/re-asserting a range scrolls a
|
||||
// textarea). That's not a user content scroll: hiding here would blink the host overlays off-and-on every
|
||||
// time a selection settles. Reposition them in place instead (geometry shifted by the reveal-scroll).
|
||||
// Crucially we do NOT touch the hide-on-scroll timer: if a real scroll-hide is somehow active, these
|
||||
// reveal-scrolls must not keep re-arming it (that would leave the overlays hidden indefinitely).
|
||||
if (el) reportState();
|
||||
else { var sr = window.getSelection(); if (sr && sr.rangeCount && !sr.isCollapsed) sendPageSel(true, sr.getRangeAt(0)); }
|
||||
return;
|
||||
}
|
||||
if (!scrolling) { scrolling = true; send({ type:'ime.scroll', active: true }); }
|
||||
if (scrollTimer) clearTimeout(scrollTimer);
|
||||
scrollTimer = setTimeout(function(){
|
||||
scrolling = false; scrollTimer = null;
|
||||
// Refresh geometry FIRST (so overlays reposition), then tell the host to show them again.
|
||||
if (el) reportState();
|
||||
else { var s = window.getSelection(); if (s && s.rangeCount && !s.isCollapsed) sendPageSel(true, s.getRangeAt(0)); }
|
||||
send({ type:'ime.scroll', active: false });
|
||||
}, 150);
|
||||
}
|
||||
document.addEventListener('scroll', onAnyScroll, true); // capture: any scroller, not just the document
|
||||
|
||||
window.__nappletImeHandle = function(msg){
|
||||
if (msg.type === 'ime.set') applyState(msg);
|
||||
else if (msg.type === 'ime.action') enter(el);
|
||||
else if (msg.type === 'ime.pageextend') pageExtend(msg.edge, msg.x, msg.y);
|
||||
else if (msg.type === 'ime.fieldextend') fieldExtend(msg.edge, msg.x, msg.y);
|
||||
else if (msg.type === 'ime.autoscroll') {
|
||||
// Host drag of a handle near the surface's top/bottom edge → scroll the content (the textarea if it
|
||||
// scrolls, else the page) so the selection can keep extending, then re-report geometry so the
|
||||
// overlays follow. Flagged so our own scroll doesn't trip the hide-on-scroll path above.
|
||||
var dy = msg.dy || 0;
|
||||
autoScrolling = true;
|
||||
try {
|
||||
if (el && (el.tagName || '').toUpperCase() === 'TEXTAREA') el.scrollTop += dy;
|
||||
window.scrollBy(0, dy);
|
||||
} catch (_) {}
|
||||
if (el) reportState();
|
||||
else { var s = window.getSelection(); if (s && s.rangeCount && !s.isCollapsed) sendPageSel(true, s.getRangeAt(0)); }
|
||||
setTimeout(function(){ autoScrolling = false; }, 0);
|
||||
}
|
||||
else if (msg.type === 'ime.caretmove') {
|
||||
if (el && !isCE(el)) {
|
||||
var off = offsetFromPoint(el, msg.x, msg.y);
|
||||
el.__nappletIme = true;
|
||||
setSel(el, off, off);
|
||||
el.__nappletIme = false;
|
||||
lastSel = selOf(el);
|
||||
reportState();
|
||||
}
|
||||
}
|
||||
};
|
||||
})();
|
||||
|
||||
|
||||
+59
@@ -0,0 +1,59 @@
|
||||
/*
|
||||
* Copyright (c) 2025 Vitor Pamplona
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
* this software and associated documentation files (the "Software"), to deal in
|
||||
* the Software without restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||
* subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in all
|
||||
* copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
package com.vitorpamplona.amethyst.napplethost
|
||||
|
||||
import android.content.Context
|
||||
import android.content.res.Configuration
|
||||
import android.view.ContextThemeWrapper
|
||||
|
||||
/**
|
||||
* A context that makes a hosted WebView (embedded surface OR full-screen activity) follow the **app** theme
|
||||
* ("DARK"/"LIGHT") rather than the device.
|
||||
*
|
||||
* WebView's dark decision (`prefers-color-scheme` via algorithmic darkening) reads the context's **theme**
|
||||
* (`?android:attr/isLightTheme`), NOT just the Configuration `uiMode` — and an off-window
|
||||
* `SurfaceControlViewHost` surface context carries neither the host window's theme nor its night mode. So we
|
||||
* force the night flag in the Configuration AND wrap it in a DayNight theme whose `isLightTheme` then resolves
|
||||
* from that flag.
|
||||
*
|
||||
* (Verified on device with a standalone repro: `createConfigurationContext` alone — a night Configuration with
|
||||
* no theme — does NOT flip the renderer; the DayNight `ContextThemeWrapper` is what does it, even across the
|
||||
* cross-process embedded surface. `setApplicationNightMode` and per-WebView config dispatch do nothing.)
|
||||
*
|
||||
* "SYSTEM" (or any unrecognized value) returns [base] unchanged, i.e. follows the device — the host already
|
||||
* resolves SYSTEM→DARK/LIGHT before handing the theme down for the embedded surfaces.
|
||||
*/
|
||||
internal fun nightThemedContext(
|
||||
base: Context,
|
||||
themeType: String,
|
||||
): Context {
|
||||
val night =
|
||||
when (themeType) {
|
||||
"DARK" -> Configuration.UI_MODE_NIGHT_YES
|
||||
"LIGHT" -> Configuration.UI_MODE_NIGHT_NO
|
||||
else -> return base
|
||||
}
|
||||
val config =
|
||||
Configuration(base.resources.configuration).apply {
|
||||
uiMode = (uiMode and Configuration.UI_MODE_NIGHT_MASK.inv()) or night
|
||||
}
|
||||
return ContextThemeWrapper(base.createConfigurationContext(config), android.R.style.Theme_DeviceDefault_DayNight)
|
||||
}
|
||||
+19
-13
@@ -35,6 +35,7 @@ import android.os.Messenger
|
||||
import android.util.Log
|
||||
import android.view.Gravity
|
||||
import android.view.View
|
||||
import android.view.ViewGroup
|
||||
import android.webkit.ConsoleMessage
|
||||
import android.webkit.WebChromeClient
|
||||
import android.webkit.WebResourceError
|
||||
@@ -153,7 +154,6 @@ class NappletBrowserActivity : ComponentActivity() {
|
||||
useTor = intent.getBooleanExtra(EXTRA_USE_TOR, true)
|
||||
title = intent.getStringExtra(EXTRA_TITLE).orEmpty()
|
||||
themeType = intent.getStringExtra(EXTRA_THEME).orEmpty().ifBlank { "SYSTEM" }
|
||||
applyNightMode()
|
||||
|
||||
if (!WebViewFeature.isFeatureSupported(WebViewFeature.WEB_MESSAGE_LISTENER)) {
|
||||
Toast.makeText(this, getString(R.string.napplet_webview_too_old), Toast.LENGTH_LONG).show()
|
||||
@@ -161,7 +161,9 @@ class NappletBrowserActivity : ComponentActivity() {
|
||||
return
|
||||
}
|
||||
|
||||
webView = WebView(this)
|
||||
// Build the WebView from a context forced to the app theme so its content follows DARK/LIGHT even when
|
||||
// the device theme differs (WebView reads the context's theme, not the window's — see nightThemedContext).
|
||||
webView = WebView(nightThemedContext(this, themeType))
|
||||
configureWebView(webView)
|
||||
webView.setBackgroundColor(resolveThemeColor(android.R.attr.colorBackground))
|
||||
webView.dropSystemBarInsets()
|
||||
@@ -231,7 +233,6 @@ class NappletBrowserActivity : ComponentActivity() {
|
||||
super.onResume()
|
||||
if (this::webView.isInitialized) {
|
||||
webView.onResume()
|
||||
webView.resumeTimers()
|
||||
}
|
||||
resumed = true
|
||||
heartbeatHandler.removeCallbacks(heartbeat)
|
||||
@@ -240,8 +241,11 @@ class NappletBrowserActivity : ComponentActivity() {
|
||||
|
||||
override fun onPause() {
|
||||
if (this::webView.isInitialized) {
|
||||
// Only pause THIS activity's WebView (onPause is per-WebView). Do NOT call pauseTimers(): it is
|
||||
// process-global — it freezes JS/layout/parsing timers for EVERY WebView in `:napplet`, including
|
||||
// the embedded ones in NappletBrowserService, which have no resume of their own. That left the
|
||||
// embed frozen (dead page/connection) after returning from a full-screen excursion.
|
||||
webView.onPause()
|
||||
webView.pauseTimers()
|
||||
}
|
||||
resumed = false
|
||||
heartbeatHandler.removeCallbacks(heartbeat)
|
||||
@@ -251,7 +255,17 @@ class NappletBrowserActivity : ComponentActivity() {
|
||||
|
||||
override fun onDestroy() {
|
||||
runCatching { unbindService(brokerConnection) }
|
||||
if (this::webView.isInitialized) webView.destroy()
|
||||
if (this::webView.isInitialized) {
|
||||
// Detach from the view tree BEFORE destroy(). Destroying a WebView while it is still attached to
|
||||
// the window corrupts the SHARED multiprocess renderer/network state, which then breaks the OTHER
|
||||
// (embedded) WebViews living in this `:napplet` process: dead DNS (ERR_NAME_NOT_RESOLVED), DOM reads
|
||||
// returning empty (`value == ""` on a field that visibly shows text), dead selection-highlight paint,
|
||||
// and broken IME — all after a full-screen excursion returns to an embed. (`destroy()` requires the
|
||||
// view to be removed from the hierarchy first; see WebView.destroy() docs.)
|
||||
webView.stopLoading()
|
||||
(webView.parent as? ViewGroup)?.removeView(webView)
|
||||
webView.destroy()
|
||||
}
|
||||
super.onDestroy()
|
||||
}
|
||||
|
||||
@@ -644,14 +658,6 @@ class NappletBrowserActivity : ComponentActivity() {
|
||||
addView(ProgressBar(this@NappletBrowserActivity))
|
||||
}
|
||||
|
||||
private fun applyNightMode() {
|
||||
val uiManager = getSystemService(android.content.Context.UI_MODE_SERVICE) as android.app.UiModeManager
|
||||
when (themeType) {
|
||||
"DARK" -> uiManager.nightMode = android.app.UiModeManager.MODE_NIGHT_YES
|
||||
"LIGHT" -> uiManager.nightMode = android.app.UiModeManager.MODE_NIGHT_NO
|
||||
}
|
||||
}
|
||||
|
||||
private fun resolveThemeColor(attr: Int): Int {
|
||||
val tv = android.util.TypedValue()
|
||||
theme.resolveAttribute(attr, tv, true)
|
||||
|
||||
+28
@@ -77,6 +77,34 @@ object NappletBrowserContract {
|
||||
*/
|
||||
const val MSG_CONSOLE_LOG = 11
|
||||
|
||||
/**
|
||||
* Client → provider: capture a magnified slice of the live page for the native-style selection loupe.
|
||||
* Host-side `PixelCopy` of the sandboxed surface returns `ERROR_SOURCE_NO_DATA` (the WebView pixels live
|
||||
* in a child SurfaceControl the host never draws into), so we capture INSIDE the provider — where the
|
||||
* WebView is a real in-window view. Carries [KEY_MAG_X]/[KEY_MAG_Y] (surface px center),
|
||||
* [KEY_MAG_BOX_W]/[KEY_MAG_BOX_H] (source rectangle, px), [KEY_MAG_ZOOM], and [KEY_MAG_REQ_T] (the
|
||||
* client's `nanoTime` send stamp, echoed back so the client can drop stale out-of-order frames).
|
||||
*/
|
||||
const val MSG_MAGNIFIER_REQUEST = 12
|
||||
|
||||
/**
|
||||
* Provider → client: the captured loupe frame. Carries [KEY_MAG_BYTES] (a PNG well under the Binder
|
||||
* limit), [KEY_MAG_W]/[KEY_MAG_H], [KEY_MAG_CAPTURE_MS] (provider-side draw+encode time), and the echoed
|
||||
* [KEY_MAG_REQ_T] so the client matches it to its request / drops stale frames.
|
||||
*/
|
||||
const val MSG_MAGNIFIER_FRAME = 13
|
||||
|
||||
const val KEY_MAG_X = "magX"
|
||||
const val KEY_MAG_Y = "magY"
|
||||
const val KEY_MAG_BOX_W = "magBoxW"
|
||||
const val KEY_MAG_BOX_H = "magBoxH"
|
||||
const val KEY_MAG_ZOOM = "magZoom"
|
||||
const val KEY_MAG_REQ_T = "magReqT"
|
||||
const val KEY_MAG_BYTES = "magBytes"
|
||||
const val KEY_MAG_W = "magW"
|
||||
const val KEY_MAG_H = "magH"
|
||||
const val KEY_MAG_CAPTURE_MS = "magCaptureMs"
|
||||
|
||||
const val KEY_IS_LOADING = "isLoading"
|
||||
const val KEY_LOAD_FAILED = "loadFailed"
|
||||
|
||||
|
||||
+50
-10
@@ -25,6 +25,8 @@ import android.content.ComponentName
|
||||
import android.content.Context
|
||||
import android.content.Intent
|
||||
import android.content.ServiceConnection
|
||||
import android.graphics.Bitmap
|
||||
import android.graphics.Canvas
|
||||
import android.net.Uri
|
||||
import android.os.Build
|
||||
import android.os.Bundle
|
||||
@@ -33,6 +35,7 @@ import android.os.IBinder
|
||||
import android.os.Looper
|
||||
import android.os.Message
|
||||
import android.os.Messenger
|
||||
import android.os.SystemClock
|
||||
import android.util.Log
|
||||
import android.webkit.ConsoleMessage
|
||||
import android.webkit.WebChromeClient
|
||||
@@ -51,6 +54,7 @@ import androidx.webkit.WebViewFeature
|
||||
import com.vitorpamplona.amethyst.commons.browser.OmniboxInput
|
||||
import com.vitorpamplona.amethyst.commons.napplet.NappletWebContract
|
||||
import org.json.JSONObject
|
||||
import java.io.ByteArrayOutputStream
|
||||
|
||||
/**
|
||||
* Provider for the **embedded** in-app browser. Runs in the keyless `:napplet` process: it hosts the
|
||||
@@ -139,14 +143,6 @@ class NappletBrowserService : Service() {
|
||||
super.onDestroy()
|
||||
}
|
||||
|
||||
private fun applyNightMode(themeType: String) {
|
||||
val uiManager = getSystemService(android.content.Context.UI_MODE_SERVICE) as android.app.UiModeManager
|
||||
when (themeType) {
|
||||
"DARK" -> uiManager.nightMode = android.app.UiModeManager.MODE_NIGHT_YES
|
||||
"LIGHT" -> uiManager.nightMode = android.app.UiModeManager.MODE_NIGHT_NO
|
||||
}
|
||||
}
|
||||
|
||||
private fun tabFor(msg: Message): BrowserTab? = msg.data?.getString(NappletBrowserContract.KEY_SESSION_ID)?.let { tabs[it] }
|
||||
|
||||
private fun onClientMessage(msg: Message): Boolean {
|
||||
@@ -164,7 +160,6 @@ class NappletBrowserService : Service() {
|
||||
bgColor = data.getInt(NappletBrowserContract.KEY_BG_COLOR, android.graphics.Color.WHITE),
|
||||
themeType = data.getString(NappletBrowserContract.KEY_THEME).orEmpty().ifBlank { "SYSTEM" },
|
||||
)
|
||||
applyNightMode(tab.themeType)
|
||||
tabs[sessionId] = tab
|
||||
// Bind the broker once; a re-sent MSG_CREATE_SESSION (e.g. client reconnect) must not
|
||||
// leak a second binding.
|
||||
@@ -190,11 +185,56 @@ class NappletBrowserService : Service() {
|
||||
// reload the one the user toggled.
|
||||
applyWebViewProxy(if (tab.useTor) tab.proxyPort else -1) { tab.webView?.reload() }
|
||||
}
|
||||
NappletBrowserContract.MSG_MAGNIFIER_REQUEST -> onMagnifierRequest(msg)
|
||||
else -> return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// One reusable output bitmap per tab would be ideal, but loupe size is fixed per drag; createBitmap each
|
||||
// frame is cheap next to the draw. Source rect is in view px (== surface px, the SCVH is 1:1).
|
||||
private fun onMagnifierRequest(msg: Message) {
|
||||
val tab = tabFor(msg) ?: return
|
||||
val wv = tab.webView ?: return
|
||||
val data = msg.data ?: return
|
||||
val cx = data.getFloat(NappletBrowserContract.KEY_MAG_X)
|
||||
val cy = data.getFloat(NappletBrowserContract.KEY_MAG_Y)
|
||||
val boxW = data.getInt(NappletBrowserContract.KEY_MAG_BOX_W, 150).coerceIn(16, 1024)
|
||||
val boxH = data.getInt(NappletBrowserContract.KEY_MAG_BOX_H, 84).coerceIn(16, 1024)
|
||||
val zoom = data.getFloat(NappletBrowserContract.KEY_MAG_ZOOM, 1.6f).coerceIn(1f, 4f)
|
||||
val reqT = data.getLong(NappletBrowserContract.KEY_MAG_REQ_T)
|
||||
|
||||
val outW = (boxW * zoom).toInt().coerceAtLeast(1)
|
||||
val outH = (boxH * zoom).toInt().coerceAtLeast(1)
|
||||
val t0 = SystemClock.elapsedRealtimeNanos()
|
||||
val bitmap = Bitmap.createBitmap(outW, outH, Bitmap.Config.ARGB_8888)
|
||||
val canvas = Canvas(bitmap)
|
||||
canvas.drawColor(tab.bgColor)
|
||||
// Map the source rect (centered on cx,cy in view px) into the zoomed output bitmap.
|
||||
canvas.scale(zoom, zoom)
|
||||
canvas.translate(-(cx - boxW / 2f), -(cy - boxH / 2f))
|
||||
wv.draw(canvas)
|
||||
|
||||
val baos = ByteArrayOutputStream()
|
||||
bitmap.compress(Bitmap.CompressFormat.PNG, 100, baos)
|
||||
val bytes = baos.toByteArray()
|
||||
bitmap.recycle()
|
||||
val captureMs = (SystemClock.elapsedRealtimeNanos() - t0) / 1_000_000.0
|
||||
|
||||
val reply =
|
||||
Message.obtain(null, NappletBrowserContract.MSG_MAGNIFIER_FRAME).apply {
|
||||
this.data =
|
||||
Bundle().apply {
|
||||
putByteArray(NappletBrowserContract.KEY_MAG_BYTES, bytes)
|
||||
putInt(NappletBrowserContract.KEY_MAG_W, outW)
|
||||
putInt(NappletBrowserContract.KEY_MAG_H, outH)
|
||||
putDouble(NappletBrowserContract.KEY_MAG_CAPTURE_MS, captureMs)
|
||||
putLong(NappletBrowserContract.KEY_MAG_REQ_T, reqT)
|
||||
}
|
||||
}
|
||||
runCatching { tab.clientMessenger?.send(reply) }
|
||||
}
|
||||
|
||||
/** Builds the SandboxedUiAdapter for [tab] and ships its cross-process handle (coreLibInfo) to the client. */
|
||||
private fun replyWithAdapter(tab: BrowserTab) {
|
||||
val adapter = NappletBrowserUiAdapter(this, tab.sessionId)
|
||||
@@ -218,7 +258,7 @@ class NappletBrowserService : Service() {
|
||||
// The session may have been closed between MSG_CREATE_SESSION and this posted call — fail rather
|
||||
// than build a WebView that no tab tracks (it would leak).
|
||||
val tab = tabs[sessionId] ?: error("No browser tab for session $sessionId")
|
||||
val wv = WebView(context)
|
||||
val wv = WebView(nightThemedContext(context, tab.themeType))
|
||||
configureWebView(wv, tab)
|
||||
// Theme the pre-load background so a blank/loading page shows Amethyst's background, not white.
|
||||
wv.setBackgroundColor(tab.bgColor)
|
||||
|
||||
+23
@@ -84,6 +84,29 @@ object NappletEmbedContract {
|
||||
*/
|
||||
const val MSG_LOAD_STATE = 15
|
||||
|
||||
/**
|
||||
* Client → provider: capture a magnified slice of the live page for the selection loupe. Host-side
|
||||
* `PixelCopy` of the sandbox surface returns `ERROR_SOURCE_NO_DATA` (the WebView pixels live in a child
|
||||
* SurfaceControl), so capture happens here, in the provider, where the WebView is a real in-window view.
|
||||
* Carries [KEY_MAG_X]/[KEY_MAG_Y] (surface px center), [KEY_MAG_BOX_W]/[KEY_MAG_BOX_H] (source rect, px),
|
||||
* [KEY_MAG_ZOOM], and [KEY_MAG_REQ_T] (echoed send stamp). Mirrors the browser path.
|
||||
*/
|
||||
const val MSG_MAGNIFIER_REQUEST = 16
|
||||
|
||||
/** Provider → client: the captured loupe frame — [KEY_MAG_BYTES] PNG, [KEY_MAG_W]/[KEY_MAG_H], [KEY_MAG_CAPTURE_MS], echoed [KEY_MAG_REQ_T]. */
|
||||
const val MSG_MAGNIFIER_FRAME = 17
|
||||
|
||||
const val KEY_MAG_X = "magX"
|
||||
const val KEY_MAG_Y = "magY"
|
||||
const val KEY_MAG_BOX_W = "magBoxW"
|
||||
const val KEY_MAG_BOX_H = "magBoxH"
|
||||
const val KEY_MAG_ZOOM = "magZoom"
|
||||
const val KEY_MAG_REQ_T = "magReqT"
|
||||
const val KEY_MAG_BYTES = "magBytes"
|
||||
const val KEY_MAG_W = "magW"
|
||||
const val KEY_MAG_H = "magH"
|
||||
const val KEY_MAG_CAPTURE_MS = "magCaptureMs"
|
||||
|
||||
const val KEY_CORE_LIB_INFO = "coreLibInfo"
|
||||
const val KEY_CAN_GO_BACK = "canGoBack"
|
||||
const val KEY_IS_LOADING = "isLoading"
|
||||
|
||||
+11
-13
@@ -205,8 +205,6 @@ class NappletHostActivity : ComponentActivity() {
|
||||
return
|
||||
}
|
||||
|
||||
applyNightMode()
|
||||
|
||||
if (!WebViewFeature.isFeatureSupported(WebViewFeature.WEB_MESSAGE_LISTENER)) {
|
||||
Toast.makeText(this, getString(R.string.napplet_webview_too_old), Toast.LENGTH_LONG).show()
|
||||
finish()
|
||||
@@ -230,7 +228,9 @@ class NappletHostActivity : ComponentActivity() {
|
||||
// Create + warm the WebView NOW so its (slow, first-in-process) Chromium init runs on the main
|
||||
// thread concurrently with the index probe below (which runs on IO) — instead of serially after
|
||||
// it. Binding the broker early overlaps too. The WebView is attached once the probe succeeds.
|
||||
webView = WebView(this)
|
||||
// Built from a context forced to the app theme so its content follows DARK/LIGHT regardless of the
|
||||
// device theme (WebView reads the context's theme, not the window's — see nightThemedContext).
|
||||
webView = WebView(nightThemedContext(this, themeType))
|
||||
hardenWebView(webView)
|
||||
// Theme the WebView's pre-paint background to the app's so it doesn't flash white when the shell
|
||||
// mounts. This activity has a themed context, so it resolves the color locally (no IPC needed).
|
||||
@@ -321,7 +321,6 @@ class NappletHostActivity : ComponentActivity() {
|
||||
super.onResume()
|
||||
if (this::webView.isInitialized) {
|
||||
webView.onResume()
|
||||
webView.resumeTimers()
|
||||
}
|
||||
// Launching this :napplet-process surface backgrounded the main process; tell the broker to
|
||||
// hold the main process resumed (Tor/relays/AUTH) while this napplet/nSite is in front, and
|
||||
@@ -335,8 +334,10 @@ class NappletHostActivity : ComponentActivity() {
|
||||
// sign/decrypt/pay request whose consent prompt would surface over (and be confused with)
|
||||
// Amethyst's own UI. Requests only happen while the user is looking at this napplet.
|
||||
if (this::webView.isInitialized) {
|
||||
// webView.onPause() pauses THIS WebView's JS/DOM (the security goal — a backgrounded napplet can't
|
||||
// fire a sign/decrypt/pay request). Do NOT call pauseTimers(): it's process-global and freezes
|
||||
// EVERY WebView in `:napplet`, including the embedded browser/napplet surfaces, which never resume.
|
||||
webView.onPause()
|
||||
webView.pauseTimers()
|
||||
}
|
||||
// No longer foreground: stop renewing and let the main process resume normal background scaling.
|
||||
resumed = false
|
||||
@@ -384,6 +385,11 @@ class NappletHostActivity : ComponentActivity() {
|
||||
runCatching { unbindService(brokerConnection) }
|
||||
keyActions.clear()
|
||||
if (this::webView.isInitialized) {
|
||||
// Detach before destroy(): destroying an attached WebView corrupts the shared multiprocess
|
||||
// renderer/network state and breaks the other (embedded) WebViews in this `:napplet` process
|
||||
// (dead DNS, empty DOM reads, dead selection paint, broken IME). See NappletBrowserActivity.
|
||||
webView.stopLoading()
|
||||
(webView.parent as? ViewGroup)?.removeView(webView)
|
||||
webView.destroy()
|
||||
}
|
||||
super.onDestroy()
|
||||
@@ -803,14 +809,6 @@ class NappletHostActivity : ComponentActivity() {
|
||||
Toast.makeText(this, message, Toast.LENGTH_SHORT).show()
|
||||
}
|
||||
|
||||
private fun applyNightMode() {
|
||||
val uiManager = getSystemService(android.content.Context.UI_MODE_SERVICE) as android.app.UiModeManager
|
||||
when (themeType) {
|
||||
"DARK" -> uiManager.nightMode = android.app.UiModeManager.MODE_NIGHT_YES
|
||||
"LIGHT" -> uiManager.nightMode = android.app.UiModeManager.MODE_NIGHT_NO
|
||||
}
|
||||
}
|
||||
|
||||
private fun resolveThemeColor(attr: Int): Int {
|
||||
val tv = TypedValue()
|
||||
theme.resolveAttribute(attr, tv, true)
|
||||
|
||||
+57
-20
@@ -25,6 +25,8 @@ import android.content.ComponentName
|
||||
import android.content.Context
|
||||
import android.content.Intent
|
||||
import android.content.ServiceConnection
|
||||
import android.graphics.Bitmap
|
||||
import android.graphics.Canvas
|
||||
import android.net.Uri
|
||||
import android.os.Build
|
||||
import android.os.Bundle
|
||||
@@ -33,6 +35,7 @@ import android.os.IBinder
|
||||
import android.os.Looper
|
||||
import android.os.Message
|
||||
import android.os.Messenger
|
||||
import android.os.SystemClock
|
||||
import android.util.Log
|
||||
import android.view.View
|
||||
import android.webkit.WebResourceError
|
||||
@@ -57,6 +60,7 @@ import com.vitorpamplona.quartz.nip01Core.core.toHexKey
|
||||
import com.vitorpamplona.quartz.nip5aStaticWebsites.tags.PathTag
|
||||
import com.vitorpamplona.quartz.utils.sha256.sha256
|
||||
import org.json.JSONObject
|
||||
import java.io.ByteArrayOutputStream
|
||||
import java.util.concurrent.Executor
|
||||
|
||||
/**
|
||||
@@ -168,34 +172,22 @@ class NappletHostService : Service() {
|
||||
}
|
||||
NappletEmbedContract.MSG_BACK -> tabFor(msg)?.webView?.let { if (it.canGoBack()) it.goBack() }
|
||||
NappletEmbedContract.MSG_RELOAD -> tabFor(msg)?.webView?.reload()
|
||||
NappletEmbedContract.MSG_PAUSE ->
|
||||
tabFor(msg)?.webView?.let {
|
||||
it.onPause()
|
||||
it.pauseTimers()
|
||||
}
|
||||
NappletEmbedContract.MSG_RESUME ->
|
||||
tabFor(msg)?.webView?.let {
|
||||
it.onResume()
|
||||
it.resumeTimers()
|
||||
}
|
||||
// onPause()/onResume() are per-WebView (pause/resume THIS surface's JS/DOM). Do NOT call
|
||||
// pauseTimers()/resumeTimers(): they are process-global and would freeze/thaw every WebView in
|
||||
// `:napplet` (the browser embed + other napplets), whose lifecycles are independent of this one.
|
||||
NappletEmbedContract.MSG_PAUSE -> tabFor(msg)?.webView?.onPause()
|
||||
NappletEmbedContract.MSG_RESUME -> tabFor(msg)?.webView?.onResume()
|
||||
NappletEmbedContract.MSG_IME_OP -> {
|
||||
val tab = tabFor(msg) ?: return true
|
||||
val payload = msg.data?.getString(NappletEmbedContract.KEY_IME_PAYLOAD) ?: return true
|
||||
tab.bridgeReplyProxy?.postMessage(payload)
|
||||
}
|
||||
NappletEmbedContract.MSG_MAGNIFIER_REQUEST -> onMagnifierRequest(msg)
|
||||
else -> return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
private fun applyNightMode(themeType: String) {
|
||||
val uiManager = getSystemService(android.content.Context.UI_MODE_SERVICE) as android.app.UiModeManager
|
||||
when (themeType) {
|
||||
"DARK" -> uiManager.nightMode = android.app.UiModeManager.MODE_NIGHT_YES
|
||||
"LIGHT" -> uiManager.nightMode = android.app.UiModeManager.MODE_NIGHT_NO
|
||||
}
|
||||
}
|
||||
|
||||
private fun buildTab(msg: Message): NappletTab? {
|
||||
val data = msg.data ?: return null
|
||||
val sessionId = data.getString(NappletEmbedContract.KEY_SESSION_ID) ?: return null
|
||||
@@ -225,10 +217,55 @@ class NappletHostService : Service() {
|
||||
themeType = data.getString(NappletHostContract.EXTRA_THEME).orEmpty().ifBlank { "SYSTEM" },
|
||||
declaredDomains = declaredDomains,
|
||||
)
|
||||
applyNightMode(tab.themeType)
|
||||
return tab
|
||||
}
|
||||
|
||||
/**
|
||||
* Draw a zoomed slice of the live WebView (the loupe content) and ship it back as a PNG. The WebView is a
|
||||
* real in-window view here in the provider, so its software draw renders real DOM pixels — unlike host-side
|
||||
* `PixelCopy` of the sandbox surface. Mirror of the browser path. Runs on the main looper (`WebView.draw`).
|
||||
*/
|
||||
private fun onMagnifierRequest(msg: Message) {
|
||||
val tab = tabFor(msg) ?: return
|
||||
val wv = tab.webView ?: return
|
||||
val data = msg.data ?: return
|
||||
val cx = data.getFloat(NappletEmbedContract.KEY_MAG_X)
|
||||
val cy = data.getFloat(NappletEmbedContract.KEY_MAG_Y)
|
||||
val boxW = data.getInt(NappletEmbedContract.KEY_MAG_BOX_W, 150).coerceIn(16, 1024)
|
||||
val boxH = data.getInt(NappletEmbedContract.KEY_MAG_BOX_H, 84).coerceIn(16, 1024)
|
||||
val zoom = data.getFloat(NappletEmbedContract.KEY_MAG_ZOOM, 1.6f).coerceIn(1f, 4f)
|
||||
val reqT = data.getLong(NappletEmbedContract.KEY_MAG_REQ_T)
|
||||
|
||||
val outW = (boxW * zoom).toInt().coerceAtLeast(1)
|
||||
val outH = (boxH * zoom).toInt().coerceAtLeast(1)
|
||||
val t0 = SystemClock.elapsedRealtimeNanos()
|
||||
val bitmap = Bitmap.createBitmap(outW, outH, Bitmap.Config.ARGB_8888)
|
||||
val canvas = Canvas(bitmap)
|
||||
canvas.drawColor(tab.bgColor)
|
||||
canvas.scale(zoom, zoom)
|
||||
canvas.translate(-(cx - boxW / 2f), -(cy - boxH / 2f))
|
||||
wv.draw(canvas)
|
||||
|
||||
val baos = ByteArrayOutputStream()
|
||||
bitmap.compress(Bitmap.CompressFormat.PNG, 100, baos)
|
||||
val bytes = baos.toByteArray()
|
||||
bitmap.recycle()
|
||||
val captureMs = (SystemClock.elapsedRealtimeNanos() - t0) / 1_000_000.0
|
||||
|
||||
val reply =
|
||||
Message.obtain(null, NappletEmbedContract.MSG_MAGNIFIER_FRAME).apply {
|
||||
this.data =
|
||||
Bundle().apply {
|
||||
putByteArray(NappletEmbedContract.KEY_MAG_BYTES, bytes)
|
||||
putInt(NappletEmbedContract.KEY_MAG_W, outW)
|
||||
putInt(NappletEmbedContract.KEY_MAG_H, outH)
|
||||
putDouble(NappletEmbedContract.KEY_MAG_CAPTURE_MS, captureMs)
|
||||
putLong(NappletEmbedContract.KEY_MAG_REQ_T, reqT)
|
||||
}
|
||||
}
|
||||
runCatching { tab.clientMessenger?.send(reply) }
|
||||
}
|
||||
|
||||
/** Builds the SandboxedUiAdapter for [tab] and ships its cross-process handle (coreLibInfo) to the client. */
|
||||
private fun replyWithAdapter(tab: NappletTab) {
|
||||
val adapter = NappletHostUiAdapter(this, tab.sessionId)
|
||||
@@ -252,7 +289,7 @@ class NappletHostService : Service() {
|
||||
// The session may have been closed between MSG_CREATE_SESSION and this posted call — fail rather
|
||||
// than build a WebView that no tab tracks (it would leak).
|
||||
val tab = tabs[sessionId] ?: error("No napplet tab for session $sessionId")
|
||||
val wv = WebView(context)
|
||||
val wv = WebView(nightThemedContext(context, tab.themeType))
|
||||
val appOrigin = NappletWebContract.appOrigin(deriveAppId(tab.author, tab.identifier))
|
||||
val effectiveProxy = if (tab.useTor) tab.proxyPort else -1
|
||||
tab.contentServer = NappletContentServer(tab.paths, tab.servers, effectiveProxy, cacheDir, shellHtml, shimJs, appOrigin, tab.profile, imeProxy = true)
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
# IME / text-selection test harness
|
||||
|
||||
A single-file web page (`index.html`) for exercising and profiling the embedded
|
||||
WebView IME + text-selection relay (see
|
||||
`amethyst/plans/2026-06-25-embed-text-selection-native-parity.md`). It has a
|
||||
plain `<input>` and a `<textarea>` plus an on-page green log that records, with
|
||||
millisecond timestamps:
|
||||
|
||||
- focus/blur, `selectionchange`, `keydown`/`beforeinput`/`input`, composition
|
||||
events, and the resulting `value`/selection — to catch erase, caret-jump, and
|
||||
focus-transfer regressions;
|
||||
- **paint latency** (`requestAnimationFrame` after each DOM change) — the metric
|
||||
that exposed the first-letter freeze;
|
||||
- **long-task** + **main-thread-block** detectors and a focus/selection
|
||||
**heartbeat** — to catch anything stalling the WebView main thread or
|
||||
spontaneously moving focus/selection.
|
||||
|
||||
The log lines are tagged `[ImeDiag]` and also go to `console.log`, so they show
|
||||
up in `adb logcat` (the `:napplet` process owns the WebView console). Nothing
|
||||
here ships in the app — it's a dev tool, which is why the `[ImeDiag]` strings
|
||||
live only under `tools/`.
|
||||
|
||||
## Run it
|
||||
|
||||
1. Serve this directory over HTTP from your dev machine:
|
||||
|
||||
```bash
|
||||
cd tools/ime-test && python3 -m http.server 8765
|
||||
```
|
||||
|
||||
2. Reach it from the device/emulator:
|
||||
- **Emulator:** the page is at `http://10.0.2.2:8765` (`10.0.2.2` is the
|
||||
emulator's alias for the host loopback).
|
||||
- **Physical device (USB):** `adb reverse tcp:8765 tcp:8765`, then the page is
|
||||
at `http://localhost:8765`.
|
||||
|
||||
3. Open that URL as an **embedded** tab (this is the path that uses the relay —
|
||||
*not* a full-screen activity):
|
||||
- Open the in-app browser (`BrowserScreen`) and type the URL into its address
|
||||
bar. The embedded browser handles `http`/`https`, so it loads into the
|
||||
`:napplet` SurfaceControlViewHost surface.
|
||||
|
||||
To compare against native behavior, open the same URL in a full-screen
|
||||
activity (where the WebView renders in-window with the native keyboard) — that
|
||||
is also how you reproduce the **full-screen round-trip highlight bug** (open
|
||||
full-screen, `back`, then selection highlight is dead across all embeds).
|
||||
|
||||
## Reading the log
|
||||
|
||||
- `INPUT … val=… sel=…` right after a keystroke with the right value = no erase.
|
||||
- `PAINT-LATENCY Nms` spiking to ~1000ms = the first-letter freeze (should stay
|
||||
low now that the surface no longer resizes on IME show).
|
||||
- `MAINTHREAD BLOCKED` / `LONGTASK` = something is stalling the WebView thread.
|
||||
- `HEARTBEAT` lines changing while idle = spontaneous focus/selection drift.
|
||||
@@ -0,0 +1,94 @@
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>IME Test</title>
|
||||
<style>
|
||||
body { font-family: sans-serif; padding: 16px; background:#f4f4fa; }
|
||||
h2 { margin: 24px 0 6px; }
|
||||
input, textarea { width: 100%; font-size: 18px; padding: 12px; box-sizing: border-box; border: 2px solid #88a; border-radius: 8px; }
|
||||
#log { white-space: pre-wrap; font: 11px monospace; background:#111; color:#0f0; padding:8px; height:200px; overflow:auto; margin-top:16px; border-radius:6px; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h2>Plain text input</h2>
|
||||
<input id="inp" type="text" placeholder="type here" value="hello world">
|
||||
<h2>Textarea</h2>
|
||||
<textarea id="ta" rows="2" placeholder="type here">hello world</textarea>
|
||||
<div id="log"></div>
|
||||
<!-- Tall spacer so the page scrolls — lets us verify the host hides selection UI during scroll. -->
|
||||
<div style="height:2000px; padding-top:16px; color:#888;">scroll region — drag up/down here</div>
|
||||
<script>
|
||||
(function(){
|
||||
var t0 = performance.now();
|
||||
function ts(){ return (performance.now() - t0).toFixed(0); }
|
||||
var logEl = document.getElementById('log');
|
||||
function log(m){
|
||||
var line = '[ImeDiag] +' + ts() + 'ms ' + m;
|
||||
try { console.log(line); } catch(_){}
|
||||
if (logEl) { logEl.textContent += line + '\n'; logEl.scrollTop = logEl.scrollHeight; }
|
||||
}
|
||||
function selOf(n){ return (n && n.selectionStart != null) ? (n.selectionStart + '..' + n.selectionEnd) : '?'; }
|
||||
function active(){ var a = document.activeElement; return a ? (a.tagName + '#' + (a.id||'')) : 'none'; }
|
||||
|
||||
['inp','ta'].forEach(function(id){
|
||||
var n = document.getElementById(id);
|
||||
n.addEventListener('focus', function(){ log(id + ' FOCUS hasFocus=' + document.hasFocus()); });
|
||||
n.addEventListener('blur', function(){ log(id + ' BLUR'); });
|
||||
n.addEventListener('keydown', function(e){ log(id + ' keydown key=' + e.key + ' isComposing=' + e.isComposing); });
|
||||
n.addEventListener('beforeinput', function(e){ log(id + ' beforeinput type=' + e.inputType + ' data=' + JSON.stringify(e.data) + ' isComposing=' + e.isComposing); });
|
||||
n.addEventListener('input', function(e){
|
||||
log(id + ' INPUT type=' + e.inputType + ' data=' + JSON.stringify(e.data) + ' isComposing=' + e.isComposing + ' val=' + JSON.stringify(n.value) + ' sel=' + selOf(n));
|
||||
// Objective paint-latency: how long until the NEXT frame is produced after this DOM change.
|
||||
var t = performance.now();
|
||||
requestAnimationFrame(function(){ log(id + ' PAINT-LATENCY ' + (performance.now() - t).toFixed(0) + 'ms (DOM change -> next frame)'); });
|
||||
});
|
||||
n.addEventListener('compositionstart', function(e){ log(id + ' compositionSTART data=' + JSON.stringify(e.data)); });
|
||||
n.addEventListener('compositionupdate', function(e){ log(id + ' compositionUPDATE data=' + JSON.stringify(e.data)); });
|
||||
n.addEventListener('compositionend', function(e){ log(id + ' compositionEND data=' + JSON.stringify(e.data)); });
|
||||
n.addEventListener('select', function(){ log(id + ' SELECT event sel=' + selOf(n)); });
|
||||
});
|
||||
|
||||
document.addEventListener('selectionchange', function(){
|
||||
var a = document.activeElement;
|
||||
if (a && (a.id === 'inp' || a.id === 'ta')) log('selectionchange ' + a.id + ' sel=' + selOf(a) + ' hasFocus=' + document.hasFocus());
|
||||
}, true);
|
||||
|
||||
window.addEventListener('focus', function(){ log('WINDOW focus hasFocus=' + document.hasFocus()); }, true);
|
||||
window.addEventListener('blur', function(){ log('WINDOW blur hasFocus=' + document.hasFocus()); }, true);
|
||||
document.addEventListener('visibilitychange', function(){ log('visibilitychange hidden=' + document.hidden); });
|
||||
|
||||
// Heartbeat: detect any spontaneous oscillation of focus/selection while idle.
|
||||
var last = '';
|
||||
setInterval(function(){
|
||||
var a = document.activeElement;
|
||||
var snap = 'hasFocus=' + document.hasFocus() + ' vis=' + document.visibilityState + ' active=' + active() + ' sel=' + (a && a.selectionStart != null ? selOf(a) : '-');
|
||||
if (snap !== last) { last = snap; log('HEARTBEAT ' + snap); }
|
||||
}, 250);
|
||||
|
||||
// Long-task observer: catches any >50ms block of the WebView main thread and its attribution.
|
||||
try {
|
||||
new PerformanceObserver(function(list){
|
||||
list.getEntries().forEach(function(e){
|
||||
var attr = (e.attribution && e.attribution[0]) ? (e.attribution[0].name + '/' + e.attribution[0].containerType) : '';
|
||||
log('LONGTASK ' + e.duration.toFixed(0) + 'ms name=' + e.name + ' ' + attr);
|
||||
});
|
||||
}).observe({ entryTypes: ['longtask'] });
|
||||
} catch(_) { log('no longtask observer'); }
|
||||
|
||||
// Main-thread block detector via MessageChannel (fires a macrotask ASAP each tick; if it's late, main was blocked).
|
||||
var mc = new MessageChannel();
|
||||
var tickExpected = performance.now();
|
||||
mc.port1.onmessage = function(){
|
||||
var now = performance.now(), late = now - tickExpected;
|
||||
if (late > 150) log('MAINTHREAD BLOCKED ~' + late.toFixed(0) + 'ms');
|
||||
tickExpected = now + 8;
|
||||
setTimeout(function(){ mc.port2.postMessage(0); }, 8);
|
||||
};
|
||||
mc.port2.postMessage(0);
|
||||
|
||||
log('test page loaded vis=' + document.visibilityState);
|
||||
})();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
Reference in New Issue
Block a user