mirror of
https://github.com/vitorpamplona/amethyst.git
synced 2026-10-05 19:28:25 +00:00
feat(embed): selection loupe + native-parity fixes for embedded text selection
Builds out host-drawn text selection for embedded napplet/nsite/browser surfaces toward native parity, and fixes the bugs found while exercising it. - Magnifier loupe (#4): EmbeddedMagnifier + provider-side pixel capture (EmbeddedMagnifierProbe) shipped over IPC for both embed paths; the caret/selection handles drive it via OnMagnify. - SelectionUiState: single source of truth for the overlay show/hide rules (insertion caret / in-field range / page-text range + dragging/scrolling). - EmbeddedSelectionDrag: suspends the nav drawer's edge swipe while a handle is dragged (auto-scroll #9). Bug fixes: - No more overlay blink on word-select: the shim's selection-reveal scrolls (a textarea auto-scrolling to show a forming/re-asserted range) no longer trip the hide-on-scroll path, and the hide self-heals instead of being re-armed indefinitely. - RemoteImeView debounces the range-lost signal so a transient collapse that gets re-asserted doesn't flicker the handles/toolbar. - Focusing a field clears any page-text selection (shim + host), so the stale page handles/Copy bar no longer linger above — and stop stealing drags from — the field overlays; also cancels any in-flight scroll-hide on focus. - Caret insertion-handle drag now actually moves the caret: read the pointer delta with positionChangeIgnoreConsumed() before consuming, so the value isn't zeroed by our own consume (or the sandbox surface consuming the move). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
8232cd50a3
commit
dec1e0bec1
@@ -31,18 +31,18 @@ rules, so we can check off coverage. Native impl lives in `android.widget.Editor
|
|||||||
|
|
||||||
| # | Native feature | Activates | Deactivates | Our status |
|
| # | Native feature | Activates | Deactivates | Our status |
|
||||||
|---|----------------|-----------|-------------|------------|
|
|---|----------------|-----------|-------------|------------|
|
||||||
| 1 | **Insertion handle** (the teardrop "blob" under the caret) | tap in editable text; tap again to re-show | typing, scroll start, focus loss, ~4s inactivity timeout | ✅ `InsertionHandle` — shows on caret geometry, hides via `onEdited`. **Missing the inactivity timeout and the re-show-on-tap.** |
|
| 1 | **Insertion handle** (the teardrop "blob" under the caret) | tap in editable text; tap again to re-show | typing, scroll start, focus loss, ~4s inactivity timeout | ✅ `InsertionHandle`. **Native availability rule now matched (2026-06-25):** only shown when the field is NON-EMPTY (`Editor` gates the handle behind `text.length() > 0`, via `SelectionUiState.fieldHasText`) — fixes it popping up on focus of an empty box; hides on typing (`onEdited`), scroll (`scrolling`), focus loss, and ~4s inactivity (`hideCaret` timeout), re-showing on the next tap — via an explicit `ime.carettap` shim signal (DOM `click`), so a tap that doesn't move the caret still re-shows it. Device-verified. |
|
||||||
| 2 | **Selection handles** (asymmetric left/right teardrops) | long-press word, double-tap word, drag-extend | tap-collapse, typing, new selection | ✅ `SelectionHandle(isStart)` + drag-to-extend via `pageExtend`/caretmove |
|
| 2 | **Selection handles** (asymmetric left/right teardrops) | long-press word, double-tap word, drag-extend | tap-collapse, typing, new selection | ✅ `SelectionHandle(isStart)` + drag-to-extend, for BOTH plain page text (`pageExtend`) AND in-field `<input>`/`<textarea>` selections (`fieldExtend`, 2026-06-25). The shim reports the selection's caret feet (`sx/sb`,`ex/eb`, flagged `rng`) via the same mirror-div as the caret; the host holds the range geometry separately so Chrome's transient collapse-to-caret (the re-assert fight) doesn't yank the handles to the field edges. **Tap-to-collapse (2026-06-25):** a single tap inside a selection dismisses it to a caret at the tapped offset + insertion handle — the shim's `click` handler collapses explicitly via `offsetFromPoint` (off-window Chrome doesn't do it itself). Device-verified. |
|
||||||
| 3 | **Floating toolbar** (Cut/Copy/Paste/Select-All/Share/…) | selection made, or tap insertion handle (Paste/Select-All) | scroll/fling (hides, returns on settle), handle drag (hides), tap-collapse | ⚠️ `EmbeddedSelectionToolbar` has Cut/Copy/Paste/Select-All only. **No hide-during-drag, no hide-during-scroll, no insertion-handle Paste popup, no overflow, no Share/Web-Search/process-text.** |
|
| 3 | **Floating toolbar** (Cut/Copy/Paste/Select-All/Share/…) | selection made, or tap insertion handle (Paste/Select-All) | scroll/fling (hides, returns on settle), handle drag (hides), tap-collapse | ⚠️ `EmbeddedSelectionToolbar` (Cut/Copy/Paste/Select-All). **Hide-during-handle-drag ✅ device-verified; hide-during-scroll ✅ implemented (2026-06-25, via `SelectionUiState.dragging`/`scrolling`; shim sends `ime.scroll` + re-reports geometry on settle) — same gate as drag-hide, but not yet exercised on device (the embedded WebView didn't scroll under injected swipes / cached short harness).** Still missing: insertion-handle Paste popup, overflow, Share/Web-Search/process-text. |
|
||||||
| 4 | **Magnifier / loupe** (the zoom bubble above the finger while dragging a handle or the caret) | finger down + moving on a handle or the caret | finger up | ❌ **Not built.** This is the "zoom bubble" called out — highest-value missing piece. |
|
| 4 | **Magnifier / loupe** (the zoom bubble above the finger while dragging a handle or the caret) | finger down + moving on a handle or the caret | finger up | ✅ **Built (2026-06-25).** `Magnifier` bubble in [EmbeddedMagnifier.kt] follows the dragged caret/selection handle, showing live magnified page pixels captured in the `:napplet` provider and shipped over IPC ([EmbeddedMagnifierProbe], option B). Both embed paths wired (browser + napplet); verified on device for the browser path. Capture Y is locked to the caret/selection line (X follows the finger). Possible further polish: RGB_565 to cut encode, themed crosshair, clamp capture X to the line so a fast drag past EOL doesn't show blank. |
|
||||||
| 5 | **Word-granularity long-press** then char-extend | long-press | — | ⚠️ partial — selection works; granularity (word-snap while dragging, then char) not modelled |
|
| 5 | **Word-granularity long-press** then char-extend | long-press | — | ✅ in-field handle-extend snaps to word boundaries (2026-06-25): `fieldExtend` snaps the dragged edge via `wordStartAt`/`wordEndAt` (end→word end, start→word start), so dragging a word selection's handle extends a word at a time. Device-verified (drag end handle slightly into "world" → whole "hello world"). Page-text extend stays char (no offset model); the "then char" fine mode within a word isn't modeled. |
|
||||||
| 6 | **Double-tap = word, long-press = word, (triple-tap/drag = paragraph)** | tap count | — | ❌ double-tap-to-select not wired (only long-press) |
|
| 6 | **Double-tap = word, long-press = word, (triple-tap/drag = paragraph)** | tap count | — | ✅ double-tap + long-press both select a word (2026-06-25). Chrome word-selects on the 2nd tap, then abandons it by collapsing to the end (off-window quirk); the host re-assert restores it. The shim `click` handler DEFERS its tap-to-collapse ~300ms and the real `dblclick` cancels that timer, so the word selection survives (a timing-only guard was flaky ~40%). Triple-tap/paragraph not done. |
|
||||||
| 7 | **Smart selection / entity expansion** (`TextClassifier`: phone, URL, address, date → entity actions in toolbar) | selection lands on an entity | — | ❌ not built (low priority) |
|
| 7 | **Smart selection / entity expansion** (`TextClassifier`: phone, URL, address, date → entity actions in toolbar) | selection lands on an entity | — | ❌ not built (low priority) |
|
||||||
| 8 | **Drag selected text** (long-press a selection → drag-and-drop to move) | long-press on existing selection | drop | ❌ not built (low priority) |
|
| 8 | **Drag selected text** (long-press a selection → drag-and-drop to move) | long-press on existing selection | drop | ❌ not built (low priority) |
|
||||||
| 9 | **Auto-scroll while dragging to a viewport edge** | handle dragged near top/bottom edge | finger leaves edge / up | ❌ not built — needed once fields can be off-screen/long |
|
| 9 | **Auto-scroll while dragging to a viewport edge** | handle dragged near top/bottom edge | finger leaves edge / up | ✅ works (2026-06-25, user-confirmed). The drag driver (`onMagnify`) detects the finger in the surface's top/bottom edge zone and sends `ime.autoscroll`; the shim scrolls the textarea (else the window) and re-reports geometry, flagged so the hide-on-scroll path doesn't fire. Scrolls per drag-move in the edge zone (not on a perfectly-held finger). **Fixed alongside:** the nav drawer's left-edge swipe was hijacking the edge drag — `EmbeddedSelectionDrag.dragging` (set by `onMagnify`) now suspends the drawer's `gesturesEnabled` while a handle is dragged. |
|
||||||
| 10 | **Caret snapping to character boundaries** | always during caret/handle drag | — | ✅ via `offsetFromPoint` binary search + Y-clamp |
|
| 10 | **Caret snapping to character boundaries** | always during caret/handle drag | — | ✅ via `offsetFromPoint` binary search + Y-clamp |
|
||||||
| 11 | **Themed handle/caret drawables + blink** | always | — | ⚠️ teardrop shape matches; color/blink not themed to `textSelectHandle`/`textCursorDrawable` |
|
| 11 | **Themed handle/caret drawables + blink** | always | — | ✅/⚠️ The host-drawn handles use `colorScheme.primary` — which IS the native `textSelectHandle`/accent color — so the handle drawables are themed (the actionable part). The caret bar + selection-highlight are drawn by Chrome inside the off-window surface: the caret already blinks natively, and theming its color/the highlight would mean injecting CSS into arbitrary third-party pages (intrusive; `::selection` was already ruled out as non-painting), so those are intentionally left to Chrome. |
|
||||||
| 12 | **Insertion-handle Paste/Select-All mini-popup** | tap the insertion handle | tap elsewhere | ❌ not built |
|
| 12 | **Insertion-handle Paste/Select-All mini-popup** | tap the insertion handle | tap elsewhere | ✅ built (2026-06-25). Tapping the bare insertion handle toggles a Paste/Select-All bar above the caret (`SelectionUiState.insertionPopup`, toggled from the handle's unified tap/drag gesture); tap-elsewhere/typing/blur/selection/scroll dismiss it. Fixed a latent bug: toolbar items now consume the *down* (not just the up) so the tap doesn't bleed through to the surface and blur the field. Device-verified (Select-all selects all text, field stays focused). |
|
||||||
|
|
||||||
Legend: ✅ done · ⚠️ partial · ❌ missing.
|
Legend: ✅ done · ⚠️ partial · ❌ missing.
|
||||||
|
|
||||||
@@ -50,30 +50,90 @@ Legend: ✅ done · ⚠️ partial · ❌ missing.
|
|||||||
|
|
||||||
Most of the bugs we already fixed were activation-timing bugs (cursor-jumps-to-end,
|
Most of the bugs we already fixed were activation-timing bugs (cursor-jumps-to-end,
|
||||||
collapse-on-tap, focus-transfer races). The remaining features each carry their
|
collapse-on-tap, focus-transfer races). The remaining features each carry their
|
||||||
own state machine. Before adding more, we should **centralize selection-UI state**
|
own state machine.
|
||||||
in one place (today it's spread across `showInsertionHandle`, `showSelectionToolbar`,
|
|
||||||
page-vs-field flags in `EmbeddedTabLayer`). A single `SelectionUiState`
|
**Done (2026-06-25): `SelectionUiState`** (`SelectionUiState.kt`) now centralizes
|
||||||
(none / insertion / range / dragging-handle / scrolling) makes the hide/show rules
|
what used to be scattered flags in `EmbeddedTabLayer` (`showInsertionHandle`,
|
||||||
(toolbar hides while dragging, magnifier shows only while dragging, handle
|
`showSelectionToolbar`, `fieldGeometry`, `rangeFieldGeometry`, `pageSelection`).
|
||||||
auto-times-out) expressible instead of emergent.
|
It holds the three mutually-exclusive contexts (insertion caret / in-field range /
|
||||||
|
page-text range) plus the transient modifiers `dragging` and `scrolling`, and
|
||||||
|
exposes derived visibility (`insertionHandle`, `fieldHandles`, `fieldToolbar`,
|
||||||
|
`pageHandles`, `pageToolbar`) so the rules are expressed once:
|
||||||
|
- **toolbar hides while a handle is dragged** (`dragging`, set from the same
|
||||||
|
`OnMagnify` lifecycle that drives the loupe) — the dragged handle also hides its
|
||||||
|
own teardrop (the loupe stands in), like Android; the other handle stays.
|
||||||
|
- **all overlays hide while scrolling** (`scrolling`, from the shim's `ime.scroll`);
|
||||||
|
the shim re-reports geometry just before `active=false` so they reappear
|
||||||
|
repositioned.
|
||||||
|
Still emergent / TODO: insertion-handle auto-timeout, tap-to-re-show.
|
||||||
|
|
||||||
|
**Selection-blink fix (2026-06-25).** A field selection — especially in a `<textarea>` —
|
||||||
|
flickered: off-window Chrome abandons the selection by collapsing the caret to an
|
||||||
|
endpoint every ~25 ms, and the FIELD re-assert round-tripped through the host EditText
|
||||||
|
(`RemoteImeView.onPageState` → `ime.set` → page), leaving a visible collapsed frame each
|
||||||
|
cycle. Fix: re-assert field selections **synchronously in the shim's `selectionchange`
|
||||||
|
handler** (mirror of the page-text path that never blinked) — `lastFieldRange`/`lastFieldAt`
|
||||||
|
tracked via `noteSel()`, and a collapse-to-endpoint within 1500 ms is reverted with `setSel`
|
||||||
|
(guarded, not re-reported) so it reverts before paint. The host re-assert stays as a
|
||||||
|
fallback. Device-verified: textarea selection is stable; input select still works.
|
||||||
|
|
||||||
## Priority order for parity work
|
## Priority order for parity work
|
||||||
|
|
||||||
1. **Magnifier (#4).** Biggest perceived gap. We already report caret/handle
|
1. **Magnifier (#4).** Biggest perceived gap. We already report caret/handle
|
||||||
geometry; the magnifier needs a *magnified pixel view of the surface* at the
|
geometry; the magnifier needs a *magnified pixel view of the surface* at the
|
||||||
drag point. Options:
|
drag point. Options:
|
||||||
- **A. Compose-side zoom of a surface snapshot.** The surface is a
|
- **A. Compose-side zoom of a surface snapshot. ❌ RULED OUT (spiked 2026-06-25).**
|
||||||
`SurfaceControlViewHost` — we can't trivially `Bitmap`-grab a remote
|
The surface is a `SurfaceControlViewHost` — we can't trivially `Bitmap`-grab a
|
||||||
surface from the main process. Would need `PixelCopy` against the
|
remote surface from the main process. We spiked `PixelCopy.request(SurfaceView, …)`
|
||||||
`SurfaceView`/`SurfaceControl`, then draw a clipped, scaled bitmap in a
|
against the live embedded surface (`SurfaceMagnifierProbe`, wired into
|
||||||
popup that follows X and stays clamped to the dragged line's Y (native
|
`EmbeddedTabLayer` behind `BuildConfig.DEBUG`, fired on field focus). The capture
|
||||||
behavior). Verify `PixelCopy` works against the sandboxed surface before
|
target is the privacysandbox `ContentView extends SurfaceView` — the only real
|
||||||
committing to this.
|
child of `SandboxedSdkView` once the session opens. **Result on a clearly-painted
|
||||||
- **B. Render the magnifier in `:napplet`.** Have `shim.js`/the host WebView
|
surface (1080×2088): every capture returns `ERROR_SOURCE_NO_DATA`** (center
|
||||||
produce the loupe content (a zoomed DOM render) and ship it. Heavier, but
|
region, repeated 3×). Cause: the WebView pixels live in a *child* `SurfaceControl`
|
||||||
sidesteps cross-process pixel capture.
|
reparented under the SurfaceView via `ContentView.setChildSurfacePackage(...)`; the
|
||||||
- Start by spiking A with `PixelCopy.request(SurfaceView, …)`; if it returns
|
host SurfaceView's *own* buffer is never drawn into, so `PixelCopy` on the parent
|
||||||
a valid bitmap for our surface, A is far simpler.
|
reads an empty buffer. Host-side pixel capture of the sandboxed content is not
|
||||||
|
available. (A `PixelCopy.request(Window, …)` against the host window would also
|
||||||
|
miss it — the sandbox layer is a *separate* SurfaceControl z-ordered below the
|
||||||
|
window.)
|
||||||
|
- **B. Capture in `:napplet` and ship the loupe content. ✅ SPIKED & VIABLE
|
||||||
|
(2026-06-25).** Inside the keyless provider the WebView IS a real in-window view, so
|
||||||
|
`WebView.draw(Canvas)` into a software bitmap renders real DOM pixels. Spike added
|
||||||
|
`MSG_MAGNIFIER_REQUEST`/`MSG_MAGNIFIER_FRAME` to `NappletBrowserContract`:
|
||||||
|
`NappletBrowserService.onMagnifierRequest` draws a zoomed slice
|
||||||
|
(`canvas.scale(zoom); translate(-(cx-box/2), -(cy-box/2)); webView.draw(canvas)`),
|
||||||
|
PNG-encodes it, and ships the bytes back; `EmbeddedBrowserController` (now also an
|
||||||
|
`EmbeddedMagnifierProbe`) requests on focus and `EmbeddedTabLayer` logs the result.
|
||||||
|
**10-frame burst, 160px source × 1.5× zoom → 240×240 PNG, center `#FF111111`
|
||||||
|
(real opaque content):** provider draw 0.7–1.2 ms steady (≈5 ms cold), provider
|
||||||
|
total draw+PNG 3–4 ms steady (≈12 ms cold), client round-trip 4–8 ms steady
|
||||||
|
(occasional ~18 ms), payload 8–15 KB (far under the 1 MB Binder limit). Comfortably
|
||||||
|
within a frame budget if throttled to ~30 fps.
|
||||||
|
|
||||||
|
**✅ Real loupe shipped (2026-06-25).** `MagnifierUiState` + `Magnifier`
|
||||||
|
(`EmbeddedMagnifier.kt`); the caret/selection handles call an `OnMagnify` callback
|
||||||
|
on drag start/move/end; `EmbeddedTabLayer` tracks the drag point, throttles capture
|
||||||
|
requests to one in flight (100 ms timeout), decodes each `MagnifierFrame` to an
|
||||||
|
`ImageBitmap`, and floats the bubble above the finger (clamped, flips below near the
|
||||||
|
top). Capture mirrored onto BOTH embed paths (browser:
|
||||||
|
`NappletBrowserContract`/`NappletBrowserService`/`EmbeddedBrowserController`;
|
||||||
|
napplet: `NappletEmbedContract`/`NappletHostService`/`EmbeddedNappletController`).
|
||||||
|
Verified on device (browser): drag → bubble shows live magnified "ello world" with
|
||||||
|
the caret, centered on the line → follows finger → hides on release. The dead
|
||||||
|
Option-A probe (`SurfaceMagnifierProbe`) was removed. **Polish done (2026-06-25):**
|
||||||
|
capture Y is locked to the authoritative caret/selection line (the handles pass a
|
||||||
|
`lineHalfPx` so the box centers on the line, not the finger or the caret foot); X
|
||||||
|
still follows the finger. Remaining nice-to-haves: RGB_565/raw to cut PNG encode,
|
||||||
|
themed crosshair, clamp capture X to the line so a fast drag past EOL isn't blank,
|
||||||
|
reuse one off-screen bitmap.
|
||||||
|
- **Gesture-routing caveat (found during the spike).** The host-drawn handles'
|
||||||
|
drag is fragile: the sandbox `ContentView.onTouchEvent` always returns `true`, so
|
||||||
|
via Compose's `AndroidView` interop it consumes the drag-move pointer and cancels
|
||||||
|
the overlay handle's `detectDragGestures` (synthetic `adb` drags on the handle
|
||||||
|
never produced an `onDrag`). The magnifier trigger should ride the existing
|
||||||
|
caret/selection-move path (which already round-trips through the shim), not a fresh
|
||||||
|
Compose drag layered over the surface.
|
||||||
2. **Toolbar state rules (#3 hide-during-drag/scroll) + insertion-handle popup
|
2. **Toolbar state rules (#3 hide-during-drag/scroll) + insertion-handle popup
|
||||||
(#12).** Pure Compose/state work, no platform unknowns. Do alongside the
|
(#12).** Pure Compose/state work, no platform unknowns. Do alongside the
|
||||||
`SelectionUiState` refactor.
|
`SelectionUiState` refactor.
|
||||||
|
|||||||
+9
-3
@@ -44,6 +44,7 @@ import com.vitorpamplona.amethyst.ui.navigation.drawer.DrawerContent
|
|||||||
import com.vitorpamplona.amethyst.ui.navigation.navs.Nav
|
import com.vitorpamplona.amethyst.ui.navigation.navs.Nav
|
||||||
import com.vitorpamplona.amethyst.ui.navigation.routes.Route
|
import com.vitorpamplona.amethyst.ui.navigation.routes.Route
|
||||||
import com.vitorpamplona.amethyst.ui.screen.AccountSessionManager
|
import com.vitorpamplona.amethyst.ui.screen.AccountSessionManager
|
||||||
|
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedSelectionDrag
|
||||||
import kotlinx.coroutines.launch
|
import kotlinx.coroutines.launch
|
||||||
|
|
||||||
@OptIn(ExperimentalMaterial3Api::class)
|
@OptIn(ExperimentalMaterial3Api::class)
|
||||||
@@ -90,9 +91,14 @@ fun AccountSwitcherAndLeftDrawerLayout(
|
|||||||
dest.hasRoute<Route.Home>() || dest.hasRoute<Route.Message>()
|
dest.hasRoute<Route.Home>() || dest.hasRoute<Route.Message>()
|
||||||
} ?: false
|
} ?: false
|
||||||
val drawerGesturesEnabled =
|
val drawerGesturesEnabled =
|
||||||
!isTabPagerRoute ||
|
(
|
||||||
nav.drawerState.isOpen ||
|
!isTabPagerRoute ||
|
||||||
nav.drawerState.targetValue != nav.drawerState.currentValue
|
nav.drawerState.isOpen ||
|
||||||
|
nav.drawerState.targetValue != nav.drawerState.currentValue
|
||||||
|
) &&
|
||||||
|
// Suspend the left-edge swipe while a selection/caret handle is dragged over an embedded surface,
|
||||||
|
// so a handle drag near the left edge (or the auto-scroll edge drag) doesn't open the drawer.
|
||||||
|
!EmbeddedSelectionDrag.dragging
|
||||||
|
|
||||||
ModalNavigationDrawer(
|
ModalNavigationDrawer(
|
||||||
drawerState = nav.drawerState,
|
drawerState = nav.drawerState,
|
||||||
|
|||||||
+39
@@ -31,6 +31,7 @@ import android.os.IBinder
|
|||||||
import android.os.Looper
|
import android.os.Looper
|
||||||
import android.os.Message
|
import android.os.Message
|
||||||
import android.os.Messenger
|
import android.os.Messenger
|
||||||
|
import android.os.SystemClock
|
||||||
import androidx.annotation.RequiresApi
|
import androidx.annotation.RequiresApi
|
||||||
import androidx.compose.runtime.mutableStateListOf
|
import androidx.compose.runtime.mutableStateListOf
|
||||||
import androidx.privacysandbox.ui.client.SandboxedUiAdapterFactory
|
import androidx.privacysandbox.ui.client.SandboxedUiAdapterFactory
|
||||||
@@ -41,8 +42,10 @@ import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.ConsoleBridge
|
|||||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.ConsoleLogEntry
|
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.ConsoleLogEntry
|
||||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedImeBridge
|
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedImeBridge
|
||||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedLoadStatus
|
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedLoadStatus
|
||||||
|
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedMagnifierProbe
|
||||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedSurfaceController
|
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedSurfaceController
|
||||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.ImeEvent
|
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.ImeEvent
|
||||||
|
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.MagnifierFrame
|
||||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.parseSelectionGeometry
|
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.parseSelectionGeometry
|
||||||
import org.json.JSONObject
|
import org.json.JSONObject
|
||||||
import java.util.concurrent.atomic.AtomicLong
|
import java.util.concurrent.atomic.AtomicLong
|
||||||
@@ -62,6 +65,7 @@ class EmbeddedWebAppController(
|
|||||||
private val themeType: String = "SYSTEM",
|
private val themeType: String = "SYSTEM",
|
||||||
) : EmbeddedSurfaceController,
|
) : EmbeddedSurfaceController,
|
||||||
EmbeddedImeBridge,
|
EmbeddedImeBridge,
|
||||||
|
EmbeddedMagnifierProbe,
|
||||||
ConsoleBridge {
|
ConsoleBridge {
|
||||||
private val incoming = Messenger(Handler(Looper.getMainLooper(), ::onServiceMessage))
|
private val incoming = Messenger(Handler(Looper.getMainLooper(), ::onServiceMessage))
|
||||||
private var serviceMessenger: Messenger? = null
|
private var serviceMessenger: Messenger? = null
|
||||||
@@ -95,6 +99,9 @@ class EmbeddedWebAppController(
|
|||||||
|
|
||||||
override var onImeEvent: ((ImeEvent) -> Unit)? = null
|
override var onImeEvent: ((ImeEvent) -> Unit)? = null
|
||||||
|
|
||||||
|
// SPIKE (magnifier #4, option B): provider-side capture round-trip. Removed once the loupe lands.
|
||||||
|
override var onMagnifierFrame: ((MagnifierFrame) -> Unit)? = null
|
||||||
|
|
||||||
private val connection =
|
private val connection =
|
||||||
object : ServiceConnection {
|
object : ServiceConnection {
|
||||||
override fun onServiceConnected(
|
override fun onServiceConnected(
|
||||||
@@ -127,6 +134,7 @@ class EmbeddedWebAppController(
|
|||||||
pendingAdapter = null
|
pendingAdapter = null
|
||||||
onUrlChanged = null
|
onUrlChanged = null
|
||||||
onImeEvent = null
|
onImeEvent = null
|
||||||
|
onMagnifierFrame = null
|
||||||
onLoadStatusChanged = null
|
onLoadStatusChanged = null
|
||||||
consoleLogs.clear()
|
consoleLogs.clear()
|
||||||
}
|
}
|
||||||
@@ -193,6 +201,19 @@ class EmbeddedWebAppController(
|
|||||||
if (consoleLogs.size >= MAX_CONSOLE_LOGS) consoleLogs.removeAt(0)
|
if (consoleLogs.size >= MAX_CONSOLE_LOGS) consoleLogs.removeAt(0)
|
||||||
consoleLogs.add(ConsoleLogEntry(level, message, source, line))
|
consoleLogs.add(ConsoleLogEntry(level, message, source, line))
|
||||||
}
|
}
|
||||||
|
NappletBrowserContract.MSG_MAGNIFIER_FRAME -> {
|
||||||
|
val data = msg.data ?: return true
|
||||||
|
val bytes = data.getByteArray(NappletBrowserContract.KEY_MAG_BYTES) ?: return true
|
||||||
|
onMagnifierFrame?.invoke(
|
||||||
|
MagnifierFrame(
|
||||||
|
bytes = bytes,
|
||||||
|
width = data.getInt(NappletBrowserContract.KEY_MAG_W),
|
||||||
|
height = data.getInt(NappletBrowserContract.KEY_MAG_H),
|
||||||
|
captureMs = data.getDouble(NappletBrowserContract.KEY_MAG_CAPTURE_MS),
|
||||||
|
requestStampNanos = data.getLong(NappletBrowserContract.KEY_MAG_REQ_T),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
else -> return false
|
else -> return false
|
||||||
}
|
}
|
||||||
return true
|
return true
|
||||||
@@ -246,6 +267,22 @@ class EmbeddedWebAppController(
|
|||||||
|
|
||||||
override fun sendImeOp(json: String) = send(NappletBrowserContract.MSG_IME_OP) { putString(NappletBrowserContract.KEY_IME_PAYLOAD, json) }
|
override fun sendImeOp(json: String) = send(NappletBrowserContract.MSG_IME_OP) { putString(NappletBrowserContract.KEY_IME_PAYLOAD, json) }
|
||||||
|
|
||||||
|
// Stamp the client send time so the reply can be matched / stale frames dropped (same-process clock).
|
||||||
|
override fun requestMagnifier(
|
||||||
|
surfaceX: Float,
|
||||||
|
surfaceY: Float,
|
||||||
|
boxWidthPx: Int,
|
||||||
|
boxHeightPx: Int,
|
||||||
|
zoom: Float,
|
||||||
|
) = send(NappletBrowserContract.MSG_MAGNIFIER_REQUEST) {
|
||||||
|
putFloat(NappletBrowserContract.KEY_MAG_X, surfaceX)
|
||||||
|
putFloat(NappletBrowserContract.KEY_MAG_Y, surfaceY)
|
||||||
|
putInt(NappletBrowserContract.KEY_MAG_BOX_W, boxWidthPx)
|
||||||
|
putInt(NappletBrowserContract.KEY_MAG_BOX_H, boxHeightPx)
|
||||||
|
putFloat(NappletBrowserContract.KEY_MAG_ZOOM, zoom)
|
||||||
|
putLong(NappletBrowserContract.KEY_MAG_REQ_T, SystemClock.elapsedRealtimeNanos())
|
||||||
|
}
|
||||||
|
|
||||||
private fun parseImeEvent(payload: String): ImeEvent? {
|
private fun parseImeEvent(payload: String): ImeEvent? {
|
||||||
val o = runCatching { JSONObject(payload) }.getOrNull() ?: return null
|
val o = runCatching { JSONObject(payload) }.getOrNull() ?: return null
|
||||||
return when (o.optString("type")) {
|
return when (o.optString("type")) {
|
||||||
@@ -273,6 +310,8 @@ class EmbeddedWebAppController(
|
|||||||
text = o.optString("text", ""),
|
text = o.optString("text", ""),
|
||||||
geometry = parseSelectionGeometry(o.optJSONObject("geom")),
|
geometry = parseSelectionGeometry(o.optJSONObject("geom")),
|
||||||
)
|
)
|
||||||
|
"ime.scroll" -> ImeEvent.Scroll(active = o.optBoolean("active", false))
|
||||||
|
"ime.carettap" -> ImeEvent.CaretTap(geometry = parseSelectionGeometry(o.optJSONObject("geom")))
|
||||||
else -> null
|
else -> null
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+23
@@ -38,6 +38,7 @@ fun parseSelectionGeometry(o: JSONObject?): SelectionGeometry? {
|
|||||||
caretX = if (o.has("cx")) o.optDouble("cx").toFloat() else null,
|
caretX = if (o.has("cx")) o.optDouble("cx").toFloat() else null,
|
||||||
caretTop = if (o.has("ct")) o.optDouble("ct").toFloat() else null,
|
caretTop = if (o.has("ct")) o.optDouble("ct").toFloat() else null,
|
||||||
caretBottom = if (o.has("cb")) o.optDouble("cb").toFloat() else null,
|
caretBottom = if (o.has("cb")) o.optDouble("cb").toFloat() else null,
|
||||||
|
isRange = o.optBoolean("rng", false),
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -89,6 +90,24 @@ sealed interface ImeEvent {
|
|||||||
val text: String,
|
val text: String,
|
||||||
val geometry: SelectionGeometry?,
|
val geometry: SelectionGeometry?,
|
||||||
) : ImeEvent
|
) : ImeEvent
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The embedded page started ([active] true) or finished ([active] false) scrolling. Host-drawn selection
|
||||||
|
* UI hides while scrolling (its geometry is stale mid-scroll) and reappears, repositioned, on settle —
|
||||||
|
* the shim re-reports geometry just before the `active=false` so the overlays land in the right place.
|
||||||
|
*/
|
||||||
|
data class Scroll(
|
||||||
|
val active: Boolean,
|
||||||
|
) : ImeEvent
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The user tapped the focused field, placing a bare caret. Re-shows the insertion handle even when the
|
||||||
|
* tap didn't move the caret (so no [State] fired) — native shows the handle on every tap. [geometry]
|
||||||
|
* carries the caret rect. Gated host-side to non-empty fields, like `Editor`.
|
||||||
|
*/
|
||||||
|
data class CaretTap(
|
||||||
|
val geometry: SelectionGeometry?,
|
||||||
|
) : ImeEvent
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -111,4 +130,8 @@ data class SelectionGeometry(
|
|||||||
val caretX: Float? = null,
|
val caretX: Float? = null,
|
||||||
val caretTop: Float? = null,
|
val caretTop: Float? = null,
|
||||||
val caretBottom: Float? = null,
|
val caretBottom: Float? = null,
|
||||||
|
// True when this is an in-field *range* selection whose [startX]/[startBottom]/[endX]/[endBottom] are the
|
||||||
|
// real selection-endpoint feet (not the field-box placeholders). Lets the host hold the range geometry
|
||||||
|
// through Chrome's transient collapse-to-caret reports during the re-assert fight.
|
||||||
|
val isRange: Boolean = false,
|
||||||
)
|
)
|
||||||
|
|||||||
+114
@@ -0,0 +1,114 @@
|
|||||||
|
/*
|
||||||
|
* Copyright (c) 2025 Vitor Pamplona
|
||||||
|
*
|
||||||
|
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||||
|
* this software and associated documentation files (the "Software"), to deal in
|
||||||
|
* the Software without restriction, including without limitation the rights to use,
|
||||||
|
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||||
|
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||||
|
* subject to the following conditions:
|
||||||
|
*
|
||||||
|
* The above copyright notice and this permission notice shall be included in all
|
||||||
|
* copies or substantial portions of the Software.
|
||||||
|
*
|
||||||
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||||
|
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||||
|
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||||
|
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||||
|
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||||
|
*/
|
||||||
|
package com.vitorpamplona.amethyst.ui.screen.loggedIn.embed
|
||||||
|
|
||||||
|
import androidx.compose.foundation.Image
|
||||||
|
import androidx.compose.foundation.background
|
||||||
|
import androidx.compose.foundation.border
|
||||||
|
import androidx.compose.foundation.layout.Box
|
||||||
|
import androidx.compose.foundation.layout.absoluteOffset
|
||||||
|
import androidx.compose.foundation.layout.fillMaxSize
|
||||||
|
import androidx.compose.foundation.layout.size
|
||||||
|
import androidx.compose.foundation.shape.RoundedCornerShape
|
||||||
|
import androidx.compose.material3.MaterialTheme
|
||||||
|
import androidx.compose.runtime.Composable
|
||||||
|
import androidx.compose.runtime.getValue
|
||||||
|
import androidx.compose.runtime.mutableStateOf
|
||||||
|
import androidx.compose.runtime.setValue
|
||||||
|
import androidx.compose.ui.Modifier
|
||||||
|
import androidx.compose.ui.draw.clip
|
||||||
|
import androidx.compose.ui.draw.shadow
|
||||||
|
import androidx.compose.ui.geometry.Offset
|
||||||
|
import androidx.compose.ui.graphics.ImageBitmap
|
||||||
|
import androidx.compose.ui.layout.ContentScale
|
||||||
|
import androidx.compose.ui.platform.LocalDensity
|
||||||
|
import androidx.compose.ui.unit.DpSize
|
||||||
|
import androidx.compose.ui.unit.IntOffset
|
||||||
|
import androidx.compose.ui.unit.IntSize
|
||||||
|
import androidx.compose.ui.unit.dp
|
||||||
|
import kotlin.math.roundToInt
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Hoisted state for the embedded-surface selection loupe (magnifier #4). The bubble shows a magnified,
|
||||||
|
* live slice of the page captured in the `:napplet` provider (host-side `PixelCopy` can't read the sandbox
|
||||||
|
* surface — see [EmbeddedMagnifierProbe]). While a caret/selection handle is dragged, [anchorPx] tracks the
|
||||||
|
* drag point in this layer's px (so the bubble follows smoothly even between captures) and [image] is the
|
||||||
|
* latest decoded frame. Throttle bookkeeping ([lastRequestUptimeMs]/[awaitingFrame]) lives here too but is
|
||||||
|
* intentionally NOT Compose state — it must not trigger recomposition.
|
||||||
|
*/
|
||||||
|
class MagnifierUiState {
|
||||||
|
var visible by mutableStateOf(false)
|
||||||
|
var anchorPx by mutableStateOf(Offset.Zero)
|
||||||
|
var image by mutableStateOf<ImageBitmap?>(null)
|
||||||
|
|
||||||
|
var lastRequestUptimeMs: Long = 0L
|
||||||
|
var awaitingFrame: Boolean = false
|
||||||
|
|
||||||
|
fun hide() {
|
||||||
|
visible = false
|
||||||
|
image = null
|
||||||
|
awaitingFrame = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The loupe bubble: a rounded, elevated rectangle showing [MagnifierUiState.image], floated above the drag
|
||||||
|
* point and centered on it horizontally, clamped inside [layerSize]. Like Android's `Magnifier`, it sits a
|
||||||
|
* little above the finger; if there's no room above (drag near the top), it flips below. Nothing is drawn
|
||||||
|
* until there's a frame to show.
|
||||||
|
*/
|
||||||
|
@Composable
|
||||||
|
fun Magnifier(
|
||||||
|
state: MagnifierUiState,
|
||||||
|
layerSize: IntSize,
|
||||||
|
bubbleSize: DpSize,
|
||||||
|
) {
|
||||||
|
if (!state.visible) return
|
||||||
|
val img = state.image ?: return
|
||||||
|
val density = LocalDensity.current
|
||||||
|
val wPx = with(density) { bubbleSize.width.toPx() }
|
||||||
|
val hPx = with(density) { bubbleSize.height.toPx() }
|
||||||
|
val gapPx = with(density) { 28.dp.toPx() }
|
||||||
|
|
||||||
|
val left = (state.anchorPx.x - wPx / 2f).coerceIn(0f, (layerSize.width - wPx).coerceAtLeast(0f))
|
||||||
|
val above = state.anchorPx.y - hPx - gapPx
|
||||||
|
val top = if (above >= 0f) above else state.anchorPx.y + gapPx
|
||||||
|
|
||||||
|
val shape = RoundedCornerShape(12.dp)
|
||||||
|
Box(
|
||||||
|
Modifier
|
||||||
|
.absoluteOffset { IntOffset(left.roundToInt(), top.roundToInt()) }
|
||||||
|
.size(bubbleSize)
|
||||||
|
.shadow(8.dp, shape)
|
||||||
|
.clip(shape)
|
||||||
|
.background(MaterialTheme.colorScheme.surface)
|
||||||
|
.border(1.dp, MaterialTheme.colorScheme.outlineVariant, shape),
|
||||||
|
) {
|
||||||
|
// The captured frame is built so its px size ≈ the bubble's px size (source rect = bubble / zoom,
|
||||||
|
// provider scaled by zoom), so FillBounds maps it 1:1 without distortion.
|
||||||
|
Image(
|
||||||
|
bitmap = img,
|
||||||
|
contentDescription = null,
|
||||||
|
contentScale = ContentScale.FillBounds,
|
||||||
|
modifier = Modifier.fillMaxSize(),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
+65
@@ -0,0 +1,65 @@
|
|||||||
|
/*
|
||||||
|
* Copyright (c) 2025 Vitor Pamplona
|
||||||
|
*
|
||||||
|
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||||
|
* this software and associated documentation files (the "Software"), to deal in
|
||||||
|
* the Software without restriction, including without limitation the rights to use,
|
||||||
|
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||||
|
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||||
|
* subject to the following conditions:
|
||||||
|
*
|
||||||
|
* The above copyright notice and this permission notice shall be included in all
|
||||||
|
* copies or substantial portions of the Software.
|
||||||
|
*
|
||||||
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||||
|
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||||
|
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||||
|
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||||
|
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||||
|
*/
|
||||||
|
package com.vitorpamplona.amethyst.ui.screen.loggedIn.embed
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Magnifier (#4) capture path — see `amethyst/plans/2026-06-25-embed-text-selection-native-parity.md`.
|
||||||
|
*
|
||||||
|
* Host-side `PixelCopy` of the sandboxed surface returns `ERROR_SOURCE_NO_DATA` (the WebView pixels live in a
|
||||||
|
* child SurfaceControl the host never draws into), so the selection loupe's content must be captured INSIDE
|
||||||
|
* the keyless `:napplet` provider — where the WebView is a real in-window view with real pixels — and shipped
|
||||||
|
* back over the Messenger channel. A controller whose surface can produce such captures implements this;
|
||||||
|
* [EmbeddedTabLayer] discovers it by `as?` cast, mirroring how it treats [EmbeddedImeBridge].
|
||||||
|
*/
|
||||||
|
interface EmbeddedMagnifierProbe {
|
||||||
|
/** A captured loupe frame arrived from the provider. */
|
||||||
|
var onMagnifierFrame: ((MagnifierFrame) -> Unit)?
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Ask the provider for a zoomed slice of the live page centered on ([surfaceX], [surfaceY]) in surface px
|
||||||
|
* (which equal the remote WebView's view px — the SCVH surface is 1:1 with the SandboxedSdkView).
|
||||||
|
* [boxWidthPx]×[boxHeightPx] is the source rectangle; the returned image is that, scaled by [zoom].
|
||||||
|
*/
|
||||||
|
fun requestMagnifier(
|
||||||
|
surfaceX: Float,
|
||||||
|
surfaceY: Float,
|
||||||
|
boxWidthPx: Int,
|
||||||
|
boxHeightPx: Int,
|
||||||
|
zoom: Float,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A loupe frame returned by the provider: a PNG ([bytes]) of [width]×[height], plus timing — [captureMs] is
|
||||||
|
* the provider-side draw+encode cost and [requestStampNanos] echoes the client's send stamp so out-of-order
|
||||||
|
* frames can be dropped.
|
||||||
|
*/
|
||||||
|
data class MagnifierFrame(
|
||||||
|
val bytes: ByteArray,
|
||||||
|
val width: Int,
|
||||||
|
val height: Int,
|
||||||
|
val captureMs: Double,
|
||||||
|
val requestStampNanos: Long,
|
||||||
|
) {
|
||||||
|
override fun equals(other: Any?) = this === other
|
||||||
|
|
||||||
|
override fun hashCode() = System.identityHashCode(this)
|
||||||
|
}
|
||||||
+37
@@ -0,0 +1,37 @@
|
|||||||
|
/*
|
||||||
|
* Copyright (c) 2025 Vitor Pamplona
|
||||||
|
*
|
||||||
|
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||||
|
* this software and associated documentation files (the "Software"), to deal in
|
||||||
|
* the Software without restriction, including without limitation the rights to use,
|
||||||
|
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||||
|
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||||
|
* subject to the following conditions:
|
||||||
|
*
|
||||||
|
* The above copyright notice and this permission notice shall be included in all
|
||||||
|
* copies or substantial portions of the Software.
|
||||||
|
*
|
||||||
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||||
|
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||||
|
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||||
|
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||||
|
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||||
|
*/
|
||||||
|
package com.vitorpamplona.amethyst.ui.screen.loggedIn.embed
|
||||||
|
|
||||||
|
import androidx.compose.runtime.getValue
|
||||||
|
import androidx.compose.runtime.mutableStateOf
|
||||||
|
import androidx.compose.runtime.setValue
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Process-level flag: true while the user is dragging a selection/caret handle over an embedded surface.
|
||||||
|
*
|
||||||
|
* The left navigation drawer's edge-swipe-to-open competes with a handle drag near the left edge (and with
|
||||||
|
* the [auto-scroll][EmbeddedTabLayer] edge drag), opening the drawer mid-drag. The drawer reads this to
|
||||||
|
* suspend its gesture while a handle is held. Kept out of the API-gated [EmbeddedTabHost] so non-R UI (the
|
||||||
|
* drawer) can read it without a version guard; [EmbeddedTabLayer] is the only writer.
|
||||||
|
*/
|
||||||
|
object EmbeddedSelectionDrag {
|
||||||
|
var dragging by mutableStateOf(false)
|
||||||
|
}
|
||||||
+401
-193
@@ -23,15 +23,19 @@ package com.vitorpamplona.amethyst.ui.screen.loggedIn.embed
|
|||||||
import android.content.ClipData
|
import android.content.ClipData
|
||||||
import android.content.ClipboardManager
|
import android.content.ClipboardManager
|
||||||
import android.content.Context
|
import android.content.Context
|
||||||
|
import android.graphics.BitmapFactory
|
||||||
import android.os.Build
|
import android.os.Build
|
||||||
|
import android.os.SystemClock
|
||||||
import android.view.ViewGroup
|
import android.view.ViewGroup
|
||||||
import androidx.annotation.RequiresApi
|
import androidx.annotation.RequiresApi
|
||||||
import androidx.compose.foundation.BorderStroke
|
import androidx.compose.foundation.BorderStroke
|
||||||
import androidx.compose.foundation.Canvas
|
import androidx.compose.foundation.Canvas
|
||||||
import androidx.compose.foundation.background
|
import androidx.compose.foundation.background
|
||||||
import androidx.compose.foundation.clickable
|
import androidx.compose.foundation.clickable
|
||||||
|
import androidx.compose.foundation.gestures.awaitEachGesture
|
||||||
|
import androidx.compose.foundation.gestures.awaitFirstDown
|
||||||
import androidx.compose.foundation.gestures.detectDragGestures
|
import androidx.compose.foundation.gestures.detectDragGestures
|
||||||
import androidx.compose.foundation.gestures.detectTapGestures
|
import androidx.compose.foundation.gestures.waitForUpOrCancellation
|
||||||
import androidx.compose.foundation.interaction.MutableInteractionSource
|
import androidx.compose.foundation.interaction.MutableInteractionSource
|
||||||
import androidx.compose.foundation.layout.Box
|
import androidx.compose.foundation.layout.Box
|
||||||
import androidx.compose.foundation.layout.ExperimentalLayoutApi
|
import androidx.compose.foundation.layout.ExperimentalLayoutApi
|
||||||
@@ -62,11 +66,15 @@ import androidx.compose.ui.Modifier
|
|||||||
import androidx.compose.ui.geometry.Offset
|
import androidx.compose.ui.geometry.Offset
|
||||||
import androidx.compose.ui.geometry.Rect
|
import androidx.compose.ui.geometry.Rect
|
||||||
import androidx.compose.ui.graphics.Path
|
import androidx.compose.ui.graphics.Path
|
||||||
|
import androidx.compose.ui.graphics.asImageBitmap
|
||||||
import androidx.compose.ui.input.pointer.pointerInput
|
import androidx.compose.ui.input.pointer.pointerInput
|
||||||
|
import androidx.compose.ui.input.pointer.positionChangeIgnoreConsumed
|
||||||
|
import androidx.compose.ui.layout.Layout
|
||||||
import androidx.compose.ui.layout.onGloballyPositioned
|
import androidx.compose.ui.layout.onGloballyPositioned
|
||||||
import androidx.compose.ui.layout.positionInWindow
|
import androidx.compose.ui.layout.positionInWindow
|
||||||
import androidx.compose.ui.platform.LocalContext
|
import androidx.compose.ui.platform.LocalContext
|
||||||
import androidx.compose.ui.platform.LocalDensity
|
import androidx.compose.ui.platform.LocalDensity
|
||||||
|
import androidx.compose.ui.unit.DpSize
|
||||||
import androidx.compose.ui.unit.IntOffset
|
import androidx.compose.ui.unit.IntOffset
|
||||||
import androidx.compose.ui.unit.IntSize
|
import androidx.compose.ui.unit.IntSize
|
||||||
import androidx.compose.ui.unit.dp
|
import androidx.compose.ui.unit.dp
|
||||||
@@ -79,6 +87,31 @@ import kotlin.math.roundToInt
|
|||||||
// How far off-screen a parked (inactive) warm tab is shifted — well past any real screen width.
|
// How far off-screen a parked (inactive) warm tab is shifted — well past any real screen width.
|
||||||
private val OFFSCREEN_SHIFT = 10_000.dp
|
private val OFFSCREEN_SHIFT = 10_000.dp
|
||||||
|
|
||||||
|
// Per-drag-move auto-scroll step (CSS px) when a handle is dragged into the surface's top/bottom edge zone.
|
||||||
|
private const val AUTOSCROLL_STEP_CSS = 40.0
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Drives the selection loupe from a handle drag: [active] shows/hides the bubble, [fingerLayerPx] is the
|
||||||
|
* drag point in the tab layer's px (where the bubble floats), and ([surfaceX], [surfaceY]) is the same point
|
||||||
|
* in surface px (what the provider captures around). Called on drag start/move with `true`, on end with `false`.
|
||||||
|
*/
|
||||||
|
private typealias OnMagnify = (active: Boolean, fingerLayerPx: Offset, surfaceX: Float, surfaceY: Float) -> Unit
|
||||||
|
|
||||||
|
/** Sends a CSS-px positional IME op (caret move / selection extend) to the page; [edge] is null for a caret move. */
|
||||||
|
private fun EmbeddedImeBridge.sendFieldOp(
|
||||||
|
type: String,
|
||||||
|
edge: String?,
|
||||||
|
cssX: Float,
|
||||||
|
cssY: Float,
|
||||||
|
) = sendImeOp(
|
||||||
|
JSONObject()
|
||||||
|
.put("type", type)
|
||||||
|
.apply { if (edge != null) put("edge", edge) }
|
||||||
|
.put("x", cssX.toDouble())
|
||||||
|
.put("y", cssY.toDouble())
|
||||||
|
.toString(),
|
||||||
|
)
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The persistent surface layer: a full-window overlay (mounted once in the app shell, below the
|
* The persistent surface layer: a full-window overlay (mounted once in the app shell, below the
|
||||||
* navigation drawer and dialogs) that renders **every** warm embedded session's [SandboxedSdkView] and
|
* navigation drawer and dialogs) that renders **every** warm embedded session's [SandboxedSdkView] and
|
||||||
@@ -295,47 +328,45 @@ fun EmbeddedTabLayer(barFavoriteIds: List<String>) {
|
|||||||
val context = LocalContext.current
|
val context = LocalContext.current
|
||||||
val imeBridge = EmbeddedTabHost.sessions.firstOrNull { it.id == activeId }?.controller as? EmbeddedImeBridge
|
val imeBridge = EmbeddedTabHost.sessions.firstOrNull { it.id == activeId }?.controller as? EmbeddedImeBridge
|
||||||
val imeView = remember { RemoteImeView(context) }
|
val imeView = remember { RemoteImeView(context) }
|
||||||
// The embedded WebView can't show Chrome's copy/paste toolbar in its cross-process surface, so when
|
// The embedded WebView can't show Chrome's copy/paste toolbar or selection handles in its cross-process
|
||||||
// the page has a non-empty selection we show our own here, over the page, and route its actions to
|
// surface, so we draw our own over the page and route actions through the hidden EditText (which mirrors
|
||||||
// the hidden EditText (which mirrors the selection) — see [RemoteImeView.onRangeSelectionChanged].
|
// the selection). All the show/hide state lives in one [SelectionUiState] so the rules — toolbar hides
|
||||||
var showSelectionToolbar by remember { mutableStateOf(false) }
|
// while dragging or scrolling, handles hide while scrolling — are expressed in one place.
|
||||||
var fieldGeometry by remember { mutableStateOf<SelectionGeometry?>(null) }
|
val sel = remember { SelectionUiState() }
|
||||||
// Insertion (cursor) handle visibility: shown when a tap places/moves a bare caret, hidden while
|
|
||||||
// typing — like Android. A caret-bearing geometry update means a tap; an edit means typing.
|
|
||||||
var showInsertionHandle by remember { mutableStateOf(false) }
|
|
||||||
var pageSelection by remember { mutableStateOf<ImeEvent.PageSelection?>(null) }
|
|
||||||
DisposableEffect(imeBridge) {
|
DisposableEffect(imeBridge) {
|
||||||
imeView.bind(imeBridge)
|
imeView.bind(imeBridge)
|
||||||
imeView.onRangeSelectionChanged = { showSelectionToolbar = it }
|
imeView.onRangeSelectionChanged = { sel.onFieldRangeToggle(it) }
|
||||||
imeView.onEdited = { showInsertionHandle = false }
|
imeView.onEdited = { sel.onEdited() }
|
||||||
imeBridge?.onImeEvent = { event ->
|
imeBridge?.onImeEvent = { event ->
|
||||||
when (event) {
|
when (event) {
|
||||||
is ImeEvent.Focus -> {
|
is ImeEvent.Focus -> {
|
||||||
imeView.onPageFocus(event)
|
imeView.onPageFocus(event)
|
||||||
event.geometry?.let { fieldGeometry = it }
|
// A field took focus → the browser dropped any page-text selection; clear its overlay so
|
||||||
if (event.geometry?.caretX != null) showInsertionHandle = true
|
// the stale page handles/Copy bar don't sit above (and steal drags from) the field UI.
|
||||||
|
sel.onPageSelection(null)
|
||||||
|
// Cancel any in-flight scroll-hide from the page phase: otherwise the field's own
|
||||||
|
// selection-reveal scrolls keep it armed and the new field handles/toolbar never appear.
|
||||||
|
sel.scrolling = false
|
||||||
|
sel.onFieldGeometry(event.geometry, event.text.isNotEmpty())
|
||||||
}
|
}
|
||||||
ImeEvent.Blur -> {
|
ImeEvent.Blur -> {
|
||||||
imeView.onPageBlur()
|
imeView.onPageBlur()
|
||||||
fieldGeometry = null
|
sel.onBlur()
|
||||||
showInsertionHandle = false
|
|
||||||
}
|
}
|
||||||
is ImeEvent.State -> {
|
is ImeEvent.State -> {
|
||||||
imeView.onPageState(event)
|
imeView.onPageState(event)
|
||||||
event.geometry?.let { fieldGeometry = it }
|
sel.onFieldGeometry(event.geometry, event.text.isNotEmpty())
|
||||||
if (event.geometry?.caretX != null) showInsertionHandle = true
|
|
||||||
}
|
}
|
||||||
is ImeEvent.PageSelection -> pageSelection = event.takeIf { it.active }
|
is ImeEvent.PageSelection -> sel.onPageSelection(event.takeIf { it.active })
|
||||||
|
is ImeEvent.Scroll -> sel.scrolling = event.active
|
||||||
|
is ImeEvent.CaretTap -> sel.onCaretTap(event.geometry)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
onDispose {
|
onDispose {
|
||||||
imeBridge?.onImeEvent = null
|
imeBridge?.onImeEvent = null
|
||||||
imeView.onRangeSelectionChanged = null
|
imeView.onRangeSelectionChanged = null
|
||||||
imeView.onEdited = null
|
imeView.onEdited = null
|
||||||
showSelectionToolbar = false
|
sel.reset()
|
||||||
fieldGeometry = null
|
|
||||||
showInsertionHandle = false
|
|
||||||
pageSelection = null
|
|
||||||
imeView.onPageBlur()
|
imeView.onPageBlur()
|
||||||
imeView.bind(null)
|
imeView.bind(null)
|
||||||
}
|
}
|
||||||
@@ -352,118 +383,209 @@ fun EmbeddedTabLayer(barFavoriteIds: List<String>) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Input-field selection: cut/copy/paste/select-all routed to the hidden EditText. The DOM exposes no
|
// Native-style auto-hide: the insertion handle disappears after a few seconds of inactivity; the next
|
||||||
// rect for a selection *inside* an input, so we anchor the bar above the field box (reported by the
|
// caret tap re-emits geometry and re-shows it. A drag keeps it alive (dragging gate) so it never
|
||||||
// shim), falling back to the top of the tab if the field geometry is missing.
|
// vanishes mid-drag; the key re-arms whenever the caret moves.
|
||||||
if (showSelectionToolbar && bounds.width > 0f && bounds.height > 0f) {
|
val caretForTimeout = sel.insertionHandle
|
||||||
val fg = fieldGeometry
|
LaunchedEffect(caretForTimeout, sel.dragging) {
|
||||||
val items =
|
if (caretForTimeout != null && !sel.dragging) {
|
||||||
listOf(
|
delay(4_000)
|
||||||
"Cut" to {
|
sel.hideCaret()
|
||||||
imeView.cutSelection()
|
|
||||||
Unit
|
|
||||||
},
|
|
||||||
"Copy" to {
|
|
||||||
imeView.copySelection()
|
|
||||||
Unit
|
|
||||||
},
|
|
||||||
"Paste" to {
|
|
||||||
imeView.pasteClipboard()
|
|
||||||
Unit
|
|
||||||
},
|
|
||||||
"Select all" to {
|
|
||||||
imeView.selectAllText()
|
|
||||||
Unit
|
|
||||||
},
|
|
||||||
)
|
|
||||||
val originX = bounds.left - layerOrigin.x
|
|
||||||
val originY = bounds.top - layerOrigin.y
|
|
||||||
val toolbarH = with(density) { 44.dp.toPx() }
|
|
||||||
val gap = with(density) { 8.dp.toPx() }
|
|
||||||
if (fg != null && fg.viewportWidth > 0f) {
|
|
||||||
val scale = bounds.width / fg.viewportWidth
|
|
||||||
val topPx = originY + fg.top * scale
|
|
||||||
val toolbarY = if (topPx - toolbarH - gap > originY) topPx - toolbarH - gap else originY + fg.bottom * scale + gap
|
|
||||||
EmbeddedSelectionToolbar(
|
|
||||||
items = items,
|
|
||||||
centerXpx = originX + ((fg.left + fg.right) / 2f) * scale,
|
|
||||||
topYpx = toolbarY,
|
|
||||||
)
|
|
||||||
} else {
|
|
||||||
EmbeddedSelectionToolbar(
|
|
||||||
items = items,
|
|
||||||
centerXpx = originX + bounds.width / 2f,
|
|
||||||
topYpx = originY + gap,
|
|
||||||
)
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Selection loupe (magnifier #4): while a caret/selection handle is dragged, show a magnified live
|
||||||
|
// slice of the page. Host-side PixelCopy can't read the sandbox surface (ERROR_SOURCE_NO_DATA), so the
|
||||||
|
// pixels are captured in the `:napplet` provider and shipped back here (see [EmbeddedMagnifierProbe]).
|
||||||
|
// The handles report the drag point via [onMagnify]; we track it for the bubble position and ask the
|
||||||
|
// provider for frames, throttled to one in flight (with a 100 ms timeout) so a fast drag can't flood
|
||||||
|
// the IPC channel. The bubble follows the finger every move; the bitmap refreshes as frames land.
|
||||||
|
val magProbe = imeBridge as? EmbeddedMagnifierProbe
|
||||||
|
val magnifier = remember { MagnifierUiState() }
|
||||||
|
val magBubble = DpSize(132.dp, 74.dp)
|
||||||
|
val magZoom = 1.5f
|
||||||
|
// Source rect (surface px) = bubble px / zoom, so the provider-scaled frame lands ≈ bubble-sized.
|
||||||
|
val magSrcW = with(density) { (magBubble.width.toPx() / magZoom).roundToInt() }
|
||||||
|
val magSrcH = with(density) { (magBubble.height.toPx() / magZoom).roundToInt() }
|
||||||
|
DisposableEffect(magProbe) {
|
||||||
|
magProbe?.onMagnifierFrame = { frame ->
|
||||||
|
if (magnifier.visible) {
|
||||||
|
magnifier.awaitingFrame = false
|
||||||
|
BitmapFactory.decodeByteArray(frame.bytes, 0, frame.bytes.size)?.let { magnifier.image = it.asImageBitmap() }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
onDispose {
|
||||||
|
magProbe?.onMagnifierFrame = null
|
||||||
|
magnifier.hide()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Auto-scroll (#9): while dragging a handle near the surface's top/bottom edge, nudge the embedded
|
||||||
|
// content so the selection can keep extending past the viewport — like Android. We scroll on each
|
||||||
|
// drag-move that's in the edge zone (the finger is usually still micro-moving); the shim keeps the
|
||||||
|
// overlays up during this programmatic scroll and re-reports geometry so they track.
|
||||||
|
val edgeZonePx = with(density) { 56.dp.toPx() }
|
||||||
|
val onMagnify: OnMagnify = { active, fingerPx, surfaceX, surfaceY ->
|
||||||
|
// Drives the loupe, the toolbar-hide-while-dragging rule (and the dragged handle hides itself
|
||||||
|
// locally), edge auto-scroll, and suspending the nav drawer's edge swipe — one drag lifecycle.
|
||||||
|
sel.dragging = active
|
||||||
|
EmbeddedSelectionDrag.dragging = active
|
||||||
|
if (active && bounds.height > 0f) {
|
||||||
|
val oy = bounds.top - layerOrigin.y
|
||||||
|
val dy =
|
||||||
|
when {
|
||||||
|
fingerPx.y < oy + edgeZonePx -> -AUTOSCROLL_STEP_CSS
|
||||||
|
fingerPx.y > oy + bounds.height - edgeZonePx -> AUTOSCROLL_STEP_CSS
|
||||||
|
else -> 0.0
|
||||||
|
}
|
||||||
|
if (dy != 0.0) imeBridge?.sendImeOp(JSONObject().put("type", "ime.autoscroll").put("dy", dy).toString())
|
||||||
|
}
|
||||||
|
if (!active || magProbe == null) {
|
||||||
|
magnifier.hide()
|
||||||
|
} else {
|
||||||
|
magnifier.visible = true
|
||||||
|
magnifier.anchorPx = fingerPx
|
||||||
|
val nowMs = SystemClock.uptimeMillis()
|
||||||
|
if (!magnifier.awaitingFrame || nowMs - magnifier.lastRequestUptimeMs > 100L) {
|
||||||
|
magnifier.awaitingFrame = true
|
||||||
|
magnifier.lastRequestUptimeMs = nowMs
|
||||||
|
magProbe.requestMagnifier(surfaceX, surfaceY, magSrcW, magSrcH, magZoom)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
val originX = bounds.left - layerOrigin.x
|
||||||
|
val originY = bounds.top - layerOrigin.y
|
||||||
|
val haveBounds = bounds.width > 0f && bounds.height > 0f
|
||||||
|
|
||||||
|
// In-field (<input>/<textarea>) range selection: cut/copy/paste/select-all routed to the hidden
|
||||||
|
// EditText, plus draggable start/end handles (drag → `ime.fieldextend`). The toolbar hides while a
|
||||||
|
// handle is dragged or the page scrolls; the handles hide only while scrolling.
|
||||||
|
val fieldItems =
|
||||||
|
listOf(
|
||||||
|
"Cut" to {
|
||||||
|
imeView.cutSelection()
|
||||||
|
Unit
|
||||||
|
},
|
||||||
|
"Copy" to {
|
||||||
|
imeView.copySelection()
|
||||||
|
Unit
|
||||||
|
},
|
||||||
|
"Paste" to {
|
||||||
|
imeView.pasteClipboard()
|
||||||
|
Unit
|
||||||
|
},
|
||||||
|
"Select all" to {
|
||||||
|
imeView.selectAllText()
|
||||||
|
Unit
|
||||||
|
},
|
||||||
|
)
|
||||||
|
val fieldHandles = sel.fieldHandles
|
||||||
|
if (haveBounds && fieldHandles != null) {
|
||||||
|
RangeSelectionOverlay(
|
||||||
|
geometry = fieldHandles,
|
||||||
|
surfaceOriginX = originX,
|
||||||
|
surfaceOriginY = originY,
|
||||||
|
scale = bounds.width / fieldHandles.viewportWidth,
|
||||||
|
showToolbar = sel.fieldToolbar != null,
|
||||||
|
toolbarItems = fieldItems,
|
||||||
|
onMagnify = onMagnify,
|
||||||
|
onExtend = { edge, cssX, cssY -> imeBridge?.sendFieldOp("ime.fieldextend", edge, cssX, cssY) },
|
||||||
|
)
|
||||||
|
} else if (haveBounds && sel.fieldToolbar != null) {
|
||||||
|
// Authoritative range but no geometry yet → a centered fallback toolbar (no handles).
|
||||||
|
EmbeddedSelectionToolbar(items = fieldItems, centerXpx = originX + bounds.width / 2f, topYpx = originY + with(density) { 8.dp.toPx() })
|
||||||
|
}
|
||||||
|
|
||||||
// Bare caret in a field (no range): a draggable insertion handle under the cursor, like Android's.
|
// Bare caret in a field (no range): a draggable insertion handle under the cursor, like Android's.
|
||||||
val fgCaret = fieldGeometry
|
val caretGeom = sel.insertionHandle
|
||||||
if (showInsertionHandle && !showSelectionToolbar && fgCaret?.caretX != null && fgCaret.caretBottom != null &&
|
if (haveBounds && caretGeom?.caretX != null && caretGeom.caretBottom != null) {
|
||||||
fgCaret.viewportWidth > 0f && bounds.width > 0f && bounds.height > 0f
|
val scale = bounds.width / caretGeom.viewportWidth
|
||||||
) {
|
// Half the caret's line height (layer px), so the loupe capture centers on the text line.
|
||||||
val scale = bounds.width / fgCaret.viewportWidth
|
val caretLineHalf =
|
||||||
val originX = bounds.left - layerOrigin.x
|
(((caretGeom.caretTop?.let { caretGeom.caretBottom - it }) ?: 0f) * scale / 2f)
|
||||||
val originY = bounds.top - layerOrigin.y
|
.coerceIn(0f, with(density) { 28.dp.toPx() })
|
||||||
InsertionHandle(
|
InsertionHandle(
|
||||||
tipPx = Offset(originX + fgCaret.caretX * scale, originY + fgCaret.caretBottom * scale),
|
tipPx = Offset(originX + caretGeom.caretX * scale, originY + caretGeom.caretBottom * scale),
|
||||||
surfaceOriginX = originX,
|
surfaceOriginX = originX,
|
||||||
surfaceOriginY = originY,
|
surfaceOriginY = originY,
|
||||||
scale = scale,
|
scale = scale,
|
||||||
onDragTo = { cssX, cssY ->
|
lineHalfPx = caretLineHalf,
|
||||||
imeBridge?.sendImeOp(
|
onMagnify = onMagnify,
|
||||||
JSONObject()
|
onTap = { sel.toggleInsertionPopup() },
|
||||||
.put("type", "ime.caretmove")
|
onDragTo = { cssX, cssY -> imeBridge?.sendFieldOp("ime.caretmove", null, cssX, cssY) },
|
||||||
.put("x", cssX.toDouble())
|
|
||||||
.put("y", cssY.toDouble())
|
|
||||||
.toString(),
|
|
||||||
)
|
|
||||||
},
|
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Plain page-text selection: a Copy bar positioned over the selection + a drag handle at each end.
|
// Tapping the bare insertion handle opens a small Paste / Select-all popup above the caret (native).
|
||||||
val pageSel = pageSelection
|
val popupCaret = sel.insertionPopupAt
|
||||||
val geom = pageSel?.geometry
|
if (haveBounds && popupCaret?.caretX != null && popupCaret.caretTop != null) {
|
||||||
if (geom != null && geom.viewportWidth > 0f && bounds.width > 0f && bounds.height > 0f) {
|
val scale = bounds.width / popupCaret.viewportWidth
|
||||||
PageSelectionOverlay(
|
val gap = with(density) { 8.dp.toPx() }
|
||||||
geometry = geom,
|
val toolbarH = with(density) { 44.dp.toPx() }
|
||||||
surfaceOriginX = bounds.left - layerOrigin.x,
|
val caretTopPx = originY + popupCaret.caretTop * scale
|
||||||
surfaceOriginY = bounds.top - layerOrigin.y,
|
val caretBottomPx = originY + (popupCaret.caretBottom ?: popupCaret.caretTop) * scale
|
||||||
scale = bounds.width / geom.viewportWidth,
|
val topY = if (caretTopPx - toolbarH - gap > originY) caretTopPx - toolbarH - gap else caretBottomPx + gap
|
||||||
onCopy = {
|
EmbeddedSelectionToolbar(
|
||||||
val clipboard = context.getSystemService(Context.CLIPBOARD_SERVICE) as ClipboardManager
|
items =
|
||||||
clipboard.setPrimaryClip(ClipData.newPlainText("selection", pageSel.text))
|
listOf(
|
||||||
},
|
"Paste" to {
|
||||||
onExtend = { edge, cssX, cssY ->
|
imeView.pasteClipboard()
|
||||||
imeBridge?.sendImeOp(
|
sel.hideInsertionPopup()
|
||||||
JSONObject()
|
},
|
||||||
.put("type", "ime.pageextend")
|
"Select all" to {
|
||||||
.put("edge", edge)
|
imeView.selectAllText()
|
||||||
.put("x", cssX.toDouble())
|
sel.hideInsertionPopup()
|
||||||
.put("y", cssY.toDouble())
|
},
|
||||||
.toString(),
|
),
|
||||||
)
|
centerXpx = originX + popupCaret.caretX * scale,
|
||||||
},
|
topYpx = topY,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Plain page-text selection: a Copy bar over the selection + a drag handle at each end (same rules).
|
||||||
|
val pageHandles = sel.pageHandles
|
||||||
|
val pageSel = sel.pageSelection
|
||||||
|
if (haveBounds && pageHandles != null && pageSel != null) {
|
||||||
|
RangeSelectionOverlay(
|
||||||
|
geometry = pageHandles,
|
||||||
|
surfaceOriginX = originX,
|
||||||
|
surfaceOriginY = originY,
|
||||||
|
scale = bounds.width / pageHandles.viewportWidth,
|
||||||
|
showToolbar = sel.pageToolbar != null,
|
||||||
|
toolbarItems =
|
||||||
|
listOf(
|
||||||
|
"Copy" to {
|
||||||
|
val clipboard = context.getSystemService(Context.CLIPBOARD_SERVICE) as ClipboardManager
|
||||||
|
clipboard.setPrimaryClip(ClipData.newPlainText("selection", pageSel.text))
|
||||||
|
Unit
|
||||||
|
},
|
||||||
|
),
|
||||||
|
onMagnify = onMagnify,
|
||||||
|
onExtend = { edge, cssX, cssY -> imeBridge?.sendFieldOp("ime.pageextend", edge, cssX, cssY) },
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The loupe sits above everything else, following the active handle/caret drag.
|
||||||
|
Magnifier(magnifier, layerSize, magBubble)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Host-drawn selection overlay for a plain page-text selection: a Copy bar above the selection plus a
|
* Host-drawn selection overlay shared by in-field (`<input>`/`<textarea>`) and plain page-text selections:
|
||||||
* draggable handle at each end. [geometry] is in the page's CSS px; [scale] and the surface origin map those
|
* a floating toolbar above the selection (only when [showToolbar] — it's hidden mid-drag) plus a draggable
|
||||||
* to this layer's px. Dragging a handle reports the new CSS-px target via [onExtend], which the shim turns
|
* handle at each end. [geometry] is in the page's CSS px; [scale] and the surface origin map those to this
|
||||||
* into a selection extension; the resulting geometry update repositions everything.
|
* layer's px. Dragging a handle reports the new CSS-px target via [onExtend], which the shim turns into a
|
||||||
|
* selection extension; the resulting geometry update repositions everything.
|
||||||
*/
|
*/
|
||||||
@Composable
|
@Composable
|
||||||
private fun PageSelectionOverlay(
|
private fun RangeSelectionOverlay(
|
||||||
geometry: SelectionGeometry,
|
geometry: SelectionGeometry,
|
||||||
surfaceOriginX: Float,
|
surfaceOriginX: Float,
|
||||||
surfaceOriginY: Float,
|
surfaceOriginY: Float,
|
||||||
scale: Float,
|
scale: Float,
|
||||||
onCopy: () -> Unit,
|
showToolbar: Boolean,
|
||||||
|
toolbarItems: List<Pair<String, () -> Unit>>,
|
||||||
|
onMagnify: OnMagnify?,
|
||||||
onExtend: (edge: String, cssX: Float, cssY: Float) -> Unit,
|
onExtend: (edge: String, cssX: Float, cssY: Float) -> Unit,
|
||||||
) {
|
) {
|
||||||
val density = LocalDensity.current
|
val density = LocalDensity.current
|
||||||
@@ -472,19 +594,24 @@ private fun PageSelectionOverlay(
|
|||||||
|
|
||||||
fun mapY(css: Float) = surfaceOriginY + css * scale
|
fun mapY(css: Float) = surfaceOriginY + css * scale
|
||||||
|
|
||||||
val toolbarH = with(density) { 44.dp.toPx() }
|
if (showToolbar) {
|
||||||
val gap = with(density) { 8.dp.toPx() }
|
val toolbarH = with(density) { 44.dp.toPx() }
|
||||||
val topPx = mapY(geometry.top)
|
val gap = with(density) { 8.dp.toPx() }
|
||||||
val toolbarY = if (topPx - toolbarH - gap > surfaceOriginY) topPx - toolbarH - gap else mapY(geometry.bottom) + gap
|
val topPx = mapY(geometry.top)
|
||||||
|
val toolbarY = if (topPx - toolbarH - gap > surfaceOriginY) topPx - toolbarH - gap else mapY(geometry.bottom) + gap
|
||||||
|
EmbeddedSelectionToolbar(
|
||||||
|
items = toolbarItems,
|
||||||
|
centerXpx = mapX((geometry.left + geometry.right) / 2f),
|
||||||
|
topYpx = toolbarY,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
EmbeddedSelectionToolbar(
|
// Half the selection's line height (layer px), clamped — the bounding box spans every selected line, so
|
||||||
items = listOf("Copy" to onCopy),
|
// cap it at a sane single-line height so the loupe centers on the dragged endpoint's line, not the middle
|
||||||
centerXpx = mapX((geometry.left + geometry.right) / 2f),
|
// of a tall multi-line box.
|
||||||
topYpx = toolbarY,
|
val lineHalfPx = ((geometry.bottom - geometry.top) * scale / 2f).coerceAtMost(with(density) { 24.dp.toPx() })
|
||||||
)
|
SelectionHandle(Offset(mapX(geometry.startX), mapY(geometry.startBottom)), isStart = true, surfaceOriginX, surfaceOriginY, scale, lineHalfPx, onMagnify) { x, y -> onExtend("start", x, y) }
|
||||||
|
SelectionHandle(Offset(mapX(geometry.endX), mapY(geometry.endBottom)), isStart = false, surfaceOriginX, surfaceOriginY, scale, lineHalfPx, onMagnify) { x, y -> onExtend("end", x, y) }
|
||||||
SelectionHandle(Offset(mapX(geometry.startX), mapY(geometry.startBottom)), isStart = true, surfaceOriginX, surfaceOriginY, scale) { x, y -> onExtend("start", x, y) }
|
|
||||||
SelectionHandle(Offset(mapX(geometry.endX), mapY(geometry.endBottom)), isStart = false, surfaceOriginX, surfaceOriginY, scale) { x, y -> onExtend("end", x, y) }
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -500,14 +627,25 @@ private fun SelectionHandle(
|
|||||||
surfaceOriginX: Float,
|
surfaceOriginX: Float,
|
||||||
surfaceOriginY: Float,
|
surfaceOriginY: Float,
|
||||||
scale: Float,
|
scale: Float,
|
||||||
|
lineHalfPx: Float,
|
||||||
|
onMagnify: OnMagnify?,
|
||||||
onDragTo: (cssX: Float, cssY: Float) -> Unit,
|
onDragTo: (cssX: Float, cssY: Float) -> Unit,
|
||||||
) {
|
) {
|
||||||
val sizeDp = 22.dp
|
val sizeDp = 22.dp
|
||||||
val sizePx = with(LocalDensity.current) { sizeDp.toPx() }
|
val sizePx = with(LocalDensity.current) { sizeDp.toPx() }
|
||||||
val color = MaterialTheme.colorScheme.primary
|
val color = MaterialTheme.colorScheme.primary
|
||||||
val currentTip by rememberUpdatedState(tipPx)
|
val currentTip by rememberUpdatedState(tipPx)
|
||||||
|
val currentMagnify by rememberUpdatedState(onMagnify)
|
||||||
var dragTip by remember { mutableStateOf<Offset?>(null) }
|
var dragTip by remember { mutableStateOf<Offset?>(null) }
|
||||||
val tip = dragTip ?: tipPx
|
val tip = dragTip ?: tipPx
|
||||||
|
|
||||||
|
// Loupe capture: X follows the finger, Y is locked to the authoritative endpoint's line ([currentTip] is
|
||||||
|
// the foot, so lift by half the line height) — so the bubble shows the line being edited, not wherever the
|
||||||
|
// finger drifts vertically.
|
||||||
|
fun magnify(
|
||||||
|
fingerPx: Offset,
|
||||||
|
active: Boolean = true,
|
||||||
|
) = currentMagnify?.invoke(active, fingerPx, fingerPx.x - surfaceOriginX, currentTip.y - surfaceOriginY - lineHalfPx)
|
||||||
// Place the box so its pointed corner lands on the tip: start = top-right corner, end = top-left corner.
|
// Place the box so its pointed corner lands on the tip: start = top-right corner, end = top-left corner.
|
||||||
val boxLeft = if (isStart) tip.x - sizePx else tip.x
|
val boxLeft = if (isStart) tip.x - sizePx else tip.x
|
||||||
Box(
|
Box(
|
||||||
@@ -516,35 +654,48 @@ private fun SelectionHandle(
|
|||||||
.size(sizeDp)
|
.size(sizeDp)
|
||||||
.pointerInput(Unit) {
|
.pointerInput(Unit) {
|
||||||
detectDragGestures(
|
detectDragGestures(
|
||||||
onDragStart = { dragTip = currentTip },
|
onDragStart = {
|
||||||
|
dragTip = currentTip
|
||||||
|
magnify(currentTip)
|
||||||
|
},
|
||||||
onDrag = { change, delta ->
|
onDrag = { change, delta ->
|
||||||
change.consume()
|
change.consume()
|
||||||
val np = (dragTip ?: currentTip) + delta
|
val np = (dragTip ?: currentTip) + delta
|
||||||
dragTip = np
|
dragTip = np
|
||||||
onDragTo((np.x - surfaceOriginX) / scale, (np.y - surfaceOriginY) / scale)
|
onDragTo((np.x - surfaceOriginX) / scale, (np.y - surfaceOriginY) / scale)
|
||||||
|
magnify(np)
|
||||||
|
},
|
||||||
|
onDragEnd = {
|
||||||
|
dragTip = null
|
||||||
|
magnify(Offset.Zero, active = false)
|
||||||
|
},
|
||||||
|
onDragCancel = {
|
||||||
|
dragTip = null
|
||||||
|
magnify(Offset.Zero, active = false)
|
||||||
},
|
},
|
||||||
onDragEnd = { dragTip = null },
|
|
||||||
onDragCancel = { dragTip = null },
|
|
||||||
)
|
)
|
||||||
},
|
},
|
||||||
) {
|
) {
|
||||||
Canvas(Modifier.fillMaxSize()) {
|
// Hide the teardrop while THIS handle is being dragged — the loupe stands in for it, like Android.
|
||||||
val d = size.minDimension
|
if (dragTip == null) {
|
||||||
val path =
|
Canvas(Modifier.fillMaxSize()) {
|
||||||
Path().apply {
|
val d = size.minDimension
|
||||||
if (isStart) {
|
val path =
|
||||||
moveTo(d, 0f)
|
Path().apply {
|
||||||
lineTo(d, d / 2f)
|
if (isStart) {
|
||||||
arcTo(Rect(0f, 0f, d, d), 0f, 270f, false)
|
moveTo(d, 0f)
|
||||||
close()
|
lineTo(d, d / 2f)
|
||||||
} else {
|
arcTo(Rect(0f, 0f, d, d), 0f, 270f, false)
|
||||||
moveTo(0f, 0f)
|
close()
|
||||||
lineTo(0f, d / 2f)
|
} else {
|
||||||
arcTo(Rect(0f, 0f, d, d), 180f, -270f, false)
|
moveTo(0f, 0f)
|
||||||
close()
|
lineTo(0f, d / 2f)
|
||||||
|
arcTo(Rect(0f, 0f, d, d), 180f, -270f, false)
|
||||||
|
close()
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
drawPath(path, color)
|
||||||
drawPath(path, color)
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -560,6 +711,9 @@ private fun InsertionHandle(
|
|||||||
surfaceOriginX: Float,
|
surfaceOriginX: Float,
|
||||||
surfaceOriginY: Float,
|
surfaceOriginY: Float,
|
||||||
scale: Float,
|
scale: Float,
|
||||||
|
lineHalfPx: Float,
|
||||||
|
onMagnify: OnMagnify?,
|
||||||
|
onTap: () -> Unit,
|
||||||
onDragTo: (cssX: Float, cssY: Float) -> Unit,
|
onDragTo: (cssX: Float, cssY: Float) -> Unit,
|
||||||
) {
|
) {
|
||||||
val density = LocalDensity.current
|
val density = LocalDensity.current
|
||||||
@@ -567,6 +721,16 @@ private fun InsertionHandle(
|
|||||||
val hPx = wPx * 1.4f
|
val hPx = wPx * 1.4f
|
||||||
val color = MaterialTheme.colorScheme.primary
|
val color = MaterialTheme.colorScheme.primary
|
||||||
val currentTip by rememberUpdatedState(tipPx)
|
val currentTip by rememberUpdatedState(tipPx)
|
||||||
|
val currentMagnify by rememberUpdatedState(onMagnify)
|
||||||
|
val currentTap by rememberUpdatedState(onTap)
|
||||||
|
|
||||||
|
// Loupe capture: X follows the finger, Y is locked to the authoritative caret's line ([currentTip] is the
|
||||||
|
// caret foot, so lift by half the line height) — so the bubble shows the edited line, not wherever the
|
||||||
|
// finger drifts vertically (e.g. a diagonal drag in a textarea).
|
||||||
|
fun magnify(
|
||||||
|
fingerPx: Offset,
|
||||||
|
active: Boolean = true,
|
||||||
|
) = currentMagnify?.invoke(active, fingerPx, fingerPx.x - surfaceOriginX, currentTip.y - surfaceOriginY - lineHalfPx)
|
||||||
// Track the finger separately from what we draw: the handle renders at the AUTHORITATIVE caret (tipPx,
|
// Track the finger separately from what we draw: the handle renders at the AUTHORITATIVE caret (tipPx,
|
||||||
// which snaps to a character position as the move round-trips through the shim), while the finger drives
|
// which snaps to a character position as the move round-trips through the shim), while the finger drives
|
||||||
// the move. So the handle stays glued to the line/text and clamps to the field instead of trailing off.
|
// the move. So the handle stays glued to the line/text and clamps to the field instead of trailing off.
|
||||||
@@ -575,33 +739,62 @@ private fun InsertionHandle(
|
|||||||
Modifier
|
Modifier
|
||||||
.absoluteOffset { IntOffset((tipPx.x - wPx / 2f).roundToInt(), tipPx.y.roundToInt()) }
|
.absoluteOffset { IntOffset((tipPx.x - wPx / 2f).roundToInt(), tipPx.y.roundToInt()) }
|
||||||
.size(with(density) { wPx.toDp() }, with(density) { hPx.toDp() })
|
.size(with(density) { wPx.toDp() }, with(density) { hPx.toDp() })
|
||||||
|
// One unified gesture so a TAP (toggle the Paste/Select-All popup) and a DRAG (move the caret)
|
||||||
|
// don't fight each other. We consume the down so the tap never bleeds to the surface (which would
|
||||||
|
// place a caret there and dismiss the popup we're opening).
|
||||||
.pointerInput(Unit) {
|
.pointerInput(Unit) {
|
||||||
detectDragGestures(
|
awaitEachGesture {
|
||||||
onDragStart = { fingerPx = currentTip },
|
val down = awaitFirstDown(requireUnconsumed = false)
|
||||||
onDrag = { change, delta ->
|
down.consume()
|
||||||
change.consume()
|
var dragging = false
|
||||||
val np = (fingerPx ?: currentTip) + delta
|
var fp = currentTip
|
||||||
fingerPx = np
|
while (true) {
|
||||||
onDragTo((np.x - surfaceOriginX) / scale, (np.y - surfaceOriginY) / scale)
|
val change = awaitPointerEvent().changes.firstOrNull() ?: break
|
||||||
},
|
if (!change.pressed) {
|
||||||
onDragEnd = { fingerPx = null },
|
if (!dragging) {
|
||||||
onDragCancel = { fingerPx = null },
|
currentTap()
|
||||||
)
|
} else {
|
||||||
|
fingerPx = null
|
||||||
|
magnify(Offset.Zero, active = false)
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if (!dragging && (change.position - down.position).getDistance() > viewConfiguration.touchSlop) {
|
||||||
|
dragging = true
|
||||||
|
fp = currentTip
|
||||||
|
magnify(currentTip)
|
||||||
|
}
|
||||||
|
if (dragging) {
|
||||||
|
// Read the delta BEFORE consuming: positionChange() returns Offset.Zero once the
|
||||||
|
// change isConsumed, so consuming first (or the sandbox surface consuming the move in
|
||||||
|
// an earlier pass) would freeze `fp` at the caret — the handle drags but the caret
|
||||||
|
// never moves. positionChangeIgnoreConsumed() is immune to both.
|
||||||
|
fp += change.positionChangeIgnoreConsumed()
|
||||||
|
change.consume()
|
||||||
|
fingerPx = fp
|
||||||
|
onDragTo((fp.x - surfaceOriginX) / scale, (fp.y - surfaceOriginY) / scale)
|
||||||
|
magnify(fp)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
},
|
},
|
||||||
) {
|
) {
|
||||||
Canvas(Modifier.fillMaxSize()) {
|
// Hide the teardrop while dragging — the loupe stands in for it, like Android.
|
||||||
val r = size.width / 2f
|
if (fingerPx == null) {
|
||||||
val cx = size.width / 2f
|
Canvas(Modifier.fillMaxSize()) {
|
||||||
val cy = size.height - r
|
val r = size.width / 2f
|
||||||
val path =
|
val cx = size.width / 2f
|
||||||
Path().apply {
|
val cy = size.height - r
|
||||||
moveTo(cx, 0f)
|
val path =
|
||||||
lineTo(cx + r, cy)
|
Path().apply {
|
||||||
arcTo(Rect(cx - r, cy - r, cx + r, cy + r), 0f, 180f, false)
|
moveTo(cx, 0f)
|
||||||
lineTo(cx, 0f)
|
lineTo(cx + r, cy)
|
||||||
close()
|
arcTo(Rect(cx - r, cy - r, cx + r, cy + r), 0f, 180f, false)
|
||||||
}
|
lineTo(cx, 0f)
|
||||||
drawPath(path, color)
|
close()
|
||||||
|
}
|
||||||
|
drawPath(path, color)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -619,32 +812,37 @@ private fun EmbeddedSelectionToolbar(
|
|||||||
centerXpx: Float,
|
centerXpx: Float,
|
||||||
topYpx: Float,
|
topYpx: Float,
|
||||||
) {
|
) {
|
||||||
// Self-center: the toolbar's width depends on its items, so measure it and offset by half-width around
|
// Center in a SINGLE layout pass: measure the bar, then place it at centerX − width/2. (The old
|
||||||
// the requested centre-x (a one-frame left-aligned flash before the first measurement is acceptable).
|
// measure-then-offset approach rendered it left-edge-at-centerX for one frame, so it visibly slid in
|
||||||
var widthPx by remember { mutableStateOf(0) }
|
// from the right each time the bar was (re)composed.)
|
||||||
Surface(
|
Layout(
|
||||||
modifier =
|
content = {
|
||||||
Modifier
|
Surface(
|
||||||
.absoluteOffset { IntOffset((centerXpx - widthPx / 2f).roundToInt(), topYpx.roundToInt()) }
|
shape = RoundedCornerShape(8.dp),
|
||||||
.onGloballyPositioned { widthPx = it.size.width },
|
color = MaterialTheme.colorScheme.surface,
|
||||||
shape = RoundedCornerShape(8.dp),
|
border = BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant),
|
||||||
color = MaterialTheme.colorScheme.surface,
|
tonalElevation = 3.dp,
|
||||||
border = BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant),
|
shadowElevation = 6.dp,
|
||||||
tonalElevation = 3.dp,
|
) {
|
||||||
shadowElevation = 6.dp,
|
Row(verticalAlignment = Alignment.CenterVertically) {
|
||||||
) {
|
items.forEachIndexed { i, (label, action) ->
|
||||||
Row(verticalAlignment = Alignment.CenterVertically) {
|
if (i > 0) {
|
||||||
items.forEachIndexed { i, (label, action) ->
|
Box(
|
||||||
if (i > 0) {
|
Modifier
|
||||||
Box(
|
.width(1.dp)
|
||||||
Modifier
|
.height(20.dp)
|
||||||
.width(1.dp)
|
.background(MaterialTheme.colorScheme.outlineVariant),
|
||||||
.height(20.dp)
|
)
|
||||||
.background(MaterialTheme.colorScheme.outlineVariant),
|
}
|
||||||
)
|
SelectionToolbarItem(label, action)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
SelectionToolbarItem(label, action)
|
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
) { measurables, constraints ->
|
||||||
|
val bar = measurables.first().measure(constraints.copy(minWidth = 0, minHeight = 0))
|
||||||
|
layout(constraints.maxWidth, constraints.maxHeight) {
|
||||||
|
bar.place((centerXpx - bar.width / 2f).roundToInt(), topYpx.roundToInt())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -659,7 +857,17 @@ private fun SelectionToolbarItem(
|
|||||||
color = MaterialTheme.colorScheme.onSurfaceVariant,
|
color = MaterialTheme.colorScheme.onSurfaceVariant,
|
||||||
modifier =
|
modifier =
|
||||||
Modifier
|
Modifier
|
||||||
.pointerInput(label) { detectTapGestures(onTap = { onClick() }) }
|
// Consume the DOWN (not just the up) so the tap never bleeds through to the embedded surface
|
||||||
.padding(horizontal = 12.dp, vertical = 10.dp),
|
// beneath — otherwise tapping the bar over non-editable page area blurs the field.
|
||||||
|
.pointerInput(label) {
|
||||||
|
awaitEachGesture {
|
||||||
|
awaitFirstDown(requireUnconsumed = false).consume()
|
||||||
|
val up = waitForUpOrCancellation()
|
||||||
|
if (up != null) {
|
||||||
|
up.consume()
|
||||||
|
onClick()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}.padding(horizontal = 12.dp, vertical = 10.dp),
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|||||||
+33
-3
@@ -121,6 +121,21 @@ class RemoteImeView(
|
|||||||
var onRangeSelectionChanged: ((Boolean) -> Unit)? = null
|
var onRangeSelectionChanged: ((Boolean) -> Unit)? = null
|
||||||
private var hadRange = false
|
private var hadRange = false
|
||||||
|
|
||||||
|
// Off-window Chrome abandons a selection by momentarily collapsing the caret to an endpoint, which we (or
|
||||||
|
// the page shim) re-assert right back — so the mirrored selection flickers range→caret→range within a few
|
||||||
|
// ms during a long-press/double-tap. The host-drawn handles/toolbar are gated on [hadRange], so reporting
|
||||||
|
// every flip blinks them off and back on, in lock-step with each collapse cycle. Native never shows that
|
||||||
|
// churn. So we DEFER the "range lost" signal by [RANGE_LOSS_DEBOUNCE_MS]: a re-assert that restores the
|
||||||
|
// range first cancels the pending hide, and only a selection that truly STAYS collapsed hides the overlays.
|
||||||
|
// Gaining a range is always reported immediately.
|
||||||
|
private val reportRangeLost =
|
||||||
|
Runnable {
|
||||||
|
if (selectionStart == selectionEnd && hadRange) {
|
||||||
|
hadRange = false
|
||||||
|
onRangeSelectionChanged?.invoke(false)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/** Fired when the user edits text via the keyboard (not on programmatic page-state applies). The host
|
/** Fired when the user edits text via the keyboard (not on programmatic page-state applies). The host
|
||||||
* hides the insertion handle while typing, the way Android does. */
|
* hides the insertion handle while typing, the way Android does. */
|
||||||
var onEdited: (() -> Unit)? = null
|
var onEdited: (() -> Unit)? = null
|
||||||
@@ -187,6 +202,7 @@ class RemoteImeView(
|
|||||||
|
|
||||||
/** The page field blurred: drop the keyboard. */
|
/** The page field blurred: drop the keyboard. */
|
||||||
fun onPageBlur() {
|
fun onPageBlur() {
|
||||||
|
removeCallbacks(reportRangeLost)
|
||||||
if (hadRange) {
|
if (hadRange) {
|
||||||
hadRange = false
|
hadRange = false
|
||||||
onRangeSelectionChanged?.invoke(false)
|
onRangeSelectionChanged?.invoke(false)
|
||||||
@@ -227,9 +243,18 @@ class RemoteImeView(
|
|||||||
) {
|
) {
|
||||||
super.onSelectionChanged(selStart, selEnd)
|
super.onSelectionChanged(selStart, selEnd)
|
||||||
val isRange = selStart != selEnd
|
val isRange = selStart != selEnd
|
||||||
if (isRange != hadRange) {
|
if (isRange) {
|
||||||
hadRange = isRange
|
// A range is back (or still here): cancel any pending hide and show immediately.
|
||||||
onRangeSelectionChanged?.invoke(isRange)
|
removeCallbacks(reportRangeLost)
|
||||||
|
if (!hadRange) {
|
||||||
|
hadRange = true
|
||||||
|
onRangeSelectionChanged?.invoke(true)
|
||||||
|
}
|
||||||
|
} else if (hadRange) {
|
||||||
|
// Collapsed — but this may be Chrome's transient abandonment we're about to re-assert. Defer the
|
||||||
|
// hide; if the range returns within the window, the show branch above cancels this.
|
||||||
|
removeCallbacks(reportRangeLost)
|
||||||
|
postDelayed(reportRangeLost, RANGE_LOSS_DEBOUNCE_MS)
|
||||||
}
|
}
|
||||||
schedule()
|
schedule()
|
||||||
}
|
}
|
||||||
@@ -321,5 +346,10 @@ class RemoteImeView(
|
|||||||
// Wider than Chrome's ~600ms re-collapse interval so consecutive abandonments stay inside the window,
|
// Wider than Chrome's ~600ms re-collapse interval so consecutive abandonments stay inside the window,
|
||||||
// yet short enough that a deliberate user tap-to-collapse (well after the gesture) is accepted.
|
// yet short enough that a deliberate user tap-to-collapse (well after the gesture) is accepted.
|
||||||
private const val REASSERT_WINDOW_MS = 800L
|
private const val REASSERT_WINDOW_MS = 800L
|
||||||
|
|
||||||
|
// How long a collapse must persist before we hide the host-drawn selection overlays. The shim/host
|
||||||
|
// re-assert restores an abandonment collapse within a frame or two (one IPC hop), so this only needs to
|
||||||
|
// outlast that round-trip — comfortably short enough that a genuine tap-to-collapse still feels instant.
|
||||||
|
private const val RANGE_LOSS_DEBOUNCE_MS = 250L
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+183
@@ -0,0 +1,183 @@
|
|||||||
|
/*
|
||||||
|
* Copyright (c) 2025 Vitor Pamplona
|
||||||
|
*
|
||||||
|
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||||
|
* this software and associated documentation files (the "Software"), to deal in
|
||||||
|
* the Software without restriction, including without limitation the rights to use,
|
||||||
|
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||||
|
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||||
|
* subject to the following conditions:
|
||||||
|
*
|
||||||
|
* The above copyright notice and this permission notice shall be included in all
|
||||||
|
* copies or substantial portions of the Software.
|
||||||
|
*
|
||||||
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||||
|
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||||
|
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||||
|
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||||
|
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||||
|
*/
|
||||||
|
package com.vitorpamplona.amethyst.ui.screen.loggedIn.embed
|
||||||
|
|
||||||
|
import androidx.compose.runtime.Stable
|
||||||
|
import androidx.compose.runtime.getValue
|
||||||
|
import androidx.compose.runtime.mutableStateOf
|
||||||
|
import androidx.compose.runtime.setValue
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Single source of truth for the host-drawn selection UI over an embedded surface. Folds together what used
|
||||||
|
* to be scattered booleans/geometries (`showSelectionToolbar`, `showInsertionHandle`, `fieldGeometry`,
|
||||||
|
* `rangeFieldGeometry`, `pageSelection`) so the show/hide rules are expressed in one place.
|
||||||
|
*
|
||||||
|
* Three mutually-exclusive selection contexts can be active: a bare caret in a field ([insertionGeometry]),
|
||||||
|
* a range inside an `<input>`/`<textarea>` ([fieldRange], gated by [fieldHasRange] — the authoritative
|
||||||
|
* "is there a selection" signal from the mirror [RemoteImeView]), or a plain page-text selection
|
||||||
|
* ([pageSelection]). [dragging] and [scrolling] are the transient modifiers:
|
||||||
|
*
|
||||||
|
* - **dragging** a handle/caret hides the floating toolbar (and the dragged handle hides itself while the
|
||||||
|
* loupe stands in), matching Android; the other handle stays so you can see the live range.
|
||||||
|
* - **scrolling** hides every overlay (its geometry is stale mid-scroll); the shim re-reports geometry on
|
||||||
|
* settle, then clears this, so they reappear in the right place.
|
||||||
|
*/
|
||||||
|
@Stable
|
||||||
|
class SelectionUiState {
|
||||||
|
/** Latest field geometry; carries the caret rect when collapsed, positioning the insertion handle. */
|
||||||
|
var fieldGeometry by mutableStateOf<SelectionGeometry?>(null)
|
||||||
|
private set
|
||||||
|
|
||||||
|
/** Held in-field range geometry (real endpoint feet) — survives Chrome's transient collapse-to-caret. */
|
||||||
|
var fieldRange by mutableStateOf<SelectionGeometry?>(null)
|
||||||
|
private set
|
||||||
|
|
||||||
|
/** Authoritative "the field selection is a non-empty range", from the mirror EditText. */
|
||||||
|
var fieldHasRange by mutableStateOf(false)
|
||||||
|
private set
|
||||||
|
|
||||||
|
/** A bare caret handle is showing — set by a tap-placed caret in non-empty text, cleared on typing/blur/timeout. */
|
||||||
|
var caretShown by mutableStateOf(false)
|
||||||
|
private set
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether the focused field currently has any text. Native (`Editor.onTouchUpEvent`) only offers the
|
||||||
|
* insertion handle when `text.length() > 0`, so an empty field never gets a draggable cursor handle
|
||||||
|
* (just the blinking caret). Gating on this is what stops the handle popping up on focus of an empty box.
|
||||||
|
*/
|
||||||
|
var fieldHasText by mutableStateOf(false)
|
||||||
|
private set
|
||||||
|
|
||||||
|
/** Plain page-text (non-editable) selection. */
|
||||||
|
var pageSelection by mutableStateOf<ImeEvent.PageSelection?>(null)
|
||||||
|
private set
|
||||||
|
|
||||||
|
/** A handle/caret is being dragged. */
|
||||||
|
var dragging by mutableStateOf(false)
|
||||||
|
|
||||||
|
/** The embedded surface is scrolling. */
|
||||||
|
var scrolling by mutableStateOf(false)
|
||||||
|
|
||||||
|
/** Tapping the bare insertion handle opens a small Paste/Select-All popup (native); tap-elsewhere closes it. */
|
||||||
|
var insertionPopup by mutableStateOf(false)
|
||||||
|
private set
|
||||||
|
|
||||||
|
private val insertionGeometry: SelectionGeometry?
|
||||||
|
get() = fieldGeometry?.takeIf { it.caretX != null && it.caretBottom != null && it.viewportWidth > 0f }
|
||||||
|
|
||||||
|
// ---- derived visibility (read in composition; track the backing state) ----
|
||||||
|
|
||||||
|
val insertionHandle: SelectionGeometry?
|
||||||
|
get() = if (caretShown && fieldHasText && !fieldHasRange && !scrolling) insertionGeometry else null
|
||||||
|
|
||||||
|
/** Geometry for the insertion-handle Paste/Select-All popup, or null when it shouldn't show. */
|
||||||
|
val insertionPopupAt: SelectionGeometry?
|
||||||
|
get() = if (insertionPopup && !dragging) insertionHandle else null
|
||||||
|
|
||||||
|
val fieldHandles: SelectionGeometry?
|
||||||
|
get() = if (fieldHasRange && !scrolling) fieldRange?.takeIf { it.viewportWidth > 0f } else null
|
||||||
|
|
||||||
|
val fieldToolbar: SelectionGeometry?
|
||||||
|
get() = if (fieldHasRange && !dragging && !scrolling) (fieldRange ?: fieldGeometry) else null
|
||||||
|
|
||||||
|
private val pageGeom: SelectionGeometry?
|
||||||
|
get() = pageSelection?.takeIf { it.active }?.geometry?.takeIf { it.viewportWidth > 0f }
|
||||||
|
|
||||||
|
val pageHandles: SelectionGeometry?
|
||||||
|
get() = if (!scrolling) pageGeom else null
|
||||||
|
|
||||||
|
val pageToolbar: SelectionGeometry?
|
||||||
|
get() = if (!dragging && !scrolling) pageGeom else null
|
||||||
|
|
||||||
|
// ---- signal sinks ----
|
||||||
|
|
||||||
|
fun onFieldGeometry(
|
||||||
|
g: SelectionGeometry?,
|
||||||
|
hasText: Boolean,
|
||||||
|
) {
|
||||||
|
fieldHasText = hasText
|
||||||
|
if (g == null) return
|
||||||
|
fieldGeometry = g
|
||||||
|
if (g.isRange) fieldRange = g
|
||||||
|
// Only a tap-placed caret in NON-empty text gets the handle — like native. (Typing suppresses the
|
||||||
|
// echo, so this fires for taps, not keystrokes; see RemoteImeView / shim __nappletIme guard.)
|
||||||
|
if (g.caretX != null && hasText) caretShown = true
|
||||||
|
}
|
||||||
|
|
||||||
|
/** From [RemoteImeView.onRangeSelectionChanged]: the field gained/lost a non-empty selection. */
|
||||||
|
fun onFieldRangeToggle(hasRange: Boolean) {
|
||||||
|
fieldHasRange = hasRange
|
||||||
|
if (!hasRange) fieldRange = null
|
||||||
|
if (hasRange) insertionPopup = false // a selection formed → no insertion popup
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The user tapped the focused field (placing a caret) — (re-)show the insertion handle even if the caret
|
||||||
|
* didn't move. Gated to non-empty text by the [insertionHandle] getter, exactly like native.
|
||||||
|
*/
|
||||||
|
fun onCaretTap(g: SelectionGeometry?) {
|
||||||
|
if (g != null) fieldGeometry = g
|
||||||
|
caretShown = true
|
||||||
|
insertionPopup = false // a tap in the text dismisses the popup (and moves the caret)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The user typed: hide the insertion handle, like Android. */
|
||||||
|
fun onEdited() {
|
||||||
|
caretShown = false
|
||||||
|
insertionPopup = false
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Native auto-hides the insertion handle after ~a few seconds of inactivity; a later tap re-shows it. */
|
||||||
|
fun hideCaret() {
|
||||||
|
caretShown = false
|
||||||
|
insertionPopup = false
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Tap on the bare insertion handle toggles its Paste/Select-All popup. */
|
||||||
|
fun toggleInsertionPopup() {
|
||||||
|
insertionPopup = !insertionPopup
|
||||||
|
}
|
||||||
|
|
||||||
|
fun hideInsertionPopup() {
|
||||||
|
insertionPopup = false
|
||||||
|
}
|
||||||
|
|
||||||
|
fun onBlur() {
|
||||||
|
fieldGeometry = null
|
||||||
|
fieldRange = null
|
||||||
|
fieldHasRange = false
|
||||||
|
fieldHasText = false
|
||||||
|
caretShown = false
|
||||||
|
insertionPopup = false
|
||||||
|
}
|
||||||
|
|
||||||
|
fun onPageSelection(sel: ImeEvent.PageSelection?) {
|
||||||
|
pageSelection = sel
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Full teardown when the active tab/controller changes. */
|
||||||
|
fun reset() {
|
||||||
|
onBlur()
|
||||||
|
pageSelection = null
|
||||||
|
dragging = false
|
||||||
|
scrolling = false
|
||||||
|
}
|
||||||
|
}
|
||||||
+38
-1
@@ -31,6 +31,7 @@ import android.os.IBinder
|
|||||||
import android.os.Looper
|
import android.os.Looper
|
||||||
import android.os.Message
|
import android.os.Message
|
||||||
import android.os.Messenger
|
import android.os.Messenger
|
||||||
|
import android.os.SystemClock
|
||||||
import androidx.annotation.RequiresApi
|
import androidx.annotation.RequiresApi
|
||||||
import androidx.privacysandbox.ui.client.SandboxedUiAdapterFactory
|
import androidx.privacysandbox.ui.client.SandboxedUiAdapterFactory
|
||||||
import androidx.privacysandbox.ui.client.view.SandboxedSdkView
|
import androidx.privacysandbox.ui.client.view.SandboxedSdkView
|
||||||
@@ -39,8 +40,10 @@ import com.vitorpamplona.amethyst.napplethost.NappletEmbedContract
|
|||||||
import com.vitorpamplona.amethyst.napplethost.NappletHostContract
|
import com.vitorpamplona.amethyst.napplethost.NappletHostContract
|
||||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedImeBridge
|
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedImeBridge
|
||||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedLoadStatus
|
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedLoadStatus
|
||||||
|
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedMagnifierProbe
|
||||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedSurfaceController
|
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedSurfaceController
|
||||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.ImeEvent
|
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.ImeEvent
|
||||||
|
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.MagnifierFrame
|
||||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.parseSelectionGeometry
|
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.parseSelectionGeometry
|
||||||
import org.json.JSONObject
|
import org.json.JSONObject
|
||||||
import java.util.concurrent.atomic.AtomicLong
|
import java.util.concurrent.atomic.AtomicLong
|
||||||
@@ -60,7 +63,8 @@ class EmbeddedNostrAppController(
|
|||||||
private val appContext: Context,
|
private val appContext: Context,
|
||||||
private val params: Bundle,
|
private val params: Bundle,
|
||||||
) : EmbeddedSurfaceController,
|
) : EmbeddedSurfaceController,
|
||||||
EmbeddedImeBridge {
|
EmbeddedImeBridge,
|
||||||
|
EmbeddedMagnifierProbe {
|
||||||
private val incoming = Messenger(Handler(Looper.getMainLooper(), ::onServiceMessage))
|
private val incoming = Messenger(Handler(Looper.getMainLooper(), ::onServiceMessage))
|
||||||
private var serviceMessenger: Messenger? = null
|
private var serviceMessenger: Messenger? = null
|
||||||
private var bound = false
|
private var bound = false
|
||||||
@@ -94,6 +98,8 @@ class EmbeddedNostrAppController(
|
|||||||
|
|
||||||
override var onImeEvent: ((ImeEvent) -> Unit)? = null
|
override var onImeEvent: ((ImeEvent) -> Unit)? = null
|
||||||
|
|
||||||
|
override var onMagnifierFrame: ((MagnifierFrame) -> Unit)? = null
|
||||||
|
|
||||||
private val connection =
|
private val connection =
|
||||||
object : ServiceConnection {
|
object : ServiceConnection {
|
||||||
override fun onServiceConnected(
|
override fun onServiceConnected(
|
||||||
@@ -126,6 +132,7 @@ class EmbeddedNostrAppController(
|
|||||||
onStateChanged = null
|
onStateChanged = null
|
||||||
onNotice = null
|
onNotice = null
|
||||||
onImeEvent = null
|
onImeEvent = null
|
||||||
|
onMagnifierFrame = null
|
||||||
onLoadStatusChanged = null
|
onLoadStatusChanged = null
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -184,6 +191,19 @@ class EmbeddedNostrAppController(
|
|||||||
val failed = msg.data?.getBoolean(NappletEmbedContract.KEY_LOAD_FAILED, false) ?: false
|
val failed = msg.data?.getBoolean(NappletEmbedContract.KEY_LOAD_FAILED, false) ?: false
|
||||||
onLoadState(isLoading, failed)
|
onLoadState(isLoading, failed)
|
||||||
}
|
}
|
||||||
|
NappletEmbedContract.MSG_MAGNIFIER_FRAME -> {
|
||||||
|
val data = msg.data ?: return true
|
||||||
|
val bytes = data.getByteArray(NappletEmbedContract.KEY_MAG_BYTES) ?: return true
|
||||||
|
onMagnifierFrame?.invoke(
|
||||||
|
MagnifierFrame(
|
||||||
|
bytes = bytes,
|
||||||
|
width = data.getInt(NappletEmbedContract.KEY_MAG_W),
|
||||||
|
height = data.getInt(NappletEmbedContract.KEY_MAG_H),
|
||||||
|
captureMs = data.getDouble(NappletEmbedContract.KEY_MAG_CAPTURE_MS),
|
||||||
|
requestStampNanos = data.getLong(NappletEmbedContract.KEY_MAG_REQ_T),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
else -> return false
|
else -> return false
|
||||||
}
|
}
|
||||||
return true
|
return true
|
||||||
@@ -191,6 +211,21 @@ class EmbeddedNostrAppController(
|
|||||||
|
|
||||||
override fun sendImeOp(json: String) = send(NappletEmbedContract.MSG_IME_OP) { putString(NappletEmbedContract.KEY_IME_PAYLOAD, json) }
|
override fun sendImeOp(json: String) = send(NappletEmbedContract.MSG_IME_OP) { putString(NappletEmbedContract.KEY_IME_PAYLOAD, json) }
|
||||||
|
|
||||||
|
override fun requestMagnifier(
|
||||||
|
surfaceX: Float,
|
||||||
|
surfaceY: Float,
|
||||||
|
boxWidthPx: Int,
|
||||||
|
boxHeightPx: Int,
|
||||||
|
zoom: Float,
|
||||||
|
) = send(NappletEmbedContract.MSG_MAGNIFIER_REQUEST) {
|
||||||
|
putFloat(NappletEmbedContract.KEY_MAG_X, surfaceX)
|
||||||
|
putFloat(NappletEmbedContract.KEY_MAG_Y, surfaceY)
|
||||||
|
putInt(NappletEmbedContract.KEY_MAG_BOX_W, boxWidthPx)
|
||||||
|
putInt(NappletEmbedContract.KEY_MAG_BOX_H, boxHeightPx)
|
||||||
|
putFloat(NappletEmbedContract.KEY_MAG_ZOOM, zoom)
|
||||||
|
putLong(NappletEmbedContract.KEY_MAG_REQ_T, SystemClock.elapsedRealtimeNanos())
|
||||||
|
}
|
||||||
|
|
||||||
private fun parseImeEvent(payload: String): ImeEvent? {
|
private fun parseImeEvent(payload: String): ImeEvent? {
|
||||||
val o = runCatching { JSONObject(payload) }.getOrNull() ?: return null
|
val o = runCatching { JSONObject(payload) }.getOrNull() ?: return null
|
||||||
return when (o.optString("type")) {
|
return when (o.optString("type")) {
|
||||||
@@ -218,6 +253,8 @@ class EmbeddedNostrAppController(
|
|||||||
text = o.optString("text", ""),
|
text = o.optString("text", ""),
|
||||||
geometry = parseSelectionGeometry(o.optJSONObject("geom")),
|
geometry = parseSelectionGeometry(o.optJSONObject("geom")),
|
||||||
)
|
)
|
||||||
|
"ime.scroll" -> ImeEvent.Scroll(active = o.optBoolean("active", false))
|
||||||
|
"ime.carettap" -> ImeEvent.CaretTap(geometry = parseSelectionGeometry(o.optJSONObject("geom")))
|
||||||
else -> null
|
else -> null
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -277,6 +277,12 @@
|
|||||||
return [n.selectionStart || 0, n.selectionEnd || 0];
|
return [n.selectionStart || 0, n.selectionEnd || 0];
|
||||||
}
|
}
|
||||||
function setSel(n, s, e){
|
function setSel(n, s, e){
|
||||||
|
// No-op if already there: re-applying the same selection still fires `select`/`selectionchange`, and
|
||||||
|
// every such redundant event ripples into a host report → geometry update → toolbar/handle recompose
|
||||||
|
// (visible churn). Our re-asserts/re-applies frequently target the current range, so guard them here.
|
||||||
|
var cur = selOf(n);
|
||||||
|
if (cur[0] === s && cur[1] === e) return;
|
||||||
|
lastSelActivityAt = perfNow(); // a real selection write → the field may auto-scroll to reveal it
|
||||||
if (isCE(n)) ceSetSel(n, s, e);
|
if (isCE(n)) ceSetSel(n, s, e);
|
||||||
else { try { n.setSelectionRange(s, e); } catch (_) {} }
|
else { try { n.setSelectionRange(s, e); } catch (_) {} }
|
||||||
}
|
}
|
||||||
@@ -329,6 +335,13 @@
|
|||||||
else if (x < c.x) hi = mid;
|
else if (x < c.x) hi = mid;
|
||||||
else lo = mid + 1;
|
else lo = mid + 1;
|
||||||
}
|
}
|
||||||
|
// The search lands on the boundary just RIGHT of x; round to the NEAREST boundary instead (native
|
||||||
|
// getOffsetForPosition) so tapping the left half of a glyph doesn't advance the caret past it. Only
|
||||||
|
// compare within the same line (skip when lo sits at a wrap, where lo-1 is on the previous row).
|
||||||
|
if (lo > 0) {
|
||||||
|
var cl = caretCoords(n, lo - 1), cr = caretCoords(n, lo);
|
||||||
|
if (cl && cr && cl.top === cr.top && (x - cl.x) < (cr.x - x)) lo = lo - 1;
|
||||||
|
}
|
||||||
return lo;
|
return lo;
|
||||||
}
|
}
|
||||||
// The focused field's bounding box in CSS px (toolbar anchor). When the selection is a bare caret it also
|
// The focused field's bounding box in CSS px (toolbar anchor). When the selection is a bare caret it also
|
||||||
@@ -337,10 +350,22 @@
|
|||||||
try {
|
try {
|
||||||
var b = n.getBoundingClientRect();
|
var b = n.getBoundingClientRect();
|
||||||
var g = { l: b.left, t: b.top, r: b.right, b: b.bottom, sx: b.left, sb: b.bottom, ex: b.right, eb: b.bottom, vw: window.innerWidth };
|
var g = { l: b.left, t: b.top, r: b.right, b: b.bottom, sx: b.left, sb: b.bottom, ex: b.right, eb: b.bottom, vw: window.innerWidth };
|
||||||
var sel = selOf(n);
|
if (!isCE(n)) {
|
||||||
if (sel[0] === sel[1] && !isCE(n)) {
|
var sel = selOf(n);
|
||||||
var c = caretCoords(n, sel[0]);
|
if (sel[0] === sel[1]) {
|
||||||
if (c) { g.cx = c.x; g.ct = c.top; g.cb = c.bottom; }
|
// Bare caret → carry the caret rect so the host shows the insertion handle.
|
||||||
|
var c = caretCoords(n, sel[0]);
|
||||||
|
if (c) { g.cx = c.x; g.ct = c.top; g.cb = c.bottom; }
|
||||||
|
} else {
|
||||||
|
// Range → carry the start/end caret feet so the host shows draggable selection handles, and
|
||||||
|
// tighten the box to the selected line span so the toolbar anchors above the selection.
|
||||||
|
var cs = caretCoords(n, sel[0]), ce = caretCoords(n, sel[1]);
|
||||||
|
if (cs && ce) {
|
||||||
|
g.rng = true; // marks a real range so the host keeps these feet through Chrome's collapse fight
|
||||||
|
g.sx = cs.x; g.sb = cs.bottom; g.ex = ce.x; g.eb = ce.bottom;
|
||||||
|
g.t = Math.min(cs.top, ce.top); g.b = Math.max(cs.bottom, ce.bottom);
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
return g;
|
return g;
|
||||||
} catch (_) { return null; }
|
} catch (_) { return null; }
|
||||||
@@ -356,17 +381,40 @@
|
|||||||
// Last selection we either applied (applyState) or already reported, so the asynchronous
|
// Last selection we either applied (applyState) or already reported, so the asynchronous
|
||||||
// selectionchange our own setSel triggers doesn't echo back to the host as a fresh edit.
|
// selectionchange our own setSel triggers doesn't echo back to the host as a fresh edit.
|
||||||
var lastSel = null;
|
var lastSel = null;
|
||||||
|
// The live non-collapsed field range we protect from Chrome's off-window abandonment (see the
|
||||||
|
// selectionchange handler). Tracked centrally so handle-extends and select-all keep it current.
|
||||||
|
var lastFieldRange = null, lastFieldAt = -1;
|
||||||
|
// When the selection last changed (Chrome-driven OR our own setSel). Forming/re-asserting a selection makes
|
||||||
|
// the browser auto-scroll the field to reveal it, firing `scroll` events that are NOT a user content scroll.
|
||||||
|
// The hide-on-scroll path uses this to ignore those: hiding the host overlays on a selection-reveal scroll
|
||||||
|
// makes the handles/toolbar blink off-and-on every time a selection settles. See onAnyScroll.
|
||||||
|
var lastSelActivityAt = -1;
|
||||||
function sameSel(a, b){ return !!(a && b && a[0] === b[0] && a[1] === b[1]); }
|
function sameSel(a, b){ return !!(a && b && a[0] === b[0] && a[1] === b[1]); }
|
||||||
|
function noteSel(sel){
|
||||||
|
lastSel = sel;
|
||||||
|
if (el && !isCE(el)) {
|
||||||
|
if (sel[0] !== sel[1]) { lastFieldRange = [sel[0], sel[1]]; lastFieldAt = perfNow(); }
|
||||||
|
else { lastFieldRange = null; }
|
||||||
|
}
|
||||||
|
}
|
||||||
function reportState(){
|
function reportState(){
|
||||||
if (!el) return;
|
if (!el) return;
|
||||||
var sel = selOf(el);
|
var sel = selOf(el);
|
||||||
lastSel = sel;
|
noteSel(sel);
|
||||||
send({ type:'ime.state', text: valOf(el), selStart: sel[0], selEnd: sel[1], geom: fieldGeom(el) });
|
send({ type:'ime.state', text: valOf(el), selStart: sel[0], selEnd: sel[1], geom: fieldGeom(el) });
|
||||||
}
|
}
|
||||||
|
|
||||||
document.addEventListener('focusin', function(e){
|
document.addEventListener('focusin', function(e){
|
||||||
if (isEditable(e.target)) {
|
if (isEditable(e.target)) {
|
||||||
el = e.target; inComposition = false; lastSel = selOf(el); send(focusInfo(el));
|
el = e.target; inComposition = false; lastSel = selOf(el); send(focusInfo(el));
|
||||||
|
// Focusing a field clears any page-text selection in the browser. The page selectionchange handler is
|
||||||
|
// muted while a field is focused (el is set), so it never emits the `active:false` — emit it here, or
|
||||||
|
// the host's page handles + Copy bar linger ABOVE the field overlays (and, being z-above, steal the
|
||||||
|
// caret/selection-handle drag so the caret can't be moved).
|
||||||
|
if (pageSelText) { lastPageRange = null; sendPageSel(false, null); }
|
||||||
|
// Cancel any in-flight scroll-hide from the page phase so the new field overlays aren't suppressed by a
|
||||||
|
// stale `scrolling` state (its settle would otherwise keep getting re-armed by the field reveal-scrolls).
|
||||||
|
scrolling = false; if (scrollTimer) { clearTimeout(scrollTimer); scrollTimer = null; }
|
||||||
// The host shrinks the surface for the keyboard, but also nudge the field into view in case IME
|
// The host shrinks the surface for the keyboard, but also nudge the field into view in case IME
|
||||||
// insets aren't delivered (some hosts) so it never sits behind the keyboard.
|
// insets aren't delivered (some hosts) so it never sits behind the keyboard.
|
||||||
try { el.scrollIntoView({ block: 'center', inline: 'nearest' }); } catch (_) {}
|
try { el.scrollIntoView({ block: 'center', inline: 'nearest' }); } catch (_) {}
|
||||||
@@ -377,14 +425,71 @@
|
|||||||
}, true);
|
}, true);
|
||||||
// The page (its own JS, autofill) changed the field: resync the host keyboard's view of it.
|
// The page (its own JS, autofill) changed the field: resync the host keyboard's view of it.
|
||||||
document.addEventListener('input', function(e){ if (e.target === el && !el.__nappletIme) reportState(); }, true);
|
document.addEventListener('input', function(e){ if (e.target === el && !el.__nappletIme) reportState(); }, true);
|
||||||
// Just mirror selection changes inside the focused editable to the host. The host owns the authoritative
|
// Mirror selection changes inside the focused editable to the host. Off-window Chrome abandons a field
|
||||||
// selection, so it (not the shim) detects and re-asserts the collapse-to-endpoint that Chrome does when it
|
// selection by collapsing the caret to one of its endpoints; we re-assert it RIGHT HERE, synchronously,
|
||||||
// abandons a selection it can't present handles for in the embedded surface — see RemoteImeView.onPageState.
|
// the same way the page-text path does — reverting before the collapse paints, so it doesn't blink (the
|
||||||
|
// old path round-tripped through the host EditText, leaving a visible collapsed frame each cycle). The
|
||||||
|
// host's own re-assert in RemoteImeView.onPageState stays as a fallback for collapses we don't catch.
|
||||||
document.addEventListener('selectionchange', function(){
|
document.addEventListener('selectionchange', function(){
|
||||||
if (!el || el.__nappletIme) return;
|
if (!el || el.__nappletIme) return; // our own applyState/setSel
|
||||||
if (sameSel(selOf(el), lastSel)) return; // our own applyState/setSel echoing back
|
lastSelActivityAt = perfNow(); // Chrome moved the selection → an imminent reveal-scroll isn't a user scroll
|
||||||
|
var sel = selOf(el);
|
||||||
|
if (sameSel(sel, lastSel)) return; // echo of what we just applied
|
||||||
|
if (!isCE(el) && lastFieldRange && sel[0] === sel[1] &&
|
||||||
|
(sel[0] === lastFieldRange[0] || sel[0] === lastFieldRange[1]) &&
|
||||||
|
(perfNow() - lastFieldAt) < 1500) {
|
||||||
|
// Chrome's off-window abandonment collapsed our live range to an endpoint → snap it back
|
||||||
|
// synchronously (reverts before paint, so no blink) and keep the window open. The host already
|
||||||
|
// holds this range, so we don't re-report (which would feed the slow round-trip loop).
|
||||||
|
el.__nappletIme = true;
|
||||||
|
setSel(el, lastFieldRange[0], lastFieldRange[1]);
|
||||||
|
el.__nappletIme = false;
|
||||||
|
lastSel = selOf(el);
|
||||||
|
lastFieldAt = perfNow();
|
||||||
|
return;
|
||||||
|
}
|
||||||
reportState();
|
reportState();
|
||||||
}, true);
|
}, true);
|
||||||
|
// Tap handling on the focused editable. A single tap collapses any selection to a caret at the tap point
|
||||||
|
// and shows the insertion handle (native; off-window Chrome won't collapse-on-tap itself). A DOUBLE tap
|
||||||
|
// selects the word — but `click` fires before `dblclick`, so instead of guessing with timing we DEFER the
|
||||||
|
// collapse and let the real `dblclick` cancel it. This is robust to Chrome's own double-click timing
|
||||||
|
// (a timing guess raced it and sometimes ate the word selection → "cursor jumps to end of word").
|
||||||
|
var collapseTimer = null;
|
||||||
|
function clearCollapse() { if (collapseTimer) { clearTimeout(collapseTimer); collapseTimer = null; } }
|
||||||
|
document.addEventListener('dblclick', function(e){
|
||||||
|
if (!el || isCE(el)) return;
|
||||||
|
clearCollapse(); // a real double-tap → don't collapse; keep Chrome's word selection
|
||||||
|
var s = selOf(el);
|
||||||
|
if (s[0] !== s[1] && !sameSel(s, lastSel)) reportState(); // report the word only if not already sent
|
||||||
|
}, true);
|
||||||
|
document.addEventListener('click', function(e){
|
||||||
|
if (!el || isCE(el) || e.target !== el) return;
|
||||||
|
var sel = selOf(el);
|
||||||
|
if (sel[0] !== sel[1]) {
|
||||||
|
// Tap landed on a selection. Defer the collapse: if a dblclick follows (within the tap window) it
|
||||||
|
// cancels this and the word stays selected; otherwise this fires and collapses to the tapped offset.
|
||||||
|
var x = e.clientX, y = e.clientY;
|
||||||
|
clearCollapse();
|
||||||
|
collapseTimer = setTimeout(function(){
|
||||||
|
collapseTimer = null;
|
||||||
|
if (!el) return;
|
||||||
|
var s = selOf(el);
|
||||||
|
if (s[0] === s[1]) return; // already collapsed
|
||||||
|
var off = offsetFromPoint(el, x, y);
|
||||||
|
el.__nappletIme = true;
|
||||||
|
setSel(el, off, off);
|
||||||
|
el.__nappletIme = false;
|
||||||
|
lastSel = selOf(el);
|
||||||
|
reportState();
|
||||||
|
send({ type:'ime.carettap', geom: fieldGeom(el) });
|
||||||
|
}, 300);
|
||||||
|
} else {
|
||||||
|
// Tap on a bare caret → (re-)show the insertion handle. If a double-tap follows, dblclick selects the
|
||||||
|
// word and supersedes this.
|
||||||
|
send({ type:'ime.carettap', geom: fieldGeom(el) });
|
||||||
|
}
|
||||||
|
}, true);
|
||||||
|
|
||||||
function enter(n){
|
function enter(n){
|
||||||
if (!n) return;
|
if (!n) return;
|
||||||
@@ -447,7 +552,7 @@
|
|||||||
}
|
}
|
||||||
setSel(n, msg.selStart, msg.selEnd);
|
setSel(n, msg.selStart, msg.selEnd);
|
||||||
if (!composingActive && inComposition) { inComposition = false; fireComp(n, 'compositionend', d.inserted || ''); }
|
if (!composingActive && inComposition) { inComposition = false; fireComp(n, 'compositionend', d.inserted || ''); }
|
||||||
} finally { n.__nappletIme = false; lastSel = selOf(n); }
|
} finally { n.__nappletIme = false; noteSel(selOf(n)); }
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- Page (non-editable) text selection re-hosting ---
|
// --- Page (non-editable) text selection re-hosting ---
|
||||||
@@ -474,6 +579,7 @@
|
|||||||
}
|
}
|
||||||
document.addEventListener('selectionchange', function(){
|
document.addEventListener('selectionchange', function(){
|
||||||
if (el || pageReasserting) return; // selections inside an editable are handled above
|
if (el || pageReasserting) return; // selections inside an editable are handled above
|
||||||
|
lastSelActivityAt = perfNow(); // page selection moved → an imminent reveal-scroll isn't a user scroll
|
||||||
var s = window.getSelection();
|
var s = window.getSelection();
|
||||||
if (s && s.rangeCount && !s.isCollapsed) {
|
if (s && s.rangeCount && !s.isCollapsed) {
|
||||||
var r = s.getRangeAt(0);
|
var r = s.getRangeAt(0);
|
||||||
@@ -510,10 +616,100 @@
|
|||||||
} catch (_) {}
|
} catch (_) {}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Word-granularity snapping (native: dragging a word selection's handle extends a word at a time). The
|
||||||
|
// end handle snaps to the end of the word at/after the offset; the start handle to the start of the word
|
||||||
|
// at/before it. Whitespace between words extends to the adjacent word so you never stop mid-gap.
|
||||||
|
function isWordChar(c){ return c != null && /\S/.test(c); }
|
||||||
|
function wordEndAt(text, off){
|
||||||
|
var i = off;
|
||||||
|
while (i < text.length && !isWordChar(text[i])) i++;
|
||||||
|
while (i < text.length && isWordChar(text[i])) i++;
|
||||||
|
return i;
|
||||||
|
}
|
||||||
|
function wordStartAt(text, off){
|
||||||
|
var i = off;
|
||||||
|
while (i > 0 && !isWordChar(text[i - 1])) i--;
|
||||||
|
while (i > 0 && isWordChar(text[i - 1])) i--;
|
||||||
|
return i;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Host drag of an in-field selection handle: move the dragged edge to the offset under (x,y) CSS px,
|
||||||
|
// keeping the other edge anchored. Snaps to word boundaries (the selection was made by word), and is
|
||||||
|
// clamped so the dragged edge can't cross the anchor (no inversion).
|
||||||
|
function fieldExtend(edge, x, y){
|
||||||
|
if (!el || isCE(el)) return;
|
||||||
|
try {
|
||||||
|
var off = offsetFromPoint(el, x, y);
|
||||||
|
var text = valOf(el);
|
||||||
|
var sel = selOf(el);
|
||||||
|
var s, e;
|
||||||
|
if (edge === 'start') {
|
||||||
|
e = sel[1];
|
||||||
|
s = Math.min(wordStartAt(text, off), e);
|
||||||
|
} else {
|
||||||
|
s = sel[0];
|
||||||
|
e = Math.max(wordEndAt(text, off), s);
|
||||||
|
}
|
||||||
|
el.__nappletIme = true;
|
||||||
|
setSel(el, s, e);
|
||||||
|
el.__nappletIme = false;
|
||||||
|
lastSel = selOf(el);
|
||||||
|
reportState();
|
||||||
|
} catch (_) {}
|
||||||
|
}
|
||||||
|
|
||||||
|
// While the page scrolls, host-drawn selection UI (toolbar + handles) would float at stale positions, so
|
||||||
|
// the host hides it on scroll-start and we re-report fresh geometry on scroll-idle so it reappears in the
|
||||||
|
// right place — like Android. Only signal when there's a selection to hide (a field range or page text).
|
||||||
|
var scrolling = false, scrollTimer = null, autoScrolling = false;
|
||||||
|
// How long after a selection change a scroll is treated as the browser's auto-reveal of that selection
|
||||||
|
// (not a user content scroll). Generous enough to catch the reveal-scroll that fires a frame or two later.
|
||||||
|
var SCROLL_SEL_GUARD_MS = 350;
|
||||||
|
function hasSelectionUi(){ return !!pageSelText || !!(el && (function(s){ return s[0] !== s[1]; })(selOf(el))); }
|
||||||
|
function onAnyScroll(){
|
||||||
|
if (autoScrolling) return; // our own drag-to-edge auto-scroll: keep the overlays up, don't hide them
|
||||||
|
if (!hasSelectionUi()) return;
|
||||||
|
if ((perfNow() - lastSelActivityAt) < SCROLL_SEL_GUARD_MS) {
|
||||||
|
// The browser auto-scrolled to reveal a just-changed selection (forming/re-asserting a range scrolls a
|
||||||
|
// textarea). That's not a user content scroll: hiding here would blink the host overlays off-and-on every
|
||||||
|
// time a selection settles. Reposition them in place instead (geometry shifted by the reveal-scroll).
|
||||||
|
// Crucially we do NOT touch the hide-on-scroll timer: if a real scroll-hide is somehow active, these
|
||||||
|
// reveal-scrolls must not keep re-arming it (that would leave the overlays hidden indefinitely).
|
||||||
|
if (el) reportState();
|
||||||
|
else { var sr = window.getSelection(); if (sr && sr.rangeCount && !sr.isCollapsed) sendPageSel(true, sr.getRangeAt(0)); }
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (!scrolling) { scrolling = true; send({ type:'ime.scroll', active: true }); }
|
||||||
|
if (scrollTimer) clearTimeout(scrollTimer);
|
||||||
|
scrollTimer = setTimeout(function(){
|
||||||
|
scrolling = false; scrollTimer = null;
|
||||||
|
// Refresh geometry FIRST (so overlays reposition), then tell the host to show them again.
|
||||||
|
if (el) reportState();
|
||||||
|
else { var s = window.getSelection(); if (s && s.rangeCount && !s.isCollapsed) sendPageSel(true, s.getRangeAt(0)); }
|
||||||
|
send({ type:'ime.scroll', active: false });
|
||||||
|
}, 150);
|
||||||
|
}
|
||||||
|
document.addEventListener('scroll', onAnyScroll, true); // capture: any scroller, not just the document
|
||||||
|
|
||||||
window.__nappletImeHandle = function(msg){
|
window.__nappletImeHandle = function(msg){
|
||||||
if (msg.type === 'ime.set') applyState(msg);
|
if (msg.type === 'ime.set') applyState(msg);
|
||||||
else if (msg.type === 'ime.action') enter(el);
|
else if (msg.type === 'ime.action') enter(el);
|
||||||
else if (msg.type === 'ime.pageextend') pageExtend(msg.edge, msg.x, msg.y);
|
else if (msg.type === 'ime.pageextend') pageExtend(msg.edge, msg.x, msg.y);
|
||||||
|
else if (msg.type === 'ime.fieldextend') fieldExtend(msg.edge, msg.x, msg.y);
|
||||||
|
else if (msg.type === 'ime.autoscroll') {
|
||||||
|
// Host drag of a handle near the surface's top/bottom edge → scroll the content (the textarea if it
|
||||||
|
// scrolls, else the page) so the selection can keep extending, then re-report geometry so the
|
||||||
|
// overlays follow. Flagged so our own scroll doesn't trip the hide-on-scroll path above.
|
||||||
|
var dy = msg.dy || 0;
|
||||||
|
autoScrolling = true;
|
||||||
|
try {
|
||||||
|
if (el && (el.tagName || '').toUpperCase() === 'TEXTAREA') el.scrollTop += dy;
|
||||||
|
window.scrollBy(0, dy);
|
||||||
|
} catch (_) {}
|
||||||
|
if (el) reportState();
|
||||||
|
else { var s = window.getSelection(); if (s && s.rangeCount && !s.isCollapsed) sendPageSel(true, s.getRangeAt(0)); }
|
||||||
|
setTimeout(function(){ autoScrolling = false; }, 0);
|
||||||
|
}
|
||||||
else if (msg.type === 'ime.caretmove') {
|
else if (msg.type === 'ime.caretmove') {
|
||||||
if (el && !isCE(el)) {
|
if (el && !isCE(el)) {
|
||||||
var off = offsetFromPoint(el, msg.x, msg.y);
|
var off = offsetFromPoint(el, msg.x, msg.y);
|
||||||
|
|||||||
+28
@@ -77,6 +77,34 @@ object NappletBrowserContract {
|
|||||||
*/
|
*/
|
||||||
const val MSG_CONSOLE_LOG = 11
|
const val MSG_CONSOLE_LOG = 11
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Client → provider: capture a magnified slice of the live page for the native-style selection loupe.
|
||||||
|
* Host-side `PixelCopy` of the sandboxed surface returns `ERROR_SOURCE_NO_DATA` (the WebView pixels live
|
||||||
|
* in a child SurfaceControl the host never draws into), so we capture INSIDE the provider — where the
|
||||||
|
* WebView is a real in-window view. Carries [KEY_MAG_X]/[KEY_MAG_Y] (surface px center),
|
||||||
|
* [KEY_MAG_BOX_W]/[KEY_MAG_BOX_H] (source rectangle, px), [KEY_MAG_ZOOM], and [KEY_MAG_REQ_T] (the
|
||||||
|
* client's `nanoTime` send stamp, echoed back so the client can drop stale out-of-order frames).
|
||||||
|
*/
|
||||||
|
const val MSG_MAGNIFIER_REQUEST = 12
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Provider → client: the captured loupe frame. Carries [KEY_MAG_BYTES] (a PNG well under the Binder
|
||||||
|
* limit), [KEY_MAG_W]/[KEY_MAG_H], [KEY_MAG_CAPTURE_MS] (provider-side draw+encode time), and the echoed
|
||||||
|
* [KEY_MAG_REQ_T] so the client matches it to its request / drops stale frames.
|
||||||
|
*/
|
||||||
|
const val MSG_MAGNIFIER_FRAME = 13
|
||||||
|
|
||||||
|
const val KEY_MAG_X = "magX"
|
||||||
|
const val KEY_MAG_Y = "magY"
|
||||||
|
const val KEY_MAG_BOX_W = "magBoxW"
|
||||||
|
const val KEY_MAG_BOX_H = "magBoxH"
|
||||||
|
const val KEY_MAG_ZOOM = "magZoom"
|
||||||
|
const val KEY_MAG_REQ_T = "magReqT"
|
||||||
|
const val KEY_MAG_BYTES = "magBytes"
|
||||||
|
const val KEY_MAG_W = "magW"
|
||||||
|
const val KEY_MAG_H = "magH"
|
||||||
|
const val KEY_MAG_CAPTURE_MS = "magCaptureMs"
|
||||||
|
|
||||||
const val KEY_IS_LOADING = "isLoading"
|
const val KEY_IS_LOADING = "isLoading"
|
||||||
const val KEY_LOAD_FAILED = "loadFailed"
|
const val KEY_LOAD_FAILED = "loadFailed"
|
||||||
|
|
||||||
|
|||||||
+49
@@ -25,6 +25,8 @@ import android.content.ComponentName
|
|||||||
import android.content.Context
|
import android.content.Context
|
||||||
import android.content.Intent
|
import android.content.Intent
|
||||||
import android.content.ServiceConnection
|
import android.content.ServiceConnection
|
||||||
|
import android.graphics.Bitmap
|
||||||
|
import android.graphics.Canvas
|
||||||
import android.net.Uri
|
import android.net.Uri
|
||||||
import android.os.Build
|
import android.os.Build
|
||||||
import android.os.Bundle
|
import android.os.Bundle
|
||||||
@@ -33,6 +35,7 @@ import android.os.IBinder
|
|||||||
import android.os.Looper
|
import android.os.Looper
|
||||||
import android.os.Message
|
import android.os.Message
|
||||||
import android.os.Messenger
|
import android.os.Messenger
|
||||||
|
import android.os.SystemClock
|
||||||
import android.util.Log
|
import android.util.Log
|
||||||
import android.webkit.ConsoleMessage
|
import android.webkit.ConsoleMessage
|
||||||
import android.webkit.WebChromeClient
|
import android.webkit.WebChromeClient
|
||||||
@@ -51,6 +54,7 @@ import androidx.webkit.WebViewFeature
|
|||||||
import com.vitorpamplona.amethyst.commons.browser.OmniboxInput
|
import com.vitorpamplona.amethyst.commons.browser.OmniboxInput
|
||||||
import com.vitorpamplona.amethyst.commons.napplet.NappletWebContract
|
import com.vitorpamplona.amethyst.commons.napplet.NappletWebContract
|
||||||
import org.json.JSONObject
|
import org.json.JSONObject
|
||||||
|
import java.io.ByteArrayOutputStream
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Provider for the **embedded** in-app browser. Runs in the keyless `:napplet` process: it hosts the
|
* Provider for the **embedded** in-app browser. Runs in the keyless `:napplet` process: it hosts the
|
||||||
@@ -190,11 +194,56 @@ class NappletBrowserService : Service() {
|
|||||||
// reload the one the user toggled.
|
// reload the one the user toggled.
|
||||||
applyWebViewProxy(if (tab.useTor) tab.proxyPort else -1) { tab.webView?.reload() }
|
applyWebViewProxy(if (tab.useTor) tab.proxyPort else -1) { tab.webView?.reload() }
|
||||||
}
|
}
|
||||||
|
NappletBrowserContract.MSG_MAGNIFIER_REQUEST -> onMagnifierRequest(msg)
|
||||||
else -> return false
|
else -> return false
|
||||||
}
|
}
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// One reusable output bitmap per tab would be ideal, but loupe size is fixed per drag; createBitmap each
|
||||||
|
// frame is cheap next to the draw. Source rect is in view px (== surface px, the SCVH is 1:1).
|
||||||
|
private fun onMagnifierRequest(msg: Message) {
|
||||||
|
val tab = tabFor(msg) ?: return
|
||||||
|
val wv = tab.webView ?: return
|
||||||
|
val data = msg.data ?: return
|
||||||
|
val cx = data.getFloat(NappletBrowserContract.KEY_MAG_X)
|
||||||
|
val cy = data.getFloat(NappletBrowserContract.KEY_MAG_Y)
|
||||||
|
val boxW = data.getInt(NappletBrowserContract.KEY_MAG_BOX_W, 150).coerceIn(16, 1024)
|
||||||
|
val boxH = data.getInt(NappletBrowserContract.KEY_MAG_BOX_H, 84).coerceIn(16, 1024)
|
||||||
|
val zoom = data.getFloat(NappletBrowserContract.KEY_MAG_ZOOM, 1.6f).coerceIn(1f, 4f)
|
||||||
|
val reqT = data.getLong(NappletBrowserContract.KEY_MAG_REQ_T)
|
||||||
|
|
||||||
|
val outW = (boxW * zoom).toInt().coerceAtLeast(1)
|
||||||
|
val outH = (boxH * zoom).toInt().coerceAtLeast(1)
|
||||||
|
val t0 = SystemClock.elapsedRealtimeNanos()
|
||||||
|
val bitmap = Bitmap.createBitmap(outW, outH, Bitmap.Config.ARGB_8888)
|
||||||
|
val canvas = Canvas(bitmap)
|
||||||
|
canvas.drawColor(tab.bgColor)
|
||||||
|
// Map the source rect (centered on cx,cy in view px) into the zoomed output bitmap.
|
||||||
|
canvas.scale(zoom, zoom)
|
||||||
|
canvas.translate(-(cx - boxW / 2f), -(cy - boxH / 2f))
|
||||||
|
wv.draw(canvas)
|
||||||
|
|
||||||
|
val baos = ByteArrayOutputStream()
|
||||||
|
bitmap.compress(Bitmap.CompressFormat.PNG, 100, baos)
|
||||||
|
val bytes = baos.toByteArray()
|
||||||
|
bitmap.recycle()
|
||||||
|
val captureMs = (SystemClock.elapsedRealtimeNanos() - t0) / 1_000_000.0
|
||||||
|
|
||||||
|
val reply =
|
||||||
|
Message.obtain(null, NappletBrowserContract.MSG_MAGNIFIER_FRAME).apply {
|
||||||
|
this.data =
|
||||||
|
Bundle().apply {
|
||||||
|
putByteArray(NappletBrowserContract.KEY_MAG_BYTES, bytes)
|
||||||
|
putInt(NappletBrowserContract.KEY_MAG_W, outW)
|
||||||
|
putInt(NappletBrowserContract.KEY_MAG_H, outH)
|
||||||
|
putDouble(NappletBrowserContract.KEY_MAG_CAPTURE_MS, captureMs)
|
||||||
|
putLong(NappletBrowserContract.KEY_MAG_REQ_T, reqT)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
runCatching { tab.clientMessenger?.send(reply) }
|
||||||
|
}
|
||||||
|
|
||||||
/** Builds the SandboxedUiAdapter for [tab] and ships its cross-process handle (coreLibInfo) to the client. */
|
/** Builds the SandboxedUiAdapter for [tab] and ships its cross-process handle (coreLibInfo) to the client. */
|
||||||
private fun replyWithAdapter(tab: BrowserTab) {
|
private fun replyWithAdapter(tab: BrowserTab) {
|
||||||
val adapter = NappletBrowserUiAdapter(this, tab.sessionId)
|
val adapter = NappletBrowserUiAdapter(this, tab.sessionId)
|
||||||
|
|||||||
+23
@@ -84,6 +84,29 @@ object NappletEmbedContract {
|
|||||||
*/
|
*/
|
||||||
const val MSG_LOAD_STATE = 15
|
const val MSG_LOAD_STATE = 15
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Client → provider: capture a magnified slice of the live page for the selection loupe. Host-side
|
||||||
|
* `PixelCopy` of the sandbox surface returns `ERROR_SOURCE_NO_DATA` (the WebView pixels live in a child
|
||||||
|
* SurfaceControl), so capture happens here, in the provider, where the WebView is a real in-window view.
|
||||||
|
* Carries [KEY_MAG_X]/[KEY_MAG_Y] (surface px center), [KEY_MAG_BOX_W]/[KEY_MAG_BOX_H] (source rect, px),
|
||||||
|
* [KEY_MAG_ZOOM], and [KEY_MAG_REQ_T] (echoed send stamp). Mirrors the browser path.
|
||||||
|
*/
|
||||||
|
const val MSG_MAGNIFIER_REQUEST = 16
|
||||||
|
|
||||||
|
/** Provider → client: the captured loupe frame — [KEY_MAG_BYTES] PNG, [KEY_MAG_W]/[KEY_MAG_H], [KEY_MAG_CAPTURE_MS], echoed [KEY_MAG_REQ_T]. */
|
||||||
|
const val MSG_MAGNIFIER_FRAME = 17
|
||||||
|
|
||||||
|
const val KEY_MAG_X = "magX"
|
||||||
|
const val KEY_MAG_Y = "magY"
|
||||||
|
const val KEY_MAG_BOX_W = "magBoxW"
|
||||||
|
const val KEY_MAG_BOX_H = "magBoxH"
|
||||||
|
const val KEY_MAG_ZOOM = "magZoom"
|
||||||
|
const val KEY_MAG_REQ_T = "magReqT"
|
||||||
|
const val KEY_MAG_BYTES = "magBytes"
|
||||||
|
const val KEY_MAG_W = "magW"
|
||||||
|
const val KEY_MAG_H = "magH"
|
||||||
|
const val KEY_MAG_CAPTURE_MS = "magCaptureMs"
|
||||||
|
|
||||||
const val KEY_CORE_LIB_INFO = "coreLibInfo"
|
const val KEY_CORE_LIB_INFO = "coreLibInfo"
|
||||||
const val KEY_CAN_GO_BACK = "canGoBack"
|
const val KEY_CAN_GO_BACK = "canGoBack"
|
||||||
const val KEY_IS_LOADING = "isLoading"
|
const val KEY_IS_LOADING = "isLoading"
|
||||||
|
|||||||
+51
@@ -25,6 +25,8 @@ import android.content.ComponentName
|
|||||||
import android.content.Context
|
import android.content.Context
|
||||||
import android.content.Intent
|
import android.content.Intent
|
||||||
import android.content.ServiceConnection
|
import android.content.ServiceConnection
|
||||||
|
import android.graphics.Bitmap
|
||||||
|
import android.graphics.Canvas
|
||||||
import android.net.Uri
|
import android.net.Uri
|
||||||
import android.os.Build
|
import android.os.Build
|
||||||
import android.os.Bundle
|
import android.os.Bundle
|
||||||
@@ -33,6 +35,7 @@ import android.os.IBinder
|
|||||||
import android.os.Looper
|
import android.os.Looper
|
||||||
import android.os.Message
|
import android.os.Message
|
||||||
import android.os.Messenger
|
import android.os.Messenger
|
||||||
|
import android.os.SystemClock
|
||||||
import android.util.Log
|
import android.util.Log
|
||||||
import android.view.View
|
import android.view.View
|
||||||
import android.webkit.WebResourceError
|
import android.webkit.WebResourceError
|
||||||
@@ -57,6 +60,7 @@ import com.vitorpamplona.quartz.nip01Core.core.toHexKey
|
|||||||
import com.vitorpamplona.quartz.nip5aStaticWebsites.tags.PathTag
|
import com.vitorpamplona.quartz.nip5aStaticWebsites.tags.PathTag
|
||||||
import com.vitorpamplona.quartz.utils.sha256.sha256
|
import com.vitorpamplona.quartz.utils.sha256.sha256
|
||||||
import org.json.JSONObject
|
import org.json.JSONObject
|
||||||
|
import java.io.ByteArrayOutputStream
|
||||||
import java.util.concurrent.Executor
|
import java.util.concurrent.Executor
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -183,6 +187,7 @@ class NappletHostService : Service() {
|
|||||||
val payload = msg.data?.getString(NappletEmbedContract.KEY_IME_PAYLOAD) ?: return true
|
val payload = msg.data?.getString(NappletEmbedContract.KEY_IME_PAYLOAD) ?: return true
|
||||||
tab.bridgeReplyProxy?.postMessage(payload)
|
tab.bridgeReplyProxy?.postMessage(payload)
|
||||||
}
|
}
|
||||||
|
NappletEmbedContract.MSG_MAGNIFIER_REQUEST -> onMagnifierRequest(msg)
|
||||||
else -> return false
|
else -> return false
|
||||||
}
|
}
|
||||||
return true
|
return true
|
||||||
@@ -229,6 +234,52 @@ class NappletHostService : Service() {
|
|||||||
return tab
|
return tab
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Draw a zoomed slice of the live WebView (the loupe content) and ship it back as a PNG. The WebView is a
|
||||||
|
* real in-window view here in the provider, so its software draw renders real DOM pixels — unlike host-side
|
||||||
|
* `PixelCopy` of the sandbox surface. Mirror of the browser path. Runs on the main looper (`WebView.draw`).
|
||||||
|
*/
|
||||||
|
private fun onMagnifierRequest(msg: Message) {
|
||||||
|
val tab = tabFor(msg) ?: return
|
||||||
|
val wv = tab.webView ?: return
|
||||||
|
val data = msg.data ?: return
|
||||||
|
val cx = data.getFloat(NappletEmbedContract.KEY_MAG_X)
|
||||||
|
val cy = data.getFloat(NappletEmbedContract.KEY_MAG_Y)
|
||||||
|
val boxW = data.getInt(NappletEmbedContract.KEY_MAG_BOX_W, 150).coerceIn(16, 1024)
|
||||||
|
val boxH = data.getInt(NappletEmbedContract.KEY_MAG_BOX_H, 84).coerceIn(16, 1024)
|
||||||
|
val zoom = data.getFloat(NappletEmbedContract.KEY_MAG_ZOOM, 1.6f).coerceIn(1f, 4f)
|
||||||
|
val reqT = data.getLong(NappletEmbedContract.KEY_MAG_REQ_T)
|
||||||
|
|
||||||
|
val outW = (boxW * zoom).toInt().coerceAtLeast(1)
|
||||||
|
val outH = (boxH * zoom).toInt().coerceAtLeast(1)
|
||||||
|
val t0 = SystemClock.elapsedRealtimeNanos()
|
||||||
|
val bitmap = Bitmap.createBitmap(outW, outH, Bitmap.Config.ARGB_8888)
|
||||||
|
val canvas = Canvas(bitmap)
|
||||||
|
canvas.drawColor(tab.bgColor)
|
||||||
|
canvas.scale(zoom, zoom)
|
||||||
|
canvas.translate(-(cx - boxW / 2f), -(cy - boxH / 2f))
|
||||||
|
wv.draw(canvas)
|
||||||
|
|
||||||
|
val baos = ByteArrayOutputStream()
|
||||||
|
bitmap.compress(Bitmap.CompressFormat.PNG, 100, baos)
|
||||||
|
val bytes = baos.toByteArray()
|
||||||
|
bitmap.recycle()
|
||||||
|
val captureMs = (SystemClock.elapsedRealtimeNanos() - t0) / 1_000_000.0
|
||||||
|
|
||||||
|
val reply =
|
||||||
|
Message.obtain(null, NappletEmbedContract.MSG_MAGNIFIER_FRAME).apply {
|
||||||
|
this.data =
|
||||||
|
Bundle().apply {
|
||||||
|
putByteArray(NappletEmbedContract.KEY_MAG_BYTES, bytes)
|
||||||
|
putInt(NappletEmbedContract.KEY_MAG_W, outW)
|
||||||
|
putInt(NappletEmbedContract.KEY_MAG_H, outH)
|
||||||
|
putDouble(NappletEmbedContract.KEY_MAG_CAPTURE_MS, captureMs)
|
||||||
|
putLong(NappletEmbedContract.KEY_MAG_REQ_T, reqT)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
runCatching { tab.clientMessenger?.send(reply) }
|
||||||
|
}
|
||||||
|
|
||||||
/** Builds the SandboxedUiAdapter for [tab] and ships its cross-process handle (coreLibInfo) to the client. */
|
/** Builds the SandboxedUiAdapter for [tab] and ships its cross-process handle (coreLibInfo) to the client. */
|
||||||
private fun replyWithAdapter(tab: NappletTab) {
|
private fun replyWithAdapter(tab: NappletTab) {
|
||||||
val adapter = NappletHostUiAdapter(this, tab.sessionId)
|
val adapter = NappletHostUiAdapter(this, tab.sessionId)
|
||||||
|
|||||||
@@ -16,6 +16,8 @@
|
|||||||
<h2>Textarea</h2>
|
<h2>Textarea</h2>
|
||||||
<textarea id="ta" rows="2" placeholder="type here">hello world</textarea>
|
<textarea id="ta" rows="2" placeholder="type here">hello world</textarea>
|
||||||
<div id="log"></div>
|
<div id="log"></div>
|
||||||
|
<!-- Tall spacer so the page scrolls — lets us verify the host hides selection UI during scroll. -->
|
||||||
|
<div style="height:2000px; padding-top:16px; color:#888;">scroll region — drag up/down here</div>
|
||||||
<script>
|
<script>
|
||||||
(function(){
|
(function(){
|
||||||
var t0 = performance.now();
|
var t0 = performance.now();
|
||||||
|
|||||||
Reference in New Issue
Block a user