mirror of
https://github.com/vitorpamplona/amethyst.git
synced 2026-10-05 11:18:24 +00:00
feat(embed): selection loupe + native-parity fixes for embedded text selection
Builds out host-drawn text selection for embedded napplet/nsite/browser surfaces toward native parity, and fixes the bugs found while exercising it. - Magnifier loupe (#4): EmbeddedMagnifier + provider-side pixel capture (EmbeddedMagnifierProbe) shipped over IPC for both embed paths; the caret/selection handles drive it via OnMagnify. - SelectionUiState: single source of truth for the overlay show/hide rules (insertion caret / in-field range / page-text range + dragging/scrolling). - EmbeddedSelectionDrag: suspends the nav drawer's edge swipe while a handle is dragged (auto-scroll #9). Bug fixes: - No more overlay blink on word-select: the shim's selection-reveal scrolls (a textarea auto-scrolling to show a forming/re-asserted range) no longer trip the hide-on-scroll path, and the hide self-heals instead of being re-armed indefinitely. - RemoteImeView debounces the range-lost signal so a transient collapse that gets re-asserted doesn't flicker the handles/toolbar. - Focusing a field clears any page-text selection (shim + host), so the stale page handles/Copy bar no longer linger above — and stop stealing drags from — the field overlays; also cancels any in-flight scroll-hide on focus. - Caret insertion-handle drag now actually moves the caret: read the pointer delta with positionChangeIgnoreConsumed() before consuming, so the value isn't zeroed by our own consume (or the sandbox surface consuming the move). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Vitor Pamplona
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
8232cd50a3
commit
dec1e0bec1
@@ -31,18 +31,18 @@ rules, so we can check off coverage. Native impl lives in `android.widget.Editor
|
||||
|
||||
| # | Native feature | Activates | Deactivates | Our status |
|
||||
|---|----------------|-----------|-------------|------------|
|
||||
| 1 | **Insertion handle** (the teardrop "blob" under the caret) | tap in editable text; tap again to re-show | typing, scroll start, focus loss, ~4s inactivity timeout | ✅ `InsertionHandle` — shows on caret geometry, hides via `onEdited`. **Missing the inactivity timeout and the re-show-on-tap.** |
|
||||
| 2 | **Selection handles** (asymmetric left/right teardrops) | long-press word, double-tap word, drag-extend | tap-collapse, typing, new selection | ✅ `SelectionHandle(isStart)` + drag-to-extend via `pageExtend`/caretmove |
|
||||
| 3 | **Floating toolbar** (Cut/Copy/Paste/Select-All/Share/…) | selection made, or tap insertion handle (Paste/Select-All) | scroll/fling (hides, returns on settle), handle drag (hides), tap-collapse | ⚠️ `EmbeddedSelectionToolbar` has Cut/Copy/Paste/Select-All only. **No hide-during-drag, no hide-during-scroll, no insertion-handle Paste popup, no overflow, no Share/Web-Search/process-text.** |
|
||||
| 4 | **Magnifier / loupe** (the zoom bubble above the finger while dragging a handle or the caret) | finger down + moving on a handle or the caret | finger up | ❌ **Not built.** This is the "zoom bubble" called out — highest-value missing piece. |
|
||||
| 5 | **Word-granularity long-press** then char-extend | long-press | — | ⚠️ partial — selection works; granularity (word-snap while dragging, then char) not modelled |
|
||||
| 6 | **Double-tap = word, long-press = word, (triple-tap/drag = paragraph)** | tap count | — | ❌ double-tap-to-select not wired (only long-press) |
|
||||
| 1 | **Insertion handle** (the teardrop "blob" under the caret) | tap in editable text; tap again to re-show | typing, scroll start, focus loss, ~4s inactivity timeout | ✅ `InsertionHandle`. **Native availability rule now matched (2026-06-25):** only shown when the field is NON-EMPTY (`Editor` gates the handle behind `text.length() > 0`, via `SelectionUiState.fieldHasText`) — fixes it popping up on focus of an empty box; hides on typing (`onEdited`), scroll (`scrolling`), focus loss, and ~4s inactivity (`hideCaret` timeout), re-showing on the next tap — via an explicit `ime.carettap` shim signal (DOM `click`), so a tap that doesn't move the caret still re-shows it. Device-verified. |
|
||||
| 2 | **Selection handles** (asymmetric left/right teardrops) | long-press word, double-tap word, drag-extend | tap-collapse, typing, new selection | ✅ `SelectionHandle(isStart)` + drag-to-extend, for BOTH plain page text (`pageExtend`) AND in-field `<input>`/`<textarea>` selections (`fieldExtend`, 2026-06-25). The shim reports the selection's caret feet (`sx/sb`,`ex/eb`, flagged `rng`) via the same mirror-div as the caret; the host holds the range geometry separately so Chrome's transient collapse-to-caret (the re-assert fight) doesn't yank the handles to the field edges. **Tap-to-collapse (2026-06-25):** a single tap inside a selection dismisses it to a caret at the tapped offset + insertion handle — the shim's `click` handler collapses explicitly via `offsetFromPoint` (off-window Chrome doesn't do it itself). Device-verified. |
|
||||
| 3 | **Floating toolbar** (Cut/Copy/Paste/Select-All/Share/…) | selection made, or tap insertion handle (Paste/Select-All) | scroll/fling (hides, returns on settle), handle drag (hides), tap-collapse | ⚠️ `EmbeddedSelectionToolbar` (Cut/Copy/Paste/Select-All). **Hide-during-handle-drag ✅ device-verified; hide-during-scroll ✅ implemented (2026-06-25, via `SelectionUiState.dragging`/`scrolling`; shim sends `ime.scroll` + re-reports geometry on settle) — same gate as drag-hide, but not yet exercised on device (the embedded WebView didn't scroll under injected swipes / cached short harness).** Still missing: insertion-handle Paste popup, overflow, Share/Web-Search/process-text. |
|
||||
| 4 | **Magnifier / loupe** (the zoom bubble above the finger while dragging a handle or the caret) | finger down + moving on a handle or the caret | finger up | ✅ **Built (2026-06-25).** `Magnifier` bubble in [EmbeddedMagnifier.kt] follows the dragged caret/selection handle, showing live magnified page pixels captured in the `:napplet` provider and shipped over IPC ([EmbeddedMagnifierProbe], option B). Both embed paths wired (browser + napplet); verified on device for the browser path. Capture Y is locked to the caret/selection line (X follows the finger). Possible further polish: RGB_565 to cut encode, themed crosshair, clamp capture X to the line so a fast drag past EOL doesn't show blank. |
|
||||
| 5 | **Word-granularity long-press** then char-extend | long-press | — | ✅ in-field handle-extend snaps to word boundaries (2026-06-25): `fieldExtend` snaps the dragged edge via `wordStartAt`/`wordEndAt` (end→word end, start→word start), so dragging a word selection's handle extends a word at a time. Device-verified (drag end handle slightly into "world" → whole "hello world"). Page-text extend stays char (no offset model); the "then char" fine mode within a word isn't modeled. |
|
||||
| 6 | **Double-tap = word, long-press = word, (triple-tap/drag = paragraph)** | tap count | — | ✅ double-tap + long-press both select a word (2026-06-25). Chrome word-selects on the 2nd tap, then abandons it by collapsing to the end (off-window quirk); the host re-assert restores it. The shim `click` handler DEFERS its tap-to-collapse ~300ms and the real `dblclick` cancels that timer, so the word selection survives (a timing-only guard was flaky ~40%). Triple-tap/paragraph not done. |
|
||||
| 7 | **Smart selection / entity expansion** (`TextClassifier`: phone, URL, address, date → entity actions in toolbar) | selection lands on an entity | — | ❌ not built (low priority) |
|
||||
| 8 | **Drag selected text** (long-press a selection → drag-and-drop to move) | long-press on existing selection | drop | ❌ not built (low priority) |
|
||||
| 9 | **Auto-scroll while dragging to a viewport edge** | handle dragged near top/bottom edge | finger leaves edge / up | ❌ not built — needed once fields can be off-screen/long |
|
||||
| 9 | **Auto-scroll while dragging to a viewport edge** | handle dragged near top/bottom edge | finger leaves edge / up | ✅ works (2026-06-25, user-confirmed). The drag driver (`onMagnify`) detects the finger in the surface's top/bottom edge zone and sends `ime.autoscroll`; the shim scrolls the textarea (else the window) and re-reports geometry, flagged so the hide-on-scroll path doesn't fire. Scrolls per drag-move in the edge zone (not on a perfectly-held finger). **Fixed alongside:** the nav drawer's left-edge swipe was hijacking the edge drag — `EmbeddedSelectionDrag.dragging` (set by `onMagnify`) now suspends the drawer's `gesturesEnabled` while a handle is dragged. |
|
||||
| 10 | **Caret snapping to character boundaries** | always during caret/handle drag | — | ✅ via `offsetFromPoint` binary search + Y-clamp |
|
||||
| 11 | **Themed handle/caret drawables + blink** | always | — | ⚠️ teardrop shape matches; color/blink not themed to `textSelectHandle`/`textCursorDrawable` |
|
||||
| 12 | **Insertion-handle Paste/Select-All mini-popup** | tap the insertion handle | tap elsewhere | ❌ not built |
|
||||
| 11 | **Themed handle/caret drawables + blink** | always | — | ✅/⚠️ The host-drawn handles use `colorScheme.primary` — which IS the native `textSelectHandle`/accent color — so the handle drawables are themed (the actionable part). The caret bar + selection-highlight are drawn by Chrome inside the off-window surface: the caret already blinks natively, and theming its color/the highlight would mean injecting CSS into arbitrary third-party pages (intrusive; `::selection` was already ruled out as non-painting), so those are intentionally left to Chrome. |
|
||||
| 12 | **Insertion-handle Paste/Select-All mini-popup** | tap the insertion handle | tap elsewhere | ✅ built (2026-06-25). Tapping the bare insertion handle toggles a Paste/Select-All bar above the caret (`SelectionUiState.insertionPopup`, toggled from the handle's unified tap/drag gesture); tap-elsewhere/typing/blur/selection/scroll dismiss it. Fixed a latent bug: toolbar items now consume the *down* (not just the up) so the tap doesn't bleed through to the surface and blur the field. Device-verified (Select-all selects all text, field stays focused). |
|
||||
|
||||
Legend: ✅ done · ⚠️ partial · ❌ missing.
|
||||
|
||||
@@ -50,30 +50,90 @@ Legend: ✅ done · ⚠️ partial · ❌ missing.
|
||||
|
||||
Most of the bugs we already fixed were activation-timing bugs (cursor-jumps-to-end,
|
||||
collapse-on-tap, focus-transfer races). The remaining features each carry their
|
||||
own state machine. Before adding more, we should **centralize selection-UI state**
|
||||
in one place (today it's spread across `showInsertionHandle`, `showSelectionToolbar`,
|
||||
page-vs-field flags in `EmbeddedTabLayer`). A single `SelectionUiState`
|
||||
(none / insertion / range / dragging-handle / scrolling) makes the hide/show rules
|
||||
(toolbar hides while dragging, magnifier shows only while dragging, handle
|
||||
auto-times-out) expressible instead of emergent.
|
||||
own state machine.
|
||||
|
||||
**Done (2026-06-25): `SelectionUiState`** (`SelectionUiState.kt`) now centralizes
|
||||
what used to be scattered flags in `EmbeddedTabLayer` (`showInsertionHandle`,
|
||||
`showSelectionToolbar`, `fieldGeometry`, `rangeFieldGeometry`, `pageSelection`).
|
||||
It holds the three mutually-exclusive contexts (insertion caret / in-field range /
|
||||
page-text range) plus the transient modifiers `dragging` and `scrolling`, and
|
||||
exposes derived visibility (`insertionHandle`, `fieldHandles`, `fieldToolbar`,
|
||||
`pageHandles`, `pageToolbar`) so the rules are expressed once:
|
||||
- **toolbar hides while a handle is dragged** (`dragging`, set from the same
|
||||
`OnMagnify` lifecycle that drives the loupe) — the dragged handle also hides its
|
||||
own teardrop (the loupe stands in), like Android; the other handle stays.
|
||||
- **all overlays hide while scrolling** (`scrolling`, from the shim's `ime.scroll`);
|
||||
the shim re-reports geometry just before `active=false` so they reappear
|
||||
repositioned.
|
||||
Still emergent / TODO: insertion-handle auto-timeout, tap-to-re-show.
|
||||
|
||||
**Selection-blink fix (2026-06-25).** A field selection — especially in a `<textarea>` —
|
||||
flickered: off-window Chrome abandons the selection by collapsing the caret to an
|
||||
endpoint every ~25 ms, and the FIELD re-assert round-tripped through the host EditText
|
||||
(`RemoteImeView.onPageState` → `ime.set` → page), leaving a visible collapsed frame each
|
||||
cycle. Fix: re-assert field selections **synchronously in the shim's `selectionchange`
|
||||
handler** (mirror of the page-text path that never blinked) — `lastFieldRange`/`lastFieldAt`
|
||||
tracked via `noteSel()`, and a collapse-to-endpoint within 1500 ms is reverted with `setSel`
|
||||
(guarded, not re-reported) so it reverts before paint. The host re-assert stays as a
|
||||
fallback. Device-verified: textarea selection is stable; input select still works.
|
||||
|
||||
## Priority order for parity work
|
||||
|
||||
1. **Magnifier (#4).** Biggest perceived gap. We already report caret/handle
|
||||
geometry; the magnifier needs a *magnified pixel view of the surface* at the
|
||||
drag point. Options:
|
||||
- **A. Compose-side zoom of a surface snapshot.** The surface is a
|
||||
`SurfaceControlViewHost` — we can't trivially `Bitmap`-grab a remote
|
||||
surface from the main process. Would need `PixelCopy` against the
|
||||
`SurfaceView`/`SurfaceControl`, then draw a clipped, scaled bitmap in a
|
||||
popup that follows X and stays clamped to the dragged line's Y (native
|
||||
behavior). Verify `PixelCopy` works against the sandboxed surface before
|
||||
committing to this.
|
||||
- **B. Render the magnifier in `:napplet`.** Have `shim.js`/the host WebView
|
||||
produce the loupe content (a zoomed DOM render) and ship it. Heavier, but
|
||||
sidesteps cross-process pixel capture.
|
||||
- Start by spiking A with `PixelCopy.request(SurfaceView, …)`; if it returns
|
||||
a valid bitmap for our surface, A is far simpler.
|
||||
- **A. Compose-side zoom of a surface snapshot. ❌ RULED OUT (spiked 2026-06-25).**
|
||||
The surface is a `SurfaceControlViewHost` — we can't trivially `Bitmap`-grab a
|
||||
remote surface from the main process. We spiked `PixelCopy.request(SurfaceView, …)`
|
||||
against the live embedded surface (`SurfaceMagnifierProbe`, wired into
|
||||
`EmbeddedTabLayer` behind `BuildConfig.DEBUG`, fired on field focus). The capture
|
||||
target is the privacysandbox `ContentView extends SurfaceView` — the only real
|
||||
child of `SandboxedSdkView` once the session opens. **Result on a clearly-painted
|
||||
surface (1080×2088): every capture returns `ERROR_SOURCE_NO_DATA`** (center
|
||||
region, repeated 3×). Cause: the WebView pixels live in a *child* `SurfaceControl`
|
||||
reparented under the SurfaceView via `ContentView.setChildSurfacePackage(...)`; the
|
||||
host SurfaceView's *own* buffer is never drawn into, so `PixelCopy` on the parent
|
||||
reads an empty buffer. Host-side pixel capture of the sandboxed content is not
|
||||
available. (A `PixelCopy.request(Window, …)` against the host window would also
|
||||
miss it — the sandbox layer is a *separate* SurfaceControl z-ordered below the
|
||||
window.)
|
||||
- **B. Capture in `:napplet` and ship the loupe content. ✅ SPIKED & VIABLE
|
||||
(2026-06-25).** Inside the keyless provider the WebView IS a real in-window view, so
|
||||
`WebView.draw(Canvas)` into a software bitmap renders real DOM pixels. Spike added
|
||||
`MSG_MAGNIFIER_REQUEST`/`MSG_MAGNIFIER_FRAME` to `NappletBrowserContract`:
|
||||
`NappletBrowserService.onMagnifierRequest` draws a zoomed slice
|
||||
(`canvas.scale(zoom); translate(-(cx-box/2), -(cy-box/2)); webView.draw(canvas)`),
|
||||
PNG-encodes it, and ships the bytes back; `EmbeddedBrowserController` (now also an
|
||||
`EmbeddedMagnifierProbe`) requests on focus and `EmbeddedTabLayer` logs the result.
|
||||
**10-frame burst, 160px source × 1.5× zoom → 240×240 PNG, center `#FF111111`
|
||||
(real opaque content):** provider draw 0.7–1.2 ms steady (≈5 ms cold), provider
|
||||
total draw+PNG 3–4 ms steady (≈12 ms cold), client round-trip 4–8 ms steady
|
||||
(occasional ~18 ms), payload 8–15 KB (far under the 1 MB Binder limit). Comfortably
|
||||
within a frame budget if throttled to ~30 fps.
|
||||
|
||||
**✅ Real loupe shipped (2026-06-25).** `MagnifierUiState` + `Magnifier`
|
||||
(`EmbeddedMagnifier.kt`); the caret/selection handles call an `OnMagnify` callback
|
||||
on drag start/move/end; `EmbeddedTabLayer` tracks the drag point, throttles capture
|
||||
requests to one in flight (100 ms timeout), decodes each `MagnifierFrame` to an
|
||||
`ImageBitmap`, and floats the bubble above the finger (clamped, flips below near the
|
||||
top). Capture mirrored onto BOTH embed paths (browser:
|
||||
`NappletBrowserContract`/`NappletBrowserService`/`EmbeddedBrowserController`;
|
||||
napplet: `NappletEmbedContract`/`NappletHostService`/`EmbeddedNappletController`).
|
||||
Verified on device (browser): drag → bubble shows live magnified "ello world" with
|
||||
the caret, centered on the line → follows finger → hides on release. The dead
|
||||
Option-A probe (`SurfaceMagnifierProbe`) was removed. **Polish done (2026-06-25):**
|
||||
capture Y is locked to the authoritative caret/selection line (the handles pass a
|
||||
`lineHalfPx` so the box centers on the line, not the finger or the caret foot); X
|
||||
still follows the finger. Remaining nice-to-haves: RGB_565/raw to cut PNG encode,
|
||||
themed crosshair, clamp capture X to the line so a fast drag past EOL isn't blank,
|
||||
reuse one off-screen bitmap.
|
||||
- **Gesture-routing caveat (found during the spike).** The host-drawn handles'
|
||||
drag is fragile: the sandbox `ContentView.onTouchEvent` always returns `true`, so
|
||||
via Compose's `AndroidView` interop it consumes the drag-move pointer and cancels
|
||||
the overlay handle's `detectDragGestures` (synthetic `adb` drags on the handle
|
||||
never produced an `onDrag`). The magnifier trigger should ride the existing
|
||||
caret/selection-move path (which already round-trips through the shim), not a fresh
|
||||
Compose drag layered over the surface.
|
||||
2. **Toolbar state rules (#3 hide-during-drag/scroll) + insertion-handle popup
|
||||
(#12).** Pure Compose/state work, no platform unknowns. Do alongside the
|
||||
`SelectionUiState` refactor.
|
||||
|
||||
+9
-3
@@ -44,6 +44,7 @@ import com.vitorpamplona.amethyst.ui.navigation.drawer.DrawerContent
|
||||
import com.vitorpamplona.amethyst.ui.navigation.navs.Nav
|
||||
import com.vitorpamplona.amethyst.ui.navigation.routes.Route
|
||||
import com.vitorpamplona.amethyst.ui.screen.AccountSessionManager
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedSelectionDrag
|
||||
import kotlinx.coroutines.launch
|
||||
|
||||
@OptIn(ExperimentalMaterial3Api::class)
|
||||
@@ -90,9 +91,14 @@ fun AccountSwitcherAndLeftDrawerLayout(
|
||||
dest.hasRoute<Route.Home>() || dest.hasRoute<Route.Message>()
|
||||
} ?: false
|
||||
val drawerGesturesEnabled =
|
||||
!isTabPagerRoute ||
|
||||
nav.drawerState.isOpen ||
|
||||
nav.drawerState.targetValue != nav.drawerState.currentValue
|
||||
(
|
||||
!isTabPagerRoute ||
|
||||
nav.drawerState.isOpen ||
|
||||
nav.drawerState.targetValue != nav.drawerState.currentValue
|
||||
) &&
|
||||
// Suspend the left-edge swipe while a selection/caret handle is dragged over an embedded surface,
|
||||
// so a handle drag near the left edge (or the auto-scroll edge drag) doesn't open the drawer.
|
||||
!EmbeddedSelectionDrag.dragging
|
||||
|
||||
ModalNavigationDrawer(
|
||||
drawerState = nav.drawerState,
|
||||
|
||||
+39
@@ -31,6 +31,7 @@ import android.os.IBinder
|
||||
import android.os.Looper
|
||||
import android.os.Message
|
||||
import android.os.Messenger
|
||||
import android.os.SystemClock
|
||||
import androidx.annotation.RequiresApi
|
||||
import androidx.compose.runtime.mutableStateListOf
|
||||
import androidx.privacysandbox.ui.client.SandboxedUiAdapterFactory
|
||||
@@ -41,8 +42,10 @@ import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.ConsoleBridge
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.ConsoleLogEntry
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedImeBridge
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedLoadStatus
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedMagnifierProbe
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedSurfaceController
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.ImeEvent
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.MagnifierFrame
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.parseSelectionGeometry
|
||||
import org.json.JSONObject
|
||||
import java.util.concurrent.atomic.AtomicLong
|
||||
@@ -62,6 +65,7 @@ class EmbeddedWebAppController(
|
||||
private val themeType: String = "SYSTEM",
|
||||
) : EmbeddedSurfaceController,
|
||||
EmbeddedImeBridge,
|
||||
EmbeddedMagnifierProbe,
|
||||
ConsoleBridge {
|
||||
private val incoming = Messenger(Handler(Looper.getMainLooper(), ::onServiceMessage))
|
||||
private var serviceMessenger: Messenger? = null
|
||||
@@ -95,6 +99,9 @@ class EmbeddedWebAppController(
|
||||
|
||||
override var onImeEvent: ((ImeEvent) -> Unit)? = null
|
||||
|
||||
// SPIKE (magnifier #4, option B): provider-side capture round-trip. Removed once the loupe lands.
|
||||
override var onMagnifierFrame: ((MagnifierFrame) -> Unit)? = null
|
||||
|
||||
private val connection =
|
||||
object : ServiceConnection {
|
||||
override fun onServiceConnected(
|
||||
@@ -127,6 +134,7 @@ class EmbeddedWebAppController(
|
||||
pendingAdapter = null
|
||||
onUrlChanged = null
|
||||
onImeEvent = null
|
||||
onMagnifierFrame = null
|
||||
onLoadStatusChanged = null
|
||||
consoleLogs.clear()
|
||||
}
|
||||
@@ -193,6 +201,19 @@ class EmbeddedWebAppController(
|
||||
if (consoleLogs.size >= MAX_CONSOLE_LOGS) consoleLogs.removeAt(0)
|
||||
consoleLogs.add(ConsoleLogEntry(level, message, source, line))
|
||||
}
|
||||
NappletBrowserContract.MSG_MAGNIFIER_FRAME -> {
|
||||
val data = msg.data ?: return true
|
||||
val bytes = data.getByteArray(NappletBrowserContract.KEY_MAG_BYTES) ?: return true
|
||||
onMagnifierFrame?.invoke(
|
||||
MagnifierFrame(
|
||||
bytes = bytes,
|
||||
width = data.getInt(NappletBrowserContract.KEY_MAG_W),
|
||||
height = data.getInt(NappletBrowserContract.KEY_MAG_H),
|
||||
captureMs = data.getDouble(NappletBrowserContract.KEY_MAG_CAPTURE_MS),
|
||||
requestStampNanos = data.getLong(NappletBrowserContract.KEY_MAG_REQ_T),
|
||||
),
|
||||
)
|
||||
}
|
||||
else -> return false
|
||||
}
|
||||
return true
|
||||
@@ -246,6 +267,22 @@ class EmbeddedWebAppController(
|
||||
|
||||
override fun sendImeOp(json: String) = send(NappletBrowserContract.MSG_IME_OP) { putString(NappletBrowserContract.KEY_IME_PAYLOAD, json) }
|
||||
|
||||
// Stamp the client send time so the reply can be matched / stale frames dropped (same-process clock).
|
||||
override fun requestMagnifier(
|
||||
surfaceX: Float,
|
||||
surfaceY: Float,
|
||||
boxWidthPx: Int,
|
||||
boxHeightPx: Int,
|
||||
zoom: Float,
|
||||
) = send(NappletBrowserContract.MSG_MAGNIFIER_REQUEST) {
|
||||
putFloat(NappletBrowserContract.KEY_MAG_X, surfaceX)
|
||||
putFloat(NappletBrowserContract.KEY_MAG_Y, surfaceY)
|
||||
putInt(NappletBrowserContract.KEY_MAG_BOX_W, boxWidthPx)
|
||||
putInt(NappletBrowserContract.KEY_MAG_BOX_H, boxHeightPx)
|
||||
putFloat(NappletBrowserContract.KEY_MAG_ZOOM, zoom)
|
||||
putLong(NappletBrowserContract.KEY_MAG_REQ_T, SystemClock.elapsedRealtimeNanos())
|
||||
}
|
||||
|
||||
private fun parseImeEvent(payload: String): ImeEvent? {
|
||||
val o = runCatching { JSONObject(payload) }.getOrNull() ?: return null
|
||||
return when (o.optString("type")) {
|
||||
@@ -273,6 +310,8 @@ class EmbeddedWebAppController(
|
||||
text = o.optString("text", ""),
|
||||
geometry = parseSelectionGeometry(o.optJSONObject("geom")),
|
||||
)
|
||||
"ime.scroll" -> ImeEvent.Scroll(active = o.optBoolean("active", false))
|
||||
"ime.carettap" -> ImeEvent.CaretTap(geometry = parseSelectionGeometry(o.optJSONObject("geom")))
|
||||
else -> null
|
||||
}
|
||||
}
|
||||
|
||||
+23
@@ -38,6 +38,7 @@ fun parseSelectionGeometry(o: JSONObject?): SelectionGeometry? {
|
||||
caretX = if (o.has("cx")) o.optDouble("cx").toFloat() else null,
|
||||
caretTop = if (o.has("ct")) o.optDouble("ct").toFloat() else null,
|
||||
caretBottom = if (o.has("cb")) o.optDouble("cb").toFloat() else null,
|
||||
isRange = o.optBoolean("rng", false),
|
||||
)
|
||||
}
|
||||
|
||||
@@ -89,6 +90,24 @@ sealed interface ImeEvent {
|
||||
val text: String,
|
||||
val geometry: SelectionGeometry?,
|
||||
) : ImeEvent
|
||||
|
||||
/**
|
||||
* The embedded page started ([active] true) or finished ([active] false) scrolling. Host-drawn selection
|
||||
* UI hides while scrolling (its geometry is stale mid-scroll) and reappears, repositioned, on settle —
|
||||
* the shim re-reports geometry just before the `active=false` so the overlays land in the right place.
|
||||
*/
|
||||
data class Scroll(
|
||||
val active: Boolean,
|
||||
) : ImeEvent
|
||||
|
||||
/**
|
||||
* The user tapped the focused field, placing a bare caret. Re-shows the insertion handle even when the
|
||||
* tap didn't move the caret (so no [State] fired) — native shows the handle on every tap. [geometry]
|
||||
* carries the caret rect. Gated host-side to non-empty fields, like `Editor`.
|
||||
*/
|
||||
data class CaretTap(
|
||||
val geometry: SelectionGeometry?,
|
||||
) : ImeEvent
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -111,4 +130,8 @@ data class SelectionGeometry(
|
||||
val caretX: Float? = null,
|
||||
val caretTop: Float? = null,
|
||||
val caretBottom: Float? = null,
|
||||
// True when this is an in-field *range* selection whose [startX]/[startBottom]/[endX]/[endBottom] are the
|
||||
// real selection-endpoint feet (not the field-box placeholders). Lets the host hold the range geometry
|
||||
// through Chrome's transient collapse-to-caret reports during the re-assert fight.
|
||||
val isRange: Boolean = false,
|
||||
)
|
||||
|
||||
+114
@@ -0,0 +1,114 @@
|
||||
/*
|
||||
* Copyright (c) 2025 Vitor Pamplona
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
* this software and associated documentation files (the "Software"), to deal in
|
||||
* the Software without restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||
* subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in all
|
||||
* copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
package com.vitorpamplona.amethyst.ui.screen.loggedIn.embed
|
||||
|
||||
import androidx.compose.foundation.Image
|
||||
import androidx.compose.foundation.background
|
||||
import androidx.compose.foundation.border
|
||||
import androidx.compose.foundation.layout.Box
|
||||
import androidx.compose.foundation.layout.absoluteOffset
|
||||
import androidx.compose.foundation.layout.fillMaxSize
|
||||
import androidx.compose.foundation.layout.size
|
||||
import androidx.compose.foundation.shape.RoundedCornerShape
|
||||
import androidx.compose.material3.MaterialTheme
|
||||
import androidx.compose.runtime.Composable
|
||||
import androidx.compose.runtime.getValue
|
||||
import androidx.compose.runtime.mutableStateOf
|
||||
import androidx.compose.runtime.setValue
|
||||
import androidx.compose.ui.Modifier
|
||||
import androidx.compose.ui.draw.clip
|
||||
import androidx.compose.ui.draw.shadow
|
||||
import androidx.compose.ui.geometry.Offset
|
||||
import androidx.compose.ui.graphics.ImageBitmap
|
||||
import androidx.compose.ui.layout.ContentScale
|
||||
import androidx.compose.ui.platform.LocalDensity
|
||||
import androidx.compose.ui.unit.DpSize
|
||||
import androidx.compose.ui.unit.IntOffset
|
||||
import androidx.compose.ui.unit.IntSize
|
||||
import androidx.compose.ui.unit.dp
|
||||
import kotlin.math.roundToInt
|
||||
|
||||
/**
|
||||
* Hoisted state for the embedded-surface selection loupe (magnifier #4). The bubble shows a magnified,
|
||||
* live slice of the page captured in the `:napplet` provider (host-side `PixelCopy` can't read the sandbox
|
||||
* surface — see [EmbeddedMagnifierProbe]). While a caret/selection handle is dragged, [anchorPx] tracks the
|
||||
* drag point in this layer's px (so the bubble follows smoothly even between captures) and [image] is the
|
||||
* latest decoded frame. Throttle bookkeeping ([lastRequestUptimeMs]/[awaitingFrame]) lives here too but is
|
||||
* intentionally NOT Compose state — it must not trigger recomposition.
|
||||
*/
|
||||
class MagnifierUiState {
|
||||
var visible by mutableStateOf(false)
|
||||
var anchorPx by mutableStateOf(Offset.Zero)
|
||||
var image by mutableStateOf<ImageBitmap?>(null)
|
||||
|
||||
var lastRequestUptimeMs: Long = 0L
|
||||
var awaitingFrame: Boolean = false
|
||||
|
||||
fun hide() {
|
||||
visible = false
|
||||
image = null
|
||||
awaitingFrame = false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The loupe bubble: a rounded, elevated rectangle showing [MagnifierUiState.image], floated above the drag
|
||||
* point and centered on it horizontally, clamped inside [layerSize]. Like Android's `Magnifier`, it sits a
|
||||
* little above the finger; if there's no room above (drag near the top), it flips below. Nothing is drawn
|
||||
* until there's a frame to show.
|
||||
*/
|
||||
@Composable
|
||||
fun Magnifier(
|
||||
state: MagnifierUiState,
|
||||
layerSize: IntSize,
|
||||
bubbleSize: DpSize,
|
||||
) {
|
||||
if (!state.visible) return
|
||||
val img = state.image ?: return
|
||||
val density = LocalDensity.current
|
||||
val wPx = with(density) { bubbleSize.width.toPx() }
|
||||
val hPx = with(density) { bubbleSize.height.toPx() }
|
||||
val gapPx = with(density) { 28.dp.toPx() }
|
||||
|
||||
val left = (state.anchorPx.x - wPx / 2f).coerceIn(0f, (layerSize.width - wPx).coerceAtLeast(0f))
|
||||
val above = state.anchorPx.y - hPx - gapPx
|
||||
val top = if (above >= 0f) above else state.anchorPx.y + gapPx
|
||||
|
||||
val shape = RoundedCornerShape(12.dp)
|
||||
Box(
|
||||
Modifier
|
||||
.absoluteOffset { IntOffset(left.roundToInt(), top.roundToInt()) }
|
||||
.size(bubbleSize)
|
||||
.shadow(8.dp, shape)
|
||||
.clip(shape)
|
||||
.background(MaterialTheme.colorScheme.surface)
|
||||
.border(1.dp, MaterialTheme.colorScheme.outlineVariant, shape),
|
||||
) {
|
||||
// The captured frame is built so its px size ≈ the bubble's px size (source rect = bubble / zoom,
|
||||
// provider scaled by zoom), so FillBounds maps it 1:1 without distortion.
|
||||
Image(
|
||||
bitmap = img,
|
||||
contentDescription = null,
|
||||
contentScale = ContentScale.FillBounds,
|
||||
modifier = Modifier.fillMaxSize(),
|
||||
)
|
||||
}
|
||||
}
|
||||
+65
@@ -0,0 +1,65 @@
|
||||
/*
|
||||
* Copyright (c) 2025 Vitor Pamplona
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
* this software and associated documentation files (the "Software"), to deal in
|
||||
* the Software without restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||
* subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in all
|
||||
* copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
package com.vitorpamplona.amethyst.ui.screen.loggedIn.embed
|
||||
|
||||
/**
|
||||
* Magnifier (#4) capture path — see `amethyst/plans/2026-06-25-embed-text-selection-native-parity.md`.
|
||||
*
|
||||
* Host-side `PixelCopy` of the sandboxed surface returns `ERROR_SOURCE_NO_DATA` (the WebView pixels live in a
|
||||
* child SurfaceControl the host never draws into), so the selection loupe's content must be captured INSIDE
|
||||
* the keyless `:napplet` provider — where the WebView is a real in-window view with real pixels — and shipped
|
||||
* back over the Messenger channel. A controller whose surface can produce such captures implements this;
|
||||
* [EmbeddedTabLayer] discovers it by `as?` cast, mirroring how it treats [EmbeddedImeBridge].
|
||||
*/
|
||||
interface EmbeddedMagnifierProbe {
|
||||
/** A captured loupe frame arrived from the provider. */
|
||||
var onMagnifierFrame: ((MagnifierFrame) -> Unit)?
|
||||
|
||||
/**
|
||||
* Ask the provider for a zoomed slice of the live page centered on ([surfaceX], [surfaceY]) in surface px
|
||||
* (which equal the remote WebView's view px — the SCVH surface is 1:1 with the SandboxedSdkView).
|
||||
* [boxWidthPx]×[boxHeightPx] is the source rectangle; the returned image is that, scaled by [zoom].
|
||||
*/
|
||||
fun requestMagnifier(
|
||||
surfaceX: Float,
|
||||
surfaceY: Float,
|
||||
boxWidthPx: Int,
|
||||
boxHeightPx: Int,
|
||||
zoom: Float,
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* A loupe frame returned by the provider: a PNG ([bytes]) of [width]×[height], plus timing — [captureMs] is
|
||||
* the provider-side draw+encode cost and [requestStampNanos] echoes the client's send stamp so out-of-order
|
||||
* frames can be dropped.
|
||||
*/
|
||||
data class MagnifierFrame(
|
||||
val bytes: ByteArray,
|
||||
val width: Int,
|
||||
val height: Int,
|
||||
val captureMs: Double,
|
||||
val requestStampNanos: Long,
|
||||
) {
|
||||
override fun equals(other: Any?) = this === other
|
||||
|
||||
override fun hashCode() = System.identityHashCode(this)
|
||||
}
|
||||
+37
@@ -0,0 +1,37 @@
|
||||
/*
|
||||
* Copyright (c) 2025 Vitor Pamplona
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
* this software and associated documentation files (the "Software"), to deal in
|
||||
* the Software without restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||
* subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in all
|
||||
* copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
package com.vitorpamplona.amethyst.ui.screen.loggedIn.embed
|
||||
|
||||
import androidx.compose.runtime.getValue
|
||||
import androidx.compose.runtime.mutableStateOf
|
||||
import androidx.compose.runtime.setValue
|
||||
|
||||
/**
|
||||
* Process-level flag: true while the user is dragging a selection/caret handle over an embedded surface.
|
||||
*
|
||||
* The left navigation drawer's edge-swipe-to-open competes with a handle drag near the left edge (and with
|
||||
* the [auto-scroll][EmbeddedTabLayer] edge drag), opening the drawer mid-drag. The drawer reads this to
|
||||
* suspend its gesture while a handle is held. Kept out of the API-gated [EmbeddedTabHost] so non-R UI (the
|
||||
* drawer) can read it without a version guard; [EmbeddedTabLayer] is the only writer.
|
||||
*/
|
||||
object EmbeddedSelectionDrag {
|
||||
var dragging by mutableStateOf(false)
|
||||
}
|
||||
+401
-193
@@ -23,15 +23,19 @@ package com.vitorpamplona.amethyst.ui.screen.loggedIn.embed
|
||||
import android.content.ClipData
|
||||
import android.content.ClipboardManager
|
||||
import android.content.Context
|
||||
import android.graphics.BitmapFactory
|
||||
import android.os.Build
|
||||
import android.os.SystemClock
|
||||
import android.view.ViewGroup
|
||||
import androidx.annotation.RequiresApi
|
||||
import androidx.compose.foundation.BorderStroke
|
||||
import androidx.compose.foundation.Canvas
|
||||
import androidx.compose.foundation.background
|
||||
import androidx.compose.foundation.clickable
|
||||
import androidx.compose.foundation.gestures.awaitEachGesture
|
||||
import androidx.compose.foundation.gestures.awaitFirstDown
|
||||
import androidx.compose.foundation.gestures.detectDragGestures
|
||||
import androidx.compose.foundation.gestures.detectTapGestures
|
||||
import androidx.compose.foundation.gestures.waitForUpOrCancellation
|
||||
import androidx.compose.foundation.interaction.MutableInteractionSource
|
||||
import androidx.compose.foundation.layout.Box
|
||||
import androidx.compose.foundation.layout.ExperimentalLayoutApi
|
||||
@@ -62,11 +66,15 @@ import androidx.compose.ui.Modifier
|
||||
import androidx.compose.ui.geometry.Offset
|
||||
import androidx.compose.ui.geometry.Rect
|
||||
import androidx.compose.ui.graphics.Path
|
||||
import androidx.compose.ui.graphics.asImageBitmap
|
||||
import androidx.compose.ui.input.pointer.pointerInput
|
||||
import androidx.compose.ui.input.pointer.positionChangeIgnoreConsumed
|
||||
import androidx.compose.ui.layout.Layout
|
||||
import androidx.compose.ui.layout.onGloballyPositioned
|
||||
import androidx.compose.ui.layout.positionInWindow
|
||||
import androidx.compose.ui.platform.LocalContext
|
||||
import androidx.compose.ui.platform.LocalDensity
|
||||
import androidx.compose.ui.unit.DpSize
|
||||
import androidx.compose.ui.unit.IntOffset
|
||||
import androidx.compose.ui.unit.IntSize
|
||||
import androidx.compose.ui.unit.dp
|
||||
@@ -79,6 +87,31 @@ import kotlin.math.roundToInt
|
||||
// How far off-screen a parked (inactive) warm tab is shifted — well past any real screen width.
|
||||
private val OFFSCREEN_SHIFT = 10_000.dp
|
||||
|
||||
// Per-drag-move auto-scroll step (CSS px) when a handle is dragged into the surface's top/bottom edge zone.
|
||||
private const val AUTOSCROLL_STEP_CSS = 40.0
|
||||
|
||||
/**
|
||||
* Drives the selection loupe from a handle drag: [active] shows/hides the bubble, [fingerLayerPx] is the
|
||||
* drag point in the tab layer's px (where the bubble floats), and ([surfaceX], [surfaceY]) is the same point
|
||||
* in surface px (what the provider captures around). Called on drag start/move with `true`, on end with `false`.
|
||||
*/
|
||||
private typealias OnMagnify = (active: Boolean, fingerLayerPx: Offset, surfaceX: Float, surfaceY: Float) -> Unit
|
||||
|
||||
/** Sends a CSS-px positional IME op (caret move / selection extend) to the page; [edge] is null for a caret move. */
|
||||
private fun EmbeddedImeBridge.sendFieldOp(
|
||||
type: String,
|
||||
edge: String?,
|
||||
cssX: Float,
|
||||
cssY: Float,
|
||||
) = sendImeOp(
|
||||
JSONObject()
|
||||
.put("type", type)
|
||||
.apply { if (edge != null) put("edge", edge) }
|
||||
.put("x", cssX.toDouble())
|
||||
.put("y", cssY.toDouble())
|
||||
.toString(),
|
||||
)
|
||||
|
||||
/**
|
||||
* The persistent surface layer: a full-window overlay (mounted once in the app shell, below the
|
||||
* navigation drawer and dialogs) that renders **every** warm embedded session's [SandboxedSdkView] and
|
||||
@@ -295,47 +328,45 @@ fun EmbeddedTabLayer(barFavoriteIds: List<String>) {
|
||||
val context = LocalContext.current
|
||||
val imeBridge = EmbeddedTabHost.sessions.firstOrNull { it.id == activeId }?.controller as? EmbeddedImeBridge
|
||||
val imeView = remember { RemoteImeView(context) }
|
||||
// The embedded WebView can't show Chrome's copy/paste toolbar in its cross-process surface, so when
|
||||
// the page has a non-empty selection we show our own here, over the page, and route its actions to
|
||||
// the hidden EditText (which mirrors the selection) — see [RemoteImeView.onRangeSelectionChanged].
|
||||
var showSelectionToolbar by remember { mutableStateOf(false) }
|
||||
var fieldGeometry by remember { mutableStateOf<SelectionGeometry?>(null) }
|
||||
// Insertion (cursor) handle visibility: shown when a tap places/moves a bare caret, hidden while
|
||||
// typing — like Android. A caret-bearing geometry update means a tap; an edit means typing.
|
||||
var showInsertionHandle by remember { mutableStateOf(false) }
|
||||
var pageSelection by remember { mutableStateOf<ImeEvent.PageSelection?>(null) }
|
||||
// The embedded WebView can't show Chrome's copy/paste toolbar or selection handles in its cross-process
|
||||
// surface, so we draw our own over the page and route actions through the hidden EditText (which mirrors
|
||||
// the selection). All the show/hide state lives in one [SelectionUiState] so the rules — toolbar hides
|
||||
// while dragging or scrolling, handles hide while scrolling — are expressed in one place.
|
||||
val sel = remember { SelectionUiState() }
|
||||
DisposableEffect(imeBridge) {
|
||||
imeView.bind(imeBridge)
|
||||
imeView.onRangeSelectionChanged = { showSelectionToolbar = it }
|
||||
imeView.onEdited = { showInsertionHandle = false }
|
||||
imeView.onRangeSelectionChanged = { sel.onFieldRangeToggle(it) }
|
||||
imeView.onEdited = { sel.onEdited() }
|
||||
imeBridge?.onImeEvent = { event ->
|
||||
when (event) {
|
||||
is ImeEvent.Focus -> {
|
||||
imeView.onPageFocus(event)
|
||||
event.geometry?.let { fieldGeometry = it }
|
||||
if (event.geometry?.caretX != null) showInsertionHandle = true
|
||||
// A field took focus → the browser dropped any page-text selection; clear its overlay so
|
||||
// the stale page handles/Copy bar don't sit above (and steal drags from) the field UI.
|
||||
sel.onPageSelection(null)
|
||||
// Cancel any in-flight scroll-hide from the page phase: otherwise the field's own
|
||||
// selection-reveal scrolls keep it armed and the new field handles/toolbar never appear.
|
||||
sel.scrolling = false
|
||||
sel.onFieldGeometry(event.geometry, event.text.isNotEmpty())
|
||||
}
|
||||
ImeEvent.Blur -> {
|
||||
imeView.onPageBlur()
|
||||
fieldGeometry = null
|
||||
showInsertionHandle = false
|
||||
sel.onBlur()
|
||||
}
|
||||
is ImeEvent.State -> {
|
||||
imeView.onPageState(event)
|
||||
event.geometry?.let { fieldGeometry = it }
|
||||
if (event.geometry?.caretX != null) showInsertionHandle = true
|
||||
sel.onFieldGeometry(event.geometry, event.text.isNotEmpty())
|
||||
}
|
||||
is ImeEvent.PageSelection -> pageSelection = event.takeIf { it.active }
|
||||
is ImeEvent.PageSelection -> sel.onPageSelection(event.takeIf { it.active })
|
||||
is ImeEvent.Scroll -> sel.scrolling = event.active
|
||||
is ImeEvent.CaretTap -> sel.onCaretTap(event.geometry)
|
||||
}
|
||||
}
|
||||
onDispose {
|
||||
imeBridge?.onImeEvent = null
|
||||
imeView.onRangeSelectionChanged = null
|
||||
imeView.onEdited = null
|
||||
showSelectionToolbar = false
|
||||
fieldGeometry = null
|
||||
showInsertionHandle = false
|
||||
pageSelection = null
|
||||
sel.reset()
|
||||
imeView.onPageBlur()
|
||||
imeView.bind(null)
|
||||
}
|
||||
@@ -352,118 +383,209 @@ fun EmbeddedTabLayer(barFavoriteIds: List<String>) {
|
||||
)
|
||||
}
|
||||
|
||||
// Input-field selection: cut/copy/paste/select-all routed to the hidden EditText. The DOM exposes no
|
||||
// rect for a selection *inside* an input, so we anchor the bar above the field box (reported by the
|
||||
// shim), falling back to the top of the tab if the field geometry is missing.
|
||||
if (showSelectionToolbar && bounds.width > 0f && bounds.height > 0f) {
|
||||
val fg = fieldGeometry
|
||||
val items =
|
||||
listOf(
|
||||
"Cut" to {
|
||||
imeView.cutSelection()
|
||||
Unit
|
||||
},
|
||||
"Copy" to {
|
||||
imeView.copySelection()
|
||||
Unit
|
||||
},
|
||||
"Paste" to {
|
||||
imeView.pasteClipboard()
|
||||
Unit
|
||||
},
|
||||
"Select all" to {
|
||||
imeView.selectAllText()
|
||||
Unit
|
||||
},
|
||||
)
|
||||
val originX = bounds.left - layerOrigin.x
|
||||
val originY = bounds.top - layerOrigin.y
|
||||
val toolbarH = with(density) { 44.dp.toPx() }
|
||||
val gap = with(density) { 8.dp.toPx() }
|
||||
if (fg != null && fg.viewportWidth > 0f) {
|
||||
val scale = bounds.width / fg.viewportWidth
|
||||
val topPx = originY + fg.top * scale
|
||||
val toolbarY = if (topPx - toolbarH - gap > originY) topPx - toolbarH - gap else originY + fg.bottom * scale + gap
|
||||
EmbeddedSelectionToolbar(
|
||||
items = items,
|
||||
centerXpx = originX + ((fg.left + fg.right) / 2f) * scale,
|
||||
topYpx = toolbarY,
|
||||
)
|
||||
} else {
|
||||
EmbeddedSelectionToolbar(
|
||||
items = items,
|
||||
centerXpx = originX + bounds.width / 2f,
|
||||
topYpx = originY + gap,
|
||||
)
|
||||
// Native-style auto-hide: the insertion handle disappears after a few seconds of inactivity; the next
|
||||
// caret tap re-emits geometry and re-shows it. A drag keeps it alive (dragging gate) so it never
|
||||
// vanishes mid-drag; the key re-arms whenever the caret moves.
|
||||
val caretForTimeout = sel.insertionHandle
|
||||
LaunchedEffect(caretForTimeout, sel.dragging) {
|
||||
if (caretForTimeout != null && !sel.dragging) {
|
||||
delay(4_000)
|
||||
sel.hideCaret()
|
||||
}
|
||||
}
|
||||
|
||||
// Selection loupe (magnifier #4): while a caret/selection handle is dragged, show a magnified live
|
||||
// slice of the page. Host-side PixelCopy can't read the sandbox surface (ERROR_SOURCE_NO_DATA), so the
|
||||
// pixels are captured in the `:napplet` provider and shipped back here (see [EmbeddedMagnifierProbe]).
|
||||
// The handles report the drag point via [onMagnify]; we track it for the bubble position and ask the
|
||||
// provider for frames, throttled to one in flight (with a 100 ms timeout) so a fast drag can't flood
|
||||
// the IPC channel. The bubble follows the finger every move; the bitmap refreshes as frames land.
|
||||
val magProbe = imeBridge as? EmbeddedMagnifierProbe
|
||||
val magnifier = remember { MagnifierUiState() }
|
||||
val magBubble = DpSize(132.dp, 74.dp)
|
||||
val magZoom = 1.5f
|
||||
// Source rect (surface px) = bubble px / zoom, so the provider-scaled frame lands ≈ bubble-sized.
|
||||
val magSrcW = with(density) { (magBubble.width.toPx() / magZoom).roundToInt() }
|
||||
val magSrcH = with(density) { (magBubble.height.toPx() / magZoom).roundToInt() }
|
||||
DisposableEffect(magProbe) {
|
||||
magProbe?.onMagnifierFrame = { frame ->
|
||||
if (magnifier.visible) {
|
||||
magnifier.awaitingFrame = false
|
||||
BitmapFactory.decodeByteArray(frame.bytes, 0, frame.bytes.size)?.let { magnifier.image = it.asImageBitmap() }
|
||||
}
|
||||
}
|
||||
onDispose {
|
||||
magProbe?.onMagnifierFrame = null
|
||||
magnifier.hide()
|
||||
}
|
||||
}
|
||||
// Auto-scroll (#9): while dragging a handle near the surface's top/bottom edge, nudge the embedded
|
||||
// content so the selection can keep extending past the viewport — like Android. We scroll on each
|
||||
// drag-move that's in the edge zone (the finger is usually still micro-moving); the shim keeps the
|
||||
// overlays up during this programmatic scroll and re-reports geometry so they track.
|
||||
val edgeZonePx = with(density) { 56.dp.toPx() }
|
||||
val onMagnify: OnMagnify = { active, fingerPx, surfaceX, surfaceY ->
|
||||
// Drives the loupe, the toolbar-hide-while-dragging rule (and the dragged handle hides itself
|
||||
// locally), edge auto-scroll, and suspending the nav drawer's edge swipe — one drag lifecycle.
|
||||
sel.dragging = active
|
||||
EmbeddedSelectionDrag.dragging = active
|
||||
if (active && bounds.height > 0f) {
|
||||
val oy = bounds.top - layerOrigin.y
|
||||
val dy =
|
||||
when {
|
||||
fingerPx.y < oy + edgeZonePx -> -AUTOSCROLL_STEP_CSS
|
||||
fingerPx.y > oy + bounds.height - edgeZonePx -> AUTOSCROLL_STEP_CSS
|
||||
else -> 0.0
|
||||
}
|
||||
if (dy != 0.0) imeBridge?.sendImeOp(JSONObject().put("type", "ime.autoscroll").put("dy", dy).toString())
|
||||
}
|
||||
if (!active || magProbe == null) {
|
||||
magnifier.hide()
|
||||
} else {
|
||||
magnifier.visible = true
|
||||
magnifier.anchorPx = fingerPx
|
||||
val nowMs = SystemClock.uptimeMillis()
|
||||
if (!magnifier.awaitingFrame || nowMs - magnifier.lastRequestUptimeMs > 100L) {
|
||||
magnifier.awaitingFrame = true
|
||||
magnifier.lastRequestUptimeMs = nowMs
|
||||
magProbe.requestMagnifier(surfaceX, surfaceY, magSrcW, magSrcH, magZoom)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
val originX = bounds.left - layerOrigin.x
|
||||
val originY = bounds.top - layerOrigin.y
|
||||
val haveBounds = bounds.width > 0f && bounds.height > 0f
|
||||
|
||||
// In-field (<input>/<textarea>) range selection: cut/copy/paste/select-all routed to the hidden
|
||||
// EditText, plus draggable start/end handles (drag → `ime.fieldextend`). The toolbar hides while a
|
||||
// handle is dragged or the page scrolls; the handles hide only while scrolling.
|
||||
val fieldItems =
|
||||
listOf(
|
||||
"Cut" to {
|
||||
imeView.cutSelection()
|
||||
Unit
|
||||
},
|
||||
"Copy" to {
|
||||
imeView.copySelection()
|
||||
Unit
|
||||
},
|
||||
"Paste" to {
|
||||
imeView.pasteClipboard()
|
||||
Unit
|
||||
},
|
||||
"Select all" to {
|
||||
imeView.selectAllText()
|
||||
Unit
|
||||
},
|
||||
)
|
||||
val fieldHandles = sel.fieldHandles
|
||||
if (haveBounds && fieldHandles != null) {
|
||||
RangeSelectionOverlay(
|
||||
geometry = fieldHandles,
|
||||
surfaceOriginX = originX,
|
||||
surfaceOriginY = originY,
|
||||
scale = bounds.width / fieldHandles.viewportWidth,
|
||||
showToolbar = sel.fieldToolbar != null,
|
||||
toolbarItems = fieldItems,
|
||||
onMagnify = onMagnify,
|
||||
onExtend = { edge, cssX, cssY -> imeBridge?.sendFieldOp("ime.fieldextend", edge, cssX, cssY) },
|
||||
)
|
||||
} else if (haveBounds && sel.fieldToolbar != null) {
|
||||
// Authoritative range but no geometry yet → a centered fallback toolbar (no handles).
|
||||
EmbeddedSelectionToolbar(items = fieldItems, centerXpx = originX + bounds.width / 2f, topYpx = originY + with(density) { 8.dp.toPx() })
|
||||
}
|
||||
|
||||
// Bare caret in a field (no range): a draggable insertion handle under the cursor, like Android's.
|
||||
val fgCaret = fieldGeometry
|
||||
if (showInsertionHandle && !showSelectionToolbar && fgCaret?.caretX != null && fgCaret.caretBottom != null &&
|
||||
fgCaret.viewportWidth > 0f && bounds.width > 0f && bounds.height > 0f
|
||||
) {
|
||||
val scale = bounds.width / fgCaret.viewportWidth
|
||||
val originX = bounds.left - layerOrigin.x
|
||||
val originY = bounds.top - layerOrigin.y
|
||||
val caretGeom = sel.insertionHandle
|
||||
if (haveBounds && caretGeom?.caretX != null && caretGeom.caretBottom != null) {
|
||||
val scale = bounds.width / caretGeom.viewportWidth
|
||||
// Half the caret's line height (layer px), so the loupe capture centers on the text line.
|
||||
val caretLineHalf =
|
||||
(((caretGeom.caretTop?.let { caretGeom.caretBottom - it }) ?: 0f) * scale / 2f)
|
||||
.coerceIn(0f, with(density) { 28.dp.toPx() })
|
||||
InsertionHandle(
|
||||
tipPx = Offset(originX + fgCaret.caretX * scale, originY + fgCaret.caretBottom * scale),
|
||||
tipPx = Offset(originX + caretGeom.caretX * scale, originY + caretGeom.caretBottom * scale),
|
||||
surfaceOriginX = originX,
|
||||
surfaceOriginY = originY,
|
||||
scale = scale,
|
||||
onDragTo = { cssX, cssY ->
|
||||
imeBridge?.sendImeOp(
|
||||
JSONObject()
|
||||
.put("type", "ime.caretmove")
|
||||
.put("x", cssX.toDouble())
|
||||
.put("y", cssY.toDouble())
|
||||
.toString(),
|
||||
)
|
||||
},
|
||||
lineHalfPx = caretLineHalf,
|
||||
onMagnify = onMagnify,
|
||||
onTap = { sel.toggleInsertionPopup() },
|
||||
onDragTo = { cssX, cssY -> imeBridge?.sendFieldOp("ime.caretmove", null, cssX, cssY) },
|
||||
)
|
||||
}
|
||||
|
||||
// Plain page-text selection: a Copy bar positioned over the selection + a drag handle at each end.
|
||||
val pageSel = pageSelection
|
||||
val geom = pageSel?.geometry
|
||||
if (geom != null && geom.viewportWidth > 0f && bounds.width > 0f && bounds.height > 0f) {
|
||||
PageSelectionOverlay(
|
||||
geometry = geom,
|
||||
surfaceOriginX = bounds.left - layerOrigin.x,
|
||||
surfaceOriginY = bounds.top - layerOrigin.y,
|
||||
scale = bounds.width / geom.viewportWidth,
|
||||
onCopy = {
|
||||
val clipboard = context.getSystemService(Context.CLIPBOARD_SERVICE) as ClipboardManager
|
||||
clipboard.setPrimaryClip(ClipData.newPlainText("selection", pageSel.text))
|
||||
},
|
||||
onExtend = { edge, cssX, cssY ->
|
||||
imeBridge?.sendImeOp(
|
||||
JSONObject()
|
||||
.put("type", "ime.pageextend")
|
||||
.put("edge", edge)
|
||||
.put("x", cssX.toDouble())
|
||||
.put("y", cssY.toDouble())
|
||||
.toString(),
|
||||
)
|
||||
},
|
||||
// Tapping the bare insertion handle opens a small Paste / Select-all popup above the caret (native).
|
||||
val popupCaret = sel.insertionPopupAt
|
||||
if (haveBounds && popupCaret?.caretX != null && popupCaret.caretTop != null) {
|
||||
val scale = bounds.width / popupCaret.viewportWidth
|
||||
val gap = with(density) { 8.dp.toPx() }
|
||||
val toolbarH = with(density) { 44.dp.toPx() }
|
||||
val caretTopPx = originY + popupCaret.caretTop * scale
|
||||
val caretBottomPx = originY + (popupCaret.caretBottom ?: popupCaret.caretTop) * scale
|
||||
val topY = if (caretTopPx - toolbarH - gap > originY) caretTopPx - toolbarH - gap else caretBottomPx + gap
|
||||
EmbeddedSelectionToolbar(
|
||||
items =
|
||||
listOf(
|
||||
"Paste" to {
|
||||
imeView.pasteClipboard()
|
||||
sel.hideInsertionPopup()
|
||||
},
|
||||
"Select all" to {
|
||||
imeView.selectAllText()
|
||||
sel.hideInsertionPopup()
|
||||
},
|
||||
),
|
||||
centerXpx = originX + popupCaret.caretX * scale,
|
||||
topYpx = topY,
|
||||
)
|
||||
}
|
||||
|
||||
// Plain page-text selection: a Copy bar over the selection + a drag handle at each end (same rules).
|
||||
val pageHandles = sel.pageHandles
|
||||
val pageSel = sel.pageSelection
|
||||
if (haveBounds && pageHandles != null && pageSel != null) {
|
||||
RangeSelectionOverlay(
|
||||
geometry = pageHandles,
|
||||
surfaceOriginX = originX,
|
||||
surfaceOriginY = originY,
|
||||
scale = bounds.width / pageHandles.viewportWidth,
|
||||
showToolbar = sel.pageToolbar != null,
|
||||
toolbarItems =
|
||||
listOf(
|
||||
"Copy" to {
|
||||
val clipboard = context.getSystemService(Context.CLIPBOARD_SERVICE) as ClipboardManager
|
||||
clipboard.setPrimaryClip(ClipData.newPlainText("selection", pageSel.text))
|
||||
Unit
|
||||
},
|
||||
),
|
||||
onMagnify = onMagnify,
|
||||
onExtend = { edge, cssX, cssY -> imeBridge?.sendFieldOp("ime.pageextend", edge, cssX, cssY) },
|
||||
)
|
||||
}
|
||||
|
||||
// The loupe sits above everything else, following the active handle/caret drag.
|
||||
Magnifier(magnifier, layerSize, magBubble)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Host-drawn selection overlay for a plain page-text selection: a Copy bar above the selection plus a
|
||||
* draggable handle at each end. [geometry] is in the page's CSS px; [scale] and the surface origin map those
|
||||
* to this layer's px. Dragging a handle reports the new CSS-px target via [onExtend], which the shim turns
|
||||
* into a selection extension; the resulting geometry update repositions everything.
|
||||
* Host-drawn selection overlay shared by in-field (`<input>`/`<textarea>`) and plain page-text selections:
|
||||
* a floating toolbar above the selection (only when [showToolbar] — it's hidden mid-drag) plus a draggable
|
||||
* handle at each end. [geometry] is in the page's CSS px; [scale] and the surface origin map those to this
|
||||
* layer's px. Dragging a handle reports the new CSS-px target via [onExtend], which the shim turns into a
|
||||
* selection extension; the resulting geometry update repositions everything.
|
||||
*/
|
||||
@Composable
|
||||
private fun PageSelectionOverlay(
|
||||
private fun RangeSelectionOverlay(
|
||||
geometry: SelectionGeometry,
|
||||
surfaceOriginX: Float,
|
||||
surfaceOriginY: Float,
|
||||
scale: Float,
|
||||
onCopy: () -> Unit,
|
||||
showToolbar: Boolean,
|
||||
toolbarItems: List<Pair<String, () -> Unit>>,
|
||||
onMagnify: OnMagnify?,
|
||||
onExtend: (edge: String, cssX: Float, cssY: Float) -> Unit,
|
||||
) {
|
||||
val density = LocalDensity.current
|
||||
@@ -472,19 +594,24 @@ private fun PageSelectionOverlay(
|
||||
|
||||
fun mapY(css: Float) = surfaceOriginY + css * scale
|
||||
|
||||
val toolbarH = with(density) { 44.dp.toPx() }
|
||||
val gap = with(density) { 8.dp.toPx() }
|
||||
val topPx = mapY(geometry.top)
|
||||
val toolbarY = if (topPx - toolbarH - gap > surfaceOriginY) topPx - toolbarH - gap else mapY(geometry.bottom) + gap
|
||||
if (showToolbar) {
|
||||
val toolbarH = with(density) { 44.dp.toPx() }
|
||||
val gap = with(density) { 8.dp.toPx() }
|
||||
val topPx = mapY(geometry.top)
|
||||
val toolbarY = if (topPx - toolbarH - gap > surfaceOriginY) topPx - toolbarH - gap else mapY(geometry.bottom) + gap
|
||||
EmbeddedSelectionToolbar(
|
||||
items = toolbarItems,
|
||||
centerXpx = mapX((geometry.left + geometry.right) / 2f),
|
||||
topYpx = toolbarY,
|
||||
)
|
||||
}
|
||||
|
||||
EmbeddedSelectionToolbar(
|
||||
items = listOf("Copy" to onCopy),
|
||||
centerXpx = mapX((geometry.left + geometry.right) / 2f),
|
||||
topYpx = toolbarY,
|
||||
)
|
||||
|
||||
SelectionHandle(Offset(mapX(geometry.startX), mapY(geometry.startBottom)), isStart = true, surfaceOriginX, surfaceOriginY, scale) { x, y -> onExtend("start", x, y) }
|
||||
SelectionHandle(Offset(mapX(geometry.endX), mapY(geometry.endBottom)), isStart = false, surfaceOriginX, surfaceOriginY, scale) { x, y -> onExtend("end", x, y) }
|
||||
// Half the selection's line height (layer px), clamped — the bounding box spans every selected line, so
|
||||
// cap it at a sane single-line height so the loupe centers on the dragged endpoint's line, not the middle
|
||||
// of a tall multi-line box.
|
||||
val lineHalfPx = ((geometry.bottom - geometry.top) * scale / 2f).coerceAtMost(with(density) { 24.dp.toPx() })
|
||||
SelectionHandle(Offset(mapX(geometry.startX), mapY(geometry.startBottom)), isStart = true, surfaceOriginX, surfaceOriginY, scale, lineHalfPx, onMagnify) { x, y -> onExtend("start", x, y) }
|
||||
SelectionHandle(Offset(mapX(geometry.endX), mapY(geometry.endBottom)), isStart = false, surfaceOriginX, surfaceOriginY, scale, lineHalfPx, onMagnify) { x, y -> onExtend("end", x, y) }
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -500,14 +627,25 @@ private fun SelectionHandle(
|
||||
surfaceOriginX: Float,
|
||||
surfaceOriginY: Float,
|
||||
scale: Float,
|
||||
lineHalfPx: Float,
|
||||
onMagnify: OnMagnify?,
|
||||
onDragTo: (cssX: Float, cssY: Float) -> Unit,
|
||||
) {
|
||||
val sizeDp = 22.dp
|
||||
val sizePx = with(LocalDensity.current) { sizeDp.toPx() }
|
||||
val color = MaterialTheme.colorScheme.primary
|
||||
val currentTip by rememberUpdatedState(tipPx)
|
||||
val currentMagnify by rememberUpdatedState(onMagnify)
|
||||
var dragTip by remember { mutableStateOf<Offset?>(null) }
|
||||
val tip = dragTip ?: tipPx
|
||||
|
||||
// Loupe capture: X follows the finger, Y is locked to the authoritative endpoint's line ([currentTip] is
|
||||
// the foot, so lift by half the line height) — so the bubble shows the line being edited, not wherever the
|
||||
// finger drifts vertically.
|
||||
fun magnify(
|
||||
fingerPx: Offset,
|
||||
active: Boolean = true,
|
||||
) = currentMagnify?.invoke(active, fingerPx, fingerPx.x - surfaceOriginX, currentTip.y - surfaceOriginY - lineHalfPx)
|
||||
// Place the box so its pointed corner lands on the tip: start = top-right corner, end = top-left corner.
|
||||
val boxLeft = if (isStart) tip.x - sizePx else tip.x
|
||||
Box(
|
||||
@@ -516,35 +654,48 @@ private fun SelectionHandle(
|
||||
.size(sizeDp)
|
||||
.pointerInput(Unit) {
|
||||
detectDragGestures(
|
||||
onDragStart = { dragTip = currentTip },
|
||||
onDragStart = {
|
||||
dragTip = currentTip
|
||||
magnify(currentTip)
|
||||
},
|
||||
onDrag = { change, delta ->
|
||||
change.consume()
|
||||
val np = (dragTip ?: currentTip) + delta
|
||||
dragTip = np
|
||||
onDragTo((np.x - surfaceOriginX) / scale, (np.y - surfaceOriginY) / scale)
|
||||
magnify(np)
|
||||
},
|
||||
onDragEnd = {
|
||||
dragTip = null
|
||||
magnify(Offset.Zero, active = false)
|
||||
},
|
||||
onDragCancel = {
|
||||
dragTip = null
|
||||
magnify(Offset.Zero, active = false)
|
||||
},
|
||||
onDragEnd = { dragTip = null },
|
||||
onDragCancel = { dragTip = null },
|
||||
)
|
||||
},
|
||||
) {
|
||||
Canvas(Modifier.fillMaxSize()) {
|
||||
val d = size.minDimension
|
||||
val path =
|
||||
Path().apply {
|
||||
if (isStart) {
|
||||
moveTo(d, 0f)
|
||||
lineTo(d, d / 2f)
|
||||
arcTo(Rect(0f, 0f, d, d), 0f, 270f, false)
|
||||
close()
|
||||
} else {
|
||||
moveTo(0f, 0f)
|
||||
lineTo(0f, d / 2f)
|
||||
arcTo(Rect(0f, 0f, d, d), 180f, -270f, false)
|
||||
close()
|
||||
// Hide the teardrop while THIS handle is being dragged — the loupe stands in for it, like Android.
|
||||
if (dragTip == null) {
|
||||
Canvas(Modifier.fillMaxSize()) {
|
||||
val d = size.minDimension
|
||||
val path =
|
||||
Path().apply {
|
||||
if (isStart) {
|
||||
moveTo(d, 0f)
|
||||
lineTo(d, d / 2f)
|
||||
arcTo(Rect(0f, 0f, d, d), 0f, 270f, false)
|
||||
close()
|
||||
} else {
|
||||
moveTo(0f, 0f)
|
||||
lineTo(0f, d / 2f)
|
||||
arcTo(Rect(0f, 0f, d, d), 180f, -270f, false)
|
||||
close()
|
||||
}
|
||||
}
|
||||
}
|
||||
drawPath(path, color)
|
||||
drawPath(path, color)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -560,6 +711,9 @@ private fun InsertionHandle(
|
||||
surfaceOriginX: Float,
|
||||
surfaceOriginY: Float,
|
||||
scale: Float,
|
||||
lineHalfPx: Float,
|
||||
onMagnify: OnMagnify?,
|
||||
onTap: () -> Unit,
|
||||
onDragTo: (cssX: Float, cssY: Float) -> Unit,
|
||||
) {
|
||||
val density = LocalDensity.current
|
||||
@@ -567,6 +721,16 @@ private fun InsertionHandle(
|
||||
val hPx = wPx * 1.4f
|
||||
val color = MaterialTheme.colorScheme.primary
|
||||
val currentTip by rememberUpdatedState(tipPx)
|
||||
val currentMagnify by rememberUpdatedState(onMagnify)
|
||||
val currentTap by rememberUpdatedState(onTap)
|
||||
|
||||
// Loupe capture: X follows the finger, Y is locked to the authoritative caret's line ([currentTip] is the
|
||||
// caret foot, so lift by half the line height) — so the bubble shows the edited line, not wherever the
|
||||
// finger drifts vertically (e.g. a diagonal drag in a textarea).
|
||||
fun magnify(
|
||||
fingerPx: Offset,
|
||||
active: Boolean = true,
|
||||
) = currentMagnify?.invoke(active, fingerPx, fingerPx.x - surfaceOriginX, currentTip.y - surfaceOriginY - lineHalfPx)
|
||||
// Track the finger separately from what we draw: the handle renders at the AUTHORITATIVE caret (tipPx,
|
||||
// which snaps to a character position as the move round-trips through the shim), while the finger drives
|
||||
// the move. So the handle stays glued to the line/text and clamps to the field instead of trailing off.
|
||||
@@ -575,33 +739,62 @@ private fun InsertionHandle(
|
||||
Modifier
|
||||
.absoluteOffset { IntOffset((tipPx.x - wPx / 2f).roundToInt(), tipPx.y.roundToInt()) }
|
||||
.size(with(density) { wPx.toDp() }, with(density) { hPx.toDp() })
|
||||
// One unified gesture so a TAP (toggle the Paste/Select-All popup) and a DRAG (move the caret)
|
||||
// don't fight each other. We consume the down so the tap never bleeds to the surface (which would
|
||||
// place a caret there and dismiss the popup we're opening).
|
||||
.pointerInput(Unit) {
|
||||
detectDragGestures(
|
||||
onDragStart = { fingerPx = currentTip },
|
||||
onDrag = { change, delta ->
|
||||
change.consume()
|
||||
val np = (fingerPx ?: currentTip) + delta
|
||||
fingerPx = np
|
||||
onDragTo((np.x - surfaceOriginX) / scale, (np.y - surfaceOriginY) / scale)
|
||||
},
|
||||
onDragEnd = { fingerPx = null },
|
||||
onDragCancel = { fingerPx = null },
|
||||
)
|
||||
awaitEachGesture {
|
||||
val down = awaitFirstDown(requireUnconsumed = false)
|
||||
down.consume()
|
||||
var dragging = false
|
||||
var fp = currentTip
|
||||
while (true) {
|
||||
val change = awaitPointerEvent().changes.firstOrNull() ?: break
|
||||
if (!change.pressed) {
|
||||
if (!dragging) {
|
||||
currentTap()
|
||||
} else {
|
||||
fingerPx = null
|
||||
magnify(Offset.Zero, active = false)
|
||||
}
|
||||
break
|
||||
}
|
||||
if (!dragging && (change.position - down.position).getDistance() > viewConfiguration.touchSlop) {
|
||||
dragging = true
|
||||
fp = currentTip
|
||||
magnify(currentTip)
|
||||
}
|
||||
if (dragging) {
|
||||
// Read the delta BEFORE consuming: positionChange() returns Offset.Zero once the
|
||||
// change isConsumed, so consuming first (or the sandbox surface consuming the move in
|
||||
// an earlier pass) would freeze `fp` at the caret — the handle drags but the caret
|
||||
// never moves. positionChangeIgnoreConsumed() is immune to both.
|
||||
fp += change.positionChangeIgnoreConsumed()
|
||||
change.consume()
|
||||
fingerPx = fp
|
||||
onDragTo((fp.x - surfaceOriginX) / scale, (fp.y - surfaceOriginY) / scale)
|
||||
magnify(fp)
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
) {
|
||||
Canvas(Modifier.fillMaxSize()) {
|
||||
val r = size.width / 2f
|
||||
val cx = size.width / 2f
|
||||
val cy = size.height - r
|
||||
val path =
|
||||
Path().apply {
|
||||
moveTo(cx, 0f)
|
||||
lineTo(cx + r, cy)
|
||||
arcTo(Rect(cx - r, cy - r, cx + r, cy + r), 0f, 180f, false)
|
||||
lineTo(cx, 0f)
|
||||
close()
|
||||
}
|
||||
drawPath(path, color)
|
||||
// Hide the teardrop while dragging — the loupe stands in for it, like Android.
|
||||
if (fingerPx == null) {
|
||||
Canvas(Modifier.fillMaxSize()) {
|
||||
val r = size.width / 2f
|
||||
val cx = size.width / 2f
|
||||
val cy = size.height - r
|
||||
val path =
|
||||
Path().apply {
|
||||
moveTo(cx, 0f)
|
||||
lineTo(cx + r, cy)
|
||||
arcTo(Rect(cx - r, cy - r, cx + r, cy + r), 0f, 180f, false)
|
||||
lineTo(cx, 0f)
|
||||
close()
|
||||
}
|
||||
drawPath(path, color)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -619,32 +812,37 @@ private fun EmbeddedSelectionToolbar(
|
||||
centerXpx: Float,
|
||||
topYpx: Float,
|
||||
) {
|
||||
// Self-center: the toolbar's width depends on its items, so measure it and offset by half-width around
|
||||
// the requested centre-x (a one-frame left-aligned flash before the first measurement is acceptable).
|
||||
var widthPx by remember { mutableStateOf(0) }
|
||||
Surface(
|
||||
modifier =
|
||||
Modifier
|
||||
.absoluteOffset { IntOffset((centerXpx - widthPx / 2f).roundToInt(), topYpx.roundToInt()) }
|
||||
.onGloballyPositioned { widthPx = it.size.width },
|
||||
shape = RoundedCornerShape(8.dp),
|
||||
color = MaterialTheme.colorScheme.surface,
|
||||
border = BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant),
|
||||
tonalElevation = 3.dp,
|
||||
shadowElevation = 6.dp,
|
||||
) {
|
||||
Row(verticalAlignment = Alignment.CenterVertically) {
|
||||
items.forEachIndexed { i, (label, action) ->
|
||||
if (i > 0) {
|
||||
Box(
|
||||
Modifier
|
||||
.width(1.dp)
|
||||
.height(20.dp)
|
||||
.background(MaterialTheme.colorScheme.outlineVariant),
|
||||
)
|
||||
// Center in a SINGLE layout pass: measure the bar, then place it at centerX − width/2. (The old
|
||||
// measure-then-offset approach rendered it left-edge-at-centerX for one frame, so it visibly slid in
|
||||
// from the right each time the bar was (re)composed.)
|
||||
Layout(
|
||||
content = {
|
||||
Surface(
|
||||
shape = RoundedCornerShape(8.dp),
|
||||
color = MaterialTheme.colorScheme.surface,
|
||||
border = BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant),
|
||||
tonalElevation = 3.dp,
|
||||
shadowElevation = 6.dp,
|
||||
) {
|
||||
Row(verticalAlignment = Alignment.CenterVertically) {
|
||||
items.forEachIndexed { i, (label, action) ->
|
||||
if (i > 0) {
|
||||
Box(
|
||||
Modifier
|
||||
.width(1.dp)
|
||||
.height(20.dp)
|
||||
.background(MaterialTheme.colorScheme.outlineVariant),
|
||||
)
|
||||
}
|
||||
SelectionToolbarItem(label, action)
|
||||
}
|
||||
}
|
||||
SelectionToolbarItem(label, action)
|
||||
}
|
||||
},
|
||||
) { measurables, constraints ->
|
||||
val bar = measurables.first().measure(constraints.copy(minWidth = 0, minHeight = 0))
|
||||
layout(constraints.maxWidth, constraints.maxHeight) {
|
||||
bar.place((centerXpx - bar.width / 2f).roundToInt(), topYpx.roundToInt())
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -659,7 +857,17 @@ private fun SelectionToolbarItem(
|
||||
color = MaterialTheme.colorScheme.onSurfaceVariant,
|
||||
modifier =
|
||||
Modifier
|
||||
.pointerInput(label) { detectTapGestures(onTap = { onClick() }) }
|
||||
.padding(horizontal = 12.dp, vertical = 10.dp),
|
||||
// Consume the DOWN (not just the up) so the tap never bleeds through to the embedded surface
|
||||
// beneath — otherwise tapping the bar over non-editable page area blurs the field.
|
||||
.pointerInput(label) {
|
||||
awaitEachGesture {
|
||||
awaitFirstDown(requireUnconsumed = false).consume()
|
||||
val up = waitForUpOrCancellation()
|
||||
if (up != null) {
|
||||
up.consume()
|
||||
onClick()
|
||||
}
|
||||
}
|
||||
}.padding(horizontal = 12.dp, vertical = 10.dp),
|
||||
)
|
||||
}
|
||||
|
||||
+33
-3
@@ -121,6 +121,21 @@ class RemoteImeView(
|
||||
var onRangeSelectionChanged: ((Boolean) -> Unit)? = null
|
||||
private var hadRange = false
|
||||
|
||||
// Off-window Chrome abandons a selection by momentarily collapsing the caret to an endpoint, which we (or
|
||||
// the page shim) re-assert right back — so the mirrored selection flickers range→caret→range within a few
|
||||
// ms during a long-press/double-tap. The host-drawn handles/toolbar are gated on [hadRange], so reporting
|
||||
// every flip blinks them off and back on, in lock-step with each collapse cycle. Native never shows that
|
||||
// churn. So we DEFER the "range lost" signal by [RANGE_LOSS_DEBOUNCE_MS]: a re-assert that restores the
|
||||
// range first cancels the pending hide, and only a selection that truly STAYS collapsed hides the overlays.
|
||||
// Gaining a range is always reported immediately.
|
||||
private val reportRangeLost =
|
||||
Runnable {
|
||||
if (selectionStart == selectionEnd && hadRange) {
|
||||
hadRange = false
|
||||
onRangeSelectionChanged?.invoke(false)
|
||||
}
|
||||
}
|
||||
|
||||
/** Fired when the user edits text via the keyboard (not on programmatic page-state applies). The host
|
||||
* hides the insertion handle while typing, the way Android does. */
|
||||
var onEdited: (() -> Unit)? = null
|
||||
@@ -187,6 +202,7 @@ class RemoteImeView(
|
||||
|
||||
/** The page field blurred: drop the keyboard. */
|
||||
fun onPageBlur() {
|
||||
removeCallbacks(reportRangeLost)
|
||||
if (hadRange) {
|
||||
hadRange = false
|
||||
onRangeSelectionChanged?.invoke(false)
|
||||
@@ -227,9 +243,18 @@ class RemoteImeView(
|
||||
) {
|
||||
super.onSelectionChanged(selStart, selEnd)
|
||||
val isRange = selStart != selEnd
|
||||
if (isRange != hadRange) {
|
||||
hadRange = isRange
|
||||
onRangeSelectionChanged?.invoke(isRange)
|
||||
if (isRange) {
|
||||
// A range is back (or still here): cancel any pending hide and show immediately.
|
||||
removeCallbacks(reportRangeLost)
|
||||
if (!hadRange) {
|
||||
hadRange = true
|
||||
onRangeSelectionChanged?.invoke(true)
|
||||
}
|
||||
} else if (hadRange) {
|
||||
// Collapsed — but this may be Chrome's transient abandonment we're about to re-assert. Defer the
|
||||
// hide; if the range returns within the window, the show branch above cancels this.
|
||||
removeCallbacks(reportRangeLost)
|
||||
postDelayed(reportRangeLost, RANGE_LOSS_DEBOUNCE_MS)
|
||||
}
|
||||
schedule()
|
||||
}
|
||||
@@ -321,5 +346,10 @@ class RemoteImeView(
|
||||
// Wider than Chrome's ~600ms re-collapse interval so consecutive abandonments stay inside the window,
|
||||
// yet short enough that a deliberate user tap-to-collapse (well after the gesture) is accepted.
|
||||
private const val REASSERT_WINDOW_MS = 800L
|
||||
|
||||
// How long a collapse must persist before we hide the host-drawn selection overlays. The shim/host
|
||||
// re-assert restores an abandonment collapse within a frame or two (one IPC hop), so this only needs to
|
||||
// outlast that round-trip — comfortably short enough that a genuine tap-to-collapse still feels instant.
|
||||
private const val RANGE_LOSS_DEBOUNCE_MS = 250L
|
||||
}
|
||||
}
|
||||
|
||||
+183
@@ -0,0 +1,183 @@
|
||||
/*
|
||||
* Copyright (c) 2025 Vitor Pamplona
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
* this software and associated documentation files (the "Software"), to deal in
|
||||
* the Software without restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||
* subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in all
|
||||
* copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
package com.vitorpamplona.amethyst.ui.screen.loggedIn.embed
|
||||
|
||||
import androidx.compose.runtime.Stable
|
||||
import androidx.compose.runtime.getValue
|
||||
import androidx.compose.runtime.mutableStateOf
|
||||
import androidx.compose.runtime.setValue
|
||||
|
||||
/**
|
||||
* Single source of truth for the host-drawn selection UI over an embedded surface. Folds together what used
|
||||
* to be scattered booleans/geometries (`showSelectionToolbar`, `showInsertionHandle`, `fieldGeometry`,
|
||||
* `rangeFieldGeometry`, `pageSelection`) so the show/hide rules are expressed in one place.
|
||||
*
|
||||
* Three mutually-exclusive selection contexts can be active: a bare caret in a field ([insertionGeometry]),
|
||||
* a range inside an `<input>`/`<textarea>` ([fieldRange], gated by [fieldHasRange] — the authoritative
|
||||
* "is there a selection" signal from the mirror [RemoteImeView]), or a plain page-text selection
|
||||
* ([pageSelection]). [dragging] and [scrolling] are the transient modifiers:
|
||||
*
|
||||
* - **dragging** a handle/caret hides the floating toolbar (and the dragged handle hides itself while the
|
||||
* loupe stands in), matching Android; the other handle stays so you can see the live range.
|
||||
* - **scrolling** hides every overlay (its geometry is stale mid-scroll); the shim re-reports geometry on
|
||||
* settle, then clears this, so they reappear in the right place.
|
||||
*/
|
||||
@Stable
|
||||
class SelectionUiState {
|
||||
/** Latest field geometry; carries the caret rect when collapsed, positioning the insertion handle. */
|
||||
var fieldGeometry by mutableStateOf<SelectionGeometry?>(null)
|
||||
private set
|
||||
|
||||
/** Held in-field range geometry (real endpoint feet) — survives Chrome's transient collapse-to-caret. */
|
||||
var fieldRange by mutableStateOf<SelectionGeometry?>(null)
|
||||
private set
|
||||
|
||||
/** Authoritative "the field selection is a non-empty range", from the mirror EditText. */
|
||||
var fieldHasRange by mutableStateOf(false)
|
||||
private set
|
||||
|
||||
/** A bare caret handle is showing — set by a tap-placed caret in non-empty text, cleared on typing/blur/timeout. */
|
||||
var caretShown by mutableStateOf(false)
|
||||
private set
|
||||
|
||||
/**
|
||||
* Whether the focused field currently has any text. Native (`Editor.onTouchUpEvent`) only offers the
|
||||
* insertion handle when `text.length() > 0`, so an empty field never gets a draggable cursor handle
|
||||
* (just the blinking caret). Gating on this is what stops the handle popping up on focus of an empty box.
|
||||
*/
|
||||
var fieldHasText by mutableStateOf(false)
|
||||
private set
|
||||
|
||||
/** Plain page-text (non-editable) selection. */
|
||||
var pageSelection by mutableStateOf<ImeEvent.PageSelection?>(null)
|
||||
private set
|
||||
|
||||
/** A handle/caret is being dragged. */
|
||||
var dragging by mutableStateOf(false)
|
||||
|
||||
/** The embedded surface is scrolling. */
|
||||
var scrolling by mutableStateOf(false)
|
||||
|
||||
/** Tapping the bare insertion handle opens a small Paste/Select-All popup (native); tap-elsewhere closes it. */
|
||||
var insertionPopup by mutableStateOf(false)
|
||||
private set
|
||||
|
||||
private val insertionGeometry: SelectionGeometry?
|
||||
get() = fieldGeometry?.takeIf { it.caretX != null && it.caretBottom != null && it.viewportWidth > 0f }
|
||||
|
||||
// ---- derived visibility (read in composition; track the backing state) ----
|
||||
|
||||
val insertionHandle: SelectionGeometry?
|
||||
get() = if (caretShown && fieldHasText && !fieldHasRange && !scrolling) insertionGeometry else null
|
||||
|
||||
/** Geometry for the insertion-handle Paste/Select-All popup, or null when it shouldn't show. */
|
||||
val insertionPopupAt: SelectionGeometry?
|
||||
get() = if (insertionPopup && !dragging) insertionHandle else null
|
||||
|
||||
val fieldHandles: SelectionGeometry?
|
||||
get() = if (fieldHasRange && !scrolling) fieldRange?.takeIf { it.viewportWidth > 0f } else null
|
||||
|
||||
val fieldToolbar: SelectionGeometry?
|
||||
get() = if (fieldHasRange && !dragging && !scrolling) (fieldRange ?: fieldGeometry) else null
|
||||
|
||||
private val pageGeom: SelectionGeometry?
|
||||
get() = pageSelection?.takeIf { it.active }?.geometry?.takeIf { it.viewportWidth > 0f }
|
||||
|
||||
val pageHandles: SelectionGeometry?
|
||||
get() = if (!scrolling) pageGeom else null
|
||||
|
||||
val pageToolbar: SelectionGeometry?
|
||||
get() = if (!dragging && !scrolling) pageGeom else null
|
||||
|
||||
// ---- signal sinks ----
|
||||
|
||||
fun onFieldGeometry(
|
||||
g: SelectionGeometry?,
|
||||
hasText: Boolean,
|
||||
) {
|
||||
fieldHasText = hasText
|
||||
if (g == null) return
|
||||
fieldGeometry = g
|
||||
if (g.isRange) fieldRange = g
|
||||
// Only a tap-placed caret in NON-empty text gets the handle — like native. (Typing suppresses the
|
||||
// echo, so this fires for taps, not keystrokes; see RemoteImeView / shim __nappletIme guard.)
|
||||
if (g.caretX != null && hasText) caretShown = true
|
||||
}
|
||||
|
||||
/** From [RemoteImeView.onRangeSelectionChanged]: the field gained/lost a non-empty selection. */
|
||||
fun onFieldRangeToggle(hasRange: Boolean) {
|
||||
fieldHasRange = hasRange
|
||||
if (!hasRange) fieldRange = null
|
||||
if (hasRange) insertionPopup = false // a selection formed → no insertion popup
|
||||
}
|
||||
|
||||
/**
|
||||
* The user tapped the focused field (placing a caret) — (re-)show the insertion handle even if the caret
|
||||
* didn't move. Gated to non-empty text by the [insertionHandle] getter, exactly like native.
|
||||
*/
|
||||
fun onCaretTap(g: SelectionGeometry?) {
|
||||
if (g != null) fieldGeometry = g
|
||||
caretShown = true
|
||||
insertionPopup = false // a tap in the text dismisses the popup (and moves the caret)
|
||||
}
|
||||
|
||||
/** The user typed: hide the insertion handle, like Android. */
|
||||
fun onEdited() {
|
||||
caretShown = false
|
||||
insertionPopup = false
|
||||
}
|
||||
|
||||
/** Native auto-hides the insertion handle after ~a few seconds of inactivity; a later tap re-shows it. */
|
||||
fun hideCaret() {
|
||||
caretShown = false
|
||||
insertionPopup = false
|
||||
}
|
||||
|
||||
/** Tap on the bare insertion handle toggles its Paste/Select-All popup. */
|
||||
fun toggleInsertionPopup() {
|
||||
insertionPopup = !insertionPopup
|
||||
}
|
||||
|
||||
fun hideInsertionPopup() {
|
||||
insertionPopup = false
|
||||
}
|
||||
|
||||
fun onBlur() {
|
||||
fieldGeometry = null
|
||||
fieldRange = null
|
||||
fieldHasRange = false
|
||||
fieldHasText = false
|
||||
caretShown = false
|
||||
insertionPopup = false
|
||||
}
|
||||
|
||||
fun onPageSelection(sel: ImeEvent.PageSelection?) {
|
||||
pageSelection = sel
|
||||
}
|
||||
|
||||
/** Full teardown when the active tab/controller changes. */
|
||||
fun reset() {
|
||||
onBlur()
|
||||
pageSelection = null
|
||||
dragging = false
|
||||
scrolling = false
|
||||
}
|
||||
}
|
||||
+38
-1
@@ -31,6 +31,7 @@ import android.os.IBinder
|
||||
import android.os.Looper
|
||||
import android.os.Message
|
||||
import android.os.Messenger
|
||||
import android.os.SystemClock
|
||||
import androidx.annotation.RequiresApi
|
||||
import androidx.privacysandbox.ui.client.SandboxedUiAdapterFactory
|
||||
import androidx.privacysandbox.ui.client.view.SandboxedSdkView
|
||||
@@ -39,8 +40,10 @@ import com.vitorpamplona.amethyst.napplethost.NappletEmbedContract
|
||||
import com.vitorpamplona.amethyst.napplethost.NappletHostContract
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedImeBridge
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedLoadStatus
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedMagnifierProbe
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.EmbeddedSurfaceController
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.ImeEvent
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.MagnifierFrame
|
||||
import com.vitorpamplona.amethyst.ui.screen.loggedIn.embed.parseSelectionGeometry
|
||||
import org.json.JSONObject
|
||||
import java.util.concurrent.atomic.AtomicLong
|
||||
@@ -60,7 +63,8 @@ class EmbeddedNostrAppController(
|
||||
private val appContext: Context,
|
||||
private val params: Bundle,
|
||||
) : EmbeddedSurfaceController,
|
||||
EmbeddedImeBridge {
|
||||
EmbeddedImeBridge,
|
||||
EmbeddedMagnifierProbe {
|
||||
private val incoming = Messenger(Handler(Looper.getMainLooper(), ::onServiceMessage))
|
||||
private var serviceMessenger: Messenger? = null
|
||||
private var bound = false
|
||||
@@ -94,6 +98,8 @@ class EmbeddedNostrAppController(
|
||||
|
||||
override var onImeEvent: ((ImeEvent) -> Unit)? = null
|
||||
|
||||
override var onMagnifierFrame: ((MagnifierFrame) -> Unit)? = null
|
||||
|
||||
private val connection =
|
||||
object : ServiceConnection {
|
||||
override fun onServiceConnected(
|
||||
@@ -126,6 +132,7 @@ class EmbeddedNostrAppController(
|
||||
onStateChanged = null
|
||||
onNotice = null
|
||||
onImeEvent = null
|
||||
onMagnifierFrame = null
|
||||
onLoadStatusChanged = null
|
||||
}
|
||||
|
||||
@@ -184,6 +191,19 @@ class EmbeddedNostrAppController(
|
||||
val failed = msg.data?.getBoolean(NappletEmbedContract.KEY_LOAD_FAILED, false) ?: false
|
||||
onLoadState(isLoading, failed)
|
||||
}
|
||||
NappletEmbedContract.MSG_MAGNIFIER_FRAME -> {
|
||||
val data = msg.data ?: return true
|
||||
val bytes = data.getByteArray(NappletEmbedContract.KEY_MAG_BYTES) ?: return true
|
||||
onMagnifierFrame?.invoke(
|
||||
MagnifierFrame(
|
||||
bytes = bytes,
|
||||
width = data.getInt(NappletEmbedContract.KEY_MAG_W),
|
||||
height = data.getInt(NappletEmbedContract.KEY_MAG_H),
|
||||
captureMs = data.getDouble(NappletEmbedContract.KEY_MAG_CAPTURE_MS),
|
||||
requestStampNanos = data.getLong(NappletEmbedContract.KEY_MAG_REQ_T),
|
||||
),
|
||||
)
|
||||
}
|
||||
else -> return false
|
||||
}
|
||||
return true
|
||||
@@ -191,6 +211,21 @@ class EmbeddedNostrAppController(
|
||||
|
||||
override fun sendImeOp(json: String) = send(NappletEmbedContract.MSG_IME_OP) { putString(NappletEmbedContract.KEY_IME_PAYLOAD, json) }
|
||||
|
||||
override fun requestMagnifier(
|
||||
surfaceX: Float,
|
||||
surfaceY: Float,
|
||||
boxWidthPx: Int,
|
||||
boxHeightPx: Int,
|
||||
zoom: Float,
|
||||
) = send(NappletEmbedContract.MSG_MAGNIFIER_REQUEST) {
|
||||
putFloat(NappletEmbedContract.KEY_MAG_X, surfaceX)
|
||||
putFloat(NappletEmbedContract.KEY_MAG_Y, surfaceY)
|
||||
putInt(NappletEmbedContract.KEY_MAG_BOX_W, boxWidthPx)
|
||||
putInt(NappletEmbedContract.KEY_MAG_BOX_H, boxHeightPx)
|
||||
putFloat(NappletEmbedContract.KEY_MAG_ZOOM, zoom)
|
||||
putLong(NappletEmbedContract.KEY_MAG_REQ_T, SystemClock.elapsedRealtimeNanos())
|
||||
}
|
||||
|
||||
private fun parseImeEvent(payload: String): ImeEvent? {
|
||||
val o = runCatching { JSONObject(payload) }.getOrNull() ?: return null
|
||||
return when (o.optString("type")) {
|
||||
@@ -218,6 +253,8 @@ class EmbeddedNostrAppController(
|
||||
text = o.optString("text", ""),
|
||||
geometry = parseSelectionGeometry(o.optJSONObject("geom")),
|
||||
)
|
||||
"ime.scroll" -> ImeEvent.Scroll(active = o.optBoolean("active", false))
|
||||
"ime.carettap" -> ImeEvent.CaretTap(geometry = parseSelectionGeometry(o.optJSONObject("geom")))
|
||||
else -> null
|
||||
}
|
||||
}
|
||||
|
||||
@@ -277,6 +277,12 @@
|
||||
return [n.selectionStart || 0, n.selectionEnd || 0];
|
||||
}
|
||||
function setSel(n, s, e){
|
||||
// No-op if already there: re-applying the same selection still fires `select`/`selectionchange`, and
|
||||
// every such redundant event ripples into a host report → geometry update → toolbar/handle recompose
|
||||
// (visible churn). Our re-asserts/re-applies frequently target the current range, so guard them here.
|
||||
var cur = selOf(n);
|
||||
if (cur[0] === s && cur[1] === e) return;
|
||||
lastSelActivityAt = perfNow(); // a real selection write → the field may auto-scroll to reveal it
|
||||
if (isCE(n)) ceSetSel(n, s, e);
|
||||
else { try { n.setSelectionRange(s, e); } catch (_) {} }
|
||||
}
|
||||
@@ -329,6 +335,13 @@
|
||||
else if (x < c.x) hi = mid;
|
||||
else lo = mid + 1;
|
||||
}
|
||||
// The search lands on the boundary just RIGHT of x; round to the NEAREST boundary instead (native
|
||||
// getOffsetForPosition) so tapping the left half of a glyph doesn't advance the caret past it. Only
|
||||
// compare within the same line (skip when lo sits at a wrap, where lo-1 is on the previous row).
|
||||
if (lo > 0) {
|
||||
var cl = caretCoords(n, lo - 1), cr = caretCoords(n, lo);
|
||||
if (cl && cr && cl.top === cr.top && (x - cl.x) < (cr.x - x)) lo = lo - 1;
|
||||
}
|
||||
return lo;
|
||||
}
|
||||
// The focused field's bounding box in CSS px (toolbar anchor). When the selection is a bare caret it also
|
||||
@@ -337,10 +350,22 @@
|
||||
try {
|
||||
var b = n.getBoundingClientRect();
|
||||
var g = { l: b.left, t: b.top, r: b.right, b: b.bottom, sx: b.left, sb: b.bottom, ex: b.right, eb: b.bottom, vw: window.innerWidth };
|
||||
var sel = selOf(n);
|
||||
if (sel[0] === sel[1] && !isCE(n)) {
|
||||
var c = caretCoords(n, sel[0]);
|
||||
if (c) { g.cx = c.x; g.ct = c.top; g.cb = c.bottom; }
|
||||
if (!isCE(n)) {
|
||||
var sel = selOf(n);
|
||||
if (sel[0] === sel[1]) {
|
||||
// Bare caret → carry the caret rect so the host shows the insertion handle.
|
||||
var c = caretCoords(n, sel[0]);
|
||||
if (c) { g.cx = c.x; g.ct = c.top; g.cb = c.bottom; }
|
||||
} else {
|
||||
// Range → carry the start/end caret feet so the host shows draggable selection handles, and
|
||||
// tighten the box to the selected line span so the toolbar anchors above the selection.
|
||||
var cs = caretCoords(n, sel[0]), ce = caretCoords(n, sel[1]);
|
||||
if (cs && ce) {
|
||||
g.rng = true; // marks a real range so the host keeps these feet through Chrome's collapse fight
|
||||
g.sx = cs.x; g.sb = cs.bottom; g.ex = ce.x; g.eb = ce.bottom;
|
||||
g.t = Math.min(cs.top, ce.top); g.b = Math.max(cs.bottom, ce.bottom);
|
||||
}
|
||||
}
|
||||
}
|
||||
return g;
|
||||
} catch (_) { return null; }
|
||||
@@ -356,17 +381,40 @@
|
||||
// Last selection we either applied (applyState) or already reported, so the asynchronous
|
||||
// selectionchange our own setSel triggers doesn't echo back to the host as a fresh edit.
|
||||
var lastSel = null;
|
||||
// The live non-collapsed field range we protect from Chrome's off-window abandonment (see the
|
||||
// selectionchange handler). Tracked centrally so handle-extends and select-all keep it current.
|
||||
var lastFieldRange = null, lastFieldAt = -1;
|
||||
// When the selection last changed (Chrome-driven OR our own setSel). Forming/re-asserting a selection makes
|
||||
// the browser auto-scroll the field to reveal it, firing `scroll` events that are NOT a user content scroll.
|
||||
// The hide-on-scroll path uses this to ignore those: hiding the host overlays on a selection-reveal scroll
|
||||
// makes the handles/toolbar blink off-and-on every time a selection settles. See onAnyScroll.
|
||||
var lastSelActivityAt = -1;
|
||||
function sameSel(a, b){ return !!(a && b && a[0] === b[0] && a[1] === b[1]); }
|
||||
function noteSel(sel){
|
||||
lastSel = sel;
|
||||
if (el && !isCE(el)) {
|
||||
if (sel[0] !== sel[1]) { lastFieldRange = [sel[0], sel[1]]; lastFieldAt = perfNow(); }
|
||||
else { lastFieldRange = null; }
|
||||
}
|
||||
}
|
||||
function reportState(){
|
||||
if (!el) return;
|
||||
var sel = selOf(el);
|
||||
lastSel = sel;
|
||||
noteSel(sel);
|
||||
send({ type:'ime.state', text: valOf(el), selStart: sel[0], selEnd: sel[1], geom: fieldGeom(el) });
|
||||
}
|
||||
|
||||
document.addEventListener('focusin', function(e){
|
||||
if (isEditable(e.target)) {
|
||||
el = e.target; inComposition = false; lastSel = selOf(el); send(focusInfo(el));
|
||||
// Focusing a field clears any page-text selection in the browser. The page selectionchange handler is
|
||||
// muted while a field is focused (el is set), so it never emits the `active:false` — emit it here, or
|
||||
// the host's page handles + Copy bar linger ABOVE the field overlays (and, being z-above, steal the
|
||||
// caret/selection-handle drag so the caret can't be moved).
|
||||
if (pageSelText) { lastPageRange = null; sendPageSel(false, null); }
|
||||
// Cancel any in-flight scroll-hide from the page phase so the new field overlays aren't suppressed by a
|
||||
// stale `scrolling` state (its settle would otherwise keep getting re-armed by the field reveal-scrolls).
|
||||
scrolling = false; if (scrollTimer) { clearTimeout(scrollTimer); scrollTimer = null; }
|
||||
// The host shrinks the surface for the keyboard, but also nudge the field into view in case IME
|
||||
// insets aren't delivered (some hosts) so it never sits behind the keyboard.
|
||||
try { el.scrollIntoView({ block: 'center', inline: 'nearest' }); } catch (_) {}
|
||||
@@ -377,14 +425,71 @@
|
||||
}, true);
|
||||
// The page (its own JS, autofill) changed the field: resync the host keyboard's view of it.
|
||||
document.addEventListener('input', function(e){ if (e.target === el && !el.__nappletIme) reportState(); }, true);
|
||||
// Just mirror selection changes inside the focused editable to the host. The host owns the authoritative
|
||||
// selection, so it (not the shim) detects and re-asserts the collapse-to-endpoint that Chrome does when it
|
||||
// abandons a selection it can't present handles for in the embedded surface — see RemoteImeView.onPageState.
|
||||
// Mirror selection changes inside the focused editable to the host. Off-window Chrome abandons a field
|
||||
// selection by collapsing the caret to one of its endpoints; we re-assert it RIGHT HERE, synchronously,
|
||||
// the same way the page-text path does — reverting before the collapse paints, so it doesn't blink (the
|
||||
// old path round-tripped through the host EditText, leaving a visible collapsed frame each cycle). The
|
||||
// host's own re-assert in RemoteImeView.onPageState stays as a fallback for collapses we don't catch.
|
||||
document.addEventListener('selectionchange', function(){
|
||||
if (!el || el.__nappletIme) return;
|
||||
if (sameSel(selOf(el), lastSel)) return; // our own applyState/setSel echoing back
|
||||
if (!el || el.__nappletIme) return; // our own applyState/setSel
|
||||
lastSelActivityAt = perfNow(); // Chrome moved the selection → an imminent reveal-scroll isn't a user scroll
|
||||
var sel = selOf(el);
|
||||
if (sameSel(sel, lastSel)) return; // echo of what we just applied
|
||||
if (!isCE(el) && lastFieldRange && sel[0] === sel[1] &&
|
||||
(sel[0] === lastFieldRange[0] || sel[0] === lastFieldRange[1]) &&
|
||||
(perfNow() - lastFieldAt) < 1500) {
|
||||
// Chrome's off-window abandonment collapsed our live range to an endpoint → snap it back
|
||||
// synchronously (reverts before paint, so no blink) and keep the window open. The host already
|
||||
// holds this range, so we don't re-report (which would feed the slow round-trip loop).
|
||||
el.__nappletIme = true;
|
||||
setSel(el, lastFieldRange[0], lastFieldRange[1]);
|
||||
el.__nappletIme = false;
|
||||
lastSel = selOf(el);
|
||||
lastFieldAt = perfNow();
|
||||
return;
|
||||
}
|
||||
reportState();
|
||||
}, true);
|
||||
// Tap handling on the focused editable. A single tap collapses any selection to a caret at the tap point
|
||||
// and shows the insertion handle (native; off-window Chrome won't collapse-on-tap itself). A DOUBLE tap
|
||||
// selects the word — but `click` fires before `dblclick`, so instead of guessing with timing we DEFER the
|
||||
// collapse and let the real `dblclick` cancel it. This is robust to Chrome's own double-click timing
|
||||
// (a timing guess raced it and sometimes ate the word selection → "cursor jumps to end of word").
|
||||
var collapseTimer = null;
|
||||
function clearCollapse() { if (collapseTimer) { clearTimeout(collapseTimer); collapseTimer = null; } }
|
||||
document.addEventListener('dblclick', function(e){
|
||||
if (!el || isCE(el)) return;
|
||||
clearCollapse(); // a real double-tap → don't collapse; keep Chrome's word selection
|
||||
var s = selOf(el);
|
||||
if (s[0] !== s[1] && !sameSel(s, lastSel)) reportState(); // report the word only if not already sent
|
||||
}, true);
|
||||
document.addEventListener('click', function(e){
|
||||
if (!el || isCE(el) || e.target !== el) return;
|
||||
var sel = selOf(el);
|
||||
if (sel[0] !== sel[1]) {
|
||||
// Tap landed on a selection. Defer the collapse: if a dblclick follows (within the tap window) it
|
||||
// cancels this and the word stays selected; otherwise this fires and collapses to the tapped offset.
|
||||
var x = e.clientX, y = e.clientY;
|
||||
clearCollapse();
|
||||
collapseTimer = setTimeout(function(){
|
||||
collapseTimer = null;
|
||||
if (!el) return;
|
||||
var s = selOf(el);
|
||||
if (s[0] === s[1]) return; // already collapsed
|
||||
var off = offsetFromPoint(el, x, y);
|
||||
el.__nappletIme = true;
|
||||
setSel(el, off, off);
|
||||
el.__nappletIme = false;
|
||||
lastSel = selOf(el);
|
||||
reportState();
|
||||
send({ type:'ime.carettap', geom: fieldGeom(el) });
|
||||
}, 300);
|
||||
} else {
|
||||
// Tap on a bare caret → (re-)show the insertion handle. If a double-tap follows, dblclick selects the
|
||||
// word and supersedes this.
|
||||
send({ type:'ime.carettap', geom: fieldGeom(el) });
|
||||
}
|
||||
}, true);
|
||||
|
||||
function enter(n){
|
||||
if (!n) return;
|
||||
@@ -447,7 +552,7 @@
|
||||
}
|
||||
setSel(n, msg.selStart, msg.selEnd);
|
||||
if (!composingActive && inComposition) { inComposition = false; fireComp(n, 'compositionend', d.inserted || ''); }
|
||||
} finally { n.__nappletIme = false; lastSel = selOf(n); }
|
||||
} finally { n.__nappletIme = false; noteSel(selOf(n)); }
|
||||
}
|
||||
|
||||
// --- Page (non-editable) text selection re-hosting ---
|
||||
@@ -474,6 +579,7 @@
|
||||
}
|
||||
document.addEventListener('selectionchange', function(){
|
||||
if (el || pageReasserting) return; // selections inside an editable are handled above
|
||||
lastSelActivityAt = perfNow(); // page selection moved → an imminent reveal-scroll isn't a user scroll
|
||||
var s = window.getSelection();
|
||||
if (s && s.rangeCount && !s.isCollapsed) {
|
||||
var r = s.getRangeAt(0);
|
||||
@@ -510,10 +616,100 @@
|
||||
} catch (_) {}
|
||||
}
|
||||
|
||||
// Word-granularity snapping (native: dragging a word selection's handle extends a word at a time). The
|
||||
// end handle snaps to the end of the word at/after the offset; the start handle to the start of the word
|
||||
// at/before it. Whitespace between words extends to the adjacent word so you never stop mid-gap.
|
||||
function isWordChar(c){ return c != null && /\S/.test(c); }
|
||||
function wordEndAt(text, off){
|
||||
var i = off;
|
||||
while (i < text.length && !isWordChar(text[i])) i++;
|
||||
while (i < text.length && isWordChar(text[i])) i++;
|
||||
return i;
|
||||
}
|
||||
function wordStartAt(text, off){
|
||||
var i = off;
|
||||
while (i > 0 && !isWordChar(text[i - 1])) i--;
|
||||
while (i > 0 && isWordChar(text[i - 1])) i--;
|
||||
return i;
|
||||
}
|
||||
|
||||
// Host drag of an in-field selection handle: move the dragged edge to the offset under (x,y) CSS px,
|
||||
// keeping the other edge anchored. Snaps to word boundaries (the selection was made by word), and is
|
||||
// clamped so the dragged edge can't cross the anchor (no inversion).
|
||||
function fieldExtend(edge, x, y){
|
||||
if (!el || isCE(el)) return;
|
||||
try {
|
||||
var off = offsetFromPoint(el, x, y);
|
||||
var text = valOf(el);
|
||||
var sel = selOf(el);
|
||||
var s, e;
|
||||
if (edge === 'start') {
|
||||
e = sel[1];
|
||||
s = Math.min(wordStartAt(text, off), e);
|
||||
} else {
|
||||
s = sel[0];
|
||||
e = Math.max(wordEndAt(text, off), s);
|
||||
}
|
||||
el.__nappletIme = true;
|
||||
setSel(el, s, e);
|
||||
el.__nappletIme = false;
|
||||
lastSel = selOf(el);
|
||||
reportState();
|
||||
} catch (_) {}
|
||||
}
|
||||
|
||||
// While the page scrolls, host-drawn selection UI (toolbar + handles) would float at stale positions, so
|
||||
// the host hides it on scroll-start and we re-report fresh geometry on scroll-idle so it reappears in the
|
||||
// right place — like Android. Only signal when there's a selection to hide (a field range or page text).
|
||||
var scrolling = false, scrollTimer = null, autoScrolling = false;
|
||||
// How long after a selection change a scroll is treated as the browser's auto-reveal of that selection
|
||||
// (not a user content scroll). Generous enough to catch the reveal-scroll that fires a frame or two later.
|
||||
var SCROLL_SEL_GUARD_MS = 350;
|
||||
function hasSelectionUi(){ return !!pageSelText || !!(el && (function(s){ return s[0] !== s[1]; })(selOf(el))); }
|
||||
function onAnyScroll(){
|
||||
if (autoScrolling) return; // our own drag-to-edge auto-scroll: keep the overlays up, don't hide them
|
||||
if (!hasSelectionUi()) return;
|
||||
if ((perfNow() - lastSelActivityAt) < SCROLL_SEL_GUARD_MS) {
|
||||
// The browser auto-scrolled to reveal a just-changed selection (forming/re-asserting a range scrolls a
|
||||
// textarea). That's not a user content scroll: hiding here would blink the host overlays off-and-on every
|
||||
// time a selection settles. Reposition them in place instead (geometry shifted by the reveal-scroll).
|
||||
// Crucially we do NOT touch the hide-on-scroll timer: if a real scroll-hide is somehow active, these
|
||||
// reveal-scrolls must not keep re-arming it (that would leave the overlays hidden indefinitely).
|
||||
if (el) reportState();
|
||||
else { var sr = window.getSelection(); if (sr && sr.rangeCount && !sr.isCollapsed) sendPageSel(true, sr.getRangeAt(0)); }
|
||||
return;
|
||||
}
|
||||
if (!scrolling) { scrolling = true; send({ type:'ime.scroll', active: true }); }
|
||||
if (scrollTimer) clearTimeout(scrollTimer);
|
||||
scrollTimer = setTimeout(function(){
|
||||
scrolling = false; scrollTimer = null;
|
||||
// Refresh geometry FIRST (so overlays reposition), then tell the host to show them again.
|
||||
if (el) reportState();
|
||||
else { var s = window.getSelection(); if (s && s.rangeCount && !s.isCollapsed) sendPageSel(true, s.getRangeAt(0)); }
|
||||
send({ type:'ime.scroll', active: false });
|
||||
}, 150);
|
||||
}
|
||||
document.addEventListener('scroll', onAnyScroll, true); // capture: any scroller, not just the document
|
||||
|
||||
window.__nappletImeHandle = function(msg){
|
||||
if (msg.type === 'ime.set') applyState(msg);
|
||||
else if (msg.type === 'ime.action') enter(el);
|
||||
else if (msg.type === 'ime.pageextend') pageExtend(msg.edge, msg.x, msg.y);
|
||||
else if (msg.type === 'ime.fieldextend') fieldExtend(msg.edge, msg.x, msg.y);
|
||||
else if (msg.type === 'ime.autoscroll') {
|
||||
// Host drag of a handle near the surface's top/bottom edge → scroll the content (the textarea if it
|
||||
// scrolls, else the page) so the selection can keep extending, then re-report geometry so the
|
||||
// overlays follow. Flagged so our own scroll doesn't trip the hide-on-scroll path above.
|
||||
var dy = msg.dy || 0;
|
||||
autoScrolling = true;
|
||||
try {
|
||||
if (el && (el.tagName || '').toUpperCase() === 'TEXTAREA') el.scrollTop += dy;
|
||||
window.scrollBy(0, dy);
|
||||
} catch (_) {}
|
||||
if (el) reportState();
|
||||
else { var s = window.getSelection(); if (s && s.rangeCount && !s.isCollapsed) sendPageSel(true, s.getRangeAt(0)); }
|
||||
setTimeout(function(){ autoScrolling = false; }, 0);
|
||||
}
|
||||
else if (msg.type === 'ime.caretmove') {
|
||||
if (el && !isCE(el)) {
|
||||
var off = offsetFromPoint(el, msg.x, msg.y);
|
||||
|
||||
+28
@@ -77,6 +77,34 @@ object NappletBrowserContract {
|
||||
*/
|
||||
const val MSG_CONSOLE_LOG = 11
|
||||
|
||||
/**
|
||||
* Client → provider: capture a magnified slice of the live page for the native-style selection loupe.
|
||||
* Host-side `PixelCopy` of the sandboxed surface returns `ERROR_SOURCE_NO_DATA` (the WebView pixels live
|
||||
* in a child SurfaceControl the host never draws into), so we capture INSIDE the provider — where the
|
||||
* WebView is a real in-window view. Carries [KEY_MAG_X]/[KEY_MAG_Y] (surface px center),
|
||||
* [KEY_MAG_BOX_W]/[KEY_MAG_BOX_H] (source rectangle, px), [KEY_MAG_ZOOM], and [KEY_MAG_REQ_T] (the
|
||||
* client's `nanoTime` send stamp, echoed back so the client can drop stale out-of-order frames).
|
||||
*/
|
||||
const val MSG_MAGNIFIER_REQUEST = 12
|
||||
|
||||
/**
|
||||
* Provider → client: the captured loupe frame. Carries [KEY_MAG_BYTES] (a PNG well under the Binder
|
||||
* limit), [KEY_MAG_W]/[KEY_MAG_H], [KEY_MAG_CAPTURE_MS] (provider-side draw+encode time), and the echoed
|
||||
* [KEY_MAG_REQ_T] so the client matches it to its request / drops stale frames.
|
||||
*/
|
||||
const val MSG_MAGNIFIER_FRAME = 13
|
||||
|
||||
const val KEY_MAG_X = "magX"
|
||||
const val KEY_MAG_Y = "magY"
|
||||
const val KEY_MAG_BOX_W = "magBoxW"
|
||||
const val KEY_MAG_BOX_H = "magBoxH"
|
||||
const val KEY_MAG_ZOOM = "magZoom"
|
||||
const val KEY_MAG_REQ_T = "magReqT"
|
||||
const val KEY_MAG_BYTES = "magBytes"
|
||||
const val KEY_MAG_W = "magW"
|
||||
const val KEY_MAG_H = "magH"
|
||||
const val KEY_MAG_CAPTURE_MS = "magCaptureMs"
|
||||
|
||||
const val KEY_IS_LOADING = "isLoading"
|
||||
const val KEY_LOAD_FAILED = "loadFailed"
|
||||
|
||||
|
||||
+49
@@ -25,6 +25,8 @@ import android.content.ComponentName
|
||||
import android.content.Context
|
||||
import android.content.Intent
|
||||
import android.content.ServiceConnection
|
||||
import android.graphics.Bitmap
|
||||
import android.graphics.Canvas
|
||||
import android.net.Uri
|
||||
import android.os.Build
|
||||
import android.os.Bundle
|
||||
@@ -33,6 +35,7 @@ import android.os.IBinder
|
||||
import android.os.Looper
|
||||
import android.os.Message
|
||||
import android.os.Messenger
|
||||
import android.os.SystemClock
|
||||
import android.util.Log
|
||||
import android.webkit.ConsoleMessage
|
||||
import android.webkit.WebChromeClient
|
||||
@@ -51,6 +54,7 @@ import androidx.webkit.WebViewFeature
|
||||
import com.vitorpamplona.amethyst.commons.browser.OmniboxInput
|
||||
import com.vitorpamplona.amethyst.commons.napplet.NappletWebContract
|
||||
import org.json.JSONObject
|
||||
import java.io.ByteArrayOutputStream
|
||||
|
||||
/**
|
||||
* Provider for the **embedded** in-app browser. Runs in the keyless `:napplet` process: it hosts the
|
||||
@@ -190,11 +194,56 @@ class NappletBrowserService : Service() {
|
||||
// reload the one the user toggled.
|
||||
applyWebViewProxy(if (tab.useTor) tab.proxyPort else -1) { tab.webView?.reload() }
|
||||
}
|
||||
NappletBrowserContract.MSG_MAGNIFIER_REQUEST -> onMagnifierRequest(msg)
|
||||
else -> return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// One reusable output bitmap per tab would be ideal, but loupe size is fixed per drag; createBitmap each
|
||||
// frame is cheap next to the draw. Source rect is in view px (== surface px, the SCVH is 1:1).
|
||||
private fun onMagnifierRequest(msg: Message) {
|
||||
val tab = tabFor(msg) ?: return
|
||||
val wv = tab.webView ?: return
|
||||
val data = msg.data ?: return
|
||||
val cx = data.getFloat(NappletBrowserContract.KEY_MAG_X)
|
||||
val cy = data.getFloat(NappletBrowserContract.KEY_MAG_Y)
|
||||
val boxW = data.getInt(NappletBrowserContract.KEY_MAG_BOX_W, 150).coerceIn(16, 1024)
|
||||
val boxH = data.getInt(NappletBrowserContract.KEY_MAG_BOX_H, 84).coerceIn(16, 1024)
|
||||
val zoom = data.getFloat(NappletBrowserContract.KEY_MAG_ZOOM, 1.6f).coerceIn(1f, 4f)
|
||||
val reqT = data.getLong(NappletBrowserContract.KEY_MAG_REQ_T)
|
||||
|
||||
val outW = (boxW * zoom).toInt().coerceAtLeast(1)
|
||||
val outH = (boxH * zoom).toInt().coerceAtLeast(1)
|
||||
val t0 = SystemClock.elapsedRealtimeNanos()
|
||||
val bitmap = Bitmap.createBitmap(outW, outH, Bitmap.Config.ARGB_8888)
|
||||
val canvas = Canvas(bitmap)
|
||||
canvas.drawColor(tab.bgColor)
|
||||
// Map the source rect (centered on cx,cy in view px) into the zoomed output bitmap.
|
||||
canvas.scale(zoom, zoom)
|
||||
canvas.translate(-(cx - boxW / 2f), -(cy - boxH / 2f))
|
||||
wv.draw(canvas)
|
||||
|
||||
val baos = ByteArrayOutputStream()
|
||||
bitmap.compress(Bitmap.CompressFormat.PNG, 100, baos)
|
||||
val bytes = baos.toByteArray()
|
||||
bitmap.recycle()
|
||||
val captureMs = (SystemClock.elapsedRealtimeNanos() - t0) / 1_000_000.0
|
||||
|
||||
val reply =
|
||||
Message.obtain(null, NappletBrowserContract.MSG_MAGNIFIER_FRAME).apply {
|
||||
this.data =
|
||||
Bundle().apply {
|
||||
putByteArray(NappletBrowserContract.KEY_MAG_BYTES, bytes)
|
||||
putInt(NappletBrowserContract.KEY_MAG_W, outW)
|
||||
putInt(NappletBrowserContract.KEY_MAG_H, outH)
|
||||
putDouble(NappletBrowserContract.KEY_MAG_CAPTURE_MS, captureMs)
|
||||
putLong(NappletBrowserContract.KEY_MAG_REQ_T, reqT)
|
||||
}
|
||||
}
|
||||
runCatching { tab.clientMessenger?.send(reply) }
|
||||
}
|
||||
|
||||
/** Builds the SandboxedUiAdapter for [tab] and ships its cross-process handle (coreLibInfo) to the client. */
|
||||
private fun replyWithAdapter(tab: BrowserTab) {
|
||||
val adapter = NappletBrowserUiAdapter(this, tab.sessionId)
|
||||
|
||||
+23
@@ -84,6 +84,29 @@ object NappletEmbedContract {
|
||||
*/
|
||||
const val MSG_LOAD_STATE = 15
|
||||
|
||||
/**
|
||||
* Client → provider: capture a magnified slice of the live page for the selection loupe. Host-side
|
||||
* `PixelCopy` of the sandbox surface returns `ERROR_SOURCE_NO_DATA` (the WebView pixels live in a child
|
||||
* SurfaceControl), so capture happens here, in the provider, where the WebView is a real in-window view.
|
||||
* Carries [KEY_MAG_X]/[KEY_MAG_Y] (surface px center), [KEY_MAG_BOX_W]/[KEY_MAG_BOX_H] (source rect, px),
|
||||
* [KEY_MAG_ZOOM], and [KEY_MAG_REQ_T] (echoed send stamp). Mirrors the browser path.
|
||||
*/
|
||||
const val MSG_MAGNIFIER_REQUEST = 16
|
||||
|
||||
/** Provider → client: the captured loupe frame — [KEY_MAG_BYTES] PNG, [KEY_MAG_W]/[KEY_MAG_H], [KEY_MAG_CAPTURE_MS], echoed [KEY_MAG_REQ_T]. */
|
||||
const val MSG_MAGNIFIER_FRAME = 17
|
||||
|
||||
const val KEY_MAG_X = "magX"
|
||||
const val KEY_MAG_Y = "magY"
|
||||
const val KEY_MAG_BOX_W = "magBoxW"
|
||||
const val KEY_MAG_BOX_H = "magBoxH"
|
||||
const val KEY_MAG_ZOOM = "magZoom"
|
||||
const val KEY_MAG_REQ_T = "magReqT"
|
||||
const val KEY_MAG_BYTES = "magBytes"
|
||||
const val KEY_MAG_W = "magW"
|
||||
const val KEY_MAG_H = "magH"
|
||||
const val KEY_MAG_CAPTURE_MS = "magCaptureMs"
|
||||
|
||||
const val KEY_CORE_LIB_INFO = "coreLibInfo"
|
||||
const val KEY_CAN_GO_BACK = "canGoBack"
|
||||
const val KEY_IS_LOADING = "isLoading"
|
||||
|
||||
+51
@@ -25,6 +25,8 @@ import android.content.ComponentName
|
||||
import android.content.Context
|
||||
import android.content.Intent
|
||||
import android.content.ServiceConnection
|
||||
import android.graphics.Bitmap
|
||||
import android.graphics.Canvas
|
||||
import android.net.Uri
|
||||
import android.os.Build
|
||||
import android.os.Bundle
|
||||
@@ -33,6 +35,7 @@ import android.os.IBinder
|
||||
import android.os.Looper
|
||||
import android.os.Message
|
||||
import android.os.Messenger
|
||||
import android.os.SystemClock
|
||||
import android.util.Log
|
||||
import android.view.View
|
||||
import android.webkit.WebResourceError
|
||||
@@ -57,6 +60,7 @@ import com.vitorpamplona.quartz.nip01Core.core.toHexKey
|
||||
import com.vitorpamplona.quartz.nip5aStaticWebsites.tags.PathTag
|
||||
import com.vitorpamplona.quartz.utils.sha256.sha256
|
||||
import org.json.JSONObject
|
||||
import java.io.ByteArrayOutputStream
|
||||
import java.util.concurrent.Executor
|
||||
|
||||
/**
|
||||
@@ -183,6 +187,7 @@ class NappletHostService : Service() {
|
||||
val payload = msg.data?.getString(NappletEmbedContract.KEY_IME_PAYLOAD) ?: return true
|
||||
tab.bridgeReplyProxy?.postMessage(payload)
|
||||
}
|
||||
NappletEmbedContract.MSG_MAGNIFIER_REQUEST -> onMagnifierRequest(msg)
|
||||
else -> return false
|
||||
}
|
||||
return true
|
||||
@@ -229,6 +234,52 @@ class NappletHostService : Service() {
|
||||
return tab
|
||||
}
|
||||
|
||||
/**
|
||||
* Draw a zoomed slice of the live WebView (the loupe content) and ship it back as a PNG. The WebView is a
|
||||
* real in-window view here in the provider, so its software draw renders real DOM pixels — unlike host-side
|
||||
* `PixelCopy` of the sandbox surface. Mirror of the browser path. Runs on the main looper (`WebView.draw`).
|
||||
*/
|
||||
private fun onMagnifierRequest(msg: Message) {
|
||||
val tab = tabFor(msg) ?: return
|
||||
val wv = tab.webView ?: return
|
||||
val data = msg.data ?: return
|
||||
val cx = data.getFloat(NappletEmbedContract.KEY_MAG_X)
|
||||
val cy = data.getFloat(NappletEmbedContract.KEY_MAG_Y)
|
||||
val boxW = data.getInt(NappletEmbedContract.KEY_MAG_BOX_W, 150).coerceIn(16, 1024)
|
||||
val boxH = data.getInt(NappletEmbedContract.KEY_MAG_BOX_H, 84).coerceIn(16, 1024)
|
||||
val zoom = data.getFloat(NappletEmbedContract.KEY_MAG_ZOOM, 1.6f).coerceIn(1f, 4f)
|
||||
val reqT = data.getLong(NappletEmbedContract.KEY_MAG_REQ_T)
|
||||
|
||||
val outW = (boxW * zoom).toInt().coerceAtLeast(1)
|
||||
val outH = (boxH * zoom).toInt().coerceAtLeast(1)
|
||||
val t0 = SystemClock.elapsedRealtimeNanos()
|
||||
val bitmap = Bitmap.createBitmap(outW, outH, Bitmap.Config.ARGB_8888)
|
||||
val canvas = Canvas(bitmap)
|
||||
canvas.drawColor(tab.bgColor)
|
||||
canvas.scale(zoom, zoom)
|
||||
canvas.translate(-(cx - boxW / 2f), -(cy - boxH / 2f))
|
||||
wv.draw(canvas)
|
||||
|
||||
val baos = ByteArrayOutputStream()
|
||||
bitmap.compress(Bitmap.CompressFormat.PNG, 100, baos)
|
||||
val bytes = baos.toByteArray()
|
||||
bitmap.recycle()
|
||||
val captureMs = (SystemClock.elapsedRealtimeNanos() - t0) / 1_000_000.0
|
||||
|
||||
val reply =
|
||||
Message.obtain(null, NappletEmbedContract.MSG_MAGNIFIER_FRAME).apply {
|
||||
this.data =
|
||||
Bundle().apply {
|
||||
putByteArray(NappletEmbedContract.KEY_MAG_BYTES, bytes)
|
||||
putInt(NappletEmbedContract.KEY_MAG_W, outW)
|
||||
putInt(NappletEmbedContract.KEY_MAG_H, outH)
|
||||
putDouble(NappletEmbedContract.KEY_MAG_CAPTURE_MS, captureMs)
|
||||
putLong(NappletEmbedContract.KEY_MAG_REQ_T, reqT)
|
||||
}
|
||||
}
|
||||
runCatching { tab.clientMessenger?.send(reply) }
|
||||
}
|
||||
|
||||
/** Builds the SandboxedUiAdapter for [tab] and ships its cross-process handle (coreLibInfo) to the client. */
|
||||
private fun replyWithAdapter(tab: NappletTab) {
|
||||
val adapter = NappletHostUiAdapter(this, tab.sessionId)
|
||||
|
||||
@@ -16,6 +16,8 @@
|
||||
<h2>Textarea</h2>
|
||||
<textarea id="ta" rows="2" placeholder="type here">hello world</textarea>
|
||||
<div id="log"></div>
|
||||
<!-- Tall spacer so the page scrolls — lets us verify the host hides selection UI during scroll. -->
|
||||
<div style="height:2000px; padding-top:16px; color:#888;">scroll region — drag up/down here</div>
|
||||
<script>
|
||||
(function(){
|
||||
var t0 = performance.now();
|
||||
|
||||
Reference in New Issue
Block a user