mirror of
https://github.com/vitorpamplona/amethyst.git
synced 2026-07-30 19:46:17 +00:00
fix(desktop,media): address PR #3175 review findings
Correctness fixes in GlobalMediaPlayer.kt
- snapshotFlow { hasMedia } collector for initial seek used `return@collect`
which only exits the lambda; the collector kept running and each
subsequent playVideo() call accumulated a live collector that would
re-fire a stale seekTo() on the wrong media. Replaced with `Flow.first`
which terminates the collection cleanly.
- playVideo()/playAudio() reset the public MediaPlaybackState to
volume=100/isMuted=false on a new URL, but the kdroidFilter player
retains its `volume` across openUri(); muting one track and starting a
new one left the engine silent while the UI showed unmuted. Reset
`player.volume = 1f` to match the public state.
- ensureVideoPlayer()/ensureAudioPlayer() called createVideoPlayerState()
synchronously from the Compose getter; if native init throws (missing
GStreamer on Linux, broken NativeLibraryLoader extraction) the whole
window would crash. Wrapped in runCatching and changed
activeVideoPlayerState to nullable. Consumers in DesktopVideoPlayer
and GlobalFullscreenOverlay handle the null path by rendering the
thumbnail / blank backdrop respectively; playVideo()/playAudio()
surface "Video playback unavailable" through the existing
errorReason -> PlaybackErrorMessage path.
Crash mitigation (kdroidFilter 0.10.0 UAF in MacVideoPlayerSurface)
- NowPlayingBar previously mounted a SECOND VideoPlayerSurface against
the same VideoPlayerState while the feed card was already mounting
one, doubling the draw rate against the shared frame bitmap and
widening the UAF window in MacVideoPlayerSurface's RasterFromBitmap
path. Mini-preview now renders the cached thumbnail (or the music
icon fallback). 0.10.1 contains an upstream fix
("recover video playback after composition removal") but is not yet
on Maven Central — single-surface mounting is the only mitigation
we can ship today.
VideoThumbnailCache.kt
- Truncated-download cache poisoning: when an origin ignored the
Range: header and returned HTTP 200 with the full body, we capped
the copy at MAX_THUMB_BYTES and persisted the truncated file
forever. Subsequent thumbnail attempts hit the broken cache file
and re-failed JCodec/ffmpeg every time. Tag download results with
whether the server actually returned 206; on 200, extract from the
temp file and delete it (no persistent cache hit).
- Tor bypass: replaced the bare OkHttpClient with
DesktopHttpClient.currentClient() so thumbnail fetches respect the
user's Tor preference (fail-closed when Tor is expected but
bootstrapping).
- ffmpeg version probe leaked the process on hang: now drains stdout
to DISCARD and calls destroyForcibly() on timeout.
- Frame-extract ffmpeg subprocess could deadlock on a chatty stderr
pipe: redirectError(DISCARD) so we never wait on stderr; a finally
block destroys the process if anything leaked through the timeout.
CI workflow cleanup
- Removed vlc-setup download cache + pre-fetch steps from
build.yml and smoke-test-desktop.yml. They were targeting an
ir.mahozad.vlc-setup plugin we no longer apply, so they wasted
~minutes of CI time per leg and tied the build to videolan.org
reachability for no reason.
- Trimmed create-release.yml's stale VLC-plugins justification on
the linuxdeploy-vs-appimagetool comment.
.gitignore + missing per-OS ffmpeg READMEs
- The pre-PR rules blanket-ignored desktopApp/src/jvmMain/appResources/{linux,macos,windows}/
so the LGPL FFmpeg drop-in slot READMEs created in 704f4f44e never
reached the commit. Refined the ignore rules to keep stale vlc/ workspace
trees out of git (still ignored) while explicitly tracking the
ffmpeg/README.md drop-in slot under each OS. The READMEs document the
recommended LGPL build source per OS for the bundled-FFmpeg packaging
path.
Verified on macOS arm64:
./gradlew :desktopApp:compileKotlin BUILD SUCCESSFUL
./gradlew :desktopApp:test BUILD SUCCESSFUL
./gradlew :desktopApp:spotlessApply clean
Refs PR #3175 review by @davotoula.
This commit is contained in:
@@ -79,82 +79,6 @@ jobs:
|
||||
with:
|
||||
cache-read-only: ${{ github.ref != 'refs/heads/main' }}
|
||||
|
||||
# Cache vlc-setup plugin downloads (VLC + UPX archives) keyed on the
|
||||
# versions pinned in desktopApp/build.gradle.kts. Each OS gets its own
|
||||
# cache namespace because the plugin downloads platform-specific archives.
|
||||
# On a hit the vlcDownload / upxDownload tasks are up-to-date and we
|
||||
# never touch get.videolan.org; on a miss (version bump or new runner)
|
||||
# we fall back to fetching, which is what the in-build retry budget
|
||||
# exists for.
|
||||
- name: Cache vlc-setup downloads
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ~/.gradle/vlcSetup
|
||||
key: vlcsetup-${{ runner.os }}-${{ hashFiles('desktopApp/build.gradle.kts') }}
|
||||
restore-keys: |
|
||||
vlcsetup-${{ runner.os }}-
|
||||
|
||||
# Pre-fetch VLC + UPX archives into ~/.gradle/vlcSetup before invoking
|
||||
# Gradle. The vlc-setup plugin (ir.mahozad.vlc-setup 0.1.0) writes its
|
||||
# downloads to ${gradleUserHomeDir}/vlcSetup/ and sets overwrite(false),
|
||||
# so an existing file there makes vlcDownload / upxDownload up-to-date
|
||||
# and Gradle never opens a socket to videolan.org.
|
||||
#
|
||||
# Why curl instead of relying on de.undercouch.gradle.tasks.download:
|
||||
# curl --retry-all-errors with a long --retry-max-time tolerates a
|
||||
# sustained get.videolan.org outage far better than the plugin's inner
|
||||
# retry budget (retries(4) + 5min readTimeout in build.gradle.kts),
|
||||
# which has been hitting SocketTimeoutException on Windows runners.
|
||||
#
|
||||
# Cache hit: the file is already on disk, fetch() short-circuits, this
|
||||
# step takes <1s. Cache miss: curl downloads with aggressive retries,
|
||||
# populating the cache for the next run.
|
||||
#
|
||||
# Versions are pinned to match desktopApp/build.gradle.kts (vlcVersion
|
||||
# = 3.0.20) and the vlc-setup extension default (upxVersion = 4.2.4).
|
||||
# NOTE: vlcVersion lags behind upstream VLC because the Linux plugins on
|
||||
# Maven Central (ir.mahozad:vlc-plugins-linux) are only published for
|
||||
# 3.0.20 / 3.0.20-2. Bump only after the Maven artifact is republished.
|
||||
# macOS does not download UPX — UPX cannot compress .dylib files.
|
||||
- name: Pre-fetch VLC + UPX archives
|
||||
env:
|
||||
VLC_VERSION: "3.0.20"
|
||||
UPX_VERSION: "4.2.4"
|
||||
run: |
|
||||
set -euo pipefail
|
||||
DEST="$HOME/.gradle/vlcSetup"
|
||||
mkdir -p "$DEST"
|
||||
fetch() {
|
||||
local url="$1" out="$2"
|
||||
if [[ -s "$out" ]]; then
|
||||
echo "cached: $out"
|
||||
return 0
|
||||
fi
|
||||
echo "fetching: $url"
|
||||
curl -fL --retry 10 --retry-delay 5 --retry-all-errors \
|
||||
--retry-max-time 900 --connect-timeout 30 \
|
||||
-o "$out.part" "$url"
|
||||
mv "$out.part" "$out"
|
||||
}
|
||||
case "${{ runner.os }}" in
|
||||
Windows)
|
||||
fetch "https://get.videolan.org/vlc/${VLC_VERSION}/win64/vlc-${VLC_VERSION}-win64.zip" \
|
||||
"$DEST/vlc-${VLC_VERSION}.zip"
|
||||
fetch "https://github.com/upx/upx/releases/download/v${UPX_VERSION}/upx-${UPX_VERSION}-win64.zip" \
|
||||
"$DEST/upx-${UPX_VERSION}.zip"
|
||||
;;
|
||||
Linux)
|
||||
fetch "https://repo1.maven.org/maven2/ir/mahozad/vlc-plugins-linux/${VLC_VERSION}/vlc-plugins-linux-${VLC_VERSION}.jar" \
|
||||
"$DEST/vlc-${VLC_VERSION}.jar"
|
||||
fetch "https://github.com/upx/upx/releases/download/v${UPX_VERSION}/upx-${UPX_VERSION}-amd64_linux.tar.xz" \
|
||||
"$DEST/upx-${UPX_VERSION}.tar.xz"
|
||||
;;
|
||||
macOS)
|
||||
fetch "https://get.videolan.org/vlc/${VLC_VERSION}/macosx/vlc-${VLC_VERSION}-universal.dmg" \
|
||||
"$DEST/vlc-${VLC_VERSION}.dmg"
|
||||
;;
|
||||
esac
|
||||
|
||||
# Compose UI smoke test (DesktopLaunchSmokeTest) uses Skiko which needs
|
||||
# a display server on Linux. xvfb provides a virtual framebuffer.
|
||||
- name: Install xvfb (Linux)
|
||||
|
||||
@@ -23,9 +23,9 @@ env:
|
||||
# Single source of truth in scripts/asset-name.sh.
|
||||
# appimagetool pinned release — bump via Dependabot, verify SHA256 via env var below.
|
||||
# We used to use linuxdeploy here, but it auto-walks the AppDir with ldd to
|
||||
# bundle deps — that fights jpackage's self-contained JRE (libjvm.so RPATH
|
||||
# mismatch) and the UPX-compressed VLC plugins. appimagetool only embeds the
|
||||
# AppDir as-is, which is what we actually want.
|
||||
# bundle deps — that fights jpackage's self-contained JRE (libjvm.so has
|
||||
# $ORIGIN RPATH so ldd can't resolve it standalone). appimagetool only
|
||||
# embeds the AppDir as-is, which is what we actually want.
|
||||
APPIMAGETOOL_URL: https://github.com/AppImage/appimagetool/releases/download/1.9.0/appimagetool-x86_64.AppImage
|
||||
APPIMAGETOOL_SHA256: 46fdd785094c7f6e545b61afcfb0f3d98d8eab243f644b4b17698c01d06083d1
|
||||
|
||||
|
||||
@@ -68,36 +68,6 @@ jobs:
|
||||
with:
|
||||
cache-read-only: true
|
||||
|
||||
- name: Cache vlc-setup downloads
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ~/.gradle/vlcSetup
|
||||
key: vlcsetup-Linux-${{ hashFiles('desktopApp/build.gradle.kts') }}
|
||||
restore-keys: |
|
||||
vlcsetup-Linux-
|
||||
|
||||
- name: Pre-fetch VLC + UPX archives
|
||||
env:
|
||||
VLC_VERSION: "3.0.20"
|
||||
UPX_VERSION: "4.2.4"
|
||||
run: |
|
||||
set -euo pipefail
|
||||
DEST="$HOME/.gradle/vlcSetup"
|
||||
mkdir -p "$DEST"
|
||||
fetch() {
|
||||
local url="$1" out="$2"
|
||||
if [[ -s "$out" ]]; then echo "cached: $out"; return 0; fi
|
||||
echo "fetching: $url"
|
||||
curl -fL --retry 10 --retry-delay 5 --retry-all-errors \
|
||||
--retry-max-time 900 --connect-timeout 30 \
|
||||
-o "$out.part" "$url"
|
||||
mv "$out.part" "$out"
|
||||
}
|
||||
fetch "https://repo1.maven.org/maven2/ir/mahozad/vlc-plugins-linux/${VLC_VERSION}/vlc-plugins-linux-${VLC_VERSION}.jar" \
|
||||
"$DEST/vlc-${VLC_VERSION}.jar"
|
||||
fetch "https://github.com/upx/upx/releases/download/v${UPX_VERSION}/upx-${UPX_VERSION}-amd64_linux.tar.xz" \
|
||||
"$DEST/upx-${UPX_VERSION}.tar.xz"
|
||||
|
||||
- name: Install xvfb + packaging deps
|
||||
run: sudo apt-get update && sudo apt-get install -y xvfb fakeroot
|
||||
|
||||
|
||||
+14
-4
@@ -161,10 +161,20 @@ TASKS.md
|
||||
.claude/settings.local.json
|
||||
.claude/scheduled_tasks.lock
|
||||
|
||||
# Downloaded VLC binaries (vlc-setup plugin)
|
||||
desktopApp/src/jvmMain/appResources/linux/
|
||||
desktopApp/src/jvmMain/appResources/macos/
|
||||
desktopApp/src/jvmMain/appResources/windows/
|
||||
# Per-OS appResources slots — historically the ir.mahozad.vlc-setup plugin
|
||||
# populated these with VLC binaries (no longer used; superseded by
|
||||
# kdroidFilter ComposeMediaPlayer). We still ignore the directory contents
|
||||
# by default to keep stale workspaces from accidentally bundling old VLC
|
||||
# trees into local packages, but explicitly track the ffmpeg/README.md
|
||||
# drop-in slot for the LGPL FFmpeg binaries used by VideoThumbnailCache.
|
||||
desktopApp/src/jvmMain/appResources/linux/*
|
||||
desktopApp/src/jvmMain/appResources/macos/*
|
||||
desktopApp/src/jvmMain/appResources/windows/*
|
||||
!desktopApp/src/jvmMain/appResources/linux/ffmpeg/
|
||||
!desktopApp/src/jvmMain/appResources/macos/ffmpeg/
|
||||
!desktopApp/src/jvmMain/appResources/windows/ffmpeg/
|
||||
desktopApp/src/jvmMain/appResources/*/ffmpeg/*
|
||||
!desktopApp/src/jvmMain/appResources/*/ffmpeg/README.md
|
||||
|
||||
# CI-fetched AppImage tooling (downloaded by create-release workflow; not committed)
|
||||
desktopApp/packaging/appimage/appimagetool-x86_64.AppImage
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
# Linux bundled FFmpeg
|
||||
|
||||
The DEB/RPM/AppImage packages **do not bundle FFmpeg** — they declare a
|
||||
runtime dependency on system FFmpeg/GStreamer instead, which keeps the
|
||||
binary's SPDX cleaner and reduces package size.
|
||||
|
||||
The Flatpak manifest similarly relies on `org.freedesktop.Platform 24.08`
|
||||
which ships an LGPL GStreamer + FFmpeg; thumbnail extraction falls through
|
||||
to the host FFmpeg if installed.
|
||||
|
||||
If you want a self-contained Linux build (e.g. AppImage with no host deps),
|
||||
drop a static LGPL `ffmpeg` binary here. johnvansickle.com publishes LGPL
|
||||
"release" builds for x86_64 and aarch64. Source: https://johnvansickle.com/ffmpeg/
|
||||
|
||||
The `VideoThumbnailCache` will prefer system `ffmpeg` on `$PATH` first, then
|
||||
fall through to this bundled binary.
|
||||
@@ -0,0 +1,28 @@
|
||||
# macOS bundled FFmpeg
|
||||
|
||||
Drop a single universal (arm64 + x86_64) LGPL FFmpeg binary here as `ffmpeg`
|
||||
(no extension) with `+x` permission. Used by `VideoThumbnailCache` for
|
||||
non-H.264 / non-faststart thumbnail extraction (HEVC, VP9, AV1, HLS).
|
||||
|
||||
## Recommended source
|
||||
|
||||
osxexperts.net "FFmpeg static (LGPL)" build:
|
||||
- Page: https://www.osxexperts.net/
|
||||
- Verify SPDX: LGPL-2.1-or-later (configure flags: `--disable-gpl --disable-nonfree`)
|
||||
- After download: `lipo -info ffmpeg` should report `arm64 x86_64`. If
|
||||
separate arch binaries, fuse with:
|
||||
`lipo -create ffmpeg-arm64 ffmpeg-x86_64 -output ffmpeg`
|
||||
- Make executable: `chmod +x ffmpeg`
|
||||
|
||||
## Codesigning
|
||||
|
||||
`jpackage --mac-sign` traverses `Contents/app/resources/` and signs nested
|
||||
executables with the same Developer ID identity as the app bundle. The
|
||||
process is launched from the JVM at runtime; hardened-runtime entitlements
|
||||
(`allow-jit`, `disable-library-validation`) are documented in
|
||||
`docs/plans/_macos-ffmpeg-signing-recipe.md`.
|
||||
|
||||
## Size
|
||||
|
||||
~30-40 MB universal binary. Acceptable trade-off given we shed ~95 MB of
|
||||
bundled VLC by switching to kdroidFilter + this thin LGPL FFmpeg.
|
||||
@@ -0,0 +1,16 @@
|
||||
# Windows bundled FFmpeg
|
||||
|
||||
Drop a single LGPL Windows x64 FFmpeg binary here as `ffmpeg.exe`.
|
||||
|
||||
## Recommended source
|
||||
|
||||
Crigges/Prebuilt-LGPL-2.1-FFmpeg-with-OpenH264:
|
||||
- Repo: https://github.com/Crigges/Prebuilt-LGPL-2.1-FFmpeg-with-OpenH264
|
||||
- Confirm: SPDX LGPL-2.1-or-later, OpenH264 (BSD-2 with patent grant from Cisco).
|
||||
- Verify: `ffmpeg.exe -version` from a `cmd.exe` prompt.
|
||||
|
||||
Used as fallback by `VideoThumbnailCache` for non-H.264 / HLS thumbnails.
|
||||
|
||||
## Size
|
||||
|
||||
~20-30 MB. Acceptable given the ~95 MB VLC plugin tree this migration retires.
|
||||
+62
-28
@@ -33,6 +33,7 @@ import kotlinx.coroutines.cancel
|
||||
import kotlinx.coroutines.flow.MutableStateFlow
|
||||
import kotlinx.coroutines.flow.StateFlow
|
||||
import kotlinx.coroutines.flow.asStateFlow
|
||||
import kotlinx.coroutines.flow.first
|
||||
import kotlinx.coroutines.launch
|
||||
|
||||
data class MediaPlaybackState(
|
||||
@@ -75,13 +76,12 @@ object GlobalMediaPlayer {
|
||||
private val initLock = Any()
|
||||
|
||||
/**
|
||||
* The kdroidFilter player driving currently-active or last-played video.
|
||||
* Mounted into a `VideoPlayerSurface(...)` by [DesktopVideoPlayer] when the
|
||||
* caller's `url` matches `videoState.value.url`.
|
||||
*
|
||||
* Lazy: first read constructs the underlying native player.
|
||||
* The kdroidFilter player driving currently-active or last-played video, or
|
||||
* `null` if the native player failed to initialize (e.g. missing GStreamer
|
||||
* on Linux). UI code reads this lazily; consumers must handle null by
|
||||
* showing the thumbnail/error fallback rather than mounting a surface.
|
||||
*/
|
||||
val activeVideoPlayerState: VideoPlayerState
|
||||
val activeVideoPlayerState: VideoPlayerState?
|
||||
get() = ensureVideoPlayer()
|
||||
|
||||
private val _videoState = MutableStateFlow(MediaPlaybackState())
|
||||
@@ -108,7 +108,17 @@ object GlobalMediaPlayer {
|
||||
seekPosition: Float = 0f,
|
||||
) {
|
||||
val current = _videoState.value
|
||||
val player = ensureVideoPlayer()
|
||||
val player =
|
||||
ensureVideoPlayer() ?: run {
|
||||
_videoState.value =
|
||||
MediaPlaybackState(
|
||||
url = url,
|
||||
type = MediaType.VIDEO,
|
||||
isBuffering = false,
|
||||
errorReason = "Video playback unavailable",
|
||||
)
|
||||
return
|
||||
}
|
||||
|
||||
if (current.url == url) {
|
||||
if (seekPosition > 0f) player.seekTo(seekPosition * 1000f)
|
||||
@@ -116,34 +126,49 @@ object GlobalMediaPlayer {
|
||||
return
|
||||
}
|
||||
|
||||
// Reset engine volume to match the UI's default for the new track. The
|
||||
// kdroidFilter player retains `volume` across openUri calls, so a mute
|
||||
// on the prior track would otherwise carry over while the UI shows the
|
||||
// default 100% / unmuted state.
|
||||
player.volume = 1f
|
||||
|
||||
_videoState.value = MediaPlaybackState(url = url, type = MediaType.VIDEO, isBuffering = true)
|
||||
|
||||
scope.launch(Dispatchers.IO) {
|
||||
player.openUri(url)
|
||||
// openUri auto-plays per InitialPlayerState.PLAY default.
|
||||
// For an initial seek we wait for hasMedia=true; cleanest is a
|
||||
// one-shot snapshotFlow collector that seeks then completes.
|
||||
// openUri auto-plays per InitialPlayerState.PLAY default. For an
|
||||
// initial seek we wait for the first hasMedia=true emission then
|
||||
// stop collecting (Flow.first terminates the collector cleanly,
|
||||
// unlike `return@collect` which only exits the lambda).
|
||||
if (seekPosition > 0f) {
|
||||
snapshotFlow { player.hasMedia }
|
||||
.collect { ready ->
|
||||
if (ready) {
|
||||
player.seekTo(seekPosition * 1000f)
|
||||
return@collect
|
||||
}
|
||||
}
|
||||
snapshotFlow { player.hasMedia }.first { it }
|
||||
player.seekTo(seekPosition * 1000f)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fun playAudio(url: String) {
|
||||
val current = _audioState.value
|
||||
val player = ensureAudioPlayer()
|
||||
val player =
|
||||
ensureAudioPlayer() ?: run {
|
||||
_audioState.value =
|
||||
MediaPlaybackState(
|
||||
url = url,
|
||||
type = MediaType.AUDIO,
|
||||
isBuffering = false,
|
||||
errorReason = "Audio playback unavailable",
|
||||
)
|
||||
return
|
||||
}
|
||||
|
||||
if (current.url == url) {
|
||||
if (!current.isPlaying) player.play()
|
||||
return
|
||||
}
|
||||
|
||||
// Reset engine volume — see playVideo() for rationale.
|
||||
player.volume = 1f
|
||||
|
||||
_audioState.value = MediaPlaybackState(url = url, type = MediaType.AUDIO, isBuffering = true)
|
||||
|
||||
scope.launch(Dispatchers.IO) {
|
||||
@@ -247,20 +272,29 @@ object GlobalMediaPlayer {
|
||||
|
||||
// --- Engine lifecycle ----------------------------------------------------
|
||||
|
||||
private fun ensureVideoPlayer(): VideoPlayerState =
|
||||
/**
|
||||
* Returns the video player, creating it on first call. Returns `null` if
|
||||
* native initialization throws (e.g. missing GStreamer on Linux, broken
|
||||
* `libNativeVideoPlayer.dylib` extraction). On failure, subsequent calls
|
||||
* keep returning `null` until the JVM is restarted — re-trying mid-session
|
||||
* is unlikely to recover from a missing native dependency.
|
||||
*/
|
||||
private fun ensureVideoPlayer(): VideoPlayerState? =
|
||||
videoPlayer ?: synchronized(initLock) {
|
||||
videoPlayer ?: createVideoPlayerState().also {
|
||||
videoPlayer = it
|
||||
startVideoSync(it)
|
||||
}
|
||||
videoPlayer ?: runCatching { createVideoPlayerState() }
|
||||
.onSuccess { startVideoSync(it) }
|
||||
.onFailure { println("kdroidFilter: video engine init failed: ${it.message}") }
|
||||
.getOrNull()
|
||||
?.also { videoPlayer = it }
|
||||
}
|
||||
|
||||
private fun ensureAudioPlayer(): VideoPlayerState =
|
||||
private fun ensureAudioPlayer(): VideoPlayerState? =
|
||||
audioPlayer ?: synchronized(initLock) {
|
||||
audioPlayer ?: createVideoPlayerState().also {
|
||||
audioPlayer = it
|
||||
startAudioSync(it)
|
||||
}
|
||||
audioPlayer ?: runCatching { createVideoPlayerState() }
|
||||
.onSuccess { startAudioSync(it) }
|
||||
.onFailure { println("kdroidFilter: audio engine init failed: ${it.message}") }
|
||||
.getOrNull()
|
||||
?.also { audioPlayer = it }
|
||||
}
|
||||
|
||||
private fun startVideoSync(player: VideoPlayerState) {
|
||||
|
||||
+56
-39
@@ -22,9 +22,9 @@ package com.vitorpamplona.amethyst.desktop.service.media
|
||||
|
||||
import androidx.compose.ui.graphics.ImageBitmap
|
||||
import androidx.compose.ui.graphics.toComposeImageBitmap
|
||||
import com.vitorpamplona.amethyst.desktop.network.DesktopHttpClient
|
||||
import kotlinx.coroutines.Dispatchers
|
||||
import kotlinx.coroutines.withContext
|
||||
import okhttp3.OkHttpClient
|
||||
import okhttp3.Request
|
||||
import org.jcodec.api.FrameGrab
|
||||
import org.jcodec.common.io.NIOUtils
|
||||
@@ -47,14 +47,15 @@ import javax.imageio.ImageIO
|
||||
*
|
||||
* Cascade:
|
||||
* 1. **JCodec** (`org.jcodec:jcodec` + `jcodec-javase`, BSD-2) — pure-Java
|
||||
* H.264 baseline/main/high decode. Handles ~80% of Nostr feed media (MP4/H.264).
|
||||
* 2. **Jaffree** (Apache-2) + **LGPL FFmpeg** subprocess — for everything else
|
||||
* (HEVC, VP9, AV1, HLS, malformed faststart MP4s). Requires a bundled
|
||||
* ffmpeg binary at `src/jvmMain/appResources/<os>/ffmpeg/ffmpeg(.exe)` or
|
||||
* a system `ffmpeg` on `$PATH`.
|
||||
* H.264 baseline/main/high decode. Handles the bulk of Nostr feed media
|
||||
* (MP4/H.264).
|
||||
* 2. **LGPL FFmpeg subprocess** (driven via raw `ProcessBuilder`) — for
|
||||
* everything else (HEVC, VP9, AV1, HLS, malformed faststart MP4s).
|
||||
* Requires either a system `ffmpeg` on `$PATH` or a bundled binary at
|
||||
* `src/jvmMain/appResources/<os>/ffmpeg/ffmpeg(.exe)`.
|
||||
*
|
||||
* Replaces the prior vlcj `RenderCallback` path. License moves from
|
||||
* GPL-3.0 (vlcj) to BSD-2 + Apache-2 + LGPL-2.1 native, MIT-dominant overall.
|
||||
* GPL-3.0 (vlcj) to BSD-2 + LGPL-2.1, MIT-dominant overall.
|
||||
*/
|
||||
object VideoThumbnailCache {
|
||||
private const val MAX_THUMB_BYTES = 4 * 1024 * 1024 // 4 MiB cap per thumbnail
|
||||
@@ -62,14 +63,6 @@ object VideoThumbnailCache {
|
||||
private val cache = ConcurrentHashMap<String, ImageBitmap>()
|
||||
private val pending = ConcurrentHashMap<String, Boolean>()
|
||||
|
||||
private val http: OkHttpClient by lazy {
|
||||
OkHttpClient
|
||||
.Builder()
|
||||
.connectTimeout(15, TimeUnit.SECONDS)
|
||||
.readTimeout(30, TimeUnit.SECONDS)
|
||||
.build()
|
||||
}
|
||||
|
||||
private val downloadCacheDir: File by lazy {
|
||||
val base =
|
||||
File(System.getProperty("user.home"), ".cache/amethyst-desktop/video-thumbs")
|
||||
@@ -78,14 +71,22 @@ object VideoThumbnailCache {
|
||||
}
|
||||
|
||||
private val ffmpegBinary: String? by lazy {
|
||||
// 1. System ffmpeg on PATH.
|
||||
// 1. System ffmpeg on PATH. Probe with `ffmpeg -version`; drain stdout
|
||||
// so the child doesn't block on a full pipe, kill it if it overruns
|
||||
// the probe budget so we don't leak the process when ffmpeg hangs.
|
||||
val onPath =
|
||||
runCatching {
|
||||
ProcessBuilder("ffmpeg", "-version")
|
||||
.redirectErrorStream(true)
|
||||
.start()
|
||||
.also { it.inputStream.close() }
|
||||
.waitFor(2, TimeUnit.SECONDS)
|
||||
val probe =
|
||||
ProcessBuilder("ffmpeg", "-version")
|
||||
.redirectErrorStream(true)
|
||||
.redirectOutput(ProcessBuilder.Redirect.DISCARD)
|
||||
.start()
|
||||
val exited = probe.waitFor(2, TimeUnit.SECONDS)
|
||||
if (!exited) {
|
||||
probe.destroyForcibly()
|
||||
return@runCatching false
|
||||
}
|
||||
probe.exitValue() == 0
|
||||
}.getOrDefault(false)
|
||||
if (onPath) return@lazy "ffmpeg"
|
||||
|
||||
@@ -121,20 +122,35 @@ object VideoThumbnailCache {
|
||||
}
|
||||
|
||||
private fun extractFirstFrame(url: String): ImageBitmap? {
|
||||
// For HLS we skip straight to Jaffree — JCodec can't read m3u8.
|
||||
// For HLS we skip straight to ffmpeg — JCodec can't read m3u8.
|
||||
val isHls = url.contains(".m3u8", ignoreCase = true) || url.contains("/hls/", ignoreCase = true)
|
||||
|
||||
if (!isHls) {
|
||||
val downloaded = runCatching { downloadFirstChunk(url) }.getOrNull()
|
||||
if (downloaded != null) {
|
||||
tryJCodec(downloaded)?.let { return it }
|
||||
tryJaffreeFile(downloaded)?.let { return it }
|
||||
try {
|
||||
tryJCodec(downloaded.file)?.let { return it }
|
||||
tryFfmpegFile(downloaded.file)?.let { return it }
|
||||
} finally {
|
||||
// Origins that ignore Range: served the full body and we
|
||||
// truncated to MAX_THUMB_BYTES; that file is unsuitable as
|
||||
// a persistent cache hit (decoders may fail on every
|
||||
// retry against a half-MP4). Discard so the next request
|
||||
// re-downloads from scratch.
|
||||
if (!downloaded.persistable) downloaded.file.delete()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return tryJaffreeUrl(url)
|
||||
return tryFfmpegUrl(url)
|
||||
}
|
||||
|
||||
/** Local result of [downloadFirstChunk]: the bytes + whether they're a real Range slice. */
|
||||
private data class Download(
|
||||
val file: File,
|
||||
val persistable: Boolean,
|
||||
)
|
||||
|
||||
/**
|
||||
* Downloads up to [MAX_THUMB_BYTES] to a cache file, returning the file (or null on failure).
|
||||
*
|
||||
@@ -146,17 +162,19 @@ object VideoThumbnailCache {
|
||||
*
|
||||
* Cleans up zero-byte cache files on failure so a transient empty response isn't sticky.
|
||||
*/
|
||||
private fun downloadFirstChunk(url: String): File? {
|
||||
private fun downloadFirstChunk(url: String): Download? {
|
||||
val hash = sha1Hex(url)
|
||||
val cached = File(downloadCacheDir, "$hash.mp4")
|
||||
if (cached.length() > 0L) return cached
|
||||
if (cached.length() > 0L) return Download(cached, persistable = true)
|
||||
if (cached.exists()) cached.delete()
|
||||
|
||||
var wrote = false
|
||||
http.newCall(buildRangeRequest(url)).execute().use { resp ->
|
||||
var rangeHonored = false
|
||||
DesktopHttpClient.currentClient().newCall(buildRangeRequest(url)).execute().use { resp ->
|
||||
if (!resp.isSuccessful && resp.code != 206) return null
|
||||
val contentType = resp.header("Content-Type")?.lowercase().orEmpty()
|
||||
if (contentType.startsWith("text/") || "html" in contentType) return null
|
||||
rangeHonored = resp.code == 206
|
||||
Files.newOutputStream(cached.toPath()).use { out ->
|
||||
val copied = copyAtMost(resp.body.byteStream(), out, MAX_THUMB_BYTES.toLong())
|
||||
wrote = copied > 0L
|
||||
@@ -166,7 +184,7 @@ object VideoThumbnailCache {
|
||||
cached.delete()
|
||||
return null
|
||||
}
|
||||
return cached
|
||||
return Download(cached, persistable = rangeHonored)
|
||||
}
|
||||
|
||||
private fun buildRangeRequest(url: String): Request =
|
||||
@@ -205,16 +223,17 @@ object VideoThumbnailCache {
|
||||
}
|
||||
}.getOrNull()
|
||||
|
||||
private fun tryJaffreeFile(file: File): ImageBitmap? = runFfmpegToImage(file.absolutePath)
|
||||
private fun tryFfmpegFile(file: File): ImageBitmap? = runFfmpegToImage(file.absolutePath)
|
||||
|
||||
private fun tryJaffreeUrl(url: String): ImageBitmap? = runFfmpegToImage(url)
|
||||
private fun tryFfmpegUrl(url: String): ImageBitmap? = runFfmpegToImage(url)
|
||||
|
||||
/**
|
||||
* Spawns `ffmpeg -ss 1 -i <input> -frames:v 1 -f image2pipe -c:v png -an pipe:1`,
|
||||
* reads PNG bytes from stdout, decodes with Skia.
|
||||
*
|
||||
* Uses raw `ProcessBuilder` rather than the Jaffree DSL — fewer API guesses,
|
||||
* easier to debug. Jaffree stays on the classpath as a future option.
|
||||
* `redirectError(DISCARD)` so a chatty ffmpeg cannot fill the stderr pipe
|
||||
* and stall our `copyTo`. `destroyForcibly()` runs on any unwind so we
|
||||
* never leak a ffmpeg process.
|
||||
*/
|
||||
private fun runFfmpegToImage(input: String): ImageBitmap? {
|
||||
val ffmpeg = ffmpegBinary ?: return null
|
||||
@@ -240,20 +259,18 @@ object VideoThumbnailCache {
|
||||
val process =
|
||||
runCatching {
|
||||
ProcessBuilder(cmd)
|
||||
.redirectErrorStream(false)
|
||||
.redirectError(ProcessBuilder.Redirect.DISCARD)
|
||||
.start()
|
||||
}.getOrNull() ?: return null
|
||||
val out = ByteArrayOutputStream(256 * 1024)
|
||||
try {
|
||||
process.inputStream.use { it.copyTo(out) }
|
||||
if (!process.waitFor(8, TimeUnit.SECONDS)) {
|
||||
process.destroyForcibly()
|
||||
return null
|
||||
}
|
||||
if (!process.waitFor(8, TimeUnit.SECONDS)) return null
|
||||
if (process.exitValue() != 0 || out.size() == 0) return null
|
||||
} catch (_: Exception) {
|
||||
process.destroyForcibly()
|
||||
return null
|
||||
} finally {
|
||||
if (process.isAlive) process.destroyForcibly()
|
||||
}
|
||||
return runCatching {
|
||||
Image.makeFromEncoded(out.toByteArray()).toComposeImageBitmap()
|
||||
|
||||
+3
-2
@@ -109,12 +109,13 @@ fun DesktopVideoPlayer(
|
||||
contentAlignment = Alignment.Center,
|
||||
) {
|
||||
val errorReason = if (isActiveVideo) videoState.errorReason else null
|
||||
val activePlayer = if (isActiveVideo) GlobalMediaPlayer.activeVideoPlayerState else null
|
||||
|
||||
if (errorReason != null) {
|
||||
PlaybackErrorMessage(url = url, reason = errorReason)
|
||||
} else if (isActiveVideo) {
|
||||
} else if (isActiveVideo && activePlayer != null) {
|
||||
VideoPlayerSurface(
|
||||
playerState = GlobalMediaPlayer.activeVideoPlayerState,
|
||||
playerState = activePlayer,
|
||||
modifier = Modifier.fillMaxSize().clip(MaterialTheme.shapes.small),
|
||||
contentScale = ContentScale.Fit,
|
||||
)
|
||||
|
||||
+10
-6
@@ -102,12 +102,16 @@ fun GlobalFullscreenOverlay() {
|
||||
},
|
||||
contentAlignment = Alignment.Center,
|
||||
) {
|
||||
// Video frame — same player state as feed card; kdroidFilter draws to Canvas
|
||||
VideoPlayerSurface(
|
||||
playerState = GlobalMediaPlayer.activeVideoPlayerState,
|
||||
modifier = Modifier.fillMaxSize(),
|
||||
contentScale = ContentScale.Fit,
|
||||
)
|
||||
// Video frame — same player state as feed card; kdroidFilter draws to Canvas.
|
||||
// If the engine failed to initialize, render a blank backdrop instead of
|
||||
// crashing the overlay.
|
||||
GlobalMediaPlayer.activeVideoPlayerState?.let { player ->
|
||||
VideoPlayerSurface(
|
||||
playerState = player,
|
||||
modifier = Modifier.fillMaxSize(),
|
||||
contentScale = ContentScale.Fit,
|
||||
)
|
||||
}
|
||||
|
||||
// Video controls overlay
|
||||
VideoControls(
|
||||
|
||||
+14
-5
@@ -23,6 +23,7 @@ package com.vitorpamplona.amethyst.desktop.ui.media
|
||||
import androidx.compose.animation.AnimatedVisibility
|
||||
import androidx.compose.animation.slideInVertically
|
||||
import androidx.compose.animation.slideOutVertically
|
||||
import androidx.compose.foundation.Image
|
||||
import androidx.compose.foundation.background
|
||||
import androidx.compose.foundation.layout.Arrangement
|
||||
import androidx.compose.foundation.layout.Row
|
||||
@@ -50,7 +51,7 @@ import androidx.compose.ui.unit.dp
|
||||
import com.vitorpamplona.amethyst.commons.icons.symbols.Icon
|
||||
import com.vitorpamplona.amethyst.commons.icons.symbols.MaterialSymbols
|
||||
import com.vitorpamplona.amethyst.desktop.service.media.GlobalMediaPlayer
|
||||
import io.github.kdroidfilter.composemediaplayer.VideoPlayerSurface
|
||||
import com.vitorpamplona.amethyst.desktop.service.media.VideoThumbnailCache
|
||||
import kotlinx.coroutines.launch
|
||||
|
||||
enum class MediaType { AUDIO, VIDEO }
|
||||
@@ -85,10 +86,18 @@ fun NowPlayingBar(modifier: Modifier = Modifier) {
|
||||
verticalAlignment = Alignment.CenterVertically,
|
||||
horizontalArrangement = Arrangement.spacedBy(8.dp),
|
||||
) {
|
||||
// Mini video preview or music icon
|
||||
if (activeType == MediaType.VIDEO) {
|
||||
VideoPlayerSurface(
|
||||
playerState = GlobalMediaPlayer.activeVideoPlayerState,
|
||||
// Mini thumbnail (cached frame, not a live VideoPlayerSurface).
|
||||
// We deliberately render the cached thumbnail rather than mounting a
|
||||
// second VideoPlayerSurface to avoid driving two simultaneous Skia
|
||||
// draws against the same player's frame bitmap — kdroidFilter
|
||||
// 0.10.0's macOS surface has a use-after-free race in that path
|
||||
// (see PR review). The mini-preview UX is preserved via the
|
||||
// poster frame extracted by VideoThumbnailCache.
|
||||
val miniThumb = activeState.url?.let { VideoThumbnailCache.getCached(it) }
|
||||
if (activeType == MediaType.VIDEO && miniThumb != null) {
|
||||
Image(
|
||||
bitmap = miniThumb,
|
||||
contentDescription = "Now playing",
|
||||
modifier =
|
||||
Modifier
|
||||
.size(width = 48.dp, height = 36.dp)
|
||||
|
||||
Reference in New Issue
Block a user