From 3e83b8ee425bb69dc118bd3840bcabbb500827e5 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 17 Sep 2026 18:28:00 +0000 Subject: [PATCH 01/11] fix(nip71): read captions from `text-track`, not from `e` tags MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit VideoEvent.textTrack() ran ETag::parse, which matches `["e", ...]` — so it never returned a caption track, and on a divine.video short it returned the event's "audio" source pointer (the reused-soundtrack reference) instead. Parse TextTrackTag, and widen that tag to keep the type and language fields publishers put at positions 3 and 4 (divine.video emits `["text-track", , , "captions", "en"]`). Also fixes the profile gallery, which passed `listOf(34236, 34236)` to LocalCache.addressables — acceptableEvent() takes any AddressableVideoEvent, so the duplicate silently kept every 34235 horizontal video out of the tab. Adds DivineVideoInteropTest, pinned to real events from relay.divine.video: extension-less Blossom URLs classified as video by their imeta MIME alone, and the dual `a`+`e` pointers divine puts on reposts, reactions and NIP-22 comments both resolving. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JLEq2G6U29mKV8cfSTamZ9 --- .../dal/UserProfileGalleryFeedFilter.kt | 6 +- .../nip71Video/DivineVideoInteropTest.kt | 213 ++++++++++++++++++ .../nip71Video/AddressableVideoEvent.kt | 6 +- .../quartz/nip71Video/RegularVideoEvent.kt | 6 +- .../quartz/nip71Video/VideoEvent.kt | 4 +- .../quartz/nip71Video/tags/TextTrackTag.kt | 41 +++- 6 files changed, 261 insertions(+), 15 deletions(-) create mode 100644 commons/src/jvmTest/kotlin/com/vitorpamplona/amethyst/commons/model/nip71Video/DivineVideoInteropTest.kt diff --git a/amethyst/src/main/java/com/vitorpamplona/amethyst/ui/screen/loggedIn/profile/gallery/dal/UserProfileGalleryFeedFilter.kt b/amethyst/src/main/java/com/vitorpamplona/amethyst/ui/screen/loggedIn/profile/gallery/dal/UserProfileGalleryFeedFilter.kt index 0425c0d86a..bb395f7540 100644 --- a/amethyst/src/main/java/com/vitorpamplona/amethyst/ui/screen/loggedIn/profile/gallery/dal/UserProfileGalleryFeedFilter.kt +++ b/amethyst/src/main/java/com/vitorpamplona/amethyst/ui/screen/loggedIn/profile/gallery/dal/UserProfileGalleryFeedFilter.kt @@ -34,6 +34,7 @@ import com.vitorpamplona.quartz.nip53LiveActivities.clip.LiveActivitiesClipEvent import com.vitorpamplona.quartz.nip68Picture.PictureEvent import com.vitorpamplona.quartz.nip71Video.AddressableVideoEvent import com.vitorpamplona.quartz.nip71Video.RegularVideoEvent +import com.vitorpamplona.quartz.nip71Video.VideoHorizontalEvent import com.vitorpamplona.quartz.nip71Video.VideoVerticalEvent class UserProfileGalleryFeedFilter( @@ -53,7 +54,10 @@ class UserProfileGalleryFeedFilter( val addressableNotes = LocalCache.addressables .filter( - listOf(VideoVerticalEvent.KIND, VideoVerticalEvent.KIND), + // Both NIP-71 addressable kinds: acceptableEvent() takes any + // AddressableVideoEvent, so listing 34236 twice silently kept every + // 34235 (horizontal) video out of the gallery. + listOf(VideoVerticalEvent.KIND, VideoHorizontalEvent.KIND), user.pubkeyHex, ) { _, it -> acceptableEvent(it, params, user) diff --git a/commons/src/jvmTest/kotlin/com/vitorpamplona/amethyst/commons/model/nip71Video/DivineVideoInteropTest.kt b/commons/src/jvmTest/kotlin/com/vitorpamplona/amethyst/commons/model/nip71Video/DivineVideoInteropTest.kt new file mode 100644 index 0000000000..34c249006b --- /dev/null +++ b/commons/src/jvmTest/kotlin/com/vitorpamplona/amethyst/commons/model/nip71Video/DivineVideoInteropTest.kt @@ -0,0 +1,213 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.amethyst.commons.model.nip71Video + +import com.vitorpamplona.amethyst.commons.relayClient.video.SUPPORTED_VIDEO_FEED_MIME_TYPES_SET +import com.vitorpamplona.amethyst.commons.richtext.MediaContentKind +import com.vitorpamplona.amethyst.commons.richtext.RichTextParser +import com.vitorpamplona.quartz.nip01Core.core.AddressableEvent +import com.vitorpamplona.quartz.nip01Core.core.Event +import com.vitorpamplona.quartz.nip18Reposts.GenericRepostEvent +import com.vitorpamplona.quartz.nip22Comments.CommentEvent +import com.vitorpamplona.quartz.nip25Reactions.ReactionEvent +import com.vitorpamplona.quartz.nip51Lists.videoCurationSet.VideoCurationSetEvent +import com.vitorpamplona.quartz.nip71Video.VideoVerticalEvent +import org.junit.Assert.assertEquals +import org.junit.Assert.assertNotNull +import org.junit.Assert.assertTrue +import org.junit.Test + +/** + * Interop guard for divine.video (), the biggest publisher of + * NIP-71 kind-34236 short videos on the network. Every fixture below is a real event pulled from + * `wss://relay.divine.video` — only the oversized `proofmode`/`device_attestation`/ + * `identity_binding` blobs were stripped, which is why the signatures are not re-verified here. + * + * Divine's events exercise two shapes Amethyst used to get wrong and can silently regress on: + * + * 1. **Extension-less media URLs.** Videos live on Blossom, so the URL is + * `https://media.divine.video/` with no `.mp4`. Only the `imeta` `m` field says it is + * a video, so every extension-driven branch (feed admission, image-vs-player classification) + * has to consult the declared MIME first. + * 2. **Dual pointers on every interaction.** Reactions, reposts and comments carry BOTH the + * addressable coordinate (`a`/`A`) and the specific version's event id (`e`/`E`), so both + * have to resolve or the counters land on a note nobody renders. + */ +class DivineVideoInteropTest { + private val video = + """{"id":"fa5a793b24edfe109f8d02ad6aa6cde77b0a923566f891df403049721b46e8d9","pubkey":"4d7dccc0a5116daa057348ef79c573873cddd9eff066fc6a5f3d37e8264afbeb","created_at":1789661586,"kind":34236,"tags":[["d","c855df3d07ba963e9097d5a151b0c14a0a3d494e4ff9b04a389bc0b5f5c16862"],["text-track","https://media.divine.video/283a420202b620a9a326598b85bfcf0e8cb4ab4d52b947c628d8c29a1313006b","wss://relay.divine.video","captions","en"],["text-track","39307:4d7dccc0a5116daa057348ef79c573873cddd9eff066fc6a5f3d37e8264afbeb:subtitles:c855df3d07ba963e9097d5a151b0c14a0a3d494e4ff9b04a389bc0b5f5c16862","wss://relay.divine.video","captions","en"],["imeta","url https://media.divine.video/c855df3d07ba963e9097d5a151b0c14a0a3d494e4ff9b04a389bc0b5f5c16862","m video/mp4","image https://media.divine.video/e1d22b85609cb105dff64b4a402f13982a942dc59aee67218e6e3652c8597d2d","dim 1080x1920","x c855df3d07ba963e9097d5a151b0c14a0a3d494e4ff9b04a389bc0b5f5c16862","size 6579464","blurhash TQHxc{D%X7_Nw{nio#RiRjaJX8oz"],["title","Xmas Already??"],["summary","In the words of Jack Skellington from The Nightmare Before Christmas, \"What's this?\" 😭"],["t","retail"],["published_at","1789661611"],["duration","6"],["alt","Xmas Already??"],["allow_audio_reuse","true"],["e","218af5fc90d66a16ce273f00a4e412a71443441c04c67d1e34ff99c654871d3d","wss://relay.divine.video","audio"],["c2pa_manifest_id","urn:c2pa:b1d12836-a60f-4398-a081-c8f047483ab4"],["verification","verified_mobile"],["client","Divine","31990:d95aa8fc0eff8e488952495b8064991d27fb96ed8652f12cdedc5a4e8b5ae540:divine-mobile","wss://relay.divine.video"]],"content":"In the words of Jack Skellington from The Nightmare Before Christmas, \"What's this?\" 😭","sig":"c32a6486480465d3c4c695d8eee52868eb9c5e085fe54827478f7e183fea0602e969fa82476053136ae766536ec9ed7f0d4c9d009c5fb6991a2099696c66fc33"}""" + + private val repost = + """{"id":"20a0663382290c769b1fbd5e6ad84593d1f848fa75689e4ee4d6263aad5c8760","pubkey":"f67d985c0bfbf87eaa33b056f1d38ad991a6aa625b138bddb2a838bdbac29f40","created_at":1789666347,"kind":16,"tags":[["k","34236"],["a","34236:5ab67f7d7fed4f781008c0ec0d26c8113f9fb46094a8346246c70c75e75db9fb:8de0dcf06982b86aca7189ae50a8c3fe8605917433044ad62c146b723b877025"],["p","5ab67f7d7fed4f781008c0ec0d26c8113f9fb46094a8346246c70c75e75db9fb"],["e","ab074d5a577635b9d34281b31ea32170d54b09cfb4cbfcc4ae6a28f52653fca1"],["client","Divine","31990:d95aa8fc0eff8e488952495b8064991d27fb96ed8652f12cdedc5a4e8b5ae540:divine-mobile","wss://relay.divine.video"]],"content":"","sig":"4513526b886a1d9e1fb12a181a48811dc8765afd901458959a674d3e9d234b761a5cd2551c75cbe4210543f30c5a74b16720281acf88e11e09a67b0b219e3d3d"}""" + + private val reaction = + """{"id":"88847c819aa18d9294de9ebc1156d6200becdd2fadad6bc6a0248b24f22d3c42","pubkey":"34257350449d357c37e93eb8aef387ff1fee8879d794da664462346a4b540aa8","created_at":1789666900,"kind":7,"tags":[["e","a75d3c1a2fb824544ae51d3d20b1a8280aea9647d13f87c06418233883e80890"],["a","34236:03c49dd3d68fdd15fc0bc7dff669d652af313408bfd6dde10daf27b02f54eb50:a983212b6a82e0d6f6a41efc085ddd0176e1cd3ebd8dcb4cac16219db0a283ef"],["p","03c49dd3d68fdd15fc0bc7dff669d652af313408bfd6dde10daf27b02f54eb50"],["k","34236"],["client","Divine","31990:d95aa8fc0eff8e488952495b8064991d27fb96ed8652f12cdedc5a4e8b5ae540:divine-mobile","wss://relay.divine.video"]],"content":"+","sig":"99d515ef57cb8425e26b85051202e305fb6f1f4265ea38bb041da24044c8ac3e95339bdcae01400307afa26417fb056b6aad1a34e826c4e267df7b4dc6c29e94"}""" + + private val comment = + """{"id":"dac3c50f5a3c22f185dc8587a2eae3357087aa0ccfd1f1230f8308d7a047515a","pubkey":"f67d985c0bfbf87eaa33b056f1d38ad991a6aa625b138bddb2a838bdbac29f40","created_at":1789666353,"kind":1111,"tags":[["E","ab074d5a577635b9d34281b31ea32170d54b09cfb4cbfcc4ae6a28f52653fca1","","5ab67f7d7fed4f781008c0ec0d26c8113f9fb46094a8346246c70c75e75db9fb"],["A","34236:5ab67f7d7fed4f781008c0ec0d26c8113f9fb46094a8346246c70c75e75db9fb:8de0dcf06982b86aca7189ae50a8c3fe8605917433044ad62c146b723b877025",""],["K","34236"],["P","5ab67f7d7fed4f781008c0ec0d26c8113f9fb46094a8346246c70c75e75db9fb"],["e","ab074d5a577635b9d34281b31ea32170d54b09cfb4cbfcc4ae6a28f52653fca1","","5ab67f7d7fed4f781008c0ec0d26c8113f9fb46094a8346246c70c75e75db9fb"],["a","34236:5ab67f7d7fed4f781008c0ec0d26c8113f9fb46094a8346246c70c75e75db9fb:8de0dcf06982b86aca7189ae50a8c3fe8605917433044ad62c146b723b877025",""],["k","34236"],["p","5ab67f7d7fed4f781008c0ec0d26c8113f9fb46094a8346246c70c75e75db9fb"],["client","Divine","31990:d95aa8fc0eff8e488952495b8064991d27fb96ed8652f12cdedc5a4e8b5ae540:divine-mobile","wss://relay.divine.video"]],"content":"⚫️","sig":"8fa4fcb27882b0622d48e21f1323e4b16c2df24f6a127c49bc30a6e277ef6069f5cfb7b13ddf9788ac18c4cde031091b1d39e349cedde812a6fc8fd517dc3d1f"}""" + + private val videoList = + """{"id":"70118cbfdd6ed7a0b774d083eb7e788a04f39660ad1aecb01cc645902d06cb9f","pubkey":"34257350449d357c37e93eb8aef387ff1fee8879d794da664462346a4b540aa8","created_at":1789666571,"kind":30005,"tags":[["d","my_vine_list"],["title","My List"],["description","My favorite vines and videos"],["playorder","chronological"],["client","Divine","31990:d95aa8fc0eff8e488952495b8064991d27fb96ed8652f12cdedc5a4e8b5ae540:divine-mobile","wss://relay.divine.video"]],"content":"Ai9Jcq6da7mTNMTcnDh+I4ZUwQcX5///rkV6ouuE7ZqaBnRIgg+rCqul8ykw4KKj6O7wwTm3jzxmTD7ZeEjD7LplcaH8kYDPnzAJWriiLC6tJ1THmXwx0SYYNYrtLs+t1cBR","sig":"7db79697f4adf9ff8b6aba918e5760508fe96fd8227b600ba41edc84d4efbf5a37af15ed7980da5e6106b9a7241583e3e4dce473728f215a0d70b0b28dbf52b2"}""" + + private val collabResponse = + """{"id":"70cd826d52989fbfd6a12f7e62e4f9aa3c01c902c1bdeec61955d0271920a248","pubkey":"ddfdea0a598ec89f1383ca83b71993540c99ea5a3734b4d5aab23719ea6bde80","created_at":1789653321,"kind":34238,"tags":[["d","34236:32d84a21bb7702c538d5ddd3f9086e86e8b73e5d5cb2e67861eaca36708597e6:d67650d7e97d50a35f491ef24f463b4b408026b5b08c0bc9dbc8359cf4b8aa05"],["a","34236:32d84a21bb7702c538d5ddd3f9086e86e8b73e5d5cb2e67861eaca36708597e6:d67650d7e97d50a35f491ef24f463b4b408026b5b08c0bc9dbc8359cf4b8aa05","wss://relay.divine.video","root"],["p","32d84a21bb7702c538d5ddd3f9086e86e8b73e5d5cb2e67861eaca36708597e6"],["role","Collaborator"],["status","accepted"],["client","Divine","31990:d95aa8fc0eff8e488952495b8064991d27fb96ed8652f12cdedc5a4e8b5ae540:divine-mobile","wss://relay.divine.video"]],"content":"","sig":"65e476c6023de6e443810e630450e6f2c3bca0201a970adcb89f18788dc9bf2ab8e5431a68ff716d411aaa05caa83288549efed8814750c2a23cec62c69bab18"}""" + + @Test + fun aShortVideoIsAnAddressableVerticalVideo() { + val event = Event.fromJson(video) + + assertTrue(event is VideoVerticalEvent) + // The address must come from the `d` tag: an `a` tag elsewhere on the network points at + // `34236::`, and a class on the wrong base would split the cache in two. + assertEquals( + "34236:4d7dccc0a5116daa057348ef79c573873cddd9eff066fc6a5f3d37e8264afbeb:c855df3d07ba963e9097d5a151b0c14a0a3d494e4ff9b04a389bc0b5f5c16862", + (event as AddressableEvent).address().toValue(), + ) + assertEquals("Xmas Already??", (event as VideoVerticalEvent).title()) + assertEquals(6, event.duration()) + } + + @Test + fun theBlossomUrlIsPlayedAsAVideoDespiteHavingNoFileExtension() { + val track = (Event.fromJson(video) as VideoVerticalEvent).selectVideoTrack() + + assertNotNull(track) + assertEquals("https://media.divine.video/c855df3d07ba963e9097d5a151b0c14a0a3d494e4ff9b04a389bc0b5f5c16862", track!!.url) + assertEquals("video/mp4", track.mimeType) + assertEquals("1080x1920", track.dimension.toString()) + assertEquals( + "https://media.divine.video/e1d22b85609cb105dff64b4a402f13982a942dc59aee67218e6e3652c8597d2d", + track.image.firstOrNull(), + ) + + // VideoDisplay diverts to the image viewer only when classifyMedia says IMAGE. The URL has + // no extension at all, so the declared MIME is the only thing keeping this in the player. + assertEquals(MediaContentKind.VIDEO, RichTextParser.classifyMedia(track.url, track.mimeType)) + + // ...and the same MIME is what admits it into the Shorts/Video feeds, whose + // SupportedContent matcher would otherwise fall through to the extension list. + assertTrue(track.mimeType in SUPPORTED_VIDEO_FEED_MIME_TYPES_SET) + } + + @Test + fun theAudioSourceETagIsNotAReplyPointer() { + // Divine marks the reused soundtrack as `["e", , , "audio"]` on the video + // itself. LocalCache.computeReplyTo has no branch for video events, so this never turns + // the post into a reply — which would drop it out of the home feed and thread it under + // whatever video the audio came from. + val event = Event.fromJson(video) as VideoVerticalEvent + val audio = event.tags.first { it[0] == "e" } + + assertEquals("218af5fc90d66a16ce273f00a4e412a71443441c04c67d1e34ff99c654871d3d", audio[1]) + assertEquals("audio", audio[3]) + } + + @Test + fun captionsComeFromTheTextTrackTagsAndNotTheAudioPointer() { + // textTrack() used to run ETag::parse, so it returned the `e` tags instead — on this + // event, the reused-soundtrack pointer, which is not a caption track at all. + val tracks = (Event.fromJson(video) as VideoVerticalEvent).textTrack() + + assertEquals(2, tracks.size) + // Divine publishes the same track twice: the WebVTT file on Blossom, and the addressable + // kind-39307 subtitle event that wraps it. + assertEquals("https://media.divine.video/283a420202b620a9a326598b85bfcf0e8cb4ab4d52b947c628d8c29a1313006b", tracks[0].ref) + assertEquals("39307:4d7dccc0a5116daa057348ef79c573873cddd9eff066fc6a5f3d37e8264afbeb:subtitles:c855df3d07ba963e9097d5a151b0c14a0a3d494e4ff9b04a389bc0b5f5c16862", tracks[1].ref) + tracks.forEach { + assertEquals("wss://relay.divine.video", it.relay) + assertEquals("captions", it.type) + assertEquals("en", it.language) + } + // No renderer consumes these yet — Amethyst plays divine.video shorts without captions. + } + + @Test + fun aRepostResolvesBothTheCoordinateAndTheVersionId() { + val event = Event.fromJson(repost) + + assertTrue(event is GenericRepostEvent) + val boost = event as GenericRepostEvent + assertEquals(34236, boost.boostedKind()) + assertEquals( + "34236:5ab67f7d7fed4f781008c0ec0d26c8113f9fb46094a8346246c70c75e75db9fb:8de0dcf06982b86aca7189ae50a8c3fe8605917433044ad62c146b723b877025", + boost.boostedAddress()?.toValue(), + ) + assertEquals("ab074d5a577635b9d34281b31ea32170d54b09cfb4cbfcc4ae6a28f52653fca1", boost.boostedEventId()) + } + + @Test + fun aReactionResolvesBothTheCoordinateAndTheVersionId() { + val event = Event.fromJson(reaction) + + assertTrue(event is ReactionEvent) + val like = event as ReactionEvent + assertEquals("+", like.content) + assertEquals(listOf("a75d3c1a2fb824544ae51d3d20b1a8280aea9647d13f87c06418233883e80890"), like.originalPost()) + assertEquals( + listOf("34236:03c49dd3d68fdd15fc0bc7dff669d652af313408bfd6dde10daf27b02f54eb50:a983212b6a82e0d6f6a41efc085ddd0176e1cd3ebd8dcb4cac16219db0a283ef"), + like.linkedAddressIds(), + ) + } + + @Test + fun aTopLevelCommentRootsAtTheVideoCoordinate() { + val event = Event.fromJson(comment) + + assertTrue(event is CommentEvent) + val reply = event as CommentEvent + val coordinate = "34236:5ab67f7d7fed4f781008c0ec0d26c8113f9fb46094a8346246c70c75e75db9fb:8de0dcf06982b86aca7189ae50a8c3fe8605917433044ad62c146b723b877025" + + assertEquals(listOf(coordinate), reply.rootAddressIds()) + assertEquals(listOf("ab074d5a577635b9d34281b31ea32170d54b09cfb4cbfcc4ae6a28f52653fca1"), reply.rootEventIds()) + // A top-level comment repeats the root as its direct parent (lowercase a/e), so both the + // version note and the addressable note collect the reply. Note.addReply() dedupes, and + // consumeBaseReplaceable migrates the version's references onto the address. + assertEquals(listOf(coordinate), reply.replyAddressIds()) + } + + @Test + fun aVideoListParsesButCarriesItsItemsEncrypted() { + val event = Event.fromJson(videoList) + + assertTrue(event is VideoCurationSetEvent) + val list = event as VideoCurationSetEvent + assertEquals("my_vine_list", list.dTag()) + assertEquals("My List", list.title()) + assertEquals("My favorite vines and videos", list.description()) + // Divine keeps every member in the NIP-51 encrypted `content`, so nothing is public. + // Amethyst has no consumer for kind 30005 today: LocalCache drops it as unsupported and + // no screen renders it. Kept as a marker for when that changes. + assertTrue(list.publicItems().isEmpty()) + } + + @Test + fun aCollabResponseIsStillAnUntypedEvent() { + // Kind 34238 is Divine's own "collaborator accepted" record. It sits in the addressable + // range and keys itself by the video coordinate, but Quartz has no class for it, so it + // parses as a bare Event, is NOT addressable, and LocalCache drops it as unsupported. + // Amethyst therefore never shows a video's collaborators. + val event = Event.fromJson(collabResponse) + + assertEquals(34238, event.kind) + assertEquals(Event::class, event::class) + assertTrue(event !is AddressableEvent) + } +} diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip71Video/AddressableVideoEvent.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip71Video/AddressableVideoEvent.kt index 433cd7866e..6f77f06f69 100644 --- a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip71Video/AddressableVideoEvent.kt +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip71Video/AddressableVideoEvent.kt @@ -23,7 +23,6 @@ package com.vitorpamplona.quartz.nip71Video import androidx.compose.runtime.Immutable import com.vitorpamplona.quartz.nip01Core.core.BaseAddressableEvent import com.vitorpamplona.quartz.nip01Core.core.HexKey -import com.vitorpamplona.quartz.nip01Core.tags.events.ETag import com.vitorpamplona.quartz.nip01Core.tags.hashtags.hashtags import com.vitorpamplona.quartz.nip01Core.tags.people.PTag import com.vitorpamplona.quartz.nip01Core.tags.publishedAt.PublishedAtProvider @@ -34,6 +33,7 @@ import com.vitorpamplona.quartz.nip50Search.IndexableFieldVisitor import com.vitorpamplona.quartz.nip50Search.SearchableEvent import com.vitorpamplona.quartz.nip71Video.tags.DurationTag import com.vitorpamplona.quartz.nip71Video.tags.SegmentTag +import com.vitorpamplona.quartz.nip71Video.tags.TextTrackTag import com.vitorpamplona.quartz.nip92IMeta.imetas import com.vitorpamplona.quartz.nip94FileMetadata.tags.HashSha256Tag import com.vitorpamplona.quartz.nip94FileMetadata.tags.MimeTypeTag @@ -82,7 +82,9 @@ abstract class AddressableVideoEvent( override fun duration() = tags.firstNotNullOfOrNull(DurationTag::parse) - override fun textTrack() = tags.mapNotNull(ETag::parse) + // `text-track`, not `e`: reading ETag here returned the event's unrelated `e` tags + // (on a divine.video short, its "audio" source pointer) and never a caption track. + override fun textTrack() = tags.mapNotNull(TextTrackTag::parse) override fun segments() = tags.mapNotNull(SegmentTag::parse) diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip71Video/RegularVideoEvent.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip71Video/RegularVideoEvent.kt index b97799e9c8..54d474b732 100644 --- a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip71Video/RegularVideoEvent.kt +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip71Video/RegularVideoEvent.kt @@ -23,7 +23,6 @@ package com.vitorpamplona.quartz.nip71Video import androidx.compose.runtime.Immutable import com.vitorpamplona.quartz.nip01Core.core.Event import com.vitorpamplona.quartz.nip01Core.core.HexKey -import com.vitorpamplona.quartz.nip01Core.tags.events.ETag import com.vitorpamplona.quartz.nip01Core.tags.hashtags.hashtags import com.vitorpamplona.quartz.nip01Core.tags.people.PTag import com.vitorpamplona.quartz.nip01Core.tags.publishedAt.PublishedAtProvider @@ -34,6 +33,7 @@ import com.vitorpamplona.quartz.nip50Search.IndexableFieldVisitor import com.vitorpamplona.quartz.nip50Search.SearchableEvent import com.vitorpamplona.quartz.nip71Video.tags.DurationTag import com.vitorpamplona.quartz.nip71Video.tags.SegmentTag +import com.vitorpamplona.quartz.nip71Video.tags.TextTrackTag import com.vitorpamplona.quartz.nip92IMeta.imetas import com.vitorpamplona.quartz.nip94FileMetadata.tags.HashSha256Tag import com.vitorpamplona.quartz.nip94FileMetadata.tags.MimeTypeTag @@ -71,7 +71,9 @@ abstract class RegularVideoEvent( override fun duration() = tags.firstNotNullOfOrNull(DurationTag::parse) - override fun textTrack() = tags.mapNotNull(ETag::parse) + // `text-track`, not `e`: reading ETag here returned the event's unrelated `e` tags + // (on a divine.video short, its "audio" source pointer) and never a caption track. + override fun textTrack() = tags.mapNotNull(TextTrackTag::parse) override fun segments() = tags.mapNotNull(SegmentTag::parse) diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip71Video/VideoEvent.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip71Video/VideoEvent.kt index f0003229f5..0398077b13 100644 --- a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip71Video/VideoEvent.kt +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip71Video/VideoEvent.kt @@ -22,9 +22,9 @@ package com.vitorpamplona.quartz.nip71Video import androidx.compose.runtime.Immutable import com.vitorpamplona.quartz.nip01Core.core.IEvent -import com.vitorpamplona.quartz.nip01Core.tags.events.ETag import com.vitorpamplona.quartz.nip01Core.tags.people.PTag import com.vitorpamplona.quartz.nip71Video.tags.SegmentTag +import com.vitorpamplona.quartz.nip71Video.tags.TextTrackTag @Immutable interface VideoEvent : IEvent { @@ -34,7 +34,7 @@ interface VideoEvent : IEvent { fun duration(): Int? - fun textTrack(): List + fun textTrack(): List fun segments(): List diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip71Video/tags/TextTrackTag.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip71Video/tags/TextTrackTag.kt index 017928ea68..589f381dcb 100644 --- a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip71Video/tags/TextTrackTag.kt +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip71Video/tags/TextTrackTag.kt @@ -20,16 +20,34 @@ */ package com.vitorpamplona.quartz.nip71Video.tags -import com.vitorpamplona.quartz.nip01Core.core.HexKey import com.vitorpamplona.quartz.nip01Core.core.has -import com.vitorpamplona.quartz.utils.arrayOfNotNull import com.vitorpamplona.quartz.utils.ensure +/** + * NIP-71 `text-track`: supplementary timed text for a video (captions, subtitles, chapters, + * metadata). + * + * The spec is inconsistent about the payload — its prose calls [ref] a "link to WebVTT file" + * while its example writes an encoded event — and publishers use both. divine.video emits a + * Blossom URL and an addressable `39307::subtitles:` coordinate for the same track, + * so [ref] is deliberately untyped: whatever identifies the track. Positions 3 and 4 carry the + * kind of information and its language code, per the prose. + */ data class TextTrackTag( - val eventId: HexKey, - var relay: String? = null, + val ref: String, + val relay: String? = null, + val type: String? = null, + val language: String? = null, ) { - fun toTagArray() = arrayOfNotNull(TAG_NAME, eventId, relay) + // Positions are meaningful, so a gap before a field that IS set has to be written as an + // empty string rather than dropped — otherwise `language` would be read as `type`. + fun toTagArray(): Array = + when { + language != null -> arrayOf(TAG_NAME, ref, relay ?: "", type ?: "", language) + type != null -> arrayOf(TAG_NAME, ref, relay ?: "", type) + relay != null -> arrayOf(TAG_NAME, ref, relay) + else -> arrayOf(TAG_NAME, ref) + } companion object { const val TAG_NAME = "text-track" @@ -38,12 +56,19 @@ data class TextTrackTag( ensure(tag.has(1)) { return null } ensure(tag[0] == TAG_NAME) { return null } ensure(tag[1].isNotEmpty()) { return null } - return TextTrackTag(tag[1], tag.getOrNull(2)) + return TextTrackTag( + ref = tag[1], + relay = tag.getOrNull(2)?.ifBlank { null }, + type = tag.getOrNull(3)?.ifBlank { null }, + language = tag.getOrNull(4)?.ifBlank { null }, + ) } fun assemble( - eventId: HexKey, + ref: String, relay: String?, - ) = arrayOfNotNull(TAG_NAME, eventId, relay) + type: String? = null, + language: String? = null, + ) = TextTrackTag(ref, relay, type, language).toTagArray() } } From 13ac7a35e4ee1fcb27739aad4c1fbcbbe014a596 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 17 Sep 2026 19:37:40 +0000 Subject: [PATCH 02/11] feat(nip71): captions, credits, video lists and collaboration responses MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Follows the divine.video interop review. Five things a NIP-71 video can say that Amethyst had no way to show: **Captions.** New kind-39307 TextTrackEvent (WebVTT in content, plus url/m/l and an `a` back-pointer), registered in EventFactory, consumed by LocalCache and indexed for NIP-50. captionTracks() splits a video's `text-track` tags into directly-loadable URLs and coordinates to fetch; rememberCaptionTracks resolves the latter through observeNoteEvent — which also puts them on an EventFinder subscription — and merges by URL, so the duplicate URL+coordinate pair divine.video publishes for one track becomes one track. They side-load as MediaItem.SubtitleConfiguration and RenderCaptions draws the cues, because media3's Compose UI has no subtitle view. CustomMediaSourceFactory carried a standing note that its explicit HLS path skips what DefaultMediaSourceFactory wraps around a source — side-loaded subtitles among them — and that anything added later must be mirrored. It is now, so the MergingMediaSource wrap is rebuilt there; otherwise a caption track on an adaptive video would load into nothing. **Credits.** VideoCredits reads the marker off `p`/`a`/`e` tags and resolves the two conventions that collide there: divine-mobile writes `[p, key, relay, "inspired-by"]`, divine-web's collaborator invite writes `[p, key, "Collaborator"]`. Both are unambiguous once you ask whether slot 2 parses as a relay. A bare `e` tag is deliberately not a credit — no marker, nothing being credited. **Kind 30005.** LocalCache dropped video curation sets as unsupported and nothing rendered them. Consumed now, with a card showing title, description, count and a poster strip. Divine keeps every member in the NIP-51 encrypted content, so private items are decrypted off-composition for the list's owner and the card says so for everyone else rather than looking broken. **Kind 34238.** VideoCollaborationEvent accepts both published shapes (coordinate-keyed `d` and random `d`); an absent `status` is an acceptance, since divine-web only emits the event on approval at all. A credited person who accepted gets a check beside their name, addressed directly as 34238::