mirror of
https://github.com/vitorpamplona/amethyst.git
synced 2026-10-06 03:38:23 +00:00
refactor(commons): move the event cache into commonMain
EventCache, LocalCache, LocalCacheHost, AntiSpamFilter, CachePruner,
CacheSearch and OnchainZapResolver are now shared; commons/jvmAndroid keeps
only the LargeSoftCache actual.
The three blockers were cleared in the preceding commits. What the move itself
turned up was the part no import grep could see, since these are JVM APIs that
need no import: @Synchronized and synchronized {} became KmpLock, @Volatile
became kotlin.concurrent.Volatile, System.nanoTime became TimeSource.Monotonic,
HashMap.merge became a local mergeMax, and Dispatchers.IO just needed
kotlinx.coroutines.IO imported.
Two references could not follow the file. The one dateFormatter call was a log
line and now prints the raw created_at. LnurlEndpointCache is a quartz
jvmAndroid singleton the outbound-zap resolver fills, so the zap-receipt
validation reads it through a new LocalCacheHost.lnurlEndpoint port; its
default null is "not cached", which is what a cache miss already meant — the
receipt takes the resolver path instead.
This is not iOS support. iOS compiles the cache now; its store and its two
observables still throw there. The gap is three named files rather than a
module boundary.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkULS5SVq4GHDdoCzajKi8
This commit is contained in:
@@ -28,6 +28,8 @@ import com.vitorpamplona.amethyst.service.checkNotInMainThread
|
||||
import com.vitorpamplona.quartz.nip01Core.core.HexKey
|
||||
import com.vitorpamplona.quartz.nip01Core.relay.client.stats.RelayStats
|
||||
import com.vitorpamplona.quartz.nip01Core.relay.normalizer.NormalizedRelayUrl
|
||||
import com.vitorpamplona.quartz.nip57Zaps.validate.LnurlEndpointCache
|
||||
import com.vitorpamplona.quartz.nip57Zaps.validate.LnurlEndpointInfo
|
||||
import kotlinx.coroutines.CoroutineScope
|
||||
|
||||
/**
|
||||
@@ -54,5 +56,9 @@ class AmethystLocalCacheHost(
|
||||
|
||||
override fun relaySelfPubKey(relay: NormalizedRelayUrl): HexKey? = modules.nip11Cache.getFromCache(relay).self
|
||||
|
||||
// The zap path's LNURL cache is a quartz-side singleton the outbound-zap resolver
|
||||
// fills; it is jvmAndroid-only, which is why the cache reads it through here.
|
||||
override fun lnurlEndpoint(lnurlpUrl: String): LnurlEndpointInfo? = LnurlEndpointCache.get(lnurlpUrl)
|
||||
|
||||
override fun assertNotMainThread() = checkNotInMainThread()
|
||||
}
|
||||
|
||||
@@ -1161,3 +1161,61 @@ Verified: `:commons:compileCommonMainKotlinMetadata`,
|
||||
`:commons:compileIosMainKotlinMetadata`, `:commons:jvmTest` (incl. the moved
|
||||
`NwcPaymentTrackerTest`, 4 tests), `:desktopApp:test`, `:cli:test`,
|
||||
`:amethyst:compileFdroidDebugKotlin`, `DvmHeartbeatTest`, `spotlessCheck`.
|
||||
|
||||
### Steps 3–5 shipped: the cache group is `commonMain`
|
||||
|
||||
`commons/src/jvmAndroid/…/model/cache/` now holds exactly one file, the
|
||||
`LargeSoftCache` actual. Everything else — `EventCache` (4k lines),
|
||||
`LocalCache`, `LocalCacheHost`, `AntiSpamFilter`, `CachePruner`, `CacheSearch`,
|
||||
`OnchainZapResolver` — is shared.
|
||||
|
||||
**The NIP-95 sink.** `nip95BlobDir: File?` became `nip95Blobs: Nip95BlobStore?`,
|
||||
because a directory is not what the cache needs — somewhere to put bytes and a
|
||||
way to ask whether they are there already is. `FileSystemNip95BlobStore` is the
|
||||
okio-backed one, and okio was already a commons dependency. Its constructor
|
||||
takes a plain path string so Android needs no okio of its own, and
|
||||
`platformFileSystem` is a two-line expect/actual because okio declares
|
||||
`FileSystem.SYSTEM` per platform.
|
||||
|
||||
Two bugs surfaced while rewriting that block. `decode()` returns `ByteArray?`
|
||||
and went straight into `FileOutputStream.write` — a platform type, so a null
|
||||
would have thrown past the `IOException` catch. And the note's copy dropped its
|
||||
content whenever a directory existed, *including* right after a failed write,
|
||||
losing the only copy of those bytes; content is now dropped only when the blob
|
||||
really is stored.
|
||||
|
||||
**`SortedSet`.** `EventCache.filter` returns a `List<Note>` in the comparator's
|
||||
order, newest first. That order is load-bearing (the napplet gateway answers
|
||||
REQs from it). The comparator also defines uniqueness by reference, so the
|
||||
sorted set was collapsing the duplicate references a filter with a repeated
|
||||
kind produces; `toSet()` before sorting keeps exactly that, since `Note`
|
||||
declares no `equals()`.
|
||||
|
||||
**The observables.** `NoteListMatchingFilter` / `EventListMatchingFilter` are
|
||||
now `expect`/`actual` with today's implementation untouched as the jvmAndroid
|
||||
actual and a throwing iOS stub. Their concurrency — every sorted-set write
|
||||
inside that key's `ConcurrentHashMap.compute` critical section — is not
|
||||
reassemblable from quartz's KMP primitives: `getOrPut` covers `new()`, but
|
||||
`remove()` needs the sorted-set removal *inside* the section, or a concurrent
|
||||
re-add inserts a comparator-equal entry that the later removal takes out
|
||||
instead, dropping the note for good. A copy-on-write `compute` cannot stand in
|
||||
either, since its CAS retry may run a side-effecting lambda twice.
|
||||
|
||||
**What the move itself turned up**, none of it visible to an import grep:
|
||||
`@Synchronized` / `@Volatile` / `synchronized {}` (→ `KmpLock`,
|
||||
`kotlin.concurrent.Volatile`), `System.nanoTime` (→ `TimeSource.Monotonic`),
|
||||
`HashMap.merge` (→ a local `mergeMax`), `Dispatchers.IO` (just needs
|
||||
`import kotlinx.coroutines.IO`), the one `dateFormatter` log line (now logs the
|
||||
raw `created_at`), and `LnurlEndpointCache` — a quartz **jvmAndroid** singleton,
|
||||
reached through a new `LocalCacheHost.lnurlEndpoint` port whose default `null`
|
||||
means "cold cache", which is what a miss already meant.
|
||||
|
||||
**Still true, and worth repeating:** commonMain is not iOS support. iOS now
|
||||
compiles the cache, and the cache's store and its two observables throw there.
|
||||
The remaining gap is three named files, not a module boundary.
|
||||
|
||||
Verified: `:commons:compileCommonMainKotlinMetadata`,
|
||||
`:commons:compileIosMainKotlinMetadata`, `:commons:verifyKmpPurity`,
|
||||
`:commons:jvmTest`, `:desktopApp:test`, `:cli:test`,
|
||||
`:amethyst:compileFdroidDebugKotlin`, `:amethyst:testPlayDebugUnitTest`,
|
||||
`spotlessCheck`.
|
||||
|
||||
+32
-26
@@ -21,7 +21,9 @@
|
||||
package com.vitorpamplona.amethyst.commons.model.cache
|
||||
|
||||
import androidx.collection.LruCache
|
||||
import com.vitorpamplona.amethyst.commons.util.KmpLock
|
||||
import com.vitorpamplona.amethyst.commons.util.njumpLink
|
||||
import com.vitorpamplona.amethyst.commons.util.withLock
|
||||
import com.vitorpamplona.quartz.nip01Core.core.Address
|
||||
import com.vitorpamplona.quartz.nip01Core.core.AddressableEvent
|
||||
import com.vitorpamplona.quartz.nip01Core.core.Event
|
||||
@@ -160,39 +162,43 @@ class AntiSpamFilter(
|
||||
return false
|
||||
}
|
||||
|
||||
@Synchronized
|
||||
// Was @Synchronized, which is a JVM-only annotation: same mutual exclusion, spelled
|
||||
// for every target. Only this method touches spamMessages for a given hash.
|
||||
private val offenderLock = KmpLock()
|
||||
|
||||
private fun logOffender(
|
||||
hashCode: Int,
|
||||
event: Event,
|
||||
): Spammer {
|
||||
val spammer = spamMessages.get(hashCode)
|
||||
): Spammer =
|
||||
offenderLock.withLock {
|
||||
val spammer = spamMessages.get(hashCode)
|
||||
|
||||
return if (spammer == null) {
|
||||
val newSpammer =
|
||||
if (event is AddressableEvent) {
|
||||
Spammer(
|
||||
pubkeyHex = event.pubKey,
|
||||
duplicatedEventIds = setOf(),
|
||||
duplicatedEventAddresses = setOfNotNull(recentAddressables[hashCode], event.address()),
|
||||
)
|
||||
} else {
|
||||
Spammer(
|
||||
pubkeyHex = event.pubKey,
|
||||
duplicatedEventIds = setOfNotNull(recentEventIds[hashCode], event.id),
|
||||
duplicatedEventAddresses = setOf(),
|
||||
)
|
||||
}
|
||||
spamMessages.put(hashCode, newSpammer)
|
||||
newSpammer
|
||||
} else {
|
||||
if (event is AddressableEvent) {
|
||||
spammer.duplicatedEventAddresses += event.address()
|
||||
if (spammer == null) {
|
||||
val newSpammer =
|
||||
if (event is AddressableEvent) {
|
||||
Spammer(
|
||||
pubkeyHex = event.pubKey,
|
||||
duplicatedEventIds = setOf(),
|
||||
duplicatedEventAddresses = setOfNotNull(recentAddressables[hashCode], event.address()),
|
||||
)
|
||||
} else {
|
||||
Spammer(
|
||||
pubkeyHex = event.pubKey,
|
||||
duplicatedEventIds = setOfNotNull(recentEventIds[hashCode], event.id),
|
||||
duplicatedEventAddresses = setOf(),
|
||||
)
|
||||
}
|
||||
spamMessages.put(hashCode, newSpammer)
|
||||
newSpammer
|
||||
} else {
|
||||
spammer.duplicatedEventIds += event.id
|
||||
if (event is AddressableEvent) {
|
||||
spammer.duplicatedEventAddresses += event.address()
|
||||
} else {
|
||||
spammer.duplicatedEventIds += event.id
|
||||
}
|
||||
spammer
|
||||
}
|
||||
spammer
|
||||
}
|
||||
}
|
||||
|
||||
val flowSpam = MutableStateFlow(AntiSpamState(this))
|
||||
}
|
||||
+15
-3
@@ -214,12 +214,12 @@ class CachePruner(
|
||||
is BaseDMGroupEvent ->
|
||||
if (giftWrapFloor != null) {
|
||||
val outerUntil = note.rumorHost?.createdAt ?: ev.createdAt
|
||||
if (outerUntil < giftWrapFloor) note.relays.forEach { giftWrapPruned.merge(it, outerUntil, ::maxOf) }
|
||||
if (outerUntil < giftWrapFloor) note.relays.forEach { giftWrapPruned.mergeMax(it, outerUntil) }
|
||||
}
|
||||
is PrivateDmEvent -> {
|
||||
val until = ev.createdAt
|
||||
if (accountNip04Floor != null && until < accountNip04Floor) note.relays.forEach { accountNip04Pruned.merge(it, until, ::maxOf) }
|
||||
if (roomNip04Floor != null && until < roomNip04Floor) note.relays.forEach { roomNip04Pruned.merge(it, until, ::maxOf) }
|
||||
if (accountNip04Floor != null && until < accountNip04Floor) note.relays.forEach { accountNip04Pruned.mergeMax(it, until) }
|
||||
if (roomNip04Floor != null && until < roomNip04Floor) note.relays.forEach { roomNip04Pruned.mergeMax(it, until) }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -500,3 +500,15 @@ class CachePruner(
|
||||
println("PRUNE: ${toBeRemoved.size} messages removed because they were Hidden")
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* `Map.merge(key, value, ::maxOf)` spelled for every target — the JVM's own merge is not in the
|
||||
* common stdlib. Keeps the highest value seen for [relay].
|
||||
*/
|
||||
private fun MutableMap<NormalizedRelayUrl, Long>.mergeMax(
|
||||
relay: NormalizedRelayUrl,
|
||||
value: Long,
|
||||
) {
|
||||
val existing = this[relay]
|
||||
this[relay] = if (existing == null) value else maxOf(existing, value)
|
||||
}
|
||||
+11
-8
@@ -58,7 +58,8 @@ import com.vitorpamplona.amethyst.commons.model.privateChats.ChatroomList
|
||||
import com.vitorpamplona.amethyst.commons.model.redirectStrayRelayGroupContent
|
||||
import com.vitorpamplona.amethyst.commons.service.BundledInsert
|
||||
import com.vitorpamplona.amethyst.commons.service.nwc.NwcPaymentTracker
|
||||
import com.vitorpamplona.amethyst.commons.util.dateFormatter
|
||||
import com.vitorpamplona.amethyst.commons.util.KmpLock
|
||||
import com.vitorpamplona.amethyst.commons.util.withLock
|
||||
import com.vitorpamplona.quartz.buzz.aeEngrams.EngramEvent
|
||||
import com.vitorpamplona.quartz.buzz.agentProfiles.AgentProfileEvent
|
||||
import com.vitorpamplona.quartz.buzz.amTurnMetrics.AgentTurnMetricEvent
|
||||
@@ -320,7 +321,6 @@ import com.vitorpamplona.quartz.nip56Reports.ReportEvent
|
||||
import com.vitorpamplona.quartz.nip57Zaps.LnZapEvent
|
||||
import com.vitorpamplona.quartz.nip57Zaps.LnZapRequestEvent
|
||||
import com.vitorpamplona.quartz.nip57Zaps.validate.LnZapReceiptValidator
|
||||
import com.vitorpamplona.quartz.nip57Zaps.validate.LnurlEndpointCache
|
||||
import com.vitorpamplona.quartz.nip57Zaps.validate.LnurlEndpointResolver
|
||||
import com.vitorpamplona.quartz.nip57Zaps.validate.LnurlForm
|
||||
import com.vitorpamplona.quartz.nip58Badges.accepted.AcceptedBadgeSetEvent
|
||||
@@ -413,6 +413,7 @@ import com.vitorpamplona.quartz.utils.TimeUtils
|
||||
import com.vitorpamplona.quartz.utils.cache.LargeCache
|
||||
import kotlinx.coroutines.CancellationException
|
||||
import kotlinx.coroutines.Dispatchers
|
||||
import kotlinx.coroutines.IO
|
||||
import kotlinx.coroutines.channels.BufferOverflow
|
||||
import kotlinx.coroutines.channels.awaitClose
|
||||
import kotlinx.coroutines.flow.Flow
|
||||
@@ -422,6 +423,8 @@ import kotlinx.coroutines.flow.buffer
|
||||
import kotlinx.coroutines.flow.callbackFlow
|
||||
import kotlinx.coroutines.flow.map
|
||||
import kotlinx.coroutines.launch
|
||||
import kotlin.concurrent.Volatile
|
||||
import kotlin.time.TimeSource
|
||||
|
||||
/**
|
||||
* The in-memory event store: every `Note`, `User` and `Channel` the app has consumed, plus the
|
||||
@@ -509,7 +512,7 @@ open class EventCache :
|
||||
val mintDirectory = MintDirectoryIndex()
|
||||
|
||||
@Volatile private var mintDirectoryBackfilled = false
|
||||
private val mintDirectoryBackfillLock = Any()
|
||||
private val mintDirectoryBackfillLock = KmpLock()
|
||||
|
||||
/**
|
||||
* Sweeps `notes` + `addressables` for any NIP-87 / NIP-61 event the
|
||||
@@ -525,7 +528,7 @@ open class EventCache :
|
||||
*/
|
||||
fun ensureMintDirectoryBackfilled() {
|
||||
if (mintDirectoryBackfilled) return
|
||||
synchronized(mintDirectoryBackfillLock) {
|
||||
mintDirectoryBackfillLock.withLock {
|
||||
if (mintDirectoryBackfilled) return
|
||||
runCatching {
|
||||
notes.forEach { _, note -> note.event?.let(::updateMintIndex) }
|
||||
@@ -2678,7 +2681,7 @@ open class EventCache :
|
||||
// stays in cache as a visible artifact but contributes 0 to zap totals.
|
||||
val recipientLnurl = recipientLnurl(event)
|
||||
val recipientLnurlpUrl = recipientLnurl?.let { LnurlForm.toUrl(it) }
|
||||
val cachedInfo = recipientLnurlpUrl?.let { LnurlEndpointCache.get(it) }
|
||||
val cachedInfo = recipientLnurlpUrl?.let { appHost.lnurlEndpoint(it) }
|
||||
|
||||
val author = getOrCreateUser(event.pubKey)
|
||||
val repliesTo = computeReplyTo(event)
|
||||
@@ -3334,9 +3337,9 @@ open class EventCache :
|
||||
val meter = verifyMeter
|
||||
if (meter == null) return justVerifyInner(event)
|
||||
|
||||
val start = System.nanoTime()
|
||||
val start = TimeSource.Monotonic.markNow()
|
||||
val valid = justVerifyInner(event)
|
||||
meter(System.nanoTime() - start, valid)
|
||||
meter(start.elapsedNow().inWholeNanoseconds, valid)
|
||||
return valid
|
||||
}
|
||||
|
||||
@@ -3346,7 +3349,7 @@ open class EventCache :
|
||||
event.checkSignature()
|
||||
} catch (e: Exception) {
|
||||
if (e is CancellationException) throw e
|
||||
Log.w("Event Verification Failed") { "Kind: ${event.kind} from ${dateFormatter(event.createdAt, "", "")} with message ${e.message}" }
|
||||
Log.w("Event Verification Failed") { "Kind: ${event.kind} created at ${event.createdAt} with message ${e.message}" }
|
||||
}
|
||||
false
|
||||
} else {
|
||||
+12
@@ -23,8 +23,10 @@ package com.vitorpamplona.amethyst.commons.model.cache
|
||||
import com.vitorpamplona.quartz.nip01Core.core.HexKey
|
||||
import com.vitorpamplona.quartz.nip01Core.relay.client.stats.RelayStats
|
||||
import com.vitorpamplona.quartz.nip01Core.relay.normalizer.NormalizedRelayUrl
|
||||
import com.vitorpamplona.quartz.nip57Zaps.validate.LnurlEndpointInfo
|
||||
import kotlinx.coroutines.CoroutineScope
|
||||
import kotlinx.coroutines.Dispatchers
|
||||
import kotlinx.coroutines.IO
|
||||
import kotlinx.coroutines.SupervisorJob
|
||||
|
||||
/**
|
||||
@@ -70,6 +72,16 @@ interface LocalCacheHost {
|
||||
*/
|
||||
fun relaySelfPubKey(relay: NormalizedRelayUrl): HexKey? = null
|
||||
|
||||
/**
|
||||
* LNURL-pay endpoint metadata the shell has already resolved for [lnurlpUrl], used to
|
||||
* validate a zap receipt's signer against the provider's `nostrPubkey` (NIP-57 Appendix F).
|
||||
*
|
||||
* `null` is "not cached", which is what a cache miss already means here: the receipt takes
|
||||
* the slower resolver path instead. A host that caches nothing therefore behaves like a
|
||||
* cold cache, not like a broken one.
|
||||
*/
|
||||
fun lnurlEndpoint(lnurlpUrl: String): LnurlEndpointInfo? = null
|
||||
|
||||
/**
|
||||
* Throws if called on the platform's main thread. Signature verification and the cache
|
||||
* sweeps are far too slow to run there. A no-op by default and on release builds.
|
||||
Reference in New Issue
Block a user