refactor(commons): move the event cache into commonMain

EventCache, LocalCache, LocalCacheHost, AntiSpamFilter, CachePruner,
CacheSearch and OnchainZapResolver are now shared; commons/jvmAndroid keeps
only the LargeSoftCache actual.

The three blockers were cleared in the preceding commits. What the move itself
turned up was the part no import grep could see, since these are JVM APIs that
need no import: @Synchronized and synchronized {} became KmpLock, @Volatile
became kotlin.concurrent.Volatile, System.nanoTime became TimeSource.Monotonic,
HashMap.merge became a local mergeMax, and Dispatchers.IO just needed
kotlinx.coroutines.IO imported.

Two references could not follow the file. The one dateFormatter call was a log
line and now prints the raw created_at. LnurlEndpointCache is a quartz
jvmAndroid singleton the outbound-zap resolver fills, so the zap-receipt
validation reads it through a new LocalCacheHost.lnurlEndpoint port; its
default null is "not cached", which is what a cache miss already meant — the
receipt takes the resolver path instead.

This is not iOS support. iOS compiles the cache now; its store and its two
observables still throw there. The gap is three named files rather than a
module boundary.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkULS5SVq4GHDdoCzajKi8
This commit is contained in:
Claude
2026-09-21 15:43:23 +00:00
parent ee636dcf17
commit f2e4454e8e
9 changed files with 134 additions and 37 deletions
@@ -28,6 +28,8 @@ import com.vitorpamplona.amethyst.service.checkNotInMainThread
import com.vitorpamplona.quartz.nip01Core.core.HexKey
import com.vitorpamplona.quartz.nip01Core.relay.client.stats.RelayStats
import com.vitorpamplona.quartz.nip01Core.relay.normalizer.NormalizedRelayUrl
import com.vitorpamplona.quartz.nip57Zaps.validate.LnurlEndpointCache
import com.vitorpamplona.quartz.nip57Zaps.validate.LnurlEndpointInfo
import kotlinx.coroutines.CoroutineScope
/**
@@ -54,5 +56,9 @@ class AmethystLocalCacheHost(
override fun relaySelfPubKey(relay: NormalizedRelayUrl): HexKey? = modules.nip11Cache.getFromCache(relay).self
// The zap path's LNURL cache is a quartz-side singleton the outbound-zap resolver
// fills; it is jvmAndroid-only, which is why the cache reads it through here.
override fun lnurlEndpoint(lnurlpUrl: String): LnurlEndpointInfo? = LnurlEndpointCache.get(lnurlpUrl)
override fun assertNotMainThread() = checkNotInMainThread()
}
@@ -1161,3 +1161,61 @@ Verified: `:commons:compileCommonMainKotlinMetadata`,
`:commons:compileIosMainKotlinMetadata`, `:commons:jvmTest` (incl. the moved
`NwcPaymentTrackerTest`, 4 tests), `:desktopApp:test`, `:cli:test`,
`:amethyst:compileFdroidDebugKotlin`, `DvmHeartbeatTest`, `spotlessCheck`.
### Steps 3–5 shipped: the cache group is `commonMain`
`commons/src/jvmAndroid/…/model/cache/` now holds exactly one file, the
`LargeSoftCache` actual. Everything else — `EventCache` (4k lines),
`LocalCache`, `LocalCacheHost`, `AntiSpamFilter`, `CachePruner`, `CacheSearch`,
`OnchainZapResolver` — is shared.
**The NIP-95 sink.** `nip95BlobDir: File?` became `nip95Blobs: Nip95BlobStore?`,
because a directory is not what the cache needs — somewhere to put bytes and a
way to ask whether they are there already is. `FileSystemNip95BlobStore` is the
okio-backed one, and okio was already a commons dependency. Its constructor
takes a plain path string so Android needs no okio of its own, and
`platformFileSystem` is a two-line expect/actual because okio declares
`FileSystem.SYSTEM` per platform.
Two bugs surfaced while rewriting that block. `decode()` returns `ByteArray?`
and went straight into `FileOutputStream.write` — a platform type, so a null
would have thrown past the `IOException` catch. And the note's copy dropped its
content whenever a directory existed, *including* right after a failed write,
losing the only copy of those bytes; content is now dropped only when the blob
really is stored.
**`SortedSet`.** `EventCache.filter` returns a `List<Note>` in the comparator's
order, newest first. That order is load-bearing (the napplet gateway answers
REQs from it). The comparator also defines uniqueness by reference, so the
sorted set was collapsing the duplicate references a filter with a repeated
kind produces; `toSet()` before sorting keeps exactly that, since `Note`
declares no `equals()`.
**The observables.** `NoteListMatchingFilter` / `EventListMatchingFilter` are
now `expect`/`actual` with today's implementation untouched as the jvmAndroid
actual and a throwing iOS stub. Their concurrency — every sorted-set write
inside that key's `ConcurrentHashMap.compute` critical section — is not
reassemblable from quartz's KMP primitives: `getOrPut` covers `new()`, but
`remove()` needs the sorted-set removal *inside* the section, or a concurrent
re-add inserts a comparator-equal entry that the later removal takes out
instead, dropping the note for good. A copy-on-write `compute` cannot stand in
either, since its CAS retry may run a side-effecting lambda twice.
**What the move itself turned up**, none of it visible to an import grep:
`@Synchronized` / `@Volatile` / `synchronized {}` (→ `KmpLock`,
`kotlin.concurrent.Volatile`), `System.nanoTime` (→ `TimeSource.Monotonic`),
`HashMap.merge` (→ a local `mergeMax`), `Dispatchers.IO` (just needs
`import kotlinx.coroutines.IO`), the one `dateFormatter` log line (now logs the
raw `created_at`), and `LnurlEndpointCache` — a quartz **jvmAndroid** singleton,
reached through a new `LocalCacheHost.lnurlEndpoint` port whose default `null`
means "cold cache", which is what a miss already meant.
**Still true, and worth repeating:** commonMain is not iOS support. iOS now
compiles the cache, and the cache's store and its two observables throw there.
The remaining gap is three named files, not a module boundary.
Verified: `:commons:compileCommonMainKotlinMetadata`,
`:commons:compileIosMainKotlinMetadata`, `:commons:verifyKmpPurity`,
`:commons:jvmTest`, `:desktopApp:test`, `:cli:test`,
`:amethyst:compileFdroidDebugKotlin`, `:amethyst:testPlayDebugUnitTest`,
`spotlessCheck`.
@@ -21,7 +21,9 @@
package com.vitorpamplona.amethyst.commons.model.cache
import androidx.collection.LruCache
import com.vitorpamplona.amethyst.commons.util.KmpLock
import com.vitorpamplona.amethyst.commons.util.njumpLink
import com.vitorpamplona.amethyst.commons.util.withLock
import com.vitorpamplona.quartz.nip01Core.core.Address
import com.vitorpamplona.quartz.nip01Core.core.AddressableEvent
import com.vitorpamplona.quartz.nip01Core.core.Event
@@ -160,39 +162,43 @@ class AntiSpamFilter(
return false
}
@Synchronized
// Was @Synchronized, which is a JVM-only annotation: same mutual exclusion, spelled
// for every target. Only this method touches spamMessages for a given hash.
private val offenderLock = KmpLock()
private fun logOffender(
hashCode: Int,
event: Event,
): Spammer {
val spammer = spamMessages.get(hashCode)
): Spammer =
offenderLock.withLock {
val spammer = spamMessages.get(hashCode)
return if (spammer == null) {
val newSpammer =
if (event is AddressableEvent) {
Spammer(
pubkeyHex = event.pubKey,
duplicatedEventIds = setOf(),
duplicatedEventAddresses = setOfNotNull(recentAddressables[hashCode], event.address()),
)
} else {
Spammer(
pubkeyHex = event.pubKey,
duplicatedEventIds = setOfNotNull(recentEventIds[hashCode], event.id),
duplicatedEventAddresses = setOf(),
)
}
spamMessages.put(hashCode, newSpammer)
newSpammer
} else {
if (event is AddressableEvent) {
spammer.duplicatedEventAddresses += event.address()
if (spammer == null) {
val newSpammer =
if (event is AddressableEvent) {
Spammer(
pubkeyHex = event.pubKey,
duplicatedEventIds = setOf(),
duplicatedEventAddresses = setOfNotNull(recentAddressables[hashCode], event.address()),
)
} else {
Spammer(
pubkeyHex = event.pubKey,
duplicatedEventIds = setOfNotNull(recentEventIds[hashCode], event.id),
duplicatedEventAddresses = setOf(),
)
}
spamMessages.put(hashCode, newSpammer)
newSpammer
} else {
spammer.duplicatedEventIds += event.id
if (event is AddressableEvent) {
spammer.duplicatedEventAddresses += event.address()
} else {
spammer.duplicatedEventIds += event.id
}
spammer
}
spammer
}
}
val flowSpam = MutableStateFlow(AntiSpamState(this))
}
@@ -214,12 +214,12 @@ class CachePruner(
is BaseDMGroupEvent ->
if (giftWrapFloor != null) {
val outerUntil = note.rumorHost?.createdAt ?: ev.createdAt
if (outerUntil < giftWrapFloor) note.relays.forEach { giftWrapPruned.merge(it, outerUntil, ::maxOf) }
if (outerUntil < giftWrapFloor) note.relays.forEach { giftWrapPruned.mergeMax(it, outerUntil) }
}
is PrivateDmEvent -> {
val until = ev.createdAt
if (accountNip04Floor != null && until < accountNip04Floor) note.relays.forEach { accountNip04Pruned.merge(it, until, ::maxOf) }
if (roomNip04Floor != null && until < roomNip04Floor) note.relays.forEach { roomNip04Pruned.merge(it, until, ::maxOf) }
if (accountNip04Floor != null && until < accountNip04Floor) note.relays.forEach { accountNip04Pruned.mergeMax(it, until) }
if (roomNip04Floor != null && until < roomNip04Floor) note.relays.forEach { roomNip04Pruned.mergeMax(it, until) }
}
}
@@ -500,3 +500,15 @@ class CachePruner(
println("PRUNE: ${toBeRemoved.size} messages removed because they were Hidden")
}
}
/**
* `Map.merge(key, value, ::maxOf)` spelled for every target — the JVM's own merge is not in the
* common stdlib. Keeps the highest value seen for [relay].
*/
private fun MutableMap<NormalizedRelayUrl, Long>.mergeMax(
relay: NormalizedRelayUrl,
value: Long,
) {
val existing = this[relay]
this[relay] = if (existing == null) value else maxOf(existing, value)
}
@@ -58,7 +58,8 @@ import com.vitorpamplona.amethyst.commons.model.privateChats.ChatroomList
import com.vitorpamplona.amethyst.commons.model.redirectStrayRelayGroupContent
import com.vitorpamplona.amethyst.commons.service.BundledInsert
import com.vitorpamplona.amethyst.commons.service.nwc.NwcPaymentTracker
import com.vitorpamplona.amethyst.commons.util.dateFormatter
import com.vitorpamplona.amethyst.commons.util.KmpLock
import com.vitorpamplona.amethyst.commons.util.withLock
import com.vitorpamplona.quartz.buzz.aeEngrams.EngramEvent
import com.vitorpamplona.quartz.buzz.agentProfiles.AgentProfileEvent
import com.vitorpamplona.quartz.buzz.amTurnMetrics.AgentTurnMetricEvent
@@ -320,7 +321,6 @@ import com.vitorpamplona.quartz.nip56Reports.ReportEvent
import com.vitorpamplona.quartz.nip57Zaps.LnZapEvent
import com.vitorpamplona.quartz.nip57Zaps.LnZapRequestEvent
import com.vitorpamplona.quartz.nip57Zaps.validate.LnZapReceiptValidator
import com.vitorpamplona.quartz.nip57Zaps.validate.LnurlEndpointCache
import com.vitorpamplona.quartz.nip57Zaps.validate.LnurlEndpointResolver
import com.vitorpamplona.quartz.nip57Zaps.validate.LnurlForm
import com.vitorpamplona.quartz.nip58Badges.accepted.AcceptedBadgeSetEvent
@@ -413,6 +413,7 @@ import com.vitorpamplona.quartz.utils.TimeUtils
import com.vitorpamplona.quartz.utils.cache.LargeCache
import kotlinx.coroutines.CancellationException
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.IO
import kotlinx.coroutines.channels.BufferOverflow
import kotlinx.coroutines.channels.awaitClose
import kotlinx.coroutines.flow.Flow
@@ -422,6 +423,8 @@ import kotlinx.coroutines.flow.buffer
import kotlinx.coroutines.flow.callbackFlow
import kotlinx.coroutines.flow.map
import kotlinx.coroutines.launch
import kotlin.concurrent.Volatile
import kotlin.time.TimeSource
/**
* The in-memory event store: every `Note`, `User` and `Channel` the app has consumed, plus the
@@ -509,7 +512,7 @@ open class EventCache :
val mintDirectory = MintDirectoryIndex()
@Volatile private var mintDirectoryBackfilled = false
private val mintDirectoryBackfillLock = Any()
private val mintDirectoryBackfillLock = KmpLock()
/**
* Sweeps `notes` + `addressables` for any NIP-87 / NIP-61 event the
@@ -525,7 +528,7 @@ open class EventCache :
*/
fun ensureMintDirectoryBackfilled() {
if (mintDirectoryBackfilled) return
synchronized(mintDirectoryBackfillLock) {
mintDirectoryBackfillLock.withLock {
if (mintDirectoryBackfilled) return
runCatching {
notes.forEach { _, note -> note.event?.let(::updateMintIndex) }
@@ -2678,7 +2681,7 @@ open class EventCache :
// stays in cache as a visible artifact but contributes 0 to zap totals.
val recipientLnurl = recipientLnurl(event)
val recipientLnurlpUrl = recipientLnurl?.let { LnurlForm.toUrl(it) }
val cachedInfo = recipientLnurlpUrl?.let { LnurlEndpointCache.get(it) }
val cachedInfo = recipientLnurlpUrl?.let { appHost.lnurlEndpoint(it) }
val author = getOrCreateUser(event.pubKey)
val repliesTo = computeReplyTo(event)
@@ -3334,9 +3337,9 @@ open class EventCache :
val meter = verifyMeter
if (meter == null) return justVerifyInner(event)
val start = System.nanoTime()
val start = TimeSource.Monotonic.markNow()
val valid = justVerifyInner(event)
meter(System.nanoTime() - start, valid)
meter(start.elapsedNow().inWholeNanoseconds, valid)
return valid
}
@@ -3346,7 +3349,7 @@ open class EventCache :
event.checkSignature()
} catch (e: Exception) {
if (e is CancellationException) throw e
Log.w("Event Verification Failed") { "Kind: ${event.kind} from ${dateFormatter(event.createdAt, "", "")} with message ${e.message}" }
Log.w("Event Verification Failed") { "Kind: ${event.kind} created at ${event.createdAt} with message ${e.message}" }
}
false
} else {
@@ -23,8 +23,10 @@ package com.vitorpamplona.amethyst.commons.model.cache
import com.vitorpamplona.quartz.nip01Core.core.HexKey
import com.vitorpamplona.quartz.nip01Core.relay.client.stats.RelayStats
import com.vitorpamplona.quartz.nip01Core.relay.normalizer.NormalizedRelayUrl
import com.vitorpamplona.quartz.nip57Zaps.validate.LnurlEndpointInfo
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.IO
import kotlinx.coroutines.SupervisorJob
/**
@@ -70,6 +72,16 @@ interface LocalCacheHost {
*/
fun relaySelfPubKey(relay: NormalizedRelayUrl): HexKey? = null
/**
* LNURL-pay endpoint metadata the shell has already resolved for [lnurlpUrl], used to
* validate a zap receipt's signer against the provider's `nostrPubkey` (NIP-57 Appendix F).
*
* `null` is "not cached", which is what a cache miss already means here: the receipt takes
* the slower resolver path instead. A host that caches nothing therefore behaves like a
* cold cache, not like a broken one.
*/
fun lnurlEndpoint(lnurlpUrl: String): LnurlEndpointInfo? = null
/**
* Throws if called on the platform's main thread. Signature verification and the cache
* sweeps are far too slow to run there. A no-op by default and on release builds.