mirror of
https://github.com/vitorpamplona/amethyst.git
synced 2026-10-06 11:48:24 +00:00
Claude
6c07dcb839
LocalCache's linuxX64 store kept a LinkedHashMap inside an AtomicReference and replaced it wholesale on every write, so each put was O(n) in the size of the cache and filling it was O(n^2). It was not thread-safe either: the read-copy-write was not a CAS loop, so concurrent writers silently dropped each other's entries. Replace it with a mutable map guarded by PlatformLock plus a lazily rebuilt read snapshot. Point operations (get/put/remove/containsKey/size) are O(1) under the lock; bulk operations run against a point-in-time copy rebuilt at most once per write epoch, which also keeps caller-supplied lambdas out of the critical section — PlatformLock is not reentrant here and LocalCache predicates call back into the cache. Two behaviour fixes fall out of matching the JVM actual's putIfAbsent: createIfAbsent now reports true only when this call inserted (it previously returned get(key) != null, which also reported true when another thread had just created the entry), and getOrCreate publishes atomically. ConcurrentHashCache.linux gets the same treatment. Its only caller, CachingEventDecoder, writes once per event arriving from a relay, so the per-write map rebuild was the worst-placed copy of the three. None of this was caught because no CI job compiled or ran linuxX64. Add LargeCacheTest to commonTest as a cross-target contract for the ~40 methods each actual reimplements by hand, a linuxTest suite covering the concurrency this actual now has to get right on its own, and a CI leg that runs both on Linux Native. That leg is scoped to the cache and concurrency packages: the full linuxX64Test suite is 3,490 tests with 78 pre-existing failures, nearly all TODO() stubs in linux actuals that were never written (MLS crypto, the SQLite driver, NIP-44, Bolt12). Filling those in is its own project; the filter keeps the job meaningful and green, and widening it later is one line. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HxQ1QuyzSkR38iFHbREjoS
437 lines
18 KiB
YAML
437 lines
18 KiB
YAML
name: Test/Build
|
||
|
||
on:
|
||
pull_request:
|
||
branches: [main]
|
||
push:
|
||
branches: [main]
|
||
|
||
permissions:
|
||
contents: read
|
||
|
||
concurrency:
|
||
group: ${{ github.workflow }}-${{ github.ref }}
|
||
cancel-in-progress: true
|
||
|
||
jobs:
|
||
lint:
|
||
runs-on: ubuntu-latest
|
||
timeout-minutes: 15
|
||
steps:
|
||
- name: Checkout code
|
||
uses: actions/checkout@v7
|
||
|
||
- name: Orphaned translations (no locale string may outlive its default key)
|
||
run: .claude/hooks/orphan_strings_check.py
|
||
|
||
- name: Set up JDK 21
|
||
uses: actions/setup-java@v6.0.0
|
||
with:
|
||
distribution: 'temurin'
|
||
java-version: 21
|
||
|
||
# Remote Gradle build cache: writes on push to main, reads on PRs and
|
||
# other branches. Caches both `~/.gradle/caches/` and individual task
|
||
# outputs, so dependency-only changes hit the cache and skip recompiling
|
||
# downstream modules / re-merging native libs (~600MB of work on
|
||
# :amethyst alone). Replaces the narrower `cache: gradle` previously on
|
||
# actions/setup-java, which only cached `modules-2`.
|
||
- name: Set up Gradle
|
||
uses: gradle/actions/setup-gradle@v6
|
||
with:
|
||
cache-read-only: ${{ github.ref != 'refs/heads/main' }}
|
||
|
||
- name: Linter (gradle)
|
||
run: ./gradlew spotlessCheck :quartz:verifyKmpPurity :commons:verifyKmpPurity
|
||
|
||
build-desktop:
|
||
needs: lint
|
||
strategy:
|
||
fail-fast: false
|
||
matrix:
|
||
include:
|
||
- os: ubuntu-latest
|
||
desktop-task: packageDeb
|
||
desktop-artifact-name: Desktop Linux DEB
|
||
desktop-artifact-path: desktopApp/build/compose/binaries/main/deb/*.deb
|
||
- os: macos-latest
|
||
desktop-task: packageDmg
|
||
desktop-artifact-name: Desktop macOS DMG
|
||
desktop-artifact-path: desktopApp/build/compose/binaries/main/dmg/*.dmg
|
||
- os: windows-latest
|
||
desktop-task: packageMsi
|
||
desktop-artifact-name: Desktop Windows MSI
|
||
desktop-artifact-path: desktopApp/build/compose/binaries/main/msi/*.msi
|
||
runs-on: ${{ matrix.os }}
|
||
timeout-minutes: 60
|
||
defaults:
|
||
run:
|
||
shell: bash
|
||
steps:
|
||
- name: Checkout code
|
||
uses: actions/checkout@v7
|
||
|
||
- name: Set up JDK 21
|
||
uses: actions/setup-java@v6.0.0
|
||
with:
|
||
distribution: 'temurin'
|
||
java-version: 21
|
||
|
||
- name: Set up Gradle
|
||
uses: gradle/actions/setup-gradle@v6
|
||
with:
|
||
cache-read-only: ${{ github.ref != 'refs/heads/main' }}
|
||
|
||
# Compose UI smoke test (DesktopLaunchSmokeTest) uses Skiko which needs
|
||
# a display server on Linux. xvfb provides a virtual framebuffer.
|
||
- name: Install xvfb (Linux)
|
||
if: runner.os == 'Linux'
|
||
run: sudo apt-get update && sudo apt-get install -y xvfb
|
||
|
||
- name: Test + Build Desktop (gradle)
|
||
run: |
|
||
CMD="./gradlew :quartz:jvmTest :commons:jvmTest :nestsClient:jvmTest :cli:test :desktopApp:test :desktopApp:${{ matrix.desktop-task }}"
|
||
if [ "${{ runner.os }}" = "Linux" ]; then
|
||
xvfb-run --auto-servernum $CMD
|
||
else
|
||
$CMD
|
||
fi
|
||
|
||
# This job runs five test suites (:quartz, :commons, :nestsClient, :cli,
|
||
# :desktopApp) but, unlike test-geode / test-quartz-ios /
|
||
# test-and-build-android, published nothing when one of them failed. The
|
||
# console line names the failing test and the exception class and stops
|
||
# there, so the message is lost with the runner. That is how the
|
||
# NostrClientNegentropySyncTest failure in run 10540 became
|
||
# undiagnosable: NegentropySyncException carries a `detail` naming which
|
||
# branch fired (connect timeout / idle silence / NEG-ERR / disconnect),
|
||
# and nobody could read it. Same action and pin as the Android job below.
|
||
- name: Desktop Test Report
|
||
uses: mikepenz/action-junit-report@a9170d5795813c01ab4901ffb045b52bab4ab09d # v6.5.0
|
||
if: always()
|
||
with:
|
||
report_paths: '**/build/test-results/**/TEST-*.xml'
|
||
annotate_only: true
|
||
detailed_summary: true
|
||
fail_on_failure: true
|
||
|
||
# The HTML reports carry the full stack traces and stdout/stderr the
|
||
# annotations truncate. Named per-OS because the three matrix legs upload
|
||
# into the same run and artifact names must be unique.
|
||
- name: Upload Desktop Test Reports
|
||
uses: actions/upload-artifact@v7
|
||
if: failure()
|
||
with:
|
||
name: Desktop Test Reports (${{ matrix.os }})
|
||
path: |
|
||
quartz/build/reports/tests
|
||
commons/build/reports/tests
|
||
nestsClient/build/reports/tests
|
||
cli/build/reports/tests
|
||
desktopApp/build/reports/tests
|
||
|
||
# jpackage pins libicu to the build host's version (libicu74 on
|
||
# ubuntu-24.04). Rewrite the .deb so testers on other Debian/Ubuntu
|
||
# releases can install the uploaded artifact.
|
||
- name: Relax libicu dependency in .deb
|
||
if: matrix.desktop-task == 'packageDeb'
|
||
run: |
|
||
set -euo pipefail
|
||
chmod +x scripts/relax-deb-libicu.sh
|
||
scripts/relax-deb-libicu.sh desktopApp/build/compose/binaries/main/deb/*.deb
|
||
|
||
- name: Upload Desktop Distribution
|
||
uses: actions/upload-artifact@v7
|
||
with:
|
||
name: ${{ matrix.desktop-artifact-name }}
|
||
path: ${{ matrix.desktop-artifact-path }}
|
||
|
||
# geode (the standalone Nostr relay) is JVM-only, so it runs in its own job
|
||
# rather than the build-desktop matrix — one runner suffices (no reason to test
|
||
# a platform-independent module 3× across the desktop OS matrix). Isolating it
|
||
# also keeps its default suite's CPU-heavy throughput benchmarks (a 1M-event
|
||
# mirror sync, WireReqFloor, NegentropyServerReconcile) from contending with the
|
||
# timing-sensitive quartz relay-client tests under org.gradle.parallel — that
|
||
# contention flakes tests like NostrClientReqBypassingRelayLimitsTest.
|
||
test-geode:
|
||
needs: lint
|
||
runs-on: ubuntu-latest
|
||
timeout-minutes: 30
|
||
steps:
|
||
- name: Checkout code
|
||
uses: actions/checkout@v7
|
||
|
||
- name: Set up JDK 21
|
||
uses: actions/setup-java@v6.0.0
|
||
with:
|
||
distribution: 'temurin'
|
||
java-version: 21
|
||
|
||
- name: Set up Gradle
|
||
uses: gradle/actions/setup-gradle@v6
|
||
with:
|
||
cache-read-only: ${{ github.ref != 'refs/heads/main' }}
|
||
|
||
# -DsyncN shrinks MirrorSyncThroughputTest's corpus from its 1,000,000-event
|
||
# default. At 1M the sink cannot keep up with the in-process source, and the
|
||
# MirrorWorker's deliberately unbounded intake channel buffers the backlog until
|
||
# the runner's heap is gone: throughput collapses (13,800 -> 86 ev/s) and an
|
||
# OutOfMemoryError lands on a coroutine thread, where the UncaughtExceptionHandler
|
||
# swallows it. JUnit never sees a failure, so the JVM wedges and the job burns to
|
||
# the timeout with no signal rather than failing. 100k keeps a real ev/s number
|
||
# while bounding the worst-case backlog to a tenth of what died.
|
||
#
|
||
# Only CI is shrunk: -DsyncN is unset everywhere else, so a local or manual run
|
||
# still measures the full 1M — the number written up in
|
||
# relayBench/plans/2026-07-04-sync-throughput-1m.md.
|
||
- name: Test geode (gradle)
|
||
run: ./gradlew :geode:test -DsyncN=100000
|
||
|
||
- name: Upload geode Test Reports
|
||
uses: actions/upload-artifact@v7
|
||
if: failure()
|
||
with:
|
||
name: geode Test Reports
|
||
path: geode/build/reports
|
||
|
||
# linuxX64 is the only target whose LargeCache / ConcurrentHashCache actuals are
|
||
# hand-written concurrent maps rather than a delegation to a platform concurrent
|
||
# collection, and until this job existed nothing ran them: the target was compiled
|
||
# by no CI leg at all. That is how a copy-on-write LargeCache with O(n) writes and a
|
||
# non-atomic read-copy-write (concurrent writers silently dropped entries) sat in
|
||
# the tree unnoticed.
|
||
#
|
||
# Scoped to the cache/concurrency packages on purpose. The full linuxX64Test suite
|
||
# is 3,490 tests with 78 pre-existing failures, essentially all of them `TODO()`
|
||
# stubs in linux actuals that were never written — MLS crypto, the SQLite driver,
|
||
# NIP-44, Bolt12 — plus two URL-handling divergences. Filling those in is its own
|
||
# project; gating PRs on them today would just mean a permanently red job. The
|
||
# filter keeps the leg meaningful and green, and widening it is a one-line change
|
||
# once the native actuals land.
|
||
#
|
||
# This still compiles and links the whole module for linuxX64, so a commonMain or
|
||
# commonTest source that reaches for a JVM-only API fails here too — on a target
|
||
# with no Foundation to fall back on the way Apple has.
|
||
test-quartz-linux-native:
|
||
needs: lint
|
||
runs-on: ubuntu-latest
|
||
timeout-minutes: 45
|
||
steps:
|
||
- name: Checkout code
|
||
uses: actions/checkout@v7
|
||
|
||
- name: Set up JDK 21
|
||
uses: actions/setup-java@v6.0.0
|
||
with:
|
||
distribution: 'temurin'
|
||
java-version: 21
|
||
|
||
- name: Set up Gradle
|
||
uses: gradle/actions/setup-gradle@v6
|
||
with:
|
||
cache-read-only: ${{ github.ref != 'refs/heads/main' }}
|
||
|
||
# The Kotlin/Native toolchain (compiler distribution + LLVM + the sysroot) lands
|
||
# in ~/.konan, which setup-gradle does not cache. Without this the job re-downloads
|
||
# well over a gigabyte on every run. Keyed on the version catalog so a Kotlin bump
|
||
# re-populates it.
|
||
- name: Cache Kotlin/Native toolchain
|
||
uses: actions/cache@v4
|
||
with:
|
||
path: ~/.konan
|
||
key: konan-${{ runner.os }}-${{ hashFiles('gradle/libs.versions.toml') }}
|
||
restore-keys: konan-${{ runner.os }}-
|
||
|
||
- name: Test Quartz caches on Linux Native
|
||
run: |
|
||
./gradlew :quartz:linuxX64Test \
|
||
--tests "com.vitorpamplona.quartz.utils.cache.*" \
|
||
--tests "com.vitorpamplona.quartz.utils.concurrent.*"
|
||
|
||
- name: Linux Native Test Report
|
||
uses: mikepenz/action-junit-report@a9170d5795813c01ab4901ffb045b52bab4ab09d # v6.5.0
|
||
if: always()
|
||
with:
|
||
report_paths: 'quartz/build/test-results/linuxX64Test/TEST-*.xml'
|
||
annotate_only: true
|
||
detailed_summary: true
|
||
fail_on_failure: true
|
||
|
||
- name: Upload Linux Native Test Reports
|
||
uses: actions/upload-artifact@v7
|
||
if: failure()
|
||
with:
|
||
name: Quartz Linux Native Test Reports
|
||
path: quartz/build/reports
|
||
|
||
test-quartz-ios:
|
||
# Phase 1 of the iOS support plan
|
||
# (amethyst/plans/2026-05-24-ios-support.md): keep :quartz green on iOS
|
||
# so JVM-only imports can't sneak into commonMain unnoticed. The
|
||
# `verifyKmpPurity` task in the lint job is the fast pre-check (Linux,
|
||
# ~1s); this job is the real one — compiles for the device variant
|
||
# and actually runs the simulator test suite.
|
||
needs: lint
|
||
runs-on: macos-latest
|
||
timeout-minutes: 45
|
||
steps:
|
||
- name: Checkout code
|
||
uses: actions/checkout@v7
|
||
|
||
- name: Set up JDK 21
|
||
uses: actions/setup-java@v6.0.0
|
||
with:
|
||
distribution: 'temurin'
|
||
java-version: 21
|
||
|
||
- name: Set up Gradle
|
||
uses: gradle/actions/setup-gradle@v6
|
||
with:
|
||
cache-read-only: ${{ github.ref != 'refs/heads/main' }}
|
||
|
||
# Two tasks, two purposes:
|
||
# - iosSimulatorArm64Test runs the existing iosTest suite on the
|
||
# simulator (NIP-04 / NIP-17 / NIP-19 / NIP-49 / AES-GCM /
|
||
# Chatroom keys), exercising secp256k1 and CryptoKit-backed
|
||
# primitives on a real Apple toolchain.
|
||
# - compileTestKotlinIosArm64 catches any device-only compile drift
|
||
# (iosArm64 = aarch64-apple-ios) without needing a physical
|
||
# device to run on. Compile-only is enough — running on-device
|
||
# would require xcodebuild + a provisioning profile.
|
||
- name: Test Quartz on iOS
|
||
run: |
|
||
./gradlew \
|
||
:quartz:iosSimulatorArm64Test \
|
||
:quartz:compileTestKotlinIosArm64
|
||
|
||
# :commons gained iosArm64 + iosSimulatorArm64 targets in Phase 2 of the
|
||
# iOS plan. Actual UI / lifecycle wiring will land with the iosApp module
|
||
# in Phase 3, but the shared commonMain + commonTest sources are already
|
||
# built here against an Apple Native frontend. Same two-task shape as
|
||
# quartz above:
|
||
# - iosSimulatorArm64Test compiles AND runs the shared commonTest suite
|
||
# on the simulator. commonTest is built for every target, so a test
|
||
# reaching for a JVM-only API (JUnit, javaClass, @JvmStatic, or a
|
||
# jvmAndroid-only symbol) breaks the Apple build even though
|
||
# :commons:jvmTest stays green — this is the job that catches it.
|
||
# - compileTestKotlinIosArm64 catches device-only compile drift
|
||
# (iosArm64 = aarch64-apple-ios) without needing a physical device.
|
||
- name: Test Commons on iOS
|
||
run: |
|
||
./gradlew \
|
||
:commons:iosSimulatorArm64Test \
|
||
:commons:compileTestKotlinIosArm64
|
||
|
||
- name: Upload iOS Test Reports
|
||
uses: actions/upload-artifact@v7
|
||
if: failure()
|
||
with:
|
||
name: Quartz iOS Test Reports
|
||
path: quartz/build/reports
|
||
|
||
test-and-build-android:
|
||
needs: lint
|
||
runs-on: ubuntu-latest
|
||
timeout-minutes: 60
|
||
steps:
|
||
- name: Checkout code
|
||
uses: actions/checkout@v7
|
||
|
||
- name: Set up JDK 21
|
||
uses: actions/setup-java@v6.0.0
|
||
with:
|
||
distribution: 'temurin'
|
||
java-version: 21
|
||
|
||
- name: Set up Gradle
|
||
uses: gradle/actions/setup-gradle@v6
|
||
with:
|
||
cache-read-only: ${{ github.ref != 'refs/heads/main' }}
|
||
|
||
# Lint + focused unit tests + benchmark assembly in one Gradle invocation.
|
||
# Previously: one invocation for lint, one for `test` (which compiled all
|
||
# six amethyst variants × all flavors), one for `assembleBenchmark`
|
||
# (re-walking the same task graph). Combining them keeps the daemon hot
|
||
# across phases and lets task-level dedup (e.g. compileKotlin) only
|
||
# happen once.
|
||
#
|
||
# `-PdisableAbiSplits=true` produces a single non-split APK per
|
||
# (flavor, buildType) instead of 5 (4 ABIs + universal). The CI only
|
||
# uploads the universal-equivalent benchmark APK; per-ABI splits were
|
||
# being built and discarded, costing ~600MB of stripped_native_libs
|
||
# intermediates and several minutes per run.
|
||
#
|
||
# Test scope: only Debug unit tests for amethyst. The release/benchmark
|
||
# variants are compile-equivalent for unit-test purposes; running all six
|
||
# adds ~5× the kotlinc work without catching new defects on PRs. Push to
|
||
# main still gets the full test matrix via the production-build path.
|
||
- name: Test + Build Android (gradle)
|
||
run: |
|
||
./gradlew \
|
||
:amethyst:lintFdroidBenchmark \
|
||
:amethyst:lintPlayBenchmark \
|
||
:quartz:jvmTest \
|
||
:commons:jvmTest \
|
||
:nestsClient:jvmTest \
|
||
:amethyst:testFdroidDebugUnitTest \
|
||
:amethyst:testPlayDebugUnitTest \
|
||
:amethyst:assembleBenchmark \
|
||
-PdisableAbiSplits=true
|
||
|
||
- name: Upload Android Lint Reports
|
||
uses: actions/upload-artifact@v7
|
||
if: always()
|
||
with:
|
||
name: Android Lint Reports
|
||
path: amethyst/build/reports/lint-results-*.html
|
||
|
||
# Publishes the JUnit XML produced by the unit-test tasks above as inline
|
||
# annotations plus a job summary. Replaces asadmansr/android-test-report-action,
|
||
# which was abandoned (last release 2020) and rebuilt an EOL Ubuntu 18.04 +
|
||
# Python 2 Docker image on every run — bionic's apt archives have since gone
|
||
# unreliable and broke this job. Pinned to a commit SHA (not the movable
|
||
# v6.4.2 tag) to close the supply-chain hole. annotate_only avoids needing
|
||
# `checks: write`, so it keeps working on pull requests from forks (where the
|
||
# GITHUB_TOKEN is read-only). fail_on_failure preserves the old step's
|
||
# behavior of marking the job red when a test fails.
|
||
- name: Android Test Report
|
||
uses: mikepenz/action-junit-report@a9170d5795813c01ab4901ffb045b52bab4ab09d # v6.5.0
|
||
if: always()
|
||
with:
|
||
report_paths: '**/build/test-results/**/TEST-*.xml'
|
||
annotate_only: true
|
||
detailed_summary: true
|
||
fail_on_failure: true
|
||
|
||
- name: Upload Test Results
|
||
uses: actions/upload-artifact@v7
|
||
if: failure()
|
||
with:
|
||
name: Test Reports
|
||
path: amethyst/build/reports
|
||
|
||
# With -PdisableAbiSplits=true the APK is named without the ABI/universal
|
||
# suffix: amethyst-<flavor>-benchmark.apk. Glob both forms so this still
|
||
# works if a contributor runs CI on a branch that doesn't pass the flag.
|
||
- name: Upload Play APK Benchmark
|
||
uses: actions/upload-artifact@v7
|
||
with:
|
||
name: Play Benchmark APK
|
||
path: |
|
||
amethyst/build/outputs/apk/play/benchmark/amethyst-play-benchmark.apk
|
||
amethyst/build/outputs/apk/play/benchmark/amethyst-play-universal-benchmark.apk
|
||
|
||
- name: Upload FDroid APK Benchmark
|
||
uses: actions/upload-artifact@v7
|
||
with:
|
||
name: FDroid Benchmark APK
|
||
path: |
|
||
amethyst/build/outputs/apk/fdroid/benchmark/amethyst-fdroid-benchmark.apk
|
||
amethyst/build/outputs/apk/fdroid/benchmark/amethyst-fdroid-universal-benchmark.apk
|
||
|
||
- name: Upload Compose Reports
|
||
uses: actions/upload-artifact@v7
|
||
with:
|
||
name: Compose Reports
|
||
path: amethyst/build/compose_compiler
|