mirror of
https://github.com/vitorpamplona/amethyst.git
synced 2026-10-05 19:28:25 +00:00
Vitor PamplonaandClaude Opus 5
8bf1ac27d8
test-geode has been dying at its 30-minute timeout on most runs since at least 18 Aug — six consecutive cancellations on main, both runs on #3955 — while the runs that pass finish the whole job in under five minutes. It is not a job that outgrew its budget; nothing lands in between. The cause is MirrorSyncThroughputTest, which preloads a 1,000,000-event corpus. The sink ingests slower than the in-process source serves, and MirrorWorker's intake channel is deliberately unbounded (its listener callback cannot suspend, so it trySends rather than block the shared OkHttp reader — see its kdoc). The backlog therefore grows until the runner's heap is gone. From the CI log: …116777/1000000 (10,225 ev/s inst) …121961/1000000 ( 1,507 ev/s inst) …122793/1000000 ( 247 ev/s inst) …123113/1000000 ( 86 ev/s inst) Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "kotlinx.coroutines.DefaultExecutor" That is a GC death spiral, then OOM. What turns it into a *timeout* rather than a failure is where the OOM lands: on coroutine threads, reaching the UncaughtExceptionHandler instead of the test thread. JUnit never sees a failure, the JVM never exits, and the job produces no further output until GitHub kills it — which is why the check has been red without ever saying why. The test already takes -DsyncN (default 1,000,000) and geode/build.gradle.kts already forwards it to the test JVM, so this is a workflow-only change. 100k keeps a real ev/s measurement while bounding the worst-case backlog to a tenth of what died. -DsyncN is unset everywhere else, so local and manual runs still measure the full 1M. This does not fix the underlying fragility — a bulk backfill can still exhaust the heap, and an OOM on a coroutine thread will still wedge rather than fail. Both are worth addressing separately; the MirrorWorker half is already noted in relayBench/plans/2026-07-04-sync-throughput-1m.md. Verified: :geode:test --tests "*MirrorSyncThroughputTest*" -DsyncN=100000 passes in 7s and logs "preloaded 100000"; the full suite passes locally in 1m39s. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
331 lines
13 KiB
YAML
331 lines
13 KiB
YAML
name: Test/Build
|
||
|
||
on:
|
||
pull_request:
|
||
branches: [main]
|
||
push:
|
||
branches: [main]
|
||
|
||
permissions:
|
||
contents: read
|
||
|
||
concurrency:
|
||
group: ${{ github.workflow }}-${{ github.ref }}
|
||
cancel-in-progress: true
|
||
|
||
jobs:
|
||
lint:
|
||
runs-on: ubuntu-latest
|
||
timeout-minutes: 15
|
||
steps:
|
||
- name: Checkout code
|
||
uses: actions/checkout@v7
|
||
|
||
- name: Set up JDK 21
|
||
uses: actions/setup-java@v5.7.0
|
||
with:
|
||
distribution: 'temurin'
|
||
java-version: 21
|
||
|
||
# Remote Gradle build cache: writes on push to main, reads on PRs and
|
||
# other branches. Caches both `~/.gradle/caches/` and individual task
|
||
# outputs, so dependency-only changes hit the cache and skip recompiling
|
||
# downstream modules / re-merging native libs (~600MB of work on
|
||
# :amethyst alone). Replaces the narrower `cache: gradle` previously on
|
||
# actions/setup-java, which only cached `modules-2`.
|
||
- name: Set up Gradle
|
||
uses: gradle/actions/setup-gradle@v6
|
||
with:
|
||
cache-read-only: ${{ github.ref != 'refs/heads/main' }}
|
||
|
||
- name: Linter (gradle)
|
||
run: ./gradlew spotlessCheck :quartz:verifyKmpPurity :commons:verifyKmpPurity
|
||
|
||
build-desktop:
|
||
needs: lint
|
||
strategy:
|
||
fail-fast: false
|
||
matrix:
|
||
include:
|
||
- os: ubuntu-latest
|
||
desktop-task: packageDeb
|
||
desktop-artifact-name: Desktop Linux DEB
|
||
desktop-artifact-path: desktopApp/build/compose/binaries/main/deb/*.deb
|
||
- os: macos-latest
|
||
desktop-task: packageDmg
|
||
desktop-artifact-name: Desktop macOS DMG
|
||
desktop-artifact-path: desktopApp/build/compose/binaries/main/dmg/*.dmg
|
||
- os: windows-latest
|
||
desktop-task: packageMsi
|
||
desktop-artifact-name: Desktop Windows MSI
|
||
desktop-artifact-path: desktopApp/build/compose/binaries/main/msi/*.msi
|
||
runs-on: ${{ matrix.os }}
|
||
timeout-minutes: 60
|
||
defaults:
|
||
run:
|
||
shell: bash
|
||
steps:
|
||
- name: Checkout code
|
||
uses: actions/checkout@v7
|
||
|
||
- name: Set up JDK 21
|
||
uses: actions/setup-java@v5.7.0
|
||
with:
|
||
distribution: 'temurin'
|
||
java-version: 21
|
||
|
||
- name: Set up Gradle
|
||
uses: gradle/actions/setup-gradle@v6
|
||
with:
|
||
cache-read-only: ${{ github.ref != 'refs/heads/main' }}
|
||
|
||
# Compose UI smoke test (DesktopLaunchSmokeTest) uses Skiko which needs
|
||
# a display server on Linux. xvfb provides a virtual framebuffer.
|
||
- name: Install xvfb (Linux)
|
||
if: runner.os == 'Linux'
|
||
run: sudo apt-get update && sudo apt-get install -y xvfb
|
||
|
||
- name: Test + Build Desktop (gradle)
|
||
run: |
|
||
CMD="./gradlew :quartz:jvmTest :commons:jvmTest :nestsClient:jvmTest :cli:test :desktopApp:test :desktopApp:${{ matrix.desktop-task }}"
|
||
if [ "${{ runner.os }}" = "Linux" ]; then
|
||
xvfb-run --auto-servernum $CMD
|
||
else
|
||
$CMD
|
||
fi
|
||
|
||
# jpackage pins libicu to the build host's version (libicu74 on
|
||
# ubuntu-24.04). Rewrite the .deb so testers on other Debian/Ubuntu
|
||
# releases can install the uploaded artifact.
|
||
- name: Relax libicu dependency in .deb
|
||
if: matrix.desktop-task == 'packageDeb'
|
||
run: |
|
||
set -euo pipefail
|
||
chmod +x scripts/relax-deb-libicu.sh
|
||
scripts/relax-deb-libicu.sh desktopApp/build/compose/binaries/main/deb/*.deb
|
||
|
||
- name: Upload Desktop Distribution
|
||
uses: actions/upload-artifact@v7
|
||
with:
|
||
name: ${{ matrix.desktop-artifact-name }}
|
||
path: ${{ matrix.desktop-artifact-path }}
|
||
|
||
# geode (the standalone Nostr relay) is JVM-only, so it runs in its own job
|
||
# rather than the build-desktop matrix — one runner suffices (no reason to test
|
||
# a platform-independent module 3× across the desktop OS matrix). Isolating it
|
||
# also keeps its default suite's CPU-heavy throughput benchmarks (a 1M-event
|
||
# mirror sync, WireReqFloor, NegentropyServerReconcile) from contending with the
|
||
# timing-sensitive quartz relay-client tests under org.gradle.parallel — that
|
||
# contention flakes tests like NostrClientReqBypassingRelayLimitsTest.
|
||
test-geode:
|
||
needs: lint
|
||
runs-on: ubuntu-latest
|
||
timeout-minutes: 30
|
||
steps:
|
||
- name: Checkout code
|
||
uses: actions/checkout@v7
|
||
|
||
- name: Set up JDK 21
|
||
uses: actions/setup-java@v5.7.0
|
||
with:
|
||
distribution: 'temurin'
|
||
java-version: 21
|
||
|
||
- name: Set up Gradle
|
||
uses: gradle/actions/setup-gradle@v6
|
||
with:
|
||
cache-read-only: ${{ github.ref != 'refs/heads/main' }}
|
||
|
||
# -DsyncN shrinks MirrorSyncThroughputTest's corpus from its 1,000,000-event
|
||
# default. At 1M the sink cannot keep up with the in-process source, and the
|
||
# MirrorWorker's deliberately unbounded intake channel buffers the backlog until
|
||
# the runner's heap is gone: throughput collapses (13,800 -> 86 ev/s) and an
|
||
# OutOfMemoryError lands on a coroutine thread, where the UncaughtExceptionHandler
|
||
# swallows it. JUnit never sees a failure, so the JVM wedges and the job burns to
|
||
# the timeout with no signal rather than failing. 100k keeps a real ev/s number
|
||
# while bounding the worst-case backlog to a tenth of what died.
|
||
#
|
||
# Only CI is shrunk: -DsyncN is unset everywhere else, so a local or manual run
|
||
# still measures the full 1M — the number written up in
|
||
# relayBench/plans/2026-07-04-sync-throughput-1m.md.
|
||
- name: Test geode (gradle)
|
||
run: ./gradlew :geode:test -DsyncN=100000
|
||
|
||
- name: Upload geode Test Reports
|
||
uses: actions/upload-artifact@v7
|
||
if: failure()
|
||
with:
|
||
name: geode Test Reports
|
||
path: geode/build/reports
|
||
|
||
test-quartz-ios:
|
||
# Phase 1 of the iOS support plan
|
||
# (amethyst/plans/2026-05-24-ios-support.md): keep :quartz green on iOS
|
||
# so JVM-only imports can't sneak into commonMain unnoticed. The
|
||
# `verifyKmpPurity` task in the lint job is the fast pre-check (Linux,
|
||
# ~1s); this job is the real one — compiles for the device variant
|
||
# and actually runs the simulator test suite.
|
||
needs: lint
|
||
runs-on: macos-latest
|
||
timeout-minutes: 45
|
||
steps:
|
||
- name: Checkout code
|
||
uses: actions/checkout@v7
|
||
|
||
- name: Set up JDK 21
|
||
uses: actions/setup-java@v5.7.0
|
||
with:
|
||
distribution: 'temurin'
|
||
java-version: 21
|
||
|
||
- name: Set up Gradle
|
||
uses: gradle/actions/setup-gradle@v6
|
||
with:
|
||
cache-read-only: ${{ github.ref != 'refs/heads/main' }}
|
||
|
||
# Two tasks, two purposes:
|
||
# - iosSimulatorArm64Test runs the existing iosTest suite on the
|
||
# simulator (NIP-04 / NIP-17 / NIP-19 / NIP-49 / AES-GCM /
|
||
# Chatroom keys), exercising secp256k1 and CryptoKit-backed
|
||
# primitives on a real Apple toolchain.
|
||
# - compileTestKotlinIosArm64 catches any device-only compile drift
|
||
# (iosArm64 = aarch64-apple-ios) without needing a physical
|
||
# device to run on. Compile-only is enough — running on-device
|
||
# would require xcodebuild + a provisioning profile.
|
||
- name: Test Quartz on iOS
|
||
run: |
|
||
./gradlew \
|
||
:quartz:iosSimulatorArm64Test \
|
||
:quartz:compileTestKotlinIosArm64
|
||
|
||
# :commons gained iosArm64 + iosSimulatorArm64 targets in Phase 2 of the
|
||
# iOS plan. Actual UI / lifecycle wiring will land with the iosApp module
|
||
# in Phase 3, but the shared commonMain + commonTest sources are already
|
||
# built here against an Apple Native frontend. Same two-task shape as
|
||
# quartz above:
|
||
# - iosSimulatorArm64Test compiles AND runs the shared commonTest suite
|
||
# on the simulator. commonTest is built for every target, so a test
|
||
# reaching for a JVM-only API (JUnit, javaClass, @JvmStatic, or a
|
||
# jvmAndroid-only symbol) breaks the Apple build even though
|
||
# :commons:jvmTest stays green — this is the job that catches it.
|
||
# - compileTestKotlinIosArm64 catches device-only compile drift
|
||
# (iosArm64 = aarch64-apple-ios) without needing a physical device.
|
||
- name: Test Commons on iOS
|
||
run: |
|
||
./gradlew \
|
||
:commons:iosSimulatorArm64Test \
|
||
:commons:compileTestKotlinIosArm64
|
||
|
||
- name: Upload iOS Test Reports
|
||
uses: actions/upload-artifact@v7
|
||
if: failure()
|
||
with:
|
||
name: Quartz iOS Test Reports
|
||
path: quartz/build/reports
|
||
|
||
test-and-build-android:
|
||
needs: lint
|
||
runs-on: ubuntu-latest
|
||
timeout-minutes: 60
|
||
steps:
|
||
- name: Checkout code
|
||
uses: actions/checkout@v7
|
||
|
||
- name: Set up JDK 21
|
||
uses: actions/setup-java@v5.7.0
|
||
with:
|
||
distribution: 'temurin'
|
||
java-version: 21
|
||
|
||
- name: Set up Gradle
|
||
uses: gradle/actions/setup-gradle@v6
|
||
with:
|
||
cache-read-only: ${{ github.ref != 'refs/heads/main' }}
|
||
|
||
# Lint + focused unit tests + benchmark assembly in one Gradle invocation.
|
||
# Previously: one invocation for lint, one for `test` (which compiled all
|
||
# six amethyst variants × all flavors), one for `assembleBenchmark`
|
||
# (re-walking the same task graph). Combining them keeps the daemon hot
|
||
# across phases and lets task-level dedup (e.g. compileKotlin) only
|
||
# happen once.
|
||
#
|
||
# `-PdisableAbiSplits=true` produces a single non-split APK per
|
||
# (flavor, buildType) instead of 5 (4 ABIs + universal). The CI only
|
||
# uploads the universal-equivalent benchmark APK; per-ABI splits were
|
||
# being built and discarded, costing ~600MB of stripped_native_libs
|
||
# intermediates and several minutes per run.
|
||
#
|
||
# Test scope: only Debug unit tests for amethyst. The release/benchmark
|
||
# variants are compile-equivalent for unit-test purposes; running all six
|
||
# adds ~5× the kotlinc work without catching new defects on PRs. Push to
|
||
# main still gets the full test matrix via the production-build path.
|
||
- name: Test + Build Android (gradle)
|
||
run: |
|
||
./gradlew \
|
||
:amethyst:lintFdroidBenchmark \
|
||
:amethyst:lintPlayBenchmark \
|
||
:quartz:jvmTest \
|
||
:commons:jvmTest \
|
||
:nestsClient:jvmTest \
|
||
:amethyst:testFdroidDebugUnitTest \
|
||
:amethyst:testPlayDebugUnitTest \
|
||
:amethyst:assembleBenchmark \
|
||
-PdisableAbiSplits=true
|
||
|
||
- name: Upload Android Lint Reports
|
||
uses: actions/upload-artifact@v7
|
||
if: always()
|
||
with:
|
||
name: Android Lint Reports
|
||
path: amethyst/build/reports/lint-results-*.html
|
||
|
||
# Publishes the JUnit XML produced by the unit-test tasks above as inline
|
||
# annotations plus a job summary. Replaces asadmansr/android-test-report-action,
|
||
# which was abandoned (last release 2020) and rebuilt an EOL Ubuntu 18.04 +
|
||
# Python 2 Docker image on every run — bionic's apt archives have since gone
|
||
# unreliable and broke this job. Pinned to a commit SHA (not the movable
|
||
# v6.4.2 tag) to close the supply-chain hole. annotate_only avoids needing
|
||
# `checks: write`, so it keeps working on pull requests from forks (where the
|
||
# GITHUB_TOKEN is read-only). fail_on_failure preserves the old step's
|
||
# behavior of marking the job red when a test fails.
|
||
- name: Android Test Report
|
||
uses: mikepenz/action-junit-report@d9f48fc87bc235f7e214acf696ca5abc0a986f16 # v6.4.2
|
||
if: always()
|
||
with:
|
||
report_paths: '**/build/test-results/**/TEST-*.xml'
|
||
annotate_only: true
|
||
detailed_summary: true
|
||
fail_on_failure: true
|
||
|
||
- name: Upload Test Results
|
||
uses: actions/upload-artifact@v7
|
||
if: failure()
|
||
with:
|
||
name: Test Reports
|
||
path: amethyst/build/reports
|
||
|
||
# With -PdisableAbiSplits=true the APK is named without the ABI/universal
|
||
# suffix: amethyst-<flavor>-benchmark.apk. Glob both forms so this still
|
||
# works if a contributor runs CI on a branch that doesn't pass the flag.
|
||
- name: Upload Play APK Benchmark
|
||
uses: actions/upload-artifact@v7
|
||
with:
|
||
name: Play Benchmark APK
|
||
path: |
|
||
amethyst/build/outputs/apk/play/benchmark/amethyst-play-benchmark.apk
|
||
amethyst/build/outputs/apk/play/benchmark/amethyst-play-universal-benchmark.apk
|
||
|
||
- name: Upload FDroid APK Benchmark
|
||
uses: actions/upload-artifact@v7
|
||
with:
|
||
name: FDroid Benchmark APK
|
||
path: |
|
||
amethyst/build/outputs/apk/fdroid/benchmark/amethyst-fdroid-benchmark.apk
|
||
amethyst/build/outputs/apk/fdroid/benchmark/amethyst-fdroid-universal-benchmark.apk
|
||
|
||
- name: Upload Compose Reports
|
||
uses: actions/upload-artifact@v7
|
||
with:
|
||
name: Compose Reports
|
||
path: amethyst/build/compose_compiler
|