Files
amethyst/RELEASE_OPS.md
T
Claude 120eef46a6 feat: assert the reflective surface survived R8, so keep rules cannot rot silently
R8 cannot see reflection. A keep rule that stops matching — a class moved to
another package, a rule dropped in a merge, a DTO renamed — produces a green
build and an APK that fails on a user's device: ClassNotFoundException from
WorkManager, JSON with one-letter property names, a settings file that no longer
parses. Nothing in the build says a word about it.

tools/r8-verify/reflection-contract.txt lists every name resolved from outside
the DEX, each with the mechanism that reads it: JNI symbols and the Rust
GetMethodID callback, Jackson's reflectively-bound DTOs, enum constants
persisted into DataStore, WorkManager's stored worker class names, the Cast
OptionsProvider named in a manifest <meta-data> value. 31 checks.
verify_reflection_contract.py asserts each against what R8 actually emitted, in
under a second, with no device. Wired into create-release.yml for both flavors
before assets are collected, so a break fails the release instead of shipping.

It reads mapping.txt AND usage.txt, because neither is sufficient:

  - mapping.txt records only what CHANGED. A class kept intact by
    `-keep ... { *; }` appears with an empty body and an unrenamed member has no
    line at all, so absence there means "preserved". A first version of this
    read absence as "missing" and reported 11 false failures against a build I
    had already verified by hand.
  - usage.txt is the other half: a constant R8 deleted leaves no trace in
    mapping.txt at all.

It also checks the two files came from the same R8 run. mapping.txt is written
during packaging, not at minify time, so a minify-only rebuild leaves a stale one
beside a fresh usage.txt — which is exactly the state this session ended up in,
and the mixed directory reads as perfectly coherent. The invariant is that a
class R8 deleted cannot also be a class R8 named; on the mixed directory it
flagged 15 such contradictions, all of them the classes whose rule had changed
between the two runs, and zero on a consistent build.

Verified by breaking it on purpose. Deleting the real NWC keep rule from both
proguard files and rebuilding produced 4 failures on genuine R8 output — three
DTOs renamed (PayInvoiceParams -> xud) and NwcTransaction shrunk away entirely,
which in production is a silent break in wallet transaction lists. Restoring the
rule and rebuilding: 31/31 pass. Eight fixture cases cover every failure mode
(class renamed, method renamed, field renamed, constant renamed, constant
deleted, class deleted, stale directory, healthy baseline).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEoxktEZyTrAS33vVZBiwm
2026-09-19 01:22:14 +00:00

22 KiB
Raw Blame History

Release Ops (Amethyst maintainers)

This is the operational checklist the Amethyst team follows to ship a release — the account-specific, push-the-buttons side of cutting a version. The generic build/release mechanics (how the CI pipeline works, the asset naming contract, the secret names a fork must set, desktop packaging) live in BUILDING.md. Read that first; this doc only covers what is specific to shipping the official Amethyst artifacts.

Forks: you do not need this file. BUILDING.md has everything you need to build and release your own fork. This describes our accounts and channels.


At a glance

A release is one tag push that fans out to four live distribution channels:

Channel Mechanism Who pushes
GitHub Releases Automatic — the Create Release Assets workflow builds + signs everything on the v* tag CI
Google Play Manual — download the signed AAB from the GH Release, upload in Play Console Maintainer
F-Droid Pull — F-Droid's build server builds the fdroid flavor from source when it sees the new tag F-Droid (we just maintain the recipe + metadata)
Zapstore zsp publish reads zapstore.yaml, signs a Nostr release event with Amethyst's nsec Maintainer
Homebrew + Winget ⚠️ Not shipping. The bump workflows run, but skip: neither package exists upstream yet Nobody (see § 3)

Maven Central (the quartz library) also publishes automatically from the same workflow — as a step at the end of the deploy-android job, not a job of its own, so don't expect to find it in the run's job list.


1. Pre-tag checklist

  1. Bump the version in gradle/libs.versions.toml — both keys:

    app     = "1.12.1"   # semver, drives every module + the tag
    appCode = "449"      # Android versionCode, monotonic — must increment
    

    That single edit propagates to Android (versionName/versionCode), Desktop & CLI (packageVersion), quartz (Maven version) and geode (RelayInfo.VERSION). Nothing else hardcodes the version.

  2. Write the changelog as docs/changelog/vMAJOR.MINOR.PP.md (zero-padded, e.g. v1.12.01.md) and add it to docs/changelog/README.md. Follow the house style: plain text, short verb-first sentences.

    For the ## Translations section, generate the credits instead of writing them by hand — no token needed:

    scripts/translators.sh
    

    This runs offline. It reads docs/changelog/translators.json (kept next to the changelogs), which CI keeps fresh: the Crowdin sync workflow's seed-translators job records everyone who has translated since the last v* tag, with their languages, in the file's sinceLastTag list, and accumulates their npubs in the forever-growing mappings registry. The script resolves that list to npubs and prints the ## Translations block grouped by language. Contributors with no npub yet are listed under UNMAPPED — credit them by hand, then add their npub under mappings so future releases pick them up automatically. (To re-query Crowdin live as a sanity check, run scripts/translators.sh --seed with CROWDIN_PROJECT_ID / CROWDIN_PERSONAL_TOKEN set, which refreshes the file.)

  3. Publish the release-notes note on Nostr — minor releases only. In practice this id has only ever been bumped on x.y.0 (1.11.0, 1.12.0, 1.13.0); patch releases leave it pointing at their minor's note. Publish with Amethyst's account and paste the event id into amethyst/build.gradle.kts:

    buildConfigField("String", "RELEASE_NOTES_ID", "\"<new-event-id-hex>\"")
    

    This id is what the in-app drawer's "Release Notes" link and the donation card open (DrawerContent.kt, ShowDonationCard.kt). It must point at the note for this version, so publish the note before tagging and commit the new id together with the version bump.

  4. Sanity-build locally (optional but cheap): ./gradlew assembleRelease and a desktop packageDistributionForCurrentOS, or run the workflow's dry-run (see BUILDING.md § Dry-run).


2. Cut the release

Commit, tag, push — see BUILDING.md § Release runbook for the exact commands. The tag must equal app from the catalog (the workflow asserts this and fails fast otherwise). A clean vMAJOR.MINOR.PATCH tag is classified stable and runs the Homebrew/Winget bump workflows; anything with a -rc/-beta/-alpha/-dev suffix is a prerelease and skips them.

Heads-up on the git push: this repo has git-credential-manager configured as a credential helper, and it blocks on an interactive prompt (a plain GIT_TERMINAL_PROMPT=0 does not stop it — the push just hangs). If that happens, push using gh's helper for the one command:

git -c credential.helper= -c credential.helper='!gh auth git-credential' push upstream main

When the Create Release Assets workflow finishes (~25–30 min) the GH Release holds 47 assets, per the asset-name contract:

  • Android (13): 5 Google Play APKs + 5 F-Droid APKs + 2 AABs + the F-Droid .apks set for Accrescent (amethyst-googleplay-*-v…apk / .aab, amethyst-fdroid-*-v…apk / .aab / .apks)
  • Desktop (14): macOS DMG (arm64 only — there is no Intel DMG), Windows MSI (x64 only — no arm64 MSI) + portable zip (x64, arm64), and Linux DEB/RPM/AppImage/flatpak/tar.gz in both x64 and arm64. BUILDING.md § Release runbook has the per-leg breakdown and why the two gaps exist.
  • CLI (10): the amy artifacts — the no-JRE jvm.tar.gz, macOS arm64, Windows x64 + arm64, and Linux DEB/RPM/tar.gz in both x64 and arm64
  • Relay (10): the geode artifacts, same matrix as amy. The geode Docker image is not a release asset — it goes to the registry, so don't count it here.
  • Maven Central: com.vitorpamplona.quartz:quartz:<version> published. repo1.maven.org lags the publish by tens of minutes — a 404 right after the run is normal. Confirm the step's log says "Deployment is being published to Maven Central", and compare against the previous version's POM before concluding anything is broken.

3. Per-channel shipping

GitHub Releases — automatic

Nothing to do beyond pushing the tag. Verify the asset count (BUILDING.md § Verify). macOS is arm64-only — there is no Intel DMG, so a single amethyst-desktop-<version>-macos-arm64.dmg is the expected, correct result.

Two of those assets are the R8 mapping files (amethyst-{googleplay,fdroid}-mapping-<version>.txt.gz). Do not prune them from old releases — they are the only way to read a crash report from a build that old (§ 7).

Google Play — manual upload

  1. Download amethyst-googleplay-<version>.aab from the GH Release.
  2. Play Console → app com.vitorpamplona.amethyst → Production (or the staged-rollout track we're using) → create release → upload the AAB.
  3. The release notes field can reuse the docs/changelog text.
  4. Roll out.

F-Droid — pull / build-from-source

F-Droid does not accept an upload from us. Its build server polls the repo, and when it sees the new v* tag it builds the fdroid product flavor from source (reproducibly) per the recipe in the separate fdroiddata repo (metadata/com.vitorpamplona.amethyst.yml), then signs and publishes to the F-Droid repo on its own cadence.

What we own to keep that working:

  • The fdroid flavor (amethyst/src/fdroid/…) must stay free of proprietary deps — it swaps Firebase/Google services for UnifiedPush and no-op/open implementations (ML Kit, writing assistant, push). Google-only libraries live behind the play flavor.
  • The fastlane metadata under fastlane/metadata/android/ (descriptions, images). F-Droid reads per-version changelogs from fastlane/metadata/android/en-US/changelogs/<versionCode>.txt if present — add one (e.g. 449.txt) when we want a changelog shown on F-Droid; otherwise none is displayed.
  • The AutoUpdateMode/UpdateCheckMode in the fdroiddata recipe tracks tags, so a correct vX.Y.Z tag + bumped versionCode is usually all F-Droid needs.

After a release, just confirm F-Droid picked up the new version (it can lag a few days): https://f-droid.org/packages/com.vitorpamplona.amethyst/.

Zapstore — zsp publish with Amethyst's nsec

Zapstore is a Nostr-native app store. The zsp CLI reads zapstore.yaml at the repo root (name, summary, description, tags, license, icon, screenshots, supported_nips, and the variants regexes that match our *-fdroid-*.apk / *-googleplay-*.apk GH-release assets), then publishes a signed software-release event to Nostr relays.

# from the repo root, after the GH Release assets exist
zsp publish

It signs with Amethyst's nsec — provide the key the way zsp expects (SIGN_WITH env var / prompt / its own config), never commit it.

Relays. zsp does not take relays from zapstore.yaml; it reads the RELAY_URLS env var (comma-separated) and defaults to wss://relay.zapstore.dev when unset. To fan the release event out to more relays for discoverability, set RELAY_URLS for the run:

RELAY_URLS="wss://relay.zapstore.dev,wss://nos.lol,wss://nostr.mom,wss://vitor.nostr1.com" \
  SIGN_WITH=<amethyst-nsec> zsp publish

Keep wss://relay.zapstore.dev in the list — that is the relay the Zapstore app itself reads from.

Homebrew + Winget — ⚠️ Winget not shipping yet

bump-homebrew.yml and bump-winget.yml are wired to open PRs against Homebrew/homebrew-cask (cask amethyst-nostr), Homebrew/homebrew-core (formula amy) and microsoft/winget-pkgs (VitorPamplona.Amethyst). They can only update a package that already exists upstream, so until the one-time bootstrap lands they detect the absence and skip with a ::warning::.

Bootstrap status:

Channel Upstream package State
Homebrew cask Homebrew/homebrew-cask → amethyst-nostr Live — merged 2026-08-24, upstream at 1.14.0
Homebrew formula Homebrew/homebrew-core → amy Live
Homebrew formula Homebrew/homebrew-core → geode-relay Not submitted — renamed from geode, which is permanently reserved for Apache Geode
Winget microsoft/winget-pkgs → VitorPamplona.Amethyst Submitted at v1.14.0 — PR #422752, still open pending CLA + review

Until each lands, that channel delivers nothing and its users get the desktop app or CLI from GitHub Releases only. Re-check before assuming — the state above is a snapshot, and two calls answer it: gh api repos/microsoft/winget-pkgs/contents/manifests/v/VitorPamplona and curl -s -o /dev/null -w '%{http_code}' https://formulae.brew.sh/api/cask/amethyst-nostr.json (404 = still absent).

Two separate faults kept this invisible until v1.13.1, both now fixed:

  1. The workflows never ran at all — for any release. They triggered on release: types: [released], and GitHub does not raise workflow-triggering events for a release created by GITHUB_TOKEN, which is exactly how create-release.yml creates it. They now trigger on workflow_run after Create Release Assets succeeds, which also fixes a latent race — the old event fired while assets were still uploading.
  2. Nothing exists upstream to bump. brew bump-cask-pr and winget-releaser can only update an existing package. The first submission is a manual, human-reviewed PR: BUILDING.md § Homebrew cask (one-time initial PR) and § Winget (one-time initial submission).

Until someone does that bootstrap, a green release run means the bump workflows skipped cleanly — not that Homebrew/Winget shipped. Check the run's warnings if you want to confirm which case you're in.

Both bumps are half-manual by design. CI does the bookkeeping with GITHUB_TOKEN only — verifying the artifact, computing hashes, and opening an in-repo PR syncing the reference packaging files. Pushing upstream needs credentials that would be dangerous as CI secrets (a repo-scoped PAT is readable by anyone with push access here), so a maintainer runs the last step:

# after merging the sync PRs
export HOMEBREW_GITHUB_API_TOKEN=ghp_...     # classic PAT, `repo` scope
scripts/bump-homebrew-cask.sh v1.16.0

scripts/bump-winget.sh v1.16.0               # no token — uses your `gh` auth

Both scripts re-verify the published artifact's sha256 before submitting, and the Homebrew one additionally refuses if the DMG is not notarized + stapled. See BUILDING.md § Package-manager credentials.

All four in-repo sync workflows (amy formula, geode formula, amethyst-nostr cask, winget manifests) open PRs against this repo on every release. Merge them to keep the reference packaging files current.


4. Operated infrastructure

Push notification server

The Google Play (FCM) flavor delivers push through a server we operate at push.amethyst.social, built from vitorpamplona/amethyst-push-notif-server. It registers devices, watches their NIP-65 inbox / NIP-17 DM relays, and sends wake-up pushes.

  • play flavor → push via this server (Firebase/FCM).
  • fdroid flavor → UnifiedPush through a distributor app the user installs (e.g. ntfy); it does not use our server.
  • Both are complemented by the on-device always-on NotificationRelayService (see PULL_NOTIFICATION.md), which keeps the user's relay connections alive without any push server at all.

The push server has its own repo, deploy, and release cadence — a normal app release does not redeploy it. Coordinate a server deploy only when the app changes the registration/push contract (token format, payload, or endpoint), so the running server stays compatible with the shipped app.


5. Secrets ownership & rotation

The workflow's required secrets and what they sign are inventoried generically in BUILDING.md § Secrets. Amethyst-specific ownership:

Secret(s) Protects Rotation
SIGNING_KEY, KEY_ALIAS, KEY_STORE_PASSWORD, KEY_PASSWORD The Android upload keystore — losing/leaking it is the worst case; Play app signing identity Keep the keystore backed up offline; never rotate casually (Play upload key reset is a support process)
SONATYPE_USERNAME, SONATYPE_PASSWORD Maven Central namespace com.vitorpamplona On compromise
SIGNING_PRIVATE_KEY, SIGNING_PASSWORD The GPG key signing Maven artifacts Per GPG key expiry
(none for Homebrew/Winget) — Both bumps run on a maintainer's machine — scripts/bump-homebrew-cask.sh and scripts/bump-winget.sh — so neither channel's PAT ever becomes a CI secret. See BUILDING.md § Package-manager credentials
CROWDIN_PERSONAL_TOKEN, CROWDIN_PROJECT_ID Translation sync On compromise

Owner assignments and rotation reminders live with the team (issue tracker).


6. Post-release verification

  • GH Release: 49 assets, sizes sane, and the asset-name set matches the previous release (see the diff one-liner in BUILDING.md § Release runbook). macOS is arm64-only — do not look for an Intel DMG.
  • Maven Central: quartz:<version> resolves (allow tens of minutes of propagation; the publish step's log is the authoritative signal).
  • Play Console: rollout started, no policy rejection.
  • Zapstore: release event visible.
  • F-Droid: new version detected (may lag days).
  • Four sync PRs opened against this repo (amy formula, geode formula, amethyst-nostr cask, winget manifests) — merge them.
  • Cask pushed upstream: scripts/bump-homebrew-cask.sh vX.Y.Z (manual, needs HOMEBREW_GITHUB_API_TOKEN in your shell).
  • Winget pushed upstream: scripts/bump-winget.sh vX.Y.Z (manual, no token — uses your gh auth). Both scripts error clearly until the one-time bootstrap PRs land (§ 3).
  • In-app "Release Notes" link opens the note matching RELEASE_NOTES_ID (only bumped on minor releases — patches keep pointing at the x.y.0 note).
  • Push still works on a play build (only if the push contract changed — see § 4); UnifiedPush still works on an fdroid build.

If anything ships broken, see BUILDING.md § Incident response.


7. Crash reports & retrace

Release builds are minified and obfuscated (they have to be: Play Console drops apps whose DEX is under 25% optimized or obfuscated out of store surfaces — see amethyst/proguard-rules.pro for the whole story). So a raw stack trace from a release build looks like this:

java.lang.IllegalStateException: something blew up
	at onh.B(r8-map-id-12c710927a584543dbe1e2e867db95460bc44482efe53283f86798648c1cfc00:7)

That is not lost information, it is encoded information. Paste the report in and run it:

scripts/retrace.sh crash-report.txt
pbpaste | scripts/retrace.sh            # or straight off the clipboard

Nothing else to supply. A report's first line names its own build — java.lang.IllegalStateException: 1.16.0-PLAY — so the script resolves the tag (v1.16.0) and the flavor (PLAY → googleplay), downloads that release's mapping asset and caches it. For a bare stack trace with no such header, name the build yourself with --release v1.16.0 --flavor play; for a build you made locally, pass its mapping.txt directly.

java.lang.IllegalStateException: something blew up
	at androidx.compose.foundation.text.input.TextFieldCharSequence.getText(TextFieldCharSequence.kt:58)
	at androidx.compose.foundation.text.input.TextFieldState.getText(TextFieldState.kt:146)
	at com.vitorpamplona.amethyst.ui.screen.loggedIn.home.ShortNotePostViewModel.onMessageChanged(ShortNotePostViewModel.kt:1727)

Note that retrace gave back three frames where the crash reported one: R8 had inlined two of them. That is worth internalising — it is the reason a raw trace's line number cannot be trusted even in the pre-obfuscation builds, where optimization was already inlining. Retracing is not a tax obfuscation imposed; it is how you read an optimized build at all.

The wrong mapping is worse than none — it produces confident, wrong names. So the script refuses to guess: the r8-map-id-<hash> in the trace is the pg_map_id header of the mapping that built it, and the two are compared before anything is printed:

error: this mapping did not build this report.
       report:  deadbeef...
       mapping: 12c71092... (amethyst-googleplay-mapping-v1.16.0.txt.gz)

--force overrides if you really mean it. (googleplay vs fdroid matters — the two flavors are separate R8 runs with different mappings.)

Per channel:

Where the report came from What to do
Play Console / Android vitals Nothing. AGP embeds the mapping in the .aab (BUNDLE-METADATA/com.android.tools.build.obfuscation/proguard.map), so Play deobfuscates automatically.
A NIP-17 DM from the in-app crash reporter, a GitHub issue, F-Droid, Zapstore, Accrescent scripts/retrace.sh <report> — it reads the build off line 1 and fetches the mapping.
A build you made locally scripts/retrace.sh amethyst/build/outputs/mapping/<variant>/mapping.txt report.txt

scripts/retrace.sh downloads the R8 version named in the mapping's own header from Google's Maven and caches it, so it needs no pinned tooling and keeps working across AGP bumps. It needs no gh auth either — release assets on a public repo are plain HTTPS downloads.

Two details of the report format in ReportAssembler exist for this and should not be "tidied" away: the headline carries the fully qualified exception class (a bare simpleName obfuscates to a, which retrace cannot resolve because it has no package), and stack frames are written as at <frame> (retrace only rewrites frames it recognises, and it recognises them by the leading at). The script repairs the missing at on reports from older builds, but new reports should not need repairing.

Do not delete mapping assets from old releases. They are the only copy — CI's are gone when the job ends, and a mapping cannot be regenerated after the fact (it would need a bit-identical rebuild, and R8's renaming is not stable across runs).

Did obfuscation break anything?

R8 cannot see reflection, so nothing in the build tells you that a keep rule stopped matching. tools/r8-verify/reflection-contract.txt lists every place something outside the DEX resolves a name at runtime — JNI symbols, Jackson DTO field names, enum constants persisted in DataStore, WorkManager's stored worker class names, the Cast OptionsProvider named in a manifest <meta-data> value — and the release workflow asserts each one against what R8 actually emitted, for both flavors, before any asset is collected:

python3 tools/r8-verify/verify_reflection_contract.py \
    amethyst/build/outputs/mapping/playRelease/

Run it on any local minified build too. It takes under a second and needs no device.

Adding reflection means adding two things: the keep rule, and a line in the contract. A rule with no contract line is unverified and will rot silently.

What this does not cover is reflection nobody wrote down. For that the honest controls are a staged Play rollout (the crash reporter retraces itself now, so a break is legible within hours) and exercising NIP-47 wallet connect, NIP-46 bunker login, Tor, scheduled posts and a settings round-trip on a minified build before shipping.