From db8089c084efa659363f300c29b5b5f2765fdff4 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 18 Sep 2026 22:24:14 +0000 Subject: [PATCH] feat: make a pasted crash report retrace itself in one command MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Follow-up to the R8 mapping work: close the last two gaps between "user sends a crash report over NIP-17" and "maintainer reads real names". `scripts/retrace.sh ` now needs nothing else. A report's first line already names its own build — `java.lang.IllegalStateException: 1.16.0-PLAY` — so the script resolves the tag (v1.16.0) and flavor (PLAY -> googleplay), downloads that release's mapping asset over plain HTTPS (no gh auth needed on a public repo) and caches it. `--release`/`--flavor` for a bare stack trace with no header, or a mapping path for a local build; all three forms tested. It refuses to guess: a report's `r8-map-id-` is the `pg_map_id` of the one mapping that built it, so the two are compared before anything is printed. The wrong mapping does not fail loudly, it prints confident wrong names — worse than no answer. `--force` overrides. A branch build (1.16.0-my-branch-PLAY) is named as such instead of 404ing on a release that never existed. Two ReportAssembler fixes, both found by running the real report format through retrace rather than assuming it worked: - Frames were written as " " with no `at`. Retrace only rewrites frames it recognises and it recognises them by the leading `at`, so a release report was passed through completely untouched — every frame still obfuscated, looking exactly like a mapping problem. Now " at ", which is also what every other stack-trace tool expects. The script still repairs the missing `at` so reports from older builds retrace too. - The headline used `e.javaClass.simpleName`, which obfuscates to "a:" and cannot be retraced back — a bare simple name has no package to resolve against. Now the fully qualified name, verified to retrace: "onh: 1.16.0-PLAY" comes back as "com.vitorpamplona.amethyst...ShortNotePostViewModel: 1.16.0-PLAY". The headline is what you skim and dedup on, so it is worth the extra chars. Verified end to end against the real playRelease mapping: a full report (device table, code fences, cause section) in via file and via stdin, frames retraced including R8's inlined frames expanded 1 -> 3, device table untouched; wrong map-id refused; missing release reported with the asset URL it looked for. ReportAssemblerTest green. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01DEoxktEZyTrAS33vVZBiwm --- RELEASE_OPS.md | 48 ++-- .../service/crashreports/ReportAssembler.kt | 14 +- .../crashreports/ReportAssemblerTest.kt | 18 +- scripts/retrace.sh | 242 ++++++++++++++---- 4 files changed, 250 insertions(+), 72 deletions(-) diff --git a/RELEASE_OPS.md b/RELEASE_OPS.md index 6cb754698f..0d5d657be5 100644 --- a/RELEASE_OPS.md +++ b/RELEASE_OPS.md @@ -345,14 +345,21 @@ java.lang.IllegalStateException: something blew up at onh.B(r8-map-id-12c710927a584543dbe1e2e867db95460bc44482efe53283f86798648c1cfc00:7) ``` -That is not lost information, it is encoded information. Run it back through the -mapping: +That is not lost information, it is encoded information. Paste the report in and +run it: ```bash -scripts/retrace.sh amethyst-googleplay-mapping-v1.13.1.txt.gz crash.txt -# or: pbpaste | scripts/retrace.sh amethyst-googleplay-mapping-v1.13.1.txt.gz +scripts/retrace.sh crash-report.txt +pbpaste | scripts/retrace.sh # or straight off the clipboard ``` +Nothing else to supply. A report's first line names its own build — +`java.lang.IllegalStateException: 1.16.0-PLAY` — so the script resolves the tag +(`v1.16.0`) and the flavor (`PLAY` → `googleplay`), downloads that release's +mapping asset and caches it. For a bare stack trace with no such header, name +the build yourself with `--release v1.16.0 --flavor play`; for a build you made +locally, pass its `mapping.txt` directly. + ``` java.lang.IllegalStateException: something blew up at androidx.compose.foundation.text.input.TextFieldCharSequence.getText(TextFieldCharSequence.kt:58) @@ -366,29 +373,40 @@ trace's line number cannot be trusted even in the pre-obfuscation builds, where optimization was already inlining. Retracing is not a tax obfuscation imposed; it is how you read an optimized build at all. -**Which mapping?** Never guess. The `r8-map-id-` in the trace *is* the -`pg_map_id` header of the mapping that produced it, so: +**The wrong mapping is worse than none** — it produces confident, wrong names. +So the script refuses to guess: the `r8-map-id-` in the trace *is* the +`pg_map_id` header of the mapping that built it, and the two are compared before +anything is printed: -```bash -gh release download -p 'amethyst-*-mapping-*.txt.gz' -zcat amethyst-googleplay-mapping-.txt.gz | grep -m1 pg_map_id +``` +error: this mapping did not build this report. + report: deadbeef... + mapping: 12c71092... (amethyst-googleplay-mapping-v1.16.0.txt.gz) ``` -If the hashes match, that is the right file, full stop. (`googleplay` vs -`fdroid` matters — the two flavors are separate R8 runs with different -mappings.) +`--force` overrides if you really mean it. (`googleplay` vs `fdroid` matters — +the two flavors are separate R8 runs with different mappings.) **Per channel:** | Where the report came from | What to do | |---|---| | Play Console / Android vitals | Nothing. AGP embeds the mapping in the `.aab` (`BUNDLE-METADATA/com.android.tools.build.obfuscation/proguard.map`), so Play deobfuscates automatically. | -| A GitHub issue, Nostr DM, F-Droid, Zapstore, Accrescent | `scripts/retrace.sh` against that release's mapping asset. | -| A build you made locally | `scripts/retrace.sh amethyst/build/outputs/mapping//mapping.txt` | +| A NIP-17 DM from the in-app crash reporter, a GitHub issue, F-Droid, Zapstore, Accrescent | `scripts/retrace.sh ` — it reads the build off line 1 and fetches the mapping. | +| A build you made locally | `scripts/retrace.sh amethyst/build/outputs/mapping//mapping.txt report.txt` | `scripts/retrace.sh` downloads the R8 version named in the mapping's own header from Google's Maven and caches it, so it needs no pinned tooling and keeps -working across AGP bumps. +working across AGP bumps. It needs no `gh` auth either — release assets on a +public repo are plain HTTPS downloads. + +Two details of the report format in `ReportAssembler` exist for this and should +not be "tidied" away: the headline carries the **fully qualified** exception +class (a bare `simpleName` obfuscates to `a`, which retrace cannot resolve +because it has no package), and stack frames are written as ` at ` +(retrace only rewrites frames it recognises, and it recognises them by the +leading `at`). The script repairs the missing `at` on reports from older builds, +but new reports should not need repairing. **Do not delete mapping assets from old releases.** They are the only copy — CI's are gone when the job ends, and a mapping cannot be regenerated after the diff --git a/amethyst/src/main/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssembler.kt b/amethyst/src/main/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssembler.kt index 5a83a988e2..db235e0e20 100644 --- a/amethyst/src/main/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssembler.kt +++ b/amethyst/src/main/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssembler.kt @@ -48,7 +48,12 @@ class ReportAssembler { threadName: String = Thread.currentThread().name, ): String = buildString { - append(e.javaClass.simpleName) + // Fully qualified, NOT simpleName. Release builds are obfuscated, so this + // headline would otherwise read "a: 1.16.0-PLAY" — and a bare simple name + // carries no package for `retrace` to resolve it against, so it would stay + // unreadable even after the rest of the report retraced cleanly. The FQN + // retraces back to the real exception class. See scripts/retrace.sh. + append(e.javaClass.name) append(": ") appendLine(BuildConfig.VERSION_NAME + "-" + BuildConfig.FLAVOR.uppercase()) appendLine() @@ -101,8 +106,11 @@ class ReportAssembler { append("Thread: ") appendLine(threadName) appendLine(e.headline()) + // " at ", not just " ": `at` is what every stack-trace + // parser keys on, R8's `retrace` included. Without it a release report is + // passed through untouched and stays obfuscated. See scripts/retrace.sh. e.stackTrace.forEach { - append(" ") + append(" at ") appendLine(it.toString()) } val cause = e.cause @@ -111,7 +119,7 @@ class ReportAssembler { append(" ") appendLine(cause.headline()) cause.stackTrace.forEach { - append(" ") + append(" at ") appendLine(it.toString()) } } diff --git a/amethyst/src/test/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssemblerTest.kt b/amethyst/src/test/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssemblerTest.kt index f618092db0..46720c57ac 100644 --- a/amethyst/src/test/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssemblerTest.kt +++ b/amethyst/src/test/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssemblerTest.kt @@ -35,7 +35,7 @@ class ReportAssemblerTest { val report = ReportAssembler().buildReport(e, "main") assertTrue(report.length < 2_000) - assertTrue(report.startsWith("IllegalArgumentException: ")) + assertTrue(report.startsWith("java.lang.IllegalArgumentException: ")) assertTrue(report.contains("Navigation destination that matches route")) assertTrue(report.trimEnd().endsWith("```")) } @@ -48,7 +48,21 @@ class ReportAssemblerTest { val report = ReportAssembler().buildReport(e, "main") assertTrue(report.contains("java.lang.RuntimeException: boom")) - assertTrue(report.contains("com.example.Foo.bar(Foo.kt:42)")) + // The "at " prefix is load-bearing: retrace only rewrites frames it recognises. + assertTrue(report.contains(" at com.example.Foo.bar(Foo.kt:42)")) assertTrue(report.contains("java.lang.IllegalStateException: root cause")) } + + @Test + fun headlineNamesTheExceptionClassInFullSoItCanBeRetraced() { + // The headline is the line a maintainer skims and dedups on. Release builds are + // obfuscated, and `retrace` can only restore a class name it can resolve — a bare + // simple name has no package, so the headline has to carry the fully qualified one. + val e = IllegalStateException("boom") + e.stackTrace = arrayOf(StackTraceElement("com.example.Foo", "bar", "Foo.kt", 42)) + + val report = ReportAssembler().buildReport(e, "main") + + assertTrue(report.startsWith("java.lang.IllegalStateException: ")) + } } diff --git a/scripts/retrace.sh b/scripts/retrace.sh index 752db089ed..27d5ee125b 100755 --- a/scripts/retrace.sh +++ b/scripts/retrace.sh @@ -1,88 +1,226 @@ #!/usr/bin/env bash # -# Retrace an obfuscated Amethyst stack trace back to real class, method, file +# Turn an obfuscated Amethyst crash report back into real class, method, file # and line names. # -# scripts/retrace.sh [stacktrace-file] # trace on stdin if omitted +# scripts/retrace.sh report.txt # figures out the release by itself +# pbpaste | scripts/retrace.sh # ... or straight off the clipboard # -# is the mapping file for the EXACT build the crash came from: -# * mapping--.txt.gz — attached to every GitHub Release -# * mapping.prt — R8's partition map (faster, same content) -# * amethyst/build/outputs/mapping//mapping.txt — a local build -# .txt, .txt.gz and .prt are all accepted. +# A report produced by ReportAssembler names its own build on line 1: # -# Picking the right one is not guesswork: since the release build is minified, -# R8 replaces every class's SourceFile attribute with `r8-map-id-`, and -# that hash is the `pg_map_id` on line 6 of the matching mapping file. A trace -# therefore names its own mapping — grep the releases for that id. +# java.lang.IllegalStateException: 1.16.0-PLAY # -# The R8 jar that does the work is fetched from Google's Maven at the version -# recorded in the mapping's own header, so this keeps working across AGP bumps -# with nothing to pin. It is cached under ~/.cache/amethyst-retrace/. +# so that is all this needs to fetch the right mapping from the matching GitHub +# Release (amethyst-googleplay-mapping-v1.16.0.txt.gz) and cache it. +# +# If you only have a bare stack trace with no such header, name the build: +# +# scripts/retrace.sh --release v1.16.0 --flavor play trace.txt +# +# ... or point at a mapping yourself, e.g. for a build you made locally: +# +# scripts/retrace.sh amethyst/build/outputs/mapping/playRelease/mapping.txt trace.txt +# +# Mappings are never guessed at. Every frame of an obfuscated trace carries +# `r8-map-id-`, which is the `pg_map_id` of the one mapping that produced +# that build, so the mapping is checked against the report before any output is +# printed — a wrong mapping produces plausible, wrong answers, which is worse +# than no answer. --force overrides. +# +# The R8 that does the work is fetched at the version recorded in the mapping's +# own header, so there is no tooling to pin and this keeps working across AGP +# bumps. Everything is cached under ~/.cache/amethyst-retrace/. set -euo pipefail -MAP="${1:-}" -TRACE="${2:-}" +REPO="${AMETHYST_REPO:-vitorpamplona/amethyst}" +CACHE="${XDG_CACHE_HOME:-$HOME/.cache}/amethyst-retrace" -if [ -z "$MAP" ] || [ ! -f "$MAP" ]; then - echo "usage: $0 [stacktrace-file]" >&2 - exit 2 +MAPPING="" +RELEASE="" +FLAVOR="" +REPORT="" +FORCE=0 + +die() { echo "error: $*" >&2; exit 2; } + +usage() { + sed -n '3,30p' "$0" | sed 's/^# \{0,1\}//' + exit "${1:-2}" +} + +# A mapping file, or the report? Decide by content, not by extension, so both +# documented argument orders keep working. +looks_like_mapping() { + local f="$1" + [ -f "$f" ] || return 1 + case "$f" in + *.prt) return 0 ;; + *.gz) gunzip -c "$f" 2>/dev/null | head -c 200 | grep -q '^# compiler' && return 0 || return 1 ;; + *) head -c 200 "$f" 2>/dev/null | grep -q '^# compiler' && return 0 || return 1 ;; + esac +} + +while [ $# -gt 0 ]; do + case "$1" in + --release) RELEASE="${2:-}"; shift 2 ;; + --flavor) FLAVOR="${2:-}"; shift 2 ;; + --mapping) MAPPING="${2:-}"; shift 2 ;; + --force) FORCE=1; shift ;; + -h|--help) usage 0 ;; + -*) die "unknown option $1 (try --help)" ;; + *) + if [ -z "$MAPPING" ] && [ -z "$RELEASE" ] && looks_like_mapping "$1"; then + MAPPING="$1" + elif [ -z "$REPORT" ]; then + REPORT="$1" + else + die "unexpected argument $1" + fi + shift ;; + esac +done + +mkdir -p "$CACHE" +TMP="$(mktemp -d)" +trap 'rm -rf "$TMP"' EXIT + +# ---- materialise the report ------------------------------------------------ +# Always to a file: we have to read it twice (header, map id) before retracing, +# and that is not possible on a pipe. +REPORT_FILE="$TMP/report.txt" +if [ -n "$REPORT" ]; then + [ -f "$REPORT" ] || die "no such file: $REPORT" + cp "$REPORT" "$REPORT_FILE" +else + [ -t 0 ] && echo "reading the report from stdin (ctrl-D when done) ..." >&2 + cat > "$REPORT_FILE" +fi +[ -s "$REPORT_FILE" ] || die "the report is empty" + +# Retrace only rewrites frames it recognises, and it recognises them by the +# leading `at`. Reports from Amethyst builds before the ReportAssembler fix +# wrote bare " com.foo.Bar.baz(File.kt:12)" lines, which would otherwise be +# passed through still obfuscated and look like a mapping problem. Put the `at` +# back on any frame-shaped line that is missing it. Anything that is not +# (:) is left exactly as it is. +sed -E 's/^([[:space:]]+)([A-Za-z_$][A-Za-z0-9_$]*(\.[A-Za-z0-9_$<>]+)+\([^()]*:[0-9]+\))[[:space:]]*$/\1at \2/' \ + "$REPORT_FILE" > "$TMP/normalised.txt" +mv "$TMP/normalised.txt" "$REPORT_FILE" + +# ---- work out which mapping -------------------------------------------------- +if [ -z "$MAPPING" ]; then + if [ -z "$RELEASE" ]; then + # --auto: parse ": -" off line 1. + header="$(head -1 "$REPORT_FILE" | tr -d '\r')" + case "$header" in + *": "*) ;; + *) die "line 1 is not an Amethyst crash-report header, so the build is unknown. + Pass --release [--flavor play|fdroid], or a mapping file." ;; + esac + vf="${header#*: }" + vf="$(echo "$vf" | tr -d '[:space:]')" + [ -n "$vf" ] || die "line 1 has no '-' after the exception name." + parsed_flavor="${vf##*-}" + version="${vf%-*}" + [ -n "$parsed_flavor" ] && [ -n "$version" ] && [ "$version" != "$vf" ] \ + || die "cannot read '-' out of line 1: $header" + [ -n "$FLAVOR" ] || FLAVOR="$parsed_flavor" + case "$version" in + [0-9]*.[0-9]*.[0-9]*) ;; + *) die "line 1 reports version '$version', which is not a release version." ;; + esac + # generateVersionName() appends the git branch on non-main builds, so a + # dev build reads e.g. 1.16.0-my-branch-PLAY. There is no release for it. + case "$version" in + *[!0-9.]*) die "version '$version' carries a branch suffix, so this is a local/CI + build, not a release — its mapping was never published. Retrace against + that build's own amethyst/build/outputs/mapping//mapping.txt." ;; + esac + RELEASE="v$version" + fi + + case "$RELEASE" in v*) ;; *) RELEASE="v$RELEASE" ;; esac + + case "$(echo "${FLAVOR:-play}" | tr '[:upper:]' '[:lower:]')" in + play|googleplay) channel=googleplay ;; + fdroid) channel=fdroid ;; + *) die "unknown flavor '$FLAVOR' (expected play or fdroid)" ;; + esac + + asset="amethyst-${channel}-mapping-${RELEASE}.txt.gz" + MAPPING="$CACHE/$asset" + if [ ! -s "$MAPPING" ]; then + url="https://github.com/${REPO}/releases/download/${RELEASE}/${asset}" + echo "fetching $asset ..." >&2 + curl -fsSL -o "$MAPPING.tmp" "$url" || { + rm -f "$MAPPING.tmp" + die "could not download $url + Either that release predates mapping publication (builds up to v1.16.0 + were not obfuscated — read those traces as-is), or the flavor is wrong. + Assets: gh release view $RELEASE --json assets --jq '.assets[].name'" + } + mv "$MAPPING.tmp" "$MAPPING" + fi fi -CACHE="${XDG_CACHE_HOME:-$HOME/.cache}/amethyst-retrace" -mkdir -p "$CACHE" +[ -f "$MAPPING" ] || die "no such mapping: $MAPPING" -# ---- resolve the mapping to a plain .txt (Retrace cannot read .gz) ---------- +# ---- decompress if needed -------------------------------------------------- PARTITION=0 -case "$MAP" in - *.prt) - PARTITION=1 - ;; +case "$MAPPING" in + *.prt) PARTITION=1 ;; *.gz) - PLAIN="$CACHE/$(basename "${MAP%.gz}")" - if [ ! -s "$PLAIN" ] || [ "$MAP" -nt "$PLAIN" ]; then - echo "decompressing $(basename "$MAP") ..." >&2 - gunzip -c "$MAP" > "$PLAIN" + plain="$CACHE/$(basename "${MAPPING%.gz}")" + if [ ! -s "$plain" ] || [ "$MAPPING" -nt "$plain" ]; then + echo "decompressing $(basename "$MAPPING") ..." >&2 + gunzip -c "$MAPPING" > "$plain" fi - MAP="$PLAIN" + MAPPING="$plain" ;; esac -# ---- fetch the matching R8 ------------------------------------------------ +# ---- make sure this mapping really built this report ------------------------ +if [ "$PARTITION" -eq 0 ]; then + trace_id="$(grep -om1 'r8-map-id-[0-9a-f]\{16,\}' "$REPORT_FILE" | sed 's/^r8-map-id-//' || true)" + map_id="$(grep -m1 '^# pg_map_id:' "$MAPPING" | sed 's/.*: *//' || true)" + if [ -z "$trace_id" ]; then + echo "note: no r8-map-id in the report (an un-obfuscated build, or a trimmed" >&2 + echo " trace) — cannot confirm the mapping matches." >&2 + elif [ "$trace_id" != "$map_id" ]; then + msg="this mapping did not build this report. + report: $trace_id + mapping: $map_id ($(basename "$MAPPING")) + Retracing anyway yields wrong names that look right. Check the release + tag and the flavor (play vs fdroid are separate R8 runs)." + [ "$FORCE" -eq 1 ] && echo "warning: $msg" >&2 || die "$msg" + fi +fi + +# ---- fetch the R8 that wrote it --------------------------------------------- if [ "$PARTITION" -eq 1 ]; then - # A partition map is a zip; its header is not greppable. Fall back to the - # newest R8 we already have, else ask for an explicit version. - R8_VERSION="${R8_VERSION:-}" - if [ -z "$R8_VERSION" ]; then - R8_VERSION=$(ls "$CACHE"/r8-*.jar 2>/dev/null | sed 's/.*r8-\(.*\)\.jar/\1/' | sort -V | tail -1 || true) - fi - if [ -z "$R8_VERSION" ]; then - echo "error: cannot read the R8 version out of a .prt partition map." >&2 - echo " Set R8_VERSION= (see 'compiler_version' in the matching" >&2 - echo " mapping.txt), or retrace against the .txt.gz instead." >&2 - exit 2 - fi + R8_VERSION="${R8_VERSION:-$(ls "$CACHE"/r8-*.jar 2>/dev/null | sed 's/.*r8-\(.*\)\.jar/\1/' | sort -V | tail -1 || true)}" + [ -n "$R8_VERSION" ] || die "a .prt partition map does not expose its R8 version. + Set R8_VERSION= (the 'compiler_version' of the matching + mapping.txt), or retrace against the .txt.gz instead." else - R8_VERSION=$(head -c 4096 "$MAP" | sed -n 's/^# compiler_version: //p' | head -1) - if [ -z "$R8_VERSION" ]; then - echo "error: no '# compiler_version:' header in $MAP — is it really an R8 mapping?" >&2 - exit 2 - fi + R8_VERSION="$(sed -n 's/^# compiler_version: //p' "$MAPPING" | head -1)" + [ -n "$R8_VERSION" ] || die "no '# compiler_version:' header in $MAPPING — not an R8 mapping?" fi R8_JAR="$CACHE/r8-${R8_VERSION}.jar" if [ ! -s "$R8_JAR" ]; then echo "fetching R8 ${R8_VERSION} ..." >&2 curl -fsSL -o "$R8_JAR.tmp" \ - "https://maven.google.com/com/android/tools/r8/${R8_VERSION}/r8-${R8_VERSION}.jar" + "https://maven.google.com/com/android/tools/r8/${R8_VERSION}/r8-${R8_VERSION}.jar" \ + || { rm -f "$R8_JAR.tmp"; die "could not download R8 ${R8_VERSION} from Google's Maven"; } mv "$R8_JAR.tmp" "$R8_JAR" fi # ---- retrace --------------------------------------------------------------- if [ "$PARTITION" -eq 1 ]; then exec java -cp "$R8_JAR" com.android.tools.r8.retrace.Retrace \ - --partition-map "$MAP" ${TRACE:+"$TRACE"} + --partition-map "$MAPPING" "$REPORT_FILE" else exec java -cp "$R8_JAR" com.android.tools.r8.retrace.Retrace \ - "$MAP" ${TRACE:+"$TRACE"} + "$MAPPING" "$REPORT_FILE" fi