laantungir/amethyst
1
0
Fork 0
mirror of https://github.com/vitorpamplona/amethyst.git synced 2026-10-05 11:18:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: make a pasted crash report retrace itself in one command

Browse Source 
Follow-up to the R8 mapping work: close the last two gaps between "user sends a
crash report over NIP-17" and "maintainer reads real names".

`scripts/retrace.sh <report>` now needs nothing else. A report's first line
already names its own build — `java.lang.IllegalStateException: 1.16.0-PLAY` —
so the script resolves the tag (v1.16.0) and flavor (PLAY -> googleplay),
downloads that release's mapping asset over plain HTTPS (no gh auth needed on a
public repo) and caches it. `--release`/`--flavor` for a bare stack trace with
no header, or a mapping path for a local build; all three forms tested.

It refuses to guess: a report's `r8-map-id-<hash>` is the `pg_map_id` of the one
mapping that built it, so the two are compared before anything is printed. The
wrong mapping does not fail loudly, it prints confident wrong names — worse than
no answer. `--force` overrides. A branch build (1.16.0-my-branch-PLAY) is named
as such instead of 404ing on a release that never existed.

Two ReportAssembler fixes, both found by running the real report format through
retrace rather than assuming it worked:

- Frames were written as "    <frame>" with no `at`. Retrace only rewrites
  frames it recognises and it recognises them by the leading `at`, so a release
  report was passed through completely untouched — every frame still
  obfuscated, looking exactly like a mapping problem. Now "    at <frame>",
  which is also what every other stack-trace tool expects. The script still
  repairs the missing `at` so reports from older builds retrace too.

- The headline used `e.javaClass.simpleName`, which obfuscates to "a:" and
  cannot be retraced back — a bare simple name has no package to resolve
  against. Now the fully qualified name, verified to retrace: "onh: 1.16.0-PLAY"
  comes back as
  "com.vitorpamplona.amethyst...ShortNotePostViewModel: 1.16.0-PLAY".
  The headline is what you skim and dedup on, so it is worth the extra chars.

Verified end to end against the real playRelease mapping: a full report (device
table, code fences, cause section) in via file and via stdin, frames retraced
including R8's inlined frames expanded 1 -> 3, device table untouched; wrong
map-id refused; missing release reported with the asset URL it looked for.
ReportAssemblerTest green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEoxktEZyTrAS33vVZBiwm
This commit is contained in:
Claude
2026-09-18 22:24:14 +00:00
parent 6aebeeb3d9
commit db8089c084
4 changed files with 250 additions and 72 deletions
Download Patch File Download Diff File Expand all files Collapse all files
RELEASE_OPS.md
+33 -15
View File
@@ -345,14 +345,21 @@ java.lang.IllegalStateException: something blew up
at onh.B(r8-map-id-12c710927a584543dbe1e2e867db95460bc44482efe53283f86798648c1cfc00:7)
```
That is not lost information, it is encoded information. Run it back through the
mapping:
That is not lost information, it is encoded information. Paste the report in and
run it:
```bash
scripts/retrace.sh amethyst-googleplay-mapping-v1.13.1.txt.gz crash.txt
# or: pbpaste | scripts/retrace.sh amethyst-googleplay-mapping-v1.13.1.txt.gz
scripts/retrace.sh crash-report.txt
pbpaste | scripts/retrace.sh # or straight off the clipboard
```
Nothing else to supply. A report's first line names its own build —
`java.lang.IllegalStateException: 1.16.0-PLAY` — so the script resolves the tag
(`v1.16.0`) and the flavor (`PLAY` → `googleplay`), downloads that release's
mapping asset and caches it. For a bare stack trace with no such header, name
the build yourself with `--release v1.16.0 --flavor play`; for a build you made
locally, pass its `mapping.txt` directly.
```
java.lang.IllegalStateException: something blew up
at androidx.compose.foundation.text.input.TextFieldCharSequence.getText(TextFieldCharSequence.kt:58)
@@ -366,29 +373,40 @@ trace's line number cannot be trusted even in the pre-obfuscation builds, where
optimization was already inlining. Retracing is not a tax obfuscation imposed;
it is how you read an optimized build at all.
**Which mapping?** Never guess. The `r8-map-id-<hash>` in the trace *is* the
`pg_map_id` header of the mapping that produced it, so:
**The wrong mapping is worse than none** — it produces confident, wrong names.
So the script refuses to guess: the `r8-map-id-<hash>` in the trace *is* the
`pg_map_id` header of the mapping that built it, and the two are compared before
anything is printed:
```bash
gh release download <tag> -p 'amethyst-*-mapping-*.txt.gz'
zcat amethyst-googleplay-mapping-<tag>.txt.gz | grep -m1 pg_map_id
```
error: this mapping did not build this report.
report: deadbeef...
mapping: 12c71092... (amethyst-googleplay-mapping-v1.16.0.txt.gz)
```
If the hashes match, that is the right file, full stop. (`googleplay` vs
`fdroid` matters — the two flavors are separate R8 runs with different
mappings.)
`--force` overrides if you really mean it. (`googleplay` vs `fdroid` matters —
the two flavors are separate R8 runs with different mappings.)
**Per channel:**
| Where the report came from | What to do |
|---|---|
| Play Console / Android vitals | Nothing. AGP embeds the mapping in the `.aab` (`BUNDLE-METADATA/com.android.tools.build.obfuscation/proguard.map`), so Play deobfuscates automatically. |
| A GitHub issue, Nostr DM, F-Droid, Zapstore, Accrescent | `scripts/retrace.sh` against that release's mapping asset. |
| A build you made locally | `scripts/retrace.sh amethyst/build/outputs/mapping/<variant>/mapping.txt` |
| A NIP-17 DM from the in-app crash reporter, a GitHub issue, F-Droid, Zapstore, Accrescent | `scripts/retrace.sh <report>` — it reads the build off line 1 and fetches the mapping. |
| A build you made locally | `scripts/retrace.sh amethyst/build/outputs/mapping/<variant>/mapping.txt report.txt` |
`scripts/retrace.sh` downloads the R8 version named in the mapping's own header
from Google's Maven and caches it, so it needs no pinned tooling and keeps
working across AGP bumps.
working across AGP bumps. It needs no `gh` auth either — release assets on a
public repo are plain HTTPS downloads.
Two details of the report format in `ReportAssembler` exist for this and should
not be "tidied" away: the headline carries the **fully qualified** exception
class (a bare `simpleName` obfuscates to `a`, which retrace cannot resolve
because it has no package), and stack frames are written as ` at <frame>`
(retrace only rewrites frames it recognises, and it recognises them by the
leading `at`). The script repairs the missing `at` on reports from older builds,
but new reports should not need repairing.
**Do not delete mapping assets from old releases.** They are the only copy —
CI's are gone when the job ends, and a mapping cannot be regenerated after the
amethyst/src/main/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssembler.kt
+11 -3
View File
@@ -48,7 +48,12 @@ class ReportAssembler {
threadName: String = Thread.currentThread().name,
): String =
buildString {
append(e.javaClass.simpleName)
// Fully qualified, NOT simpleName. Release builds are obfuscated, so this
// headline would otherwise read "a: 1.16.0-PLAY" — and a bare simple name
// carries no package for `retrace` to resolve it against, so it would stay
// unreadable even after the rest of the report retraced cleanly. The FQN
// retraces back to the real exception class. See scripts/retrace.sh.
append(e.javaClass.name)
append(": ")
appendLine(BuildConfig.VERSION_NAME + "-" + BuildConfig.FLAVOR.uppercase())
appendLine()
@@ -101,8 +106,11 @@ class ReportAssembler {
append("Thread: ")
appendLine(threadName)
appendLine(e.headline())
// " at <frame>", not just " <frame>": `at` is what every stack-trace
// parser keys on, R8's `retrace` included. Without it a release report is
// passed through untouched and stays obfuscated. See scripts/retrace.sh.
e.stackTrace.forEach {
append(" ")
append(" at ")
appendLine(it.toString())
}
val cause = e.cause
@@ -111,7 +119,7 @@ class ReportAssembler {
append(" ")
appendLine(cause.headline())
cause.stackTrace.forEach {
append(" ")
append(" at ")
appendLine(it.toString())
}
}
amethyst/src/test/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssemblerTest.kt
+16 -2
View File
@@ -35,7 +35,7 @@ class ReportAssemblerTest {
val report = ReportAssembler().buildReport(e, "main")
assertTrue(report.length < 2_000)
assertTrue(report.startsWith("IllegalArgumentException: "))
assertTrue(report.startsWith("java.lang.IllegalArgumentException: "))
assertTrue(report.contains("Navigation destination that matches route"))
assertTrue(report.trimEnd().endsWith("```"))
}
@@ -48,7 +48,21 @@ class ReportAssemblerTest {
val report = ReportAssembler().buildReport(e, "main")
assertTrue(report.contains("java.lang.RuntimeException: boom"))
assertTrue(report.contains("com.example.Foo.bar(Foo.kt:42)"))
// The "at " prefix is load-bearing: retrace only rewrites frames it recognises.
assertTrue(report.contains(" at com.example.Foo.bar(Foo.kt:42)"))
assertTrue(report.contains("java.lang.IllegalStateException: root cause"))
}
@Test
fun headlineNamesTheExceptionClassInFullSoItCanBeRetraced() {
// The headline is the line a maintainer skims and dedups on. Release builds are
// obfuscated, and `retrace` can only restore a class name it can resolve — a bare
// simple name has no package, so the headline has to carry the fully qualified one.
val e = IllegalStateException("boom")
e.stackTrace = arrayOf(StackTraceElement("com.example.Foo", "bar", "Foo.kt", 42))
val report = ReportAssembler().buildReport(e, "main")
assertTrue(report.startsWith("java.lang.IllegalStateException: "))
}
}
scripts/retrace.sh
+190 -52
View File
@@ -1,88 +1,226 @@
#!/usr/bin/env bash
#
# Retrace an obfuscated Amethyst stack trace back to real class, method, file
# Turn an obfuscated Amethyst crash report back into real class, method, file
# and line names.
#
# scripts/retrace.sh <mapping> [stacktrace-file] # trace on stdin if omitted
# scripts/retrace.sh report.txt # figures out the release by itself
# pbpaste | scripts/retrace.sh # ... or straight off the clipboard
#
# <mapping> is the mapping file for the EXACT build the crash came from:
# * mapping-<flavor>-<tag>.txt.gz — attached to every GitHub Release
# * mapping.prt — R8's partition map (faster, same content)
# * amethyst/build/outputs/mapping/<variant>/mapping.txt — a local build
# .txt, .txt.gz and .prt are all accepted.
# A report produced by ReportAssembler names its own build on line 1:
#
# Picking the right one is not guesswork: since the release build is minified,
# R8 replaces every class's SourceFile attribute with `r8-map-id-<hash>`, and
# that hash is the `pg_map_id` on line 6 of the matching mapping file. A trace
# therefore names its own mapping — grep the releases for that id.
# java.lang.IllegalStateException: 1.16.0-PLAY
#
# The R8 jar that does the work is fetched from Google's Maven at the version
# recorded in the mapping's own header, so this keeps working across AGP bumps
# with nothing to pin. It is cached under ~/.cache/amethyst-retrace/.
# so that is all this needs to fetch the right mapping from the matching GitHub
# Release (amethyst-googleplay-mapping-v1.16.0.txt.gz) and cache it.
#
# If you only have a bare stack trace with no such header, name the build:
#
# scripts/retrace.sh --release v1.16.0 --flavor play trace.txt
#
# ... or point at a mapping yourself, e.g. for a build you made locally:
#
# scripts/retrace.sh amethyst/build/outputs/mapping/playRelease/mapping.txt trace.txt
#
# Mappings are never guessed at. Every frame of an obfuscated trace carries
# `r8-map-id-<hash>`, which is the `pg_map_id` of the one mapping that produced
# that build, so the mapping is checked against the report before any output is
# printed — a wrong mapping produces plausible, wrong answers, which is worse
# than no answer. --force overrides.
#
# The R8 that does the work is fetched at the version recorded in the mapping's
# own header, so there is no tooling to pin and this keeps working across AGP
# bumps. Everything is cached under ~/.cache/amethyst-retrace/.
set -euo pipefail
MAP="${1:-}"
TRACE="${2:-}"
REPO="${AMETHYST_REPO:-vitorpamplona/amethyst}"
CACHE="${XDG_CACHE_HOME:-$HOME/.cache}/amethyst-retrace"
if [ -z "$MAP" ] || [ ! -f "$MAP" ]; then
echo "usage: $0 <mapping.txt|mapping.txt.gz|mapping.prt> [stacktrace-file]" >&2
exit 2
MAPPING=""
RELEASE=""
FLAVOR=""
REPORT=""
FORCE=0
die() { echo "error: $*" >&2; exit 2; }
usage() {
sed -n '3,30p' "$0" | sed 's/^# \{0,1\}//'
exit "${1:-2}"
}
# A mapping file, or the report? Decide by content, not by extension, so both
# documented argument orders keep working.
looks_like_mapping() {
local f="$1"
[ -f "$f" ] || return 1
case "$f" in
*.prt) return 0 ;;
*.gz) gunzip -c "$f" 2>/dev/null | head -c 200 | grep -q '^# compiler' && return 0 || return 1 ;;
*) head -c 200 "$f" 2>/dev/null | grep -q '^# compiler' && return 0 || return 1 ;;
esac
}
while [ $# -gt 0 ]; do
case "$1" in
--release) RELEASE="${2:-}"; shift 2 ;;
--flavor) FLAVOR="${2:-}"; shift 2 ;;
--mapping) MAPPING="${2:-}"; shift 2 ;;
--force) FORCE=1; shift ;;
-h|--help) usage 0 ;;
-*) die "unknown option $1 (try --help)" ;;
*)
if [ -z "$MAPPING" ] && [ -z "$RELEASE" ] && looks_like_mapping "$1"; then
MAPPING="$1"
elif [ -z "$REPORT" ]; then
REPORT="$1"
else
die "unexpected argument $1"
fi
shift ;;
esac
done
mkdir -p "$CACHE"
TMP="$(mktemp -d)"
trap 'rm -rf "$TMP"' EXIT
# ---- materialise the report ------------------------------------------------
# Always to a file: we have to read it twice (header, map id) before retracing,
# and that is not possible on a pipe.
REPORT_FILE="$TMP/report.txt"
if [ -n "$REPORT" ]; then
[ -f "$REPORT" ] || die "no such file: $REPORT"
cp "$REPORT" "$REPORT_FILE"
else
[ -t 0 ] && echo "reading the report from stdin (ctrl-D when done) ..." >&2
cat > "$REPORT_FILE"
fi
[ -s "$REPORT_FILE" ] || die "the report is empty"
# Retrace only rewrites frames it recognises, and it recognises them by the
# leading `at`. Reports from Amethyst builds before the ReportAssembler fix
# wrote bare " com.foo.Bar.baz(File.kt:12)" lines, which would otherwise be
# passed through still obfuscated and look like a mapping problem. Put the `at`
# back on any frame-shaped line that is missing it. Anything that is not
# <indent><dotted.name>(<something>:<digits>) is left exactly as it is.
sed -E 's/^([[:space:]]+)([A-Za-z_$][A-Za-z0-9_$]*(\.[A-Za-z0-9_$<>]+)+\([^()]*:[0-9]+\))[[:space:]]*$/\1at \2/' \
"$REPORT_FILE" > "$TMP/normalised.txt"
mv "$TMP/normalised.txt" "$REPORT_FILE"
# ---- work out which mapping --------------------------------------------------
if [ -z "$MAPPING" ]; then
if [ -z "$RELEASE" ]; then
# --auto: parse "<exception>: <version>-<FLAVOR>" off line 1.
header="$(head -1 "$REPORT_FILE" | tr -d '\r')"
case "$header" in
*": "*) ;;
*) die "line 1 is not an Amethyst crash-report header, so the build is unknown.
Pass --release <tag> [--flavor play|fdroid], or a mapping file." ;;
esac
vf="${header#*: }"
vf="$(echo "$vf" | tr -d '[:space:]')"
[ -n "$vf" ] || die "line 1 has no '<version>-<FLAVOR>' after the exception name."
parsed_flavor="${vf##*-}"
version="${vf%-*}"
[ -n "$parsed_flavor" ] && [ -n "$version" ] && [ "$version" != "$vf" ] \
|| die "cannot read '<version>-<FLAVOR>' out of line 1: $header"
[ -n "$FLAVOR" ] || FLAVOR="$parsed_flavor"
case "$version" in
[0-9]*.[0-9]*.[0-9]*) ;;
*) die "line 1 reports version '$version', which is not a release version." ;;
esac
# generateVersionName() appends the git branch on non-main builds, so a
# dev build reads e.g. 1.16.0-my-branch-PLAY. There is no release for it.
case "$version" in
*[!0-9.]*) die "version '$version' carries a branch suffix, so this is a local/CI
build, not a release — its mapping was never published. Retrace against
that build's own amethyst/build/outputs/mapping/<variant>/mapping.txt." ;;
esac
RELEASE="v$version"
fi
case "$RELEASE" in v*) ;; *) RELEASE="v$RELEASE" ;; esac
case "$(echo "${FLAVOR:-play}" | tr '[:upper:]' '[:lower:]')" in
play|googleplay) channel=googleplay ;;
fdroid) channel=fdroid ;;
*) die "unknown flavor '$FLAVOR' (expected play or fdroid)" ;;
esac
asset="amethyst-${channel}-mapping-${RELEASE}.txt.gz"
MAPPING="$CACHE/$asset"
if [ ! -s "$MAPPING" ]; then
url="https://github.com/${REPO}/releases/download/${RELEASE}/${asset}"
echo "fetching $asset ..." >&2
curl -fsSL -o "$MAPPING.tmp" "$url" || {
rm -f "$MAPPING.tmp"
die "could not download $url
Either that release predates mapping publication (builds up to v1.16.0
were not obfuscated — read those traces as-is), or the flavor is wrong.
Assets: gh release view $RELEASE --json assets --jq '.assets[].name'"
}
mv "$MAPPING.tmp" "$MAPPING"
fi
fi
CACHE="${XDG_CACHE_HOME:-$HOME/.cache}/amethyst-retrace"
mkdir -p "$CACHE"
[ -f "$MAPPING" ] || die "no such mapping: $MAPPING"
# ---- resolve the mapping to a plain .txt (Retrace cannot read .gz) ----------
# ---- decompress if needed --------------------------------------------------
PARTITION=0
case "$MAP" in
*.prt)
PARTITION=1
;;
case "$MAPPING" in
*.prt) PARTITION=1 ;;
*.gz)
PLAIN="$CACHE/$(basename "${MAP%.gz}")"
if [ ! -s "$PLAIN" ] || [ "$MAP" -nt "$PLAIN" ]; then
echo "decompressing $(basename "$MAP") ..." >&2
gunzip -c "$MAP" > "$PLAIN"
plain="$CACHE/$(basename "${MAPPING%.gz}")"
if [ ! -s "$plain" ] || [ "$MAPPING" -nt "$plain" ]; then
echo "decompressing $(basename "$MAPPING") ..." >&2
gunzip -c "$MAPPING" > "$plain"
fi
MAP="$PLAIN"
MAPPING="$plain"
;;
esac
# ---- fetch the matching R8 ------------------------------------------------
# ---- make sure this mapping really built this report ------------------------
if [ "$PARTITION" -eq 0 ]; then
trace_id="$(grep -om1 'r8-map-id-[0-9a-f]\{16,\}' "$REPORT_FILE" | sed 's/^r8-map-id-//' || true)"
map_id="$(grep -m1 '^# pg_map_id:' "$MAPPING" | sed 's/.*: *//' || true)"
if [ -z "$trace_id" ]; then
echo "note: no r8-map-id in the report (an un-obfuscated build, or a trimmed" >&2
echo " trace) — cannot confirm the mapping matches." >&2
elif [ "$trace_id" != "$map_id" ]; then
msg="this mapping did not build this report.
report: $trace_id
mapping: $map_id ($(basename "$MAPPING"))
Retracing anyway yields wrong names that look right. Check the release
tag and the flavor (play vs fdroid are separate R8 runs)."
[ "$FORCE" -eq 1 ] && echo "warning: $msg" >&2 || die "$msg"
fi
fi
# ---- fetch the R8 that wrote it ---------------------------------------------
if [ "$PARTITION" -eq 1 ]; then
# A partition map is a zip; its header is not greppable. Fall back to the
# newest R8 we already have, else ask for an explicit version.
R8_VERSION="${R8_VERSION:-}"
if [ -z "$R8_VERSION" ]; then
R8_VERSION=$(ls "$CACHE"/r8-*.jar 2>/dev/null | sed 's/.*r8-\(.*\)\.jar/\1/' | sort -V | tail -1 || true)
fi
if [ -z "$R8_VERSION" ]; then
echo "error: cannot read the R8 version out of a .prt partition map." >&2
echo " Set R8_VERSION=<x.y.z> (see 'compiler_version' in the matching" >&2
echo " mapping.txt), or retrace against the .txt.gz instead." >&2
exit 2
fi
R8_VERSION="${R8_VERSION:-$(ls "$CACHE"/r8-*.jar 2>/dev/null | sed 's/.*r8-\(.*\)\.jar/\1/' | sort -V | tail -1 || true)}"
[ -n "$R8_VERSION" ] || die "a .prt partition map does not expose its R8 version.
Set R8_VERSION=<x.y.z> (the 'compiler_version' of the matching
mapping.txt), or retrace against the .txt.gz instead."
else
R8_VERSION=$(head -c 4096 "$MAP" | sed -n 's/^# compiler_version: //p' | head -1)
if [ -z "$R8_VERSION" ]; then
echo "error: no '# compiler_version:' header in $MAP — is it really an R8 mapping?" >&2
exit 2
fi
R8_VERSION="$(sed -n 's/^# compiler_version: //p' "$MAPPING" | head -1)"
[ -n "$R8_VERSION" ] || die "no '# compiler_version:' header in $MAPPING — not an R8 mapping?"
fi
R8_JAR="$CACHE/r8-${R8_VERSION}.jar"
if [ ! -s "$R8_JAR" ]; then
echo "fetching R8 ${R8_VERSION} ..." >&2
curl -fsSL -o "$R8_JAR.tmp" \
"https://maven.google.com/com/android/tools/r8/${R8_VERSION}/r8-${R8_VERSION}.jar"
"https://maven.google.com/com/android/tools/r8/${R8_VERSION}/r8-${R8_VERSION}.jar" \
|| { rm -f "$R8_JAR.tmp"; die "could not download R8 ${R8_VERSION} from Google's Maven"; }
mv "$R8_JAR.tmp" "$R8_JAR"
fi
# ---- retrace ---------------------------------------------------------------
if [ "$PARTITION" -eq 1 ]; then
exec java -cp "$R8_JAR" com.android.tools.r8.retrace.Retrace \
--partition-map "$MAP" ${TRACE:+"$TRACE"}
--partition-map "$MAPPING" "$REPORT_FILE"
else
exec java -cp "$R8_JAR" com.android.tools.r8.retrace.Retrace \
"$MAP" ${TRACE:+"$TRACE"}
"$MAPPING" "$REPORT_FILE"
fi